2024-04-22T23:18:02.177 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-22T23:18:02.178 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_adking@teuthology --description orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} --verbose --archive /home/teuthworker/archive/adking-2024-04-22_22:45:33-orch:cephadm-wip-adk-testing-2024-04-22-1618-distro-default-smithi/7669223 --name adking-2024-04-22_22:45:33-orch:cephadm-wip-adk-testing-2024-04-22-1618-distro-default-smithi -- /home/teuthworker/archive/adking-2024-04-22_22:45:33-orch:cephadm-wip-adk-testing-2024-04-22-1618-distro-default-smithi/7669223/orig.config.yaml 2024-04-22T23:18:02.230 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-22T23:18:02.331 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/adking-2024-04-22_22:45:33-orch:cephadm-wip-adk-testing-2024-04-22-1618-distro-default-smithi/7669223 branch: wip-adk-testing-2024-04-22-1618 description: orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: adking@redhat.com first_in_suite: false job_id: '7669223' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/quincy ' name: adking-2024-04-22_22:45:33-orch:cephadm-wip-adk-testing-2024-04-22-1618-distro-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-adk-testing-2024-04-22-1618 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - OSD_DOWN - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: 43be020184947e53516056c9931e1ac5bdbbb1a5 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 43be020184947e53516056c9931e1ac5bdbbb1a5 kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-adk-testing-2024-04-22-1618 sha1: 43be020184947e53516056c9931e1ac5bdbbb1a5 owner: scheduled_adking@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 6551 sha1: 43be020184947e53516056c9931e1ac5bdbbb1a5 sleep_before_teardown: 0 subset: 1/16 suite: orch:cephadm suite_branch: wip-adk-testing-2024-04-22-1618 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_43be020184947e53516056c9931e1ac5bdbbb1a5/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 43be020184947e53516056c9931e1ac5bdbbb1a5 targets: smithi107.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB+DTvQAnUBeyS/bLqLhzS7sorox8muSN8gqBNapoUj1mivuWT3erhKYzFcA+NzRrRmz3Qw6fWIKkUU7eebmS0I= smithi157.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMeZUlG6mdz2mxTsSJmYKj0JPQJm6K4MO+DH8xaIK+f8c8Dfx+Y34lUB3z8FFDMUOmXEL2TsyqIsZ3zFX5yvkAk= tasks: - install: branch: quincy exclude_packages: - ceph-volume - print: '**** done install task...' - cephadm: cephadm_branch: quincy cephadm_git_url: https://github.com/ceph/ceph conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.ceph.io/ceph-ci/ceph:quincy roleless: true - print: '**** done end installing quincy cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 2 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay false - cephadm.shell: host.a: - ceph fs set cephfs inline_data false - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - kclient: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: - /cephfs/home/adking/src/git.ceph.com_ceph-c_wip-adk-testing-2024-04-22-1618/qa/suites/orch/cephadm/mds_upgrade_sequence/tasks/3-upgrade-mgr-staggered.yaml meta: {} postmerge: [] variables: fail_fs: false teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-22_22:45:33 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs false || true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: adking verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-04-22T23:18:02.334 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_43be020184947e53516056c9931e1ac5bdbbb1a5/qa; will attempt to use it 2024-04-22T23:18:02.334 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_43be020184947e53516056c9931e1ac5bdbbb1a5/qa/tasks 2024-04-22T23:18:02.335 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-22T23:18:02.337 INFO:teuthology.task.internal:Checking packages... 2024-04-22T23:18:02.362 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '43be020184947e53516056c9931e1ac5bdbbb1a5' 2024-04-22T23:18:02.362 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-22T23:18:02.362 INFO:teuthology.packaging:ref: None 2024-04-22T23:18:02.363 INFO:teuthology.packaging:tag: None 2024-04-22T23:18:02.363 INFO:teuthology.packaging:branch: wip-adk-testing-2024-04-22-1618 2024-04-22T23:18:02.363 INFO:teuthology.packaging:sha1: 43be020184947e53516056c9931e1ac5bdbbb1a5 2024-04-22T23:18:02.363 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-adk-testing-2024-04-22-1618 2024-04-22T23:18:02.588 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3252.g43be0201 2024-04-22T23:18:02.589 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-22T23:18:02.597 INFO:teuthology.task.internal:no buildpackages task found 2024-04-22T23:18:02.597 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-22T23:18:02.631 INFO:teuthology.task.internal:Saving configuration 2024-04-22T23:18:02.658 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-22T23:18:02.665 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-22T23:18:02.689 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi107.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/adking-2024-04-22_22:45:33-orch:cephadm-wip-adk-testing-2024-04-22-1618-distro-default-smithi/7669223', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-22 23:12:21.079576', 'locked_by': 'scheduled_adking@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB+DTvQAnUBeyS/bLqLhzS7sorox8muSN8gqBNapoUj1mivuWT3erhKYzFcA+NzRrRmz3Qw6fWIKkUU7eebmS0I='} 2024-04-22T23:18:02.712 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi157.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/adking-2024-04-22_22:45:33-orch:cephadm-wip-adk-testing-2024-04-22-1618-distro-default-smithi/7669223', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-22 23:12:21.078218', 'locked_by': 'scheduled_adking@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMeZUlG6mdz2mxTsSJmYKj0JPQJm6K4MO+DH8xaIK+f8c8Dfx+Y34lUB3z8FFDMUOmXEL2TsyqIsZ3zFX5yvkAk='} 2024-04-22T23:18:02.712 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-22T23:18:02.718 INFO:teuthology.task.internal:roles: ubuntu@smithi107.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-04-22T23:18:02.718 INFO:teuthology.task.internal:roles: ubuntu@smithi157.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-04-22T23:18:02.718 INFO:teuthology.run_tasks:Running task console_log... 2024-04-22T23:18:02.827 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f3e59ee5160>, signals=[15]) 2024-04-22T23:18:02.828 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-22T23:18:02.837 INFO:teuthology.task.internal:Opening connections... 2024-04-22T23:18:02.837 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi107.front.sepia.ceph.com 2024-04-22T23:18:02.839 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-22T23:18:02.923 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi157.front.sepia.ceph.com 2024-04-22T23:18:02.924 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-22T23:18:03.002 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-22T23:18:03.009 DEBUG:teuthology.orchestra.run.smithi107:> uname -m 2024-04-22T23:18:03.027 INFO:teuthology.orchestra.run.smithi107.stdout:x86_64 2024-04-22T23:18:03.027 DEBUG:teuthology.orchestra.run.smithi107:> cat /etc/os-release 2024-04-22T23:18:03.085 INFO:teuthology.orchestra.run.smithi107.stdout:NAME="CentOS Stream" 2024-04-22T23:18:03.085 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION="9" 2024-04-22T23:18:03.085 INFO:teuthology.orchestra.run.smithi107.stdout:ID="centos" 2024-04-22T23:18:03.085 INFO:teuthology.orchestra.run.smithi107.stdout:ID_LIKE="rhel fedora" 2024-04-22T23:18:03.085 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION_ID="9" 2024-04-22T23:18:03.085 INFO:teuthology.orchestra.run.smithi107.stdout:PLATFORM_ID="platform:el9" 2024-04-22T23:18:03.085 INFO:teuthology.orchestra.run.smithi107.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-22T23:18:03.085 INFO:teuthology.orchestra.run.smithi107.stdout:ANSI_COLOR="0;31" 2024-04-22T23:18:03.085 INFO:teuthology.orchestra.run.smithi107.stdout:LOGO="fedora-logo-icon" 2024-04-22T23:18:03.085 INFO:teuthology.orchestra.run.smithi107.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-22T23:18:03.086 INFO:teuthology.orchestra.run.smithi107.stdout:HOME_URL="https://centos.org/" 2024-04-22T23:18:03.086 INFO:teuthology.orchestra.run.smithi107.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-22T23:18:03.086 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-22T23:18:03.086 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-22T23:18:03.087 INFO:teuthology.lock.ops:Updating smithi107.front.sepia.ceph.com on lock server 2024-04-22T23:18:03.107 DEBUG:teuthology.orchestra.run.smithi157:> uname -m 2024-04-22T23:18:03.124 INFO:teuthology.orchestra.run.smithi157.stdout:x86_64 2024-04-22T23:18:03.125 DEBUG:teuthology.orchestra.run.smithi157:> cat /etc/os-release 2024-04-22T23:18:03.183 INFO:teuthology.orchestra.run.smithi157.stdout:NAME="CentOS Stream" 2024-04-22T23:18:03.183 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION="9" 2024-04-22T23:18:03.183 INFO:teuthology.orchestra.run.smithi157.stdout:ID="centos" 2024-04-22T23:18:03.183 INFO:teuthology.orchestra.run.smithi157.stdout:ID_LIKE="rhel fedora" 2024-04-22T23:18:03.183 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION_ID="9" 2024-04-22T23:18:03.183 INFO:teuthology.orchestra.run.smithi157.stdout:PLATFORM_ID="platform:el9" 2024-04-22T23:18:03.183 INFO:teuthology.orchestra.run.smithi157.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-22T23:18:03.184 INFO:teuthology.orchestra.run.smithi157.stdout:ANSI_COLOR="0;31" 2024-04-22T23:18:03.184 INFO:teuthology.orchestra.run.smithi157.stdout:LOGO="fedora-logo-icon" 2024-04-22T23:18:03.184 INFO:teuthology.orchestra.run.smithi157.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-22T23:18:03.184 INFO:teuthology.orchestra.run.smithi157.stdout:HOME_URL="https://centos.org/" 2024-04-22T23:18:03.184 INFO:teuthology.orchestra.run.smithi157.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-22T23:18:03.184 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-22T23:18:03.184 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-22T23:18:03.184 INFO:teuthology.lock.ops:Updating smithi157.front.sepia.ceph.com on lock server 2024-04-22T23:18:03.207 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-22T23:18:03.217 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-22T23:18:03.240 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-22T23:18:03.240 DEBUG:teuthology.orchestra.run.smithi107:> test '!' -e /home/ubuntu/cephtest 2024-04-22T23:18:03.243 DEBUG:teuthology.orchestra.run.smithi157:> test '!' -e /home/ubuntu/cephtest 2024-04-22T23:18:03.259 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-22T23:18:03.265 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-22T23:18:03.266 DEBUG:teuthology.orchestra.run.smithi107:> test -z $(ls -A /var/lib/ceph) 2024-04-22T23:18:03.299 DEBUG:teuthology.orchestra.run.smithi157:> test -z $(ls -A /var/lib/ceph) 2024-04-22T23:18:03.404 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-22T23:18:03.459 INFO:teuthology.run_tasks:Running task kernel... 2024-04-22T23:18:03.474 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-04-22T23:18:03.474 INFO:teuthology.task.kernel:config {'host.a': {'kdb': 1, 'sha1': 'distro'}, 'host.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-04-22T23:18:03.475 DEBUG:teuthology.orchestra.run.smithi107:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-22T23:18:03.475 DEBUG:teuthology.orchestra.run.smithi157:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-22T23:18:03.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T23:18:03.492 DEBUG:teuthology.orchestra.run.smithi107:> uname -r 2024-04-22T23:18:03.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T23:18:03.493 DEBUG:teuthology.orchestra.run.smithi157:> uname -r 2024-04-22T23:18:03.549 INFO:teuthology.orchestra.run.smithi107.stdout:5.14.0-437.el9.x86_64 2024-04-22T23:18:03.550 INFO:teuthology.task.kernel:Running kernel on smithi107: 5.14.0-437.el9.x86_64 2024-04-22T23:18:03.550 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum install -y kernel 2024-04-22T23:18:03.551 INFO:teuthology.orchestra.run.smithi157.stdout:5.14.0-437.el9.x86_64 2024-04-22T23:18:03.551 INFO:teuthology.task.kernel:Running kernel on smithi157: 5.14.0-437.el9.x86_64 2024-04-22T23:18:03.551 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum install -y kernel 2024-04-22T23:18:05.790 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-04-22T23:18:14.455 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - BaseOS 896 kB/s | 8.0 MB 00:09 2024-04-22T23:18:16.865 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - AppStream 14 MB/s | 19 MB 00:01 2024-04-22T23:18:20.915 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.2 MB 00:00 2024-04-22T23:18:22.532 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - Extras packages 61 kB/s | 16 kB 00:00 2024-04-22T23:18:24.134 INFO:teuthology.orchestra.run.smithi157.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-04-22T23:18:27.629 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - AppStream 937 kB/s | 19 MB 00:20 2024-04-22T23:18:29.175 INFO:teuthology.orchestra.run.smithi157.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-04-22T23:18:31.123 INFO:teuthology.orchestra.run.smithi157.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-22T23:18:31.123 INFO:teuthology.orchestra.run.smithi157.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-22T23:18:31.216 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-04-22T23:18:31.218 INFO:teuthology.orchestra.run.smithi157.stdout:Nothing to do. 2024-04-22T23:18:31.218 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-04-22T23:18:31.345 DEBUG:teuthology.orchestra.run.smithi157:> echo no | sudo yum reinstall kernel || true 2024-04-22T23:18:31.886 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:00:02 ago on Mon 22 Apr 2024 11:18:29 PM UTC. 2024-04-22T23:18:32.020 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-04-22T23:18:32.027 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-22T23:18:32.028 INFO:teuthology.orchestra.run.smithi157.stdout: Package Architecture Version Repository Size 2024-04-22T23:18:32.028 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-22T23:18:32.028 INFO:teuthology.orchestra.run.smithi157.stdout:Reinstalling: 2024-04-22T23:18:32.028 INFO:teuthology.orchestra.run.smithi157.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-22T23:18:32.028 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:18:32.028 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-04-22T23:18:32.028 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-22T23:18:32.028 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:18:32.028 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 255 k 2024-04-22T23:18:32.028 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 0 2024-04-22T23:18:32.028 INFO:teuthology.orchestra.run.smithi157.stderr:Operation aborted. 2024-04-22T23:18:32.089 INFO:teuthology.orchestra.run.smithi157.stdout:Is this ok [y/N]: 2024-04-22T23:18:32.090 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum reinstall -y kernel || true 2024-04-22T23:18:32.589 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:00:03 ago on Mon 22 Apr 2024 11:18:29 PM UTC. 2024-04-22T23:18:32.704 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-04-22T23:18:32.705 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-22T23:18:32.705 INFO:teuthology.orchestra.run.smithi157.stdout: Package Architecture Version Repository Size 2024-04-22T23:18:32.705 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-22T23:18:32.705 INFO:teuthology.orchestra.run.smithi157.stdout:Reinstalling: 2024-04-22T23:18:32.705 INFO:teuthology.orchestra.run.smithi157.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-22T23:18:32.705 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:18:32.705 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-04-22T23:18:32.705 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-22T23:18:32.705 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:18:32.706 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 255 k 2024-04-22T23:18:32.706 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 0 2024-04-22T23:18:32.706 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-04-22T23:18:33.027 INFO:teuthology.orchestra.run.smithi157.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.3 MB/s | 255 kB 00:00 2024-04-22T23:18:33.027 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:18:33.027 INFO:teuthology.orchestra.run.smithi157.stdout:Total 792 kB/s | 255 kB 00:00 2024-04-22T23:18:33.046 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-04-22T23:18:33.233 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-04-22T23:18:33.234 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-04-22T23:18:33.238 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-04-22T23:18:33.239 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-04-22T23:18:33.368 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-04-22T23:18:33.470 INFO:teuthology.orchestra.run.smithi157.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-22T23:18:33.470 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-22T23:18:34.097 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-22T23:18:34.097 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-22T23:18:34.803 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-22T23:18:34.803 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:18:34.803 INFO:teuthology.orchestra.run.smithi157.stdout:Reinstalled: 2024-04-22T23:18:34.803 INFO:teuthology.orchestra.run.smithi157.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-22T23:18:34.803 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:18:34.803 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-04-22T23:18:34.992 DEBUG:teuthology.orchestra.run.smithi157:> rpm -q kernel | sort -rV | head -n 1 2024-04-22T23:18:35.039 INFO:teuthology.orchestra.run.smithi157.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-22T23:18:35.039 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-22T23:18:35.039 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-22T23:18:35.039 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-04-22T23:18:35.039 DEBUG:teuthology.orchestra.run.smithi157:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-22T23:18:35.073 INFO:teuthology.orchestra.run.smithi157.stdout:ttyS1 2024-04-22T23:18:35.093 DEBUG:teuthology.parallel:result is None 2024-04-22T23:18:39.061 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - CRB 810 kB/s | 6.2 MB 00:07 2024-04-22T23:18:40.578 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - Extras packages 46 kB/s | 16 kB 00:00 2024-04-22T23:18:41.604 INFO:teuthology.orchestra.run.smithi107.stdout:Extra Packages for Enterprise Linux 23 MB/s | 21 MB 00:00 2024-04-22T23:18:46.646 INFO:teuthology.orchestra.run.smithi107.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-04-22T23:18:48.476 INFO:teuthology.orchestra.run.smithi107.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-22T23:18:48.476 INFO:teuthology.orchestra.run.smithi107.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-22T23:18:48.563 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-04-22T23:18:48.564 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-04-22T23:18:48.564 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-04-22T23:18:48.686 DEBUG:teuthology.orchestra.run.smithi107:> echo no | sudo yum reinstall kernel || true 2024-04-22T23:18:49.182 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:00:03 ago on Mon 22 Apr 2024 11:18:46 PM UTC. 2024-04-22T23:18:49.301 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-04-22T23:18:49.307 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-22T23:18:49.307 INFO:teuthology.orchestra.run.smithi107.stdout: Package Architecture Version Repository Size 2024-04-22T23:18:49.307 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-22T23:18:49.308 INFO:teuthology.orchestra.run.smithi107.stdout:Reinstalling: 2024-04-22T23:18:49.308 INFO:teuthology.orchestra.run.smithi107.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-22T23:18:49.308 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:18:49.308 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-04-22T23:18:49.308 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-22T23:18:49.308 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:18:49.308 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 255 k 2024-04-22T23:18:49.308 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 0 2024-04-22T23:18:49.308 INFO:teuthology.orchestra.run.smithi107.stderr:Operation aborted. 2024-04-22T23:18:49.376 INFO:teuthology.orchestra.run.smithi107.stdout:Is this ok [y/N]: 2024-04-22T23:18:49.376 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum reinstall -y kernel || true 2024-04-22T23:18:49.867 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:00:03 ago on Mon 22 Apr 2024 11:18:46 PM UTC. 2024-04-22T23:18:49.983 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-04-22T23:18:49.984 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-22T23:18:49.984 INFO:teuthology.orchestra.run.smithi107.stdout: Package Architecture Version Repository Size 2024-04-22T23:18:49.984 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-22T23:18:49.984 INFO:teuthology.orchestra.run.smithi107.stdout:Reinstalling: 2024-04-22T23:18:49.984 INFO:teuthology.orchestra.run.smithi107.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-22T23:18:49.984 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:18:49.984 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-04-22T23:18:49.984 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-22T23:18:49.984 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:18:49.984 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 255 k 2024-04-22T23:18:49.984 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 0 2024-04-22T23:18:49.984 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-04-22T23:18:50.189 INFO:teuthology.orchestra.run.smithi107.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.8 MB/s | 255 kB 00:00 2024-04-22T23:18:50.189 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:18:50.189 INFO:teuthology.orchestra.run.smithi107.stdout:Total 1.2 MB/s | 255 kB 00:00 2024-04-22T23:18:50.207 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-04-22T23:18:50.398 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-04-22T23:18:50.398 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-04-22T23:18:50.403 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-04-22T23:18:50.404 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-04-22T23:18:50.542 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-04-22T23:18:50.615 INFO:teuthology.orchestra.run.smithi107.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-22T23:18:50.615 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-22T23:18:51.300 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-22T23:18:51.300 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-22T23:18:51.994 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-22T23:18:51.994 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:18:51.994 INFO:teuthology.orchestra.run.smithi107.stdout:Reinstalled: 2024-04-22T23:18:51.994 INFO:teuthology.orchestra.run.smithi107.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-22T23:18:51.994 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:18:51.994 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-04-22T23:18:52.159 DEBUG:teuthology.orchestra.run.smithi107:> rpm -q kernel | sort -rV | head -n 1 2024-04-22T23:18:52.206 INFO:teuthology.orchestra.run.smithi107.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-22T23:18:52.206 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-22T23:18:52.206 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-22T23:18:52.206 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-04-22T23:18:52.207 DEBUG:teuthology.orchestra.run.smithi107:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-22T23:18:52.244 INFO:teuthology.orchestra.run.smithi107.stdout:ttyS1 2024-04-22T23:18:52.262 DEBUG:teuthology.parallel:result is None 2024-04-22T23:18:52.262 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-22T23:18:52.267 INFO:teuthology.task.internal:Creating test directory... 2024-04-22T23:18:52.267 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-22T23:18:52.305 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-22T23:18:52.324 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-22T23:18:52.330 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-22T23:18:52.355 INFO:teuthology.task.internal:Creating archive directory... 2024-04-22T23:18:52.355 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-22T23:18:52.360 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-22T23:18:52.585 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-22T23:18:52.592 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-22T23:18:52.592 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-22T23:18:52.594 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-22T23:18:52.621 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-22T23:18:52.630 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-22T23:18:52.631 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-22T23:18:52.641 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-22T23:18:52.643 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-22T23:18:52.650 INFO:teuthology.task.internal:Configuring sudo... 2024-04-22T23:18:52.650 DEBUG:teuthology.orchestra.run.smithi107:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-22T23:18:52.674 DEBUG:teuthology.orchestra.run.smithi157:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-22T23:18:52.710 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-22T23:18:52.720 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-22T23:18:52.720 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-22T23:18:52.737 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-22T23:18:52.767 DEBUG:teuthology.orchestra.run.smithi107:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-22T23:18:52.834 DEBUG:teuthology.orchestra.run.smithi107:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-22T23:18:52.879 DEBUG:teuthology.orchestra.run.smithi107:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-22T23:18:52.961 DEBUG:teuthology.orchestra.run.smithi107:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-22T23:18:53.000 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-22T23:18:53.000 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-22T23:18:53.064 DEBUG:teuthology.orchestra.run.smithi157:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-22T23:18:53.109 DEBUG:teuthology.orchestra.run.smithi157:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-22T23:18:53.154 DEBUG:teuthology.orchestra.run.smithi157:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-22T23:18:53.239 DEBUG:teuthology.orchestra.run.smithi157:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-22T23:18:53.279 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-22T23:18:53.280 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-22T23:18:53.345 DEBUG:teuthology.orchestra.run.smithi107:> sudo service rsyslog restart 2024-04-22T23:18:53.347 DEBUG:teuthology.orchestra.run.smithi157:> sudo service rsyslog restart 2024-04-22T23:18:53.381 INFO:teuthology.orchestra.run.smithi107.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-22T23:18:53.435 INFO:teuthology.orchestra.run.smithi157.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-22T23:18:53.925 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-22T23:18:53.933 INFO:teuthology.task.internal:Starting timer... 2024-04-22T23:18:53.933 INFO:teuthology.run_tasks:Running task pcp... 2024-04-22T23:18:53.981 INFO:teuthology.run_tasks:Running task selinux... 2024-04-22T23:18:54.014 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-22T23:18:54.014 DEBUG:teuthology.orchestra.run.smithi107:> sudo service auditd rotate 2024-04-22T23:18:54.059 INFO:teuthology.orchestra.run.smithi107.stdout:Rotating logs: 2024-04-22T23:18:54.061 DEBUG:teuthology.orchestra.run.smithi157:> sudo service auditd rotate 2024-04-22T23:18:54.130 INFO:teuthology.orchestra.run.smithi157.stdout:Rotating logs: 2024-04-22T23:18:54.133 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-22T23:18:54.133 DEBUG:teuthology.orchestra.run.smithi107:> /usr/sbin/getenforce 2024-04-22T23:18:54.154 INFO:teuthology.orchestra.run.smithi107.stdout:Permissive 2024-04-22T23:18:54.154 DEBUG:teuthology.orchestra.run.smithi157:> /usr/sbin/getenforce 2024-04-22T23:18:54.174 INFO:teuthology.orchestra.run.smithi157.stdout:Permissive 2024-04-22T23:18:54.174 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi107.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi157.front.sepia.ceph.com': 'permissive'} 2024-04-22T23:18:54.175 DEBUG:teuthology.orchestra.run.smithi107:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-22T23:18:54.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T23:18:54.219 DEBUG:teuthology.orchestra.run.smithi157:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-22T23:18:54.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T23:18:54.245 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-22T23:18:54.246 DEBUG:teuthology.orchestra.run.smithi107:> sudo /usr/sbin/setenforce permissive 2024-04-22T23:18:54.283 DEBUG:teuthology.orchestra.run.smithi157:> sudo /usr/sbin/setenforce permissive 2024-04-22T23:18:54.310 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-22T23:18:54.319 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-22T23:18:54.325 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-22T23:18:54.437 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-22T23:18:54.453 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-22T23:18:54.455 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi107.front.sepia.ceph.com,smithi157.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-22T23:22:21.119 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi107.front.sepia.ceph.com'), Remote(name='ubuntu@smithi157.front.sepia.ceph.com')] 2024-04-22T23:22:21.120 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-04-22T23:22:21.121 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-22T23:22:21.198 DEBUG:teuthology.orchestra.run.smithi107:> true 2024-04-22T23:22:21.281 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-04-22T23:22:21.282 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-04-22T23:22:21.282 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-22T23:22:21.357 DEBUG:teuthology.orchestra.run.smithi157:> true 2024-04-22T23:22:21.396 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-04-22T23:22:21.397 INFO:teuthology.run_tasks:Running task clock... 2024-04-22T23:22:21.409 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-22T23:22:21.409 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-22T23:22:21.409 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-22T23:22:21.411 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-22T23:22:21.412 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-22T23:22:21.438 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-22T23:22:21.451 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-22T23:22:21.464 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-22T23:22:21.473 INFO:teuthology.orchestra.run.smithi107.stderr:sudo: ntpd: command not found 2024-04-22T23:22:21.477 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-22T23:22:21.484 INFO:teuthology.orchestra.run.smithi107.stdout:506 Cannot talk to daemon 2024-04-22T23:22:21.497 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-22T23:22:21.499 INFO:teuthology.orchestra.run.smithi157.stderr:sudo: ntpd: command not found 2024-04-22T23:22:21.509 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-22T23:22:21.510 INFO:teuthology.orchestra.run.smithi157.stdout:506 Cannot talk to daemon 2024-04-22T23:22:21.523 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-22T23:22:21.536 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-22T23:22:21.569 INFO:teuthology.orchestra.run.smithi107.stderr:bash: line 1: ntpq: command not found 2024-04-22T23:22:21.572 INFO:teuthology.orchestra.run.smithi107.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-22T23:22:21.572 INFO:teuthology.orchestra.run.smithi107.stdout:=============================================================================== 2024-04-22T23:22:21.573 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T23:22:21.573 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T23:22:21.573 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T23:22:21.573 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T23:22:21.597 INFO:teuthology.orchestra.run.smithi157.stderr:bash: line 1: ntpq: command not found 2024-04-22T23:22:21.600 INFO:teuthology.orchestra.run.smithi157.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-22T23:22:21.601 INFO:teuthology.orchestra.run.smithi157.stdout:=============================================================================== 2024-04-22T23:22:21.601 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T23:22:21.601 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T23:22:21.601 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T23:22:21.601 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T23:22:21.602 INFO:teuthology.run_tasks:Running task install... 2024-04-22T23:22:21.610 DEBUG:teuthology.task.install:project ceph 2024-04-22T23:22:21.610 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '43be020184947e53516056c9931e1ac5bdbbb1a5'}} 2024-04-22T23:22:21.610 DEBUG:teuthology.task.install:config {'branch': 'quincy', 'exclude_packages': ['ceph-volume'], 'flavor': 'default', 'sha1': '43be020184947e53516056c9931e1ac5bdbbb1a5'} 2024-04-22T23:22:21.611 INFO:teuthology.task.install:Using flavor: default 2024-04-22T23:22:21.616 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-22T23:22:21.617 INFO:teuthology.task.install:extra packages: [] 2024-04-22T23:22:21.617 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '43be020184947e53516056c9931e1ac5bdbbb1a5', 'tag': None, 'wait_for_package': False} 2024-04-22T23:22:21.617 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-22T23:22:21.617 INFO:teuthology.packaging:ref: None 2024-04-22T23:22:21.617 INFO:teuthology.packaging:tag: None 2024-04-22T23:22:21.618 INFO:teuthology.packaging:branch: quincy 2024-04-22T23:22:21.618 INFO:teuthology.packaging:sha1: 43be020184947e53516056c9931e1ac5bdbbb1a5 2024-04-22T23:22:21.618 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-04-22T23:22:21.620 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '43be020184947e53516056c9931e1ac5bdbbb1a5', 'tag': None, 'wait_for_package': False} 2024-04-22T23:22:21.620 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-22T23:22:21.620 INFO:teuthology.packaging:ref: None 2024-04-22T23:22:21.621 INFO:teuthology.packaging:tag: None 2024-04-22T23:22:21.621 INFO:teuthology.packaging:branch: quincy 2024-04-22T23:22:21.621 INFO:teuthology.packaging:sha1: 43be020184947e53516056c9931e1ac5bdbbb1a5 2024-04-22T23:22:21.621 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-04-22T23:22:21.890 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/quincy/70d0cb4b9d576828afb0a05dcf8732894d343ece/centos/9/flavors/default/ 2024-04-22T23:22:21.891 INFO:teuthology.task.install.rpm:Package version is 17.2.7-884.g70d0cb4b 2024-04-22T23:22:21.924 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/quincy/70d0cb4b9d576828afb0a05dcf8732894d343ece/centos/9/flavors/default/ 2024-04-22T23:22:21.924 INFO:teuthology.task.install.rpm:Package version is 17.2.7-884.g70d0cb4b 2024-04-22T23:22:22.059 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/quincy/70d0cb4b9d576828afb0a05dcf8732894d343ece/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/quincy/70d0cb4b9d576828afb0a05dcf8732894d343ece/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/quincy/70d0cb4b9d576828afb0a05dcf8732894d343ece/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-22T23:22:22.059 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-22T23:22:22.059 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-22T23:22:22.083 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-22T23:22:22.083 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-22T23:22:22.084 INFO:teuthology.packaging:ref: None 2024-04-22T23:22:22.084 INFO:teuthology.packaging:tag: None 2024-04-22T23:22:22.084 INFO:teuthology.packaging:branch: quincy 2024-04-22T23:22:22.084 INFO:teuthology.packaging:sha1: 43be020184947e53516056c9931e1ac5bdbbb1a5 2024-04-22T23:22:22.084 DEBUG:teuthology.orchestra.run.smithi157:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-22T23:22:22.089 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/quincy/70d0cb4b9d576828afb0a05dcf8732894d343ece/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/quincy/70d0cb4b9d576828afb0a05dcf8732894d343ece/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/quincy/70d0cb4b9d576828afb0a05dcf8732894d343ece/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-22T23:22:22.089 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-22T23:22:22.089 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-22T23:22:22.114 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-22T23:22:22.114 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-22T23:22:22.114 INFO:teuthology.packaging:ref: None 2024-04-22T23:22:22.114 INFO:teuthology.packaging:tag: None 2024-04-22T23:22:22.114 INFO:teuthology.packaging:branch: quincy 2024-04-22T23:22:22.114 INFO:teuthology.packaging:sha1: 43be020184947e53516056c9931e1ac5bdbbb1a5 2024-04-22T23:22:22.115 DEBUG:teuthology.orchestra.run.smithi107:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-22T23:22:22.149 DEBUG:teuthology.orchestra.run.smithi157:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-22T23:22:22.178 DEBUG:teuthology.orchestra.run.smithi107:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-22T23:22:22.221 DEBUG:teuthology.orchestra.run.smithi157:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-22T23:22:22.252 DEBUG:teuthology.orchestra.run.smithi107:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-22T23:22:22.285 INFO:teuthology.orchestra.run.smithi157.stdout:check_obsoletes = 1 2024-04-22T23:22:22.286 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum clean all 2024-04-22T23:22:22.314 INFO:teuthology.orchestra.run.smithi107.stdout:check_obsoletes = 1 2024-04-22T23:22:22.316 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum clean all 2024-04-22T23:22:22.510 INFO:teuthology.orchestra.run.smithi157.stdout:48 files removed 2024-04-22T23:22:22.538 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-radosgw 2024-04-22T23:22:22.540 INFO:teuthology.orchestra.run.smithi107.stdout:48 files removed 2024-04-22T23:22:22.567 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-radosgw 2024-04-22T23:22:23.362 INFO:teuthology.orchestra.run.smithi157.stdout:ceph packages for x86_64 136 kB/s | 77 kB 00:00 2024-04-22T23:22:23.388 INFO:teuthology.orchestra.run.smithi107.stdout:ceph packages for x86_64 137 kB/s | 77 kB 00:00 2024-04-22T23:22:23.772 INFO:teuthology.orchestra.run.smithi107.stdout:ceph noarch packages 31 kB/s | 11 kB 00:00 2024-04-22T23:22:23.787 INFO:teuthology.orchestra.run.smithi157.stdout:ceph noarch packages 28 kB/s | 11 kB 00:00 2024-04-22T23:22:24.147 INFO:teuthology.orchestra.run.smithi107.stdout:ceph source packages 5.3 kB/s | 1.8 kB 00:00 2024-04-22T23:22:24.255 INFO:teuthology.orchestra.run.smithi157.stdout:ceph source packages 4.1 kB/s | 1.8 kB 00:00 2024-04-22T23:22:24.681 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.0 MB 00:00 2024-04-22T23:22:24.980 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-04-22T23:22:26.857 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - AppStream 17 MB/s | 19 MB 00:01 2024-04-22T23:22:27.656 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - AppStream 12 MB/s | 19 MB 00:01 2024-04-22T23:22:30.933 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.2 MB 00:00 2024-04-22T23:22:32.392 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - Extras packages 52 kB/s | 16 kB 00:00 2024-04-22T23:22:33.132 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - CRB 3.2 MB/s | 6.2 MB 00:01 2024-04-22T23:22:33.921 INFO:teuthology.orchestra.run.smithi107.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-04-22T23:22:35.524 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS Stream 9 - Extras packages 13 kB/s | 16 kB 00:01 2024-04-22T23:22:37.636 INFO:teuthology.orchestra.run.smithi157.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-04-22T23:22:38.984 INFO:teuthology.orchestra.run.smithi107.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-22T23:22:41.100 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-04-22T23:22:41.102 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-22T23:22:41.102 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-04-22T23:22:41.102 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-22T23:22:41.102 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-04-22T23:22:41.102 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-radosgw x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 14 M 2024-04-22T23:22:41.102 INFO:teuthology.orchestra.run.smithi107.stdout:Upgrading: 2024-04-22T23:22:41.102 INFO:teuthology.orchestra.run.smithi107.stdout: librados2 x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 3.1 M 2024-04-22T23:22:41.102 INFO:teuthology.orchestra.run.smithi107.stdout: librbd1 x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 2.9 M 2024-04-22T23:22:41.103 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-04-22T23:22:41.103 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-base x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 5.0 M 2024-04-22T23:22:41.103 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-common x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 22 M 2024-04-22T23:22:41.103 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-selinux x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 25 k 2024-04-22T23:22:41.103 INFO:teuthology.orchestra.run.smithi107.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-22T23:22:41.103 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs2 x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 638 k 2024-04-22T23:22:41.103 INFO:teuthology.orchestra.run.smithi107.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-22T23:22:41.103 INFO:teuthology.orchestra.run.smithi107.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-22T23:22:41.103 INFO:teuthology.orchestra.run.smithi107.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-22T23:22:41.103 INFO:teuthology.orchestra.run.smithi107.stdout: libradosstriper1 x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 457 k 2024-04-22T23:22:41.103 INFO:teuthology.orchestra.run.smithi107.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-22T23:22:41.103 INFO:teuthology.orchestra.run.smithi107.stdout: librgw2 x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 4.9 M 2024-04-22T23:22:41.103 INFO:teuthology.orchestra.run.smithi107.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-22T23:22:41.103 INFO:teuthology.orchestra.run.smithi107.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-22T23:22:41.104 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-argparse x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 45 k 2024-04-22T23:22:41.104 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-common x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 118 k 2024-04-22T23:22:41.104 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cephfs x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 156 k 2024-04-22T23:22:41.104 INFO:teuthology.orchestra.run.smithi107.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-22T23:22:41.104 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rados x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 320 k 2024-04-22T23:22:41.104 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rbd x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 293 k 2024-04-22T23:22:41.104 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rgw x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 99 k 2024-04-22T23:22:41.104 INFO:teuthology.orchestra.run.smithi107.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-22T23:22:41.104 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:22:41.104 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-04-22T23:22:41.104 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-22T23:22:41.104 INFO:teuthology.orchestra.run.smithi107.stdout:Install 22 Packages 2024-04-22T23:22:41.104 INFO:teuthology.orchestra.run.smithi107.stdout:Upgrade 2 Packages 2024-04-22T23:22:41.104 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:22:41.105 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 58 M 2024-04-22T23:22:41.105 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-04-22T23:22:41.886 INFO:teuthology.orchestra.run.smithi107.stdout:(1/24): ceph-base-17.2.7-884.g70d0cb4b.el7.x86_ 8.8 MB/s | 5.0 MB 00:00 2024-04-22T23:22:41.936 INFO:teuthology.orchestra.run.smithi107.stdout:(2/24): ceph-selinux-17.2.7-884.g70d0cb4b.el7.x 506 kB/s | 25 kB 00:00 2024-04-22T23:22:42.028 INFO:teuthology.orchestra.run.smithi107.stdout:(3/24): libcephfs2-17.2.7-884.g70d0cb4b.el7.x86 6.8 MB/s | 638 kB 00:00 2024-04-22T23:22:42.119 INFO:teuthology.orchestra.run.smithi107.stdout:(4/24): libradosstriper1-17.2.7-884.g70d0cb4b.e 4.9 MB/s | 457 kB 00:00 2024-04-22T23:22:42.561 INFO:teuthology.orchestra.run.smithi107.stdout:(5/24): librgw2-17.2.7-884.g70d0cb4b.el7.x86_64 11 MB/s | 4.9 MB 00:00 2024-04-22T23:22:42.703 INFO:teuthology.orchestra.run.smithi107.stdout:(6/24): ceph-radosgw-17.2.7-884.g70d0cb4b.el7.x 10 MB/s | 14 MB 00:01 2024-04-22T23:22:42.729 INFO:teuthology.orchestra.run.smithi107.stdout:(7/24): python3-ceph-argparse-17.2.7-884.g70d0c 270 kB/s | 45 kB 00:00 2024-04-22T23:22:42.734 INFO:teuthology.orchestra.run.smithi157.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-22T23:22:42.754 INFO:teuthology.orchestra.run.smithi107.stdout:(8/24): python3-ceph-common-17.2.7-884.g70d0cb4 2.3 MB/s | 118 kB 00:00 2024-04-22T23:22:42.788 INFO:teuthology.orchestra.run.smithi107.stdout:(9/24): python3-cephfs-17.2.7-884.g70d0cb4b.el7 2.6 MB/s | 156 kB 00:00 2024-04-22T23:22:42.813 INFO:teuthology.orchestra.run.smithi107.stdout:(10/24): python3-rados-17.2.7-884.g70d0cb4b.el7 5.3 MB/s | 320 kB 00:00 2024-04-22T23:22:42.846 INFO:teuthology.orchestra.run.smithi107.stdout:(11/24): python3-rbd-17.2.7-884.g70d0cb4b.el7.x 5.0 MB/s | 293 kB 00:00 2024-04-22T23:22:42.871 INFO:teuthology.orchestra.run.smithi107.stdout:(12/24): python3-rgw-17.2.7-884.g70d0cb4b.el7.x 1.7 MB/s | 99 kB 00:00 2024-04-22T23:22:42.972 INFO:teuthology.orchestra.run.smithi107.stdout:(13/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 453 kB/s | 45 kB 00:00 2024-04-22T23:22:42.997 INFO:teuthology.orchestra.run.smithi107.stdout:(14/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 160 kB 00:00 2024-04-22T23:22:43.056 INFO:teuthology.orchestra.run.smithi107.stdout:(15/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-04-22T23:22:43.089 INFO:teuthology.orchestra.run.smithi107.stdout:(16/24): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-04-22T23:22:43.115 INFO:teuthology.orchestra.run.smithi107.stdout:(17/24): python3-prettytable-0.7.2-27.el9.noarc 721 kB/s | 42 kB 00:00 2024-04-22T23:22:43.273 INFO:teuthology.orchestra.run.smithi107.stdout:(18/24): liboath-2.6.7-2.el9.x86_64.rpm 307 kB/s | 49 kB 00:00 2024-04-22T23:22:43.382 INFO:teuthology.orchestra.run.smithi107.stdout:(19/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.0 MB/s | 309 kB 00:00 2024-04-22T23:22:43.407 INFO:teuthology.orchestra.run.smithi107.stdout:(20/24): libunwind-1.6.2-1.el9.x86_64.rpm 504 kB/s | 67 kB 00:00 2024-04-22T23:22:43.549 INFO:teuthology.orchestra.run.smithi107.stdout:(21/24): thrift-0.15.0-2.el9.x86_64.rpm 9.5 MB/s | 1.6 MB 00:00 2024-04-22T23:22:43.666 INFO:teuthology.orchestra.run.smithi107.stdout:(22/24): librados2-17.2.7-884.g70d0cb4b.el7.x86 12 MB/s | 3.1 MB 00:00 2024-04-22T23:22:43.866 INFO:teuthology.orchestra.run.smithi107.stdout:(23/24): librbd1-17.2.7-884.g70d0cb4b.el7.x86_6 9.3 MB/s | 2.9 MB 00:00 2024-04-22T23:22:44.275 INFO:teuthology.orchestra.run.smithi107.stdout:(24/24): ceph-common-17.2.7-884.g70d0cb4b.el7.x 7.6 MB/s | 22 MB 00:02 2024-04-22T23:22:44.278 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:22:44.278 INFO:teuthology.orchestra.run.smithi107.stdout:Total 18 MB/s | 58 MB 00:03 2024-04-22T23:22:44.340 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-04-22T23:22:44.373 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-04-22T23:22:44.373 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-04-22T23:22:44.809 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-04-22T23:22:44.810 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-04-22T23:22:44.875 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-04-22T23:22:44.876 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-22T23:22:44.877 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2024-04-22T23:22:44.877 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-22T23:22:44.877 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-04-22T23:22:44.877 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-radosgw x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 14 M 2024-04-22T23:22:44.877 INFO:teuthology.orchestra.run.smithi157.stdout:Upgrading: 2024-04-22T23:22:44.877 INFO:teuthology.orchestra.run.smithi157.stdout: librados2 x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 3.1 M 2024-04-22T23:22:44.877 INFO:teuthology.orchestra.run.smithi157.stdout: librbd1 x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 2.9 M 2024-04-22T23:22:44.877 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2024-04-22T23:22:44.877 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-base x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 5.0 M 2024-04-22T23:22:44.877 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-common x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 22 M 2024-04-22T23:22:44.877 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-selinux x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 25 k 2024-04-22T23:22:44.877 INFO:teuthology.orchestra.run.smithi157.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-22T23:22:44.878 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs2 x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 638 k 2024-04-22T23:22:44.878 INFO:teuthology.orchestra.run.smithi157.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-22T23:22:44.878 INFO:teuthology.orchestra.run.smithi157.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-22T23:22:44.878 INFO:teuthology.orchestra.run.smithi157.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-22T23:22:44.878 INFO:teuthology.orchestra.run.smithi157.stdout: libradosstriper1 x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 457 k 2024-04-22T23:22:44.878 INFO:teuthology.orchestra.run.smithi157.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-22T23:22:44.878 INFO:teuthology.orchestra.run.smithi157.stdout: librgw2 x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 4.9 M 2024-04-22T23:22:44.878 INFO:teuthology.orchestra.run.smithi157.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-22T23:22:44.878 INFO:teuthology.orchestra.run.smithi157.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-22T23:22:44.878 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-argparse x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 45 k 2024-04-22T23:22:44.878 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-common x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 118 k 2024-04-22T23:22:44.878 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cephfs x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 156 k 2024-04-22T23:22:44.878 INFO:teuthology.orchestra.run.smithi157.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-22T23:22:44.879 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rados x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 320 k 2024-04-22T23:22:44.879 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rbd x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 293 k 2024-04-22T23:22:44.879 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rgw x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 99 k 2024-04-22T23:22:44.879 INFO:teuthology.orchestra.run.smithi157.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-22T23:22:44.879 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:22:44.879 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-04-22T23:22:44.879 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-22T23:22:44.879 INFO:teuthology.orchestra.run.smithi157.stdout:Install 22 Packages 2024-04-22T23:22:44.879 INFO:teuthology.orchestra.run.smithi157.stdout:Upgrade 2 Packages 2024-04-22T23:22:44.879 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:22:44.880 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 58 M 2024-04-22T23:22:44.880 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-04-22T23:22:45.685 INFO:teuthology.orchestra.run.smithi157.stdout:(1/24): ceph-base-17.2.7-884.g70d0cb4b.el7.x86_ 9.2 MB/s | 5.0 MB 00:00 2024-04-22T23:22:45.744 INFO:teuthology.orchestra.run.smithi157.stdout:(2/24): ceph-selinux-17.2.7-884.g70d0cb4b.el7.x 433 kB/s | 25 kB 00:00 2024-04-22T23:22:45.819 INFO:teuthology.orchestra.run.smithi157.stdout:(3/24): libcephfs2-17.2.7-884.g70d0cb4b.el7.x86 8.3 MB/s | 638 kB 00:00 2024-04-22T23:22:45.877 INFO:teuthology.orchestra.run.smithi157.stdout:(4/24): libradosstriper1-17.2.7-884.g70d0cb4b.e 7.7 MB/s | 457 kB 00:00 2024-04-22T23:22:46.052 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-04-22T23:22:46.261 INFO:teuthology.orchestra.run.smithi157.stdout:(5/24): librgw2-17.2.7-884.g70d0cb4b.el7.x86_64 13 MB/s | 4.9 MB 00:00 2024-04-22T23:22:46.312 INFO:teuthology.orchestra.run.smithi157.stdout:(6/24): python3-ceph-argparse-17.2.7-884.g70d0c 902 kB/s | 45 kB 00:00 2024-04-22T23:22:46.370 INFO:teuthology.orchestra.run.smithi157.stdout:(7/24): python3-ceph-common-17.2.7-884.g70d0cb4 2.0 MB/s | 118 kB 00:00 2024-04-22T23:22:46.372 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-04-22T23:22:46.437 INFO:teuthology.orchestra.run.smithi157.stdout:(8/24): python3-cephfs-17.2.7-884.g70d0cb4b.el7 2.3 MB/s | 156 kB 00:00 2024-04-22T23:22:46.486 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : librados2-2:17.2.7-884.g70d0cb4b.el7.x86_64 2/26 2024-04-22T23:22:46.496 INFO:teuthology.orchestra.run.smithi157.stdout:(9/24): python3-rados-17.2.7-884.g70d0cb4b.el7. 5.4 MB/s | 320 kB 00:00 2024-04-22T23:22:46.561 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librados2-2:17.2.7-884.g70d0cb4b.el7.x86_64 2/26 2024-04-22T23:22:46.562 INFO:teuthology.orchestra.run.smithi157.stdout:(10/24): python3-rbd-17.2.7-884.g70d0cb4b.el7.x 4.3 MB/s | 293 kB 00:00 2024-04-22T23:22:46.621 INFO:teuthology.orchestra.run.smithi157.stdout:(11/24): python3-rgw-17.2.7-884.g70d0cb4b.el7.x 1.7 MB/s | 99 kB 00:00 2024-04-22T23:22:46.639 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rados-2:17.2.7-884.g70d0cb4b.el7.x86_64 3/26 2024-04-22T23:22:46.688 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephfs2-2:17.2.7-884.g70d0cb4b.el7.x86_64 4/26 2024-04-22T23:22:46.759 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libcephfs2-2:17.2.7-884.g70d0cb4b.el7.x86_64 4/26 2024-04-22T23:22:46.804 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-04-22T23:22:46.842 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-04-22T23:22:47.171 INFO:teuthology.orchestra.run.smithi157.stdout:(12/24): ceph-radosgw-17.2.7-884.g70d0cb4b.el7. 7.0 MB/s | 14 MB 00:02 2024-04-22T23:22:47.298 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-04-22T23:22:47.332 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librgw2-2:17.2.7-884.g70d0cb4b.el7.x86_64 8/26 2024-04-22T23:22:47.394 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librgw2-2:17.2.7-884.g70d0cb4b.el7.x86_64 8/26 2024-04-22T23:22:47.423 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-04-22T23:22:47.466 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ceph-argparse-2:17.2.7-884.g70d0cb4b.el7.x 10/26 2024-04-22T23:22:47.503 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cephfs-2:17.2.7-884.g70d0cb4b.el7.x86_64 11/26 2024-04-22T23:22:47.513 INFO:teuthology.orchestra.run.smithi157.stdout:(13/24): ceph-common-17.2.7-884.g70d0cb4b.el7.x 9.5 MB/s | 22 MB 00:02 2024-04-22T23:22:47.538 INFO:teuthology.orchestra.run.smithi157.stdout:(14/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 175 kB/s | 160 kB 00:00 2024-04-22T23:22:47.565 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rgw-2:17.2.7-884.g70d0cb4b.el7.x86_64 12/26 2024-04-22T23:22:47.593 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libradosstriper1-2:17.2.7-884.g70d0cb4b.el7.x86_64 13/26 2024-04-22T23:22:47.666 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libradosstriper1-2:17.2.7-884.g70d0cb4b.el7.x86_64 13/26 2024-04-22T23:22:47.705 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-04-22T23:22:47.735 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-04-22T23:22:47.768 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-04-22T23:22:48.014 INFO:teuthology.orchestra.run.smithi157.stdout:(15/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 54 kB/s | 45 kB 00:00 2024-04-22T23:22:48.062 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-04-22T23:22:48.106 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : librbd1-2:17.2.7-884.g70d0cb4b.el7.x86_64 18/26 2024-04-22T23:22:48.195 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librbd1-2:17.2.7-884.g70d0cb4b.el7.x86_64 18/26 2024-04-22T23:22:48.269 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rbd-2:17.2.7-884.g70d0cb4b.el7.x86_64 19/26 2024-04-22T23:22:48.308 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ceph-common-2:17.2.7-884.g70d0cb4b.el7.x86 20/26 2024-04-22T23:22:48.497 INFO:teuthology.orchestra.run.smithi157.stdout:(16/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 305 kB/s | 292 kB 00:00 2024-04-22T23:22:48.639 INFO:teuthology.orchestra.run.smithi157.stdout:(17/24): python3-prettytable-0.7.2-27.el9.noarc 68 kB/s | 42 kB 00:00 2024-04-22T23:22:49.040 INFO:teuthology.orchestra.run.smithi157.stdout:(18/24): liboath-2.6.7-2.el9.x86_64.rpm 121 kB/s | 49 kB 00:00 2024-04-22T23:22:49.090 INFO:teuthology.orchestra.run.smithi157.stdout:(19/24): librdkafka-1.6.1-102.el9.x86_64.rpm 420 kB/s | 662 kB 00:01 2024-04-22T23:22:49.223 INFO:teuthology.orchestra.run.smithi157.stdout:(20/24): libunwind-1.6.2-1.el9.x86_64.rpm 367 kB/s | 67 kB 00:00 2024-04-22T23:22:49.299 INFO:teuthology.orchestra.run.smithi157.stdout:(21/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 386 kB/s | 309 kB 00:00 2024-04-22T23:22:49.482 INFO:teuthology.orchestra.run.smithi157.stdout:(22/24): librados2-17.2.7-884.g70d0cb4b.el7.x86 12 MB/s | 3.1 MB 00:00 2024-04-22T23:22:49.724 INFO:teuthology.orchestra.run.smithi157.stdout:(23/24): librbd1-17.2.7-884.g70d0cb4b.el7.x86_6 6.9 MB/s | 2.9 MB 00:00 2024-04-22T23:22:50.449 INFO:teuthology.orchestra.run.smithi157.stdout:(24/24): thrift-0.15.0-2.el9.x86_64.rpm 1.2 MB/s | 1.6 MB 00:01 2024-04-22T23:22:50.452 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:22:50.452 INFO:teuthology.orchestra.run.smithi157.stdout:Total 10 MB/s | 58 MB 00:05 2024-04-22T23:22:50.516 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-04-22T23:22:50.549 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-04-22T23:22:50.549 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-04-22T23:22:50.708 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-common-2:17.2.7-884.g70d0cb4b.el7.x86_64 21/26 2024-04-22T23:22:50.737 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-common-2:17.2.7-884.g70d0cb4b.el7.x86_64 21/26 2024-04-22T23:22:50.983 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-04-22T23:22:50.983 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-04-22T23:22:51.246 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-common-2:17.2.7-884.g70d0cb4b.el7.x86_64 21/26 2024-04-22T23:22:51.290 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-base-2:17.2.7-884.g70d0cb4b.el7.x86_64 22/26 2024-04-22T23:22:51.377 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-base-2:17.2.7-884.g70d0cb4b.el7.x86_64 22/26 2024-04-22T23:22:51.377 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-22T23:22:51.377 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-22T23:22:51.377 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:22:51.417 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-selinux-2:17.2.7-884.g70d0cb4b.el7.x86_64 23/26 2024-04-22T23:22:52.257 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-04-22T23:22:52.595 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-04-22T23:22:52.702 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : librados2-2:17.2.7-884.g70d0cb4b.el7.x86_64 2/26 2024-04-22T23:22:52.794 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librados2-2:17.2.7-884.g70d0cb4b.el7.x86_64 2/26 2024-04-22T23:22:52.873 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rados-2:17.2.7-884.g70d0cb4b.el7.x86_64 3/26 2024-04-22T23:22:52.904 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephfs2-2:17.2.7-884.g70d0cb4b.el7.x86_64 4/26 2024-04-22T23:22:52.991 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libcephfs2-2:17.2.7-884.g70d0cb4b.el7.x86_64 4/26 2024-04-22T23:22:53.037 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-04-22T23:22:53.066 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-04-22T23:22:53.501 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-04-22T23:22:53.531 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librgw2-2:17.2.7-884.g70d0cb4b.el7.x86_64 8/26 2024-04-22T23:22:53.594 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librgw2-2:17.2.7-884.g70d0cb4b.el7.x86_64 8/26 2024-04-22T23:22:53.631 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-04-22T23:22:53.676 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-ceph-argparse-2:17.2.7-884.g70d0cb4b.el7.x 10/26 2024-04-22T23:22:53.713 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cephfs-2:17.2.7-884.g70d0cb4b.el7.x86_64 11/26 2024-04-22T23:22:53.776 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rgw-2:17.2.7-884.g70d0cb4b.el7.x86_64 12/26 2024-04-22T23:22:53.803 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libradosstriper1-2:17.2.7-884.g70d0cb4b.el7.x86_64 13/26 2024-04-22T23:22:53.868 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libradosstriper1-2:17.2.7-884.g70d0cb4b.el7.x86_64 13/26 2024-04-22T23:22:53.907 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-04-22T23:22:53.936 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-04-22T23:22:53.969 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-04-22T23:22:54.261 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-04-22T23:22:54.299 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : librbd1-2:17.2.7-884.g70d0cb4b.el7.x86_64 18/26 2024-04-22T23:22:54.389 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librbd1-2:17.2.7-884.g70d0cb4b.el7.x86_64 18/26 2024-04-22T23:22:54.461 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rbd-2:17.2.7-884.g70d0cb4b.el7.x86_64 19/26 2024-04-22T23:22:54.501 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-ceph-common-2:17.2.7-884.g70d0cb4b.el7.x86 20/26 2024-04-22T23:22:56.901 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-common-2:17.2.7-884.g70d0cb4b.el7.x86_64 21/26 2024-04-22T23:22:57.056 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-common-2:17.2.7-884.g70d0cb4b.el7.x86_64 21/26 2024-04-22T23:22:57.561 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-common-2:17.2.7-884.g70d0cb4b.el7.x86_64 21/26 2024-04-22T23:22:57.617 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-base-2:17.2.7-884.g70d0cb4b.el7.x86_64 22/26 2024-04-22T23:22:57.717 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-base-2:17.2.7-884.g70d0cb4b.el7.x86_64 22/26 2024-04-22T23:22:57.717 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-22T23:22:57.717 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-22T23:22:57.717 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:22:57.752 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-selinux-2:17.2.7-884.g70d0cb4b.el7.x86_64 23/26 2024-04-22T23:23:01.000 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-selinux-2:17.2.7-884.g70d0cb4b.el7.x86_64 23/26 2024-04-22T23:23:01.001 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /sys 2024-04-22T23:23:01.001 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /proc 2024-04-22T23:23:01.001 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /mnt 2024-04-22T23:23:01.001 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /var/tmp 2024-04-22T23:23:01.001 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /home 2024-04-22T23:23:01.001 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /root 2024-04-22T23:23:01.001 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /tmp 2024-04-22T23:23:01.001 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:23:02.277 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-radosgw-2:17.2.7-884.g70d0cb4b.el7.x86_64 24/26 2024-04-22T23:23:02.338 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-884.g70d0cb4b.el7.x86_64 24/26 2024-04-22T23:23:02.338 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T23:23:02.338 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-22T23:23:02.338 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-22T23:23:02.338 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-22T23:23:02.338 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:23:02.340 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-04-22T23:23:02.421 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-04-22T23:23:02.422 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-04-22T23:23:03.776 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-04-22T23:23:03.776 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-base-2:17.2.7-884.g70d0cb4b.el7.x86_64 1/26 2024-04-22T23:23:03.776 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-common-2:17.2.7-884.g70d0cb4b.el7.x86_64 2/26 2024-04-22T23:23:03.776 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-radosgw-2:17.2.7-884.g70d0cb4b.el7.x86_64 3/26 2024-04-22T23:23:03.776 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-selinux-2:17.2.7-884.g70d0cb4b.el7.x86_64 4/26 2024-04-22T23:23:03.777 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephfs2-2:17.2.7-884.g70d0cb4b.el7.x86_64 5/26 2024-04-22T23:23:03.777 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libradosstriper1-2:17.2.7-884.g70d0cb4b.el7.x86_64 6/26 2024-04-22T23:23:03.777 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librgw2-2:17.2.7-884.g70d0cb4b.el7.x86_64 7/26 2024-04-22T23:23:03.777 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ceph-argparse-2:17.2.7-884.g70d0cb4b.el7.x 8/26 2024-04-22T23:23:03.777 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ceph-common-2:17.2.7-884.g70d0cb4b.el7.x86 9/26 2024-04-22T23:23:03.777 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cephfs-2:17.2.7-884.g70d0cb4b.el7.x86_64 10/26 2024-04-22T23:23:03.777 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rados-2:17.2.7-884.g70d0cb4b.el7.x86_64 11/26 2024-04-22T23:23:03.777 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rbd-2:17.2.7-884.g70d0cb4b.el7.x86_64 12/26 2024-04-22T23:23:03.777 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rgw-2:17.2.7-884.g70d0cb4b.el7.x86_64 13/26 2024-04-22T23:23:03.777 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-04-22T23:23:03.777 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-04-22T23:23:03.777 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-04-22T23:23:03.777 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-04-22T23:23:03.778 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-04-22T23:23:03.778 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-04-22T23:23:03.778 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-04-22T23:23:03.778 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-04-22T23:23:03.778 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-04-22T23:23:03.778 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados2-2:17.2.7-884.g70d0cb4b.el7.x86_64 23/26 2024-04-22T23:23:03.778 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-04-22T23:23:03.778 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librbd1-2:17.2.7-884.g70d0cb4b.el7.x86_64 25/26 2024-04-22T23:23:04.480 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-04-22T23:23:04.481 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:23:04.481 INFO:teuthology.orchestra.run.smithi107.stdout:Upgraded: 2024-04-22T23:23:04.481 INFO:teuthology.orchestra.run.smithi107.stdout: librados2-2:17.2.7-884.g70d0cb4b.el7.x86_64 2024-04-22T23:23:04.481 INFO:teuthology.orchestra.run.smithi107.stdout: librbd1-2:17.2.7-884.g70d0cb4b.el7.x86_64 2024-04-22T23:23:04.481 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-04-22T23:23:04.481 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-base-2:17.2.7-884.g70d0cb4b.el7.x86_64 2024-04-22T23:23:04.481 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-common-2:17.2.7-884.g70d0cb4b.el7.x86_64 2024-04-22T23:23:04.481 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-radosgw-2:17.2.7-884.g70d0cb4b.el7.x86_64 2024-04-22T23:23:04.481 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-selinux-2:17.2.7-884.g70d0cb4b.el7.x86_64 2024-04-22T23:23:04.481 INFO:teuthology.orchestra.run.smithi107.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-22T23:23:04.481 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs2-2:17.2.7-884.g70d0cb4b.el7.x86_64 2024-04-22T23:23:04.481 INFO:teuthology.orchestra.run.smithi107.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-22T23:23:04.482 INFO:teuthology.orchestra.run.smithi107.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-22T23:23:04.482 INFO:teuthology.orchestra.run.smithi107.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-22T23:23:04.482 INFO:teuthology.orchestra.run.smithi107.stdout: libradosstriper1-2:17.2.7-884.g70d0cb4b.el7.x86_64 2024-04-22T23:23:04.482 INFO:teuthology.orchestra.run.smithi107.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-22T23:23:04.482 INFO:teuthology.orchestra.run.smithi107.stdout: librgw2-2:17.2.7-884.g70d0cb4b.el7.x86_64 2024-04-22T23:23:04.482 INFO:teuthology.orchestra.run.smithi107.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-22T23:23:04.482 INFO:teuthology.orchestra.run.smithi107.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-22T23:23:04.482 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-argparse-2:17.2.7-884.g70d0cb4b.el7.x86_64 2024-04-22T23:23:04.482 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-common-2:17.2.7-884.g70d0cb4b.el7.x86_64 2024-04-22T23:23:04.482 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cephfs-2:17.2.7-884.g70d0cb4b.el7.x86_64 2024-04-22T23:23:04.482 INFO:teuthology.orchestra.run.smithi107.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-22T23:23:04.482 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rados-2:17.2.7-884.g70d0cb4b.el7.x86_64 2024-04-22T23:23:04.482 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rbd-2:17.2.7-884.g70d0cb4b.el7.x86_64 2024-04-22T23:23:04.483 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rgw-2:17.2.7-884.g70d0cb4b.el7.x86_64 2024-04-22T23:23:04.483 INFO:teuthology.orchestra.run.smithi107.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-22T23:23:04.483 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:23:04.483 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-04-22T23:23:04.776 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-test 2024-04-22T23:23:05.311 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:00:27 ago on Mon 22 Apr 2024 11:22:38 PM UTC. 2024-04-22T23:23:05.433 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-04-22T23:23:05.434 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-22T23:23:05.434 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-04-22T23:23:05.434 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-22T23:23:05.434 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-04-22T23:23:05.434 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-test x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 48 M 2024-04-22T23:23:05.434 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-04-22T23:23:05.435 INFO:teuthology.orchestra.run.smithi107.stdout: libcephsqlite x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 163 k 2024-04-22T23:23:05.435 INFO:teuthology.orchestra.run.smithi107.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-22T23:23:05.435 INFO:teuthology.orchestra.run.smithi107.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-22T23:23:05.435 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:23:05.435 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-04-22T23:23:05.435 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-22T23:23:05.435 INFO:teuthology.orchestra.run.smithi107.stdout:Install 4 Packages 2024-04-22T23:23:05.435 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:23:05.436 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 48 M 2024-04-22T23:23:05.436 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 192 M 2024-04-22T23:23:05.436 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-04-22T23:23:05.829 INFO:teuthology.orchestra.run.smithi107.stdout:(1/4): libcephsqlite-17.2.7-884.g70d0cb4b.el7.x 667 kB/s | 163 kB 00:00 2024-04-22T23:23:05.905 INFO:teuthology.orchestra.run.smithi107.stdout:(2/4): socat-1.7.4.1-5.el9.x86_64.rpm 953 kB/s | 305 kB 00:00 2024-04-22T23:23:05.998 INFO:teuthology.orchestra.run.smithi107.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 378 kB/s | 64 kB 00:00 2024-04-22T23:23:06.961 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-selinux-2:17.2.7-884.g70d0cb4b.el7.x86_64 23/26 2024-04-22T23:23:06.961 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /sys 2024-04-22T23:23:06.961 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /proc 2024-04-22T23:23:06.961 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /mnt 2024-04-22T23:23:06.961 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /var/tmp 2024-04-22T23:23:06.961 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /home 2024-04-22T23:23:06.961 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /root 2024-04-22T23:23:06.962 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /tmp 2024-04-22T23:23:06.962 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:23:07.899 INFO:teuthology.orchestra.run.smithi107.stdout:(4/4): ceph-test-17.2.7-884.g70d0cb4b.el7.x86_6 21 MB/s | 48 MB 00:02 2024-04-22T23:23:07.901 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:23:07.901 INFO:teuthology.orchestra.run.smithi107.stdout:Total 20 MB/s | 48 MB 00:02 2024-04-22T23:23:07.928 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-04-22T23:23:07.947 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-04-22T23:23:07.947 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-04-22T23:23:08.194 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-radosgw-2:17.2.7-884.g70d0cb4b.el7.x86_64 24/26 2024-04-22T23:23:08.272 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-884.g70d0cb4b.el7.x86_64 24/26 2024-04-22T23:23:08.272 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T23:23:08.272 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-22T23:23:08.272 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-22T23:23:08.272 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-22T23:23:08.272 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:23:08.274 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-04-22T23:23:08.307 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-04-22T23:23:08.308 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-04-22T23:23:08.363 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-04-22T23:23:08.363 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-04-22T23:23:08.846 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-04-22T23:23:08.924 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-04-22T23:23:08.973 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-04-22T23:23:09.001 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephsqlite-2:17.2.7-884.g70d0cb4b.el7.x86_64 3/4 2024-04-22T23:23:09.711 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-04-22T23:23:09.711 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-base-2:17.2.7-884.g70d0cb4b.el7.x86_64 1/26 2024-04-22T23:23:09.711 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-common-2:17.2.7-884.g70d0cb4b.el7.x86_64 2/26 2024-04-22T23:23:09.711 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-radosgw-2:17.2.7-884.g70d0cb4b.el7.x86_64 3/26 2024-04-22T23:23:09.711 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-selinux-2:17.2.7-884.g70d0cb4b.el7.x86_64 4/26 2024-04-22T23:23:09.711 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephfs2-2:17.2.7-884.g70d0cb4b.el7.x86_64 5/26 2024-04-22T23:23:09.711 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libradosstriper1-2:17.2.7-884.g70d0cb4b.el7.x86_64 6/26 2024-04-22T23:23:09.711 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librgw2-2:17.2.7-884.g70d0cb4b.el7.x86_64 7/26 2024-04-22T23:23:09.711 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-ceph-argparse-2:17.2.7-884.g70d0cb4b.el7.x 8/26 2024-04-22T23:23:09.712 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-ceph-common-2:17.2.7-884.g70d0cb4b.el7.x86 9/26 2024-04-22T23:23:09.712 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cephfs-2:17.2.7-884.g70d0cb4b.el7.x86_64 10/26 2024-04-22T23:23:09.712 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rados-2:17.2.7-884.g70d0cb4b.el7.x86_64 11/26 2024-04-22T23:23:09.712 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rbd-2:17.2.7-884.g70d0cb4b.el7.x86_64 12/26 2024-04-22T23:23:09.712 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rgw-2:17.2.7-884.g70d0cb4b.el7.x86_64 13/26 2024-04-22T23:23:09.712 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-04-22T23:23:09.712 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-04-22T23:23:09.712 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-04-22T23:23:09.712 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-04-22T23:23:09.712 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-04-22T23:23:09.712 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-04-22T23:23:09.712 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-04-22T23:23:09.713 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-04-22T23:23:09.713 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-04-22T23:23:09.713 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados2-2:17.2.7-884.g70d0cb4b.el7.x86_64 23/26 2024-04-22T23:23:09.713 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-04-22T23:23:09.713 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librbd1-2:17.2.7-884.g70d0cb4b.el7.x86_64 25/26 2024-04-22T23:23:10.433 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-04-22T23:23:10.433 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:23:10.433 INFO:teuthology.orchestra.run.smithi157.stdout:Upgraded: 2024-04-22T23:23:10.433 INFO:teuthology.orchestra.run.smithi157.stdout: librados2-2:17.2.7-884.g70d0cb4b.el7.x86_64 2024-04-22T23:23:10.433 INFO:teuthology.orchestra.run.smithi157.stdout: librbd1-2:17.2.7-884.g70d0cb4b.el7.x86_64 2024-04-22T23:23:10.433 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-04-22T23:23:10.433 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-base-2:17.2.7-884.g70d0cb4b.el7.x86_64 2024-04-22T23:23:10.433 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-common-2:17.2.7-884.g70d0cb4b.el7.x86_64 2024-04-22T23:23:10.433 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-radosgw-2:17.2.7-884.g70d0cb4b.el7.x86_64 2024-04-22T23:23:10.433 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-selinux-2:17.2.7-884.g70d0cb4b.el7.x86_64 2024-04-22T23:23:10.433 INFO:teuthology.orchestra.run.smithi157.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-22T23:23:10.434 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs2-2:17.2.7-884.g70d0cb4b.el7.x86_64 2024-04-22T23:23:10.434 INFO:teuthology.orchestra.run.smithi157.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-22T23:23:10.434 INFO:teuthology.orchestra.run.smithi157.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-22T23:23:10.434 INFO:teuthology.orchestra.run.smithi157.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-22T23:23:10.434 INFO:teuthology.orchestra.run.smithi157.stdout: libradosstriper1-2:17.2.7-884.g70d0cb4b.el7.x86_64 2024-04-22T23:23:10.434 INFO:teuthology.orchestra.run.smithi157.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-22T23:23:10.434 INFO:teuthology.orchestra.run.smithi157.stdout: librgw2-2:17.2.7-884.g70d0cb4b.el7.x86_64 2024-04-22T23:23:10.434 INFO:teuthology.orchestra.run.smithi157.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-22T23:23:10.434 INFO:teuthology.orchestra.run.smithi157.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-22T23:23:10.434 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-argparse-2:17.2.7-884.g70d0cb4b.el7.x86_64 2024-04-22T23:23:10.434 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-common-2:17.2.7-884.g70d0cb4b.el7.x86_64 2024-04-22T23:23:10.434 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cephfs-2:17.2.7-884.g70d0cb4b.el7.x86_64 2024-04-22T23:23:10.434 INFO:teuthology.orchestra.run.smithi157.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-22T23:23:10.434 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rados-2:17.2.7-884.g70d0cb4b.el7.x86_64 2024-04-22T23:23:10.435 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rbd-2:17.2.7-884.g70d0cb4b.el7.x86_64 2024-04-22T23:23:10.435 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rgw-2:17.2.7-884.g70d0cb4b.el7.x86_64 2024-04-22T23:23:10.435 INFO:teuthology.orchestra.run.smithi157.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-22T23:23:10.435 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:23:10.435 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-04-22T23:23:10.714 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-test 2024-04-22T23:23:11.247 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:00:29 ago on Mon 22 Apr 2024 11:22:42 PM UTC. 2024-04-22T23:23:11.368 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-04-22T23:23:11.370 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-22T23:23:11.370 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2024-04-22T23:23:11.370 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-22T23:23:11.371 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-04-22T23:23:11.371 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-test x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 48 M 2024-04-22T23:23:11.371 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2024-04-22T23:23:11.371 INFO:teuthology.orchestra.run.smithi157.stdout: libcephsqlite x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 163 k 2024-04-22T23:23:11.371 INFO:teuthology.orchestra.run.smithi157.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-22T23:23:11.371 INFO:teuthology.orchestra.run.smithi157.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-22T23:23:11.372 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:23:11.372 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-04-22T23:23:11.372 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-22T23:23:11.372 INFO:teuthology.orchestra.run.smithi157.stdout:Install 4 Packages 2024-04-22T23:23:11.372 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:23:11.372 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 48 M 2024-04-22T23:23:11.373 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 192 M 2024-04-22T23:23:11.373 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-04-22T23:23:11.696 INFO:teuthology.orchestra.run.smithi157.stdout:(1/4): libcephsqlite-17.2.7-884.g70d0cb4b.el7.x 633 kB/s | 163 kB 00:00 2024-04-22T23:23:11.805 INFO:teuthology.orchestra.run.smithi157.stdout:(2/4): socat-1.7.4.1-5.el9.x86_64.rpm 833 kB/s | 305 kB 00:00 2024-04-22T23:23:11.906 INFO:teuthology.orchestra.run.smithi157.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 304 kB/s | 64 kB 00:00 2024-04-22T23:23:13.345 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libcephsqlite-2:17.2.7-884.g70d0cb4b.el7.x86_64 3/4 2024-04-22T23:23:13.414 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-test-2:17.2.7-884.g70d0cb4b.el7.x86_64 4/4 2024-04-22T23:23:13.741 INFO:teuthology.orchestra.run.smithi157.stdout:(4/4): ceph-test-17.2.7-884.g70d0cb4b.el7.x86_6 21 MB/s | 48 MB 00:02 2024-04-22T23:23:13.745 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:23:13.745 INFO:teuthology.orchestra.run.smithi157.stdout:Total 20 MB/s | 48 MB 00:02 2024-04-22T23:23:13.771 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-04-22T23:23:13.791 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-04-22T23:23:13.792 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-04-22T23:23:14.156 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-04-22T23:23:14.157 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-04-22T23:23:14.673 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-04-22T23:23:14.742 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-04-22T23:23:14.790 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-04-22T23:23:14.818 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephsqlite-2:17.2.7-884.g70d0cb4b.el7.x86_64 3/4 2024-04-22T23:23:15.720 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-test-2:17.2.7-884.g70d0cb4b.el7.x86_64 4/4 2024-04-22T23:23:15.720 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-test-2:17.2.7-884.g70d0cb4b.el7.x86_64 1/4 2024-04-22T23:23:15.721 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephsqlite-2:17.2.7-884.g70d0cb4b.el7.x86_64 2/4 2024-04-22T23:23:15.721 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-04-22T23:23:15.958 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-04-22T23:23:15.959 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:23:15.959 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-04-22T23:23:15.959 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-test-2:17.2.7-884.g70d0cb4b.el7.x86_64 2024-04-22T23:23:15.959 INFO:teuthology.orchestra.run.smithi107.stdout: libcephsqlite-2:17.2.7-884.g70d0cb4b.el7.x86_64 2024-04-22T23:23:15.959 INFO:teuthology.orchestra.run.smithi107.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-22T23:23:15.959 INFO:teuthology.orchestra.run.smithi107.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-22T23:23:15.959 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:23:15.959 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-04-22T23:23:16.240 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph 2024-04-22T23:23:16.751 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:00:38 ago on Mon 22 Apr 2024 11:22:38 PM UTC. 2024-04-22T23:23:16.876 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-04-22T23:23:16.878 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-22T23:23:16.878 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-04-22T23:23:16.878 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-22T23:23:16.878 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-04-22T23:23:16.879 INFO:teuthology.orchestra.run.smithi107.stdout: ceph x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 6.5 k 2024-04-22T23:23:16.879 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-04-22T23:23:16.879 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mds x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 2.1 M 2024-04-22T23:23:16.879 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 1.4 M 2024-04-22T23:23:16.879 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-modules-core noarch 2:17.2.7-884.g70d0cb4b.el7 ceph-noarch 240 k 2024-04-22T23:23:16.879 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mon x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 3.9 M 2024-04-22T23:23:16.879 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-osd x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 15 M 2024-04-22T23:23:16.879 INFO:teuthology.orchestra.run.smithi107.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-22T23:23:16.879 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-22T23:23:16.879 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-22T23:23:16.879 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-22T23:23:16.879 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-22T23:23:16.879 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-22T23:23:16.879 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-22T23:23:16.880 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-22T23:23:16.880 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-22T23:23:16.880 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-22T23:23:16.880 INFO:teuthology.orchestra.run.smithi107.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-22T23:23:16.880 INFO:teuthology.orchestra.run.smithi107.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-22T23:23:16.880 INFO:teuthology.orchestra.run.smithi107.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-22T23:23:16.880 INFO:teuthology.orchestra.run.smithi107.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-22T23:23:16.880 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-22T23:23:16.880 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-22T23:23:16.880 INFO:teuthology.orchestra.run.smithi107.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-22T23:23:16.880 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-22T23:23:16.880 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-22T23:23:16.881 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-22T23:23:16.881 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-22T23:23:16.881 INFO:teuthology.orchestra.run.smithi107.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-22T23:23:16.881 INFO:teuthology.orchestra.run.smithi107.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-22T23:23:16.881 INFO:teuthology.orchestra.run.smithi107.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-22T23:23:16.881 INFO:teuthology.orchestra.run.smithi107.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-22T23:23:16.881 INFO:teuthology.orchestra.run.smithi107.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-22T23:23:16.881 INFO:teuthology.orchestra.run.smithi107.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-22T23:23:16.881 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:23:16.881 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-04-22T23:23:16.881 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-22T23:23:16.881 INFO:teuthology.orchestra.run.smithi107.stdout:Install 33 Packages 2024-04-22T23:23:16.881 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:23:16.882 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 27 M 2024-04-22T23:23:16.882 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 96 M 2024-04-22T23:23:16.882 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-04-22T23:23:17.874 INFO:teuthology.orchestra.run.smithi107.stdout:(1/33): ceph-17.2.7-884.g70d0cb4b.el7.x86_64.rp 38 kB/s | 6.5 kB 00:00 2024-04-22T23:23:18.091 INFO:teuthology.orchestra.run.smithi107.stdout:(2/33): ceph-mgr-17.2.7-884.g70d0cb4b.el7.x86_6 3.7 MB/s | 1.4 MB 00:00 2024-04-22T23:23:18.516 INFO:teuthology.orchestra.run.smithi107.stdout:(3/33): ceph-mon-17.2.7-884.g70d0cb4b.el7.x86_6 6.1 MB/s | 3.9 MB 00:00 2024-04-22T23:23:18.624 INFO:teuthology.orchestra.run.smithi107.stdout:(4/33): ceph-mgr-modules-core-17.2.7-884.g70d0c 2.2 MB/s | 240 kB 00:00 2024-04-22T23:23:18.917 INFO:teuthology.orchestra.run.smithi107.stdout:(5/33): ceph-osd-17.2.7-884.g70d0cb4b.el7.x86_6 19 MB/s | 15 MB 00:00 2024-04-22T23:23:19.033 INFO:teuthology.orchestra.run.smithi107.stdout:(6/33): ceph-mds-17.2.7-884.g70d0cb4b.el7.x86_6 1.6 MB/s | 2.1 MB 00:01 2024-04-22T23:23:19.058 INFO:teuthology.orchestra.run.smithi107.stdout:(7/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 584 kB/s | 253 kB 00:00 2024-04-22T23:23:19.092 INFO:teuthology.orchestra.run.smithi107.stdout:(8/33): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-22T23:23:19.170 INFO:teuthology.orchestra.run.smithi107.stdout:(9/33): python3-requests-2.25.1-8.el9.noarch.rp 1.6 MB/s | 125 kB 00:00 2024-04-22T23:23:19.219 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libcephsqlite-2:17.2.7-884.g70d0cb4b.el7.x86_64 3/4 2024-04-22T23:23:19.243 INFO:teuthology.orchestra.run.smithi107.stdout:(10/33): python3-cryptography-36.0.1-4.el9.x86_ 3.8 MB/s | 1.2 MB 00:00 2024-04-22T23:23:19.268 INFO:teuthology.orchestra.run.smithi107.stdout:(11/33): python3-urllib3-1.26.5-5.el9.noarch.rp 2.1 MB/s | 215 kB 00:00 2024-04-22T23:23:19.298 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-test-2:17.2.7-884.g70d0cb4b.el7.x86_64 4/4 2024-04-22T23:23:19.302 INFO:teuthology.orchestra.run.smithi107.stdout:(12/33): python3-pycparser-2.20-6.el9.noarch.rp 556 kB/s | 135 kB 00:00 2024-04-22T23:23:19.394 INFO:teuthology.orchestra.run.smithi107.stdout:(13/33): python3-markupsafe-1.1.1-12.el9.x86_64 277 kB/s | 35 kB 00:00 2024-04-22T23:23:19.452 INFO:teuthology.orchestra.run.smithi107.stdout:(14/33): python3-toml-0.10.2-6.el9.noarch.rpm 715 kB/s | 42 kB 00:00 2024-04-22T23:23:19.478 INFO:teuthology.orchestra.run.smithi107.stdout:(15/33): python3-pytz-2021.1-5.el9.noarch.rpm 290 kB/s | 51 kB 00:00 2024-04-22T23:23:19.511 INFO:teuthology.orchestra.run.smithi107.stdout:(16/33): python3-mako-1.1.4-6.el9.noarch.rpm 643 kB/s | 172 kB 00:00 2024-04-22T23:23:19.628 INFO:teuthology.orchestra.run.smithi107.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 247 kB/s | 43 kB 00:00 2024-04-22T23:23:19.687 INFO:teuthology.orchestra.run.smithi107.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 184 kB/s | 11 kB 00:00 2024-04-22T23:23:19.712 INFO:teuthology.orchestra.run.smithi107.stdout:(19/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 734 kB/s | 172 kB 00:00 2024-04-22T23:23:19.745 INFO:teuthology.orchestra.run.smithi107.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 301 kB/s | 18 kB 00:00 2024-04-22T23:23:19.779 INFO:teuthology.orchestra.run.smithi107.stdout:(21/33): python3-cherrypy-18.6.1-2.el9.noarch.r 1.3 MB/s | 358 kB 00:00 2024-04-22T23:23:19.804 INFO:teuthology.orchestra.run.smithi107.stdout:(22/33): python3-jaraco-collections-3.0.0-8.el9 251 kB/s | 23 kB 00:00 2024-04-22T23:23:19.830 INFO:teuthology.orchestra.run.smithi107.stdout:(23/33): python3-jaraco-functools-3.5.0-2.el9.n 232 kB/s | 19 kB 00:00 2024-04-22T23:23:19.863 INFO:teuthology.orchestra.run.smithi107.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 336 kB/s | 20 kB 00:00 2024-04-22T23:23:19.888 INFO:teuthology.orchestra.run.smithi107.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 553 kB/s | 46 kB 00:00 2024-04-22T23:23:19.914 INFO:teuthology.orchestra.run.smithi107.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 941 kB/s | 79 kB 00:00 2024-04-22T23:23:19.939 INFO:teuthology.orchestra.run.smithi107.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 325 kB/s | 16 kB 00:00 2024-04-22T23:23:19.964 INFO:teuthology.orchestra.run.smithi107.stdout:(28/33): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-04-22T23:23:19.981 INFO:teuthology.orchestra.run.smithi107.stdout:(29/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.3 MB/s | 90 kB 00:00 2024-04-22T23:23:20.006 INFO:teuthology.orchestra.run.smithi107.stdout:(30/33): python3-tempora-5.0.0-2.el9.noarch.rpm 532 kB/s | 36 kB 00:00 2024-04-22T23:23:20.032 INFO:teuthology.orchestra.run.smithi107.stdout:(31/33): python3-webob-1.8.7-6.el9.noarch.rpm 3.3 MB/s | 230 kB 00:00 2024-04-22T23:23:20.057 INFO:teuthology.orchestra.run.smithi107.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 399 kB/s | 20 kB 00:00 2024-04-22T23:23:20.099 INFO:teuthology.orchestra.run.smithi107.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.6 MB/s | 427 kB 00:00 2024-04-22T23:23:20.102 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:23:20.103 INFO:teuthology.orchestra.run.smithi107.stdout:Total 8.4 MB/s | 27 MB 00:03 2024-04-22T23:23:20.220 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-04-22T23:23:20.257 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-04-22T23:23:20.258 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-04-22T23:23:20.523 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-04-22T23:23:20.524 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-04-22T23:23:21.428 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-test-2:17.2.7-884.g70d0cb4b.el7.x86_64 4/4 2024-04-22T23:23:21.428 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-test-2:17.2.7-884.g70d0cb4b.el7.x86_64 1/4 2024-04-22T23:23:21.428 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephsqlite-2:17.2.7-884.g70d0cb4b.el7.x86_64 2/4 2024-04-22T23:23:21.428 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-04-22T23:23:21.656 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-04-22T23:23:21.707 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-04-22T23:23:21.707 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:23:21.707 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-04-22T23:23:21.707 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-test-2:17.2.7-884.g70d0cb4b.el7.x86_64 2024-04-22T23:23:21.707 INFO:teuthology.orchestra.run.smithi157.stdout: libcephsqlite-2:17.2.7-884.g70d0cb4b.el7.x86_64 2024-04-22T23:23:21.707 INFO:teuthology.orchestra.run.smithi157.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-22T23:23:21.707 INFO:teuthology.orchestra.run.smithi157.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-22T23:23:21.707 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:23:21.707 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-04-22T23:23:21.743 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-04-22T23:23:21.775 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-04-22T23:23:21.808 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-04-22T23:23:21.900 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-04-22T23:23:21.954 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph 2024-04-22T23:23:21.978 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-04-22T23:23:22.019 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-04-22T23:23:22.054 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-04-22T23:23:22.084 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-04-22T23:23:22.118 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-04-22T23:23:22.152 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-04-22T23:23:22.192 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-04-22T23:23:22.227 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-04-22T23:23:22.262 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-04-22T23:23:22.300 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-04-22T23:23:22.372 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-04-22T23:23:22.470 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:00:40 ago on Mon 22 Apr 2024 11:22:42 PM UTC. 2024-04-22T23:23:22.518 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-04-22T23:23:22.595 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-04-22T23:23:22.597 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-22T23:23:22.597 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2024-04-22T23:23:22.598 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-22T23:23:22.598 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-04-22T23:23:22.598 INFO:teuthology.orchestra.run.smithi157.stdout: ceph x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 6.5 k 2024-04-22T23:23:22.598 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2024-04-22T23:23:22.598 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mds x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 2.1 M 2024-04-22T23:23:22.598 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 1.4 M 2024-04-22T23:23:22.598 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-modules-core noarch 2:17.2.7-884.g70d0cb4b.el7 ceph-noarch 240 k 2024-04-22T23:23:22.598 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mon x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 3.9 M 2024-04-22T23:23:22.598 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-osd x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 15 M 2024-04-22T23:23:22.598 INFO:teuthology.orchestra.run.smithi157.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-22T23:23:22.598 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-22T23:23:22.598 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-22T23:23:22.598 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-22T23:23:22.598 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-22T23:23:22.599 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-22T23:23:22.599 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-22T23:23:22.599 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-22T23:23:22.599 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-22T23:23:22.599 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-22T23:23:22.599 INFO:teuthology.orchestra.run.smithi157.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-22T23:23:22.599 INFO:teuthology.orchestra.run.smithi157.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-22T23:23:22.599 INFO:teuthology.orchestra.run.smithi157.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-22T23:23:22.599 INFO:teuthology.orchestra.run.smithi157.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-22T23:23:22.599 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-22T23:23:22.599 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-22T23:23:22.599 INFO:teuthology.orchestra.run.smithi157.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-22T23:23:22.599 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-22T23:23:22.600 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-22T23:23:22.600 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-22T23:23:22.600 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-22T23:23:22.600 INFO:teuthology.orchestra.run.smithi157.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-22T23:23:22.600 INFO:teuthology.orchestra.run.smithi157.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-22T23:23:22.600 INFO:teuthology.orchestra.run.smithi157.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-22T23:23:22.600 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-22T23:23:22.600 INFO:teuthology.orchestra.run.smithi157.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-22T23:23:22.600 INFO:teuthology.orchestra.run.smithi157.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-22T23:23:22.600 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:23:22.600 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-04-22T23:23:22.600 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-22T23:23:22.600 INFO:teuthology.orchestra.run.smithi157.stdout:Install 33 Packages 2024-04-22T23:23:22.600 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:23:22.601 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-04-22T23:23:22.601 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 27 M 2024-04-22T23:23:22.601 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 96 M 2024-04-22T23:23:22.602 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-04-22T23:23:22.652 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 18/33 2024-04-22T23:23:22.691 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-04-22T23:23:22.748 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-04-22T23:23:22.803 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-04-22T23:23:22.989 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-04-22T23:23:23.037 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-04-22T23:23:23.106 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-04-22T23:23:23.202 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-04-22T23:23:23.244 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-04-22T23:23:23.358 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-04-22T23:23:23.392 INFO:teuthology.orchestra.run.smithi157.stdout:(1/33): ceph-17.2.7-884.g70d0cb4b.el7.x86_64.rp 40 kB/s | 6.5 kB 00:00 2024-04-22T23:23:23.507 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-884.g70d0cb4b.el7.n 28/33 2024-04-22T23:23:23.530 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-2:17.2.7-884.g70d0cb4b.el7.x86_64 29/33 2024-04-22T23:23:23.554 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-2:17.2.7-884.g70d0cb4b.el7.x86_64 29/33 2024-04-22T23:23:23.554 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T23:23:23.554 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-22T23:23:23.554 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-22T23:23:23.554 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-22T23:23:23.555 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:23:23.667 INFO:teuthology.orchestra.run.smithi157.stdout:(2/33): ceph-mgr-17.2.7-884.g70d0cb4b.el7.x86_6 3.2 MB/s | 1.4 MB 00:00 2024-04-22T23:23:23.742 INFO:teuthology.orchestra.run.smithi157.stdout:(3/33): ceph-mon-17.2.7-884.g70d0cb4b.el7.x86_6 11 MB/s | 3.9 MB 00:00 2024-04-22T23:23:23.851 INFO:teuthology.orchestra.run.smithi157.stdout:(4/33): ceph-mgr-modules-core-17.2.7-884.g70d0c 2.2 MB/s | 240 kB 00:00 2024-04-22T23:23:24.010 INFO:teuthology.orchestra.run.smithi157.stdout:(5/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.6 MB/s | 253 kB 00:00 2024-04-22T23:23:24.109 INFO:teuthology.orchestra.run.smithi157.stdout:(6/33): python3-cryptography-36.0.1-4.el9.x86_6 12 MB/s | 1.2 MB 00:00 2024-04-22T23:23:24.160 INFO:teuthology.orchestra.run.smithi157.stdout:(7/33): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-04-22T23:23:24.210 INFO:teuthology.orchestra.run.smithi157.stdout:(8/33): python3-pycparser-2.20-6.el9.noarch.rpm 2.6 MB/s | 135 kB 00:00 2024-04-22T23:23:24.302 INFO:teuthology.orchestra.run.smithi157.stdout:(9/33): python3-requests-2.25.1-8.el9.noarch.rp 1.3 MB/s | 125 kB 00:00 2024-04-22T23:23:24.469 INFO:teuthology.orchestra.run.smithi157.stdout:(10/33): ceph-osd-17.2.7-884.g70d0cb4b.el7.x86_ 19 MB/s | 15 MB 00:00 2024-04-22T23:23:24.511 INFO:teuthology.orchestra.run.smithi157.stdout:(11/33): ceph-mds-17.2.7-884.g70d0cb4b.el7.x86_ 1.6 MB/s | 2.1 MB 00:01 2024-04-22T23:23:24.536 INFO:teuthology.orchestra.run.smithi157.stdout:(12/33): python3-urllib3-1.26.5-5.el9.noarch.rp 918 kB/s | 215 kB 00:00 2024-04-22T23:23:24.620 INFO:teuthology.orchestra.run.smithi157.stdout:(13/33): python3-markupsafe-1.1.1-12.el9.x86_64 319 kB/s | 35 kB 00:00 2024-04-22T23:23:24.662 INFO:teuthology.orchestra.run.smithi157.stdout:(14/33): python3-mako-1.1.4-6.el9.noarch.rpm 895 kB/s | 172 kB 00:00 2024-04-22T23:23:24.679 INFO:teuthology.orchestra.run.smithi157.stdout:(15/33): python3-pytz-2021.1-5.el9.noarch.rpm 358 kB/s | 51 kB 00:00 2024-04-22T23:23:24.695 INFO:teuthology.orchestra.run.smithi157.stdout:(16/33): python3-toml-0.10.2-6.el9.noarch.rpm 551 kB/s | 42 kB 00:00 2024-04-22T23:23:24.864 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-osd-2:17.2.7-884.g70d0cb4b.el7.x86_64 30/33 2024-04-22T23:23:24.890 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-osd-2:17.2.7-884.g70d0cb4b.el7.x86_64 30/33 2024-04-22T23:23:24.890 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T23:23:24.890 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-22T23:23:24.890 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-22T23:23:24.890 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-22T23:23:24.890 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:23:25.104 INFO:teuthology.orchestra.run.smithi157.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 98 kB/s | 43 kB 00:00 2024-04-22T23:23:25.230 INFO:teuthology.orchestra.run.smithi157.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 85 kB/s | 11 kB 00:00 2024-04-22T23:23:25.264 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mon-2:17.2.7-884.g70d0cb4b.el7.x86_64 31/33 2024-04-22T23:23:25.287 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mon-2:17.2.7-884.g70d0cb4b.el7.x86_64 31/33 2024-04-22T23:23:25.287 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T23:23:25.287 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-22T23:23:25.287 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-22T23:23:25.287 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-22T23:23:25.287 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:23:25.296 INFO:teuthology.orchestra.run.smithi157.stdout:(19/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 278 kB/s | 172 kB 00:00 2024-04-22T23:23:25.355 INFO:teuthology.orchestra.run.smithi157.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 141 kB/s | 18 kB 00:00 2024-04-22T23:23:25.405 INFO:teuthology.orchestra.run.smithi157.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 213 kB/s | 23 kB 00:00 2024-04-22T23:23:25.431 INFO:teuthology.orchestra.run.smithi157.stdout:(22/33): python3-cherrypy-18.6.1-2.el9.noarch.r 488 kB/s | 358 kB 00:00 2024-04-22T23:23:25.481 INFO:teuthology.orchestra.run.smithi157.stdout:(23/33): python3-jaraco-functools-3.5.0-2.el9.n 155 kB/s | 19 kB 00:00 2024-04-22T23:23:25.499 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mds-2:17.2.7-884.g70d0cb4b.el7.x86_64 32/33 2024-04-22T23:23:25.514 INFO:teuthology.orchestra.run.smithi157.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 181 kB/s | 20 kB 00:00 2024-04-22T23:23:25.521 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mds-2:17.2.7-884.g70d0cb4b.el7.x86_64 32/33 2024-04-22T23:23:25.522 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T23:23:25.522 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-22T23:23:25.522 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-22T23:23:25.522 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-22T23:23:25.522 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:23:25.540 INFO:teuthology.orchestra.run.smithi157.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 425 kB/s | 46 kB 00:00 2024-04-22T23:23:25.585 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-2:17.2.7-884.g70d0cb4b.el7.x86_64 33/33 2024-04-22T23:23:25.648 INFO:teuthology.orchestra.run.smithi157.stdout:(26/33): python3-portend-3.1.0-2.el9.noarch.rpm 152 kB/s | 16 kB 00:00 2024-04-22T23:23:25.690 INFO:teuthology.orchestra.run.smithi157.stdout:(27/33): python3-more-itertools-8.12.0-2.el9.no 377 kB/s | 79 kB 00:00 2024-04-22T23:23:25.715 INFO:teuthology.orchestra.run.smithi157.stdout:(28/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.3 MB/s | 272 kB 00:00 2024-04-22T23:23:25.766 INFO:teuthology.orchestra.run.smithi157.stdout:(29/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 765 kB/s | 90 kB 00:00 2024-04-22T23:23:25.808 INFO:teuthology.orchestra.run.smithi157.stdout:(30/33): python3-tempora-5.0.0-2.el9.noarch.rpm 304 kB/s | 36 kB 00:00 2024-04-22T23:23:25.850 INFO:teuthology.orchestra.run.smithi157.stdout:(31/33): python3-webob-1.8.7-6.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-04-22T23:23:25.942 INFO:teuthology.orchestra.run.smithi157.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 150 kB/s | 20 kB 00:00 2024-04-22T23:23:25.967 INFO:teuthology.orchestra.run.smithi157.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 2.1 MB/s | 427 kB 00:00 2024-04-22T23:23:25.971 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:23:25.971 INFO:teuthology.orchestra.run.smithi157.stdout:Total 8.0 MB/s | 27 MB 00:03 2024-04-22T23:23:26.089 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-04-22T23:23:26.127 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-04-22T23:23:26.127 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-04-22T23:23:26.393 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-04-22T23:23:26.393 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-04-22T23:23:26.982 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-2:17.2.7-884.g70d0cb4b.el7.x86_64 33/33 2024-04-22T23:23:26.983 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-2:17.2.7-884.g70d0cb4b.el7.x86_64 1/33 2024-04-22T23:23:26.983 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mds-2:17.2.7-884.g70d0cb4b.el7.x86_64 2/33 2024-04-22T23:23:26.983 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-2:17.2.7-884.g70d0cb4b.el7.x86_64 3/33 2024-04-22T23:23:26.983 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mon-2:17.2.7-884.g70d0cb4b.el7.x86_64 4/33 2024-04-22T23:23:26.983 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-osd-2:17.2.7-884.g70d0cb4b.el7.x86_64 5/33 2024-04-22T23:23:26.983 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-884.g70d0cb4b.el7.n 6/33 2024-04-22T23:23:26.983 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-04-22T23:23:26.983 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-04-22T23:23:26.983 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-04-22T23:23:26.983 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-04-22T23:23:26.983 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-04-22T23:23:26.983 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/33 2024-04-22T23:23:26.983 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-04-22T23:23:26.983 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-04-22T23:23:26.984 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-04-22T23:23:26.984 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-04-22T23:23:26.984 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-04-22T23:23:26.984 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-04-22T23:23:26.984 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-04-22T23:23:26.984 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-04-22T23:23:26.984 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-04-22T23:23:26.984 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-04-22T23:23:26.984 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-04-22T23:23:26.984 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-04-22T23:23:26.984 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-04-22T23:23:26.984 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-04-22T23:23:26.984 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-04-22T23:23:26.984 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-04-22T23:23:26.985 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-04-22T23:23:26.985 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-04-22T23:23:26.985 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-04-22T23:23:26.985 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-04-22T23:23:27.561 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-04-22T23:23:27.620 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-04-22T23:23:27.652 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-04-22T23:23:27.693 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-04-22T23:23:27.786 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-04-22T23:23:27.838 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-04-22T23:23:27.838 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:23:27.838 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-04-22T23:23:27.838 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-2:17.2.7-884.g70d0cb4b.el7.x86_64 2024-04-22T23:23:27.838 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mds-2:17.2.7-884.g70d0cb4b.el7.x86_64 2024-04-22T23:23:27.838 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-2:17.2.7-884.g70d0cb4b.el7.x86_64 2024-04-22T23:23:27.838 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-modules-core-2:17.2.7-884.g70d0cb4b.el7.noarch 2024-04-22T23:23:27.838 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mon-2:17.2.7-884.g70d0cb4b.el7.x86_64 2024-04-22T23:23:27.839 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-osd-2:17.2.7-884.g70d0cb4b.el7.x86_64 2024-04-22T23:23:27.839 INFO:teuthology.orchestra.run.smithi107.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-22T23:23:27.839 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-22T23:23:27.839 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-22T23:23:27.839 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-22T23:23:27.839 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-22T23:23:27.839 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-22T23:23:27.839 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-22T23:23:27.839 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-22T23:23:27.839 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-22T23:23:27.839 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-22T23:23:27.839 INFO:teuthology.orchestra.run.smithi107.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-22T23:23:27.839 INFO:teuthology.orchestra.run.smithi107.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-22T23:23:27.840 INFO:teuthology.orchestra.run.smithi107.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-22T23:23:27.840 INFO:teuthology.orchestra.run.smithi107.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-22T23:23:27.840 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-22T23:23:27.840 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ply-3.11-14.el9.noarch 2024-04-22T23:23:27.840 INFO:teuthology.orchestra.run.smithi107.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-22T23:23:27.840 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-22T23:23:27.840 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-22T23:23:27.840 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-22T23:23:27.840 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-22T23:23:27.840 INFO:teuthology.orchestra.run.smithi107.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-22T23:23:27.840 INFO:teuthology.orchestra.run.smithi107.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-22T23:23:27.840 INFO:teuthology.orchestra.run.smithi107.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-22T23:23:27.840 INFO:teuthology.orchestra.run.smithi107.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-22T23:23:27.841 INFO:teuthology.orchestra.run.smithi107.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-22T23:23:27.841 INFO:teuthology.orchestra.run.smithi107.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-22T23:23:27.841 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:23:27.841 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-04-22T23:23:27.871 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-04-22T23:23:27.913 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-04-22T23:23:27.947 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-04-22T23:23:27.977 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-04-22T23:23:28.011 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-04-22T23:23:28.045 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-04-22T23:23:28.077 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-04-22T23:23:28.121 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-04-22T23:23:28.155 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-04-22T23:23:28.174 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-base 2024-04-22T23:23:28.193 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-04-22T23:23:28.267 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-04-22T23:23:28.404 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-04-22T23:23:28.477 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-04-22T23:23:28.530 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 18/33 2024-04-22T23:23:28.568 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-04-22T23:23:28.617 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-04-22T23:23:28.672 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-04-22T23:23:28.684 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:00:50 ago on Mon 22 Apr 2024 11:22:38 PM UTC. 2024-04-22T23:23:28.762 INFO:teuthology.orchestra.run.smithi107.stdout:Package ceph-base-2:17.2.7-884.g70d0cb4b.el7.x86_64 is already installed. 2024-04-22T23:23:28.802 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-04-22T23:23:28.803 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-04-22T23:23:28.803 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-04-22T23:23:28.866 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install cephadm 2024-04-22T23:23:28.867 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-04-22T23:23:28.922 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-04-22T23:23:28.992 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-04-22T23:23:29.081 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-04-22T23:23:29.121 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-04-22T23:23:29.236 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-04-22T23:23:29.368 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:00:51 ago on Mon 22 Apr 2024 11:22:38 PM UTC. 2024-04-22T23:23:29.385 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-884.g70d0cb4b.el7.n 28/33 2024-04-22T23:23:29.407 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-2:17.2.7-884.g70d0cb4b.el7.x86_64 29/33 2024-04-22T23:23:29.432 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-2:17.2.7-884.g70d0cb4b.el7.x86_64 29/33 2024-04-22T23:23:29.432 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T23:23:29.432 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-22T23:23:29.432 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-22T23:23:29.432 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-22T23:23:29.432 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:23:29.486 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-04-22T23:23:29.486 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-22T23:23:29.487 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-04-22T23:23:29.487 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-22T23:23:29.487 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-04-22T23:23:29.487 INFO:teuthology.orchestra.run.smithi107.stdout: cephadm noarch 2:17.2.7-884.g70d0cb4b.el7 ceph-noarch 82 k 2024-04-22T23:23:29.487 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:23:29.487 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-04-22T23:23:29.487 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-22T23:23:29.487 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2024-04-22T23:23:29.487 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:23:29.487 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 82 k 2024-04-22T23:23:29.487 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 372 k 2024-04-22T23:23:29.487 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-04-22T23:23:29.705 INFO:teuthology.orchestra.run.smithi107.stdout:cephadm-17.2.7-884.g70d0cb4b.el7.noarch.rpm 377 kB/s | 82 kB 00:00 2024-04-22T23:23:29.706 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:23:29.706 INFO:teuthology.orchestra.run.smithi107.stdout:Total 375 kB/s | 82 kB 00:00 2024-04-22T23:23:29.706 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-04-22T23:23:29.711 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-04-22T23:23:29.711 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-04-22T23:23:29.713 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-04-22T23:23:29.713 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-04-22T23:23:29.793 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-04-22T23:23:30.236 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: cephadm-2:17.2.7-884.g70d0cb4b.el7.noarch 1/1 2024-04-22T23:23:30.334 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephadm-2:17.2.7-884.g70d0cb4b.el7.noarch 1/1 2024-04-22T23:23:30.664 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: cephadm-2:17.2.7-884.g70d0cb4b.el7.noarch 1/1 2024-04-22T23:23:30.793 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-osd-2:17.2.7-884.g70d0cb4b.el7.x86_64 30/33 2024-04-22T23:23:30.821 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-osd-2:17.2.7-884.g70d0cb4b.el7.x86_64 30/33 2024-04-22T23:23:30.821 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T23:23:30.821 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-22T23:23:30.821 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-22T23:23:30.821 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-22T23:23:30.821 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:23:30.881 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephadm-2:17.2.7-884.g70d0cb4b.el7.noarch 1/1 2024-04-22T23:23:30.882 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:23:30.882 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-04-22T23:23:30.882 INFO:teuthology.orchestra.run.smithi107.stdout: cephadm-2:17.2.7-884.g70d0cb4b.el7.noarch 2024-04-22T23:23:30.882 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:23:30.882 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-04-22T23:23:31.127 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-immutable-object-cache 2024-04-22T23:23:31.203 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mon-2:17.2.7-884.g70d0cb4b.el7.x86_64 31/33 2024-04-22T23:23:31.226 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mon-2:17.2.7-884.g70d0cb4b.el7.x86_64 31/33 2024-04-22T23:23:31.227 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T23:23:31.227 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-22T23:23:31.227 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-22T23:23:31.227 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-22T23:23:31.227 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:23:31.429 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mds-2:17.2.7-884.g70d0cb4b.el7.x86_64 32/33 2024-04-22T23:23:31.452 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mds-2:17.2.7-884.g70d0cb4b.el7.x86_64 32/33 2024-04-22T23:23:31.452 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T23:23:31.452 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-22T23:23:31.452 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-22T23:23:31.452 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-22T23:23:31.453 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:23:31.507 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-2:17.2.7-884.g70d0cb4b.el7.x86_64 33/33 2024-04-22T23:23:31.655 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:00:53 ago on Mon 22 Apr 2024 11:22:38 PM UTC. 2024-04-22T23:23:31.777 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-04-22T23:23:31.778 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-22T23:23:31.778 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repo Size 2024-04-22T23:23:31.778 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-22T23:23:31.778 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-04-22T23:23:31.778 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 137 k 2024-04-22T23:23:31.778 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:23:31.778 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-04-22T23:23:31.778 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-22T23:23:31.778 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2024-04-22T23:23:31.778 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:23:31.778 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 137 k 2024-04-22T23:23:31.779 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 379 k 2024-04-22T23:23:31.779 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-04-22T23:23:31.995 INFO:teuthology.orchestra.run.smithi107.stdout:ceph-immutable-object-cache-17.2.7-884.g70d0cb4 635 kB/s | 137 kB 00:00 2024-04-22T23:23:31.996 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:23:31.996 INFO:teuthology.orchestra.run.smithi107.stdout:Total 629 kB/s | 137 kB 00:00 2024-04-22T23:23:31.996 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-04-22T23:23:32.004 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-04-22T23:23:32.004 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-04-22T23:23:32.059 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-04-22T23:23:32.060 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-04-22T23:23:32.206 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-04-22T23:23:32.288 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-884.g70d0cb4b.e 1/1 2024-04-22T23:23:32.312 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-884.g70d0cb4b.e 1/1 2024-04-22T23:23:32.313 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T23:23:32.313 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-22T23:23:32.313 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:23:33.079 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-2:17.2.7-884.g70d0cb4b.el7.x86_64 33/33 2024-04-22T23:23:33.080 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-2:17.2.7-884.g70d0cb4b.el7.x86_64 1/33 2024-04-22T23:23:33.080 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mds-2:17.2.7-884.g70d0cb4b.el7.x86_64 2/33 2024-04-22T23:23:33.080 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-2:17.2.7-884.g70d0cb4b.el7.x86_64 3/33 2024-04-22T23:23:33.080 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mon-2:17.2.7-884.g70d0cb4b.el7.x86_64 4/33 2024-04-22T23:23:33.080 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-osd-2:17.2.7-884.g70d0cb4b.el7.x86_64 5/33 2024-04-22T23:23:33.080 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-884.g70d0cb4b.el7.n 6/33 2024-04-22T23:23:33.080 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-04-22T23:23:33.080 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-04-22T23:23:33.080 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-04-22T23:23:33.080 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-04-22T23:23:33.080 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-04-22T23:23:33.081 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/33 2024-04-22T23:23:33.081 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-04-22T23:23:33.081 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-04-22T23:23:33.081 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-04-22T23:23:33.081 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-04-22T23:23:33.081 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-04-22T23:23:33.081 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-04-22T23:23:33.081 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-04-22T23:23:33.081 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-04-22T23:23:33.081 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-04-22T23:23:33.081 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-04-22T23:23:33.081 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-04-22T23:23:33.081 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-04-22T23:23:33.081 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-04-22T23:23:33.082 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-04-22T23:23:33.083 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-04-22T23:23:33.083 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-04-22T23:23:33.083 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-04-22T23:23:33.083 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-04-22T23:23:33.083 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-04-22T23:23:33.083 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-04-22T23:23:33.136 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-884.g70d0cb4b.e 1/1 2024-04-22T23:23:33.137 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:23:33.137 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-04-22T23:23:33.137 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-immutable-object-cache-2:17.2.7-884.g70d0cb4b.el7.x86_64 2024-04-22T23:23:33.137 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:23:33.137 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-04-22T23:23:33.366 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-mgr 2024-04-22T23:23:33.879 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:00:55 ago on Mon 22 Apr 2024 11:22:38 PM UTC. 2024-04-22T23:23:33.958 INFO:teuthology.orchestra.run.smithi107.stdout:Package ceph-mgr-2:17.2.7-884.g70d0cb4b.el7.x86_64 is already installed. 2024-04-22T23:23:34.000 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-04-22T23:23:34.001 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-04-22T23:23:34.001 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-04-22T23:23:34.029 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-04-22T23:23:34.029 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:23:34.029 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-04-22T23:23:34.029 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-2:17.2.7-884.g70d0cb4b.el7.x86_64 2024-04-22T23:23:34.029 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mds-2:17.2.7-884.g70d0cb4b.el7.x86_64 2024-04-22T23:23:34.030 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-2:17.2.7-884.g70d0cb4b.el7.x86_64 2024-04-22T23:23:34.030 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-modules-core-2:17.2.7-884.g70d0cb4b.el7.noarch 2024-04-22T23:23:34.030 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mon-2:17.2.7-884.g70d0cb4b.el7.x86_64 2024-04-22T23:23:34.030 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-osd-2:17.2.7-884.g70d0cb4b.el7.x86_64 2024-04-22T23:23:34.030 INFO:teuthology.orchestra.run.smithi157.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-22T23:23:34.030 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-22T23:23:34.030 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-22T23:23:34.030 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-22T23:23:34.030 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-22T23:23:34.030 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-22T23:23:34.030 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-22T23:23:34.030 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-22T23:23:34.030 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-22T23:23:34.030 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-22T23:23:34.031 INFO:teuthology.orchestra.run.smithi157.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-22T23:23:34.031 INFO:teuthology.orchestra.run.smithi157.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-22T23:23:34.031 INFO:teuthology.orchestra.run.smithi157.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-22T23:23:34.031 INFO:teuthology.orchestra.run.smithi157.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-22T23:23:34.031 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-22T23:23:34.031 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ply-3.11-14.el9.noarch 2024-04-22T23:23:34.031 INFO:teuthology.orchestra.run.smithi157.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-22T23:23:34.031 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-22T23:23:34.031 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-22T23:23:34.031 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-22T23:23:34.031 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-22T23:23:34.031 INFO:teuthology.orchestra.run.smithi157.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-22T23:23:34.031 INFO:teuthology.orchestra.run.smithi157.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-22T23:23:34.031 INFO:teuthology.orchestra.run.smithi157.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-22T23:23:34.032 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-22T23:23:34.032 INFO:teuthology.orchestra.run.smithi157.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-22T23:23:34.032 INFO:teuthology.orchestra.run.smithi157.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-22T23:23:34.032 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:23:34.032 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-04-22T23:23:34.071 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-mgr-dashboard 2024-04-22T23:23:34.384 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-base 2024-04-22T23:23:34.630 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:00:56 ago on Mon 22 Apr 2024 11:22:38 PM UTC. 2024-04-22T23:23:34.752 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-04-22T23:23:34.753 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-22T23:23:34.753 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-04-22T23:23:34.753 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-22T23:23:34.753 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-04-22T23:23:34.753 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-dashboard noarch 2:17.2.7-884.g70d0cb4b.el7 ceph-noarch 3.4 M 2024-04-22T23:23:34.753 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-04-22T23:23:34.753 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-grafana-dashboards noarch 2:17.2.7-884.g70d0cb4b.el7 ceph-noarch 24 k 2024-04-22T23:23:34.753 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-prometheus-alerts noarch 2:17.2.7-884.g70d0cb4b.el7 ceph-noarch 15 k 2024-04-22T23:23:34.753 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-04-22T23:23:34.753 INFO:teuthology.orchestra.run.smithi107.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-22T23:23:34.754 INFO:teuthology.orchestra.run.smithi107.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-22T23:23:34.754 INFO:teuthology.orchestra.run.smithi107.stdout:Installing weak dependencies: 2024-04-22T23:23:34.754 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-04-22T23:23:34.754 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:23:34.754 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-04-22T23:23:34.754 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-22T23:23:34.754 INFO:teuthology.orchestra.run.smithi107.stdout:Install 7 Packages 2024-04-22T23:23:34.754 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:23:34.755 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 3.7 M 2024-04-22T23:23:34.755 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 80 M 2024-04-22T23:23:34.755 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-04-22T23:23:34.896 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:00:52 ago on Mon 22 Apr 2024 11:22:42 PM UTC. 2024-04-22T23:23:34.975 INFO:teuthology.orchestra.run.smithi157.stdout:Package ceph-base-2:17.2.7-884.g70d0cb4b.el7.x86_64 is already installed. 2024-04-22T23:23:35.015 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-04-22T23:23:35.016 INFO:teuthology.orchestra.run.smithi157.stdout:Nothing to do. 2024-04-22T23:23:35.016 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-04-22T23:23:35.087 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install cephadm 2024-04-22T23:23:35.315 INFO:teuthology.orchestra.run.smithi107.stdout:(1/7): ceph-grafana-dashboards-17.2.7-884.g70d0 124 kB/s | 24 kB 00:00 2024-04-22T23:23:35.424 INFO:teuthology.orchestra.run.smithi107.stdout:(2/7): ceph-prometheus-alerts-17.2.7-884.g70d0c 48 kB/s | 15 kB 00:00 2024-04-22T23:23:35.549 INFO:teuthology.orchestra.run.smithi107.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 39 kB/s | 9.0 kB 00:00 2024-04-22T23:23:35.589 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:00:53 ago on Mon 22 Apr 2024 11:22:42 PM UTC. 2024-04-22T23:23:35.683 INFO:teuthology.orchestra.run.smithi107.stdout:(4/7): ceph-mgr-dashboard-17.2.7-884.g70d0cb4b. 5.9 MB/s | 3.4 MB 00:00 2024-04-22T23:23:35.708 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-04-22T23:23:35.709 INFO:teuthology.orchestra.run.smithi107.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 144 kB/s | 41 kB 00:00 2024-04-22T23:23:35.709 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-22T23:23:35.710 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2024-04-22T23:23:35.710 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-22T23:23:35.710 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-04-22T23:23:35.710 INFO:teuthology.orchestra.run.smithi157.stdout: cephadm noarch 2:17.2.7-884.g70d0cb4b.el7 ceph-noarch 82 k 2024-04-22T23:23:35.710 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:23:35.710 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-04-22T23:23:35.711 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-22T23:23:35.711 INFO:teuthology.orchestra.run.smithi157.stdout:Install 1 Package 2024-04-22T23:23:35.711 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:23:35.711 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 82 k 2024-04-22T23:23:35.711 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 372 k 2024-04-22T23:23:35.711 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-04-22T23:23:35.741 INFO:teuthology.orchestra.run.smithi107.stdout:(6/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 161 kB/s | 31 kB 00:00 2024-04-22T23:23:35.858 INFO:teuthology.orchestra.run.smithi107.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 1.0 MB/s | 188 kB 00:00 2024-04-22T23:23:35.862 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:23:35.862 INFO:teuthology.orchestra.run.smithi107.stdout:Total 3.3 MB/s | 3.7 MB 00:01 2024-04-22T23:23:35.862 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-04-22T23:23:35.870 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-04-22T23:23:35.870 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-04-22T23:23:35.932 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-04-22T23:23:35.932 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-04-22T23:23:36.020 INFO:teuthology.orchestra.run.smithi157.stdout:cephadm-17.2.7-884.g70d0cb4b.el7.noarch.rpm 265 kB/s | 82 kB 00:00 2024-04-22T23:23:36.021 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:23:36.021 INFO:teuthology.orchestra.run.smithi157.stdout:Total 263 kB/s | 82 kB 00:00 2024-04-22T23:23:36.021 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-04-22T23:23:36.026 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-04-22T23:23:36.026 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-04-22T23:23:36.027 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-04-22T23:23:36.028 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-04-22T23:23:36.105 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-04-22T23:23:36.232 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-04-22T23:23:36.293 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-04-22T23:23:36.327 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-04-22T23:23:36.374 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-04-22T23:23:36.397 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-04-22T23:23:36.426 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-884.g70d0cb4b.el7.no 5/7 2024-04-22T23:23:36.559 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: cephadm-2:17.2.7-884.g70d0cb4b.el7.noarch 1/1 2024-04-22T23:23:36.647 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephadm-2:17.2.7-884.g70d0cb4b.el7.noarch 1/1 2024-04-22T23:23:36.984 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: cephadm-2:17.2.7-884.g70d0cb4b.el7.noarch 1/1 2024-04-22T23:23:37.195 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephadm-2:17.2.7-884.g70d0cb4b.el7.noarch 1/1 2024-04-22T23:23:37.195 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:23:37.195 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-04-22T23:23:37.195 INFO:teuthology.orchestra.run.smithi157.stdout: cephadm-2:17.2.7-884.g70d0cb4b.el7.noarch 2024-04-22T23:23:37.195 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:23:37.195 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-04-22T23:23:37.390 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-immutable-object-cache 2024-04-22T23:23:37.416 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-884.g70d0cb4b.el7.n 6/7 2024-04-22T23:23:37.455 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-884.g70d0cb4b.el7.noarch 7/7 2024-04-22T23:23:37.911 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:00:55 ago on Mon 22 Apr 2024 11:22:42 PM UTC. 2024-04-22T23:23:38.034 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-04-22T23:23:38.035 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-22T23:23:38.035 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repo Size 2024-04-22T23:23:38.035 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-22T23:23:38.035 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-04-22T23:23:38.035 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 137 k 2024-04-22T23:23:38.035 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:23:38.035 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-04-22T23:23:38.035 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-22T23:23:38.035 INFO:teuthology.orchestra.run.smithi157.stdout:Install 1 Package 2024-04-22T23:23:38.035 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:23:38.036 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 137 k 2024-04-22T23:23:38.036 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 379 k 2024-04-22T23:23:38.036 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-04-22T23:23:38.342 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-immutable-object-cache-17.2.7-884.g70d0cb4 449 kB/s | 137 kB 00:00 2024-04-22T23:23:38.343 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:23:38.343 INFO:teuthology.orchestra.run.smithi157.stdout:Total 446 kB/s | 137 kB 00:00 2024-04-22T23:23:38.344 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-04-22T23:23:38.351 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-04-22T23:23:38.351 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-04-22T23:23:38.407 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-04-22T23:23:38.407 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-04-22T23:23:38.475 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-884.g70d0cb4b.el7.noarch 7/7 2024-04-22T23:23:38.475 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-884.g70d0cb4b.el7.n 1/7 2024-04-22T23:23:38.475 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-884.g70d0cb4b.el7.noarch 2/7 2024-04-22T23:23:38.475 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-884.g70d0cb4b.el7.no 3/7 2024-04-22T23:23:38.476 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-04-22T23:23:38.476 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-04-22T23:23:38.476 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-04-22T23:23:38.552 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-04-22T23:23:38.627 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-884.g70d0cb4b.e 1/1 2024-04-22T23:23:38.651 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-884.g70d0cb4b.e 1/1 2024-04-22T23:23:38.651 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T23:23:38.651 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-22T23:23:38.651 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:23:38.729 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-04-22T23:23:38.729 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:23:38.729 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-04-22T23:23:38.729 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-grafana-dashboards-2:17.2.7-884.g70d0cb4b.el7.noarch 2024-04-22T23:23:38.729 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-dashboard-2:17.2.7-884.g70d0cb4b.el7.noarch 2024-04-22T23:23:38.729 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-prometheus-alerts-2:17.2.7-884.g70d0cb4b.el7.noarch 2024-04-22T23:23:38.729 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-04-22T23:23:38.729 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-04-22T23:23:38.729 INFO:teuthology.orchestra.run.smithi107.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-22T23:23:38.729 INFO:teuthology.orchestra.run.smithi107.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-22T23:23:38.729 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:23:38.729 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-04-22T23:23:38.927 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-22T23:23:39.434 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:01 ago on Mon 22 Apr 2024 11:22:38 PM UTC. 2024-04-22T23:23:39.504 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-884.g70d0cb4b.e 1/1 2024-04-22T23:23:39.504 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:23:39.504 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-04-22T23:23:39.504 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-immutable-object-cache-2:17.2.7-884.g70d0cb4b.el7.x86_64 2024-04-22T23:23:39.504 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:23:39.504 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-04-22T23:23:39.554 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-04-22T23:23:39.555 INFO:teuthology.orchestra.run.smithi107.stdout:====================================================================================== 2024-04-22T23:23:39.555 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-04-22T23:23:39.555 INFO:teuthology.orchestra.run.smithi107.stdout:====================================================================================== 2024-04-22T23:23:39.555 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-04-22T23:23:39.555 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-884.g70d0cb4b.el7 ceph-noarch 7.4 M 2024-04-22T23:23:39.555 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-04-22T23:23:39.555 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-22T23:23:39.555 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-22T23:23:39.555 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-22T23:23:39.556 INFO:teuthology.orchestra.run.smithi107.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-22T23:23:39.556 INFO:teuthology.orchestra.run.smithi107.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-22T23:23:39.556 INFO:teuthology.orchestra.run.smithi107.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-22T23:23:39.556 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-22T23:23:39.556 INFO:teuthology.orchestra.run.smithi107.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-22T23:23:39.556 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-22T23:23:39.556 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-22T23:23:39.556 INFO:teuthology.orchestra.run.smithi107.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-22T23:23:39.556 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:23:39.556 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-04-22T23:23:39.556 INFO:teuthology.orchestra.run.smithi107.stdout:====================================================================================== 2024-04-22T23:23:39.556 INFO:teuthology.orchestra.run.smithi107.stdout:Install 12 Packages 2024-04-22T23:23:39.556 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:23:39.557 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 38 M 2024-04-22T23:23:39.557 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 209 M 2024-04-22T23:23:39.557 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-04-22T23:23:39.725 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-mgr 2024-04-22T23:23:40.133 INFO:teuthology.orchestra.run.smithi107.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 195 kB 00:00 2024-04-22T23:23:40.237 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:00:58 ago on Mon 22 Apr 2024 11:22:42 PM UTC. 2024-04-22T23:23:40.258 INFO:teuthology.orchestra.run.smithi107.stdout:(2/12): libgfortran-11.4.1-3.el9.x86_64.rpm 2.6 MB/s | 803 kB 00:00 2024-04-22T23:23:40.317 INFO:teuthology.orchestra.run.smithi157.stdout:Package ceph-mgr-2:17.2.7-884.g70d0cb4b.el7.x86_64 is already installed. 2024-04-22T23:23:40.359 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-04-22T23:23:40.360 INFO:teuthology.orchestra.run.smithi157.stdout:Nothing to do. 2024-04-22T23:23:40.360 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-04-22T23:23:40.425 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-mgr-dashboard 2024-04-22T23:23:40.692 INFO:teuthology.orchestra.run.smithi107.stdout:(3/12): ceph-mgr-diskprediction-local-17.2.7-88 10 MB/s | 7.4 MB 00:00 2024-04-22T23:23:40.726 INFO:teuthology.orchestra.run.smithi107.stdout:(4/12): flexiblas-3.0.4-8.el9.x86_64.rpm 55 kB/s | 33 kB 00:00 2024-04-22T23:23:40.942 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:00:58 ago on Mon 22 Apr 2024 11:22:42 PM UTC. 2024-04-22T23:23:41.063 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-04-22T23:23:41.064 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-22T23:23:41.064 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2024-04-22T23:23:41.064 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-22T23:23:41.064 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-04-22T23:23:41.064 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-dashboard noarch 2:17.2.7-884.g70d0cb4b.el7 ceph-noarch 3.4 M 2024-04-22T23:23:41.064 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2024-04-22T23:23:41.064 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-grafana-dashboards noarch 2:17.2.7-884.g70d0cb4b.el7 ceph-noarch 24 k 2024-04-22T23:23:41.064 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-prometheus-alerts noarch 2:17.2.7-884.g70d0cb4b.el7 ceph-noarch 15 k 2024-04-22T23:23:41.065 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-04-22T23:23:41.065 INFO:teuthology.orchestra.run.smithi157.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-22T23:23:41.065 INFO:teuthology.orchestra.run.smithi157.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-22T23:23:41.065 INFO:teuthology.orchestra.run.smithi157.stdout:Installing weak dependencies: 2024-04-22T23:23:41.065 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-04-22T23:23:41.065 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:23:41.065 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-04-22T23:23:41.065 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-22T23:23:41.065 INFO:teuthology.orchestra.run.smithi157.stdout:Install 7 Packages 2024-04-22T23:23:41.065 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:23:41.066 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 3.7 M 2024-04-22T23:23:41.066 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 80 M 2024-04-22T23:23:41.066 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-04-22T23:23:41.234 INFO:teuthology.orchestra.run.smithi107.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 34 kB/s | 18 kB 00:00 2024-04-22T23:23:41.334 INFO:teuthology.orchestra.run.smithi107.stdout:(6/12): openblas-0.3.21-2.el9.x86_64.rpm 58 kB/s | 35 kB 00:00 2024-04-22T23:23:41.921 INFO:teuthology.orchestra.run.smithi157.stdout:(1/7): ceph-grafana-dashboards-17.2.7-884.g70d0 50 kB/s | 24 kB 00:00 2024-04-22T23:23:41.947 INFO:teuthology.orchestra.run.smithi157.stdout:(2/7): ceph-prometheus-alerts-17.2.7-884.g70d0c 28 kB/s | 15 kB 00:00 2024-04-22T23:23:42.064 INFO:teuthology.orchestra.run.smithi157.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 64 kB/s | 9.0 kB 00:00 2024-04-22T23:23:42.106 INFO:teuthology.orchestra.run.smithi157.stdout:(4/7): python3-jwt-2.4.0-1.el9.noarch.rpm 258 kB/s | 41 kB 00:00 2024-04-22T23:23:42.131 INFO:teuthology.orchestra.run.smithi157.stdout:(5/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 462 kB/s | 31 kB 00:00 2024-04-22T23:23:42.198 INFO:teuthology.orchestra.run.smithi157.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 2.0 MB/s | 188 kB 00:00 2024-04-22T23:23:42.251 INFO:teuthology.orchestra.run.smithi107.stdout:(7/12): python3-devel-3.9.18-3.el9.x86_64.rpm 266 kB/s | 244 kB 00:00 2024-04-22T23:23:43.481 INFO:teuthology.orchestra.run.smithi157.stdout:(7/7): ceph-mgr-dashboard-17.2.7-884.g70d0cb4b. 1.6 MB/s | 3.4 MB 00:02 2024-04-22T23:23:43.486 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:23:43.486 INFO:teuthology.orchestra.run.smithi157.stdout:Total 1.5 MB/s | 3.7 MB 00:02 2024-04-22T23:23:43.486 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-04-22T23:23:43.495 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-04-22T23:23:43.495 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-04-22T23:23:43.560 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-04-22T23:23:43.562 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-04-22T23:23:43.860 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-04-22T23:23:43.933 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-04-22T23:23:43.967 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-04-22T23:23:44.022 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-04-22T23:23:44.053 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-04-22T23:23:44.082 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-884.g70d0cb4b.el7.no 5/7 2024-04-22T23:23:44.751 INFO:teuthology.orchestra.run.smithi107.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 681 kB/s | 3.0 MB 00:04 2024-04-22T23:23:45.096 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-884.g70d0cb4b.el7.n 6/7 2024-04-22T23:23:45.128 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-884.g70d0cb4b.el7.noarch 7/7 2024-04-22T23:23:45.643 INFO:teuthology.orchestra.run.smithi107.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 281 kB/s | 250 kB 00:00 2024-04-22T23:23:46.181 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-884.g70d0cb4b.el7.noarch 7/7 2024-04-22T23:23:46.181 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-884.g70d0cb4b.el7.n 1/7 2024-04-22T23:23:46.182 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-884.g70d0cb4b.el7.noarch 2/7 2024-04-22T23:23:46.182 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-884.g70d0cb4b.el7.no 3/7 2024-04-22T23:23:46.182 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-04-22T23:23:46.182 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-04-22T23:23:46.182 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-04-22T23:23:46.427 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-04-22T23:23:46.427 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:23:46.427 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-04-22T23:23:46.428 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-grafana-dashboards-2:17.2.7-884.g70d0cb4b.el7.noarch 2024-04-22T23:23:46.428 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-dashboard-2:17.2.7-884.g70d0cb4b.el7.noarch 2024-04-22T23:23:46.428 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-prometheus-alerts-2:17.2.7-884.g70d0cb4b.el7.noarch 2024-04-22T23:23:46.428 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-04-22T23:23:46.428 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-04-22T23:23:46.428 INFO:teuthology.orchestra.run.smithi157.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-22T23:23:46.428 INFO:teuthology.orchestra.run.smithi157.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-22T23:23:46.428 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:23:46.428 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-04-22T23:23:46.645 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-22T23:23:47.158 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:05 ago on Mon 22 Apr 2024 11:22:42 PM UTC. 2024-04-22T23:23:47.279 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-04-22T23:23:47.280 INFO:teuthology.orchestra.run.smithi157.stdout:====================================================================================== 2024-04-22T23:23:47.280 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2024-04-22T23:23:47.280 INFO:teuthology.orchestra.run.smithi157.stdout:====================================================================================== 2024-04-22T23:23:47.281 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-04-22T23:23:47.281 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-884.g70d0cb4b.el7 ceph-noarch 7.4 M 2024-04-22T23:23:47.281 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2024-04-22T23:23:47.281 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-22T23:23:47.281 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-22T23:23:47.281 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-22T23:23:47.281 INFO:teuthology.orchestra.run.smithi157.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-22T23:23:47.281 INFO:teuthology.orchestra.run.smithi157.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-22T23:23:47.281 INFO:teuthology.orchestra.run.smithi157.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-22T23:23:47.281 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-22T23:23:47.281 INFO:teuthology.orchestra.run.smithi157.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-22T23:23:47.281 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-22T23:23:47.282 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-22T23:23:47.282 INFO:teuthology.orchestra.run.smithi157.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-22T23:23:47.282 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:23:47.282 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-04-22T23:23:47.282 INFO:teuthology.orchestra.run.smithi157.stdout:====================================================================================== 2024-04-22T23:23:47.282 INFO:teuthology.orchestra.run.smithi157.stdout:Install 12 Packages 2024-04-22T23:23:47.282 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:23:47.283 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 38 M 2024-04-22T23:23:47.283 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 209 M 2024-04-22T23:23:47.283 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-04-22T23:23:47.723 INFO:teuthology.orchestra.run.smithi157.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 791 kB/s | 195 kB 00:00 2024-04-22T23:23:47.726 INFO:teuthology.orchestra.run.smithi107.stdout:(10/12): openblas-openmp-0.3.21-2.el9.x86_64.rp 747 kB/s | 4.7 MB 00:06 2024-04-22T23:23:47.815 INFO:teuthology.orchestra.run.smithi157.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 357 kB/s | 33 kB 00:00 2024-04-22T23:23:47.857 INFO:teuthology.orchestra.run.smithi157.stdout:(3/12): libgfortran-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 803 kB 00:00 2024-04-22T23:23:47.940 INFO:teuthology.orchestra.run.smithi157.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 218 kB/s | 18 kB 00:00 2024-04-22T23:23:47.999 INFO:teuthology.orchestra.run.smithi157.stdout:(5/12): openblas-0.3.21-2.el9.x86_64.rpm 606 kB/s | 35 kB 00:00 2024-04-22T23:23:48.049 INFO:teuthology.orchestra.run.smithi157.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 13 MB/s | 3.0 MB 00:00 2024-04-22T23:23:48.099 INFO:teuthology.orchestra.run.smithi157.stdout:(7/12): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-04-22T23:23:48.275 INFO:teuthology.orchestra.run.smithi157.stdout:(8/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 17 MB/s | 4.7 MB 00:00 2024-04-22T23:23:48.341 INFO:teuthology.orchestra.run.smithi157.stdout:(9/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 21 MB/s | 5.1 MB 00:00 2024-04-22T23:23:48.367 INFO:teuthology.orchestra.run.smithi157.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 2.7 MB/s | 250 kB 00:00 2024-04-22T23:23:48.801 INFO:teuthology.orchestra.run.smithi157.stdout:(11/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2024-04-22T23:23:49.427 INFO:teuthology.orchestra.run.smithi107.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 728 kB/s | 5.1 MB 00:07 2024-04-22T23:23:51.900 INFO:teuthology.orchestra.run.smithi157.stdout:(12/12): ceph-mgr-diskprediction-local-17.2.7-8 1.7 MB/s | 7.4 MB 00:04 2024-04-22T23:23:51.904 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:23:51.904 INFO:teuthology.orchestra.run.smithi157.stdout:Total 8.2 MB/s | 38 MB 00:04 2024-04-22T23:23:52.261 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-04-22T23:23:52.288 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-04-22T23:23:52.288 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-04-22T23:23:52.695 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-04-22T23:23:52.695 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-04-22T23:23:53.463 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-04-22T23:23:53.538 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-04-22T23:23:53.582 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-04-22T23:23:53.682 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-04-22T23:23:53.725 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-04-22T23:23:53.903 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-22T23:23:53.932 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-22T23:23:54.009 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-04-22T23:23:54.656 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-04-22T23:23:54.771 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-04-22T23:23:55.999 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-04-22T23:23:57.059 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-04-22T23:23:57.090 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-884.g70d0cb 12/12 2024-04-22T23:23:59.588 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-884.g70d0cb 12/12 2024-04-22T23:23:59.588 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-884.g70d0cb 1/12 2024-04-22T23:23:59.588 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-04-22T23:23:59.588 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-04-22T23:23:59.588 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-04-22T23:23:59.588 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-04-22T23:23:59.588 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-04-22T23:23:59.588 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-04-22T23:23:59.588 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-04-22T23:23:59.589 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-04-22T23:23:59.589 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-04-22T23:23:59.589 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-04-22T23:24:00.027 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-04-22T23:24:00.027 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:24:00.027 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-04-22T23:24:00.027 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-diskprediction-local-2:17.2.7-884.g70d0cb4b.el7.noarch 2024-04-22T23:24:00.027 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-22T23:24:00.027 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-22T23:24:00.027 INFO:teuthology.orchestra.run.smithi157.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-22T23:24:00.027 INFO:teuthology.orchestra.run.smithi157.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-22T23:24:00.027 INFO:teuthology.orchestra.run.smithi157.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-22T23:24:00.027 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-22T23:24:00.028 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-22T23:24:00.028 INFO:teuthology.orchestra.run.smithi157.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-22T23:24:00.028 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-22T23:24:00.028 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-22T23:24:00.028 INFO:teuthology.orchestra.run.smithi157.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-22T23:24:00.028 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:24:00.028 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-04-22T23:24:00.380 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-mgr-rook 2024-04-22T23:24:00.914 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:18 ago on Mon 22 Apr 2024 11:22:42 PM UTC. 2024-04-22T23:24:01.047 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-04-22T23:24:01.047 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-22T23:24:01.047 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2024-04-22T23:24:01.047 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-22T23:24:01.047 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-04-22T23:24:01.047 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-rook noarch 2:17.2.7-884.g70d0cb4b.el7 ceph-noarch 47 k 2024-04-22T23:24:01.047 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2024-04-22T23:24:01.047 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-22T23:24:01.047 INFO:teuthology.orchestra.run.smithi157.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-22T23:24:01.048 INFO:teuthology.orchestra.run.smithi157.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-22T23:24:01.048 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-22T23:24:01.048 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-22T23:24:01.048 INFO:teuthology.orchestra.run.smithi157.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-22T23:24:01.048 INFO:teuthology.orchestra.run.smithi157.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-22T23:24:01.048 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-22T23:24:01.048 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-22T23:24:01.048 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-22T23:24:01.048 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-22T23:24:01.048 INFO:teuthology.orchestra.run.smithi157.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-22T23:24:01.048 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:24:01.048 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-04-22T23:24:01.048 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-22T23:24:01.048 INFO:teuthology.orchestra.run.smithi157.stdout:Install 13 Packages 2024-04-22T23:24:01.049 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:24:01.049 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 2.2 M 2024-04-22T23:24:01.049 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 26 M 2024-04-22T23:24:01.049 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-04-22T23:24:01.865 INFO:teuthology.orchestra.run.smithi157.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 108 kB/s | 19 kB 00:00 2024-04-22T23:24:01.890 INFO:teuthology.orchestra.run.smithi157.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 126 kB/s | 26 kB 00:00 2024-04-22T23:24:01.982 INFO:teuthology.orchestra.run.smithi157.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.9 MB/s | 222 kB 00:00 2024-04-22T23:24:02.008 INFO:teuthology.orchestra.run.smithi157.stdout:(4/13): ceph-mgr-rook-17.2.7-884.g70d0cb4b.el7. 148 kB/s | 47 kB 00:00 2024-04-22T23:24:02.033 INFO:teuthology.orchestra.run.smithi157.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.1 MB/s | 159 kB 00:00 2024-04-22T23:24:02.075 INFO:teuthology.orchestra.run.smithi157.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 3.0 MB/s | 279 kB 00:00 2024-04-22T23:24:02.100 INFO:teuthology.orchestra.run.smithi157.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 583 kB/s | 54 kB 00:00 2024-04-22T23:24:02.417 INFO:teuthology.orchestra.run.smithi157.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 84 kB/s | 32 kB 00:00 2024-04-22T23:24:02.451 INFO:teuthology.orchestra.run.smithi157.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 40 kB/s | 15 kB 00:00 2024-04-22T23:24:02.659 INFO:teuthology.orchestra.run.smithi157.stdout:(10/13): python3-google-auth-2.28.2-1.el9.noarc 363 kB/s | 203 kB 00:00 2024-04-22T23:24:02.685 INFO:teuthology.orchestra.run.smithi157.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 253 kB/s | 59 kB 00:00 2024-04-22T23:24:02.826 INFO:teuthology.orchestra.run.smithi157.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 538 kB/s | 90 kB 00:00 2024-04-22T23:24:02.885 INFO:teuthology.orchestra.run.smithi157.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 2.2 MB/s | 1.0 MB 00:00 2024-04-22T23:24:02.890 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:24:02.890 INFO:teuthology.orchestra.run.smithi157.stdout:Total 1.2 MB/s | 2.2 MB 00:01 2024-04-22T23:24:02.954 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-04-22T23:24:02.968 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-04-22T23:24:02.968 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-04-22T23:24:03.052 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-04-22T23:24:03.053 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-04-22T23:24:03.645 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-04-22T23:24:03.808 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-04-22T23:24:03.861 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-04-22T23:24:03.917 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-04-22T23:24:03.954 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-04-22T23:24:03.996 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-04-22T23:24:04.091 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-04-22T23:24:04.200 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 7/13 2024-04-22T23:24:04.247 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-04-22T23:24:04.969 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-04-22T23:24:05.028 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-04-22T23:24:05.061 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-04-22T23:24:05.107 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-04-22T23:24:05.132 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-rook-2:17.2.7-884.g70d0cb4b.el7.noarch 13/13 2024-04-22T23:24:05.871 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-884.g70d0cb4b.el7.noarch 13/13 2024-04-22T23:24:05.871 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-rook-2:17.2.7-884.g70d0cb4b.el7.noarch 1/13 2024-04-22T23:24:05.872 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-04-22T23:24:05.872 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-04-22T23:24:05.872 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-04-22T23:24:05.872 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-04-22T23:24:05.872 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-04-22T23:24:05.872 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-04-22T23:24:05.872 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-04-22T23:24:05.872 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-04-22T23:24:05.872 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 10/13 2024-04-22T23:24:05.872 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-04-22T23:24:05.872 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-04-22T23:24:06.190 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-04-22T23:24:06.190 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:24:06.190 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-04-22T23:24:06.190 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-rook-2:17.2.7-884.g70d0cb4b.el7.noarch 2024-04-22T23:24:06.190 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-22T23:24:06.190 INFO:teuthology.orchestra.run.smithi157.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-22T23:24:06.190 INFO:teuthology.orchestra.run.smithi157.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-22T23:24:06.191 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-22T23:24:06.191 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-22T23:24:06.191 INFO:teuthology.orchestra.run.smithi157.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-22T23:24:06.191 INFO:teuthology.orchestra.run.smithi157.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-22T23:24:06.191 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-22T23:24:06.191 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-22T23:24:06.191 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-22T23:24:06.191 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-22T23:24:06.191 INFO:teuthology.orchestra.run.smithi157.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-22T23:24:06.191 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:24:06.191 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-04-22T23:24:06.409 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-mgr-cephadm 2024-04-22T23:24:06.764 INFO:teuthology.orchestra.run.smithi107.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 788 kB/s | 16 MB 00:21 2024-04-22T23:24:06.766 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:24:06.766 INFO:teuthology.orchestra.run.smithi107.stdout:Total 1.4 MB/s | 38 MB 00:27 2024-04-22T23:24:06.925 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:24 ago on Mon 22 Apr 2024 11:22:42 PM UTC. 2024-04-22T23:24:07.045 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-04-22T23:24:07.046 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-22T23:24:07.046 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2024-04-22T23:24:07.046 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-22T23:24:07.046 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-04-22T23:24:07.046 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-cephadm noarch 2:17.2.7-884.g70d0cb4b.el7 ceph-noarch 123 k 2024-04-22T23:24:07.046 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2024-04-22T23:24:07.046 INFO:teuthology.orchestra.run.smithi157.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-22T23:24:07.047 INFO:teuthology.orchestra.run.smithi157.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-22T23:24:07.047 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-22T23:24:07.047 INFO:teuthology.orchestra.run.smithi157.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-22T23:24:07.047 INFO:teuthology.orchestra.run.smithi157.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-22T23:24:07.047 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:24:07.047 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-04-22T23:24:07.047 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-22T23:24:07.047 INFO:teuthology.orchestra.run.smithi157.stdout:Install 6 Packages 2024-04-22T23:24:07.047 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:24:07.048 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 7.0 M 2024-04-22T23:24:07.048 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 33 M 2024-04-22T23:24:07.048 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-04-22T23:24:07.118 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-04-22T23:24:07.146 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-04-22T23:24:07.146 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-04-22T23:24:07.553 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-04-22T23:24:07.553 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-04-22T23:24:07.645 INFO:teuthology.orchestra.run.smithi157.stdout:(1/6): ceph-mgr-cephadm-17.2.7-884.g70d0cb4b.el 375 kB/s | 123 kB 00:00 2024-04-22T23:24:07.821 INFO:teuthology.orchestra.run.smithi157.stdout:(2/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 492 kB/s | 248 kB 00:00 2024-04-22T23:24:07.921 INFO:teuthology.orchestra.run.smithi157.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 1.9 MB/s | 548 kB 00:00 2024-04-22T23:24:07.980 INFO:teuthology.orchestra.run.smithi157.stdout:(4/6): python3-typing-extensions-4.4.0-2.el9.no 873 kB/s | 51 kB 00:00 2024-04-22T23:24:08.004 INFO:teuthology.orchestra.run.smithi157.stdout:(5/6): python3-natsort-7.1.1-5.el9.noarch.rpm 314 kB/s | 58 kB 00:00 2024-04-22T23:24:08.308 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-04-22T23:24:08.360 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-04-22T23:24:08.404 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-04-22T23:24:08.503 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-04-22T23:24:08.547 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-04-22T23:24:08.724 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-22T23:24:08.762 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-22T23:24:08.855 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-04-22T23:24:09.515 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-04-22T23:24:09.642 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-04-22T23:24:10.870 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-04-22T23:24:10.996 INFO:teuthology.orchestra.run.smithi157.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 1.6 MB/s | 6.0 MB 00:03 2024-04-22T23:24:10.996 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:24:10.997 INFO:teuthology.orchestra.run.smithi157.stdout:Total 1.8 MB/s | 7.0 MB 00:03 2024-04-22T23:24:11.070 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-04-22T23:24:11.082 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-04-22T23:24:11.082 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-04-22T23:24:11.177 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-04-22T23:24:11.178 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-04-22T23:24:11.483 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-04-22T23:24:11.654 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-04-22T23:24:11.698 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-04-22T23:24:11.913 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-04-22T23:24:11.937 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-884.g70d0cb 12/12 2024-04-22T23:24:12.227 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-04-22T23:24:12.318 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-04-22T23:24:12.373 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-04-22T23:24:12.402 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-884.g70d0cb4b.el7.noarch 6/6 2024-04-22T23:24:13.119 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-884.g70d0cb4b.el7.noarch 6/6 2024-04-22T23:24:13.119 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-884.g70d0cb4b.el7.noarch 1/6 2024-04-22T23:24:13.119 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-04-22T23:24:13.119 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-04-22T23:24:13.119 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-04-22T23:24:13.119 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-04-22T23:24:13.373 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-04-22T23:24:13.373 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:24:13.373 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-04-22T23:24:13.373 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-cephadm-2:17.2.7-884.g70d0cb4b.el7.noarch 2024-04-22T23:24:13.373 INFO:teuthology.orchestra.run.smithi157.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-22T23:24:13.373 INFO:teuthology.orchestra.run.smithi157.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-22T23:24:13.373 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-22T23:24:13.373 INFO:teuthology.orchestra.run.smithi157.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-22T23:24:13.374 INFO:teuthology.orchestra.run.smithi157.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-22T23:24:13.374 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:24:13.374 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-04-22T23:24:13.618 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-fuse 2024-04-22T23:24:14.136 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:32 ago on Mon 22 Apr 2024 11:22:42 PM UTC. 2024-04-22T23:24:14.257 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-04-22T23:24:14.258 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-22T23:24:14.258 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2024-04-22T23:24:14.258 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-22T23:24:14.258 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-04-22T23:24:14.258 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-fuse x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 812 k 2024-04-22T23:24:14.258 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2024-04-22T23:24:14.258 INFO:teuthology.orchestra.run.smithi157.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-22T23:24:14.258 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:24:14.258 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-04-22T23:24:14.258 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-22T23:24:14.258 INFO:teuthology.orchestra.run.smithi157.stdout:Install 2 Packages 2024-04-22T23:24:14.258 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:24:14.259 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 892 k 2024-04-22T23:24:14.259 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 2.6 M 2024-04-22T23:24:14.259 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-04-22T23:24:14.452 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-884.g70d0cb 12/12 2024-04-22T23:24:14.452 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-884.g70d0cb 1/12 2024-04-22T23:24:14.453 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-04-22T23:24:14.453 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-04-22T23:24:14.453 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-04-22T23:24:14.453 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-04-22T23:24:14.453 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-04-22T23:24:14.453 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-04-22T23:24:14.453 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-04-22T23:24:14.453 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-04-22T23:24:14.454 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-04-22T23:24:14.454 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-04-22T23:24:14.632 INFO:teuthology.orchestra.run.smithi157.stdout:(1/2): ceph-fuse-17.2.7-884.g70d0cb4b.el7.x86_6 2.6 MB/s | 812 kB 00:00 2024-04-22T23:24:14.677 INFO:teuthology.orchestra.run.smithi157.stdout:(2/2): fuse-2.9.9-15.el9.x86_64.rpm 234 kB/s | 80 kB 00:00 2024-04-22T23:24:14.678 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:24:14.678 INFO:teuthology.orchestra.run.smithi157.stdout:Total 2.1 MB/s | 892 kB 00:00 2024-04-22T23:24:14.690 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-04-22T23:24:14.712 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-04-22T23:24:14.712 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-04-22T23:24:14.789 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-04-22T23:24:14.789 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-04-22T23:24:14.828 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-04-22T23:24:14.828 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:24:14.828 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-04-22T23:24:14.828 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-diskprediction-local-2:17.2.7-884.g70d0cb4b.el7.noarch 2024-04-22T23:24:14.828 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-22T23:24:14.828 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-22T23:24:14.828 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-22T23:24:14.828 INFO:teuthology.orchestra.run.smithi107.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-22T23:24:14.829 INFO:teuthology.orchestra.run.smithi107.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-22T23:24:14.829 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-22T23:24:14.829 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-22T23:24:14.829 INFO:teuthology.orchestra.run.smithi107.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-22T23:24:14.829 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-22T23:24:14.829 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-22T23:24:14.829 INFO:teuthology.orchestra.run.smithi107.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-22T23:24:14.829 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:24:14.829 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-04-22T23:24:14.974 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-04-22T23:24:15.110 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-04-22T23:24:15.187 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-fuse-2:17.2.7-884.g70d0cb4b.el7.x86_64 2/2 2024-04-22T23:24:15.224 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-mgr-rook 2024-04-22T23:24:15.737 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:37 ago on Mon 22 Apr 2024 11:22:38 PM UTC. 2024-04-22T23:24:15.788 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-fuse-2:17.2.7-884.g70d0cb4b.el7.x86_64 2/2 2024-04-22T23:24:15.788 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-fuse-2:17.2.7-884.g70d0cb4b.el7.x86_64 1/2 2024-04-22T23:24:15.859 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-04-22T23:24:15.860 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-22T23:24:15.860 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-04-22T23:24:15.860 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-22T23:24:15.860 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-04-22T23:24:15.860 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-rook noarch 2:17.2.7-884.g70d0cb4b.el7 ceph-noarch 47 k 2024-04-22T23:24:15.861 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-04-22T23:24:15.861 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-22T23:24:15.861 INFO:teuthology.orchestra.run.smithi107.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-22T23:24:15.861 INFO:teuthology.orchestra.run.smithi107.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-22T23:24:15.861 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-22T23:24:15.861 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-22T23:24:15.861 INFO:teuthology.orchestra.run.smithi107.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-22T23:24:15.861 INFO:teuthology.orchestra.run.smithi107.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-22T23:24:15.861 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-22T23:24:15.861 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-22T23:24:15.861 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-22T23:24:15.861 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-22T23:24:15.861 INFO:teuthology.orchestra.run.smithi107.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-22T23:24:15.862 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:24:15.862 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-04-22T23:24:15.862 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-22T23:24:15.862 INFO:teuthology.orchestra.run.smithi107.stdout:Install 13 Packages 2024-04-22T23:24:15.862 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:24:15.862 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 2.2 M 2024-04-22T23:24:15.862 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 26 M 2024-04-22T23:24:15.863 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-04-22T23:24:16.034 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-04-22T23:24:16.034 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:24:16.034 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-04-22T23:24:16.034 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-fuse-2:17.2.7-884.g70d0cb4b.el7.x86_64 fuse-2.9.9-15.el9.x86_64 2024-04-22T23:24:16.035 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:24:16.035 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-04-22T23:24:16.296 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install librados-devel 2024-04-22T23:24:16.337 INFO:teuthology.orchestra.run.smithi107.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 148 kB/s | 26 kB 00:00 2024-04-22T23:24:16.362 INFO:teuthology.orchestra.run.smithi107.stdout:(2/13): ceph-mgr-rook-17.2.7-884.g70d0cb4b.el7. 240 kB/s | 47 kB 00:00 2024-04-22T23:24:16.387 INFO:teuthology.orchestra.run.smithi107.stdout:(3/13): python3-jsonpointer-2.0-4.el9.noarch.rp 86 kB/s | 19 kB 00:00 2024-04-22T23:24:16.479 INFO:teuthology.orchestra.run.smithi107.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.5 MB/s | 222 kB 00:00 2024-04-22T23:24:16.504 INFO:teuthology.orchestra.run.smithi107.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-04-22T23:24:16.538 INFO:teuthology.orchestra.run.smithi107.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 914 kB/s | 54 kB 00:00 2024-04-22T23:24:16.596 INFO:teuthology.orchestra.run.smithi107.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.3 MB/s | 279 kB 00:00 2024-04-22T23:24:16.672 INFO:teuthology.orchestra.run.smithi107.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 193 kB/s | 32 kB 00:00 2024-04-22T23:24:16.714 INFO:teuthology.orchestra.run.smithi107.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 85 kB/s | 15 kB 00:00 2024-04-22T23:24:16.821 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:34 ago on Mon 22 Apr 2024 11:22:42 PM UTC. 2024-04-22T23:24:16.839 INFO:teuthology.orchestra.run.smithi107.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 473 kB/s | 59 kB 00:00 2024-04-22T23:24:16.898 INFO:teuthology.orchestra.run.smithi107.stdout:(11/13): python3-google-auth-2.28.2-1.el9.noarc 674 kB/s | 203 kB 00:00 2024-04-22T23:24:16.945 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-04-22T23:24:16.945 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-22T23:24:16.946 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repo Size 2024-04-22T23:24:16.946 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-22T23:24:16.946 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-04-22T23:24:16.946 INFO:teuthology.orchestra.run.smithi157.stdout: librados-devel x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 126 k 2024-04-22T23:24:16.946 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:24:16.946 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-04-22T23:24:16.946 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-22T23:24:16.946 INFO:teuthology.orchestra.run.smithi157.stdout:Install 1 Package 2024-04-22T23:24:16.946 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:24:16.946 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 126 k 2024-04-22T23:24:16.946 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 448 k 2024-04-22T23:24:16.946 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-04-22T23:24:16.956 INFO:teuthology.orchestra.run.smithi107.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 768 kB/s | 90 kB 00:00 2024-04-22T23:24:17.014 INFO:teuthology.orchestra.run.smithi107.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 3.0 MB/s | 1.0 MB 00:00 2024-04-22T23:24:17.015 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:24:17.015 INFO:teuthology.orchestra.run.smithi107.stdout:Total 1.9 MB/s | 2.2 MB 00:01 2024-04-22T23:24:17.078 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-04-22T23:24:17.093 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-04-22T23:24:17.093 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-04-22T23:24:17.172 INFO:teuthology.orchestra.run.smithi157.stdout:librados-devel-17.2.7-884.g70d0cb4b.el7.x86_64. 560 kB/s | 126 kB 00:00 2024-04-22T23:24:17.173 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:24:17.173 INFO:teuthology.orchestra.run.smithi157.stdout:Total 556 kB/s | 126 kB 00:00 2024-04-22T23:24:17.173 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-04-22T23:24:17.177 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-04-22T23:24:17.179 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-04-22T23:24:17.179 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-04-22T23:24:17.179 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-04-22T23:24:17.241 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-04-22T23:24:17.241 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-04-22T23:24:17.391 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-04-22T23:24:17.526 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librados-devel-2:17.2.7-884.g70d0cb4b.el7.x86_64 1/1 2024-04-22T23:24:17.764 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-04-22T23:24:17.922 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-04-22T23:24:17.937 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librados-devel-2:17.2.7-884.g70d0cb4b.el7.x86_64 1/1 2024-04-22T23:24:17.982 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-04-22T23:24:18.038 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-04-22T23:24:18.076 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-04-22T23:24:18.117 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-04-22T23:24:18.188 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados-devel-2:17.2.7-884.g70d0cb4b.el7.x86_64 1/1 2024-04-22T23:24:18.188 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:24:18.189 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-04-22T23:24:18.189 INFO:teuthology.orchestra.run.smithi157.stdout: librados-devel-2:17.2.7-884.g70d0cb4b.el7.x86_64 2024-04-22T23:24:18.189 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:24:18.189 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-04-22T23:24:18.219 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-04-22T23:24:18.321 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 7/13 2024-04-22T23:24:18.368 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-04-22T23:24:18.433 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install libcephfs2 2024-04-22T23:24:18.954 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:36 ago on Mon 22 Apr 2024 11:22:42 PM UTC. 2024-04-22T23:24:19.035 INFO:teuthology.orchestra.run.smithi157.stdout:Package libcephfs2-2:17.2.7-884.g70d0cb4b.el7.x86_64 is already installed. 2024-04-22T23:24:19.077 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-04-22T23:24:19.078 INFO:teuthology.orchestra.run.smithi157.stdout:Nothing to do. 2024-04-22T23:24:19.078 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-04-22T23:24:19.087 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-04-22T23:24:19.150 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-04-22T23:24:19.154 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install libcephfs-devel 2024-04-22T23:24:19.183 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-04-22T23:24:19.229 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-04-22T23:24:19.253 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-rook-2:17.2.7-884.g70d0cb4b.el7.noarch 13/13 2024-04-22T23:24:19.674 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:37 ago on Mon 22 Apr 2024 11:22:42 PM UTC. 2024-04-22T23:24:19.794 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-04-22T23:24:19.795 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-22T23:24:19.795 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repo Size 2024-04-22T23:24:19.795 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-22T23:24:19.795 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-04-22T23:24:19.795 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs-devel x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 25 k 2024-04-22T23:24:19.795 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:24:19.795 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-04-22T23:24:19.795 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-22T23:24:19.795 INFO:teuthology.orchestra.run.smithi157.stdout:Install 1 Package 2024-04-22T23:24:19.795 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:24:19.795 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 25 k 2024-04-22T23:24:19.795 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 110 k 2024-04-22T23:24:19.796 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-04-22T23:24:20.043 INFO:teuthology.orchestra.run.smithi157.stdout:libcephfs-devel-17.2.7-884.g70d0cb4b.el7.x86_64 101 kB/s | 25 kB 00:00 2024-04-22T23:24:20.044 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:24:20.044 INFO:teuthology.orchestra.run.smithi157.stdout:Total 100 kB/s | 25 kB 00:00 2024-04-22T23:24:20.044 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-04-22T23:24:20.047 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-884.g70d0cb4b.el7.noarch 13/13 2024-04-22T23:24:20.047 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-rook-2:17.2.7-884.g70d0cb4b.el7.noarch 1/13 2024-04-22T23:24:20.047 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-04-22T23:24:20.047 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-04-22T23:24:20.047 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-04-22T23:24:20.047 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-04-22T23:24:20.048 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-04-22T23:24:20.048 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-04-22T23:24:20.048 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-04-22T23:24:20.048 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-04-22T23:24:20.048 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 10/13 2024-04-22T23:24:20.048 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-04-22T23:24:20.048 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-04-22T23:24:20.049 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-04-22T23:24:20.049 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-04-22T23:24:20.049 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-04-22T23:24:20.049 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-04-22T23:24:20.130 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-04-22T23:24:20.230 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephfs-devel-2:17.2.7-884.g70d0cb4b.el7.x86_64 1/1 2024-04-22T23:24:20.353 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-04-22T23:24:20.353 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:24:20.353 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-04-22T23:24:20.354 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-rook-2:17.2.7-884.g70d0cb4b.el7.noarch 2024-04-22T23:24:20.354 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-22T23:24:20.354 INFO:teuthology.orchestra.run.smithi107.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-22T23:24:20.354 INFO:teuthology.orchestra.run.smithi107.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-22T23:24:20.354 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-22T23:24:20.354 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-22T23:24:20.354 INFO:teuthology.orchestra.run.smithi107.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-22T23:24:20.354 INFO:teuthology.orchestra.run.smithi107.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-22T23:24:20.354 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-22T23:24:20.354 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-22T23:24:20.354 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-22T23:24:20.354 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-22T23:24:20.354 INFO:teuthology.orchestra.run.smithi107.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-22T23:24:20.355 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:24:20.355 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-04-22T23:24:20.583 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-884.g70d0cb4b.el7.x86_64 1/1 2024-04-22T23:24:20.586 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-mgr-cephadm 2024-04-22T23:24:20.787 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephfs-devel-2:17.2.7-884.g70d0cb4b.el7.x86_64 1/1 2024-04-22T23:24:20.787 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:24:20.787 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-04-22T23:24:20.787 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs-devel-2:17.2.7-884.g70d0cb4b.el7.x86_64 2024-04-22T23:24:20.787 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:24:20.787 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-04-22T23:24:20.957 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install librados2 2024-04-22T23:24:21.094 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:43 ago on Mon 22 Apr 2024 11:22:38 PM UTC. 2024-04-22T23:24:21.211 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-04-22T23:24:21.212 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-22T23:24:21.212 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-04-22T23:24:21.212 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-22T23:24:21.213 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-04-22T23:24:21.213 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-cephadm noarch 2:17.2.7-884.g70d0cb4b.el7 ceph-noarch 123 k 2024-04-22T23:24:21.213 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-04-22T23:24:21.213 INFO:teuthology.orchestra.run.smithi107.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-22T23:24:21.213 INFO:teuthology.orchestra.run.smithi107.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-22T23:24:21.213 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-22T23:24:21.213 INFO:teuthology.orchestra.run.smithi107.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-22T23:24:21.213 INFO:teuthology.orchestra.run.smithi107.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-22T23:24:21.213 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:24:21.213 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-04-22T23:24:21.213 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-22T23:24:21.213 INFO:teuthology.orchestra.run.smithi107.stdout:Install 6 Packages 2024-04-22T23:24:21.213 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:24:21.214 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 7.0 M 2024-04-22T23:24:21.214 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 33 M 2024-04-22T23:24:21.214 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-04-22T23:24:21.475 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:39 ago on Mon 22 Apr 2024 11:22:42 PM UTC. 2024-04-22T23:24:21.554 INFO:teuthology.orchestra.run.smithi157.stdout:Package librados2-2:17.2.7-884.g70d0cb4b.el7.x86_64 is already installed. 2024-04-22T23:24:21.594 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-04-22T23:24:21.595 INFO:teuthology.orchestra.run.smithi157.stdout:Nothing to do. 2024-04-22T23:24:21.595 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-04-22T23:24:21.659 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install librbd1 2024-04-22T23:24:21.850 INFO:teuthology.orchestra.run.smithi107.stdout:(1/6): ceph-mgr-cephadm-17.2.7-884.g70d0cb4b.el 602 kB/s | 123 kB 00:00 2024-04-22T23:24:21.967 INFO:teuthology.orchestra.run.smithi107.stdout:(2/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 770 kB/s | 248 kB 00:00 2024-04-22T23:24:22.170 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:40 ago on Mon 22 Apr 2024 11:22:42 PM UTC. 2024-04-22T23:24:22.248 INFO:teuthology.orchestra.run.smithi157.stdout:Package librbd1-2:17.2.7-884.g70d0cb4b.el7.x86_64 is already installed. 2024-04-22T23:24:22.267 INFO:teuthology.orchestra.run.smithi107.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 192 kB/s | 58 kB 00:00 2024-04-22T23:24:22.289 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-04-22T23:24:22.290 INFO:teuthology.orchestra.run.smithi157.stdout:Nothing to do. 2024-04-22T23:24:22.290 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-04-22T23:24:22.334 INFO:teuthology.orchestra.run.smithi107.stdout:(4/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 1.1 MB/s | 548 kB 00:00 2024-04-22T23:24:22.358 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install python3-rados 2024-04-22T23:24:22.367 INFO:teuthology.orchestra.run.smithi107.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 509 kB/s | 51 kB 00:00 2024-04-22T23:24:22.878 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:40 ago on Mon 22 Apr 2024 11:22:42 PM UTC. 2024-04-22T23:24:22.958 INFO:teuthology.orchestra.run.smithi157.stdout:Package python3-rados-2:17.2.7-884.g70d0cb4b.el7.x86_64 is already installed. 2024-04-22T23:24:22.999 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-04-22T23:24:23.000 INFO:teuthology.orchestra.run.smithi157.stdout:Nothing to do. 2024-04-22T23:24:23.000 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-04-22T23:24:23.067 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install python3-rgw 2024-04-22T23:24:23.143 INFO:teuthology.orchestra.run.smithi107.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 4.0 MB/s | 6.0 MB 00:01 2024-04-22T23:24:23.145 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:24:23.145 INFO:teuthology.orchestra.run.smithi107.stdout:Total 3.6 MB/s | 7.0 MB 00:01 2024-04-22T23:24:23.217 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-04-22T23:24:23.228 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-04-22T23:24:23.228 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-04-22T23:24:23.322 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-04-22T23:24:23.324 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-04-22T23:24:23.572 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:41 ago on Mon 22 Apr 2024 11:22:42 PM UTC. 2024-04-22T23:24:23.627 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-04-22T23:24:23.652 INFO:teuthology.orchestra.run.smithi157.stdout:Package python3-rgw-2:17.2.7-884.g70d0cb4b.el7.x86_64 is already installed. 2024-04-22T23:24:23.692 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-04-22T23:24:23.693 INFO:teuthology.orchestra.run.smithi157.stdout:Nothing to do. 2024-04-22T23:24:23.693 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-04-22T23:24:23.759 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install python3-cephfs 2024-04-22T23:24:23.810 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-04-22T23:24:23.869 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-04-22T23:24:24.272 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:42 ago on Mon 22 Apr 2024 11:22:42 PM UTC. 2024-04-22T23:24:24.352 INFO:teuthology.orchestra.run.smithi157.stdout:Package python3-cephfs-2:17.2.7-884.g70d0cb4b.el7.x86_64 is already installed. 2024-04-22T23:24:24.386 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-04-22T23:24:24.392 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-04-22T23:24:24.393 INFO:teuthology.orchestra.run.smithi157.stdout:Nothing to do. 2024-04-22T23:24:24.393 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-04-22T23:24:24.459 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install python3-rbd 2024-04-22T23:24:24.463 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-04-22T23:24:24.528 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-04-22T23:24:24.556 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-884.g70d0cb4b.el7.noarch 6/6 2024-04-22T23:24:24.972 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:42 ago on Mon 22 Apr 2024 11:22:42 PM UTC. 2024-04-22T23:24:25.051 INFO:teuthology.orchestra.run.smithi157.stdout:Package python3-rbd-2:17.2.7-884.g70d0cb4b.el7.x86_64 is already installed. 2024-04-22T23:24:25.092 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-04-22T23:24:25.093 INFO:teuthology.orchestra.run.smithi157.stdout:Nothing to do. 2024-04-22T23:24:25.093 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-04-22T23:24:25.161 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install rbd-fuse 2024-04-22T23:24:25.293 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-884.g70d0cb4b.el7.noarch 6/6 2024-04-22T23:24:25.293 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-884.g70d0cb4b.el7.noarch 1/6 2024-04-22T23:24:25.293 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-04-22T23:24:25.293 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-04-22T23:24:25.293 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-04-22T23:24:25.293 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-04-22T23:24:25.543 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-04-22T23:24:25.561 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:24:25.562 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-04-22T23:24:25.562 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-cephadm-2:17.2.7-884.g70d0cb4b.el7.noarch 2024-04-22T23:24:25.562 INFO:teuthology.orchestra.run.smithi107.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-22T23:24:25.562 INFO:teuthology.orchestra.run.smithi107.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-22T23:24:25.562 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-22T23:24:25.562 INFO:teuthology.orchestra.run.smithi107.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-22T23:24:25.562 INFO:teuthology.orchestra.run.smithi107.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-22T23:24:25.562 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:24:25.563 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-04-22T23:24:25.667 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:43 ago on Mon 22 Apr 2024 11:22:42 PM UTC. 2024-04-22T23:24:25.785 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-04-22T23:24:25.786 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-22T23:24:25.786 INFO:teuthology.orchestra.run.smithi157.stdout: Package Architecture Version Repository Size 2024-04-22T23:24:25.786 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-22T23:24:25.787 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-04-22T23:24:25.787 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-fuse x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 82 k 2024-04-22T23:24:25.787 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:24:25.787 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-04-22T23:24:25.787 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-22T23:24:25.787 INFO:teuthology.orchestra.run.smithi157.stdout:Install 1 Package 2024-04-22T23:24:25.787 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:24:25.787 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 82 k 2024-04-22T23:24:25.787 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 222 k 2024-04-22T23:24:25.787 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-04-22T23:24:25.798 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-fuse 2024-04-22T23:24:25.996 INFO:teuthology.orchestra.run.smithi157.stdout:rbd-fuse-17.2.7-884.g70d0cb4b.el7.x86_64.rpm 391 kB/s | 82 kB 00:00 2024-04-22T23:24:25.997 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:24:25.997 INFO:teuthology.orchestra.run.smithi157.stdout:Total 389 kB/s | 82 kB 00:00 2024-04-22T23:24:25.997 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-04-22T23:24:26.003 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-04-22T23:24:26.003 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-04-22T23:24:26.065 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-04-22T23:24:26.065 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-04-22T23:24:26.210 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-04-22T23:24:26.307 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:48 ago on Mon 22 Apr 2024 11:22:38 PM UTC. 2024-04-22T23:24:26.324 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-fuse-2:17.2.7-884.g70d0cb4b.el7.x86_64 1/1 2024-04-22T23:24:26.427 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-04-22T23:24:26.427 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-22T23:24:26.428 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-04-22T23:24:26.428 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-22T23:24:26.428 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-04-22T23:24:26.428 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-fuse x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 812 k 2024-04-22T23:24:26.428 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-04-22T23:24:26.428 INFO:teuthology.orchestra.run.smithi107.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-22T23:24:26.428 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:24:26.428 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-04-22T23:24:26.428 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-22T23:24:26.428 INFO:teuthology.orchestra.run.smithi107.stdout:Install 2 Packages 2024-04-22T23:24:26.428 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:24:26.428 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 892 k 2024-04-22T23:24:26.428 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 2.6 M 2024-04-22T23:24:26.428 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-04-22T23:24:26.662 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: rbd-fuse-2:17.2.7-884.g70d0cb4b.el7.x86_64 1/1 2024-04-22T23:24:26.902 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-fuse-2:17.2.7-884.g70d0cb4b.el7.x86_64 1/1 2024-04-22T23:24:26.902 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:24:26.902 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-04-22T23:24:26.903 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-fuse-2:17.2.7-884.g70d0cb4b.el7.x86_64 2024-04-22T23:24:26.903 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:24:26.903 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-04-22T23:24:26.994 INFO:teuthology.orchestra.run.smithi107.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 275 kB/s | 80 kB 00:00 2024-04-22T23:24:27.156 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install rbd-mirror 2024-04-22T23:24:27.369 INFO:teuthology.orchestra.run.smithi107.stdout:(2/2): ceph-fuse-17.2.7-884.g70d0cb4b.el7.x86_6 1.2 MB/s | 812 kB 00:00 2024-04-22T23:24:27.370 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:24:27.370 INFO:teuthology.orchestra.run.smithi107.stdout:Total 947 kB/s | 892 kB 00:00 2024-04-22T23:24:27.382 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-04-22T23:24:27.404 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-04-22T23:24:27.404 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-04-22T23:24:27.480 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-04-22T23:24:27.481 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-04-22T23:24:27.668 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-04-22T23:24:27.673 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:45 ago on Mon 22 Apr 2024 11:22:42 PM UTC. 2024-04-22T23:24:27.795 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-04-22T23:24:27.796 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-22T23:24:27.796 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repo Size 2024-04-22T23:24:27.796 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-22T23:24:27.797 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-04-22T23:24:27.797 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-mirror x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 2.9 M 2024-04-22T23:24:27.797 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:24:27.797 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-04-22T23:24:27.797 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-22T23:24:27.797 INFO:teuthology.orchestra.run.smithi157.stdout:Install 1 Package 2024-04-22T23:24:27.797 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:24:27.797 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 2.9 M 2024-04-22T23:24:27.797 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 12 M 2024-04-22T23:24:27.797 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-04-22T23:24:27.815 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-04-22T23:24:27.874 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-fuse-2:17.2.7-884.g70d0cb4b.el7.x86_64 2/2 2024-04-22T23:24:28.524 INFO:teuthology.orchestra.run.smithi157.stdout:rbd-mirror-17.2.7-884.g70d0cb4b.el7.x86_64.rpm 4.0 MB/s | 2.9 MB 00:00 2024-04-22T23:24:28.525 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:24:28.525 INFO:teuthology.orchestra.run.smithi157.stdout:Total 4.0 MB/s | 2.9 MB 00:00 2024-04-22T23:24:28.526 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-04-22T23:24:28.534 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-04-22T23:24:28.535 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-04-22T23:24:28.563 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-fuse-2:17.2.7-884.g70d0cb4b.el7.x86_64 2/2 2024-04-22T23:24:28.563 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-fuse-2:17.2.7-884.g70d0cb4b.el7.x86_64 1/2 2024-04-22T23:24:28.613 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-04-22T23:24:28.614 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-04-22T23:24:28.811 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-04-22T23:24:28.811 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:24:28.811 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-04-22T23:24:28.811 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-fuse-2:17.2.7-884.g70d0cb4b.el7.x86_64 fuse-2.9.9-15.el9.x86_64 2024-04-22T23:24:28.811 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:24:28.811 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-04-22T23:24:29.032 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-04-22T23:24:29.082 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install librados-devel 2024-04-22T23:24:29.092 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-mirror-2:17.2.7-884.g70d0cb4b.el7.x86_64 1/1 2024-04-22T23:24:29.116 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: rbd-mirror-2:17.2.7-884.g70d0cb4b.el7.x86_64 1/1 2024-04-22T23:24:29.117 INFO:teuthology.orchestra.run.smithi157.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T23:24:29.117 INFO:teuthology.orchestra.run.smithi157.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-22T23:24:29.117 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-22T23:24:29.117 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-22T23:24:29.117 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:24:29.609 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:51 ago on Mon 22 Apr 2024 11:22:38 PM UTC. 2024-04-22T23:24:29.732 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-04-22T23:24:29.733 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-22T23:24:29.733 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repo Size 2024-04-22T23:24:29.733 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-22T23:24:29.733 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-04-22T23:24:29.733 INFO:teuthology.orchestra.run.smithi107.stdout: librados-devel x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 126 k 2024-04-22T23:24:29.733 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:24:29.733 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-04-22T23:24:29.733 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-22T23:24:29.733 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2024-04-22T23:24:29.733 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:24:29.733 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 126 k 2024-04-22T23:24:29.733 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 448 k 2024-04-22T23:24:29.734 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-04-22T23:24:30.041 INFO:teuthology.orchestra.run.smithi107.stdout:librados-devel-17.2.7-884.g70d0cb4b.el7.x86_64. 411 kB/s | 126 kB 00:00 2024-04-22T23:24:30.041 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:24:30.042 INFO:teuthology.orchestra.run.smithi107.stdout:Total 409 kB/s | 126 kB 00:00 2024-04-22T23:24:30.042 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-04-22T23:24:30.047 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-04-22T23:24:30.047 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-04-22T23:24:30.109 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-04-22T23:24:30.109 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-04-22T23:24:30.152 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-mirror-2:17.2.7-884.g70d0cb4b.el7.x86_64 1/1 2024-04-22T23:24:30.152 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:24:30.153 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-04-22T23:24:30.153 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-mirror-2:17.2.7-884.g70d0cb4b.el7.x86_64 2024-04-22T23:24:30.153 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:24:30.153 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-04-22T23:24:30.262 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-04-22T23:24:30.396 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install rbd-nbd 2024-04-22T23:24:30.403 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librados-devel-2:17.2.7-884.g70d0cb4b.el7.x86_64 1/1 2024-04-22T23:24:30.781 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librados-devel-2:17.2.7-884.g70d0cb4b.el7.x86_64 1/1 2024-04-22T23:24:30.923 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:48 ago on Mon 22 Apr 2024 11:22:42 PM UTC. 2024-04-22T23:24:31.023 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados-devel-2:17.2.7-884.g70d0cb4b.el7.x86_64 1/1 2024-04-22T23:24:31.023 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:24:31.023 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-04-22T23:24:31.023 INFO:teuthology.orchestra.run.smithi107.stdout: librados-devel-2:17.2.7-884.g70d0cb4b.el7.x86_64 2024-04-22T23:24:31.023 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:24:31.023 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-04-22T23:24:31.048 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2024-04-22T23:24:31.049 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-22T23:24:31.049 INFO:teuthology.orchestra.run.smithi157.stdout: Package Architecture Version Repository Size 2024-04-22T23:24:31.049 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-22T23:24:31.049 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2024-04-22T23:24:31.049 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-nbd x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 165 k 2024-04-22T23:24:31.049 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:24:31.049 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2024-04-22T23:24:31.049 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2024-04-22T23:24:31.049 INFO:teuthology.orchestra.run.smithi157.stdout:Install 1 Package 2024-04-22T23:24:31.049 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:24:31.050 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 165 k 2024-04-22T23:24:31.050 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 477 k 2024-04-22T23:24:31.050 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2024-04-22T23:24:31.275 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install libcephfs2 2024-04-22T23:24:31.278 INFO:teuthology.orchestra.run.smithi157.stdout:rbd-nbd-17.2.7-884.g70d0cb4b.el7.x86_64.rpm 722 kB/s | 165 kB 00:00 2024-04-22T23:24:31.279 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:24:31.279 INFO:teuthology.orchestra.run.smithi157.stdout:Total 718 kB/s | 165 kB 00:00 2024-04-22T23:24:31.279 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2024-04-22T23:24:31.287 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2024-04-22T23:24:31.287 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2024-04-22T23:24:31.349 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2024-04-22T23:24:31.349 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2024-04-22T23:24:31.504 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2024-04-22T23:24:31.607 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-nbd-2:17.2.7-884.g70d0cb4b.el7.x86_64 1/1 2024-04-22T23:24:31.787 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:53 ago on Mon 22 Apr 2024 11:22:38 PM UTC. 2024-04-22T23:24:31.865 INFO:teuthology.orchestra.run.smithi107.stdout:Package libcephfs2-2:17.2.7-884.g70d0cb4b.el7.x86_64 is already installed. 2024-04-22T23:24:31.906 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-04-22T23:24:31.907 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-04-22T23:24:31.907 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-04-22T23:24:32.000 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: rbd-nbd-2:17.2.7-884.g70d0cb4b.el7.x86_64 1/1 2024-04-22T23:24:32.004 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install libcephfs-devel 2024-04-22T23:24:32.249 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-nbd-2:17.2.7-884.g70d0cb4b.el7.x86_64 1/1 2024-04-22T23:24:32.250 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:24:32.250 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2024-04-22T23:24:32.250 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-nbd-2:17.2.7-884.g70d0cb4b.el7.x86_64 2024-04-22T23:24:32.250 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:24:32.250 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2024-04-22T23:24:32.500 DEBUG:teuthology.parallel:result is None 2024-04-22T23:24:32.527 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:54 ago on Mon 22 Apr 2024 11:22:38 PM UTC. 2024-04-22T23:24:32.646 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-04-22T23:24:32.646 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-22T23:24:32.647 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repo Size 2024-04-22T23:24:32.647 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-22T23:24:32.647 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-04-22T23:24:32.647 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs-devel x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 25 k 2024-04-22T23:24:32.647 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:24:32.647 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-04-22T23:24:32.647 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-22T23:24:32.647 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2024-04-22T23:24:32.647 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:24:32.647 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 25 k 2024-04-22T23:24:32.647 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 110 k 2024-04-22T23:24:32.647 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-04-22T23:24:32.894 INFO:teuthology.orchestra.run.smithi107.stdout:libcephfs-devel-17.2.7-884.g70d0cb4b.el7.x86_64 101 kB/s | 25 kB 00:00 2024-04-22T23:24:32.895 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:24:32.895 INFO:teuthology.orchestra.run.smithi107.stdout:Total 100 kB/s | 25 kB 00:00 2024-04-22T23:24:32.895 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-04-22T23:24:32.899 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-04-22T23:24:32.899 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-04-22T23:24:32.900 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-04-22T23:24:32.900 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-04-22T23:24:32.983 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-04-22T23:24:33.081 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephfs-devel-2:17.2.7-884.g70d0cb4b.el7.x86_64 1/1 2024-04-22T23:24:33.391 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-884.g70d0cb4b.el7.x86_64 1/1 2024-04-22T23:24:33.606 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephfs-devel-2:17.2.7-884.g70d0cb4b.el7.x86_64 1/1 2024-04-22T23:24:33.606 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:24:33.606 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-04-22T23:24:33.606 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs-devel-2:17.2.7-884.g70d0cb4b.el7.x86_64 2024-04-22T23:24:33.606 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:24:33.606 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-04-22T23:24:33.799 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install librados2 2024-04-22T23:24:34.314 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:56 ago on Mon 22 Apr 2024 11:22:38 PM UTC. 2024-04-22T23:24:34.396 INFO:teuthology.orchestra.run.smithi107.stdout:Package librados2-2:17.2.7-884.g70d0cb4b.el7.x86_64 is already installed. 2024-04-22T23:24:34.437 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-04-22T23:24:34.438 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-04-22T23:24:34.438 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-04-22T23:24:34.520 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install librbd1 2024-04-22T23:24:35.026 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:57 ago on Mon 22 Apr 2024 11:22:38 PM UTC. 2024-04-22T23:24:35.104 INFO:teuthology.orchestra.run.smithi107.stdout:Package librbd1-2:17.2.7-884.g70d0cb4b.el7.x86_64 is already installed. 2024-04-22T23:24:35.143 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-04-22T23:24:35.144 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-04-22T23:24:35.145 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-04-22T23:24:35.213 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install python3-rados 2024-04-22T23:24:35.718 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:57 ago on Mon 22 Apr 2024 11:22:38 PM UTC. 2024-04-22T23:24:35.796 INFO:teuthology.orchestra.run.smithi107.stdout:Package python3-rados-2:17.2.7-884.g70d0cb4b.el7.x86_64 is already installed. 2024-04-22T23:24:35.837 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-04-22T23:24:35.838 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-04-22T23:24:35.838 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-04-22T23:24:35.920 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install python3-rgw 2024-04-22T23:24:36.426 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:58 ago on Mon 22 Apr 2024 11:22:38 PM UTC. 2024-04-22T23:24:36.505 INFO:teuthology.orchestra.run.smithi107.stdout:Package python3-rgw-2:17.2.7-884.g70d0cb4b.el7.x86_64 is already installed. 2024-04-22T23:24:36.547 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-04-22T23:24:36.548 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-04-22T23:24:36.548 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-04-22T23:24:36.630 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install python3-cephfs 2024-04-22T23:24:37.134 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:59 ago on Mon 22 Apr 2024 11:22:38 PM UTC. 2024-04-22T23:24:37.211 INFO:teuthology.orchestra.run.smithi107.stdout:Package python3-cephfs-2:17.2.7-884.g70d0cb4b.el7.x86_64 is already installed. 2024-04-22T23:24:37.251 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-04-22T23:24:37.252 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-04-22T23:24:37.252 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-04-22T23:24:37.322 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install python3-rbd 2024-04-22T23:24:37.831 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:59 ago on Mon 22 Apr 2024 11:22:38 PM UTC. 2024-04-22T23:24:37.909 INFO:teuthology.orchestra.run.smithi107.stdout:Package python3-rbd-2:17.2.7-884.g70d0cb4b.el7.x86_64 is already installed. 2024-04-22T23:24:37.950 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-04-22T23:24:37.951 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-04-22T23:24:37.951 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-04-22T23:24:38.027 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install rbd-fuse 2024-04-22T23:24:38.536 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:02:00 ago on Mon 22 Apr 2024 11:22:38 PM UTC. 2024-04-22T23:24:38.654 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-04-22T23:24:38.655 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-22T23:24:38.655 INFO:teuthology.orchestra.run.smithi107.stdout: Package Architecture Version Repository Size 2024-04-22T23:24:38.655 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-22T23:24:38.655 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-04-22T23:24:38.656 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-fuse x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 82 k 2024-04-22T23:24:38.656 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:24:38.656 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-04-22T23:24:38.656 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-22T23:24:38.656 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2024-04-22T23:24:38.656 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:24:38.656 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 82 k 2024-04-22T23:24:38.656 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 222 k 2024-04-22T23:24:38.656 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-04-22T23:24:39.205 INFO:teuthology.orchestra.run.smithi107.stdout:rbd-fuse-17.2.7-884.g70d0cb4b.el7.x86_64.rpm 149 kB/s | 82 kB 00:00 2024-04-22T23:24:39.206 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:24:39.206 INFO:teuthology.orchestra.run.smithi107.stdout:Total 148 kB/s | 82 kB 00:00 2024-04-22T23:24:39.207 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-04-22T23:24:39.213 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-04-22T23:24:39.213 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-04-22T23:24:39.276 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-04-22T23:24:39.276 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-04-22T23:24:39.414 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-04-22T23:24:39.519 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-fuse-2:17.2.7-884.g70d0cb4b.el7.x86_64 1/1 2024-04-22T23:24:39.879 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: rbd-fuse-2:17.2.7-884.g70d0cb4b.el7.x86_64 1/1 2024-04-22T23:24:40.130 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-fuse-2:17.2.7-884.g70d0cb4b.el7.x86_64 1/1 2024-04-22T23:24:40.130 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:24:40.130 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-04-22T23:24:40.130 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-fuse-2:17.2.7-884.g70d0cb4b.el7.x86_64 2024-04-22T23:24:40.130 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:24:40.130 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-04-22T23:24:40.370 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install rbd-mirror 2024-04-22T23:24:40.883 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:02:02 ago on Mon 22 Apr 2024 11:22:38 PM UTC. 2024-04-22T23:24:41.002 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-04-22T23:24:41.003 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-22T23:24:41.003 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repo Size 2024-04-22T23:24:41.003 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-22T23:24:41.003 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-04-22T23:24:41.003 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-mirror x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 2.9 M 2024-04-22T23:24:41.003 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:24:41.003 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-04-22T23:24:41.003 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-22T23:24:41.003 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2024-04-22T23:24:41.004 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:24:41.004 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 2.9 M 2024-04-22T23:24:41.004 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 12 M 2024-04-22T23:24:41.004 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-04-22T23:24:41.768 INFO:teuthology.orchestra.run.smithi107.stdout:rbd-mirror-17.2.7-884.g70d0cb4b.el7.x86_64.rpm 3.8 MB/s | 2.9 MB 00:00 2024-04-22T23:24:41.769 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:24:41.769 INFO:teuthology.orchestra.run.smithi107.stdout:Total 3.8 MB/s | 2.9 MB 00:00 2024-04-22T23:24:41.769 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-04-22T23:24:41.778 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-04-22T23:24:41.778 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-04-22T23:24:41.856 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-04-22T23:24:41.856 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-04-22T23:24:42.272 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-04-22T23:24:42.336 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-mirror-2:17.2.7-884.g70d0cb4b.el7.x86_64 1/1 2024-04-22T23:24:42.360 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: rbd-mirror-2:17.2.7-884.g70d0cb4b.el7.x86_64 1/1 2024-04-22T23:24:42.360 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T23:24:42.360 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-22T23:24:42.360 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-22T23:24:42.360 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-22T23:24:42.360 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:24:43.374 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-mirror-2:17.2.7-884.g70d0cb4b.el7.x86_64 1/1 2024-04-22T23:24:43.374 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:24:43.374 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-04-22T23:24:43.374 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-mirror-2:17.2.7-884.g70d0cb4b.el7.x86_64 2024-04-22T23:24:43.374 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:24:43.374 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-04-22T23:24:43.643 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install rbd-nbd 2024-04-22T23:24:44.162 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:02:06 ago on Mon 22 Apr 2024 11:22:38 PM UTC. 2024-04-22T23:24:44.285 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-04-22T23:24:44.286 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-22T23:24:44.286 INFO:teuthology.orchestra.run.smithi107.stdout: Package Architecture Version Repository Size 2024-04-22T23:24:44.286 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-22T23:24:44.286 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-04-22T23:24:44.286 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-nbd x86_64 2:17.2.7-884.g70d0cb4b.el7 ceph 165 k 2024-04-22T23:24:44.286 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:24:44.286 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-04-22T23:24:44.286 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-22T23:24:44.286 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2024-04-22T23:24:44.286 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:24:44.286 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 165 k 2024-04-22T23:24:44.286 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 477 k 2024-04-22T23:24:44.286 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-04-22T23:24:44.622 INFO:teuthology.orchestra.run.smithi107.stdout:rbd-nbd-17.2.7-884.g70d0cb4b.el7.x86_64.rpm 492 kB/s | 165 kB 00:00 2024-04-22T23:24:44.623 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:24:44.623 INFO:teuthology.orchestra.run.smithi107.stdout:Total 490 kB/s | 165 kB 00:00 2024-04-22T23:24:44.623 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-04-22T23:24:44.631 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-04-22T23:24:44.631 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-04-22T23:24:44.693 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-04-22T23:24:44.693 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-04-22T23:24:44.847 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-04-22T23:24:44.958 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-nbd-2:17.2.7-884.g70d0cb4b.el7.x86_64 1/1 2024-04-22T23:24:45.328 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: rbd-nbd-2:17.2.7-884.g70d0cb4b.el7.x86_64 1/1 2024-04-22T23:24:45.565 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-nbd-2:17.2.7-884.g70d0cb4b.el7.x86_64 1/1 2024-04-22T23:24:45.566 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:24:45.566 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-04-22T23:24:45.566 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-nbd-2:17.2.7-884.g70d0cb4b.el7.x86_64 2024-04-22T23:24:45.566 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:24:45.566 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-04-22T23:24:45.827 DEBUG:teuthology.parallel:result is None 2024-04-22T23:24:45.828 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-22T23:24:45.828 INFO:teuthology.packaging:ref: None 2024-04-22T23:24:45.828 INFO:teuthology.packaging:tag: None 2024-04-22T23:24:45.828 INFO:teuthology.packaging:branch: quincy 2024-04-22T23:24:45.828 INFO:teuthology.packaging:sha1: 43be020184947e53516056c9931e1ac5bdbbb1a5 2024-04-22T23:24:45.828 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-04-22T23:24:46.216 DEBUG:teuthology.orchestra.run.smithi107:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-22T23:24:46.238 INFO:teuthology.orchestra.run.smithi107.stdout:17.2.7-884.g70d0cb4b.el7 2024-04-22T23:24:46.238 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-884.g70d0cb4b.el7 2024-04-22T23:24:46.239 INFO:teuthology.task.install:The correct ceph version 17.2.7-884.g70d0cb4b is installed. 2024-04-22T23:24:46.241 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-22T23:24:46.241 INFO:teuthology.packaging:ref: None 2024-04-22T23:24:46.241 INFO:teuthology.packaging:tag: None 2024-04-22T23:24:46.241 INFO:teuthology.packaging:branch: quincy 2024-04-22T23:24:46.242 INFO:teuthology.packaging:sha1: 43be020184947e53516056c9931e1ac5bdbbb1a5 2024-04-22T23:24:46.242 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-04-22T23:24:46.461 DEBUG:teuthology.orchestra.run.smithi157:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-22T23:24:46.487 INFO:teuthology.orchestra.run.smithi157.stdout:17.2.7-884.g70d0cb4b.el7 2024-04-22T23:24:46.487 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-884.g70d0cb4b.el7 2024-04-22T23:24:46.487 INFO:teuthology.task.install:The correct ceph version 17.2.7-884.g70d0cb4b is installed. 2024-04-22T23:24:46.489 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-22T23:24:46.489 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-22T23:24:46.489 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-22T23:24:46.527 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-22T23:24:46.527 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-22T23:24:46.556 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-22T23:24:46.556 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-22T23:24:46.557 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/daemon-helper 2024-04-22T23:24:46.591 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-22T23:24:46.655 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-22T23:24:46.655 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/daemon-helper 2024-04-22T23:24:46.682 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-22T23:24:46.747 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-22T23:24:46.748 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-22T23:24:46.748 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-22T23:24:46.771 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-22T23:24:46.834 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-22T23:24:46.834 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-22T23:24:46.859 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-22T23:24:46.924 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-22T23:24:46.924 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-22T23:24:46.925 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/stdin-killer 2024-04-22T23:24:46.948 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-22T23:24:47.012 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-22T23:24:47.012 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/stdin-killer 2024-04-22T23:24:47.036 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-22T23:24:47.100 INFO:teuthology.run_tasks:Running task print... 2024-04-22T23:24:47.110 INFO:teuthology.task.print:**** done install task... 2024-04-22T23:24:47.111 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-22T23:24:47.490 INFO:tasks.cephadm:Config: {'cephadm_branch': 'quincy', 'cephadm_git_url': 'https://github.com/ceph/ceph', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}}, 'image': 'quay.ceph.io/ceph-ci/ceph:quincy', 'roleless': True, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'OSD_DOWN', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': '43be020184947e53516056c9931e1ac5bdbbb1a5'} 2024-04-22T23:24:47.490 INFO:tasks.cephadm:Cluster image is quay.ceph.io/ceph-ci/ceph:quincy 2024-04-22T23:24:47.491 INFO:tasks.cephadm:Cluster fsid is 8252f6f2-00ff-11ef-bc93-c7b262605968 2024-04-22T23:24:47.491 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-22T23:24:47.491 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-04-22T23:24:47.491 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi107': '172.21.15.107', 'mon.smithi157': '172.21.15.157'} 2024-04-22T23:24:47.491 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-22T23:24:47.491 DEBUG:teuthology.orchestra.run.smithi107:> sudo hostname $(hostname -s) 2024-04-22T23:24:47.516 DEBUG:teuthology.orchestra.run.smithi157:> sudo hostname $(hostname -s) 2024-04-22T23:24:47.541 INFO:tasks.cephadm:Downloading cephadm (repo https://github.com/ceph/ceph ref quincy)... 2024-04-22T23:24:47.541 DEBUG:teuthology.orchestra.run.smithi107:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-22T23:24:47.747 INFO:teuthology.orchestra.run.smithi107.stdout:-rw-r--r--. 1 ubuntu ubuntu 375153 Apr 22 23:24 /home/ubuntu/cephtest/cephadm 2024-04-22T23:24:47.747 DEBUG:teuthology.orchestra.run.smithi157:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-22T23:24:47.932 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r--. 1 ubuntu ubuntu 375153 Apr 22 23:24 /home/ubuntu/cephtest/cephadm 2024-04-22T23:24:47.932 DEBUG:teuthology.orchestra.run.smithi107:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-22T23:24:47.948 DEBUG:teuthology.orchestra.run.smithi157:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-22T23:24:47.978 INFO:tasks.cephadm:Pulling image quay.ceph.io/ceph-ci/ceph:quincy on all hosts... 2024-04-22T23:24:47.978 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-04-22T23:24:47.990 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-04-22T23:24:49.415 INFO:teuthology.orchestra.run.smithi157.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-04-22T23:24:49.625 INFO:teuthology.orchestra.run.smithi107.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-04-22T23:25:16.257 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-22T23:25:16.257 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_version": "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)", 2024-04-22T23:25:16.257 INFO:teuthology.orchestra.run.smithi107.stdout: "image_id": "cafd8ddac8fa176bf00aeacab8beaeeffcb0c5261d2d415c33683cb8bed0534e", 2024-04-22T23:25:16.257 INFO:teuthology.orchestra.run.smithi107.stdout: "repo_digests": [ 2024-04-22T23:25:16.257 INFO:teuthology.orchestra.run.smithi107.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:3890cd726bd5e73ae623dabdf4ed76dd5df81c936b8bd8d761026babda969466" 2024-04-22T23:25:16.257 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-04-22T23:25:16.257 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-22T23:25:16.712 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2024-04-22T23:25:16.712 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)", 2024-04-22T23:25:16.713 INFO:teuthology.orchestra.run.smithi157.stdout: "image_id": "cafd8ddac8fa176bf00aeacab8beaeeffcb0c5261d2d415c33683cb8bed0534e", 2024-04-22T23:25:16.713 INFO:teuthology.orchestra.run.smithi157.stdout: "repo_digests": [ 2024-04-22T23:25:16.713 INFO:teuthology.orchestra.run.smithi157.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:3890cd726bd5e73ae623dabdf4ed76dd5df81c936b8bd8d761026babda969466" 2024-04-22T23:25:16.713 INFO:teuthology.orchestra.run.smithi157.stdout: ] 2024-04-22T23:25:16.713 INFO:teuthology.orchestra.run.smithi157.stdout:} 2024-04-22T23:25:16.731 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /etc/ceph 2024-04-22T23:25:16.758 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /etc/ceph 2024-04-22T23:25:16.784 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 777 /etc/ceph 2024-04-22T23:25:16.821 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 777 /etc/ceph 2024-04-22T23:25:16.847 INFO:tasks.cephadm:Writing seed config... 2024-04-22T23:25:16.848 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-04-22T23:25:16.848 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-04-22T23:25:16.848 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-04-22T23:25:16.848 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-04-22T23:25:16.848 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-04-22T23:25:16.849 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-22T23:25:16.849 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-22T23:25:16.849 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-04-22T23:25:16.849 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-04-22T23:25:16.849 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-22T23:25:16.849 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-22T23:25:16.849 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-04-22T23:25:16.849 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-22T23:25:16.849 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-22T23:25:16.849 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-22T23:25:16.849 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-22T23:25:16.849 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-22T23:25:16.849 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-22T23:25:16.849 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-22T23:25:16.850 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-22T23:25:16.850 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-22T23:25:16.850 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-22T23:25:16.850 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-22T23:25:16.850 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-04-22T23:25:16.850 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-22T23:25:16.850 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-22T23:25:16.850 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-22T23:25:16.850 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-22T23:25:16.850 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-22T23:25:16.850 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-22T23:25:16.850 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-22T23:25:16.850 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-22T23:25:16.850 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-22T23:25:16.850 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-22T23:25:16.851 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-22T23:25:16.851 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-22T23:25:16.851 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-22T23:25:16.851 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-22T23:25:16.851 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-22T23:25:16.851 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-22T23:25:16.851 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-22T23:25:16.851 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-22T23:25:16.852 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-22T23:25:16.852 DEBUG:teuthology.orchestra.run.smithi107:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-22T23:25:16.876 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 8252f6f2-00ff-11ef-bc93-c7b262605968 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-22T23:25:16.876 DEBUG:teuthology.orchestra.run.smithi107:mon.smithi107> sudo journalctl -f -n 0 -u ceph-8252f6f2-00ff-11ef-bc93-c7b262605968@mon.smithi107.service 2024-04-22T23:25:16.918 INFO:tasks.cephadm:Bootstrapping... 2024-04-22T23:25:16.919 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy -v bootstrap --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.107 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-22T23:25:17.119 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-04-22T23:25:17.119 INFO:teuthology.orchestra.run.smithi107.stdout:cephadm ['--image', 'quay.ceph.io/ceph-ci/ceph:quincy', '-v', 'bootstrap', '--fsid', '8252f6f2-00ff-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.107', '--skip-admin-label'] 2024-04-22T23:25:17.145 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-22T23:25:17.145 INFO:teuthology.orchestra.run.smithi107.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-22T23:25:17.145 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying podman|docker is present... 2024-04-22T23:25:17.170 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-22T23:25:17.171 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying lvm2 is present... 2024-04-22T23:25:17.171 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying time synchronization is in place... 2024-04-22T23:25:17.178 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-22T23:25:17.179 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-22T23:25:17.184 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-22T23:25:17.185 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-04-22T23:25:17.191 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout enabled 2024-04-22T23:25:17.197 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout active 2024-04-22T23:25:17.197 INFO:teuthology.orchestra.run.smithi107.stdout:Unit chronyd.service is enabled and running 2024-04-22T23:25:17.198 INFO:teuthology.orchestra.run.smithi107.stdout:Repeating the final host check... 2024-04-22T23:25:17.221 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-22T23:25:17.221 INFO:teuthology.orchestra.run.smithi107.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-22T23:25:17.221 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl is present 2024-04-22T23:25:17.221 INFO:teuthology.orchestra.run.smithi107.stdout:lvcreate is present 2024-04-22T23:25:17.227 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-22T23:25:17.227 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-22T23:25:17.233 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-22T23:25:17.234 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-04-22T23:25:17.240 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout enabled 2024-04-22T23:25:17.247 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout active 2024-04-22T23:25:17.247 INFO:teuthology.orchestra.run.smithi107.stdout:Unit chronyd.service is enabled and running 2024-04-22T23:25:17.247 INFO:teuthology.orchestra.run.smithi107.stdout:Host looks OK 2024-04-22T23:25:17.247 INFO:teuthology.orchestra.run.smithi107.stdout:Cluster fsid: 8252f6f2-00ff-11ef-bc93-c7b262605968 2024-04-22T23:25:17.247 INFO:teuthology.orchestra.run.smithi107.stdout:Acquiring lock 140100106568608 on /run/cephadm/8252f6f2-00ff-11ef-bc93-c7b262605968.lock 2024-04-22T23:25:17.247 INFO:teuthology.orchestra.run.smithi107.stdout:Lock 140100106568608 acquired on /run/cephadm/8252f6f2-00ff-11ef-bc93-c7b262605968.lock 2024-04-22T23:25:17.247 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying IP 172.21.15.107 port 3300 ... 2024-04-22T23:25:17.248 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying IP 172.21.15.107 port 6789 ... 2024-04-22T23:25:17.248 INFO:teuthology.orchestra.run.smithi107.stdout:Base mon IP(s) is [172.21.15.107:3300, 172.21.15.107:6789], mon addrv is [v2:172.21.15.107:3300,v1:172.21.15.107:6789] 2024-04-22T23:25:17.251 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.107 metric 100 2024-04-22T23:25:17.251 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.107 metric 100 2024-04-22T23:25:17.254 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-22T23:25:17.254 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-04-22T23:25:17.254 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-04-22T23:25:17.256 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-22T23:25:17.257 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-22T23:25:17.257 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-22T23:25:17.257 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-04-22T23:25:17.257 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout inet6 fe80::5dc3:f4ce:498d:c8c6/64 scope link noprefixroute 2024-04-22T23:25:17.257 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-22T23:25:17.257 INFO:teuthology.orchestra.run.smithi107.stdout:Mon IP `172.21.15.107` is in CIDR network `172.21.0.0/20` 2024-04-22T23:25:17.258 INFO:teuthology.orchestra.run.smithi107.stdout:Mon IP `172.21.15.107` is in CIDR network `172.21.0.0/20` 2024-04-22T23:25:17.258 INFO:teuthology.orchestra.run.smithi107.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-22T23:25:17.258 INFO:teuthology.orchestra.run.smithi107.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-22T23:25:17.259 INFO:teuthology.orchestra.run.smithi107.stdout:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-04-22T23:25:17.723 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stdout cafd8ddac8fa176bf00aeacab8beaeeffcb0c5261d2d415c33683cb8bed0534e 2024-04-22T23:25:17.723 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Trying to pull quay.ceph.io/ceph-ci/ceph:quincy... 2024-04-22T23:25:17.723 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Getting image source signatures 2024-04-22T23:25:17.723 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Copying blob sha256:9977ec9f18cf69a6eaba4ac817ba680893fdb6756bbeabcd26f0bbb94288ba4a 2024-04-22T23:25:17.723 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Copying blob sha256:426fede9fffa2c7dfe40a4f9322a34794b72405170134d54d2c30d9cc6c6a5be 2024-04-22T23:25:17.724 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Copying config sha256:cafd8ddac8fa176bf00aeacab8beaeeffcb0c5261d2d415c33683cb8bed0534e 2024-04-22T23:25:17.724 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-22T23:25:18.668 INFO:teuthology.orchestra.run.smithi107.stdout:ceph: stdout ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable) 2024-04-22T23:25:18.668 INFO:teuthology.orchestra.run.smithi107.stdout:Ceph version: ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable) 2024-04-22T23:25:18.668 INFO:teuthology.orchestra.run.smithi107.stdout:Extracting ceph user uid/gid from container image... 2024-04-22T23:25:19.632 INFO:teuthology.orchestra.run.smithi107.stdout:stat: stdout 167 167 2024-04-22T23:25:19.632 INFO:teuthology.orchestra.run.smithi107.stdout:Creating initial keys... 2024-04-22T23:25:20.565 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph-authtool: stdout AQDf8SZm391oORAAOBtyWDgbMvqBNtSsUCalKA== 2024-04-22T23:25:21.448 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph-authtool: stdout AQDg8SZm1ChfNRAAhDyh+fpCSGjEVOjJR+M+jg== 2024-04-22T23:25:22.390 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph-authtool: stdout AQDh8SZmetl2LBAADry/pYNeb7+cfmq3gejapg== 2024-04-22T23:25:22.391 INFO:teuthology.orchestra.run.smithi107.stdout:Creating initial monmap... 2024-04-22T23:25:23.259 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-22T23:25:23.259 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = octopus 2024-04-22T23:25:23.259 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 8252f6f2-00ff-11ef-bc93-c7b262605968 2024-04-22T23:25:23.259 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-22T23:25:23.259 INFO:teuthology.orchestra.run.smithi107.stdout:monmaptool for smithi107 [v2:172.21.15.107:3300,v1:172.21.15.107:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-22T23:25:23.260 INFO:teuthology.orchestra.run.smithi107.stdout:setting min_mon_release = octopus 2024-04-22T23:25:23.260 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: set fsid to 8252f6f2-00ff-11ef-bc93-c7b262605968 2024-04-22T23:25:23.260 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-22T23:25:23.260 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:25:23.260 INFO:teuthology.orchestra.run.smithi107.stdout:Creating mon... 2024-04-22T23:25:24.190 INFO:teuthology.orchestra.run.smithi107.stdout:create mon.smithi107 on 2024-04-22T23:25:24.394 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-22T23:25:24.536 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-22T23:25:24.687 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-8252f6f2-00ff-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-8252f6f2-00ff-11ef-bc93-c7b262605968.target. 2024-04-22T23:25:24.687 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-8252f6f2-00ff-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-8252f6f2-00ff-11ef-bc93-c7b262605968.target. 2024-04-22T23:25:24.861 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-8252f6f2-00ff-11ef-bc93-c7b262605968@mon.smithi107 2024-04-22T23:25:24.861 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Failed to reset failed state of unit ceph-8252f6f2-00ff-11ef-bc93-c7b262605968@mon.smithi107.service: Unit ceph-8252f6f2-00ff-11ef-bc93-c7b262605968@mon.smithi107.service not loaded. 2024-04-22T23:25:25.019 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-8252f6f2-00ff-11ef-bc93-c7b262605968.target.wants/ceph-8252f6f2-00ff-11ef-bc93-c7b262605968@mon.smithi107.service → /etc/systemd/system/ceph-8252f6f2-00ff-11ef-bc93-c7b262605968@.service. 2024-04-22T23:25:25.571 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-22T23:25:25.572 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout disabled 2024-04-22T23:25:25.579 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-22T23:25:25.579 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-04-22T23:25:25.579 INFO:teuthology.orchestra.run.smithi107.stdout:firewalld.service is not enabled 2024-04-22T23:25:25.579 INFO:teuthology.orchestra.run.smithi107.stdout:Not possible to enable service . firewalld.service is not available 2024-04-22T23:25:25.580 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mon to start... 2024-04-22T23:25:25.580 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mon... 2024-04-22T23:25:25.705 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:25 smithi107 systemd[1]: Started Ceph mon.smithi107 for 8252f6f2-00ff-11ef-bc93-c7b262605968. 2024-04-22T23:25:26.716 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout cluster: 2024-04-22T23:25:26.716 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout id: 8252f6f2-00ff-11ef-bc93-c7b262605968 2024-04-22T23:25:26.716 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-22T23:25:26.716 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-04-22T23:25:26.716 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout services: 2024-04-22T23:25:26.716 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi107 (age 0.634064s) 2024-04-22T23:25:26.716 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-22T23:25:26.716 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-22T23:25:26.716 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-04-22T23:25:26.716 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout data: 2024-04-22T23:25:26.717 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-22T23:25:26.717 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-22T23:25:26.717 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-22T23:25:26.717 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout pgs: 2024-04-22T23:25:26.717 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-04-22T23:25:26.717 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:26.054+0000 7f7f1983f700 1 Processor -- start 2024-04-22T23:25:26.717 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:26.054+0000 7f7f1983f700 1 -- start start 2024-04-22T23:25:26.717 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:26.054+0000 7f7f1983f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f14102930 0x7f7f14102d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:25:26.717 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:26.054+0000 7f7f1983f700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f14103280 con 0x7f7f14102930 2024-04-22T23:25:26.717 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:26.054+0000 7f7f12ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f14102930 0x7f7f14102d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:25:26.717 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:26.054+0000 7f7f12ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f14102930 0x7f7f14102d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42318/0 (socket says 172.21.15.107:42318) 2024-04-22T23:25:26.717 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:26.054+0000 7f7f12ffd700 1 -- 172.21.15.107:0/2798152599 learned_addr learned my addr 172.21.15.107:0/2798152599 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:25:26.717 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:26.055+0000 7f7f12ffd700 1 -- 172.21.15.107:0/2798152599 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f141033c0 con 0x7f7f14102930 2024-04-22T23:25:26.718 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:26.055+0000 7f7f12ffd700 1 --2- 172.21.15.107:0/2798152599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f14102930 0x7f7f14102d40 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7f040098e0 tx=0x7f7f04009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1266768dc0868d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:25:26.718 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:26.055+0000 7f7f11ffb700 1 -- 172.21.15.107:0/2798152599 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7f0400b8f0 con 0x7f7f14102930 2024-04-22T23:25:26.718 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:26.055+0000 7f7f11ffb700 1 -- 172.21.15.107:0/2798152599 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f0400ba50 con 0x7f7f14102930 2024-04-22T23:25:26.718 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:26.055+0000 7f7f11ffb700 1 -- 172.21.15.107:0/2798152599 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7f0400bd20 con 0x7f7f14102930 2024-04-22T23:25:26.718 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:26.055+0000 7f7f1983f700 1 -- 172.21.15.107:0/2798152599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f14102930 msgr2=0x7f7f14102d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:25:26.718 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:26.055+0000 7f7f1983f700 1 --2- 172.21.15.107:0/2798152599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f14102930 0x7f7f14102d40 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7f040098e0 tx=0x7f7f04009250 comp rx=0 tx=0).stop 2024-04-22T23:25:26.718 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:26.055+0000 7f7f1983f700 1 -- 172.21.15.107:0/2798152599 shutdown_connections 2024-04-22T23:25:26.718 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:26.055+0000 7f7f1983f700 1 --2- 172.21.15.107:0/2798152599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f14102930 0x7f7f14102d40 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:25:26.718 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:26.055+0000 7f7f1983f700 1 -- 172.21.15.107:0/2798152599 >> 172.21.15.107:0/2798152599 conn(0x7f7f140fe140 msgr2=0x7f7f14100550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:25:26.718 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:26.055+0000 7f7f1983f700 1 -- 172.21.15.107:0/2798152599 shutdown_connections 2024-04-22T23:25:26.718 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:26.055+0000 7f7f1983f700 1 -- 172.21.15.107:0/2798152599 wait complete. 2024-04-22T23:25:26.718 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:26.056+0000 7f7f1983f700 1 Processor -- start 2024-04-22T23:25:26.718 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:26.056+0000 7f7f1983f700 1 -- start start 2024-04-22T23:25:26.719 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:26.056+0000 7f7f1983f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f14102930 0x7f7f14110f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:25:26.719 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:26.056+0000 7f7f1983f700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f14111440 con 0x7f7f14102930 2024-04-22T23:25:26.719 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:26.056+0000 7f7f12ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f14102930 0x7f7f14110f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:25:26.719 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:26.056+0000 7f7f12ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f14102930 0x7f7f14110f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42322/0 (socket says 172.21.15.107:42322) 2024-04-22T23:25:26.719 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:26.056+0000 7f7f12ffd700 1 -- 172.21.15.107:0/3451122117 learned_addr learned my addr 172.21.15.107:0/3451122117 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:25:26.719 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:26.057+0000 7f7f12ffd700 1 -- 172.21.15.107:0/3451122117 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f040094f0 con 0x7f7f14102930 2024-04-22T23:25:26.719 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:26.057+0000 7f7f12ffd700 1 --2- 172.21.15.107:0/3451122117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f14102930 0x7f7f14110f00 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7f04009290 tx=0x7f7f04016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:25:26.719 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:26.057+0000 7f7efb7fe700 1 -- 172.21.15.107:0/3451122117 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7f0401c670 con 0x7f7f14102930 2024-04-22T23:25:26.719 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:26.057+0000 7f7efb7fe700 1 -- 172.21.15.107:0/3451122117 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f0401c7d0 con 0x7f7f14102930 2024-04-22T23:25:26.719 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:26.057+0000 7f7efb7fe700 1 -- 172.21.15.107:0/3451122117 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7f0401cae0 con 0x7f7f14102930 2024-04-22T23:25:26.719 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:26.057+0000 7f7f1983f700 1 -- 172.21.15.107:0/3451122117 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f14111640 con 0x7f7f14102930 2024-04-22T23:25:26.719 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:26.057+0000 7f7f1983f700 1 -- 172.21.15.107:0/3451122117 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f141120d0 con 0x7f7f14102930 2024-04-22T23:25:26.719 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:26.057+0000 7f7efb7fe700 1 -- 172.21.15.107:0/3451122117 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f7f0401cc40 con 0x7f7f14102930 2024-04-22T23:25:26.719 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:26.057+0000 7f7efb7fe700 1 -- 172.21.15.107:0/3451122117 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f7f0401ad30 con 0x7f7f14102930 2024-04-22T23:25:26.720 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:26.058+0000 7f7f1983f700 1 -- 172.21.15.107:0/3451122117 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f1405f5f0 con 0x7f7f14102930 2024-04-22T23:25:26.720 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:26.060+0000 7f7efb7fe700 1 -- 172.21.15.107:0/3451122117 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7f7f0401a450 con 0x7f7f14102930 2024-04-22T23:25:26.720 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:26.129+0000 7f7f1983f700 1 -- 172.21.15.107:0/3451122117 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f7f14103ba0 con 0x7f7f14102930 2024-04-22T23:25:26.720 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:26.129+0000 7f7efb7fe700 1 -- 172.21.15.107:0/3451122117 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+325 (secure 0 0 0) 0x7f7f04025440 con 0x7f7f14102930 2024-04-22T23:25:26.720 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:26.130+0000 7f7f1983f700 1 -- 172.21.15.107:0/3451122117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f14102930 msgr2=0x7f7f14110f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:25:26.720 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:26.130+0000 7f7f1983f700 1 --2- 172.21.15.107:0/3451122117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f14102930 0x7f7f14110f00 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7f04009290 tx=0x7f7f04016000 comp rx=0 tx=0).stop 2024-04-22T23:25:26.720 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:26.130+0000 7f7f1983f700 1 -- 172.21.15.107:0/3451122117 shutdown_connections 2024-04-22T23:25:26.720 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:26.130+0000 7f7f1983f700 1 --2- 172.21.15.107:0/3451122117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f14102930 0x7f7f14110f00 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:25:26.720 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:26.130+0000 7f7f1983f700 1 -- 172.21.15.107:0/3451122117 >> 172.21.15.107:0/3451122117 conn(0x7f7f140fe140 msgr2=0x7f7f14105320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:25:26.720 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:26.130+0000 7f7f1983f700 1 -- 172.21.15.107:0/3451122117 shutdown_connections 2024-04-22T23:25:26.720 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:26.130+0000 7f7f1983f700 1 -- 172.21.15.107:0/3451122117 wait complete. 2024-04-22T23:25:26.720 INFO:teuthology.orchestra.run.smithi107.stdout:mon is available 2024-04-22T23:25:26.720 INFO:teuthology.orchestra.run.smithi107.stdout:Assimilating anything we can from ceph.conf... 2024-04-22T23:25:26.740 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:26 smithi107 ceph-mon[25980]: mon.smithi107 is new leader, mons smithi107 in quorum (ranks 0) 2024-04-22T23:25:26.740 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:26 smithi107 ceph-mon[25980]: monmap e1: 1 mons at {smithi107=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-22T23:25:26.740 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:26 smithi107 ceph-mon[25980]: fsmap 2024-04-22T23:25:26.740 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:26 smithi107 ceph-mon[25980]: osdmap e1: 0 total, 0 up, 0 in 2024-04-22T23:25:26.741 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:26 smithi107 ceph-mon[25980]: mgrmap e1: no daemons active 2024-04-22T23:25:26.741 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:26 smithi107 ceph-mon[25980]: from='client.? 172.21.15.107:0/3451122117' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-22T23:25:27.799 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:27 smithi107 ceph-mon[25980]: from='client.? 172.21.15.107:0/2090210473' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-22T23:25:27.799 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:27 smithi107 ceph-mon[25980]: from='client.? 172.21.15.107:0/2090210473' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-22T23:25:27.801 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-04-22T23:25:27.808 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [global] 2024-04-22T23:25:27.808 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout fsid = 8252f6f2-00ff-11ef-bc93-c7b262605968 2024-04-22T23:25:27.808 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.107:3300,v1:172.21.15.107:6789] 2024-04-22T23:25:27.808 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-22T23:25:27.808 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-22T23:25:27.808 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-22T23:25:27.808 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-22T23:25:27.808 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-04-22T23:25:27.808 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-22T23:25:27.808 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-22T23:25:27.808 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-04-22T23:25:27.808 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [osd] 2024-04-22T23:25:27.808 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-22T23:25:27.809 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-22T23:25:27.809 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-22T23:25:27.809 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-22T23:25:27.809 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-22T23:25:27.809 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:27.157+0000 7fcb5ec1f700 1 Processor -- start 2024-04-22T23:25:27.809 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:27.158+0000 7fcb5ec1f700 1 -- start start 2024-04-22T23:25:27.809 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:27.158+0000 7fcb5ec1f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb58100cd0 0x7fcb581010e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:25:27.809 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:27.158+0000 7fcb5ec1f700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb58101620 con 0x7fcb58100cd0 2024-04-22T23:25:27.809 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:27.158+0000 7fcb5c9bb700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb58100cd0 0x7fcb581010e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:25:27.809 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:27.158+0000 7fcb5c9bb700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb58100cd0 0x7fcb581010e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42332/0 (socket says 172.21.15.107:42332) 2024-04-22T23:25:27.809 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:27.158+0000 7fcb5c9bb700 1 -- 172.21.15.107:0/3233939613 learned_addr learned my addr 172.21.15.107:0/3233939613 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:25:27.809 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:27.158+0000 7fcb5c9bb700 1 -- 172.21.15.107:0/3233939613 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb58101760 con 0x7fcb58100cd0 2024-04-22T23:25:27.809 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:27.158+0000 7fcb5c9bb700 1 --2- 172.21.15.107:0/3233939613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb58100cd0 0x7fcb581010e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c0098e0 tx=0x7fcb4c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5094c12d69db78e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:25:27.810 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:27.158+0000 7fcb577fe700 1 -- 172.21.15.107:0/3233939613 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcb4c00b860 con 0x7fcb58100cd0 2024-04-22T23:25:27.810 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:27.159+0000 7fcb577fe700 1 -- 172.21.15.107:0/3233939613 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb4c00b9c0 con 0x7fcb58100cd0 2024-04-22T23:25:27.810 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:27.159+0000 7fcb577fe700 1 -- 172.21.15.107:0/3233939613 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcb4c00bc90 con 0x7fcb58100cd0 2024-04-22T23:25:27.810 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:27.159+0000 7fcb5ec1f700 1 -- 172.21.15.107:0/3233939613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb58100cd0 msgr2=0x7fcb581010e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:25:27.810 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:27.159+0000 7fcb5ec1f700 1 --2- 172.21.15.107:0/3233939613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb58100cd0 0x7fcb581010e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c0098e0 tx=0x7fcb4c009250 comp rx=0 tx=0).stop 2024-04-22T23:25:27.810 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:27.159+0000 7fcb5ec1f700 1 -- 172.21.15.107:0/3233939613 shutdown_connections 2024-04-22T23:25:27.810 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:27.159+0000 7fcb5ec1f700 1 --2- 172.21.15.107:0/3233939613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb58100cd0 0x7fcb581010e0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:25:27.810 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:27.159+0000 7fcb5ec1f700 1 -- 172.21.15.107:0/3233939613 >> 172.21.15.107:0/3233939613 conn(0x7fcb580fc770 msgr2=0x7fcb580feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:25:27.810 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:27.159+0000 7fcb5ec1f700 1 -- 172.21.15.107:0/3233939613 shutdown_connections 2024-04-22T23:25:27.810 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:27.159+0000 7fcb5ec1f700 1 -- 172.21.15.107:0/3233939613 wait complete. 2024-04-22T23:25:27.810 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:27.160+0000 7fcb5ec1f700 1 Processor -- start 2024-04-22T23:25:27.810 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:27.160+0000 7fcb5ec1f700 1 -- start start 2024-04-22T23:25:27.811 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:27.160+0000 7fcb5ec1f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb58100cd0 0x7fcb58110f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:25:27.811 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:27.160+0000 7fcb5ec1f700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb58111440 con 0x7fcb58100cd0 2024-04-22T23:25:27.811 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:27.160+0000 7fcb5c9bb700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb58100cd0 0x7fcb58110f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:25:27.811 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:27.160+0000 7fcb5c9bb700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb58100cd0 0x7fcb58110f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42344/0 (socket says 172.21.15.107:42344) 2024-04-22T23:25:27.811 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:27.160+0000 7fcb5c9bb700 1 -- 172.21.15.107:0/2090210473 learned_addr learned my addr 172.21.15.107:0/2090210473 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:25:27.811 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:27.160+0000 7fcb5c9bb700 1 -- 172.21.15.107:0/2090210473 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb4c0094f0 con 0x7fcb58100cd0 2024-04-22T23:25:27.811 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:27.160+0000 7fcb5c9bb700 1 --2- 172.21.15.107:0/2090210473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb58100cd0 0x7fcb58110f00 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c003080 tx=0x7fcb4c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:25:27.811 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:27.161+0000 7fcb557fa700 1 -- 172.21.15.107:0/2090210473 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcb4c01c5c0 con 0x7fcb58100cd0 2024-04-22T23:25:27.811 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:27.161+0000 7fcb557fa700 1 -- 172.21.15.107:0/2090210473 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb4c01c720 con 0x7fcb58100cd0 2024-04-22T23:25:27.811 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:27.161+0000 7fcb5ec1f700 1 -- 172.21.15.107:0/2090210473 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb58111640 con 0x7fcb58100cd0 2024-04-22T23:25:27.811 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:27.161+0000 7fcb557fa700 1 -- 172.21.15.107:0/2090210473 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcb4c01ca30 con 0x7fcb58100cd0 2024-04-22T23:25:27.811 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:27.161+0000 7fcb5ec1f700 1 -- 172.21.15.107:0/2090210473 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb581120d0 con 0x7fcb58100cd0 2024-04-22T23:25:27.811 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:27.161+0000 7fcb557fa700 1 -- 172.21.15.107:0/2090210473 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fcb4c01cc60 con 0x7fcb58100cd0 2024-04-22T23:25:27.811 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:27.161+0000 7fcb557fa700 1 -- 172.21.15.107:0/2090210473 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fcb4c01ac80 con 0x7fcb58100cd0 2024-04-22T23:25:27.812 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:27.162+0000 7fcb5ec1f700 1 -- 172.21.15.107:0/2090210473 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb5805f5f0 con 0x7fcb58100cd0 2024-04-22T23:25:27.812 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:27.164+0000 7fcb557fa700 1 -- 172.21.15.107:0/2090210473 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7fcb4c01a450 con 0x7fcb58100cd0 2024-04-22T23:25:27.812 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:27.230+0000 7fcb5ec1f700 1 -- 172.21.15.107:0/2090210473 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fcb5804bed0 con 0x7fcb58100cd0 2024-04-22T23:25:27.812 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:27.233+0000 7fcb557fa700 1 -- 172.21.15.107:0/2090210473 <== mon.0 v2:172.21.15.107:3300/0 7 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fcb4c024870 con 0x7fcb58100cd0 2024-04-22T23:25:27.812 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:27.233+0000 7fcb557fa700 1 -- 172.21.15.107:0/2090210473 <== mon.0 v2:172.21.15.107:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+431 (secure 0 0 0) 0x7fcb4c0364f0 con 0x7fcb58100cd0 2024-04-22T23:25:27.812 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:27.234+0000 7fcb5ec1f700 1 -- 172.21.15.107:0/2090210473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb58100cd0 msgr2=0x7fcb58110f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:25:27.812 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:27.234+0000 7fcb5ec1f700 1 --2- 172.21.15.107:0/2090210473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb58100cd0 0x7fcb58110f00 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c003080 tx=0x7fcb4c016000 comp rx=0 tx=0).stop 2024-04-22T23:25:27.812 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:27.234+0000 7fcb5ec1f700 1 -- 172.21.15.107:0/2090210473 shutdown_connections 2024-04-22T23:25:27.812 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:27.234+0000 7fcb5ec1f700 1 --2- 172.21.15.107:0/2090210473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb58100cd0 0x7fcb58110f00 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:25:27.812 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:27.234+0000 7fcb5ec1f700 1 -- 172.21.15.107:0/2090210473 >> 172.21.15.107:0/2090210473 conn(0x7fcb580fc770 msgr2=0x7fcb580fcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:25:27.812 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:27.234+0000 7fcb5ec1f700 1 -- 172.21.15.107:0/2090210473 shutdown_connections 2024-04-22T23:25:27.812 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:27.234+0000 7fcb5ec1f700 1 -- 172.21.15.107:0/2090210473 wait complete. 2024-04-22T23:25:27.812 INFO:teuthology.orchestra.run.smithi107.stdout:Generating new minimal ceph.conf... 2024-04-22T23:25:28.816 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:28 smithi107 ceph-mon[25980]: from='client.? 172.21.15.107:0/2453801326' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:25:28.823 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:28.203+0000 7feb64c42700 1 Processor -- start 2024-04-22T23:25:28.824 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:28.203+0000 7feb64c42700 1 -- start start 2024-04-22T23:25:28.824 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:28.203+0000 7feb64c42700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb60077ab0 0x7feb60076150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:25:28.824 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:28.203+0000 7feb64c42700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb60077ec0 con 0x7feb60077ab0 2024-04-22T23:25:28.824 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:28.204+0000 7feb5e59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb60077ab0 0x7feb60076150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:25:28.824 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:28.204+0000 7feb5e59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb60077ab0 0x7feb60076150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42348/0 (socket says 172.21.15.107:42348) 2024-04-22T23:25:28.824 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:28.204+0000 7feb5e59c700 1 -- 172.21.15.107:0/3699415485 learned_addr learned my addr 172.21.15.107:0/3699415485 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:25:28.824 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:28.204+0000 7feb5e59c700 1 -- 172.21.15.107:0/3699415485 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb60078000 con 0x7feb60077ab0 2024-04-22T23:25:28.824 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:28.204+0000 7feb5e59c700 1 --2- 172.21.15.107:0/3699415485 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb60077ab0 0x7feb60076150 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7feb58009e50 tx=0x7feb58009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1f143c5256ecdc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:25:28.824 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:28.204+0000 7feb5d59a700 1 -- 172.21.15.107:0/3699415485 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7feb580092e0 con 0x7feb60077ab0 2024-04-22T23:25:28.824 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:28.204+0000 7feb5d59a700 1 -- 172.21.15.107:0/3699415485 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7feb58012430 con 0x7feb60077ab0 2024-04-22T23:25:28.824 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:28.204+0000 7feb5d59a700 1 -- 172.21.15.107:0/3699415485 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7feb580075c0 con 0x7feb60077ab0 2024-04-22T23:25:28.824 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:28.205+0000 7feb64c42700 1 -- 172.21.15.107:0/3699415485 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb60077ab0 msgr2=0x7feb60076150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:25:28.825 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:28.205+0000 7feb64c42700 1 --2- 172.21.15.107:0/3699415485 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb60077ab0 0x7feb60076150 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7feb58009e50 tx=0x7feb58009840 comp rx=0 tx=0).stop 2024-04-22T23:25:28.825 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:28.205+0000 7feb64c42700 1 -- 172.21.15.107:0/3699415485 shutdown_connections 2024-04-22T23:25:28.825 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:28.205+0000 7feb64c42700 1 --2- 172.21.15.107:0/3699415485 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb60077ab0 0x7feb60076150 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:25:28.825 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:28.205+0000 7feb64c42700 1 -- 172.21.15.107:0/3699415485 >> 172.21.15.107:0/3699415485 conn(0x7feb600fe0b0 msgr2=0x7feb60100500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:25:28.825 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:28.205+0000 7feb64c42700 1 -- 172.21.15.107:0/3699415485 shutdown_connections 2024-04-22T23:25:28.825 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:28.205+0000 7feb64c42700 1 -- 172.21.15.107:0/3699415485 wait complete. 2024-04-22T23:25:28.825 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:28.205+0000 7feb64c42700 1 Processor -- start 2024-04-22T23:25:28.825 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:28.206+0000 7feb64c42700 1 -- start start 2024-04-22T23:25:28.825 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:28.206+0000 7feb64c42700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb60077ab0 0x7feb60113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:25:28.825 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:28.206+0000 7feb64c42700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb60113ec0 con 0x7feb60077ab0 2024-04-22T23:25:28.825 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:28.206+0000 7feb5e59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb60077ab0 0x7feb60113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:25:28.825 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:28.206+0000 7feb5e59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb60077ab0 0x7feb60113980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42356/0 (socket says 172.21.15.107:42356) 2024-04-22T23:25:28.825 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:28.206+0000 7feb5e59c700 1 -- 172.21.15.107:0/2453801326 learned_addr learned my addr 172.21.15.107:0/2453801326 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:25:28.825 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:28.206+0000 7feb5e59c700 1 -- 172.21.15.107:0/2453801326 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb58009ae0 con 0x7feb60077ab0 2024-04-22T23:25:28.825 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:28.206+0000 7feb5e59c700 1 --2- 172.21.15.107:0/2453801326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb60077ab0 0x7feb60113980 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7feb58000f80 tx=0x7feb58009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:25:28.826 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:28.206+0000 7feb56ffd700 1 -- 172.21.15.107:0/2453801326 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7feb580092e0 con 0x7feb60077ab0 2024-04-22T23:25:28.826 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:28.206+0000 7feb64c42700 1 -- 172.21.15.107:0/2453801326 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb601140c0 con 0x7feb60077ab0 2024-04-22T23:25:28.826 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:28.206+0000 7feb56ffd700 1 -- 172.21.15.107:0/2453801326 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7feb58007cc0 con 0x7feb60077ab0 2024-04-22T23:25:28.826 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:28.206+0000 7feb56ffd700 1 -- 172.21.15.107:0/2453801326 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7feb5801c960 con 0x7feb60077ab0 2024-04-22T23:25:28.826 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:28.206+0000 7feb64c42700 1 -- 172.21.15.107:0/2453801326 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb60114c30 con 0x7feb60077ab0 2024-04-22T23:25:28.826 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:28.207+0000 7feb56ffd700 1 -- 172.21.15.107:0/2453801326 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7feb58019070 con 0x7feb60077ab0 2024-04-22T23:25:28.826 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:28.207+0000 7feb56ffd700 1 -- 172.21.15.107:0/2453801326 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7feb58025b90 con 0x7feb60077ab0 2024-04-22T23:25:28.826 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:28.207+0000 7feb64c42700 1 -- 172.21.15.107:0/2453801326 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb6005f5f0 con 0x7feb60077ab0 2024-04-22T23:25:28.826 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:28.210+0000 7feb56ffd700 1 -- 172.21.15.107:0/2453801326 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7feb5802a060 con 0x7feb60077ab0 2024-04-22T23:25:28.826 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:28.276+0000 7feb64c42700 1 -- 172.21.15.107:0/2453801326 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7feb60108ac0 con 0x7feb60077ab0 2024-04-22T23:25:28.826 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:28.276+0000 7feb56ffd700 1 -- 172.21.15.107:0/2453801326 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+177 (secure 0 0 0) 0x7feb58020070 con 0x7feb60077ab0 2024-04-22T23:25:28.826 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:28.276+0000 7feb64c42700 1 -- 172.21.15.107:0/2453801326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb60077ab0 msgr2=0x7feb60113980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:25:28.826 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:28.276+0000 7feb64c42700 1 --2- 172.21.15.107:0/2453801326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb60077ab0 0x7feb60113980 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7feb58000f80 tx=0x7feb58009840 comp rx=0 tx=0).stop 2024-04-22T23:25:28.826 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:28.277+0000 7feb64c42700 1 -- 172.21.15.107:0/2453801326 shutdown_connections 2024-04-22T23:25:28.827 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:28.277+0000 7feb64c42700 1 --2- 172.21.15.107:0/2453801326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb60077ab0 0x7feb60113980 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:25:28.827 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:28.277+0000 7feb64c42700 1 -- 172.21.15.107:0/2453801326 >> 172.21.15.107:0/2453801326 conn(0x7feb600fe0b0 msgr2=0x7feb600fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:25:28.827 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:28.277+0000 7feb64c42700 1 -- 172.21.15.107:0/2453801326 shutdown_connections 2024-04-22T23:25:28.827 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:28.277+0000 7feb64c42700 1 -- 172.21.15.107:0/2453801326 wait complete. 2024-04-22T23:25:28.827 INFO:teuthology.orchestra.run.smithi107.stdout:Restarting the monitor... 2024-04-22T23:25:29.095 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:28 smithi107 systemd[1]: Stopping Ceph mon.smithi107 for 8252f6f2-00ff-11ef-bc93-c7b262605968... 2024-04-22T23:25:29.491 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:29 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-mon-smithi107[25976]: 2024-04-22T23:25:29.093+0000 7f15f5c51700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi107 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T23:25:29.491 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:29 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-mon-smithi107[25976]: 2024-04-22T23:25:29.093+0000 7f15f5c51700 -1 mon.smithi107@0(leader) e1 *** Got Signal Terminated *** 2024-04-22T23:25:29.491 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:29 smithi107 podman[26228]: 2024-04-22 23:25:29.277844101 +0000 UTC m=+0.290202557 container died c560428f3fddd0eabc017d532aa85e9a3e564295322b7dbcbf5e3590f85614a6 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-mon-smithi107, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_CLEAN=True, RELEASE=quincy-70d0cb4, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-22T23:25:33.241 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:32 smithi107 podman[26228]: 2024-04-22 23:25:32.910361137 +0000 UTC m=+3.922719582 container cleanup c560428f3fddd0eabc017d532aa85e9a3e564295322b7dbcbf5e3590f85614a6 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-mon-smithi107, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=quincy-70d0cb4, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416) 2024-04-22T23:25:33.241 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:32 smithi107 bash[26228]: ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-mon-smithi107 2024-04-22T23:25:34.991 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:34 smithi107 podman[26240]: 2024-04-22 23:25:34.685722127 +0000 UTC m=+5.411684648 container remove c560428f3fddd0eabc017d532aa85e9a3e564295322b7dbcbf5e3590f85614a6 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-mon-smithi107, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=quincy-70d0cb4, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-22T23:25:35.491 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:35 smithi107 systemd[1]: ceph-8252f6f2-00ff-11ef-bc93-c7b262605968@mon.smithi107.service: Deactivated successfully. 2024-04-22T23:25:35.491 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:35 smithi107 systemd[1]: Stopped Ceph mon.smithi107 for 8252f6f2-00ff-11ef-bc93-c7b262605968. 2024-04-22T23:25:35.491 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:35 smithi107 systemd[1]: Starting Ceph mon.smithi107 for 8252f6f2-00ff-11ef-bc93-c7b262605968... 2024-04-22T23:25:35.491 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:35 smithi107 podman[26336]: 2024-04-22 23:25:35.14934304 +0000 UTC m=+0.018312884 image pull cafd8ddac8fa176bf00aeacab8beaeeffcb0c5261d2d415c33683cb8bed0534e quay.ceph.io/ceph-ci/ceph:quincy 2024-04-22T23:25:36.740 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:36 smithi107 podman[26336]: 2024-04-22 23:25:36.262749392 +0000 UTC m=+1.131719216 container create 8314962fa1b933e18567c31af8eaa108e6792030a367debbd28d139d120a8450 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-mon-smithi107, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=quincy-70d0cb4, maintainer=Guillaume Abrioux ) 2024-04-22T23:25:38.491 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:37 smithi107 podman[26336]: 2024-04-22 23:25:37.94564777 +0000 UTC m=+2.814617594 container init 8314962fa1b933e18567c31af8eaa108e6792030a367debbd28d139d120a8450 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-mon-smithi107, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=quincy-70d0cb4, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2) 2024-04-22T23:25:38.491 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 podman[26336]: 2024-04-22 23:25:38.001618538 +0000 UTC m=+2.870588361 container start 8314962fa1b933e18567c31af8eaa108e6792030a367debbd28d139d120a8450 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-mon-smithi107, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=quincy-70d0cb4, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-22T23:25:38.491 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: set uid:gid to 167:167 (ceph:ceph) 2024-04-22T23:25:38.491 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable), process ceph-mon, pid 2 2024-04-22T23:25:38.491 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: pidfile_write: ignore empty --pid-file 2024-04-22T23:25:38.491 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: load: jerasure load: lrc 2024-04-22T23:25:38.491 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: RocksDB version: 6.15.5 2024-04-22T23:25:38.491 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-04-22T23:25:38.492 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Compile date Apr 22 2024 2024-04-22T23:25:38.492 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: DB SUMMARY 2024-04-22T23:25:38.492 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: DB Session ID: 53ZE2LRF3349PO4XZHEP 2024-04-22T23:25:38.492 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: CURRENT file: CURRENT 2024-04-22T23:25:38.492 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: IDENTITY file: IDENTITY 2024-04-22T23:25:38.492 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: MANIFEST file: MANIFEST-000009 size: 131 Bytes 2024-04-22T23:25:38.492 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi107/store.db dir, Total Num: 1, files: 000008.sst 2024-04-22T23:25:38.492 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi107/store.db: 000010.log size: 98192 ; 2024-04-22T23:25:38.492 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.error_if_exists: 0 2024-04-22T23:25:38.492 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.create_if_missing: 0 2024-04-22T23:25:38.492 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.paranoid_checks: 1 2024-04-22T23:25:38.492 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-22T23:25:38.492 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.env: 0x5594ae026b00 2024-04-22T23:25:38.493 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.fs: Posix File System 2024-04-22T23:25:38.493 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.info_log: 0x5594af41bc00 2024-04-22T23:25:38.493 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.max_file_opening_threads: 16 2024-04-22T23:25:38.493 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.statistics: (nil) 2024-04-22T23:25:38.493 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.use_fsync: 0 2024-04-22T23:25:38.493 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.max_log_file_size: 0 2024-04-22T23:25:38.493 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-22T23:25:38.493 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-22T23:25:38.493 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.keep_log_file_num: 1000 2024-04-22T23:25:38.493 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.recycle_log_file_num: 0 2024-04-22T23:25:38.493 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.allow_fallocate: 1 2024-04-22T23:25:38.493 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.allow_mmap_reads: 0 2024-04-22T23:25:38.493 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.allow_mmap_writes: 0 2024-04-22T23:25:38.493 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.use_direct_reads: 0 2024-04-22T23:25:38.493 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-22T23:25:38.494 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.create_missing_column_families: 0 2024-04-22T23:25:38.494 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.db_log_dir: 2024-04-22T23:25:38.494 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi107/store.db 2024-04-22T23:25:38.494 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-22T23:25:38.494 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-22T23:25:38.494 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-22T23:25:38.494 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-22T23:25:38.494 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-22T23:25:38.494 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-22T23:25:38.494 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.advise_random_on_open: 1 2024-04-22T23:25:38.494 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.db_write_buffer_size: 0 2024-04-22T23:25:38.494 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.write_buffer_manager: 0x5594b01909c0 2024-04-22T23:25:38.494 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-22T23:25:38.494 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-04-22T23:25:38.495 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-22T23:25:38.495 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-22T23:25:38.495 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.rate_limiter: (nil) 2024-04-22T23:25:38.495 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-22T23:25:38.495 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.wal_recovery_mode: 2 2024-04-22T23:25:38.495 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.enable_thread_tracking: 0 2024-04-22T23:25:38.495 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.enable_pipelined_write: 0 2024-04-22T23:25:38.495 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.unordered_write: 0 2024-04-22T23:25:38.495 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-22T23:25:38.495 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-22T23:25:38.495 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-22T23:25:38.495 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-22T23:25:38.495 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.row_cache: None 2024-04-22T23:25:38.495 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.wal_filter: None 2024-04-22T23:25:38.496 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-22T23:25:38.496 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.allow_ingest_behind: 0 2024-04-22T23:25:38.497 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.preserve_deletes: 0 2024-04-22T23:25:38.497 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.two_write_queues: 0 2024-04-22T23:25:38.497 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.manual_wal_flush: 0 2024-04-22T23:25:38.497 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.atomic_flush: 0 2024-04-22T23:25:38.497 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-22T23:25:38.497 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-22T23:25:38.497 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-22T23:25:38.497 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.log_readahead_size: 0 2024-04-22T23:25:38.498 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-22T23:25:38.498 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.best_efforts_recovery: 0 2024-04-22T23:25:38.498 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-22T23:25:38.498 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-22T23:25:38.498 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.allow_data_in_errors: 0 2024-04-22T23:25:38.498 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.db_host_id: __hostname__ 2024-04-22T23:25:38.498 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.max_background_jobs: 2 2024-04-22T23:25:38.498 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.max_background_compactions: -1 2024-04-22T23:25:38.498 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.max_subcompactions: 1 2024-04-22T23:25:38.498 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-22T23:25:38.498 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-22T23:25:38.498 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-22T23:25:38.498 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.max_total_wal_size: 0 2024-04-22T23:25:38.498 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-22T23:25:38.499 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-22T23:25:38.499 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-22T23:25:38.499 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-22T23:25:38.499 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.max_open_files: -1 2024-04-22T23:25:38.499 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.bytes_per_sync: 0 2024-04-22T23:25:38.499 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-22T23:25:38.499 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-22T23:25:38.499 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.compaction_readahead_size: 0 2024-04-22T23:25:38.499 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.max_background_flushes: -1 2024-04-22T23:25:38.499 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Compression algorithms supported: 2024-04-22T23:25:38.499 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-22T23:25:38.499 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: kZSTD supported: 0 2024-04-22T23:25:38.499 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: kXpressCompression supported: 0 2024-04-22T23:25:38.499 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: kLZ4HCCompression supported: 1 2024-04-22T23:25:38.499 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: kZlibCompression supported: 1 2024-04-22T23:25:38.499 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: kSnappyCompression supported: 1 2024-04-22T23:25:38.500 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: kLZ4Compression supported: 1 2024-04-22T23:25:38.500 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: kBZip2Compression supported: 0 2024-04-22T23:25:38.500 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-22T23:25:38.500 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi107/store.db/MANIFEST-000009 2024-04-22T23:25:38.500 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-04-22T23:25:38.500 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-22T23:25:38.500 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.merge_operator: 2024-04-22T23:25:38.500 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.compaction_filter: None 2024-04-22T23:25:38.500 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.compaction_filter_factory: None 2024-04-22T23:25:38.500 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.sst_partitioner_factory: None 2024-04-22T23:25:38.500 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-22T23:25:38.500 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-22T23:25:38.500 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5594af470f58) 2024-04-22T23:25:38.500 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: cache_index_and_filter_blocks: 1 2024-04-22T23:25:38.500 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-22T23:25:38.500 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-22T23:25:38.501 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: pin_top_level_index_and_filter: 1 2024-04-22T23:25:38.501 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: index_type: 0 2024-04-22T23:25:38.501 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: data_block_index_type: 0 2024-04-22T23:25:38.501 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: index_shortening: 1 2024-04-22T23:25:38.501 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-22T23:25:38.501 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: hash_index_allow_collision: 1 2024-04-22T23:25:38.501 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: checksum: 1 2024-04-22T23:25:38.501 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: no_block_cache: 0 2024-04-22T23:25:38.501 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: block_cache: 0x5594af4a3090 2024-04-22T23:25:38.501 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: block_cache_name: BinnedLRUCache 2024-04-22T23:25:38.501 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: block_cache_options: 2024-04-22T23:25:38.501 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: capacity : 536870912 2024-04-22T23:25:38.501 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: num_shard_bits : 4 2024-04-22T23:25:38.501 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: strict_capacity_limit : 0 2024-04-22T23:25:38.501 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: high_pri_pool_ratio: 0.000 2024-04-22T23:25:38.502 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: block_cache_compressed: (nil) 2024-04-22T23:25:38.502 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: persistent_cache: (nil) 2024-04-22T23:25:38.502 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: block_size: 4096 2024-04-22T23:25:38.502 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: block_size_deviation: 10 2024-04-22T23:25:38.502 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: block_restart_interval: 16 2024-04-22T23:25:38.502 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: index_block_restart_interval: 1 2024-04-22T23:25:38.502 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: metadata_block_size: 4096 2024-04-22T23:25:38.502 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: partition_filters: 0 2024-04-22T23:25:38.502 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: use_delta_encoding: 1 2024-04-22T23:25:38.502 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-04-22T23:25:38.502 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: whole_key_filtering: 1 2024-04-22T23:25:38.502 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: verify_compression: 0 2024-04-22T23:25:38.502 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: read_amp_bytes_per_bit: 0 2024-04-22T23:25:38.503 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: format_version: 4 2024-04-22T23:25:38.503 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: enable_index_compression: 1 2024-04-22T23:25:38.503 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: block_align: 0 2024-04-22T23:25:38.503 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.write_buffer_size: 33554432 2024-04-22T23:25:38.503 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.max_write_buffer_number: 2 2024-04-22T23:25:38.503 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.compression: NoCompression 2024-04-22T23:25:38.503 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.bottommost_compression: Disabled 2024-04-22T23:25:38.503 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.prefix_extractor: nullptr 2024-04-22T23:25:38.503 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-22T23:25:38.503 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.num_levels: 7 2024-04-22T23:25:38.503 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-22T23:25:38.503 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-22T23:25:38.503 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-22T23:25:38.503 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-22T23:25:38.504 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-22T23:25:38.504 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-22T23:25:38.504 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-22T23:25:38.504 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-22T23:25:38.504 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-22T23:25:38.504 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-22T23:25:38.504 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-22T23:25:38.504 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.compression_opts.level: 32767 2024-04-22T23:25:38.504 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.compression_opts.strategy: 0 2024-04-22T23:25:38.504 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-22T23:25:38.504 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-22T23:25:38.504 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-22T23:25:38.504 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.compression_opts.enabled: false 2024-04-22T23:25:38.504 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-22T23:25:38.504 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-22T23:25:38.504 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-22T23:25:38.505 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.target_file_size_base: 67108864 2024-04-22T23:25:38.505 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-22T23:25:38.505 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-22T23:25:38.505 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-22T23:25:38.505 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-22T23:25:38.505 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-22T23:25:38.505 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-22T23:25:38.505 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-22T23:25:38.505 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-22T23:25:38.505 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-22T23:25:38.505 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-22T23:25:38.505 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-22T23:25:38.505 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-22T23:25:38.505 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-22T23:25:38.505 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.arena_block_size: 4194304 2024-04-22T23:25:38.506 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-22T23:25:38.506 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-22T23:25:38.506 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-04-22T23:25:38.506 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.disable_auto_compactions: 0 2024-04-22T23:25:38.506 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-22T23:25:38.506 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-22T23:25:38.506 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-22T23:25:38.506 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-22T23:25:38.506 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-22T23:25:38.506 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-22T23:25:38.506 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-22T23:25:38.506 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-22T23:25:38.506 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-22T23:25:38.506 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-22T23:25:38.507 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.table_properties_collectors: 2024-04-22T23:25:38.507 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.inplace_update_support: 0 2024-04-22T23:25:38.507 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-22T23:25:38.507 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-22T23:25:38.507 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-22T23:25:38.507 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-22T23:25:38.507 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.bloom_locality: 0 2024-04-22T23:25:38.507 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.max_successive_merges: 0 2024-04-22T23:25:38.507 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-22T23:25:38.507 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.paranoid_file_checks: 0 2024-04-22T23:25:38.507 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.force_consistency_checks: 1 2024-04-22T23:25:38.507 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.report_bg_io_stats: 0 2024-04-22T23:25:38.507 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.ttl: 2592000 2024-04-22T23:25:38.507 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-22T23:25:38.507 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.enable_blob_files: false 2024-04-22T23:25:38.508 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.min_blob_size: 0 2024-04-22T23:25:38.508 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.blob_file_size: 268435456 2024-04-22T23:25:38.508 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-22T23:25:38.508 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-22T23:25:38.508 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-22T23:25:38.508 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi107/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 11, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-22T23:25:38.508 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 5 2024-04-22T23:25:38.508 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: [db/version_set.cc:4082] Creating manifest 13 2024-04-22T23:25:38.508 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713828338031091, "job": 1, "event": "recovery_started", "wal_files": [10]} 2024-04-22T23:25:38.508 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #10 mode 2 2024-04-22T23:25:38.508 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-04-22T23:25:38.508 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713828338032316, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 14, "file_size": 93398, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 91510, "index_size": 248, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13576, "raw_average_key_size": 49, "raw_value_size": 84074, "raw_average_value_size": 307, "num_data_blocks": 11, "num_entries": 273, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1713828338, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "e7f29ee8-a57f-4b35-a184-082fab568ff3", "db_session_id": "53ZE2LRF3349PO4XZHEP"}} 2024-04-22T23:25:38.508 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: [db/version_set.cc:4082] Creating manifest 15 2024-04-22T23:25:38.508 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713828338032815, "job": 1, "event": "recovery_finished"} 2024-04-22T23:25:38.508 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-smithi107/store.db/000010.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-22T23:25:38.509 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x5594af437180 2024-04-22T23:25:38.509 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: DB pointer 0x5594af50c000 2024-04-22T23:25:38.509 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-04-22T23:25:38.509 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-04-22T23:25:38.509 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: ** DB Stats ** 2024-04-22T23:25:38.509 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-22T23:25:38.509 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-22T23:25:38.509 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-22T23:25:38.509 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-22T23:25:38.509 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-22T23:25:38.509 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-04-22T23:25:38.509 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-22T23:25:38.509 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: 2024-04-22T23:25:38.509 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: ** Compaction Stats [default] ** 2024-04-22T23:25:38.509 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-04-22T23:25:38.509 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-22T23:25:38.510 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: L0 2/0 93.01 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 108.2 0.00 0.00 1 0.001 0 0 2024-04-22T23:25:38.510 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Sum 2/0 93.01 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 108.2 0.00 0.00 1 0.001 0 0 2024-04-22T23:25:38.510 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 108.2 0.00 0.00 1 0.001 0 0 2024-04-22T23:25:38.510 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: 2024-04-22T23:25:38.510 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: ** Compaction Stats [default] ** 2024-04-22T23:25:38.510 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-04-22T23:25:38.510 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-22T23:25:38.510 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 108.2 0.00 0.00 1 0.001 0 0 2024-04-22T23:25:38.510 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-22T23:25:38.510 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-22T23:25:38.510 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-22T23:25:38.510 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-22T23:25:38.510 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-22T23:25:38.510 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-22T23:25:38.511 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Cumulative compaction: 0.00 GB write, 20.46 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-22T23:25:38.511 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Interval compaction: 0.00 GB write, 20.46 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-22T23:25:38.511 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-22T23:25:38.511 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: 2024-04-22T23:25:38.511 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-22T23:25:38.511 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: 2024-04-22T23:25:38.511 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: ** Compaction Stats [default] ** 2024-04-22T23:25:38.511 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-04-22T23:25:38.511 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-22T23:25:38.511 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: L0 2/0 93.01 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 108.2 0.00 0.00 1 0.001 0 0 2024-04-22T23:25:38.511 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Sum 2/0 93.01 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 108.2 0.00 0.00 1 0.001 0 0 2024-04-22T23:25:38.511 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-04-22T23:25:38.511 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: 2024-04-22T23:25:38.511 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: ** Compaction Stats [default] ** 2024-04-22T23:25:38.511 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-04-22T23:25:38.512 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-22T23:25:38.512 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 108.2 0.00 0.00 1 0.001 0 0 2024-04-22T23:25:38.512 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-22T23:25:38.512 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-22T23:25:38.512 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-22T23:25:38.512 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-22T23:25:38.512 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-22T23:25:38.512 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-22T23:25:38.512 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Cumulative compaction: 0.00 GB write, 20.32 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-22T23:25:38.512 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-22T23:25:38.512 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-22T23:25:38.512 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: 2024-04-22T23:25:38.512 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-22T23:25:38.512 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: starting mon.smithi107 rank 0 at public addrs [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] at bind addrs [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi107 fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 2024-04-22T23:25:38.513 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: mon.smithi107@-1(???) e1 preinit fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 2024-04-22T23:25:38.513 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: mon.smithi107@-1(???).mds e1 new map 2024-04-22T23:25:38.513 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: mon.smithi107@-1(???).mds e1 print_map 2024-04-22T23:25:38.513 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: e1 2024-04-22T23:25:38.513 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T23:25:38.513 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:25:38.513 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: legacy client fscid: -1 2024-04-22T23:25:38.513 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: 2024-04-22T23:25:38.513 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: No filesystems configured 2024-04-22T23:25:38.513 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: mon.smithi107@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-22T23:25:38.513 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: mon.smithi107@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-22T23:25:38.513 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: mon.smithi107@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-22T23:25:38.513 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: mon.smithi107@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-22T23:25:38.513 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: mon.smithi107@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-22T23:25:38.514 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: mon.smithi107 is new leader, mons smithi107 in quorum (ranks 0) 2024-04-22T23:25:38.514 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: monmap e1: 1 mons at {smithi107=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-22T23:25:38.514 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: fsmap 2024-04-22T23:25:38.514 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: osdmap e1: 0 total, 0 up, 0 in 2024-04-22T23:25:38.514 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 ceph-mon[26355]: mgrmap e1: no daemons active 2024-04-22T23:25:38.811 INFO:teuthology.orchestra.run.smithi107.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-22T23:25:39.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 bash[26336]: 8314962fa1b933e18567c31af8eaa108e6792030a367debbd28d139d120a8450 2024-04-22T23:25:39.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:38 smithi107 systemd[1]: Started Ceph mon.smithi107 for 8252f6f2-00ff-11ef-bc93-c7b262605968. 2024-04-22T23:25:40.145 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:39.489+0000 7fc502ea6700 1 Processor -- start 2024-04-22T23:25:40.145 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:39.489+0000 7fc502ea6700 1 -- start start 2024-04-22T23:25:40.146 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:39.489+0000 7fc502ea6700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc4fc102960 0x7fc4fc102d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:25:40.146 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:39.489+0000 7fc502ea6700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4fc1032b0 con 0x7fc4fc102960 2024-04-22T23:25:40.146 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:39.489+0000 7fc500c42700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc4fc102960 0x7fc4fc102d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:25:40.146 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:39.490+0000 7fc500c42700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc4fc102960 0x7fc4fc102d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33800/0 (socket says 172.21.15.107:33800) 2024-04-22T23:25:40.146 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:39.490+0000 7fc500c42700 1 -- 172.21.15.107:0/2417083410 learned_addr learned my addr 172.21.15.107:0/2417083410 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:25:40.146 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:39.490+0000 7fc500c42700 1 -- 172.21.15.107:0/2417083410 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4fc1033f0 con 0x7fc4fc102960 2024-04-22T23:25:40.146 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:39.490+0000 7fc500c42700 1 --2- 172.21.15.107:0/2417083410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc4fc102960 0x7fc4fc102d70 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc4f00098e0 tx=0x7fc4f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c58a44275adb0363 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:25:40.146 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:39.490+0000 7fc4fb7fe700 1 -- 172.21.15.107:0/2417083410 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc4f000b860 con 0x7fc4fc102960 2024-04-22T23:25:40.146 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:39.490+0000 7fc4fb7fe700 1 -- 172.21.15.107:0/2417083410 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fc4f000be50 con 0x7fc4fc102960 2024-04-22T23:25:40.146 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:39.490+0000 7fc4fb7fe700 1 -- 172.21.15.107:0/2417083410 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc4f0010380 con 0x7fc4fc102960 2024-04-22T23:25:40.146 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:39.491+0000 7fc502ea6700 1 -- 172.21.15.107:0/2417083410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc4fc102960 msgr2=0x7fc4fc102d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:25:40.146 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:39.491+0000 7fc502ea6700 1 --2- 172.21.15.107:0/2417083410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc4fc102960 0x7fc4fc102d70 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc4f00098e0 tx=0x7fc4f0009250 comp rx=0 tx=0).stop 2024-04-22T23:25:40.147 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:39.491+0000 7fc502ea6700 1 -- 172.21.15.107:0/2417083410 shutdown_connections 2024-04-22T23:25:40.147 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:39.491+0000 7fc502ea6700 1 --2- 172.21.15.107:0/2417083410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc4fc102960 0x7fc4fc102d70 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:25:40.147 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:39.491+0000 7fc502ea6700 1 -- 172.21.15.107:0/2417083410 >> 172.21.15.107:0/2417083410 conn(0x7fc4fc0fe0e0 msgr2=0x7fc4fc1004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:25:40.147 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:39.491+0000 7fc502ea6700 1 -- 172.21.15.107:0/2417083410 shutdown_connections 2024-04-22T23:25:40.147 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:39.491+0000 7fc502ea6700 1 -- 172.21.15.107:0/2417083410 wait complete. 2024-04-22T23:25:40.147 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:39.491+0000 7fc502ea6700 1 Processor -- start 2024-04-22T23:25:40.147 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:39.491+0000 7fc502ea6700 1 -- start start 2024-04-22T23:25:40.147 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:39.492+0000 7fc502ea6700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc4fc102960 0x7fc4fc1116f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:25:40.147 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:39.492+0000 7fc502ea6700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4fc111c30 con 0x7fc4fc102960 2024-04-22T23:25:40.147 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:39.492+0000 7fc500c42700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc4fc102960 0x7fc4fc1116f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:25:40.147 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:39.492+0000 7fc500c42700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc4fc102960 0x7fc4fc1116f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33804/0 (socket says 172.21.15.107:33804) 2024-04-22T23:25:40.147 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:39.492+0000 7fc500c42700 1 -- 172.21.15.107:0/3260961718 learned_addr learned my addr 172.21.15.107:0/3260961718 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:25:40.147 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:39.492+0000 7fc500c42700 1 -- 172.21.15.107:0/3260961718 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4f00094f0 con 0x7fc4fc102960 2024-04-22T23:25:40.147 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:39.492+0000 7fc500c42700 1 --2- 172.21.15.107:0/3260961718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc4fc102960 0x7fc4fc1116f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc4f000b440 tx=0x7fc4f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:25:40.147 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:39.492+0000 7fc4f97fa700 1 -- 172.21.15.107:0/3260961718 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc4f0010760 con 0x7fc4fc102960 2024-04-22T23:25:40.147 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:39.492+0000 7fc4f97fa700 1 -- 172.21.15.107:0/3260961718 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fc4f0010d50 con 0x7fc4fc102960 2024-04-22T23:25:40.148 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:39.492+0000 7fc502ea6700 1 -- 172.21.15.107:0/3260961718 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4fc111e30 con 0x7fc4fc102960 2024-04-22T23:25:40.148 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:39.492+0000 7fc4f97fa700 1 -- 172.21.15.107:0/3260961718 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc4f001ae20 con 0x7fc4fc102960 2024-04-22T23:25:40.148 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:39.492+0000 7fc502ea6700 1 -- 172.21.15.107:0/3260961718 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4fc1129a0 con 0x7fc4fc102960 2024-04-22T23:25:40.148 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:39.493+0000 7fc4f97fa700 1 -- 172.21.15.107:0/3260961718 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fc4f002e440 con 0x7fc4fc102960 2024-04-22T23:25:40.148 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:39.493+0000 7fc4f97fa700 1 -- 172.21.15.107:0/3260961718 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fc4f0024860 con 0x7fc4fc102960 2024-04-22T23:25:40.148 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:39.493+0000 7fc502ea6700 1 -- 172.21.15.107:0/3260961718 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4fc05f5f0 con 0x7fc4fc102960 2024-04-22T23:25:40.148 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:39.496+0000 7fc4f97fa700 1 -- 172.21.15.107:0/3260961718 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7fc4f0029070 con 0x7fc4fc102960 2024-04-22T23:25:40.148 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:39.561+0000 7fc502ea6700 1 -- 172.21.15.107:0/3260961718 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7fc4fc104690 con 0x7fc4fc102960 2024-04-22T23:25:40.148 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:39.563+0000 7fc4f97fa700 1 -- 172.21.15.107:0/3260961718 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7fc4f001a6a0 con 0x7fc4fc102960 2024-04-22T23:25:40.148 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:39.564+0000 7fc502ea6700 1 -- 172.21.15.107:0/3260961718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc4fc102960 msgr2=0x7fc4fc1116f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:25:40.148 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:39.564+0000 7fc502ea6700 1 --2- 172.21.15.107:0/3260961718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc4fc102960 0x7fc4fc1116f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc4f000b440 tx=0x7fc4f0009250 comp rx=0 tx=0).stop 2024-04-22T23:25:40.148 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:39.564+0000 7fc502ea6700 1 -- 172.21.15.107:0/3260961718 shutdown_connections 2024-04-22T23:25:40.148 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:39.564+0000 7fc502ea6700 1 --2- 172.21.15.107:0/3260961718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc4fc102960 0x7fc4fc1116f0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:25:40.148 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:39.564+0000 7fc502ea6700 1 -- 172.21.15.107:0/3260961718 >> 172.21.15.107:0/3260961718 conn(0x7fc4fc0fe0e0 msgr2=0x7fc4fc0fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:25:40.148 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:39.564+0000 7fc502ea6700 1 -- 172.21.15.107:0/3260961718 shutdown_connections 2024-04-22T23:25:40.148 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:39.564+0000 7fc502ea6700 1 -- 172.21.15.107:0/3260961718 wait complete. 2024-04-22T23:25:40.149 INFO:teuthology.orchestra.run.smithi107.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-22T23:25:40.149 INFO:teuthology.orchestra.run.smithi107.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-22T23:25:40.149 INFO:teuthology.orchestra.run.smithi107.stdout:Creating mgr... 2024-04-22T23:25:40.149 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying port 9283 ... 2024-04-22T23:25:40.313 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-8252f6f2-00ff-11ef-bc93-c7b262605968@mgr.smithi107.uipxvm 2024-04-22T23:25:40.313 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Failed to reset failed state of unit ceph-8252f6f2-00ff-11ef-bc93-c7b262605968@mgr.smithi107.uipxvm.service: Unit ceph-8252f6f2-00ff-11ef-bc93-c7b262605968@mgr.smithi107.uipxvm.service not loaded. 2024-04-22T23:25:40.468 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-8252f6f2-00ff-11ef-bc93-c7b262605968.target.wants/ceph-8252f6f2-00ff-11ef-bc93-c7b262605968@mgr.smithi107.uipxvm.service → /etc/systemd/system/ceph-8252f6f2-00ff-11ef-bc93-c7b262605968@.service. 2024-04-22T23:25:40.740 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:40 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/3260961718' entity='client.admin' 2024-04-22T23:25:40.999 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-22T23:25:40.999 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout disabled 2024-04-22T23:25:41.007 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-22T23:25:41.008 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-04-22T23:25:41.008 INFO:teuthology.orchestra.run.smithi107.stdout:firewalld.service is not enabled 2024-04-22T23:25:41.008 INFO:teuthology.orchestra.run.smithi107.stdout:Not possible to enable service . firewalld.service is not available 2024-04-22T23:25:41.018 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-22T23:25:41.018 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout disabled 2024-04-22T23:25:41.026 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-22T23:25:41.029 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-04-22T23:25:41.029 INFO:teuthology.orchestra.run.smithi107.stdout:firewalld.service is not enabled 2024-04-22T23:25:41.030 INFO:teuthology.orchestra.run.smithi107.stdout:Not possible to open ports <[9283]>. firewalld.service is not available 2024-04-22T23:25:41.030 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mgr to start... 2024-04-22T23:25:41.030 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mgr... 2024-04-22T23:25:41.924 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:41 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/2498182552' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-22T23:25:42.188 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-04-22T23:25:42.189 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-04-22T23:25:42.189 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsid": "8252f6f2-00ff-11ef-bc93-c7b262605968", 2024-04-22T23:25:42.189 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "health": { 2024-04-22T23:25:42.189 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-22T23:25:42.189 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-22T23:25:42.189 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-22T23:25:42.189 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-04-22T23:25:42.189 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-22T23:25:42.189 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-22T23:25:42.189 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 0 2024-04-22T23:25:42.189 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-04-22T23:25:42.189 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-22T23:25:42.189 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "smithi107" 2024-04-22T23:25:42.189 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-04-22T23:25:42.190 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-04-22T23:25:42.190 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-22T23:25:42.190 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T23:25:42.190 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-04-22T23:25:42.190 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-22T23:25:42.190 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-04-22T23:25:42.190 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-22T23:25:42.190 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T23:25:42.190 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-22T23:25:42.190 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-22T23:25:42.190 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-22T23:25:42.198 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-22T23:25:42.199 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-22T23:25:42.199 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-22T23:25:42.199 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-04-22T23:25:42.199 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-22T23:25:42.199 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-22T23:25:42.199 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-22T23:25:42.199 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-22T23:25:42.199 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-22T23:25:42.199 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-22T23:25:42.199 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-22T23:25:42.199 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-22T23:25:42.199 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-22T23:25:42.199 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-04-22T23:25:42.200 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-22T23:25:42.200 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T23:25:42.200 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-22T23:25:42.200 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-22T23:25:42.200 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-04-22T23:25:42.200 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-22T23:25:42.200 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-22T23:25:42.200 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-22T23:25:42.200 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-22T23:25:42.200 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-22T23:25:42.200 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-22T23:25:42.200 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "restful" 2024-04-22T23:25:42.200 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-04-22T23:25:42.200 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-22T23:25:42.201 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-04-22T23:25:42.201 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-22T23:25:42.201 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T23:25:42.201 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modified": "2024-04-22T23:25:25.497563+0000", 2024-04-22T23:25:42.201 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-22T23:25:42.201 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-04-22T23:25:42.201 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-22T23:25:42.201 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-04-22T23:25:42.201 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:41.483+0000 7fade815f700 1 Processor -- start 2024-04-22T23:25:42.201 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:41.483+0000 7fade815f700 1 -- start start 2024-04-22T23:25:42.201 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:41.483+0000 7fade815f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fade0071300 0x7fade01054f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:25:42.201 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:41.483+0000 7fade815f700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade0071710 con 0x7fade0071300 2024-04-22T23:25:42.201 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:41.484+0000 7fade5efb700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fade0071300 0x7fade01054f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:25:42.201 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:41.484+0000 7fade5efb700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fade0071300 0x7fade01054f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37200/0 (socket says 172.21.15.107:37200) 2024-04-22T23:25:42.202 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:41.484+0000 7fade5efb700 1 -- 172.21.15.107:0/2900061121 learned_addr learned my addr 172.21.15.107:0/2900061121 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:25:42.202 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:41.484+0000 7fade5efb700 1 -- 172.21.15.107:0/2900061121 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade0105a30 con 0x7fade0071300 2024-04-22T23:25:42.202 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:41.484+0000 7fade5efb700 1 --2- 172.21.15.107:0/2900061121 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fade0071300 0x7fade01054f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fadd00098e0 tx=0x7fadd0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b9c55fe9b33cdb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:25:42.202 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:41.484+0000 7fade4ef9700 1 -- 172.21.15.107:0/2900061121 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fadd000b860 con 0x7fade0071300 2024-04-22T23:25:42.202 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:41.484+0000 7fade4ef9700 1 -- 172.21.15.107:0/2900061121 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fadd000be50 con 0x7fade0071300 2024-04-22T23:25:42.202 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:41.485+0000 7fade4ef9700 1 -- 172.21.15.107:0/2900061121 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fadd0010380 con 0x7fade0071300 2024-04-22T23:25:42.202 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:41.485+0000 7fade815f700 1 -- 172.21.15.107:0/2900061121 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fade0071300 msgr2=0x7fade01054f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:25:42.202 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:41.485+0000 7fade815f700 1 --2- 172.21.15.107:0/2900061121 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fade0071300 0x7fade01054f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fadd00098e0 tx=0x7fadd0009250 comp rx=0 tx=0).stop 2024-04-22T23:25:42.202 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:41.485+0000 7fade815f700 1 -- 172.21.15.107:0/2900061121 shutdown_connections 2024-04-22T23:25:42.202 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:41.485+0000 7fade815f700 1 --2- 172.21.15.107:0/2900061121 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fade0071300 0x7fade01054f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:25:42.202 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:41.485+0000 7fade815f700 1 -- 172.21.15.107:0/2900061121 >> 172.21.15.107:0/2900061121 conn(0x7fade00fcf80 msgr2=0x7fade00ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:25:42.202 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:41.485+0000 7fade815f700 1 -- 172.21.15.107:0/2900061121 shutdown_connections 2024-04-22T23:25:42.202 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:41.486+0000 7fade815f700 1 -- 172.21.15.107:0/2900061121 wait complete. 2024-04-22T23:25:42.202 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:41.486+0000 7fade815f700 1 Processor -- start 2024-04-22T23:25:42.203 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:41.486+0000 7fade815f700 1 -- start start 2024-04-22T23:25:42.203 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:41.486+0000 7fade815f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fade0071300 0x7fade0111750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:25:42.203 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:41.486+0000 7fade815f700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade0111c90 con 0x7fade0071300 2024-04-22T23:25:42.203 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:41.486+0000 7fade5efb700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fade0071300 0x7fade0111750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:25:42.203 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:41.486+0000 7fade5efb700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fade0071300 0x7fade0111750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37216/0 (socket says 172.21.15.107:37216) 2024-04-22T23:25:42.203 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:41.486+0000 7fade5efb700 1 -- 172.21.15.107:0/2498182552 learned_addr learned my addr 172.21.15.107:0/2498182552 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:25:42.203 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:41.487+0000 7fade5efb700 1 -- 172.21.15.107:0/2498182552 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd00094f0 con 0x7fade0071300 2024-04-22T23:25:42.203 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:41.487+0000 7fade5efb700 1 --2- 172.21.15.107:0/2498182552 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fade0071300 0x7fade0111750 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fadd0011bd0 tx=0x7fadd0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:25:42.203 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:41.487+0000 7fadd67fc700 1 -- 172.21.15.107:0/2498182552 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fadd00107c0 con 0x7fade0071300 2024-04-22T23:25:42.203 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:41.487+0000 7fadd67fc700 1 -- 172.21.15.107:0/2498182552 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fadd0010db0 con 0x7fade0071300 2024-04-22T23:25:42.203 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:41.487+0000 7fade815f700 1 -- 172.21.15.107:0/2498182552 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fade0111e90 con 0x7fade0071300 2024-04-22T23:25:42.203 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:41.487+0000 7fade815f700 1 -- 172.21.15.107:0/2498182552 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fade0112a00 con 0x7fade0071300 2024-04-22T23:25:42.203 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:41.487+0000 7fadd67fc700 1 -- 172.21.15.107:0/2498182552 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fadd001ad60 con 0x7fade0071300 2024-04-22T23:25:42.204 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:41.487+0000 7fadd67fc700 1 -- 172.21.15.107:0/2498182552 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fadd002e440 con 0x7fade0071300 2024-04-22T23:25:42.204 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:41.487+0000 7fadd67fc700 1 -- 172.21.15.107:0/2498182552 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fadd0024740 con 0x7fade0071300 2024-04-22T23:25:42.204 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:41.488+0000 7fade815f700 1 -- 172.21.15.107:0/2498182552 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadc40052e0 con 0x7fade0071300 2024-04-22T23:25:42.204 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:41.492+0000 7fadd67fc700 1 -- 172.21.15.107:0/2498182552 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7fadd001a5c0 con 0x7fade0071300 2024-04-22T23:25:42.204 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:41.563+0000 7fade815f700 1 -- 172.21.15.107:0/2498182552 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fadc4005860 con 0x7fade0071300 2024-04-22T23:25:42.204 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:41.564+0000 7fadd67fc700 1 -- 172.21.15.107:0/2498182552 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7fadd0010920 con 0x7fade0071300 2024-04-22T23:25:42.204 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:41.565+0000 7fade815f700 1 -- 172.21.15.107:0/2498182552 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fade0071300 msgr2=0x7fade0111750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:25:42.204 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:41.565+0000 7fade815f700 1 --2- 172.21.15.107:0/2498182552 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fade0071300 0x7fade0111750 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fadd0011bd0 tx=0x7fadd0009250 comp rx=0 tx=0).stop 2024-04-22T23:25:42.204 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:41.565+0000 7fade815f700 1 -- 172.21.15.107:0/2498182552 shutdown_connections 2024-04-22T23:25:42.204 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:41.565+0000 7fade815f700 1 --2- 172.21.15.107:0/2498182552 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fade0071300 0x7fade0111750 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:25:42.204 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:41.565+0000 7fade815f700 1 -- 172.21.15.107:0/2498182552 >> 172.21.15.107:0/2498182552 conn(0x7fade00fcf80 msgr2=0x7fade00fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:25:42.204 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:41.565+0000 7fade815f700 1 -- 172.21.15.107:0/2498182552 shutdown_connections 2024-04-22T23:25:42.204 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:41.565+0000 7fade815f700 1 -- 172.21.15.107:0/2498182552 wait complete. 2024-04-22T23:25:42.204 INFO:teuthology.orchestra.run.smithi107.stdout:mgr not available, waiting (1/15)... 2024-04-22T23:25:44.920 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:44 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/944295600' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-22T23:25:46.979 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-04-22T23:25:46.979 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-04-22T23:25:46.979 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsid": "8252f6f2-00ff-11ef-bc93-c7b262605968", 2024-04-22T23:25:46.979 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "health": { 2024-04-22T23:25:46.979 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-22T23:25:46.979 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-22T23:25:46.979 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-22T23:25:46.980 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-04-22T23:25:46.980 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-22T23:25:46.980 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-22T23:25:46.980 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 0 2024-04-22T23:25:46.980 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-04-22T23:25:46.980 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-22T23:25:46.980 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "smithi107" 2024-04-22T23:25:46.980 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-04-22T23:25:46.980 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-04-22T23:25:46.980 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-22T23:25:46.980 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T23:25:46.980 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-04-22T23:25:46.980 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-22T23:25:46.980 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-04-22T23:25:46.981 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-22T23:25:46.981 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T23:25:46.981 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-22T23:25:46.981 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-22T23:25:46.981 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-22T23:25:46.981 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-22T23:25:46.981 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-22T23:25:46.981 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-22T23:25:46.981 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-04-22T23:25:46.981 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-22T23:25:46.981 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-22T23:25:46.989 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-22T23:25:46.989 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-22T23:25:46.989 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-22T23:25:46.989 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-22T23:25:46.990 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-22T23:25:46.990 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-22T23:25:46.990 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-22T23:25:46.990 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-04-22T23:25:46.990 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-22T23:25:46.990 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T23:25:46.990 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-22T23:25:46.990 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-22T23:25:46.990 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-04-22T23:25:46.990 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-22T23:25:46.990 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-22T23:25:46.990 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-22T23:25:46.990 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-22T23:25:46.991 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-22T23:25:46.991 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-22T23:25:46.991 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "restful" 2024-04-22T23:25:46.991 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-04-22T23:25:46.991 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-22T23:25:46.991 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-04-22T23:25:46.991 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-22T23:25:46.991 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T23:25:46.991 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modified": "2024-04-22T23:25:25.497563+0000", 2024-04-22T23:25:46.991 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-22T23:25:46.991 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-04-22T23:25:46.991 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-22T23:25:46.991 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-04-22T23:25:46.991 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:44.704+0000 7f38d9c26700 1 Processor -- start 2024-04-22T23:25:46.991 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:44.704+0000 7f38d9c26700 1 -- start start 2024-04-22T23:25:46.992 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:44.704+0000 7f38d9c26700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38d4101880 0x7f38d4101c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:25:46.992 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:44.704+0000 7f38d9c26700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d41021d0 con 0x7f38d4101880 2024-04-22T23:25:46.992 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:44.705+0000 7f38d37fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38d4101880 0x7f38d4101c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:25:46.992 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:44.705+0000 7f38d37fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38d4101880 0x7f38d4101c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37222/0 (socket says 172.21.15.107:37222) 2024-04-22T23:25:46.992 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:44.705+0000 7f38d37fe700 1 -- 172.21.15.107:0/1796712645 learned_addr learned my addr 172.21.15.107:0/1796712645 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:25:46.992 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:44.705+0000 7f38d37fe700 1 -- 172.21.15.107:0/1796712645 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d4102310 con 0x7f38d4101880 2024-04-22T23:25:46.992 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:44.705+0000 7f38d37fe700 1 --2- 172.21.15.107:0/1796712645 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38d4101880 0x7f38d4101c90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f38c40098e0 tx=0x7f38c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef97c1c8ea0ba700 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:25:46.992 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:44.705+0000 7f38d27fc700 1 -- 172.21.15.107:0/1796712645 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f38c400b8f0 con 0x7f38d4101880 2024-04-22T23:25:46.992 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:44.705+0000 7f38d27fc700 1 -- 172.21.15.107:0/1796712645 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f38c4011420 con 0x7f38d4101880 2024-04-22T23:25:46.992 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:44.705+0000 7f38d27fc700 1 -- 172.21.15.107:0/1796712645 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f38c4010540 con 0x7f38d4101880 2024-04-22T23:25:46.992 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:44.706+0000 7f38d9c26700 1 -- 172.21.15.107:0/1796712645 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38d4101880 msgr2=0x7f38d4101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:25:46.992 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:44.706+0000 7f38d9c26700 1 --2- 172.21.15.107:0/1796712645 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38d4101880 0x7f38d4101c90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f38c40098e0 tx=0x7f38c4009250 comp rx=0 tx=0).stop 2024-04-22T23:25:46.992 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:44.706+0000 7f38d9c26700 1 -- 172.21.15.107:0/1796712645 shutdown_connections 2024-04-22T23:25:46.992 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:44.706+0000 7f38d9c26700 1 --2- 172.21.15.107:0/1796712645 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38d4101880 0x7f38d4101c90 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:25:46.993 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:44.706+0000 7f38d9c26700 1 -- 172.21.15.107:0/1796712645 >> 172.21.15.107:0/1796712645 conn(0x7f38d40fcfc0 msgr2=0x7f38d40ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:25:46.993 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:44.706+0000 7f38d9c26700 1 -- 172.21.15.107:0/1796712645 shutdown_connections 2024-04-22T23:25:46.993 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:44.706+0000 7f38d9c26700 1 -- 172.21.15.107:0/1796712645 wait complete. 2024-04-22T23:25:46.993 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:44.706+0000 7f38d9c26700 1 Processor -- start 2024-04-22T23:25:46.993 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:44.707+0000 7f38d9c26700 1 -- start start 2024-04-22T23:25:46.993 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:44.707+0000 7f38d9c26700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38d4101880 0x7f38d410f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:25:46.993 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:44.707+0000 7f38d9c26700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d410fa60 con 0x7f38d4101880 2024-04-22T23:25:46.994 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:44.707+0000 7f38d37fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38d4101880 0x7f38d410f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:25:46.994 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:44.707+0000 7f38d37fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38d4101880 0x7f38d410f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37226/0 (socket says 172.21.15.107:37226) 2024-04-22T23:25:46.994 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:44.707+0000 7f38d37fe700 1 -- 172.21.15.107:0/944295600 learned_addr learned my addr 172.21.15.107:0/944295600 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:25:46.994 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:44.707+0000 7f38d37fe700 1 -- 172.21.15.107:0/944295600 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c40094f0 con 0x7f38d4101880 2024-04-22T23:25:46.994 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:44.707+0000 7f38d37fe700 1 --2- 172.21.15.107:0/944295600 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38d4101880 0x7f38d410f520 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f38c4011ef0 tx=0x7f38c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:25:46.994 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:44.707+0000 7f38bbfff700 1 -- 172.21.15.107:0/944295600 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f38c4010910 con 0x7f38d4101880 2024-04-22T23:25:46.994 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:44.708+0000 7f38d9c26700 1 -- 172.21.15.107:0/944295600 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38d410fc60 con 0x7f38d4101880 2024-04-22T23:25:46.994 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:44.708+0000 7f38bbfff700 1 -- 172.21.15.107:0/944295600 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f38c4010f00 con 0x7f38d4101880 2024-04-22T23:25:46.994 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:44.708+0000 7f38bbfff700 1 -- 172.21.15.107:0/944295600 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f38c401ae50 con 0x7f38d4101880 2024-04-22T23:25:46.994 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:44.708+0000 7f38d9c26700 1 -- 172.21.15.107:0/944295600 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38d41107d0 con 0x7f38d4101880 2024-04-22T23:25:46.994 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:44.708+0000 7f38bbfff700 1 -- 172.21.15.107:0/944295600 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f38c4024480 con 0x7f38d4101880 2024-04-22T23:25:46.994 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:44.708+0000 7f38bbfff700 1 -- 172.21.15.107:0/944295600 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f38c402d740 con 0x7f38d4101880 2024-04-22T23:25:46.994 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:44.708+0000 7f38d9c26700 1 -- 172.21.15.107:0/944295600 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38d405f5f0 con 0x7f38d4101880 2024-04-22T23:25:46.995 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:44.711+0000 7f38bbfff700 1 -- 172.21.15.107:0/944295600 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7f38c4029070 con 0x7f38d4101880 2024-04-22T23:25:46.995 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:44.783+0000 7f38d9c26700 1 -- 172.21.15.107:0/944295600 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f38d4103570 con 0x7f38d4101880 2024-04-22T23:25:46.995 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:44.783+0000 7f38bbfff700 1 -- 172.21.15.107:0/944295600 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7f38c401a430 con 0x7f38d4101880 2024-04-22T23:25:46.995 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:44.784+0000 7f38d9c26700 1 -- 172.21.15.107:0/944295600 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38d4101880 msgr2=0x7f38d410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:25:46.995 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:44.784+0000 7f38d9c26700 1 --2- 172.21.15.107:0/944295600 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38d4101880 0x7f38d410f520 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f38c4011ef0 tx=0x7f38c4009250 comp rx=0 tx=0).stop 2024-04-22T23:25:46.995 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:44.784+0000 7f38d9c26700 1 -- 172.21.15.107:0/944295600 shutdown_connections 2024-04-22T23:25:46.995 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:44.784+0000 7f38d9c26700 1 --2- 172.21.15.107:0/944295600 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38d4101880 0x7f38d410f520 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:25:46.995 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:44.784+0000 7f38d9c26700 1 -- 172.21.15.107:0/944295600 >> 172.21.15.107:0/944295600 conn(0x7f38d40fcfc0 msgr2=0x7f38d4105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:25:46.995 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:44.784+0000 7f38d9c26700 1 -- 172.21.15.107:0/944295600 shutdown_connections 2024-04-22T23:25:46.995 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:44.784+0000 7f38d9c26700 1 -- 172.21.15.107:0/944295600 wait complete. 2024-04-22T23:25:46.995 INFO:teuthology.orchestra.run.smithi107.stdout:mgr not available, waiting (2/15)... 2024-04-22T23:25:47.490 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:47 smithi107 ceph-mon[26355]: Activating manager daemon smithi107.uipxvm 2024-04-22T23:25:47.491 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:47 smithi107 ceph-mon[26355]: mgrmap e2: smithi107.uipxvm(active, starting, since 0.00314234s) 2024-04-22T23:25:47.491 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:47 smithi107 ceph-mon[26355]: from='mgr.14100 172.21.15.107:0/2886987763' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-22T23:25:47.491 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:47 smithi107 ceph-mon[26355]: from='mgr.14100 172.21.15.107:0/2886987763' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-22T23:25:47.491 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:47 smithi107 ceph-mon[26355]: from='mgr.14100 172.21.15.107:0/2886987763' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-22T23:25:47.491 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:47 smithi107 ceph-mon[26355]: from='mgr.14100 172.21.15.107:0/2886987763' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-04-22T23:25:47.491 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:47 smithi107 ceph-mon[26355]: from='mgr.14100 172.21.15.107:0/2886987763' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mgr metadata", "who": "smithi107.uipxvm", "id": "smithi107.uipxvm"}]: dispatch 2024-04-22T23:25:47.491 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:47 smithi107 ceph-mon[26355]: Manager daemon smithi107.uipxvm is now available 2024-04-22T23:25:47.491 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:47 smithi107 ceph-mon[26355]: from='mgr.14100 172.21.15.107:0/2886987763' entity='mgr.smithi107.uipxvm' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi107.uipxvm/mirror_snapshot_schedule"}]: dispatch 2024-04-22T23:25:47.491 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:47 smithi107 ceph-mon[26355]: from='mgr.14100 172.21.15.107:0/2886987763' entity='mgr.smithi107.uipxvm' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi107.uipxvm/trash_purge_schedule"}]: dispatch 2024-04-22T23:25:47.491 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:47 smithi107 ceph-mon[26355]: from='mgr.14100 172.21.15.107:0/2886987763' entity='mgr.smithi107.uipxvm' 2024-04-22T23:25:47.491 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:47 smithi107 ceph-mon[26355]: from='mgr.14100 172.21.15.107:0/2886987763' entity='mgr.smithi107.uipxvm' 2024-04-22T23:25:47.491 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:47 smithi107 ceph-mon[26355]: from='mgr.14100 172.21.15.107:0/2886987763' entity='mgr.smithi107.uipxvm' 2024-04-22T23:25:49.710 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:49 smithi107 ceph-mon[26355]: mgrmap e3: smithi107.uipxvm(active, since 1.00682s) 2024-04-22T23:25:50.282 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-04-22T23:25:50.285 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-04-22T23:25:50.285 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsid": "8252f6f2-00ff-11ef-bc93-c7b262605968", 2024-04-22T23:25:50.285 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "health": { 2024-04-22T23:25:50.285 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-22T23:25:50.285 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-22T23:25:50.285 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-22T23:25:50.285 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-04-22T23:25:50.285 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-22T23:25:50.285 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-22T23:25:50.285 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 0 2024-04-22T23:25:50.285 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-04-22T23:25:50.285 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-22T23:25:50.285 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "smithi107" 2024-04-22T23:25:50.286 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-04-22T23:25:50.286 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-04-22T23:25:50.286 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-22T23:25:50.286 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T23:25:50.286 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-04-22T23:25:50.286 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-22T23:25:50.286 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-04-22T23:25:50.286 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-22T23:25:50.286 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T23:25:50.286 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-22T23:25:50.286 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-22T23:25:50.286 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-22T23:25:50.295 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-22T23:25:50.295 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-22T23:25:50.295 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-22T23:25:50.295 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-04-22T23:25:50.295 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-22T23:25:50.295 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-22T23:25:50.295 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-22T23:25:50.295 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-22T23:25:50.295 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-22T23:25:50.295 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-22T23:25:50.295 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-22T23:25:50.295 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-22T23:25:50.295 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-22T23:25:50.295 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-04-22T23:25:50.295 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-22T23:25:50.296 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T23:25:50.296 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-22T23:25:50.296 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-22T23:25:50.296 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-04-22T23:25:50.296 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-22T23:25:50.296 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-22T23:25:50.296 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-22T23:25:50.296 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-22T23:25:50.296 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-22T23:25:50.296 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-22T23:25:50.296 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "restful" 2024-04-22T23:25:50.296 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-04-22T23:25:50.296 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-22T23:25:50.296 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-04-22T23:25:50.296 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-22T23:25:50.296 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T23:25:50.297 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modified": "2024-04-22T23:25:25.497563+0000", 2024-04-22T23:25:50.297 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-22T23:25:50.297 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-04-22T23:25:50.297 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-22T23:25:50.297 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-04-22T23:25:50.297 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:49.436+0000 7f34d5dfb700 1 Processor -- start 2024-04-22T23:25:50.297 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:49.436+0000 7f34d5dfb700 1 -- start start 2024-04-22T23:25:50.297 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:49.437+0000 7f34d5dfb700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34d0101780 0x7f34d0101b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:25:50.297 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:49.437+0000 7f34d5dfb700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d01020d0 con 0x7f34d0101780 2024-04-22T23:25:50.297 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:49.437+0000 7f34cf7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34d0101780 0x7f34d0101b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:25:50.297 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:49.437+0000 7f34cf7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34d0101780 0x7f34d0101b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37296/0 (socket says 172.21.15.107:37296) 2024-04-22T23:25:50.297 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:49.437+0000 7f34cf7fe700 1 -- 172.21.15.107:0/1321648133 learned_addr learned my addr 172.21.15.107:0/1321648133 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:25:50.297 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:49.437+0000 7f34cf7fe700 1 -- 172.21.15.107:0/1321648133 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34d0102210 con 0x7f34d0101780 2024-04-22T23:25:50.298 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:49.437+0000 7f34cf7fe700 1 --2- 172.21.15.107:0/1321648133 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34d0101780 0x7f34d0101b90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f34c00098e0 tx=0x7f34c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3669271d0969e07e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:25:50.298 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:49.438+0000 7f34ceffd700 1 -- 172.21.15.107:0/1321648133 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f34c000b860 con 0x7f34d0101780 2024-04-22T23:25:50.298 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:49.438+0000 7f34ceffd700 1 -- 172.21.15.107:0/1321648133 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f34c000be50 con 0x7f34d0101780 2024-04-22T23:25:50.298 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:49.438+0000 7f34ceffd700 1 -- 172.21.15.107:0/1321648133 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f34c0010380 con 0x7f34d0101780 2024-04-22T23:25:50.298 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:49.438+0000 7f34d5dfb700 1 -- 172.21.15.107:0/1321648133 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34d0101780 msgr2=0x7f34d0101b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:25:50.298 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:49.438+0000 7f34d5dfb700 1 --2- 172.21.15.107:0/1321648133 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34d0101780 0x7f34d0101b90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f34c00098e0 tx=0x7f34c0009250 comp rx=0 tx=0).stop 2024-04-22T23:25:50.298 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:49.438+0000 7f34d5dfb700 1 -- 172.21.15.107:0/1321648133 shutdown_connections 2024-04-22T23:25:50.298 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:49.438+0000 7f34d5dfb700 1 --2- 172.21.15.107:0/1321648133 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34d0101780 0x7f34d0101b90 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:25:50.298 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:49.438+0000 7f34d5dfb700 1 -- 172.21.15.107:0/1321648133 >> 172.21.15.107:0/1321648133 conn(0x7f34d00fcf00 msgr2=0x7f34d00ff310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:25:50.298 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:49.438+0000 7f34d5dfb700 1 -- 172.21.15.107:0/1321648133 shutdown_connections 2024-04-22T23:25:50.298 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:49.438+0000 7f34d5dfb700 1 -- 172.21.15.107:0/1321648133 wait complete. 2024-04-22T23:25:50.298 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:49.439+0000 7f34d5dfb700 1 Processor -- start 2024-04-22T23:25:50.298 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:49.439+0000 7f34d5dfb700 1 -- start start 2024-04-22T23:25:50.298 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:49.439+0000 7f34d5dfb700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34d0101780 0x7f34d010f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:25:50.299 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:49.439+0000 7f34d5dfb700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d010f980 con 0x7f34d0101780 2024-04-22T23:25:50.299 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:49.439+0000 7f34cf7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34d0101780 0x7f34d010f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:25:50.299 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:49.439+0000 7f34cf7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34d0101780 0x7f34d010f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37300/0 (socket says 172.21.15.107:37300) 2024-04-22T23:25:50.299 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:49.439+0000 7f34cf7fe700 1 -- 172.21.15.107:0/3978682650 learned_addr learned my addr 172.21.15.107:0/3978682650 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:25:50.299 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:49.440+0000 7f34cf7fe700 1 -- 172.21.15.107:0/3978682650 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34c00094f0 con 0x7f34d0101780 2024-04-22T23:25:50.299 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:49.440+0000 7f34cf7fe700 1 --2- 172.21.15.107:0/3978682650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34d0101780 0x7f34d010f440 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f34c0000c00 tx=0x7f34c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:25:50.299 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:49.440+0000 7f34ccff9700 1 -- 172.21.15.107:0/3978682650 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f34c00107c0 con 0x7f34d0101780 2024-04-22T23:25:50.299 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:49.440+0000 7f34ccff9700 1 -- 172.21.15.107:0/3978682650 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f34c0010db0 con 0x7f34d0101780 2024-04-22T23:25:50.299 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:49.440+0000 7f34d5dfb700 1 -- 172.21.15.107:0/3978682650 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34d010fb80 con 0x7f34d0101780 2024-04-22T23:25:50.299 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:49.440+0000 7f34ccff9700 1 -- 172.21.15.107:0/3978682650 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f34c001ad00 con 0x7f34d0101780 2024-04-22T23:25:50.299 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:49.440+0000 7f34d5dfb700 1 -- 172.21.15.107:0/3978682650 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34d01106f0 con 0x7f34d0101780 2024-04-22T23:25:50.299 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:49.441+0000 7f34ccff9700 1 -- 172.21.15.107:0/3978682650 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 4) v1 ==== 43365+0+0 (secure 0 0 0) 0x7f34c00193c0 con 0x7f34d0101780 2024-04-22T23:25:50.299 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:49.441+0000 7f34d5dfb700 1 -- 172.21.15.107:0/3978682650 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34d005f5f0 con 0x7f34d0101780 2024-04-22T23:25:50.299 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:49.441+0000 7f34ccff9700 1 --2- 172.21.15.107:0/3978682650 >> [v2:172.21.15.107:6800/467262774,v1:172.21.15.107:6801/467262774] conn(0x7f34bc0389c0 0x7f34bc03ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:25:50.299 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:49.441+0000 7f34ccff9700 1 -- 172.21.15.107:0/3978682650 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f34c004a590 con 0x7f34d0101780 2024-04-22T23:25:50.300 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:49.441+0000 7f34c6ffd700 1 --2- 172.21.15.107:0/3978682650 >> [v2:172.21.15.107:6800/467262774,v1:172.21.15.107:6801/467262774] conn(0x7f34bc0389c0 0x7f34bc03ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:25:50.300 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:49.441+0000 7f34c6ffd700 1 --2- 172.21.15.107:0/3978682650 >> [v2:172.21.15.107:6800/467262774,v1:172.21.15.107:6801/467262774] conn(0x7f34bc0389c0 0x7f34bc03ae70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f34b8009990 tx=0x7f34b8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:25:50.300 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:49.446+0000 7f34ccff9700 1 -- 172.21.15.107:0/3978682650 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f34c0022930 con 0x7f34d0101780 2024-04-22T23:25:50.300 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:49.709+0000 7f34d5dfb700 1 -- 172.21.15.107:0/3978682650 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f34d004bed0 con 0x7f34d0101780 2024-04-22T23:25:50.300 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:49.709+0000 7f34ccff9700 1 -- 172.21.15.107:0/3978682650 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7f34c0022340 con 0x7f34d0101780 2024-04-22T23:25:50.300 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:49.711+0000 7f34d5dfb700 1 -- 172.21.15.107:0/3978682650 >> [v2:172.21.15.107:6800/467262774,v1:172.21.15.107:6801/467262774] conn(0x7f34bc0389c0 msgr2=0x7f34bc03ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:25:50.300 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:49.711+0000 7f34d5dfb700 1 --2- 172.21.15.107:0/3978682650 >> [v2:172.21.15.107:6800/467262774,v1:172.21.15.107:6801/467262774] conn(0x7f34bc0389c0 0x7f34bc03ae70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f34b8009990 tx=0x7f34b8006e40 comp rx=0 tx=0).stop 2024-04-22T23:25:50.300 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:49.711+0000 7f34d5dfb700 1 -- 172.21.15.107:0/3978682650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34d0101780 msgr2=0x7f34d010f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:25:50.300 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:49.711+0000 7f34d5dfb700 1 --2- 172.21.15.107:0/3978682650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34d0101780 0x7f34d010f440 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f34c0000c00 tx=0x7f34c0009250 comp rx=0 tx=0).stop 2024-04-22T23:25:50.300 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:49.711+0000 7f34d5dfb700 1 -- 172.21.15.107:0/3978682650 shutdown_connections 2024-04-22T23:25:50.300 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:49.711+0000 7f34d5dfb700 1 --2- 172.21.15.107:0/3978682650 >> [v2:172.21.15.107:6800/467262774,v1:172.21.15.107:6801/467262774] conn(0x7f34bc0389c0 0x7f34bc03ae70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:25:50.300 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:49.711+0000 7f34d5dfb700 1 --2- 172.21.15.107:0/3978682650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34d0101780 0x7f34d010f440 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:25:50.300 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:49.711+0000 7f34d5dfb700 1 -- 172.21.15.107:0/3978682650 >> 172.21.15.107:0/3978682650 conn(0x7f34d00fcf00 msgr2=0x7f34d0105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:25:50.300 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:49.711+0000 7f34d5dfb700 1 -- 172.21.15.107:0/3978682650 shutdown_connections 2024-04-22T23:25:50.301 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:49.711+0000 7f34d5dfb700 1 -- 172.21.15.107:0/3978682650 wait complete. 2024-04-22T23:25:50.301 INFO:teuthology.orchestra.run.smithi107.stdout:mgr is available 2024-04-22T23:25:50.666 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:50 smithi107 ceph-mon[26355]: mgrmap e4: smithi107.uipxvm(active, since 2s) 2024-04-22T23:25:50.666 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:50 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/3978682650' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-22T23:25:51.490 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:51 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/2813398447' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-22T23:25:51.512 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-04-22T23:25:51.512 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [global] 2024-04-22T23:25:51.512 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout fsid = 8252f6f2-00ff-11ef-bc93-c7b262605968 2024-04-22T23:25:51.512 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-22T23:25:51.512 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-22T23:25:51.512 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-22T23:25:51.512 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-22T23:25:51.512 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-04-22T23:25:51.512 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-22T23:25:51.512 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-22T23:25:51.512 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-04-22T23:25:51.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [osd] 2024-04-22T23:25:51.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-22T23:25:51.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-22T23:25:51.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-22T23:25:51.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-22T23:25:51.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-22T23:25:51.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:50.686+0000 7fd108dab700 1 Processor -- start 2024-04-22T23:25:51.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:50.686+0000 7fd108dab700 1 -- start start 2024-04-22T23:25:51.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:50.686+0000 7fd108dab700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd104101820 0x7fd104101c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:25:51.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:50.686+0000 7fd108dab700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd104102170 con 0x7fd104101820 2024-04-22T23:25:51.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:50.686+0000 7fd10259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd104101820 0x7fd104101c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:25:51.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:50.686+0000 7fd10259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd104101820 0x7fd104101c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37304/0 (socket says 172.21.15.107:37304) 2024-04-22T23:25:51.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:50.686+0000 7fd10259c700 1 -- 172.21.15.107:0/3757542763 learned_addr learned my addr 172.21.15.107:0/3757542763 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:25:51.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:50.686+0000 7fd10259c700 1 -- 172.21.15.107:0/3757542763 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1041022b0 con 0x7fd104101820 2024-04-22T23:25:51.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:50.687+0000 7fd10259c700 1 --2- 172.21.15.107:0/3757542763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd104101820 0x7fd104101c30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd0f40098e0 tx=0x7fd0f4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87493edc9659578d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:25:51.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:50.687+0000 7fd10159a700 1 -- 172.21.15.107:0/3757542763 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd0f400b860 con 0x7fd104101820 2024-04-22T23:25:51.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:50.687+0000 7fd10159a700 1 -- 172.21.15.107:0/3757542763 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fd0f400be50 con 0x7fd104101820 2024-04-22T23:25:51.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:50.687+0000 7fd10159a700 1 -- 172.21.15.107:0/3757542763 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd0f4010380 con 0x7fd104101820 2024-04-22T23:25:51.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:50.687+0000 7fd108dab700 1 -- 172.21.15.107:0/3757542763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd104101820 msgr2=0x7fd104101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:25:51.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:50.687+0000 7fd108dab700 1 --2- 172.21.15.107:0/3757542763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd104101820 0x7fd104101c30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd0f40098e0 tx=0x7fd0f4009250 comp rx=0 tx=0).stop 2024-04-22T23:25:51.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:50.687+0000 7fd108dab700 1 -- 172.21.15.107:0/3757542763 shutdown_connections 2024-04-22T23:25:51.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:50.687+0000 7fd108dab700 1 --2- 172.21.15.107:0/3757542763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd104101820 0x7fd104101c30 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:25:51.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:50.687+0000 7fd108dab700 1 -- 172.21.15.107:0/3757542763 >> 172.21.15.107:0/3757542763 conn(0x7fd1040fcfe0 msgr2=0x7fd1040ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:25:51.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:50.688+0000 7fd108dab700 1 -- 172.21.15.107:0/3757542763 shutdown_connections 2024-04-22T23:25:51.515 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:50.688+0000 7fd108dab700 1 -- 172.21.15.107:0/3757542763 wait complete. 2024-04-22T23:25:51.515 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:50.688+0000 7fd108dab700 1 Processor -- start 2024-04-22T23:25:51.515 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:50.688+0000 7fd108dab700 1 -- start start 2024-04-22T23:25:51.515 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:50.688+0000 7fd108dab700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd104101820 0x7fd104076b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:25:51.515 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:50.688+0000 7fd108dab700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1041143c0 con 0x7fd104101820 2024-04-22T23:25:51.515 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:50.688+0000 7fd10259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd104101820 0x7fd104076b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:25:51.515 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:50.688+0000 7fd10259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd104101820 0x7fd104076b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37306/0 (socket says 172.21.15.107:37306) 2024-04-22T23:25:51.515 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:50.688+0000 7fd10259c700 1 -- 172.21.15.107:0/2813398447 learned_addr learned my addr 172.21.15.107:0/2813398447 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:25:51.515 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:50.688+0000 7fd10259c700 1 -- 172.21.15.107:0/2813398447 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f40094f0 con 0x7fd104101820 2024-04-22T23:25:51.515 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:50.689+0000 7fd10259c700 1 --2- 172.21.15.107:0/2813398447 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd104101820 0x7fd104076b90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4006b00 tx=0x7fd0f4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:25:51.515 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:50.689+0000 7fd0f2ffd700 1 -- 172.21.15.107:0/2813398447 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd0f4010760 con 0x7fd104101820 2024-04-22T23:25:51.515 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:50.689+0000 7fd0f2ffd700 1 -- 172.21.15.107:0/2813398447 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fd0f4010d50 con 0x7fd104101820 2024-04-22T23:25:51.515 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:50.689+0000 7fd108dab700 1 -- 172.21.15.107:0/2813398447 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd104114500 con 0x7fd104101820 2024-04-22T23:25:51.515 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:50.689+0000 7fd0f2ffd700 1 -- 172.21.15.107:0/2813398447 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd0f401ad40 con 0x7fd104101820 2024-04-22T23:25:51.516 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:50.689+0000 7fd108dab700 1 -- 172.21.15.107:0/2813398447 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1041149f0 con 0x7fd104101820 2024-04-22T23:25:51.516 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:50.690+0000 7fd0f2ffd700 1 -- 172.21.15.107:0/2813398447 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 4) v1 ==== 43365+0+0 (secure 0 0 0) 0x7fd0f402e440 con 0x7fd104101820 2024-04-22T23:25:51.516 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:50.690+0000 7fd0f2ffd700 1 --2- 172.21.15.107:0/2813398447 >> [v2:172.21.15.107:6800/467262774,v1:172.21.15.107:6801/467262774] conn(0x7fd0ec036790 0x7fd0ec038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:25:51.516 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:50.690+0000 7fd0f2ffd700 1 -- 172.21.15.107:0/2813398447 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fd0f404a630 con 0x7fd104101820 2024-04-22T23:25:51.516 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:50.690+0000 7fd101d9b700 1 --2- 172.21.15.107:0/2813398447 >> [v2:172.21.15.107:6800/467262774,v1:172.21.15.107:6801/467262774] conn(0x7fd0ec036790 0x7fd0ec038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:25:51.516 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:50.690+0000 7fd108dab700 1 -- 172.21.15.107:0/2813398447 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd10404bed0 con 0x7fd104101820 2024-04-22T23:25:51.516 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:50.690+0000 7fd101d9b700 1 --2- 172.21.15.107:0/2813398447 >> [v2:172.21.15.107:6800/467262774,v1:172.21.15.107:6801/467262774] conn(0x7fd0ec036790 0x7fd0ec038c40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8009990 tx=0x7fd0f8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:25:51.516 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:50.695+0000 7fd0f2ffd700 1 -- 172.21.15.107:0/2813398447 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd0f4010aa0 con 0x7fd104101820 2024-04-22T23:25:51.516 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:50.879+0000 7fd108dab700 1 -- 172.21.15.107:0/2813398447 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fd1041035a0 con 0x7fd104101820 2024-04-22T23:25:51.516 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:50.880+0000 7fd0f2ffd700 1 -- 172.21.15.107:0/2813398447 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7fd0f40108c0 con 0x7fd104101820 2024-04-22T23:25:51.516 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:50.882+0000 7fd108dab700 1 -- 172.21.15.107:0/2813398447 >> [v2:172.21.15.107:6800/467262774,v1:172.21.15.107:6801/467262774] conn(0x7fd0ec036790 msgr2=0x7fd0ec038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:25:51.516 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:50.882+0000 7fd108dab700 1 --2- 172.21.15.107:0/2813398447 >> [v2:172.21.15.107:6800/467262774,v1:172.21.15.107:6801/467262774] conn(0x7fd0ec036790 0x7fd0ec038c40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8009990 tx=0x7fd0f8006e40 comp rx=0 tx=0).stop 2024-04-22T23:25:51.516 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:50.882+0000 7fd108dab700 1 -- 172.21.15.107:0/2813398447 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd104101820 msgr2=0x7fd104076b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:25:51.516 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:50.882+0000 7fd108dab700 1 --2- 172.21.15.107:0/2813398447 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd104101820 0x7fd104076b90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4006b00 tx=0x7fd0f4009250 comp rx=0 tx=0).stop 2024-04-22T23:25:51.517 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:50.882+0000 7fd108dab700 1 -- 172.21.15.107:0/2813398447 shutdown_connections 2024-04-22T23:25:51.517 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:50.882+0000 7fd108dab700 1 --2- 172.21.15.107:0/2813398447 >> [v2:172.21.15.107:6800/467262774,v1:172.21.15.107:6801/467262774] conn(0x7fd0ec036790 0x7fd0ec038c40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:25:51.517 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:50.882+0000 7fd108dab700 1 --2- 172.21.15.107:0/2813398447 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd104101820 0x7fd104076b90 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:25:51.517 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:50.882+0000 7fd108dab700 1 -- 172.21.15.107:0/2813398447 >> 172.21.15.107:0/2813398447 conn(0x7fd1040fcfe0 msgr2=0x7fd1040fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:25:51.517 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:50.882+0000 7fd108dab700 1 -- 172.21.15.107:0/2813398447 shutdown_connections 2024-04-22T23:25:51.517 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:50.882+0000 7fd108dab700 1 -- 172.21.15.107:0/2813398447 wait complete. 2024-04-22T23:25:51.517 INFO:teuthology.orchestra.run.smithi107.stdout:Enabling cephadm module... 2024-04-22T23:25:52.422 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:52 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/366192975' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-22T23:25:53.017 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:51.918+0000 7f02aa222700 1 Processor -- start 2024-04-22T23:25:53.018 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:51.918+0000 7f02aa222700 1 -- start start 2024-04-22T23:25:53.018 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:51.918+0000 7f02aa222700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02a4101820 0x7f02a4101c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:25:53.018 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:51.918+0000 7f02aa222700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a4102170 con 0x7f02a4101820 2024-04-22T23:25:53.018 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:51.918+0000 7f02a37fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02a4101820 0x7f02a4101c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:25:53.018 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:51.918+0000 7f02a37fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02a4101820 0x7f02a4101c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36304/0 (socket says 172.21.15.107:36304) 2024-04-22T23:25:53.018 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:51.918+0000 7f02a37fe700 1 -- 172.21.15.107:0/118890757 learned_addr learned my addr 172.21.15.107:0/118890757 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:25:53.018 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:51.918+0000 7f02a37fe700 1 -- 172.21.15.107:0/118890757 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02a41022b0 con 0x7f02a4101820 2024-04-22T23:25:53.018 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:51.919+0000 7f02a37fe700 1 --2- 172.21.15.107:0/118890757 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02a4101820 0x7f02a4101c30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f02940098e0 tx=0x7f0294009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d569e0245e50e373 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:25:53.018 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:51.919+0000 7f02a27fc700 1 -- 172.21.15.107:0/118890757 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f029400b860 con 0x7f02a4101820 2024-04-22T23:25:53.018 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:51.919+0000 7f02a27fc700 1 -- 172.21.15.107:0/118890757 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f029400be50 con 0x7f02a4101820 2024-04-22T23:25:53.018 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:51.919+0000 7f02a27fc700 1 -- 172.21.15.107:0/118890757 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0294010380 con 0x7f02a4101820 2024-04-22T23:25:53.018 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:51.919+0000 7f02aa222700 1 -- 172.21.15.107:0/118890757 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02a4101820 msgr2=0x7f02a4101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:25:53.018 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:51.919+0000 7f02aa222700 1 --2- 172.21.15.107:0/118890757 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02a4101820 0x7f02a4101c30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f02940098e0 tx=0x7f0294009250 comp rx=0 tx=0).stop 2024-04-22T23:25:53.018 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:51.919+0000 7f02aa222700 1 -- 172.21.15.107:0/118890757 shutdown_connections 2024-04-22T23:25:53.019 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:51.919+0000 7f02aa222700 1 --2- 172.21.15.107:0/118890757 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02a4101820 0x7f02a4101c30 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:25:53.019 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:51.919+0000 7f02aa222700 1 -- 172.21.15.107:0/118890757 >> 172.21.15.107:0/118890757 conn(0x7f02a40fcfe0 msgr2=0x7f02a40ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:25:53.019 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:51.919+0000 7f02aa222700 1 -- 172.21.15.107:0/118890757 shutdown_connections 2024-04-22T23:25:53.019 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:51.919+0000 7f02aa222700 1 -- 172.21.15.107:0/118890757 wait complete. 2024-04-22T23:25:53.019 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:51.920+0000 7f02aa222700 1 Processor -- start 2024-04-22T23:25:53.019 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:51.920+0000 7f02aa222700 1 -- start start 2024-04-22T23:25:53.019 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:51.920+0000 7f02aa222700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02a4101820 0x7f02a4076b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:25:53.019 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:51.920+0000 7f02aa222700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a4114420 con 0x7f02a4101820 2024-04-22T23:25:53.019 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:51.920+0000 7f02a37fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02a4101820 0x7f02a4076b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:25:53.019 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:51.920+0000 7f02a37fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02a4101820 0x7f02a4076b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36308/0 (socket says 172.21.15.107:36308) 2024-04-22T23:25:53.019 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:51.920+0000 7f02a37fe700 1 -- 172.21.15.107:0/366192975 learned_addr learned my addr 172.21.15.107:0/366192975 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:25:53.019 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:51.921+0000 7f02a37fe700 1 -- 172.21.15.107:0/366192975 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02940094f0 con 0x7f02a4101820 2024-04-22T23:25:53.019 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:51.921+0000 7f02a37fe700 1 --2- 172.21.15.107:0/366192975 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02a4101820 0x7f02a4076b30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f029400ba90 tx=0x7f0294009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:25:53.019 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:51.921+0000 7f028bfff700 1 -- 172.21.15.107:0/366192975 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f02940107c0 con 0x7f02a4101820 2024-04-22T23:25:53.020 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:51.921+0000 7f02aa222700 1 -- 172.21.15.107:0/366192975 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02a4114560 con 0x7f02a4101820 2024-04-22T23:25:53.020 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:51.921+0000 7f028bfff700 1 -- 172.21.15.107:0/366192975 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f0294010db0 con 0x7f02a4101820 2024-04-22T23:25:53.020 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:51.921+0000 7f028bfff700 1 -- 172.21.15.107:0/366192975 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f029401add0 con 0x7f02a4101820 2024-04-22T23:25:53.020 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:51.921+0000 7f02aa222700 1 -- 172.21.15.107:0/366192975 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02a41149f0 con 0x7f02a4101820 2024-04-22T23:25:53.020 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:51.922+0000 7f028bfff700 1 -- 172.21.15.107:0/366192975 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 4) v1 ==== 43365+0+0 (secure 0 0 0) 0x7f029402e440 con 0x7f02a4101820 2024-04-22T23:25:53.020 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:51.922+0000 7f028bfff700 1 --2- 172.21.15.107:0/366192975 >> [v2:172.21.15.107:6800/467262774,v1:172.21.15.107:6801/467262774] conn(0x7f028c036740 0x7f028c038bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:25:53.020 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:51.922+0000 7f02aa222700 1 -- 172.21.15.107:0/366192975 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02a404bed0 con 0x7f02a4101820 2024-04-22T23:25:53.020 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:51.922+0000 7f028bfff700 1 -- 172.21.15.107:0/366192975 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f029404a730 con 0x7f02a4101820 2024-04-22T23:25:53.020 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:51.922+0000 7f02a2ffd700 1 --2- 172.21.15.107:0/366192975 >> [v2:172.21.15.107:6800/467262774,v1:172.21.15.107:6801/467262774] conn(0x7f028c036740 0x7f028c038bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:25:53.020 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:51.923+0000 7f02a2ffd700 1 --2- 172.21.15.107:0/366192975 >> [v2:172.21.15.107:6800/467262774,v1:172.21.15.107:6801/467262774] conn(0x7f028c036740 0x7f028c038bf0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0298009990 tx=0x7f0298006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:25:53.020 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:51.927+0000 7f028bfff700 1 -- 172.21.15.107:0/366192975 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0294018940 con 0x7f02a4101820 2024-04-22T23:25:53.020 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:52.161+0000 7f02aa222700 1 -- 172.21.15.107:0/366192975 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f02a41035a0 con 0x7f02a4101820 2024-04-22T23:25:53.020 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:52.404+0000 7f028bfff700 1 -- 172.21.15.107:0/366192975 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 5) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f029402e740 con 0x7f02a4101820 2024-04-22T23:25:53.020 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:52.406+0000 7f028bfff700 1 -- 172.21.15.107:0/366192975 <== mon.0 v2:172.21.15.107:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f02940498d0 con 0x7f02a4101820 2024-04-22T23:25:53.020 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:52.408+0000 7f02aa222700 1 -- 172.21.15.107:0/366192975 >> [v2:172.21.15.107:6800/467262774,v1:172.21.15.107:6801/467262774] conn(0x7f028c036740 msgr2=0x7f028c038bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:25:53.021 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:52.408+0000 7f02aa222700 1 --2- 172.21.15.107:0/366192975 >> [v2:172.21.15.107:6800/467262774,v1:172.21.15.107:6801/467262774] conn(0x7f028c036740 0x7f028c038bf0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0298009990 tx=0x7f0298006e40 comp rx=0 tx=0).stop 2024-04-22T23:25:53.021 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:52.408+0000 7f02aa222700 1 -- 172.21.15.107:0/366192975 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02a4101820 msgr2=0x7f02a4076b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:25:53.021 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:52.408+0000 7f02aa222700 1 --2- 172.21.15.107:0/366192975 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02a4101820 0x7f02a4076b30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f029400ba90 tx=0x7f0294009250 comp rx=0 tx=0).stop 2024-04-22T23:25:53.021 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:52.408+0000 7f02aa222700 1 -- 172.21.15.107:0/366192975 shutdown_connections 2024-04-22T23:25:53.021 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:52.408+0000 7f02aa222700 1 --2- 172.21.15.107:0/366192975 >> [v2:172.21.15.107:6800/467262774,v1:172.21.15.107:6801/467262774] conn(0x7f028c036740 0x7f028c038bf0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:25:53.021 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:52.408+0000 7f02aa222700 1 --2- 172.21.15.107:0/366192975 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02a4101820 0x7f02a4076b30 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:25:53.021 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:52.408+0000 7f02aa222700 1 -- 172.21.15.107:0/366192975 >> 172.21.15.107:0/366192975 conn(0x7f02a40fcfe0 msgr2=0x7f02a40fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:25:53.021 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:52.408+0000 7f02aa222700 1 -- 172.21.15.107:0/366192975 shutdown_connections 2024-04-22T23:25:53.021 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:52.408+0000 7f02aa222700 1 -- 172.21.15.107:0/366192975 wait complete. 2024-04-22T23:25:53.651 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:53 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/366192975' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-22T23:25:53.652 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:53 smithi107 ceph-mon[26355]: mgrmap e5: smithi107.uipxvm(active, since 5s) 2024-04-22T23:25:54.408 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-04-22T23:25:54.408 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-22T23:25:54.408 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-22T23:25:54.408 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "active_name": "smithi107.uipxvm", 2024-04-22T23:25:54.409 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-22T23:25:54.409 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-04-22T23:25:54.409 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:53.443+0000 7f24da8ae700 1 Processor -- start 2024-04-22T23:25:54.409 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:53.443+0000 7f24da8ae700 1 -- start start 2024-04-22T23:25:54.409 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:53.443+0000 7f24da8ae700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24d4078c00 0x7f24d40772a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:25:54.409 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:53.443+0000 7f24da8ae700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d4079010 con 0x7f24d4078c00 2024-04-22T23:25:54.409 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:53.444+0000 7f24d3fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24d4078c00 0x7f24d40772a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:25:54.409 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:53.444+0000 7f24d3fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24d4078c00 0x7f24d40772a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36336/0 (socket says 172.21.15.107:36336) 2024-04-22T23:25:54.409 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:53.444+0000 7f24d3fff700 1 -- 172.21.15.107:0/3804082177 learned_addr learned my addr 172.21.15.107:0/3804082177 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:25:54.409 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:53.444+0000 7f24d3fff700 1 -- 172.21.15.107:0/3804082177 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d4079150 con 0x7f24d4078c00 2024-04-22T23:25:54.409 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:53.444+0000 7f24d3fff700 1 --2- 172.21.15.107:0/3804082177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24d4078c00 0x7f24d40772a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f24c40098e0 tx=0x7f24c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4befb06b1a91146b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:25:54.409 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:53.444+0000 7f24d2ffd700 1 -- 172.21.15.107:0/3804082177 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f24c400b8f0 con 0x7f24d4078c00 2024-04-22T23:25:54.409 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:53.444+0000 7f24d2ffd700 1 -- 172.21.15.107:0/3804082177 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f24c4011420 con 0x7f24d4078c00 2024-04-22T23:25:54.409 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:53.444+0000 7f24d2ffd700 1 -- 172.21.15.107:0/3804082177 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f24c4010540 con 0x7f24d4078c00 2024-04-22T23:25:54.410 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:53.445+0000 7f24da8ae700 1 -- 172.21.15.107:0/3804082177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24d4078c00 msgr2=0x7f24d40772a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:25:54.410 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:53.445+0000 7f24da8ae700 1 --2- 172.21.15.107:0/3804082177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24d4078c00 0x7f24d40772a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f24c40098e0 tx=0x7f24c4009250 comp rx=0 tx=0).stop 2024-04-22T23:25:54.410 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:53.445+0000 7f24da8ae700 1 -- 172.21.15.107:0/3804082177 shutdown_connections 2024-04-22T23:25:54.410 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:53.445+0000 7f24da8ae700 1 --2- 172.21.15.107:0/3804082177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24d4078c00 0x7f24d40772a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:25:54.410 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:53.445+0000 7f24da8ae700 1 -- 172.21.15.107:0/3804082177 >> 172.21.15.107:0/3804082177 conn(0x7f24d40fe0e0 msgr2=0x7f24d4100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:25:54.410 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:53.445+0000 7f24da8ae700 1 -- 172.21.15.107:0/3804082177 shutdown_connections 2024-04-22T23:25:54.410 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:53.445+0000 7f24da8ae700 1 -- 172.21.15.107:0/3804082177 wait complete. 2024-04-22T23:25:54.410 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:53.445+0000 7f24da8ae700 1 Processor -- start 2024-04-22T23:25:54.410 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:53.446+0000 7f24da8ae700 1 -- start start 2024-04-22T23:25:54.410 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:53.446+0000 7f24da8ae700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24d4078c00 0x7f24d4117e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:25:54.410 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:53.446+0000 7f24da8ae700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d4118340 con 0x7f24d4078c00 2024-04-22T23:25:54.410 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:53.446+0000 7f24d3fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24d4078c00 0x7f24d4117e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:25:54.410 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:53.446+0000 7f24d3fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24d4078c00 0x7f24d4117e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36342/0 (socket says 172.21.15.107:36342) 2024-04-22T23:25:54.410 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:53.446+0000 7f24d3fff700 1 -- 172.21.15.107:0/3595722110 learned_addr learned my addr 172.21.15.107:0/3595722110 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:25:54.410 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:53.446+0000 7f24d3fff700 1 -- 172.21.15.107:0/3595722110 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c40094f0 con 0x7f24d4078c00 2024-04-22T23:25:54.411 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:53.446+0000 7f24d3fff700 1 --2- 172.21.15.107:0/3595722110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24d4078c00 0x7f24d4117e00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f24c40094c0 tx=0x7f24c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:25:54.411 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:53.446+0000 7f24d0ff9700 1 -- 172.21.15.107:0/3595722110 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f24c4010800 con 0x7f24d4078c00 2024-04-22T23:25:54.411 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:53.446+0000 7f24d0ff9700 1 -- 172.21.15.107:0/3595722110 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f24c4010df0 con 0x7f24d4078c00 2024-04-22T23:25:54.411 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:53.446+0000 7f24da8ae700 1 -- 172.21.15.107:0/3595722110 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24d4118540 con 0x7f24d4078c00 2024-04-22T23:25:54.411 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:53.446+0000 7f24d0ff9700 1 -- 172.21.15.107:0/3595722110 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f24c4024e50 con 0x7f24d4078c00 2024-04-22T23:25:54.411 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:53.446+0000 7f24da8ae700 1 -- 172.21.15.107:0/3595722110 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24d4118f70 con 0x7f24d4078c00 2024-04-22T23:25:54.411 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:53.447+0000 7f24d0ff9700 1 -- 172.21.15.107:0/3595722110 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 5) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f24c401a540 con 0x7f24d4078c00 2024-04-22T23:25:54.411 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:53.447+0000 7f24da8ae700 1 -- 172.21.15.107:0/3595722110 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24d405f5f0 con 0x7f24d4078c00 2024-04-22T23:25:54.411 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:53.447+0000 7f24d0ff9700 1 --2- 172.21.15.107:0/3595722110 >> [v2:172.21.15.107:6800/467262774,v1:172.21.15.107:6801/467262774] conn(0x7f24bc036740 0x7f24bc038bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:25:54.411 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:53.448+0000 7f24d0ff9700 1 -- 172.21.15.107:0/3595722110 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f24c404a7e0 con 0x7f24d4078c00 2024-04-22T23:25:54.411 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:53.448+0000 7f24d37fe700 1 -- 172.21.15.107:0/3595722110 >> [v2:172.21.15.107:6800/467262774,v1:172.21.15.107:6801/467262774] conn(0x7f24bc036740 msgr2=0x7f24bc038bf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/467262774 2024-04-22T23:25:54.411 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:53.448+0000 7f24d37fe700 1 --2- 172.21.15.107:0/3595722110 >> [v2:172.21.15.107:6800/467262774,v1:172.21.15.107:6801/467262774] conn(0x7f24bc036740 0x7f24bc038bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T23:25:54.411 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:53.452+0000 7f24d0ff9700 1 -- 172.21.15.107:0/3595722110 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f24c4018940 con 0x7f24d4078c00 2024-04-22T23:25:54.412 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:53.649+0000 7f24d37fe700 1 -- 172.21.15.107:0/3595722110 >> [v2:172.21.15.107:6800/467262774,v1:172.21.15.107:6801/467262774] conn(0x7f24bc036740 msgr2=0x7f24bc038bf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/467262774 2024-04-22T23:25:54.412 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:53.649+0000 7f24d37fe700 1 --2- 172.21.15.107:0/3595722110 >> [v2:172.21.15.107:6800/467262774,v1:172.21.15.107:6801/467262774] conn(0x7f24bc036740 0x7f24bc038bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T23:25:54.412 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:53.729+0000 7f24da8ae700 1 -- 172.21.15.107:0/3595722110 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f24d4079150 con 0x7f24d4078c00 2024-04-22T23:25:54.412 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:53.730+0000 7f24d0ff9700 1 -- 172.21.15.107:0/3595722110 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7f24d4079150 con 0x7f24d4078c00 2024-04-22T23:25:54.412 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:53.732+0000 7f24da8ae700 1 -- 172.21.15.107:0/3595722110 >> [v2:172.21.15.107:6800/467262774,v1:172.21.15.107:6801/467262774] conn(0x7f24bc036740 msgr2=0x7f24bc038bf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T23:25:54.412 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:53.732+0000 7f24da8ae700 1 --2- 172.21.15.107:0/3595722110 >> [v2:172.21.15.107:6800/467262774,v1:172.21.15.107:6801/467262774] conn(0x7f24bc036740 0x7f24bc038bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:25:54.412 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:53.732+0000 7f24da8ae700 1 -- 172.21.15.107:0/3595722110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24d4078c00 msgr2=0x7f24d4117e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:25:54.412 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:53.732+0000 7f24da8ae700 1 --2- 172.21.15.107:0/3595722110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24d4078c00 0x7f24d4117e00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f24c40094c0 tx=0x7f24c4009250 comp rx=0 tx=0).stop 2024-04-22T23:25:54.412 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:53.732+0000 7f24da8ae700 1 -- 172.21.15.107:0/3595722110 shutdown_connections 2024-04-22T23:25:54.412 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:53.732+0000 7f24da8ae700 1 --2- 172.21.15.107:0/3595722110 >> [v2:172.21.15.107:6800/467262774,v1:172.21.15.107:6801/467262774] conn(0x7f24bc036740 0x7f24bc038bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:25:54.412 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:53.732+0000 7f24da8ae700 1 --2- 172.21.15.107:0/3595722110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24d4078c00 0x7f24d4117e00 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:25:54.412 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:53.732+0000 7f24da8ae700 1 -- 172.21.15.107:0/3595722110 >> 172.21.15.107:0/3595722110 conn(0x7f24d40fe0e0 msgr2=0x7f24d410f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:25:54.412 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:53.732+0000 7f24da8ae700 1 -- 172.21.15.107:0/3595722110 shutdown_connections 2024-04-22T23:25:54.412 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:53.732+0000 7f24da8ae700 1 -- 172.21.15.107:0/3595722110 wait complete. 2024-04-22T23:25:54.412 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for the mgr to restart... 2024-04-22T23:25:54.412 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mgr epoch 5... 2024-04-22T23:25:54.490 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:54 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/3595722110' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-22T23:25:58.990 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:58 smithi107 ceph-mon[26355]: Active manager daemon smithi107.uipxvm restarted 2024-04-22T23:25:58.991 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:58 smithi107 ceph-mon[26355]: Activating manager daemon smithi107.uipxvm 2024-04-22T23:25:58.991 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:58 smithi107 ceph-mon[26355]: osdmap e2: 0 total, 0 up, 0 in 2024-04-22T23:25:58.991 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:58 smithi107 ceph-mon[26355]: mgrmap e6: smithi107.uipxvm(active, starting, since 0.00333504s) 2024-04-22T23:25:58.991 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:58 smithi107 ceph-mon[26355]: from='mgr.14118 172.21.15.107:0/4176396201' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-04-22T23:25:58.991 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:58 smithi107 ceph-mon[26355]: from='mgr.14118 172.21.15.107:0/4176396201' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mgr metadata", "who": "smithi107.uipxvm", "id": "smithi107.uipxvm"}]: dispatch 2024-04-22T23:25:58.991 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:58 smithi107 ceph-mon[26355]: from='mgr.14118 172.21.15.107:0/4176396201' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-22T23:25:58.991 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:58 smithi107 ceph-mon[26355]: from='mgr.14118 172.21.15.107:0/4176396201' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-22T23:25:58.991 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:58 smithi107 ceph-mon[26355]: from='mgr.14118 172.21.15.107:0/4176396201' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-22T23:25:58.991 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:58 smithi107 ceph-mon[26355]: Manager daemon smithi107.uipxvm is now available 2024-04-22T23:25:58.991 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:58 smithi107 ceph-mon[26355]: from='mgr.14118 172.21.15.107:0/4176396201' entity='mgr.smithi107.uipxvm' 2024-04-22T23:25:58.991 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:58 smithi107 ceph-mon[26355]: from='mgr.14118 172.21.15.107:0/4176396201' entity='mgr.smithi107.uipxvm' 2024-04-22T23:25:58.991 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:58 smithi107 ceph-mon[26355]: from='mgr.14118 172.21.15.107:0/4176396201' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:25:58.991 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:58 smithi107 ceph-mon[26355]: from='mgr.14118 172.21.15.107:0/4176396201' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:25:58.992 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:58 smithi107 ceph-mon[26355]: from='mgr.14118 172.21.15.107:0/4176396201' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:25:58.992 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:58 smithi107 ceph-mon[26355]: from='mgr.14118 172.21.15.107:0/4176396201' entity='mgr.smithi107.uipxvm' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi107.uipxvm/mirror_snapshot_schedule"}]: dispatch 2024-04-22T23:25:58.992 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:58 smithi107 ceph-mon[26355]: from='mgr.14118 172.21.15.107:0/4176396201' entity='mgr.smithi107.uipxvm' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi107.uipxvm/trash_purge_schedule"}]: dispatch 2024-04-22T23:26:00.189 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:59 smithi107 ceph-mon[26355]: Found migration_current of "None". Setting to last migration. 2024-04-22T23:26:00.189 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:59 smithi107 ceph-mon[26355]: from='mgr.14118 172.21.15.107:0/4176396201' entity='mgr.smithi107.uipxvm' 2024-04-22T23:26:00.189 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:59 smithi107 ceph-mon[26355]: from='mgr.14118 172.21.15.107:0/4176396201' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:26:00.189 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:25:59 smithi107 ceph-mon[26355]: mgrmap e7: smithi107.uipxvm(active, since 1.0053s) 2024-04-22T23:26:00.460 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-04-22T23:26:00.460 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-22T23:26:00.460 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-22T23:26:00.461 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-04-22T23:26:00.461 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:54.857+0000 7f20dc456700 1 Processor -- start 2024-04-22T23:26:00.461 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:54.857+0000 7f20dc456700 1 -- start start 2024-04-22T23:26:00.461 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:54.857+0000 7f20dc456700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20d4103f20 0x7f20d4104330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:00.461 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:54.857+0000 7f20dc456700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d4070e20 con 0x7f20d4103f20 2024-04-22T23:26:00.461 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:54.858+0000 7f20da1f2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20d4103f20 0x7f20d4104330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:00.461 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:54.858+0000 7f20da1f2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20d4103f20 0x7f20d4104330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36346/0 (socket says 172.21.15.107:36346) 2024-04-22T23:26:00.461 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:54.858+0000 7f20da1f2700 1 -- 172.21.15.107:0/3922209016 learned_addr learned my addr 172.21.15.107:0/3922209016 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:26:00.461 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:54.858+0000 7f20da1f2700 1 -- 172.21.15.107:0/3922209016 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d4104870 con 0x7f20d4103f20 2024-04-22T23:26:00.461 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:54.858+0000 7f20da1f2700 1 --2- 172.21.15.107:0/3922209016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20d4103f20 0x7f20d4104330 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f20c40098e0 tx=0x7f20c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a42c6eb331804e55 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:00.461 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:54.858+0000 7f20d91f0700 1 -- 172.21.15.107:0/3922209016 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f20c400b860 con 0x7f20d4103f20 2024-04-22T23:26:00.461 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:54.858+0000 7f20d91f0700 1 -- 172.21.15.107:0/3922209016 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f20c400be50 con 0x7f20d4103f20 2024-04-22T23:26:00.461 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:54.858+0000 7f20d91f0700 1 -- 172.21.15.107:0/3922209016 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f20c4010380 con 0x7f20d4103f20 2024-04-22T23:26:00.462 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:54.859+0000 7f20dc456700 1 -- 172.21.15.107:0/3922209016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20d4103f20 msgr2=0x7f20d4104330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:00.462 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:54.859+0000 7f20dc456700 1 --2- 172.21.15.107:0/3922209016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20d4103f20 0x7f20d4104330 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f20c40098e0 tx=0x7f20c4009250 comp rx=0 tx=0).stop 2024-04-22T23:26:00.462 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:54.859+0000 7f20dc456700 1 -- 172.21.15.107:0/3922209016 shutdown_connections 2024-04-22T23:26:00.462 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:54.859+0000 7f20dc456700 1 --2- 172.21.15.107:0/3922209016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20d4103f20 0x7f20d4104330 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:00.462 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:54.859+0000 7f20dc456700 1 -- 172.21.15.107:0/3922209016 >> 172.21.15.107:0/3922209016 conn(0x7f20d40fc760 msgr2=0x7f20d40feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:26:00.462 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:54.859+0000 7f20dc456700 1 -- 172.21.15.107:0/3922209016 shutdown_connections 2024-04-22T23:26:00.462 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:54.859+0000 7f20dc456700 1 -- 172.21.15.107:0/3922209016 wait complete. 2024-04-22T23:26:00.462 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:54.859+0000 7f20dc456700 1 Processor -- start 2024-04-22T23:26:00.462 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:54.859+0000 7f20dc456700 1 -- start start 2024-04-22T23:26:00.462 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:54.860+0000 7f20dc456700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20d4103f20 0x7f20d41116a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:00.462 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:54.860+0000 7f20dc456700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d4111be0 con 0x7f20d4103f20 2024-04-22T23:26:00.462 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:54.860+0000 7f20da1f2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20d4103f20 0x7f20d41116a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:00.462 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:54.860+0000 7f20da1f2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20d4103f20 0x7f20d41116a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36362/0 (socket says 172.21.15.107:36362) 2024-04-22T23:26:00.462 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:54.860+0000 7f20da1f2700 1 -- 172.21.15.107:0/4116434970 learned_addr learned my addr 172.21.15.107:0/4116434970 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:26:00.463 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:54.860+0000 7f20da1f2700 1 -- 172.21.15.107:0/4116434970 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c40094f0 con 0x7f20d4103f20 2024-04-22T23:26:00.463 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:54.860+0000 7f20da1f2700 1 --2- 172.21.15.107:0/4116434970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20d4103f20 0x7f20d41116a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f20c400b440 tx=0x7f20c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:00.463 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:54.860+0000 7f20caffd700 1 -- 172.21.15.107:0/4116434970 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f20c4010760 con 0x7f20d4103f20 2024-04-22T23:26:00.463 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:54.860+0000 7f20caffd700 1 -- 172.21.15.107:0/4116434970 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f20c4010d50 con 0x7f20d4103f20 2024-04-22T23:26:00.463 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:54.860+0000 7f20dc456700 1 -- 172.21.15.107:0/4116434970 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20d4111de0 con 0x7f20d4103f20 2024-04-22T23:26:00.463 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:54.860+0000 7f20caffd700 1 -- 172.21.15.107:0/4116434970 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f20c401ae20 con 0x7f20d4103f20 2024-04-22T23:26:00.463 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:54.860+0000 7f20dc456700 1 -- 172.21.15.107:0/4116434970 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20d4112950 con 0x7f20d4103f20 2024-04-22T23:26:00.463 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:54.861+0000 7f20caffd700 1 -- 172.21.15.107:0/4116434970 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 5) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f20c40108c0 con 0x7f20d4103f20 2024-04-22T23:26:00.463 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:54.861+0000 7f20caffd700 1 --2- 172.21.15.107:0/4116434970 >> [v2:172.21.15.107:6800/467262774,v1:172.21.15.107:6801/467262774] conn(0x7f20c0036770 0x7f20c0038c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:00.463 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:54.861+0000 7f20caffd700 1 -- 172.21.15.107:0/4116434970 --> [v2:172.21.15.107:6800/467262774,v1:172.21.15.107:6801/467262774] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f20c0039330 con 0x7f20c0036770 2024-04-22T23:26:00.463 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:54.861+0000 7f20caffd700 1 -- 172.21.15.107:0/4116434970 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f20c404a5d0 con 0x7f20d4103f20 2024-04-22T23:26:00.463 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:54.861+0000 7f20d99f1700 1 -- 172.21.15.107:0/4116434970 >> [v2:172.21.15.107:6800/467262774,v1:172.21.15.107:6801/467262774] conn(0x7f20c0036770 msgr2=0x7f20c0038c20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/467262774 2024-04-22T23:26:00.463 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:54.861+0000 7f20d99f1700 1 --2- 172.21.15.107:0/4116434970 >> [v2:172.21.15.107:6800/467262774,v1:172.21.15.107:6801/467262774] conn(0x7f20c0036770 0x7f20c0038c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T23:26:00.463 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:55.062+0000 7f20d99f1700 1 -- 172.21.15.107:0/4116434970 >> [v2:172.21.15.107:6800/467262774,v1:172.21.15.107:6801/467262774] conn(0x7f20c0036770 msgr2=0x7f20c0038c20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/467262774 2024-04-22T23:26:00.464 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:55.062+0000 7f20d99f1700 1 --2- 172.21.15.107:0/4116434970 >> [v2:172.21.15.107:6800/467262774,v1:172.21.15.107:6801/467262774] conn(0x7f20c0036770 0x7f20c0038c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T23:26:00.464 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:55.463+0000 7f20d99f1700 1 -- 172.21.15.107:0/4116434970 >> [v2:172.21.15.107:6800/467262774,v1:172.21.15.107:6801/467262774] conn(0x7f20c0036770 msgr2=0x7f20c0038c20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/467262774 2024-04-22T23:26:00.464 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:55.463+0000 7f20d99f1700 1 --2- 172.21.15.107:0/4116434970 >> [v2:172.21.15.107:6800/467262774,v1:172.21.15.107:6801/467262774] conn(0x7f20c0036770 0x7f20c0038c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T23:26:00.464 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:56.264+0000 7f20d99f1700 1 -- 172.21.15.107:0/4116434970 >> [v2:172.21.15.107:6800/467262774,v1:172.21.15.107:6801/467262774] conn(0x7f20c0036770 msgr2=0x7f20c0038c20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/467262774 2024-04-22T23:26:00.464 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:56.264+0000 7f20d99f1700 1 --2- 172.21.15.107:0/4116434970 >> [v2:172.21.15.107:6800/467262774,v1:172.21.15.107:6801/467262774] conn(0x7f20c0036770 0x7f20c0038c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T23:26:00.464 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:57.865+0000 7f20d99f1700 1 -- 172.21.15.107:0/4116434970 >> [v2:172.21.15.107:6800/467262774,v1:172.21.15.107:6801/467262774] conn(0x7f20c0036770 msgr2=0x7f20c0038c20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/467262774 2024-04-22T23:26:00.464 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:57.865+0000 7f20d99f1700 1 --2- 172.21.15.107:0/4116434970 >> [v2:172.21.15.107:6800/467262774,v1:172.21.15.107:6801/467262774] conn(0x7f20c0036770 0x7f20c0038c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-22T23:26:00.464 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:58.817+0000 7f20caffd700 1 -- 172.21.15.107:0/4116434970 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mgrmap(e 6) v1 ==== 43143+0+0 (secure 0 0 0) 0x7f20c402e7c0 con 0x7f20d4103f20 2024-04-22T23:26:00.464 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:58.817+0000 7f20caffd700 1 -- 172.21.15.107:0/4116434970 >> [v2:172.21.15.107:6800/467262774,v1:172.21.15.107:6801/467262774] conn(0x7f20c0036770 msgr2=0x7f20c0038c20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T23:26:00.464 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:58.817+0000 7f20caffd700 1 --2- 172.21.15.107:0/4116434970 >> [v2:172.21.15.107:6800/467262774,v1:172.21.15.107:6801/467262774] conn(0x7f20c0036770 0x7f20c0038c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:00.464 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:59.818+0000 7f20caffd700 1 -- 172.21.15.107:0/4116434970 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 7) v1 ==== 43270+0+0 (secure 0 0 0) 0x7f20c402dc70 con 0x7f20d4103f20 2024-04-22T23:26:00.464 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:59.818+0000 7f20caffd700 1 --2- 172.21.15.107:0/4116434970 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f20c0039ff0 0x7f20c003c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:00.464 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:59.818+0000 7f20caffd700 1 -- 172.21.15.107:0/4116434970 --> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f20c0039330 con 0x7f20c0039ff0 2024-04-22T23:26:00.464 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:59.818+0000 7f20d99f1700 1 --2- 172.21.15.107:0/4116434970 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f20c0039ff0 0x7f20c003c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:00.465 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:59.819+0000 7f20d99f1700 1 --2- 172.21.15.107:0/4116434970 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f20c0039ff0 0x7f20c003c3d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f20d00039d0 tx=0x7f20d0009350 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:00.465 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:59.819+0000 7f20caffd700 1 -- 172.21.15.107:0/4116434970 <== mgr.14118 v2:172.21.15.107:6800/307474287 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7f20c0039330 con 0x7f20c0039ff0 2024-04-22T23:26:00.465 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:59.825+0000 7f20dc456700 1 -- 172.21.15.107:0/4116434970 --> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f20d4112270 con 0x7f20c0039ff0 2024-04-22T23:26:00.465 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:59.825+0000 7f20caffd700 1 -- 172.21.15.107:0/4116434970 <== mgr.14118 v2:172.21.15.107:6800/307474287 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f20d4112270 con 0x7f20c0039ff0 2024-04-22T23:26:00.465 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:59.826+0000 7f20dc456700 1 -- 172.21.15.107:0/4116434970 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f20c0039ff0 msgr2=0x7f20c003c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:00.465 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:59.826+0000 7f20dc456700 1 --2- 172.21.15.107:0/4116434970 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f20c0039ff0 0x7f20c003c3d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f20d00039d0 tx=0x7f20d0009350 comp rx=0 tx=0).stop 2024-04-22T23:26:00.465 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:59.826+0000 7f20dc456700 1 -- 172.21.15.107:0/4116434970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20d4103f20 msgr2=0x7f20d41116a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:00.465 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:59.826+0000 7f20dc456700 1 --2- 172.21.15.107:0/4116434970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20d4103f20 0x7f20d41116a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f20c400b440 tx=0x7f20c4009250 comp rx=0 tx=0).stop 2024-04-22T23:26:00.465 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:59.826+0000 7f20dc456700 1 -- 172.21.15.107:0/4116434970 shutdown_connections 2024-04-22T23:26:00.465 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:59.826+0000 7f20dc456700 1 --2- 172.21.15.107:0/4116434970 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f20c0039ff0 0x7f20c003c3d0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:00.465 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:59.826+0000 7f20dc456700 1 --2- 172.21.15.107:0/4116434970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20d4103f20 0x7f20d41116a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:00.465 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:59.826+0000 7f20dc456700 1 --2- 172.21.15.107:0/4116434970 >> [v2:172.21.15.107:6800/467262774,v1:172.21.15.107:6801/467262774] conn(0x7f20c0036770 0x7f20c0038c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:00.465 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:59.826+0000 7f20dc456700 1 -- 172.21.15.107:0/4116434970 >> 172.21.15.107:0/4116434970 conn(0x7f20d40fc760 msgr2=0x7f20d40fd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:26:00.465 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:59.826+0000 7f20dc456700 1 -- 172.21.15.107:0/4116434970 shutdown_connections 2024-04-22T23:26:00.466 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:25:59.826+0000 7f20dc456700 1 -- 172.21.15.107:0/4116434970 wait complete. 2024-04-22T23:26:00.466 INFO:teuthology.orchestra.run.smithi107.stdout:mgr epoch 5 is available 2024-04-22T23:26:00.466 INFO:teuthology.orchestra.run.smithi107.stdout:Setting orchestrator backend to cephadm... 2024-04-22T23:26:01.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:00 smithi107 ceph-mon[26355]: [22/Apr/2024:23:25:59] ENGINE Bus STARTING 2024-04-22T23:26:01.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:00 smithi107 ceph-mon[26355]: [22/Apr/2024:23:25:59] ENGINE Serving on https://172.21.15.107:7150 2024-04-22T23:26:01.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:00 smithi107 ceph-mon[26355]: [22/Apr/2024:23:25:59] ENGINE Bus STARTED 2024-04-22T23:26:01.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:00 smithi107 ceph-mon[26355]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-22T23:26:01.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:00 smithi107 ceph-mon[26355]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-22T23:26:01.662 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:00.890+0000 7f8e65db9700 1 Processor -- start 2024-04-22T23:26:01.663 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:00.890+0000 7f8e65db9700 1 -- start start 2024-04-22T23:26:01.663 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:00.890+0000 7f8e65db9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e60102930 0x7f8e60102d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:01.663 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:00.891+0000 7f8e65db9700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e60103280 con 0x7f8e60102930 2024-04-22T23:26:01.663 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:00.891+0000 7f8e5f7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e60102930 0x7f8e60102d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:01.663 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:00.891+0000 7f8e5f7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e60102930 0x7f8e60102d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36446/0 (socket says 172.21.15.107:36446) 2024-04-22T23:26:01.663 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:00.891+0000 7f8e5f7fe700 1 -- 172.21.15.107:0/2160721430 learned_addr learned my addr 172.21.15.107:0/2160721430 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:26:01.663 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:00.891+0000 7f8e5f7fe700 1 -- 172.21.15.107:0/2160721430 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e601033c0 con 0x7f8e60102930 2024-04-22T23:26:01.663 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:00.891+0000 7f8e5f7fe700 1 --2- 172.21.15.107:0/2160721430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e60102930 0x7f8e60102d40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8e58009e50 tx=0x7f8e58009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e85b099e40406225 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:01.663 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:00.891+0000 7f8e5e7fc700 1 -- 172.21.15.107:0/2160721430 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8e580092e0 con 0x7f8e60102930 2024-04-22T23:26:01.663 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:00.891+0000 7f8e5e7fc700 1 -- 172.21.15.107:0/2160721430 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f8e58012430 con 0x7f8e60102930 2024-04-22T23:26:01.663 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:00.891+0000 7f8e5e7fc700 1 -- 172.21.15.107:0/2160721430 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8e580075c0 con 0x7f8e60102930 2024-04-22T23:26:01.663 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:00.892+0000 7f8e65db9700 1 -- 172.21.15.107:0/2160721430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e60102930 msgr2=0x7f8e60102d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:01.663 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:00.892+0000 7f8e65db9700 1 --2- 172.21.15.107:0/2160721430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e60102930 0x7f8e60102d40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8e58009e50 tx=0x7f8e58009840 comp rx=0 tx=0).stop 2024-04-22T23:26:01.663 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:00.892+0000 7f8e65db9700 1 -- 172.21.15.107:0/2160721430 shutdown_connections 2024-04-22T23:26:01.664 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:00.892+0000 7f8e65db9700 1 --2- 172.21.15.107:0/2160721430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e60102930 0x7f8e60102d40 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:01.664 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:00.892+0000 7f8e65db9700 1 -- 172.21.15.107:0/2160721430 >> 172.21.15.107:0/2160721430 conn(0x7f8e600fe140 msgr2=0x7f8e60100550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:26:01.664 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:00.892+0000 7f8e65db9700 1 -- 172.21.15.107:0/2160721430 shutdown_connections 2024-04-22T23:26:01.664 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:00.892+0000 7f8e65db9700 1 -- 172.21.15.107:0/2160721430 wait complete. 2024-04-22T23:26:01.664 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:00.892+0000 7f8e65db9700 1 Processor -- start 2024-04-22T23:26:01.664 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:00.893+0000 7f8e65db9700 1 -- start start 2024-04-22T23:26:01.664 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:00.893+0000 7f8e65db9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e60102930 0x7f8e601116f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:01.664 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:00.893+0000 7f8e65db9700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e60111c30 con 0x7f8e60102930 2024-04-22T23:26:01.664 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:00.893+0000 7f8e5f7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e60102930 0x7f8e601116f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:01.664 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:00.893+0000 7f8e5f7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e60102930 0x7f8e601116f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36460/0 (socket says 172.21.15.107:36460) 2024-04-22T23:26:01.664 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:00.893+0000 7f8e5f7fe700 1 -- 172.21.15.107:0/4246014109 learned_addr learned my addr 172.21.15.107:0/4246014109 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:26:01.664 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:00.893+0000 7f8e5f7fe700 1 -- 172.21.15.107:0/4246014109 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e58009ae0 con 0x7f8e60102930 2024-04-22T23:26:01.664 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:00.893+0000 7f8e5f7fe700 1 --2- 172.21.15.107:0/4246014109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e60102930 0x7f8e601116f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8e58009ab0 tx=0x7f8e58009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:01.665 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:00.893+0000 7f8e57fff700 1 -- 172.21.15.107:0/4246014109 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8e580092e0 con 0x7f8e60102930 2024-04-22T23:26:01.665 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:00.893+0000 7f8e65db9700 1 -- 172.21.15.107:0/4246014109 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e60111e30 con 0x7f8e60102930 2024-04-22T23:26:01.665 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:00.893+0000 7f8e57fff700 1 -- 172.21.15.107:0/4246014109 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f8e58007d10 con 0x7f8e60102930 2024-04-22T23:26:01.665 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:00.893+0000 7f8e57fff700 1 -- 172.21.15.107:0/4246014109 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8e5801c9b0 con 0x7f8e60102930 2024-04-22T23:26:01.665 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:00.894+0000 7f8e65db9700 1 -- 172.21.15.107:0/4246014109 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e601129a0 con 0x7f8e60102930 2024-04-22T23:26:01.665 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:00.894+0000 7f8e57fff700 1 -- 172.21.15.107:0/4246014109 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f8e58019070 con 0x7f8e60102930 2024-04-22T23:26:01.665 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:00.894+0000 7f8e57fff700 1 --2- 172.21.15.107:0/4246014109 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f8e4c0367e0 0x7f8e4c038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:01.665 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:00.894+0000 7f8e57fff700 1 -- 172.21.15.107:0/4246014109 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f8e58049e10 con 0x7f8e60102930 2024-04-22T23:26:01.665 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:00.895+0000 7f8e5effd700 1 --2- 172.21.15.107:0/4246014109 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f8e4c0367e0 0x7f8e4c038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:01.665 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:00.895+0000 7f8e65db9700 1 -- 172.21.15.107:0/4246014109 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e6005f5f0 con 0x7f8e60102930 2024-04-22T23:26:01.665 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:00.895+0000 7f8e5effd700 1 --2- 172.21.15.107:0/4246014109 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f8e4c0367e0 0x7f8e4c038c90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8e48009990 tx=0x7f8e48006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:01.665 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:00.900+0000 7f8e57fff700 1 -- 172.21.15.107:0/4246014109 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8e58017080 con 0x7f8e60102930 2024-04-22T23:26:01.665 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:01.097+0000 7f8e65db9700 1 -- 172.21.15.107:0/4246014109 --> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e600ff3d0 con 0x7f8e4c0367e0 2024-04-22T23:26:01.666 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:01.106+0000 7f8e57fff700 1 -- 172.21.15.107:0/4246014109 <== mgr.14118 v2:172.21.15.107:6800/307474287 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f8e600ff3d0 con 0x7f8e4c0367e0 2024-04-22T23:26:01.666 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:01.108+0000 7f8e65db9700 1 -- 172.21.15.107:0/4246014109 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f8e4c0367e0 msgr2=0x7f8e4c038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:01.666 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:01.108+0000 7f8e65db9700 1 --2- 172.21.15.107:0/4246014109 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f8e4c0367e0 0x7f8e4c038c90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8e48009990 tx=0x7f8e48006e40 comp rx=0 tx=0).stop 2024-04-22T23:26:01.666 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:01.108+0000 7f8e65db9700 1 -- 172.21.15.107:0/4246014109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e60102930 msgr2=0x7f8e601116f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:01.666 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:01.108+0000 7f8e65db9700 1 --2- 172.21.15.107:0/4246014109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e60102930 0x7f8e601116f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8e58009ab0 tx=0x7f8e58009840 comp rx=0 tx=0).stop 2024-04-22T23:26:01.666 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:01.108+0000 7f8e65db9700 1 -- 172.21.15.107:0/4246014109 shutdown_connections 2024-04-22T23:26:01.666 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:01.108+0000 7f8e65db9700 1 --2- 172.21.15.107:0/4246014109 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f8e4c0367e0 0x7f8e4c038c90 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:01.666 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:01.108+0000 7f8e65db9700 1 --2- 172.21.15.107:0/4246014109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e60102930 0x7f8e601116f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:01.666 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:01.108+0000 7f8e65db9700 1 -- 172.21.15.107:0/4246014109 >> 172.21.15.107:0/4246014109 conn(0x7f8e600fe140 msgr2=0x7f8e600feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:26:01.666 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:01.109+0000 7f8e65db9700 1 -- 172.21.15.107:0/4246014109 shutdown_connections 2024-04-22T23:26:01.666 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:01.109+0000 7f8e65db9700 1 -- 172.21.15.107:0/4246014109 wait complete. 2024-04-22T23:26:02.041 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:01 smithi107 ceph-mon[26355]: mgrmap e8: smithi107.uipxvm(active, since 2s) 2024-04-22T23:26:02.041 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:01 smithi107 ceph-mon[26355]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:26:02.041 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:01 smithi107 ceph-mon[26355]: from='mgr.14118 172.21.15.107:0/4176396201' entity='mgr.smithi107.uipxvm' 2024-04-22T23:26:02.041 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:01 smithi107 ceph-mon[26355]: from='mgr.14118 172.21.15.107:0/4176396201' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:26:02.877 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-22T23:26:02.877 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:02.051+0000 7fefa3927700 1 Processor -- start 2024-04-22T23:26:02.878 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:02.052+0000 7fefa3927700 1 -- start start 2024-04-22T23:26:02.878 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:02.052+0000 7fefa3927700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef9c076ae0 0x7fef9c1055a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:02.878 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:02.052+0000 7fefa3927700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c076ef0 con 0x7fef9c076ae0 2024-04-22T23:26:02.878 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:02.052+0000 7fefa16c3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef9c076ae0 0x7fef9c1055a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:02.878 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:02.052+0000 7fefa16c3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef9c076ae0 0x7fef9c1055a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34296/0 (socket says 172.21.15.107:34296) 2024-04-22T23:26:02.878 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:02.052+0000 7fefa16c3700 1 -- 172.21.15.107:0/1868815553 learned_addr learned my addr 172.21.15.107:0/1868815553 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:26:02.878 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:02.052+0000 7fefa16c3700 1 -- 172.21.15.107:0/1868815553 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef9c105ae0 con 0x7fef9c076ae0 2024-04-22T23:26:02.878 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:02.053+0000 7fefa16c3700 1 --2- 172.21.15.107:0/1868815553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef9c076ae0 0x7fef9c1055a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fef8c0098e0 tx=0x7fef8c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc879b6463bb2db8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:02.878 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:02.053+0000 7fef93fff700 1 -- 172.21.15.107:0/1868815553 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fef8c00b8f0 con 0x7fef9c076ae0 2024-04-22T23:26:02.878 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:02.053+0000 7fef93fff700 1 -- 172.21.15.107:0/1868815553 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fef8c011420 con 0x7fef9c076ae0 2024-04-22T23:26:02.878 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:02.053+0000 7fef93fff700 1 -- 172.21.15.107:0/1868815553 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fef8c010540 con 0x7fef9c076ae0 2024-04-22T23:26:02.878 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:02.053+0000 7fefa3927700 1 -- 172.21.15.107:0/1868815553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef9c076ae0 msgr2=0x7fef9c1055a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:02.879 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:02.053+0000 7fefa3927700 1 --2- 172.21.15.107:0/1868815553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef9c076ae0 0x7fef9c1055a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fef8c0098e0 tx=0x7fef8c009250 comp rx=0 tx=0).stop 2024-04-22T23:26:02.879 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:02.053+0000 7fefa3927700 1 -- 172.21.15.107:0/1868815553 shutdown_connections 2024-04-22T23:26:02.879 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:02.053+0000 7fefa3927700 1 --2- 172.21.15.107:0/1868815553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef9c076ae0 0x7fef9c1055a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:02.879 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:02.053+0000 7fefa3927700 1 -- 172.21.15.107:0/1868815553 >> 172.21.15.107:0/1868815553 conn(0x7fef9c0fcfb0 msgr2=0x7fef9c0ff400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:26:02.879 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:02.053+0000 7fefa3927700 1 -- 172.21.15.107:0/1868815553 shutdown_connections 2024-04-22T23:26:02.879 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:02.053+0000 7fefa3927700 1 -- 172.21.15.107:0/1868815553 wait complete. 2024-04-22T23:26:02.879 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:02.054+0000 7fefa3927700 1 Processor -- start 2024-04-22T23:26:02.879 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:02.054+0000 7fefa3927700 1 -- start start 2024-04-22T23:26:02.879 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:02.054+0000 7fefa3927700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef9c076ae0 0x7fef9c102ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:02.880 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:02.054+0000 7fefa3927700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c103430 con 0x7fef9c076ae0 2024-04-22T23:26:02.880 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:02.054+0000 7fefa16c3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef9c076ae0 0x7fef9c102ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:02.880 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:02.054+0000 7fefa16c3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef9c076ae0 0x7fef9c102ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34302/0 (socket says 172.21.15.107:34302) 2024-04-22T23:26:02.880 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:02.054+0000 7fefa16c3700 1 -- 172.21.15.107:0/728363825 learned_addr learned my addr 172.21.15.107:0/728363825 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:26:02.880 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:02.054+0000 7fefa16c3700 1 -- 172.21.15.107:0/728363825 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef8c0094f0 con 0x7fef9c076ae0 2024-04-22T23:26:02.880 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:02.054+0000 7fefa16c3700 1 --2- 172.21.15.107:0/728363825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef9c076ae0 0x7fef9c102ef0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fef8c0094c0 tx=0x7fef8c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:02.880 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:02.055+0000 7fef91ffb700 1 -- 172.21.15.107:0/728363825 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fef8c010800 con 0x7fef9c076ae0 2024-04-22T23:26:02.880 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:02.055+0000 7fef91ffb700 1 -- 172.21.15.107:0/728363825 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fef8c010df0 con 0x7fef9c076ae0 2024-04-22T23:26:02.880 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:02.055+0000 7fefa3927700 1 -- 172.21.15.107:0/728363825 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef9c1014a0 con 0x7fef9c076ae0 2024-04-22T23:26:02.880 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:02.055+0000 7fef91ffb700 1 -- 172.21.15.107:0/728363825 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fef8c024e50 con 0x7fef9c076ae0 2024-04-22T23:26:02.880 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:02.055+0000 7fefa3927700 1 -- 172.21.15.107:0/728363825 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef9c101990 con 0x7fef9c076ae0 2024-04-22T23:26:02.880 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:02.056+0000 7fef91ffb700 1 -- 172.21.15.107:0/728363825 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fef8c01a490 con 0x7fef9c076ae0 2024-04-22T23:26:02.880 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:02.056+0000 7fefa3927700 1 -- 172.21.15.107:0/728363825 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef9c04bed0 con 0x7fef9c076ae0 2024-04-22T23:26:02.880 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:02.056+0000 7fef91ffb700 1 --2- 172.21.15.107:0/728363825 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7fef88036790 0x7fef88038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:02.881 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:02.056+0000 7fef91ffb700 1 -- 172.21.15.107:0/728363825 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fef8c04a5a0 con 0x7fef9c076ae0 2024-04-22T23:26:02.881 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:02.056+0000 7fefa0ec2700 1 --2- 172.21.15.107:0/728363825 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7fef88036790 0x7fef88038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:02.881 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:02.056+0000 7fefa0ec2700 1 --2- 172.21.15.107:0/728363825 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7fef88036790 0x7fef88038c40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fef98009990 tx=0x7fef98006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:02.881 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:02.060+0000 7fef91ffb700 1 -- 172.21.15.107:0/728363825 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fef8c018940 con 0x7fef9c076ae0 2024-04-22T23:26:02.881 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:02.249+0000 7fefa3927700 1 -- 172.21.15.107:0/728363825 --> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7fef9c1020a0 con 0x7fef88036790 2024-04-22T23:26:02.881 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:02.250+0000 7fef91ffb700 1 -- 172.21.15.107:0/728363825 <== mgr.14118 v2:172.21.15.107:6800/307474287 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7fef9c1020a0 con 0x7fef88036790 2024-04-22T23:26:02.881 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:02.251+0000 7fefa3927700 1 -- 172.21.15.107:0/728363825 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7fef88036790 msgr2=0x7fef88038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:02.881 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:02.251+0000 7fefa3927700 1 --2- 172.21.15.107:0/728363825 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7fef88036790 0x7fef88038c40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fef98009990 tx=0x7fef98006e40 comp rx=0 tx=0).stop 2024-04-22T23:26:02.881 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:02.251+0000 7fefa3927700 1 -- 172.21.15.107:0/728363825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef9c076ae0 msgr2=0x7fef9c102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:02.881 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:02.251+0000 7fefa3927700 1 --2- 172.21.15.107:0/728363825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef9c076ae0 0x7fef9c102ef0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fef8c0094c0 tx=0x7fef8c009250 comp rx=0 tx=0).stop 2024-04-22T23:26:02.881 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:02.252+0000 7fefa3927700 1 -- 172.21.15.107:0/728363825 shutdown_connections 2024-04-22T23:26:02.881 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:02.252+0000 7fefa3927700 1 --2- 172.21.15.107:0/728363825 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7fef88036790 0x7fef88038c40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:02.882 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:02.252+0000 7fefa3927700 1 --2- 172.21.15.107:0/728363825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef9c076ae0 0x7fef9c102ef0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:02.882 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:02.252+0000 7fefa3927700 1 -- 172.21.15.107:0/728363825 >> 172.21.15.107:0/728363825 conn(0x7fef9c0fcfb0 msgr2=0x7fef9c0fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:26:02.882 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:02.252+0000 7fefa3927700 1 -- 172.21.15.107:0/728363825 shutdown_connections 2024-04-22T23:26:02.882 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:02.252+0000 7fefa3927700 1 -- 172.21.15.107:0/728363825 wait complete. 2024-04-22T23:26:02.882 INFO:teuthology.orchestra.run.smithi107.stdout:Generating ssh key... 2024-04-22T23:26:03.198 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:02 smithi107 ceph-mon[26355]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:26:04.276 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:03.289+0000 7f47dd8ee700 1 Processor -- start 2024-04-22T23:26:04.277 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:03.289+0000 7f47dd8ee700 1 -- start start 2024-04-22T23:26:04.277 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:03.289+0000 7f47dd8ee700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47d80712e0 0x7f47d8105510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:04.277 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:03.289+0000 7f47dd8ee700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d80716f0 con 0x7f47d80712e0 2024-04-22T23:26:04.277 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:03.290+0000 7f47d6ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47d80712e0 0x7f47d8105510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:04.277 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:03.290+0000 7f47d6ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47d80712e0 0x7f47d8105510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34316/0 (socket says 172.21.15.107:34316) 2024-04-22T23:26:04.277 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:03.290+0000 7f47d6ffd700 1 -- 172.21.15.107:0/1936594783 learned_addr learned my addr 172.21.15.107:0/1936594783 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:26:04.277 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:03.290+0000 7f47d6ffd700 1 -- 172.21.15.107:0/1936594783 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d8105a50 con 0x7f47d80712e0 2024-04-22T23:26:04.277 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:03.290+0000 7f47d6ffd700 1 --2- 172.21.15.107:0/1936594783 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47d80712e0 0x7f47d8105510 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f47cc0098e0 tx=0x7f47cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcc5268db3bb41fe server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:04.277 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:03.290+0000 7f47d5ffb700 1 -- 172.21.15.107:0/1936594783 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f47cc00b860 con 0x7f47d80712e0 2024-04-22T23:26:04.277 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:03.290+0000 7f47d5ffb700 1 -- 172.21.15.107:0/1936594783 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f47cc00be50 con 0x7f47d80712e0 2024-04-22T23:26:04.277 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:03.290+0000 7f47d5ffb700 1 -- 172.21.15.107:0/1936594783 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f47cc010380 con 0x7f47d80712e0 2024-04-22T23:26:04.277 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:03.291+0000 7f47dd8ee700 1 -- 172.21.15.107:0/1936594783 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47d80712e0 msgr2=0x7f47d8105510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:04.278 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:03.291+0000 7f47dd8ee700 1 --2- 172.21.15.107:0/1936594783 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47d80712e0 0x7f47d8105510 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f47cc0098e0 tx=0x7f47cc009250 comp rx=0 tx=0).stop 2024-04-22T23:26:04.278 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:03.291+0000 7f47dd8ee700 1 -- 172.21.15.107:0/1936594783 shutdown_connections 2024-04-22T23:26:04.278 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:03.291+0000 7f47dd8ee700 1 --2- 172.21.15.107:0/1936594783 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47d80712e0 0x7f47d8105510 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:04.278 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:03.291+0000 7f47dd8ee700 1 -- 172.21.15.107:0/1936594783 >> 172.21.15.107:0/1936594783 conn(0x7f47d80fcf80 msgr2=0x7f47d80ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:26:04.278 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:03.291+0000 7f47dd8ee700 1 -- 172.21.15.107:0/1936594783 shutdown_connections 2024-04-22T23:26:04.278 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:03.291+0000 7f47dd8ee700 1 -- 172.21.15.107:0/1936594783 wait complete. 2024-04-22T23:26:04.278 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:03.291+0000 7f47dd8ee700 1 Processor -- start 2024-04-22T23:26:04.278 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:03.292+0000 7f47dd8ee700 1 -- start start 2024-04-22T23:26:04.278 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:03.292+0000 7f47dd8ee700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47d80712e0 0x7f47d8102e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:04.278 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:03.292+0000 7f47dd8ee700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d81033a0 con 0x7f47d80712e0 2024-04-22T23:26:04.278 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:03.292+0000 7f47d6ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47d80712e0 0x7f47d8102e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:04.278 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:03.292+0000 7f47d6ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47d80712e0 0x7f47d8102e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34318/0 (socket says 172.21.15.107:34318) 2024-04-22T23:26:04.278 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:03.292+0000 7f47d6ffd700 1 -- 172.21.15.107:0/985104033 learned_addr learned my addr 172.21.15.107:0/985104033 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:26:04.279 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:03.292+0000 7f47d6ffd700 1 -- 172.21.15.107:0/985104033 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47cc0094f0 con 0x7f47d80712e0 2024-04-22T23:26:04.279 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:03.292+0000 7f47d6ffd700 1 --2- 172.21.15.107:0/985104033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47d80712e0 0x7f47d8102e60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f47cc011bd0 tx=0x7f47cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:04.279 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:03.292+0000 7f47bf7fe700 1 -- 172.21.15.107:0/985104033 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f47cc0107c0 con 0x7f47d80712e0 2024-04-22T23:26:04.279 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:03.292+0000 7f47bf7fe700 1 -- 172.21.15.107:0/985104033 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f47cc010db0 con 0x7f47d80712e0 2024-04-22T23:26:04.279 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:03.292+0000 7f47bf7fe700 1 -- 172.21.15.107:0/985104033 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f47cc01ada0 con 0x7f47d80712e0 2024-04-22T23:26:04.279 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:03.292+0000 7f47dd8ee700 1 -- 172.21.15.107:0/985104033 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47d8101410 con 0x7f47d80712e0 2024-04-22T23:26:04.279 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:03.292+0000 7f47dd8ee700 1 -- 172.21.15.107:0/985104033 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47d8101900 con 0x7f47d80712e0 2024-04-22T23:26:04.279 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:03.293+0000 7f47bf7fe700 1 -- 172.21.15.107:0/985104033 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f47cc010920 con 0x7f47d80712e0 2024-04-22T23:26:04.279 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:03.293+0000 7f47bf7fe700 1 --2- 172.21.15.107:0/985104033 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f47c00367e0 0x7f47c0038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:04.279 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:03.293+0000 7f47bf7fe700 1 -- 172.21.15.107:0/985104033 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f47cc04ab00 con 0x7f47d80712e0 2024-04-22T23:26:04.279 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:03.293+0000 7f47dd8ee700 1 -- 172.21.15.107:0/985104033 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47d804bed0 con 0x7f47d80712e0 2024-04-22T23:26:04.279 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:03.294+0000 7f47d67fc700 1 --2- 172.21.15.107:0/985104033 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f47c00367e0 0x7f47c0038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:04.279 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:03.294+0000 7f47d67fc700 1 --2- 172.21.15.107:0/985104033 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f47c00367e0 0x7f47c0038c90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f47c8009990 tx=0x7f47c8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:04.280 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:03.298+0000 7f47bf7fe700 1 -- 172.21.15.107:0/985104033 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f47cc01a430 con 0x7f47d80712e0 2024-04-22T23:26:04.280 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:03.483+0000 7f47dd8ee700 1 -- 172.21.15.107:0/985104033 --> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f47d8101db0 con 0x7f47c00367e0 2024-04-22T23:26:04.280 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:03.711+0000 7f47bf7fe700 1 -- 172.21.15.107:0/985104033 <== mgr.14118 v2:172.21.15.107:6800/307474287 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f47d8101db0 con 0x7f47c00367e0 2024-04-22T23:26:04.280 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:03.713+0000 7f47dd8ee700 1 -- 172.21.15.107:0/985104033 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f47c00367e0 msgr2=0x7f47c0038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:04.280 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:03.713+0000 7f47dd8ee700 1 --2- 172.21.15.107:0/985104033 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f47c00367e0 0x7f47c0038c90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f47c8009990 tx=0x7f47c8006e40 comp rx=0 tx=0).stop 2024-04-22T23:26:04.280 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:03.713+0000 7f47dd8ee700 1 -- 172.21.15.107:0/985104033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47d80712e0 msgr2=0x7f47d8102e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:04.280 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:03.713+0000 7f47dd8ee700 1 --2- 172.21.15.107:0/985104033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47d80712e0 0x7f47d8102e60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f47cc011bd0 tx=0x7f47cc009250 comp rx=0 tx=0).stop 2024-04-22T23:26:04.280 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:03.714+0000 7f47dd8ee700 1 -- 172.21.15.107:0/985104033 shutdown_connections 2024-04-22T23:26:04.280 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:03.714+0000 7f47dd8ee700 1 --2- 172.21.15.107:0/985104033 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f47c00367e0 0x7f47c0038c90 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:04.280 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:03.714+0000 7f47dd8ee700 1 --2- 172.21.15.107:0/985104033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47d80712e0 0x7f47d8102e60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:04.280 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:03.714+0000 7f47dd8ee700 1 -- 172.21.15.107:0/985104033 >> 172.21.15.107:0/985104033 conn(0x7f47d80fcf80 msgr2=0x7f47d80fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:26:04.280 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:03.714+0000 7f47dd8ee700 1 -- 172.21.15.107:0/985104033 shutdown_connections 2024-04-22T23:26:04.280 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:03.714+0000 7f47dd8ee700 1 -- 172.21.15.107:0/985104033 wait complete. 2024-04-22T23:26:04.885 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:04 smithi107 ceph-mon[26355]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:26:04.885 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:04 smithi107 ceph-mon[26355]: Generating ssh key... 2024-04-22T23:26:04.885 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:04 smithi107 ceph-mon[26355]: from='mgr.14118 172.21.15.107:0/4176396201' entity='mgr.smithi107.uipxvm' 2024-04-22T23:26:04.885 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:04 smithi107 ceph-mon[26355]: from='mgr.14118 172.21.15.107:0/4176396201' entity='mgr.smithi107.uipxvm' 2024-04-22T23:26:05.475 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1WWJuwZTZzGSBHUvFJLoPGJ5EFcXCcshdrIUBKc5qGJzAS2XSOuel0GZnv4ZfjQ8v9flO/XgV+Dv6WLYUbw2CqiEq4kp3dWiauXSh/87wXychQeDCY39g2OQfSokyX2BfKW3cnp62pTsEP5cDP5+elbXFz5/932saD8SAx8R/dsa88XLuVSar4Rt9DIkWHoqwqqYu5hFTtBKbDb1D2gTrTSs6rYXr5+J3qjM1hpjZzF+xw3eg6uimfVFFjmNpvU7tGum3SXtmFrFSfohJcN8p9bXgrBNsBYMNQUkT6dlDF1Sl3Z2Trg64ncGCC0lusUOyEi4Uuh+7W7amYVp7AZSUs+/80EsNGBARvVSVtzpmsA+GXn24sq/EzTumJ91K0lesNX+7rPsLP98KRsVsLACkwWVPD9SctKbx0BUjJfu4vI7QzhS9rLgFgQ6l2Xl/xEL2/PNzN8vtbKSbbDR5shRMhZu9OBW9DXuw8BsdhOg1BNmXZbJwjTi+2BnltAsw9Rk= ceph-8252f6f2-00ff-11ef-bc93-c7b262605968 2024-04-22T23:26:05.476 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:04.684+0000 7f29eaf37700 1 Processor -- start 2024-04-22T23:26:05.476 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:04.684+0000 7f29eaf37700 1 -- start start 2024-04-22T23:26:05.476 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:04.684+0000 7f29eaf37700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29e4078bf0 0x7f29e4077280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:05.476 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:04.684+0000 7f29eaf37700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e4079000 con 0x7f29e4078bf0 2024-04-22T23:26:05.476 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:04.684+0000 7f29e8cd3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29e4078bf0 0x7f29e4077280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:05.476 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:04.684+0000 7f29e8cd3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29e4078bf0 0x7f29e4077280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34330/0 (socket says 172.21.15.107:34330) 2024-04-22T23:26:05.476 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:04.684+0000 7f29e8cd3700 1 -- 172.21.15.107:0/1166081269 learned_addr learned my addr 172.21.15.107:0/1166081269 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:26:05.476 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:04.684+0000 7f29e8cd3700 1 -- 172.21.15.107:0/1166081269 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e4079140 con 0x7f29e4078bf0 2024-04-22T23:26:05.476 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:04.685+0000 7f29e8cd3700 1 --2- 172.21.15.107:0/1166081269 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29e4078bf0 0x7f29e4077280 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f29dc009e50 tx=0x7f29dc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a19d9e1adf79f620 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:05.476 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:04.685+0000 7f29e37fe700 1 -- 172.21.15.107:0/1166081269 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f29dc0092e0 con 0x7f29e4078bf0 2024-04-22T23:26:05.476 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:04.685+0000 7f29e37fe700 1 -- 172.21.15.107:0/1166081269 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f29dc012430 con 0x7f29e4078bf0 2024-04-22T23:26:05.476 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:04.685+0000 7f29e37fe700 1 -- 172.21.15.107:0/1166081269 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f29dc0075c0 con 0x7f29e4078bf0 2024-04-22T23:26:05.477 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:04.685+0000 7f29eaf37700 1 -- 172.21.15.107:0/1166081269 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29e4078bf0 msgr2=0x7f29e4077280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:05.477 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:04.685+0000 7f29eaf37700 1 --2- 172.21.15.107:0/1166081269 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29e4078bf0 0x7f29e4077280 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f29dc009e50 tx=0x7f29dc009840 comp rx=0 tx=0).stop 2024-04-22T23:26:05.477 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:04.685+0000 7f29eaf37700 1 -- 172.21.15.107:0/1166081269 shutdown_connections 2024-04-22T23:26:05.477 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:04.685+0000 7f29eaf37700 1 --2- 172.21.15.107:0/1166081269 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29e4078bf0 0x7f29e4077280 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:05.477 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:04.685+0000 7f29eaf37700 1 -- 172.21.15.107:0/1166081269 >> 172.21.15.107:0/1166081269 conn(0x7f29e40fe140 msgr2=0x7f29e4100550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:26:05.477 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:04.685+0000 7f29eaf37700 1 -- 172.21.15.107:0/1166081269 shutdown_connections 2024-04-22T23:26:05.477 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:04.686+0000 7f29eaf37700 1 -- 172.21.15.107:0/1166081269 wait complete. 2024-04-22T23:26:05.477 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:04.686+0000 7f29eaf37700 1 Processor -- start 2024-04-22T23:26:05.477 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:04.686+0000 7f29eaf37700 1 -- start start 2024-04-22T23:26:05.477 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:04.686+0000 7f29eaf37700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29e4078bf0 0x7f29e4113930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:05.477 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:04.686+0000 7f29eaf37700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e4113e70 con 0x7f29e4078bf0 2024-04-22T23:26:05.477 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:04.686+0000 7f29e8cd3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29e4078bf0 0x7f29e4113930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:05.477 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:04.686+0000 7f29e8cd3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29e4078bf0 0x7f29e4113930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34344/0 (socket says 172.21.15.107:34344) 2024-04-22T23:26:05.477 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:04.686+0000 7f29e8cd3700 1 -- 172.21.15.107:0/3618305620 learned_addr learned my addr 172.21.15.107:0/3618305620 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:26:05.478 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:04.687+0000 7f29e8cd3700 1 -- 172.21.15.107:0/3618305620 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29dc009ae0 con 0x7f29e4078bf0 2024-04-22T23:26:05.478 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:04.687+0000 7f29e8cd3700 1 --2- 172.21.15.107:0/3618305620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29e4078bf0 0x7f29e4113930 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f29dc009fd0 tx=0x7f29dc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:05.478 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:04.687+0000 7f29e17fa700 1 -- 172.21.15.107:0/3618305620 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f29dc0092e0 con 0x7f29e4078bf0 2024-04-22T23:26:05.478 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:04.687+0000 7f29e17fa700 1 -- 172.21.15.107:0/3618305620 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f29dc007cc0 con 0x7f29e4078bf0 2024-04-22T23:26:05.478 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:04.687+0000 7f29eaf37700 1 -- 172.21.15.107:0/3618305620 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29e4114070 con 0x7f29e4078bf0 2024-04-22T23:26:05.478 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:04.687+0000 7f29e17fa700 1 -- 172.21.15.107:0/3618305620 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f29dc01c970 con 0x7f29e4078bf0 2024-04-22T23:26:05.478 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:04.687+0000 7f29eaf37700 1 -- 172.21.15.107:0/3618305620 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29e4114be0 con 0x7f29e4078bf0 2024-04-22T23:26:05.478 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:04.688+0000 7f29e17fa700 1 -- 172.21.15.107:0/3618305620 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f29dc019070 con 0x7f29e4078bf0 2024-04-22T23:26:05.478 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:04.688+0000 7f29e17fa700 1 --2- 172.21.15.107:0/3618305620 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f29d00367e0 0x7f29d0038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:05.478 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:04.688+0000 7f29e17fa700 1 -- 172.21.15.107:0/3618305620 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f29dc049eb0 con 0x7f29e4078bf0 2024-04-22T23:26:05.478 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:04.688+0000 7f29eaf37700 1 -- 172.21.15.107:0/3618305620 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29e405f5f0 con 0x7f29e4078bf0 2024-04-22T23:26:05.478 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:04.688+0000 7f29e3fff700 1 --2- 172.21.15.107:0/3618305620 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f29d00367e0 0x7f29d0038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:05.478 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:04.689+0000 7f29e3fff700 1 --2- 172.21.15.107:0/3618305620 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f29d00367e0 0x7f29d0038c90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f29cc009990 tx=0x7f29cc006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:05.478 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:04.694+0000 7f29e17fa700 1 -- 172.21.15.107:0/3618305620 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f29dc017080 con 0x7f29e4078bf0 2024-04-22T23:26:05.479 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:04.883+0000 7f29eaf37700 1 -- 172.21.15.107:0/3618305620 --> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f29e40ff4b0 con 0x7f29d00367e0 2024-04-22T23:26:05.479 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:04.883+0000 7f29e17fa700 1 -- 172.21.15.107:0/3618305620 <== mgr.14118 v2:172.21.15.107:6800/307474287 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f29e40ff4b0 con 0x7f29d00367e0 2024-04-22T23:26:05.479 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:04.886+0000 7f29eaf37700 1 -- 172.21.15.107:0/3618305620 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f29d00367e0 msgr2=0x7f29d0038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:05.479 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:04.886+0000 7f29eaf37700 1 --2- 172.21.15.107:0/3618305620 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f29d00367e0 0x7f29d0038c90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f29cc009990 tx=0x7f29cc006e40 comp rx=0 tx=0).stop 2024-04-22T23:26:05.479 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:04.886+0000 7f29eaf37700 1 -- 172.21.15.107:0/3618305620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29e4078bf0 msgr2=0x7f29e4113930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:05.479 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:04.886+0000 7f29eaf37700 1 --2- 172.21.15.107:0/3618305620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29e4078bf0 0x7f29e4113930 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f29dc009fd0 tx=0x7f29dc009840 comp rx=0 tx=0).stop 2024-04-22T23:26:05.479 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:04.886+0000 7f29eaf37700 1 -- 172.21.15.107:0/3618305620 shutdown_connections 2024-04-22T23:26:05.479 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:04.886+0000 7f29eaf37700 1 --2- 172.21.15.107:0/3618305620 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f29d00367e0 0x7f29d0038c90 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:05.479 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:04.886+0000 7f29eaf37700 1 --2- 172.21.15.107:0/3618305620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29e4078bf0 0x7f29e4113930 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:05.479 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:04.886+0000 7f29eaf37700 1 -- 172.21.15.107:0/3618305620 >> 172.21.15.107:0/3618305620 conn(0x7f29e40fe140 msgr2=0x7f29e40fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:26:05.479 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:04.886+0000 7f29eaf37700 1 -- 172.21.15.107:0/3618305620 shutdown_connections 2024-04-22T23:26:05.479 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:04.886+0000 7f29eaf37700 1 -- 172.21.15.107:0/3618305620 wait complete. 2024-04-22T23:26:05.479 INFO:teuthology.orchestra.run.smithi107.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-22T23:26:05.479 INFO:teuthology.orchestra.run.smithi107.stdout:Adding key to root@localhost authorized_keys... 2024-04-22T23:26:05.479 INFO:teuthology.orchestra.run.smithi107.stdout:Adding host smithi107... 2024-04-22T23:26:05.990 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:05 smithi107 ceph-mon[26355]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:26:06.905 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:06 smithi107 ceph-mon[26355]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi107", "addr": "172.21.15.107", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:26:07.990 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:07 smithi107 ceph-mon[26355]: Deploying cephadm binary to smithi107 2024-04-22T23:26:08.458 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Added host 'smithi107' with addr '172.21.15.107' 2024-04-22T23:26:08.458 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:05.857+0000 7ffb4c31d700 1 Processor -- start 2024-04-22T23:26:08.459 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:05.857+0000 7ffb4c31d700 1 -- start start 2024-04-22T23:26:08.459 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:05.857+0000 7ffb4c31d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb44078b90 0x7ffb44077220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:08.459 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:05.857+0000 7ffb4c31d700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb44078fa0 con 0x7ffb44078b90 2024-04-22T23:26:08.459 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:05.857+0000 7ffb4a0b9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb44078b90 0x7ffb44077220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:08.459 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:05.857+0000 7ffb4a0b9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb44078b90 0x7ffb44077220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34352/0 (socket says 172.21.15.107:34352) 2024-04-22T23:26:08.459 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:05.857+0000 7ffb4a0b9700 1 -- 172.21.15.107:0/2159425297 learned_addr learned my addr 172.21.15.107:0/2159425297 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:26:08.459 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:05.857+0000 7ffb4a0b9700 1 -- 172.21.15.107:0/2159425297 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb440790e0 con 0x7ffb44078b90 2024-04-22T23:26:08.459 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:05.858+0000 7ffb4a0b9700 1 --2- 172.21.15.107:0/2159425297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb44078b90 0x7ffb44077220 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ffb400098e0 tx=0x7ffb40009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e945db7a2d954be server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:08.459 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:05.858+0000 7ffb490b7700 1 -- 172.21.15.107:0/2159425297 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffb4000b8f0 con 0x7ffb44078b90 2024-04-22T23:26:08.459 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:05.858+0000 7ffb490b7700 1 -- 172.21.15.107:0/2159425297 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7ffb40011420 con 0x7ffb44078b90 2024-04-22T23:26:08.459 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:05.858+0000 7ffb490b7700 1 -- 172.21.15.107:0/2159425297 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffb40010540 con 0x7ffb44078b90 2024-04-22T23:26:08.459 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:05.858+0000 7ffb4c31d700 1 -- 172.21.15.107:0/2159425297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb44078b90 msgr2=0x7ffb44077220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:08.459 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:05.858+0000 7ffb4c31d700 1 --2- 172.21.15.107:0/2159425297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb44078b90 0x7ffb44077220 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ffb400098e0 tx=0x7ffb40009250 comp rx=0 tx=0).stop 2024-04-22T23:26:08.460 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:05.858+0000 7ffb4c31d700 1 -- 172.21.15.107:0/2159425297 shutdown_connections 2024-04-22T23:26:08.460 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:05.858+0000 7ffb4c31d700 1 --2- 172.21.15.107:0/2159425297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb44078b90 0x7ffb44077220 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:08.460 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:05.858+0000 7ffb4c31d700 1 -- 172.21.15.107:0/2159425297 >> 172.21.15.107:0/2159425297 conn(0x7ffb440fe140 msgr2=0x7ffb44100550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:26:08.460 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:05.858+0000 7ffb4c31d700 1 -- 172.21.15.107:0/2159425297 shutdown_connections 2024-04-22T23:26:08.460 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:05.858+0000 7ffb4c31d700 1 -- 172.21.15.107:0/2159425297 wait complete. 2024-04-22T23:26:08.460 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:05.859+0000 7ffb4c31d700 1 Processor -- start 2024-04-22T23:26:08.460 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:05.859+0000 7ffb4c31d700 1 -- start start 2024-04-22T23:26:08.460 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:05.859+0000 7ffb4c31d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb44078b90 0x7ffb44113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:08.460 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:05.859+0000 7ffb4c31d700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb44113eb0 con 0x7ffb44078b90 2024-04-22T23:26:08.460 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:05.859+0000 7ffb4a0b9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb44078b90 0x7ffb44113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:08.460 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:05.859+0000 7ffb4a0b9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb44078b90 0x7ffb44113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34368/0 (socket says 172.21.15.107:34368) 2024-04-22T23:26:08.460 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:05.859+0000 7ffb4a0b9700 1 -- 172.21.15.107:0/2161556258 learned_addr learned my addr 172.21.15.107:0/2161556258 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:26:08.460 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:05.859+0000 7ffb4a0b9700 1 -- 172.21.15.107:0/2161556258 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb400094f0 con 0x7ffb44078b90 2024-04-22T23:26:08.460 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:05.860+0000 7ffb4a0b9700 1 --2- 172.21.15.107:0/2161556258 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb44078b90 0x7ffb44113970 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ffb400094c0 tx=0x7ffb40009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:08.461 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:05.860+0000 7ffb3affd700 1 -- 172.21.15.107:0/2161556258 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffb40010800 con 0x7ffb44078b90 2024-04-22T23:26:08.461 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:05.860+0000 7ffb3affd700 1 -- 172.21.15.107:0/2161556258 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7ffb40010df0 con 0x7ffb44078b90 2024-04-22T23:26:08.461 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:05.860+0000 7ffb4c31d700 1 -- 172.21.15.107:0/2161556258 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb441140b0 con 0x7ffb44078b90 2024-04-22T23:26:08.461 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:05.860+0000 7ffb3affd700 1 -- 172.21.15.107:0/2161556258 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffb40024e50 con 0x7ffb44078b90 2024-04-22T23:26:08.461 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:05.860+0000 7ffb4c31d700 1 -- 172.21.15.107:0/2161556258 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb44114c20 con 0x7ffb44078b90 2024-04-22T23:26:08.461 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:05.861+0000 7ffb3affd700 1 -- 172.21.15.107:0/2161556258 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7ffb40010960 con 0x7ffb44078b90 2024-04-22T23:26:08.461 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:05.861+0000 7ffb4c31d700 1 -- 172.21.15.107:0/2161556258 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb4405f5f0 con 0x7ffb44078b90 2024-04-22T23:26:08.461 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:05.861+0000 7ffb3affd700 1 --2- 172.21.15.107:0/2161556258 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7ffb30036790 0x7ffb30038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:08.461 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:05.861+0000 7ffb3affd700 1 -- 172.21.15.107:0/2161556258 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7ffb4004a550 con 0x7ffb44078b90 2024-04-22T23:26:08.461 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:05.861+0000 7ffb498b8700 1 --2- 172.21.15.107:0/2161556258 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7ffb30036790 0x7ffb30038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:08.461 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:05.861+0000 7ffb498b8700 1 --2- 172.21.15.107:0/2161556258 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7ffb30036790 0x7ffb30038c40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ffb34009990 tx=0x7ffb34006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:08.461 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:05.865+0000 7ffb3affd700 1 -- 172.21.15.107:0/2161556258 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ffb4002d3f0 con 0x7ffb44078b90 2024-04-22T23:26:08.461 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:06.053+0000 7ffb4c31d700 1 -- 172.21.15.107:0/2161556258 --> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi107", "addr": "172.21.15.107", "target": ["mon-mgr", ""]}) v1 -- 0x7ffb440ff560 con 0x7ffb30036790 2024-04-22T23:26:08.462 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:07.892+0000 7ffb3affd700 1 -- 172.21.15.107:0/2161556258 <== mgr.14118 v2:172.21.15.107:6800/307474287 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7ffb440ff560 con 0x7ffb30036790 2024-04-22T23:26:08.462 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:07.895+0000 7ffb4c31d700 1 -- 172.21.15.107:0/2161556258 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7ffb30036790 msgr2=0x7ffb30038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:08.462 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:07.895+0000 7ffb4c31d700 1 --2- 172.21.15.107:0/2161556258 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7ffb30036790 0x7ffb30038c40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ffb34009990 tx=0x7ffb34006e40 comp rx=0 tx=0).stop 2024-04-22T23:26:08.462 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:07.895+0000 7ffb4c31d700 1 -- 172.21.15.107:0/2161556258 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb44078b90 msgr2=0x7ffb44113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:08.462 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:07.895+0000 7ffb4c31d700 1 --2- 172.21.15.107:0/2161556258 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb44078b90 0x7ffb44113970 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ffb400094c0 tx=0x7ffb40009250 comp rx=0 tx=0).stop 2024-04-22T23:26:08.462 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:07.895+0000 7ffb4c31d700 1 -- 172.21.15.107:0/2161556258 shutdown_connections 2024-04-22T23:26:08.462 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:07.895+0000 7ffb4c31d700 1 --2- 172.21.15.107:0/2161556258 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7ffb30036790 0x7ffb30038c40 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:08.462 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:07.895+0000 7ffb4c31d700 1 --2- 172.21.15.107:0/2161556258 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb44078b90 0x7ffb44113970 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:08.462 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:07.895+0000 7ffb4c31d700 1 -- 172.21.15.107:0/2161556258 >> 172.21.15.107:0/2161556258 conn(0x7ffb440fe140 msgr2=0x7ffb440fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:26:08.462 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:07.895+0000 7ffb4c31d700 1 -- 172.21.15.107:0/2161556258 shutdown_connections 2024-04-22T23:26:08.462 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:07.895+0000 7ffb4c31d700 1 -- 172.21.15.107:0/2161556258 wait complete. 2024-04-22T23:26:08.462 INFO:teuthology.orchestra.run.smithi107.stdout:Deploying mon service with default placement... 2024-04-22T23:26:09.133 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:08 smithi107 ceph-mon[26355]: from='mgr.14118 172.21.15.107:0/4176396201' entity='mgr.smithi107.uipxvm' 2024-04-22T23:26:09.133 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:08 smithi107 ceph-mon[26355]: Added host smithi107 2024-04-22T23:26:09.133 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:08 smithi107 ceph-mon[26355]: from='mgr.14118 172.21.15.107:0/4176396201' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:26:09.959 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-22T23:26:09.960 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:08.900+0000 7fc0eb30c700 1 Processor -- start 2024-04-22T23:26:09.960 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:08.900+0000 7fc0eb30c700 1 -- start start 2024-04-22T23:26:09.960 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:08.900+0000 7fc0eb30c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0e4100d10 0x7fc0e4101120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:09.960 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:08.900+0000 7fc0eb30c700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e4101660 con 0x7fc0e4100d10 2024-04-22T23:26:09.960 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:08.900+0000 7fc0e90a8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0e4100d10 0x7fc0e4101120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:09.960 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:08.900+0000 7fc0e90a8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0e4100d10 0x7fc0e4101120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34374/0 (socket says 172.21.15.107:34374) 2024-04-22T23:26:09.960 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:08.900+0000 7fc0e90a8700 1 -- 172.21.15.107:0/2680642089 learned_addr learned my addr 172.21.15.107:0/2680642089 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:26:09.960 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:08.900+0000 7fc0e90a8700 1 -- 172.21.15.107:0/2680642089 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0e41017a0 con 0x7fc0e4100d10 2024-04-22T23:26:09.960 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:08.901+0000 7fc0e90a8700 1 --2- 172.21.15.107:0/2680642089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0e4100d10 0x7fc0e4101120 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc0e00098e0 tx=0x7fc0e0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2e0f1f050d7cc4a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:09.960 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:08.901+0000 7fc0dbfff700 1 -- 172.21.15.107:0/2680642089 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc0e000b860 con 0x7fc0e4100d10 2024-04-22T23:26:09.960 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:08.901+0000 7fc0dbfff700 1 -- 172.21.15.107:0/2680642089 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fc0e000be50 con 0x7fc0e4100d10 2024-04-22T23:26:09.961 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:08.901+0000 7fc0dbfff700 1 -- 172.21.15.107:0/2680642089 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc0e0010380 con 0x7fc0e4100d10 2024-04-22T23:26:09.961 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:08.901+0000 7fc0eb30c700 1 -- 172.21.15.107:0/2680642089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0e4100d10 msgr2=0x7fc0e4101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:09.961 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:08.901+0000 7fc0eb30c700 1 --2- 172.21.15.107:0/2680642089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0e4100d10 0x7fc0e4101120 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc0e00098e0 tx=0x7fc0e0009250 comp rx=0 tx=0).stop 2024-04-22T23:26:09.961 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:08.901+0000 7fc0eb30c700 1 -- 172.21.15.107:0/2680642089 shutdown_connections 2024-04-22T23:26:09.961 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:08.901+0000 7fc0eb30c700 1 --2- 172.21.15.107:0/2680642089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0e4100d10 0x7fc0e4101120 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:09.961 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:08.901+0000 7fc0eb30c700 1 -- 172.21.15.107:0/2680642089 >> 172.21.15.107:0/2680642089 conn(0x7fc0e40fc770 msgr2=0x7fc0e40feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:26:09.961 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:08.901+0000 7fc0eb30c700 1 -- 172.21.15.107:0/2680642089 shutdown_connections 2024-04-22T23:26:09.961 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:08.902+0000 7fc0eb30c700 1 -- 172.21.15.107:0/2680642089 wait complete. 2024-04-22T23:26:09.961 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:08.902+0000 7fc0eb30c700 1 Processor -- start 2024-04-22T23:26:09.961 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:08.902+0000 7fc0eb30c700 1 -- start start 2024-04-22T23:26:09.961 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:08.902+0000 7fc0eb30c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0e4100d10 0x7fc0e4103740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:09.961 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:08.902+0000 7fc0eb30c700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e4103c80 con 0x7fc0e4100d10 2024-04-22T23:26:09.961 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:08.902+0000 7fc0e90a8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0e4100d10 0x7fc0e4103740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:09.961 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:08.902+0000 7fc0e90a8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0e4100d10 0x7fc0e4103740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34384/0 (socket says 172.21.15.107:34384) 2024-04-22T23:26:09.962 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:08.902+0000 7fc0e90a8700 1 -- 172.21.15.107:0/3849061034 learned_addr learned my addr 172.21.15.107:0/3849061034 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:26:09.962 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:08.902+0000 7fc0e90a8700 1 -- 172.21.15.107:0/3849061034 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0e00094f0 con 0x7fc0e4100d10 2024-04-22T23:26:09.962 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:08.903+0000 7fc0e90a8700 1 --2- 172.21.15.107:0/3849061034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0e4100d10 0x7fc0e4103740 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0006b00 tx=0x7fc0e0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:09.962 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:08.903+0000 7fc0d9ffb700 1 -- 172.21.15.107:0/3849061034 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc0e0010760 con 0x7fc0e4100d10 2024-04-22T23:26:09.962 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:08.903+0000 7fc0d9ffb700 1 -- 172.21.15.107:0/3849061034 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fc0e0010d50 con 0x7fc0e4100d10 2024-04-22T23:26:09.962 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:08.903+0000 7fc0d9ffb700 1 -- 172.21.15.107:0/3849061034 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc0e001ad40 con 0x7fc0e4100d10 2024-04-22T23:26:09.962 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:08.903+0000 7fc0eb30c700 1 -- 172.21.15.107:0/3849061034 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0e4101cf0 con 0x7fc0e4100d10 2024-04-22T23:26:09.962 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:08.903+0000 7fc0eb30c700 1 -- 172.21.15.107:0/3849061034 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0e41021e0 con 0x7fc0e4100d10 2024-04-22T23:26:09.962 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:08.904+0000 7fc0d9ffb700 1 -- 172.21.15.107:0/3849061034 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fc0e00108c0 con 0x7fc0e4100d10 2024-04-22T23:26:09.962 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:08.904+0000 7fc0eb30c700 1 -- 172.21.15.107:0/3849061034 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0e404bed0 con 0x7fc0e4100d10 2024-04-22T23:26:09.962 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:08.904+0000 7fc0d9ffb700 1 --2- 172.21.15.107:0/3849061034 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7fc0d00367e0 0x7fc0d0038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:09.962 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:08.904+0000 7fc0d9ffb700 1 -- 172.21.15.107:0/3849061034 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fc0e004a510 con 0x7fc0e4100d10 2024-04-22T23:26:09.962 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:08.904+0000 7fc0e88a7700 1 --2- 172.21.15.107:0/3849061034 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7fc0d00367e0 0x7fc0d0038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:09.962 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:08.904+0000 7fc0e88a7700 1 --2- 172.21.15.107:0/3849061034 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7fc0d00367e0 0x7fc0d0038c90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4009990 tx=0x7fc0d4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:09.963 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:08.909+0000 7fc0d9ffb700 1 -- 172.21.15.107:0/3849061034 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fc0e0018b40 con 0x7fc0e4100d10 2024-04-22T23:26:09.963 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:09.132+0000 7fc0eb30c700 1 -- 172.21.15.107:0/3849061034 --> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7fc0e41029c0 con 0x7fc0d00367e0 2024-04-22T23:26:09.963 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:09.139+0000 7fc0d9ffb700 1 -- 172.21.15.107:0/3849061034 <== mgr.14118 v2:172.21.15.107:6800/307474287 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fc0e41029c0 con 0x7fc0d00367e0 2024-04-22T23:26:09.963 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:09.141+0000 7fc0eb30c700 1 -- 172.21.15.107:0/3849061034 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7fc0d00367e0 msgr2=0x7fc0d0038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:09.963 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:09.141+0000 7fc0eb30c700 1 --2- 172.21.15.107:0/3849061034 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7fc0d00367e0 0x7fc0d0038c90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4009990 tx=0x7fc0d4006e40 comp rx=0 tx=0).stop 2024-04-22T23:26:09.963 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:09.141+0000 7fc0eb30c700 1 -- 172.21.15.107:0/3849061034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0e4100d10 msgr2=0x7fc0e4103740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:09.963 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:09.141+0000 7fc0eb30c700 1 --2- 172.21.15.107:0/3849061034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0e4100d10 0x7fc0e4103740 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0006b00 tx=0x7fc0e0009250 comp rx=0 tx=0).stop 2024-04-22T23:26:09.963 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:09.141+0000 7fc0eb30c700 1 -- 172.21.15.107:0/3849061034 shutdown_connections 2024-04-22T23:26:09.963 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:09.142+0000 7fc0eb30c700 1 --2- 172.21.15.107:0/3849061034 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7fc0d00367e0 0x7fc0d0038c90 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:09.963 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:09.142+0000 7fc0eb30c700 1 --2- 172.21.15.107:0/3849061034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0e4100d10 0x7fc0e4103740 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:09.963 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:09.142+0000 7fc0eb30c700 1 -- 172.21.15.107:0/3849061034 >> 172.21.15.107:0/3849061034 conn(0x7fc0e40fc770 msgr2=0x7fc0e40fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:26:09.963 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:09.142+0000 7fc0eb30c700 1 -- 172.21.15.107:0/3849061034 shutdown_connections 2024-04-22T23:26:09.963 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:09.142+0000 7fc0eb30c700 1 -- 172.21.15.107:0/3849061034 wait complete. 2024-04-22T23:26:09.963 INFO:teuthology.orchestra.run.smithi107.stdout:Deploying mgr service with default placement... 2024-04-22T23:26:10.414 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:10 smithi107 ceph-mon[26355]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:26:10.414 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:10 smithi107 ceph-mon[26355]: Saving service mon spec with placement count:5 2024-04-22T23:26:10.414 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:10 smithi107 ceph-mon[26355]: from='mgr.14118 172.21.15.107:0/4176396201' entity='mgr.smithi107.uipxvm' 2024-04-22T23:26:11.957 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:11 smithi107 ceph-mon[26355]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:26:11.957 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:11 smithi107 ceph-mon[26355]: Saving service mgr spec with placement count:2 2024-04-22T23:26:11.957 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:11 smithi107 ceph-mon[26355]: from='mgr.14118 172.21.15.107:0/4176396201' entity='mgr.smithi107.uipxvm' 2024-04-22T23:26:11.957 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:11 smithi107 ceph-mon[26355]: from='mgr.14118 172.21.15.107:0/4176396201' entity='mgr.smithi107.uipxvm' 2024-04-22T23:26:11.957 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:11 smithi107 ceph-mon[26355]: from='mgr.14118 172.21.15.107:0/4176396201' entity='mgr.smithi107.uipxvm' 2024-04-22T23:26:11.964 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-22T23:26:11.964 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:10.509+0000 7f1f4936d700 1 Processor -- start 2024-04-22T23:26:11.964 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:10.509+0000 7f1f4936d700 1 -- start start 2024-04-22T23:26:11.964 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:10.509+0000 7f1f4936d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f44100c30 0x7f1f44101040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:11.964 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:10.509+0000 7f1f4936d700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f44101580 con 0x7f1f44100c30 2024-04-22T23:26:11.965 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:10.509+0000 7f1f42ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f44100c30 0x7f1f44101040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:11.965 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:10.509+0000 7f1f42ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f44100c30 0x7f1f44101040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34400/0 (socket says 172.21.15.107:34400) 2024-04-22T23:26:11.965 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:10.509+0000 7f1f42ffd700 1 -- 172.21.15.107:0/211032274 learned_addr learned my addr 172.21.15.107:0/211032274 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:26:11.965 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:10.509+0000 7f1f42ffd700 1 -- 172.21.15.107:0/211032274 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f441016c0 con 0x7f1f44100c30 2024-04-22T23:26:11.965 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:10.509+0000 7f1f42ffd700 1 --2- 172.21.15.107:0/211032274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f44100c30 0x7f1f44101040 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1f340098e0 tx=0x7f1f34009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62278c649c47af7a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:11.965 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:10.510+0000 7f1f41ffb700 1 -- 172.21.15.107:0/211032274 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1f3400b860 con 0x7f1f44100c30 2024-04-22T23:26:11.965 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:10.510+0000 7f1f41ffb700 1 -- 172.21.15.107:0/211032274 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f1f3400be50 con 0x7f1f44100c30 2024-04-22T23:26:11.965 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:10.510+0000 7f1f41ffb700 1 -- 172.21.15.107:0/211032274 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1f34010380 con 0x7f1f44100c30 2024-04-22T23:26:11.965 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:10.510+0000 7f1f4936d700 1 -- 172.21.15.107:0/211032274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f44100c30 msgr2=0x7f1f44101040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:11.965 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:10.510+0000 7f1f4936d700 1 --2- 172.21.15.107:0/211032274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f44100c30 0x7f1f44101040 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1f340098e0 tx=0x7f1f34009250 comp rx=0 tx=0).stop 2024-04-22T23:26:11.965 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:10.510+0000 7f1f4936d700 1 -- 172.21.15.107:0/211032274 shutdown_connections 2024-04-22T23:26:11.965 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:10.510+0000 7f1f4936d700 1 --2- 172.21.15.107:0/211032274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f44100c30 0x7f1f44101040 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:11.965 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:10.510+0000 7f1f4936d700 1 -- 172.21.15.107:0/211032274 >> 172.21.15.107:0/211032274 conn(0x7f1f440fc770 msgr2=0x7f1f440feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:26:11.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:10.510+0000 7f1f4936d700 1 -- 172.21.15.107:0/211032274 shutdown_connections 2024-04-22T23:26:11.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:10.510+0000 7f1f4936d700 1 -- 172.21.15.107:0/211032274 wait complete. 2024-04-22T23:26:11.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:10.511+0000 7f1f4936d700 1 Processor -- start 2024-04-22T23:26:11.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:10.511+0000 7f1f4936d700 1 -- start start 2024-04-22T23:26:11.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:10.511+0000 7f1f4936d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f44100c30 0x7f1f44103660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:11.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:10.511+0000 7f1f4936d700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f44103ba0 con 0x7f1f44100c30 2024-04-22T23:26:11.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:10.511+0000 7f1f42ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f44100c30 0x7f1f44103660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:11.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:10.511+0000 7f1f42ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f44100c30 0x7f1f44103660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34410/0 (socket says 172.21.15.107:34410) 2024-04-22T23:26:11.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:10.511+0000 7f1f42ffd700 1 -- 172.21.15.107:0/3927495940 learned_addr learned my addr 172.21.15.107:0/3927495940 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:26:11.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:10.511+0000 7f1f42ffd700 1 -- 172.21.15.107:0/3927495940 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f340094f0 con 0x7f1f44100c30 2024-04-22T23:26:11.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:10.512+0000 7f1f42ffd700 1 --2- 172.21.15.107:0/3927495940 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f44100c30 0x7f1f44103660 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1f34006b00 tx=0x7f1f34009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:11.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:10.512+0000 7f1f2b7fe700 1 -- 172.21.15.107:0/3927495940 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1f34010760 con 0x7f1f44100c30 2024-04-22T23:26:11.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:10.512+0000 7f1f2b7fe700 1 -- 172.21.15.107:0/3927495940 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f1f34010d50 con 0x7f1f44100c30 2024-04-22T23:26:11.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:10.512+0000 7f1f4936d700 1 -- 172.21.15.107:0/3927495940 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f44101c10 con 0x7f1f44100c30 2024-04-22T23:26:11.967 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:10.512+0000 7f1f2b7fe700 1 -- 172.21.15.107:0/3927495940 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1f3401ad40 con 0x7f1f44100c30 2024-04-22T23:26:11.967 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:10.512+0000 7f1f4936d700 1 -- 172.21.15.107:0/3927495940 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f44102100 con 0x7f1f44100c30 2024-04-22T23:26:11.967 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:10.513+0000 7f1f2b7fe700 1 -- 172.21.15.107:0/3927495940 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f1f3402e440 con 0x7f1f44100c30 2024-04-22T23:26:11.967 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:10.513+0000 7f1f4936d700 1 -- 172.21.15.107:0/3927495940 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f4404bed0 con 0x7f1f44100c30 2024-04-22T23:26:11.967 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:10.513+0000 7f1f2b7fe700 1 --2- 172.21.15.107:0/3927495940 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f1f2c036790 0x7f1f2c038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:11.967 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:10.513+0000 7f1f2b7fe700 1 -- 172.21.15.107:0/3927495940 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f1f3404a590 con 0x7f1f44100c30 2024-04-22T23:26:11.967 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:10.513+0000 7f1f427fc700 1 --2- 172.21.15.107:0/3927495940 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f1f2c036790 0x7f1f2c038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:11.967 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:10.513+0000 7f1f427fc700 1 --2- 172.21.15.107:0/3927495940 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f1f2c036790 0x7f1f2c038c40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1f38009990 tx=0x7f1f38006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:11.967 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:10.517+0000 7f1f2b7fe700 1 -- 172.21.15.107:0/3927495940 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f1f34018940 con 0x7f1f44100c30 2024-04-22T23:26:11.967 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:10.718+0000 7f1f4936d700 1 -- 172.21.15.107:0/3927495940 --> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f1f440fdaf0 con 0x7f1f2c036790 2024-04-22T23:26:11.967 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:10.722+0000 7f1f2b7fe700 1 -- 172.21.15.107:0/3927495940 <== mgr.14118 v2:172.21.15.107:6800/307474287 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f1f440fdaf0 con 0x7f1f2c036790 2024-04-22T23:26:11.967 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:10.725+0000 7f1f4936d700 1 -- 172.21.15.107:0/3927495940 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f1f2c036790 msgr2=0x7f1f2c038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:11.967 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:10.725+0000 7f1f4936d700 1 --2- 172.21.15.107:0/3927495940 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f1f2c036790 0x7f1f2c038c40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1f38009990 tx=0x7f1f38006e40 comp rx=0 tx=0).stop 2024-04-22T23:26:11.967 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:10.725+0000 7f1f4936d700 1 -- 172.21.15.107:0/3927495940 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f44100c30 msgr2=0x7f1f44103660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:11.968 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:10.725+0000 7f1f4936d700 1 --2- 172.21.15.107:0/3927495940 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f44100c30 0x7f1f44103660 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1f34006b00 tx=0x7f1f34009250 comp rx=0 tx=0).stop 2024-04-22T23:26:11.968 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:10.725+0000 7f1f4936d700 1 -- 172.21.15.107:0/3927495940 shutdown_connections 2024-04-22T23:26:11.968 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:10.725+0000 7f1f4936d700 1 --2- 172.21.15.107:0/3927495940 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f1f2c036790 0x7f1f2c038c40 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:11.968 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:10.725+0000 7f1f4936d700 1 --2- 172.21.15.107:0/3927495940 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f44100c30 0x7f1f44103660 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:11.968 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:10.725+0000 7f1f4936d700 1 -- 172.21.15.107:0/3927495940 >> 172.21.15.107:0/3927495940 conn(0x7f1f440fc770 msgr2=0x7f1f440fd320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:26:11.968 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:10.726+0000 7f1f4936d700 1 -- 172.21.15.107:0/3927495940 shutdown_connections 2024-04-22T23:26:11.968 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:10.726+0000 7f1f4936d700 1 -- 172.21.15.107:0/3927495940 wait complete. 2024-04-22T23:26:11.968 INFO:teuthology.orchestra.run.smithi107.stdout:Deploying crash service with default placement... 2024-04-22T23:26:13.399 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-04-22T23:26:13.400 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:12.372+0000 7fa142ee4700 1 Processor -- start 2024-04-22T23:26:13.400 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:12.372+0000 7fa142ee4700 1 -- start start 2024-04-22T23:26:13.400 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:12.373+0000 7fa142ee4700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa13c100d30 0x7fa13c101140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:13.400 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:12.373+0000 7fa142ee4700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c101680 con 0x7fa13c100d30 2024-04-22T23:26:13.400 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:12.373+0000 7fa140c80700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa13c100d30 0x7fa13c101140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:13.400 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:12.373+0000 7fa140c80700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa13c100d30 0x7fa13c101140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34064/0 (socket says 172.21.15.107:34064) 2024-04-22T23:26:13.400 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:12.373+0000 7fa140c80700 1 -- 172.21.15.107:0/3255675432 learned_addr learned my addr 172.21.15.107:0/3255675432 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:26:13.400 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:12.373+0000 7fa140c80700 1 -- 172.21.15.107:0/3255675432 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa13c1017c0 con 0x7fa13c100d30 2024-04-22T23:26:13.400 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:12.373+0000 7fa140c80700 1 --2- 172.21.15.107:0/3255675432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa13c100d30 0x7fa13c101140 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa1300098e0 tx=0x7fa130009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68076fa0e0475a7e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:13.400 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:12.373+0000 7fa13b7fe700 1 -- 172.21.15.107:0/3255675432 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa13000b860 con 0x7fa13c100d30 2024-04-22T23:26:13.400 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:12.373+0000 7fa13b7fe700 1 -- 172.21.15.107:0/3255675432 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa13000be90 con 0x7fa13c100d30 2024-04-22T23:26:13.400 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:12.373+0000 7fa13b7fe700 1 -- 172.21.15.107:0/3255675432 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa1300103c0 con 0x7fa13c100d30 2024-04-22T23:26:13.401 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:12.374+0000 7fa142ee4700 1 -- 172.21.15.107:0/3255675432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa13c100d30 msgr2=0x7fa13c101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:13.401 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:12.374+0000 7fa142ee4700 1 --2- 172.21.15.107:0/3255675432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa13c100d30 0x7fa13c101140 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa1300098e0 tx=0x7fa130009250 comp rx=0 tx=0).stop 2024-04-22T23:26:13.401 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:12.374+0000 7fa142ee4700 1 -- 172.21.15.107:0/3255675432 shutdown_connections 2024-04-22T23:26:13.401 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:12.374+0000 7fa142ee4700 1 --2- 172.21.15.107:0/3255675432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa13c100d30 0x7fa13c101140 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:13.401 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:12.374+0000 7fa142ee4700 1 -- 172.21.15.107:0/3255675432 >> 172.21.15.107:0/3255675432 conn(0x7fa13c0fc730 msgr2=0x7fa13c0feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:26:13.401 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:12.374+0000 7fa142ee4700 1 -- 172.21.15.107:0/3255675432 shutdown_connections 2024-04-22T23:26:13.401 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:12.374+0000 7fa142ee4700 1 -- 172.21.15.107:0/3255675432 wait complete. 2024-04-22T23:26:13.401 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:12.375+0000 7fa142ee4700 1 Processor -- start 2024-04-22T23:26:13.401 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:12.375+0000 7fa142ee4700 1 -- start start 2024-04-22T23:26:13.401 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:12.375+0000 7fa142ee4700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa13c100d30 0x7fa13c103760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:13.401 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:12.375+0000 7fa142ee4700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c103ca0 con 0x7fa13c100d30 2024-04-22T23:26:13.402 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:12.376+0000 7fa140c80700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa13c100d30 0x7fa13c103760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:13.402 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:12.376+0000 7fa140c80700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa13c100d30 0x7fa13c103760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34078/0 (socket says 172.21.15.107:34078) 2024-04-22T23:26:13.402 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:12.376+0000 7fa140c80700 1 -- 172.21.15.107:0/1056557713 learned_addr learned my addr 172.21.15.107:0/1056557713 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:26:13.402 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:12.376+0000 7fa140c80700 1 -- 172.21.15.107:0/1056557713 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1300094f0 con 0x7fa13c100d30 2024-04-22T23:26:13.402 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:12.376+0000 7fa140c80700 1 --2- 172.21.15.107:0/1056557713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa13c100d30 0x7fa13c103760 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa1300090b0 tx=0x7fa130009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:13.402 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:12.376+0000 7fa1397fa700 1 -- 172.21.15.107:0/1056557713 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa130010840 con 0x7fa13c100d30 2024-04-22T23:26:13.402 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:12.376+0000 7fa1397fa700 1 -- 172.21.15.107:0/1056557713 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa130010e70 con 0x7fa13c100d30 2024-04-22T23:26:13.403 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:12.376+0000 7fa142ee4700 1 -- 172.21.15.107:0/1056557713 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa13c101d10 con 0x7fa13c100d30 2024-04-22T23:26:13.403 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:12.376+0000 7fa142ee4700 1 -- 172.21.15.107:0/1056557713 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa13c102200 con 0x7fa13c100d30 2024-04-22T23:26:13.403 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:12.376+0000 7fa1397fa700 1 -- 172.21.15.107:0/1056557713 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa130024e40 con 0x7fa13c100d30 2024-04-22T23:26:13.403 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:12.377+0000 7fa1397fa700 1 -- 172.21.15.107:0/1056557713 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fa13001a520 con 0x7fa13c100d30 2024-04-22T23:26:13.404 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:12.377+0000 7fa1397fa700 1 --2- 172.21.15.107:0/1056557713 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7fa12c036790 0x7fa12c038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:13.404 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:12.377+0000 7fa1397fa700 1 -- 172.21.15.107:0/1056557713 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fa13004b910 con 0x7fa13c100d30 2024-04-22T23:26:13.404 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:12.378+0000 7fa142ee4700 1 -- 172.21.15.107:0/1056557713 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa13c04bed0 con 0x7fa13c100d30 2024-04-22T23:26:13.404 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:12.378+0000 7fa13bfff700 1 --2- 172.21.15.107:0/1056557713 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7fa12c036790 0x7fa12c038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:13.404 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:12.378+0000 7fa13bfff700 1 --2- 172.21.15.107:0/1056557713 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7fa12c036790 0x7fa12c038c40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa128009990 tx=0x7fa128006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:13.405 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:12.383+0000 7fa1397fa700 1 -- 172.21.15.107:0/1056557713 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fa130018bd0 con 0x7fa13c100d30 2024-04-22T23:26:13.405 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:12.601+0000 7fa142ee4700 1 -- 172.21.15.107:0/1056557713 --> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7fa13c102910 con 0x7fa12c036790 2024-04-22T23:26:13.405 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:12.605+0000 7fa1397fa700 1 -- 172.21.15.107:0/1056557713 <== mgr.14118 v2:172.21.15.107:6800/307474287 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7fa13c102910 con 0x7fa12c036790 2024-04-22T23:26:13.405 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:12.607+0000 7fa142ee4700 1 -- 172.21.15.107:0/1056557713 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7fa12c036790 msgr2=0x7fa12c038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:13.405 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:12.607+0000 7fa142ee4700 1 --2- 172.21.15.107:0/1056557713 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7fa12c036790 0x7fa12c038c40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa128009990 tx=0x7fa128006e40 comp rx=0 tx=0).stop 2024-04-22T23:26:13.405 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:12.607+0000 7fa142ee4700 1 -- 172.21.15.107:0/1056557713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa13c100d30 msgr2=0x7fa13c103760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:13.406 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:12.607+0000 7fa142ee4700 1 --2- 172.21.15.107:0/1056557713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa13c100d30 0x7fa13c103760 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa1300090b0 tx=0x7fa130009250 comp rx=0 tx=0).stop 2024-04-22T23:26:13.406 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:12.607+0000 7fa142ee4700 1 -- 172.21.15.107:0/1056557713 shutdown_connections 2024-04-22T23:26:13.406 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:12.607+0000 7fa142ee4700 1 --2- 172.21.15.107:0/1056557713 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7fa12c036790 0x7fa12c038c40 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:13.406 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:12.607+0000 7fa142ee4700 1 --2- 172.21.15.107:0/1056557713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa13c100d30 0x7fa13c103760 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:13.406 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:12.607+0000 7fa142ee4700 1 -- 172.21.15.107:0/1056557713 >> 172.21.15.107:0/1056557713 conn(0x7fa13c0fc730 msgr2=0x7fa13c0fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:26:13.406 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:12.607+0000 7fa142ee4700 1 -- 172.21.15.107:0/1056557713 shutdown_connections 2024-04-22T23:26:13.406 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:12.607+0000 7fa142ee4700 1 -- 172.21.15.107:0/1056557713 wait complete. 2024-04-22T23:26:13.406 INFO:teuthology.orchestra.run.smithi107.stdout:Deploying prometheus service with default placement... 2024-04-22T23:26:13.777 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:13 smithi107 ceph-mon[26355]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:26:13.777 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:13 smithi107 ceph-mon[26355]: Saving service crash spec with placement * 2024-04-22T23:26:13.778 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:13 smithi107 ceph-mon[26355]: from='mgr.14118 172.21.15.107:0/4176396201' entity='mgr.smithi107.uipxvm' 2024-04-22T23:26:14.724 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-04-22T23:26:14.724 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:13.788+0000 7fb087883700 1 Processor -- start 2024-04-22T23:26:14.724 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:13.788+0000 7fb087883700 1 -- start start 2024-04-22T23:26:14.724 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:13.788+0000 7fb087883700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb080102930 0x7fb080102d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:14.724 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:13.788+0000 7fb087883700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb080103280 con 0x7fb080102930 2024-04-22T23:26:14.724 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:13.788+0000 7fb08561f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb080102930 0x7fb080102d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:14.724 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:13.788+0000 7fb08561f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb080102930 0x7fb080102d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34094/0 (socket says 172.21.15.107:34094) 2024-04-22T23:26:14.725 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:13.788+0000 7fb08561f700 1 -- 172.21.15.107:0/2576888696 learned_addr learned my addr 172.21.15.107:0/2576888696 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:26:14.725 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:13.788+0000 7fb08561f700 1 -- 172.21.15.107:0/2576888696 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0801033c0 con 0x7fb080102930 2024-04-22T23:26:14.725 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:13.789+0000 7fb08561f700 1 --2- 172.21.15.107:0/2576888696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb080102930 0x7fb080102d40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb07c0098e0 tx=0x7fb07c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d2d631ba8d76045 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:14.725 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:13.789+0000 7fb077fff700 1 -- 172.21.15.107:0/2576888696 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb07c00b8f0 con 0x7fb080102930 2024-04-22T23:26:14.725 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:13.789+0000 7fb077fff700 1 -- 172.21.15.107:0/2576888696 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb07c011420 con 0x7fb080102930 2024-04-22T23:26:14.725 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:13.789+0000 7fb077fff700 1 -- 172.21.15.107:0/2576888696 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb07c010540 con 0x7fb080102930 2024-04-22T23:26:14.725 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:13.789+0000 7fb087883700 1 -- 172.21.15.107:0/2576888696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb080102930 msgr2=0x7fb080102d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:14.725 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:13.789+0000 7fb087883700 1 --2- 172.21.15.107:0/2576888696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb080102930 0x7fb080102d40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb07c0098e0 tx=0x7fb07c009250 comp rx=0 tx=0).stop 2024-04-22T23:26:14.725 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:13.789+0000 7fb087883700 1 -- 172.21.15.107:0/2576888696 shutdown_connections 2024-04-22T23:26:14.725 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:13.789+0000 7fb087883700 1 --2- 172.21.15.107:0/2576888696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb080102930 0x7fb080102d40 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:14.725 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:13.789+0000 7fb087883700 1 -- 172.21.15.107:0/2576888696 >> 172.21.15.107:0/2576888696 conn(0x7fb0800fe140 msgr2=0x7fb080100550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:26:14.725 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:13.790+0000 7fb087883700 1 -- 172.21.15.107:0/2576888696 shutdown_connections 2024-04-22T23:26:14.725 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:13.790+0000 7fb087883700 1 -- 172.21.15.107:0/2576888696 wait complete. 2024-04-22T23:26:14.725 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:13.790+0000 7fb087883700 1 Processor -- start 2024-04-22T23:26:14.726 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:13.790+0000 7fb087883700 1 -- start start 2024-04-22T23:26:14.726 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:13.790+0000 7fb087883700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb080102930 0x7fb0801116f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:14.726 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:13.790+0000 7fb087883700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb080111c30 con 0x7fb080102930 2024-04-22T23:26:14.726 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:13.790+0000 7fb08561f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb080102930 0x7fb0801116f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:14.726 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:13.790+0000 7fb08561f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb080102930 0x7fb0801116f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34110/0 (socket says 172.21.15.107:34110) 2024-04-22T23:26:14.726 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:13.790+0000 7fb08561f700 1 -- 172.21.15.107:0/4060050595 learned_addr learned my addr 172.21.15.107:0/4060050595 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:26:14.726 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:13.791+0000 7fb08561f700 1 -- 172.21.15.107:0/4060050595 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb07c0094f0 con 0x7fb080102930 2024-04-22T23:26:14.726 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:13.791+0000 7fb08561f700 1 --2- 172.21.15.107:0/4060050595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb080102930 0x7fb0801116f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb07c006b00 tx=0x7fb07c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:14.726 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:13.791+0000 7fb075ffb700 1 -- 172.21.15.107:0/4060050595 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb07c010800 con 0x7fb080102930 2024-04-22T23:26:14.726 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:13.791+0000 7fb075ffb700 1 -- 172.21.15.107:0/4060050595 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb07c010e30 con 0x7fb080102930 2024-04-22T23:26:14.726 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:13.791+0000 7fb087883700 1 -- 172.21.15.107:0/4060050595 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb080111e30 con 0x7fb080102930 2024-04-22T23:26:14.726 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:13.791+0000 7fb075ffb700 1 -- 172.21.15.107:0/4060050595 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb07c01ae40 con 0x7fb080102930 2024-04-22T23:26:14.726 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:13.791+0000 7fb087883700 1 -- 172.21.15.107:0/4060050595 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0801129a0 con 0x7fb080102930 2024-04-22T23:26:14.726 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:13.792+0000 7fb075ffb700 1 -- 172.21.15.107:0/4060050595 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fb07c010960 con 0x7fb080102930 2024-04-22T23:26:14.726 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:13.792+0000 7fb075ffb700 1 --2- 172.21.15.107:0/4060050595 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7fb06c036720 0x7fb06c038bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:14.727 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:13.792+0000 7fb087883700 1 -- 172.21.15.107:0/4060050595 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb08005f5f0 con 0x7fb080102930 2024-04-22T23:26:14.727 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:13.792+0000 7fb075ffb700 1 -- 172.21.15.107:0/4060050595 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fb07c04a660 con 0x7fb080102930 2024-04-22T23:26:14.727 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:13.792+0000 7fb084e1e700 1 --2- 172.21.15.107:0/4060050595 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7fb06c036720 0x7fb06c038bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:14.727 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:13.793+0000 7fb084e1e700 1 --2- 172.21.15.107:0/4060050595 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7fb06c036720 0x7fb06c038bd0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb070009990 tx=0x7fb070006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:14.727 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:13.797+0000 7fb075ffb700 1 -- 172.21.15.107:0/4060050595 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb07c02e440 con 0x7fb080102930 2024-04-22T23:26:14.727 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:13.996+0000 7fb087883700 1 -- 172.21.15.107:0/4060050595 --> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7fb0800ff380 con 0x7fb06c036720 2024-04-22T23:26:14.727 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:14.003+0000 7fb075ffb700 1 -- 172.21.15.107:0/4060050595 <== mgr.14118 v2:172.21.15.107:6800/307474287 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7fb0800ff380 con 0x7fb06c036720 2024-04-22T23:26:14.727 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:14.005+0000 7fb087883700 1 -- 172.21.15.107:0/4060050595 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7fb06c036720 msgr2=0x7fb06c038bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:14.727 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:14.005+0000 7fb087883700 1 --2- 172.21.15.107:0/4060050595 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7fb06c036720 0x7fb06c038bd0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb070009990 tx=0x7fb070006e40 comp rx=0 tx=0).stop 2024-04-22T23:26:14.727 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:14.005+0000 7fb087883700 1 -- 172.21.15.107:0/4060050595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb080102930 msgr2=0x7fb0801116f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:14.727 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:14.005+0000 7fb087883700 1 --2- 172.21.15.107:0/4060050595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb080102930 0x7fb0801116f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb07c006b00 tx=0x7fb07c009250 comp rx=0 tx=0).stop 2024-04-22T23:26:14.727 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:14.005+0000 7fb087883700 1 -- 172.21.15.107:0/4060050595 shutdown_connections 2024-04-22T23:26:14.727 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:14.005+0000 7fb087883700 1 --2- 172.21.15.107:0/4060050595 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7fb06c036720 0x7fb06c038bd0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:14.727 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:14.005+0000 7fb087883700 1 --2- 172.21.15.107:0/4060050595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb080102930 0x7fb0801116f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:14.728 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:14.005+0000 7fb087883700 1 -- 172.21.15.107:0/4060050595 >> 172.21.15.107:0/4060050595 conn(0x7fb0800fe140 msgr2=0x7fb0800fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:26:14.728 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:14.005+0000 7fb087883700 1 -- 172.21.15.107:0/4060050595 shutdown_connections 2024-04-22T23:26:14.728 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:14.005+0000 7fb087883700 1 -- 172.21.15.107:0/4060050595 wait complete. 2024-04-22T23:26:14.728 INFO:teuthology.orchestra.run.smithi107.stdout:Deploying grafana service with default placement... 2024-04-22T23:26:15.399 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:15 smithi107 ceph-mon[26355]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:26:15.399 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:15 smithi107 ceph-mon[26355]: Saving service prometheus spec with placement count:1 2024-04-22T23:26:15.399 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:15 smithi107 ceph-mon[26355]: from='mgr.14118 172.21.15.107:0/4176396201' entity='mgr.smithi107.uipxvm' 2024-04-22T23:26:16.023 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-04-22T23:26:16.023 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:15.183+0000 7f341226b700 1 Processor -- start 2024-04-22T23:26:16.024 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:15.183+0000 7f341226b700 1 -- start start 2024-04-22T23:26:16.024 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:15.183+0000 7f341226b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f340c100c90 0x7f340c1010a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:16.024 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:15.183+0000 7f341226b700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f340c1015e0 con 0x7f340c100c90 2024-04-22T23:26:16.024 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:15.183+0000 7f340bfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f340c100c90 0x7f340c1010a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:16.024 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:15.183+0000 7f340bfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f340c100c90 0x7f340c1010a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34114/0 (socket says 172.21.15.107:34114) 2024-04-22T23:26:16.024 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:15.183+0000 7f340bfff700 1 -- 172.21.15.107:0/1363098582 learned_addr learned my addr 172.21.15.107:0/1363098582 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:26:16.024 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:15.183+0000 7f340bfff700 1 -- 172.21.15.107:0/1363098582 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f340c101720 con 0x7f340c100c90 2024-04-22T23:26:16.024 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:15.183+0000 7f340bfff700 1 --2- 172.21.15.107:0/1363098582 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f340c100c90 0x7f340c1010a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f33fc0098e0 tx=0x7f33fc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acb035dc96822590 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:16.024 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:15.184+0000 7f340affd700 1 -- 172.21.15.107:0/1363098582 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f33fc00b860 con 0x7f340c100c90 2024-04-22T23:26:16.024 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:15.184+0000 7f340affd700 1 -- 172.21.15.107:0/1363098582 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f33fc00be90 con 0x7f340c100c90 2024-04-22T23:26:16.024 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:15.184+0000 7f340affd700 1 -- 172.21.15.107:0/1363098582 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f33fc0103c0 con 0x7f340c100c90 2024-04-22T23:26:16.024 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:15.184+0000 7f341226b700 1 -- 172.21.15.107:0/1363098582 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f340c100c90 msgr2=0x7f340c1010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:16.024 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:15.184+0000 7f341226b700 1 --2- 172.21.15.107:0/1363098582 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f340c100c90 0x7f340c1010a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f33fc0098e0 tx=0x7f33fc009250 comp rx=0 tx=0).stop 2024-04-22T23:26:16.024 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:15.184+0000 7f341226b700 1 -- 172.21.15.107:0/1363098582 shutdown_connections 2024-04-22T23:26:16.025 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:15.184+0000 7f341226b700 1 --2- 172.21.15.107:0/1363098582 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f340c100c90 0x7f340c1010a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:16.025 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:15.184+0000 7f341226b700 1 -- 172.21.15.107:0/1363098582 >> 172.21.15.107:0/1363098582 conn(0x7f340c0fc6b0 msgr2=0x7f340c0feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:26:16.025 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:15.184+0000 7f341226b700 1 -- 172.21.15.107:0/1363098582 shutdown_connections 2024-04-22T23:26:16.025 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:15.184+0000 7f341226b700 1 -- 172.21.15.107:0/1363098582 wait complete. 2024-04-22T23:26:16.025 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:15.185+0000 7f341226b700 1 Processor -- start 2024-04-22T23:26:16.025 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:15.185+0000 7f341226b700 1 -- start start 2024-04-22T23:26:16.025 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:15.185+0000 7f341226b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f340c100c90 0x7f340c1116c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:16.025 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:15.185+0000 7f341226b700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f340c111c00 con 0x7f340c100c90 2024-04-22T23:26:16.025 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:15.185+0000 7f340bfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f340c100c90 0x7f340c1116c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:16.025 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:15.185+0000 7f340bfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f340c100c90 0x7f340c1116c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34128/0 (socket says 172.21.15.107:34128) 2024-04-22T23:26:16.025 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:15.185+0000 7f340bfff700 1 -- 172.21.15.107:0/3703318968 learned_addr learned my addr 172.21.15.107:0/3703318968 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:26:16.025 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:15.185+0000 7f340bfff700 1 -- 172.21.15.107:0/3703318968 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33fc0094f0 con 0x7f340c100c90 2024-04-22T23:26:16.025 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:15.186+0000 7f340bfff700 1 --2- 172.21.15.107:0/3703318968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f340c100c90 0x7f340c1116c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f33fc011650 tx=0x7f33fc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:16.025 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:15.186+0000 7f3408ff9700 1 -- 172.21.15.107:0/3703318968 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f33fc010890 con 0x7f340c100c90 2024-04-22T23:26:16.026 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:15.186+0000 7f3408ff9700 1 -- 172.21.15.107:0/3703318968 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f33fc010ec0 con 0x7f340c100c90 2024-04-22T23:26:16.026 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:15.186+0000 7f341226b700 1 -- 172.21.15.107:0/3703318968 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f340c111e00 con 0x7f340c100c90 2024-04-22T23:26:16.026 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:15.186+0000 7f3408ff9700 1 -- 172.21.15.107:0/3703318968 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f33fc0193c0 con 0x7f340c100c90 2024-04-22T23:26:16.026 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:15.186+0000 7f341226b700 1 -- 172.21.15.107:0/3703318968 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f340c112970 con 0x7f340c100c90 2024-04-22T23:26:16.026 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:15.186+0000 7f3408ff9700 1 -- 172.21.15.107:0/3703318968 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f33fc019600 con 0x7f340c100c90 2024-04-22T23:26:16.026 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:15.187+0000 7f3408ff9700 1 --2- 172.21.15.107:0/3703318968 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f33f4036790 0x7f33f4038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:16.026 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:15.187+0000 7f341226b700 1 -- 172.21.15.107:0/3703318968 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f340c05f5f0 con 0x7f340c100c90 2024-04-22T23:26:16.026 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:15.187+0000 7f3408ff9700 1 -- 172.21.15.107:0/3703318968 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f33fc04a590 con 0x7f340c100c90 2024-04-22T23:26:16.026 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:15.187+0000 7f340b7fe700 1 --2- 172.21.15.107:0/3703318968 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f33f4036790 0x7f33f4038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:16.026 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:15.187+0000 7f340b7fe700 1 --2- 172.21.15.107:0/3703318968 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f33f4036790 0x7f33f4038c40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3400009990 tx=0x7f3400006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:16.027 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:15.191+0000 7f3408ff9700 1 -- 172.21.15.107:0/3703318968 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f33fc02b950 con 0x7f340c100c90 2024-04-22T23:26:16.027 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:15.397+0000 7f341226b700 1 -- 172.21.15.107:0/3703318968 --> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7f340c0fd930 con 0x7f33f4036790 2024-04-22T23:26:16.027 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:15.402+0000 7f3408ff9700 1 -- 172.21.15.107:0/3703318968 <== mgr.14118 v2:172.21.15.107:6800/307474287 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f340c0fd930 con 0x7f33f4036790 2024-04-22T23:26:16.027 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:15.405+0000 7f341226b700 1 -- 172.21.15.107:0/3703318968 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f33f4036790 msgr2=0x7f33f4038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:16.027 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:15.405+0000 7f341226b700 1 --2- 172.21.15.107:0/3703318968 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f33f4036790 0x7f33f4038c40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3400009990 tx=0x7f3400006e40 comp rx=0 tx=0).stop 2024-04-22T23:26:16.027 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:15.405+0000 7f341226b700 1 -- 172.21.15.107:0/3703318968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f340c100c90 msgr2=0x7f340c1116c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:16.027 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:15.405+0000 7f341226b700 1 --2- 172.21.15.107:0/3703318968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f340c100c90 0x7f340c1116c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f33fc011650 tx=0x7f33fc009250 comp rx=0 tx=0).stop 2024-04-22T23:26:16.027 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:15.406+0000 7f341226b700 1 -- 172.21.15.107:0/3703318968 shutdown_connections 2024-04-22T23:26:16.027 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:15.406+0000 7f341226b700 1 --2- 172.21.15.107:0/3703318968 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f33f4036790 0x7f33f4038c40 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:16.027 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:15.406+0000 7f341226b700 1 --2- 172.21.15.107:0/3703318968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f340c100c90 0x7f340c1116c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:16.027 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:15.406+0000 7f341226b700 1 -- 172.21.15.107:0/3703318968 >> 172.21.15.107:0/3703318968 conn(0x7f340c0fc6b0 msgr2=0x7f340c0fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:26:16.027 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:15.406+0000 7f341226b700 1 -- 172.21.15.107:0/3703318968 shutdown_connections 2024-04-22T23:26:16.027 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:15.406+0000 7f341226b700 1 -- 172.21.15.107:0/3703318968 wait complete. 2024-04-22T23:26:16.027 INFO:teuthology.orchestra.run.smithi107.stdout:Deploying node-exporter service with default placement... 2024-04-22T23:26:16.270 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:16 smithi107 ceph-mon[26355]: from='mgr.14118 172.21.15.107:0/4176396201' entity='mgr.smithi107.uipxvm' 2024-04-22T23:26:16.270 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:16 smithi107 ceph-mon[26355]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:26:16.270 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:16 smithi107 ceph-mon[26355]: Saving service grafana spec with placement count:1 2024-04-22T23:26:16.271 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:16 smithi107 ceph-mon[26355]: from='mgr.14118 172.21.15.107:0/4176396201' entity='mgr.smithi107.uipxvm' 2024-04-22T23:26:16.271 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:16 smithi107 ceph-mon[26355]: from='mgr.14118 172.21.15.107:0/4176396201' entity='mgr.smithi107.uipxvm' 2024-04-22T23:26:17.827 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-04-22T23:26:17.828 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:16.486+0000 7f61d150f700 1 Processor -- start 2024-04-22T23:26:17.828 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:16.486+0000 7f61d150f700 1 -- start start 2024-04-22T23:26:17.828 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:16.486+0000 7f61d150f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61cc101800 0x7f61cc101c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:17.828 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:16.486+0000 7f61d150f700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61cc102150 con 0x7f61cc101800 2024-04-22T23:26:17.828 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:16.486+0000 7f61caffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61cc101800 0x7f61cc101c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:17.828 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:16.486+0000 7f61caffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61cc101800 0x7f61cc101c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34136/0 (socket says 172.21.15.107:34136) 2024-04-22T23:26:17.828 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:16.486+0000 7f61caffd700 1 -- 172.21.15.107:0/2297637357 learned_addr learned my addr 172.21.15.107:0/2297637357 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:26:17.828 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:16.487+0000 7f61caffd700 1 -- 172.21.15.107:0/2297637357 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61cc102290 con 0x7f61cc101800 2024-04-22T23:26:17.828 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:16.487+0000 7f61caffd700 1 --2- 172.21.15.107:0/2297637357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61cc101800 0x7f61cc101c10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f61bc0098e0 tx=0x7f61bc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=959b75cb4c399454 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:17.828 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:16.487+0000 7f61c9ffb700 1 -- 172.21.15.107:0/2297637357 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f61bc00b860 con 0x7f61cc101800 2024-04-22T23:26:17.828 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:16.487+0000 7f61c9ffb700 1 -- 172.21.15.107:0/2297637357 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f61bc00be90 con 0x7f61cc101800 2024-04-22T23:26:17.828 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:16.487+0000 7f61c9ffb700 1 -- 172.21.15.107:0/2297637357 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f61bc0103c0 con 0x7f61cc101800 2024-04-22T23:26:17.828 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:16.488+0000 7f61d150f700 1 -- 172.21.15.107:0/2297637357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61cc101800 msgr2=0x7f61cc101c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:17.828 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:16.488+0000 7f61d150f700 1 --2- 172.21.15.107:0/2297637357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61cc101800 0x7f61cc101c10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f61bc0098e0 tx=0x7f61bc009250 comp rx=0 tx=0).stop 2024-04-22T23:26:17.829 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:16.488+0000 7f61d150f700 1 -- 172.21.15.107:0/2297637357 shutdown_connections 2024-04-22T23:26:17.829 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:16.488+0000 7f61d150f700 1 --2- 172.21.15.107:0/2297637357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61cc101800 0x7f61cc101c10 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:17.829 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:16.488+0000 7f61d150f700 1 -- 172.21.15.107:0/2297637357 >> 172.21.15.107:0/2297637357 conn(0x7f61cc0fcf80 msgr2=0x7f61cc0ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:26:17.829 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:16.488+0000 7f61d150f700 1 -- 172.21.15.107:0/2297637357 shutdown_connections 2024-04-22T23:26:17.829 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:16.488+0000 7f61d150f700 1 -- 172.21.15.107:0/2297637357 wait complete. 2024-04-22T23:26:17.829 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:16.489+0000 7f61d150f700 1 Processor -- start 2024-04-22T23:26:17.829 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:16.489+0000 7f61d150f700 1 -- start start 2024-04-22T23:26:17.829 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:16.489+0000 7f61d150f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61cc101800 0x7f61cc076b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:17.829 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:16.489+0000 7f61d150f700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61cc1143c0 con 0x7f61cc101800 2024-04-22T23:26:17.829 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:16.489+0000 7f61caffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61cc101800 0x7f61cc076b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:17.830 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:16.489+0000 7f61caffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61cc101800 0x7f61cc076b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34138/0 (socket says 172.21.15.107:34138) 2024-04-22T23:26:17.830 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:16.489+0000 7f61caffd700 1 -- 172.21.15.107:0/3295350909 learned_addr learned my addr 172.21.15.107:0/3295350909 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:26:17.830 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:16.489+0000 7f61caffd700 1 -- 172.21.15.107:0/3295350909 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61bc0094f0 con 0x7f61cc101800 2024-04-22T23:26:17.830 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:16.489+0000 7f61caffd700 1 --2- 172.21.15.107:0/3295350909 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61cc101800 0x7f61cc076b50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f61bc011680 tx=0x7f61bc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:17.830 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:16.490+0000 7f61b37fe700 1 -- 172.21.15.107:0/3295350909 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f61bc010890 con 0x7f61cc101800 2024-04-22T23:26:17.830 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:16.490+0000 7f61d150f700 1 -- 172.21.15.107:0/3295350909 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61cc114500 con 0x7f61cc101800 2024-04-22T23:26:17.830 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:16.490+0000 7f61b37fe700 1 -- 172.21.15.107:0/3295350909 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f61bc010ec0 con 0x7f61cc101800 2024-04-22T23:26:17.830 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:16.490+0000 7f61b37fe700 1 -- 172.21.15.107:0/3295350909 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f61bc0243f0 con 0x7f61cc101800 2024-04-22T23:26:17.830 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:16.490+0000 7f61d150f700 1 -- 172.21.15.107:0/3295350909 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61cc114990 con 0x7f61cc101800 2024-04-22T23:26:17.830 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:16.491+0000 7f61d150f700 1 -- 172.21.15.107:0/3295350909 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61cc04bed0 con 0x7f61cc101800 2024-04-22T23:26:17.830 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:16.491+0000 7f61b37fe700 1 -- 172.21.15.107:0/3295350909 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f61bc01a8c0 con 0x7f61cc101800 2024-04-22T23:26:17.830 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:16.492+0000 7f61b37fe700 1 --2- 172.21.15.107:0/3295350909 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f61b4036790 0x7f61b4038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:17.830 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:16.492+0000 7f61b37fe700 1 -- 172.21.15.107:0/3295350909 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f61bc04a470 con 0x7f61cc101800 2024-04-22T23:26:17.831 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:16.492+0000 7f61ca7fc700 1 --2- 172.21.15.107:0/3295350909 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f61b4036790 0x7f61b4038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:17.831 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:16.492+0000 7f61ca7fc700 1 --2- 172.21.15.107:0/3295350909 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f61b4036790 0x7f61b4038c40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f61c0009990 tx=0x7f61c0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:17.831 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:16.496+0000 7f61b37fe700 1 -- 172.21.15.107:0/3295350909 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f61bc018760 con 0x7f61cc101800 2024-04-22T23:26:17.831 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:16.729+0000 7f61d150f700 1 -- 172.21.15.107:0/3295350909 --> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7f61cc0fe1e0 con 0x7f61b4036790 2024-04-22T23:26:17.831 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:16.733+0000 7f61b37fe700 1 -- 172.21.15.107:0/3295350909 <== mgr.14118 v2:172.21.15.107:6800/307474287 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f61cc0fe1e0 con 0x7f61b4036790 2024-04-22T23:26:17.831 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:16.735+0000 7f61d150f700 1 -- 172.21.15.107:0/3295350909 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f61b4036790 msgr2=0x7f61b4038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:17.831 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:16.735+0000 7f61d150f700 1 --2- 172.21.15.107:0/3295350909 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f61b4036790 0x7f61b4038c40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f61c0009990 tx=0x7f61c0006e40 comp rx=0 tx=0).stop 2024-04-22T23:26:17.831 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:16.735+0000 7f61d150f700 1 -- 172.21.15.107:0/3295350909 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61cc101800 msgr2=0x7f61cc076b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:17.831 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:16.735+0000 7f61d150f700 1 --2- 172.21.15.107:0/3295350909 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61cc101800 0x7f61cc076b50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f61bc011680 tx=0x7f61bc009250 comp rx=0 tx=0).stop 2024-04-22T23:26:17.831 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:16.735+0000 7f61d150f700 1 -- 172.21.15.107:0/3295350909 shutdown_connections 2024-04-22T23:26:17.831 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:16.735+0000 7f61d150f700 1 --2- 172.21.15.107:0/3295350909 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f61b4036790 0x7f61b4038c40 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:17.832 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:16.735+0000 7f61d150f700 1 --2- 172.21.15.107:0/3295350909 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61cc101800 0x7f61cc076b50 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:17.832 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:16.735+0000 7f61d150f700 1 -- 172.21.15.107:0/3295350909 >> 172.21.15.107:0/3295350909 conn(0x7f61cc0fcf80 msgr2=0x7f61cc0fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:26:17.832 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:16.735+0000 7f61d150f700 1 -- 172.21.15.107:0/3295350909 shutdown_connections 2024-04-22T23:26:17.832 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:16.735+0000 7f61d150f700 1 -- 172.21.15.107:0/3295350909 wait complete. 2024-04-22T23:26:17.832 INFO:teuthology.orchestra.run.smithi107.stdout:Deploying alertmanager service with default placement... 2024-04-22T23:26:17.990 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:17 smithi107 ceph-mon[26355]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:26:17.990 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:17 smithi107 ceph-mon[26355]: Saving service node-exporter spec with placement * 2024-04-22T23:26:17.990 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:17 smithi107 ceph-mon[26355]: from='mgr.14118 172.21.15.107:0/4176396201' entity='mgr.smithi107.uipxvm' 2024-04-22T23:26:19.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-04-22T23:26:19.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:18.690+0000 7f207e0f4700 1 Processor -- start 2024-04-22T23:26:19.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:18.690+0000 7f207e0f4700 1 -- start start 2024-04-22T23:26:19.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:18.690+0000 7f207e0f4700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20781018b0 0x7f2078101cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:19.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:18.690+0000 7f207e0f4700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2078102200 con 0x7f20781018b0 2024-04-22T23:26:19.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:18.690+0000 7f20777fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20781018b0 0x7f2078101cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:19.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:18.691+0000 7f20777fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20781018b0 0x7f2078101cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34142/0 (socket says 172.21.15.107:34142) 2024-04-22T23:26:19.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:18.691+0000 7f20777fe700 1 -- 172.21.15.107:0/2714526756 learned_addr learned my addr 172.21.15.107:0/2714526756 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:26:19.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:18.691+0000 7f20777fe700 1 -- 172.21.15.107:0/2714526756 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2078102340 con 0x7f20781018b0 2024-04-22T23:26:19.438 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:18.691+0000 7f20777fe700 1 --2- 172.21.15.107:0/2714526756 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20781018b0 0x7f2078101cc0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f20680098e0 tx=0x7f2068009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f259fbb819d5b5ea server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:19.438 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:18.691+0000 7f20767fc700 1 -- 172.21.15.107:0/2714526756 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f206800b8f0 con 0x7f20781018b0 2024-04-22T23:26:19.438 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:18.691+0000 7f20767fc700 1 -- 172.21.15.107:0/2714526756 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2068011420 con 0x7f20781018b0 2024-04-22T23:26:19.438 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:18.691+0000 7f20767fc700 1 -- 172.21.15.107:0/2714526756 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2068010540 con 0x7f20781018b0 2024-04-22T23:26:19.438 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:18.692+0000 7f207e0f4700 1 -- 172.21.15.107:0/2714526756 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20781018b0 msgr2=0x7f2078101cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:19.438 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:18.692+0000 7f207e0f4700 1 --2- 172.21.15.107:0/2714526756 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20781018b0 0x7f2078101cc0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f20680098e0 tx=0x7f2068009250 comp rx=0 tx=0).stop 2024-04-22T23:26:19.438 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:18.692+0000 7f207e0f4700 1 -- 172.21.15.107:0/2714526756 shutdown_connections 2024-04-22T23:26:19.438 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:18.692+0000 7f207e0f4700 1 --2- 172.21.15.107:0/2714526756 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20781018b0 0x7f2078101cc0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:19.438 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:18.692+0000 7f207e0f4700 1 -- 172.21.15.107:0/2714526756 >> 172.21.15.107:0/2714526756 conn(0x7f20780fd010 msgr2=0x7f20780ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:26:19.438 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:18.692+0000 7f207e0f4700 1 -- 172.21.15.107:0/2714526756 shutdown_connections 2024-04-22T23:26:19.438 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:18.692+0000 7f207e0f4700 1 -- 172.21.15.107:0/2714526756 wait complete. 2024-04-22T23:26:19.438 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:18.692+0000 7f207e0f4700 1 Processor -- start 2024-04-22T23:26:19.438 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:18.693+0000 7f207e0f4700 1 -- start start 2024-04-22T23:26:19.438 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:18.693+0000 7f207e0f4700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20781018b0 0x7f207810f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:19.439 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:18.693+0000 7f207e0f4700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207810fab0 con 0x7f20781018b0 2024-04-22T23:26:19.439 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:18.693+0000 7f20777fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20781018b0 0x7f207810f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:19.439 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:18.693+0000 7f20777fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20781018b0 0x7f207810f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34146/0 (socket says 172.21.15.107:34146) 2024-04-22T23:26:19.439 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:18.693+0000 7f20777fe700 1 -- 172.21.15.107:0/3260664288 learned_addr learned my addr 172.21.15.107:0/3260664288 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:26:19.439 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:18.693+0000 7f20777fe700 1 -- 172.21.15.107:0/3260664288 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20680094f0 con 0x7f20781018b0 2024-04-22T23:26:19.439 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:18.693+0000 7f20777fe700 1 --2- 172.21.15.107:0/3260664288 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20781018b0 0x7f207810f570 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2068006d40 tx=0x7f2068009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:19.439 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:18.693+0000 7f205ffff700 1 -- 172.21.15.107:0/3260664288 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f206800beb0 con 0x7f20781018b0 2024-04-22T23:26:19.439 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:18.693+0000 7f207e0f4700 1 -- 172.21.15.107:0/3260664288 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f207810fcb0 con 0x7f20781018b0 2024-04-22T23:26:19.439 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:18.693+0000 7f205ffff700 1 -- 172.21.15.107:0/3260664288 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2068010c80 con 0x7f20781018b0 2024-04-22T23:26:19.439 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:18.693+0000 7f205ffff700 1 -- 172.21.15.107:0/3260664288 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2068024d40 con 0x7f20781018b0 2024-04-22T23:26:19.439 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:18.693+0000 7f207e0f4700 1 -- 172.21.15.107:0/3260664288 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2078110820 con 0x7f20781018b0 2024-04-22T23:26:19.439 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:18.694+0000 7f205ffff700 1 -- 172.21.15.107:0/3260664288 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f206801a430 con 0x7f20781018b0 2024-04-22T23:26:19.439 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:18.694+0000 7f205ffff700 1 --2- 172.21.15.107:0/3260664288 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f2060038a20 0x7f206003aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:19.439 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:18.694+0000 7f207e0f4700 1 -- 172.21.15.107:0/3260664288 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f207805f5f0 con 0x7f20781018b0 2024-04-22T23:26:19.440 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:18.694+0000 7f205ffff700 1 -- 172.21.15.107:0/3260664288 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f206804a220 con 0x7f20781018b0 2024-04-22T23:26:19.440 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:18.695+0000 7f2076ffd700 1 --2- 172.21.15.107:0/3260664288 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f2060038a20 0x7f206003aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:19.440 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:18.695+0000 7f2076ffd700 1 --2- 172.21.15.107:0/3260664288 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f2060038a20 0x7f206003aed0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f206c009990 tx=0x7f206c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:19.440 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:18.699+0000 7f205ffff700 1 -- 172.21.15.107:0/3260664288 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f2068018710 con 0x7f20781018b0 2024-04-22T23:26:19.440 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:18.897+0000 7f207e0f4700 1 -- 172.21.15.107:0/3260664288 --> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7f2078110bd0 con 0x7f2060038a20 2024-04-22T23:26:19.440 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:18.903+0000 7f205ffff700 1 -- 172.21.15.107:0/3260664288 <== mgr.14118 v2:172.21.15.107:6800/307474287 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7f2078110bd0 con 0x7f2060038a20 2024-04-22T23:26:19.440 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:18.905+0000 7f207e0f4700 1 -- 172.21.15.107:0/3260664288 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f2060038a20 msgr2=0x7f206003aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:19.440 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:18.905+0000 7f207e0f4700 1 --2- 172.21.15.107:0/3260664288 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f2060038a20 0x7f206003aed0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f206c009990 tx=0x7f206c006e40 comp rx=0 tx=0).stop 2024-04-22T23:26:19.440 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:18.905+0000 7f207e0f4700 1 -- 172.21.15.107:0/3260664288 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20781018b0 msgr2=0x7f207810f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:19.440 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:18.905+0000 7f207e0f4700 1 --2- 172.21.15.107:0/3260664288 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20781018b0 0x7f207810f570 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2068006d40 tx=0x7f2068009250 comp rx=0 tx=0).stop 2024-04-22T23:26:19.440 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:18.905+0000 7f207e0f4700 1 -- 172.21.15.107:0/3260664288 shutdown_connections 2024-04-22T23:26:19.440 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:18.905+0000 7f207e0f4700 1 --2- 172.21.15.107:0/3260664288 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f2060038a20 0x7f206003aed0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:19.440 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:18.905+0000 7f207e0f4700 1 --2- 172.21.15.107:0/3260664288 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20781018b0 0x7f207810f570 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:19.440 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:18.905+0000 7f207e0f4700 1 -- 172.21.15.107:0/3260664288 >> 172.21.15.107:0/3260664288 conn(0x7f20780fd010 msgr2=0x7f20780ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:26:19.440 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:18.905+0000 7f207e0f4700 1 -- 172.21.15.107:0/3260664288 shutdown_connections 2024-04-22T23:26:19.441 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:18.906+0000 7f207e0f4700 1 -- 172.21.15.107:0/3260664288 wait complete. 2024-04-22T23:26:19.990 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:19 smithi107 ceph-mon[26355]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:26:19.990 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:19 smithi107 ceph-mon[26355]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:26:19.990 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:19 smithi107 ceph-mon[26355]: Saving service alertmanager spec with placement count:1 2024-04-22T23:26:19.990 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:19 smithi107 ceph-mon[26355]: from='mgr.14118 172.21.15.107:0/4176396201' entity='mgr.smithi107.uipxvm' 2024-04-22T23:26:20.633 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:19.887+0000 7fa53b947700 1 Processor -- start 2024-04-22T23:26:20.634 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:19.888+0000 7fa53b947700 1 -- start start 2024-04-22T23:26:20.634 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:19.888+0000 7fa53b947700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa534102930 0x7fa534102d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:20.634 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:19.888+0000 7fa53b947700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa534103280 con 0x7fa534102930 2024-04-22T23:26:20.634 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:19.888+0000 7fa5396e3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa534102930 0x7fa534102d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:20.634 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:19.888+0000 7fa5396e3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa534102930 0x7fa534102d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34150/0 (socket says 172.21.15.107:34150) 2024-04-22T23:26:20.634 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:19.888+0000 7fa5396e3700 1 -- 172.21.15.107:0/1612052288 learned_addr learned my addr 172.21.15.107:0/1612052288 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:26:20.634 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:19.888+0000 7fa5396e3700 1 -- 172.21.15.107:0/1612052288 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5341033c0 con 0x7fa534102930 2024-04-22T23:26:20.634 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:19.888+0000 7fa5396e3700 1 --2- 172.21.15.107:0/1612052288 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa534102930 0x7fa534102d40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa5300098e0 tx=0x7fa530009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18e6e421597b7161 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:20.634 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:19.889+0000 7fa52bfff700 1 -- 172.21.15.107:0/1612052288 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa53000b8f0 con 0x7fa534102930 2024-04-22T23:26:20.634 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:19.889+0000 7fa52bfff700 1 -- 172.21.15.107:0/1612052288 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa530011420 con 0x7fa534102930 2024-04-22T23:26:20.634 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:19.889+0000 7fa52bfff700 1 -- 172.21.15.107:0/1612052288 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa530010540 con 0x7fa534102930 2024-04-22T23:26:20.634 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:19.889+0000 7fa53b947700 1 -- 172.21.15.107:0/1612052288 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa534102930 msgr2=0x7fa534102d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:20.635 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:19.889+0000 7fa53b947700 1 --2- 172.21.15.107:0/1612052288 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa534102930 0x7fa534102d40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa5300098e0 tx=0x7fa530009250 comp rx=0 tx=0).stop 2024-04-22T23:26:20.635 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:19.889+0000 7fa53b947700 1 -- 172.21.15.107:0/1612052288 shutdown_connections 2024-04-22T23:26:20.635 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:19.889+0000 7fa53b947700 1 --2- 172.21.15.107:0/1612052288 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa534102930 0x7fa534102d40 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:20.635 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:19.889+0000 7fa53b947700 1 -- 172.21.15.107:0/1612052288 >> 172.21.15.107:0/1612052288 conn(0x7fa5340fe140 msgr2=0x7fa534100550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:26:20.635 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:19.889+0000 7fa53b947700 1 -- 172.21.15.107:0/1612052288 shutdown_connections 2024-04-22T23:26:20.635 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:19.889+0000 7fa53b947700 1 -- 172.21.15.107:0/1612052288 wait complete. 2024-04-22T23:26:20.635 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:19.890+0000 7fa53b947700 1 Processor -- start 2024-04-22T23:26:20.635 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:19.890+0000 7fa53b947700 1 -- start start 2024-04-22T23:26:20.635 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:19.890+0000 7fa53b947700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa534102930 0x7fa5341116e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:20.635 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:19.890+0000 7fa53b947700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa534111c20 con 0x7fa534102930 2024-04-22T23:26:20.635 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:19.890+0000 7fa5396e3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa534102930 0x7fa5341116e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:20.635 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:19.890+0000 7fa5396e3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa534102930 0x7fa5341116e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34154/0 (socket says 172.21.15.107:34154) 2024-04-22T23:26:20.635 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:19.890+0000 7fa5396e3700 1 -- 172.21.15.107:0/387069417 learned_addr learned my addr 172.21.15.107:0/387069417 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:26:20.635 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:19.890+0000 7fa5396e3700 1 -- 172.21.15.107:0/387069417 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5300094f0 con 0x7fa534102930 2024-04-22T23:26:20.636 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:19.891+0000 7fa5396e3700 1 --2- 172.21.15.107:0/387069417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa534102930 0x7fa5341116e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa530006b00 tx=0x7fa530009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:20.636 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:19.891+0000 7fa529ffb700 1 -- 172.21.15.107:0/387069417 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa530010800 con 0x7fa534102930 2024-04-22T23:26:20.636 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:19.891+0000 7fa529ffb700 1 -- 172.21.15.107:0/387069417 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa530010e30 con 0x7fa534102930 2024-04-22T23:26:20.636 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:19.891+0000 7fa53b947700 1 -- 172.21.15.107:0/387069417 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa534111e20 con 0x7fa534102930 2024-04-22T23:26:20.636 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:19.891+0000 7fa529ffb700 1 -- 172.21.15.107:0/387069417 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa53001ae40 con 0x7fa534102930 2024-04-22T23:26:20.636 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:19.891+0000 7fa53b947700 1 -- 172.21.15.107:0/387069417 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa534112990 con 0x7fa534102930 2024-04-22T23:26:20.636 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:19.892+0000 7fa529ffb700 1 -- 172.21.15.107:0/387069417 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fa53002e440 con 0x7fa534102930 2024-04-22T23:26:20.636 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:19.892+0000 7fa53b947700 1 -- 172.21.15.107:0/387069417 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa53405f5f0 con 0x7fa534102930 2024-04-22T23:26:20.636 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:19.892+0000 7fa529ffb700 1 --2- 172.21.15.107:0/387069417 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7fa520036790 0x7fa520038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:20.636 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:19.892+0000 7fa529ffb700 1 -- 172.21.15.107:0/387069417 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fa53004a600 con 0x7fa534102930 2024-04-22T23:26:20.636 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:19.892+0000 7fa538ee2700 1 --2- 172.21.15.107:0/387069417 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7fa520036790 0x7fa520038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:20.636 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:19.893+0000 7fa538ee2700 1 --2- 172.21.15.107:0/387069417 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7fa520036790 0x7fa520038c40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa524009990 tx=0x7fa524006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:20.636 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:19.897+0000 7fa529ffb700 1 -- 172.21.15.107:0/387069417 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fa530018940 con 0x7fa534102930 2024-04-22T23:26:20.636 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:20.081+0000 7fa53b947700 1 -- 172.21.15.107:0/387069417 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7fa534112d30 con 0x7fa534102930 2024-04-22T23:26:20.637 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:20.086+0000 7fa529ffb700 1 -- 172.21.15.107:0/387069417 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7fa530018350 con 0x7fa534102930 2024-04-22T23:26:20.637 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:20.089+0000 7fa53b947700 1 -- 172.21.15.107:0/387069417 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7fa520036790 msgr2=0x7fa520038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:20.637 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:20.089+0000 7fa53b947700 1 --2- 172.21.15.107:0/387069417 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7fa520036790 0x7fa520038c40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa524009990 tx=0x7fa524006e40 comp rx=0 tx=0).stop 2024-04-22T23:26:20.637 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:20.089+0000 7fa53b947700 1 -- 172.21.15.107:0/387069417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa534102930 msgr2=0x7fa5341116e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:20.637 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:20.089+0000 7fa53b947700 1 --2- 172.21.15.107:0/387069417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa534102930 0x7fa5341116e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa530006b00 tx=0x7fa530009250 comp rx=0 tx=0).stop 2024-04-22T23:26:20.637 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:20.089+0000 7fa53b947700 1 -- 172.21.15.107:0/387069417 shutdown_connections 2024-04-22T23:26:20.637 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:20.089+0000 7fa53b947700 1 --2- 172.21.15.107:0/387069417 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7fa520036790 0x7fa520038c40 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:20.637 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:20.089+0000 7fa53b947700 1 --2- 172.21.15.107:0/387069417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa534102930 0x7fa5341116e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:20.637 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:20.089+0000 7fa53b947700 1 -- 172.21.15.107:0/387069417 >> 172.21.15.107:0/387069417 conn(0x7fa5340fe140 msgr2=0x7fa5340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:26:20.637 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:20.089+0000 7fa53b947700 1 -- 172.21.15.107:0/387069417 shutdown_connections 2024-04-22T23:26:20.637 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:20.089+0000 7fa53b947700 1 -- 172.21.15.107:0/387069417 wait complete. 2024-04-22T23:26:21.490 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:21 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/387069417' entity='client.admin' 2024-04-22T23:26:21.858 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:21.063+0000 7f9cc40e2700 1 Processor -- start 2024-04-22T23:26:21.858 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:21.063+0000 7f9cc40e2700 1 -- start start 2024-04-22T23:26:21.858 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:21.063+0000 7f9cc40e2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cbc078bb0 0x7f9cbc077240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:21.858 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:21.063+0000 7f9cc40e2700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cbc078fc0 con 0x7f9cbc078bb0 2024-04-22T23:26:21.858 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:21.064+0000 7f9cc1e7e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cbc078bb0 0x7f9cbc077240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:21.858 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:21.064+0000 7f9cc1e7e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cbc078bb0 0x7f9cbc077240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34160/0 (socket says 172.21.15.107:34160) 2024-04-22T23:26:21.858 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:21.064+0000 7f9cc1e7e700 1 -- 172.21.15.107:0/934009308 learned_addr learned my addr 172.21.15.107:0/934009308 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:26:21.859 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:21.064+0000 7f9cc1e7e700 1 -- 172.21.15.107:0/934009308 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cbc079100 con 0x7f9cbc078bb0 2024-04-22T23:26:21.859 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:21.064+0000 7f9cc1e7e700 1 --2- 172.21.15.107:0/934009308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cbc078bb0 0x7f9cbc077240 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9cb80098e0 tx=0x7f9cb8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c3d4f7ce96745ef server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:21.859 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:21.064+0000 7f9cc0e7c700 1 -- 172.21.15.107:0/934009308 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9cb800b8f0 con 0x7f9cbc078bb0 2024-04-22T23:26:21.859 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:21.064+0000 7f9cc0e7c700 1 -- 172.21.15.107:0/934009308 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9cb8011420 con 0x7f9cbc078bb0 2024-04-22T23:26:21.859 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:21.064+0000 7f9cc0e7c700 1 -- 172.21.15.107:0/934009308 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9cb8010540 con 0x7f9cbc078bb0 2024-04-22T23:26:21.859 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:21.065+0000 7f9cc40e2700 1 -- 172.21.15.107:0/934009308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cbc078bb0 msgr2=0x7f9cbc077240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:21.859 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:21.065+0000 7f9cc40e2700 1 --2- 172.21.15.107:0/934009308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cbc078bb0 0x7f9cbc077240 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9cb80098e0 tx=0x7f9cb8009250 comp rx=0 tx=0).stop 2024-04-22T23:26:21.859 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:21.065+0000 7f9cc40e2700 1 -- 172.21.15.107:0/934009308 shutdown_connections 2024-04-22T23:26:21.859 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:21.065+0000 7f9cc40e2700 1 --2- 172.21.15.107:0/934009308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cbc078bb0 0x7f9cbc077240 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:21.859 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:21.065+0000 7f9cc40e2700 1 -- 172.21.15.107:0/934009308 >> 172.21.15.107:0/934009308 conn(0x7f9cbc0fe140 msgr2=0x7f9cbc100550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:26:21.860 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:21.065+0000 7f9cc40e2700 1 -- 172.21.15.107:0/934009308 shutdown_connections 2024-04-22T23:26:21.860 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:21.065+0000 7f9cc40e2700 1 -- 172.21.15.107:0/934009308 wait complete. 2024-04-22T23:26:21.860 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:21.065+0000 7f9cc40e2700 1 Processor -- start 2024-04-22T23:26:21.860 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:21.065+0000 7f9cc40e2700 1 -- start start 2024-04-22T23:26:21.860 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:21.065+0000 7f9cc40e2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cbc078bb0 0x7f9cbc113910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:21.860 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:21.065+0000 7f9cc40e2700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cbc113e50 con 0x7f9cbc078bb0 2024-04-22T23:26:21.860 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:21.066+0000 7f9cc1e7e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cbc078bb0 0x7f9cbc113910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:21.860 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:21.066+0000 7f9cc1e7e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cbc078bb0 0x7f9cbc113910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34176/0 (socket says 172.21.15.107:34176) 2024-04-22T23:26:21.860 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:21.066+0000 7f9cc1e7e700 1 -- 172.21.15.107:0/1177780556 learned_addr learned my addr 172.21.15.107:0/1177780556 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:26:21.860 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:21.066+0000 7f9cc1e7e700 1 -- 172.21.15.107:0/1177780556 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cb80094f0 con 0x7f9cbc078bb0 2024-04-22T23:26:21.860 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:21.066+0000 7f9cc1e7e700 1 --2- 172.21.15.107:0/1177780556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cbc078bb0 0x7f9cbc113910 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8011e90 tx=0x7f9cb8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:21.860 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:21.066+0000 7f9cb27fc700 1 -- 172.21.15.107:0/1177780556 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9cb800beb0 con 0x7f9cbc078bb0 2024-04-22T23:26:21.860 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:21.066+0000 7f9cb27fc700 1 -- 172.21.15.107:0/1177780556 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9cb8010c80 con 0x7f9cbc078bb0 2024-04-22T23:26:21.861 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:21.066+0000 7f9cc40e2700 1 -- 172.21.15.107:0/1177780556 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cbc114050 con 0x7f9cbc078bb0 2024-04-22T23:26:21.861 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:21.066+0000 7f9cb27fc700 1 -- 172.21.15.107:0/1177780556 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9cb801ad40 con 0x7f9cbc078bb0 2024-04-22T23:26:21.861 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:21.066+0000 7f9cc40e2700 1 -- 172.21.15.107:0/1177780556 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cbc114bc0 con 0x7f9cbc078bb0 2024-04-22T23:26:21.861 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:21.067+0000 7f9cb27fc700 1 -- 172.21.15.107:0/1177780556 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f9cb802d3f0 con 0x7f9cbc078bb0 2024-04-22T23:26:21.861 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:21.067+0000 7f9cb27fc700 1 --2- 172.21.15.107:0/1177780556 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f9ca8036810 0x7f9ca8038cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:21.861 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:21.067+0000 7f9cc40e2700 1 -- 172.21.15.107:0/1177780556 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cbc05f5f0 con 0x7f9cbc078bb0 2024-04-22T23:26:21.861 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:21.067+0000 7f9cb27fc700 1 -- 172.21.15.107:0/1177780556 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f9cb804a570 con 0x7f9cbc078bb0 2024-04-22T23:26:21.861 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:21.067+0000 7f9cc167d700 1 --2- 172.21.15.107:0/1177780556 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f9ca8036810 0x7f9ca8038cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:21.861 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:21.068+0000 7f9cc167d700 1 --2- 172.21.15.107:0/1177780556 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f9ca8036810 0x7f9ca8038cc0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9cac009990 tx=0x7f9cac006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:21.861 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:21.072+0000 7f9cb27fc700 1 -- 172.21.15.107:0/1177780556 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f9cb802b6c0 con 0x7f9cbc078bb0 2024-04-22T23:26:21.861 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:21.258+0000 7f9cc40e2700 1 -- 172.21.15.107:0/1177780556 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f9cbc04bed0 con 0x7f9cbc078bb0 2024-04-22T23:26:21.861 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:21.262+0000 7f9cb27fc700 1 -- 172.21.15.107:0/1177780556 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f9cb802b300 con 0x7f9cbc078bb0 2024-04-22T23:26:21.862 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:21.263+0000 7f9cc40e2700 1 -- 172.21.15.107:0/1177780556 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f9ca8036810 msgr2=0x7f9ca8038cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:21.862 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:21.263+0000 7f9cc40e2700 1 --2- 172.21.15.107:0/1177780556 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f9ca8036810 0x7f9ca8038cc0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9cac009990 tx=0x7f9cac006e40 comp rx=0 tx=0).stop 2024-04-22T23:26:21.862 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:21.263+0000 7f9cc40e2700 1 -- 172.21.15.107:0/1177780556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cbc078bb0 msgr2=0x7f9cbc113910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:21.862 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:21.263+0000 7f9cc40e2700 1 --2- 172.21.15.107:0/1177780556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cbc078bb0 0x7f9cbc113910 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8011e90 tx=0x7f9cb8009250 comp rx=0 tx=0).stop 2024-04-22T23:26:21.862 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:21.264+0000 7f9cc40e2700 1 -- 172.21.15.107:0/1177780556 shutdown_connections 2024-04-22T23:26:21.862 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:21.264+0000 7f9cc40e2700 1 --2- 172.21.15.107:0/1177780556 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f9ca8036810 0x7f9ca8038cc0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:21.862 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:21.264+0000 7f9cc40e2700 1 --2- 172.21.15.107:0/1177780556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cbc078bb0 0x7f9cbc113910 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:21.862 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:21.264+0000 7f9cc40e2700 1 -- 172.21.15.107:0/1177780556 >> 172.21.15.107:0/1177780556 conn(0x7f9cbc0fe140 msgr2=0x7f9cbc0fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:26:21.862 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:21.264+0000 7f9cc40e2700 1 -- 172.21.15.107:0/1177780556 shutdown_connections 2024-04-22T23:26:21.862 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:21.264+0000 7f9cc40e2700 1 -- 172.21.15.107:0/1177780556 wait complete. 2024-04-22T23:26:21.862 INFO:teuthology.orchestra.run.smithi107.stdout:Enabling the dashboard module... 2024-04-22T23:26:22.490 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:22 smithi107 ceph-mon[26355]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:26:22.490 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:22 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/1177780556' entity='client.admin' 2024-04-22T23:26:23.588 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:23 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/2967814187' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-22T23:26:23.866 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:22.264+0000 7fb2bf7a8700 1 Processor -- start 2024-04-22T23:26:23.866 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:22.264+0000 7fb2bf7a8700 1 -- start start 2024-04-22T23:26:23.866 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:22.265+0000 7fb2bf7a8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2b8101800 0x7fb2b8101c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:23.866 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:22.265+0000 7fb2bf7a8700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b8102150 con 0x7fb2b8101800 2024-04-22T23:26:23.866 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:22.265+0000 7fb2bd544700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2b8101800 0x7fb2b8101c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:23.866 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:22.265+0000 7fb2bd544700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2b8101800 0x7fb2b8101c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44404/0 (socket says 172.21.15.107:44404) 2024-04-22T23:26:23.866 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:22.265+0000 7fb2bd544700 1 -- 172.21.15.107:0/2396211027 learned_addr learned my addr 172.21.15.107:0/2396211027 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:26:23.866 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:22.265+0000 7fb2bd544700 1 -- 172.21.15.107:0/2396211027 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b8102290 con 0x7fb2b8101800 2024-04-22T23:26:23.866 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:22.265+0000 7fb2bd544700 1 --2- 172.21.15.107:0/2396211027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2b8101800 0x7fb2b8101c10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb2a80098e0 tx=0x7fb2a8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a3754bc845ba297 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:23.866 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:22.265+0000 7fb2affff700 1 -- 172.21.15.107:0/2396211027 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb2a800b860 con 0x7fb2b8101800 2024-04-22T23:26:23.866 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:22.265+0000 7fb2affff700 1 -- 172.21.15.107:0/2396211027 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb2a800be90 con 0x7fb2b8101800 2024-04-22T23:26:23.867 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:22.265+0000 7fb2affff700 1 -- 172.21.15.107:0/2396211027 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb2a80103c0 con 0x7fb2b8101800 2024-04-22T23:26:23.867 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:22.266+0000 7fb2bf7a8700 1 -- 172.21.15.107:0/2396211027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2b8101800 msgr2=0x7fb2b8101c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:23.867 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:22.266+0000 7fb2bf7a8700 1 --2- 172.21.15.107:0/2396211027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2b8101800 0x7fb2b8101c10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb2a80098e0 tx=0x7fb2a8009250 comp rx=0 tx=0).stop 2024-04-22T23:26:23.867 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:22.266+0000 7fb2bf7a8700 1 -- 172.21.15.107:0/2396211027 shutdown_connections 2024-04-22T23:26:23.867 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:22.266+0000 7fb2bf7a8700 1 --2- 172.21.15.107:0/2396211027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2b8101800 0x7fb2b8101c10 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:23.867 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:22.266+0000 7fb2bf7a8700 1 -- 172.21.15.107:0/2396211027 >> 172.21.15.107:0/2396211027 conn(0x7fb2b80fcfa0 msgr2=0x7fb2b80ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:26:23.867 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:22.266+0000 7fb2bf7a8700 1 -- 172.21.15.107:0/2396211027 shutdown_connections 2024-04-22T23:26:23.867 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:22.266+0000 7fb2bf7a8700 1 -- 172.21.15.107:0/2396211027 wait complete. 2024-04-22T23:26:23.867 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:22.266+0000 7fb2bf7a8700 1 Processor -- start 2024-04-22T23:26:23.867 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:22.267+0000 7fb2bf7a8700 1 -- start start 2024-04-22T23:26:23.867 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:22.267+0000 7fb2bf7a8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2b8101800 0x7fb2b81116d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:23.867 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:22.267+0000 7fb2bf7a8700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b8111c10 con 0x7fb2b8101800 2024-04-22T23:26:23.867 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:22.267+0000 7fb2bd544700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2b8101800 0x7fb2b81116d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:23.867 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:22.267+0000 7fb2bd544700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2b8101800 0x7fb2b81116d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44408/0 (socket says 172.21.15.107:44408) 2024-04-22T23:26:23.867 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:22.267+0000 7fb2bd544700 1 -- 172.21.15.107:0/2967814187 learned_addr learned my addr 172.21.15.107:0/2967814187 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:26:23.868 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:22.267+0000 7fb2bd544700 1 -- 172.21.15.107:0/2967814187 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a80094f0 con 0x7fb2b8101800 2024-04-22T23:26:23.868 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:22.267+0000 7fb2bd544700 1 --2- 172.21.15.107:0/2967814187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2b8101800 0x7fb2b81116d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb2a80090b0 tx=0x7fb2a8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:23.868 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:22.267+0000 7fb2adffb700 1 -- 172.21.15.107:0/2967814187 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb2a8010840 con 0x7fb2b8101800 2024-04-22T23:26:23.868 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:22.267+0000 7fb2adffb700 1 -- 172.21.15.107:0/2967814187 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb2a8010e70 con 0x7fb2b8101800 2024-04-22T23:26:23.868 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:22.267+0000 7fb2adffb700 1 -- 172.21.15.107:0/2967814187 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb2a8024e40 con 0x7fb2b8101800 2024-04-22T23:26:23.868 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:22.267+0000 7fb2bf7a8700 1 -- 172.21.15.107:0/2967814187 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2b8111e10 con 0x7fb2b8101800 2024-04-22T23:26:23.868 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:22.268+0000 7fb2bf7a8700 1 -- 172.21.15.107:0/2967814187 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2b8112980 con 0x7fb2b8101800 2024-04-22T23:26:23.868 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:22.268+0000 7fb2adffb700 1 -- 172.21.15.107:0/2967814187 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fb2a8024430 con 0x7fb2b8101800 2024-04-22T23:26:23.868 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:22.269+0000 7fb2bf7a8700 1 -- 172.21.15.107:0/2967814187 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2b805f5f0 con 0x7fb2b8101800 2024-04-22T23:26:23.868 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:22.269+0000 7fb2adffb700 1 --2- 172.21.15.107:0/2967814187 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7fb2a4036740 0x7fb2a4038bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:23.868 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:22.269+0000 7fb2adffb700 1 -- 172.21.15.107:0/2967814187 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fb2a804a7a0 con 0x7fb2b8101800 2024-04-22T23:26:23.868 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:22.269+0000 7fb2bcd43700 1 --2- 172.21.15.107:0/2967814187 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7fb2a4036740 0x7fb2a4038bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:23.868 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:22.269+0000 7fb2bcd43700 1 --2- 172.21.15.107:0/2967814187 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7fb2a4036740 0x7fb2a4038bf0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4009990 tx=0x7fb2b4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:23.868 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:22.273+0000 7fb2adffb700 1 -- 172.21.15.107:0/2967814187 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb2a802c910 con 0x7fb2b8101800 2024-04-22T23:26:23.869 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:22.515+0000 7fb2bf7a8700 1 -- 172.21.15.107:0/2967814187 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7fb2b804bed0 con 0x7fb2b8101800 2024-04-22T23:26:23.869 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:23.265+0000 7fb2adffb700 1 -- 172.21.15.107:0/2967814187 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 9) v1 ==== 43389+0+0 (secure 0 0 0) 0x7fb2a802e440 con 0x7fb2b8101800 2024-04-22T23:26:23.869 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:23.267+0000 7fb2adffb700 1 -- 172.21.15.107:0/2967814187 <== mon.0 v2:172.21.15.107:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7fb2a8018350 con 0x7fb2b8101800 2024-04-22T23:26:23.869 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:23.269+0000 7fb2bf7a8700 1 -- 172.21.15.107:0/2967814187 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7fb2a4036740 msgr2=0x7fb2a4038bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:23.869 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:23.269+0000 7fb2bf7a8700 1 --2- 172.21.15.107:0/2967814187 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7fb2a4036740 0x7fb2a4038bf0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4009990 tx=0x7fb2b4006e40 comp rx=0 tx=0).stop 2024-04-22T23:26:23.869 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:23.269+0000 7fb2bf7a8700 1 -- 172.21.15.107:0/2967814187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2b8101800 msgr2=0x7fb2b81116d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:23.869 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:23.269+0000 7fb2bf7a8700 1 --2- 172.21.15.107:0/2967814187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2b8101800 0x7fb2b81116d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb2a80090b0 tx=0x7fb2a8009250 comp rx=0 tx=0).stop 2024-04-22T23:26:23.869 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:23.269+0000 7fb2bf7a8700 1 -- 172.21.15.107:0/2967814187 shutdown_connections 2024-04-22T23:26:23.869 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:23.269+0000 7fb2bf7a8700 1 --2- 172.21.15.107:0/2967814187 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7fb2a4036740 0x7fb2a4038bf0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:23.869 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:23.269+0000 7fb2bf7a8700 1 --2- 172.21.15.107:0/2967814187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2b8101800 0x7fb2b81116d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:23.869 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:23.269+0000 7fb2bf7a8700 1 -- 172.21.15.107:0/2967814187 >> 172.21.15.107:0/2967814187 conn(0x7fb2b80fcfa0 msgr2=0x7fb2b80fdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:26:23.869 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:23.269+0000 7fb2bf7a8700 1 -- 172.21.15.107:0/2967814187 shutdown_connections 2024-04-22T23:26:23.869 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:23.269+0000 7fb2bf7a8700 1 -- 172.21.15.107:0/2967814187 wait complete. 2024-04-22T23:26:24.490 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:24 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/2967814187' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-22T23:26:24.490 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:24 smithi107 ceph-mon[26355]: mgrmap e9: smithi107.uipxvm(active, since 24s) 2024-04-22T23:26:25.182 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-04-22T23:26:25.182 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-22T23:26:25.182 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-22T23:26:25.182 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "active_name": "smithi107.uipxvm", 2024-04-22T23:26:25.182 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-22T23:26:25.182 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-04-22T23:26:25.182 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:24.333+0000 7fec85af8700 1 Processor -- start 2024-04-22T23:26:25.183 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:24.333+0000 7fec85af8700 1 -- start start 2024-04-22T23:26:25.183 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:24.333+0000 7fec85af8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec80076b00 0x7fec80105590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:25.183 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:24.333+0000 7fec85af8700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec80076f10 con 0x7fec80076b00 2024-04-22T23:26:25.183 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:24.333+0000 7fec7f7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec80076b00 0x7fec80105590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:25.183 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:24.334+0000 7fec7f7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec80076b00 0x7fec80105590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44430/0 (socket says 172.21.15.107:44430) 2024-04-22T23:26:25.183 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:24.334+0000 7fec7f7fe700 1 -- 172.21.15.107:0/4023608067 learned_addr learned my addr 172.21.15.107:0/4023608067 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:26:25.183 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:24.334+0000 7fec7f7fe700 1 -- 172.21.15.107:0/4023608067 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec80105ad0 con 0x7fec80076b00 2024-04-22T23:26:25.183 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:24.334+0000 7fec7f7fe700 1 --2- 172.21.15.107:0/4023608067 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec80076b00 0x7fec80105590 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fec740098e0 tx=0x7fec74009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae35d071653b1532 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:25.183 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:24.334+0000 7fec7e7fc700 1 -- 172.21.15.107:0/4023608067 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fec7400b8f0 con 0x7fec80076b00 2024-04-22T23:26:25.183 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:24.334+0000 7fec7e7fc700 1 -- 172.21.15.107:0/4023608067 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fec74011420 con 0x7fec80076b00 2024-04-22T23:26:25.183 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:24.334+0000 7fec7e7fc700 1 -- 172.21.15.107:0/4023608067 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fec74010540 con 0x7fec80076b00 2024-04-22T23:26:25.183 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:24.335+0000 7fec85af8700 1 -- 172.21.15.107:0/4023608067 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec80076b00 msgr2=0x7fec80105590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:25.183 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:24.335+0000 7fec85af8700 1 --2- 172.21.15.107:0/4023608067 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec80076b00 0x7fec80105590 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fec740098e0 tx=0x7fec74009250 comp rx=0 tx=0).stop 2024-04-22T23:26:25.183 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:24.335+0000 7fec85af8700 1 -- 172.21.15.107:0/4023608067 shutdown_connections 2024-04-22T23:26:25.184 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:24.335+0000 7fec85af8700 1 --2- 172.21.15.107:0/4023608067 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec80076b00 0x7fec80105590 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:25.184 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:24.335+0000 7fec85af8700 1 -- 172.21.15.107:0/4023608067 >> 172.21.15.107:0/4023608067 conn(0x7fec800fd040 msgr2=0x7fec800ff450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:26:25.184 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:24.335+0000 7fec85af8700 1 -- 172.21.15.107:0/4023608067 shutdown_connections 2024-04-22T23:26:25.184 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:24.335+0000 7fec85af8700 1 -- 172.21.15.107:0/4023608067 wait complete. 2024-04-22T23:26:25.184 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:24.335+0000 7fec85af8700 1 Processor -- start 2024-04-22T23:26:25.184 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:24.336+0000 7fec85af8700 1 -- start start 2024-04-22T23:26:25.184 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:24.336+0000 7fec85af8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec80076b00 0x7fec80102ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:25.184 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:24.336+0000 7fec85af8700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec80103420 con 0x7fec80076b00 2024-04-22T23:26:25.184 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:24.336+0000 7fec7f7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec80076b00 0x7fec80102ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:25.184 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:24.336+0000 7fec7f7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec80076b00 0x7fec80102ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44436/0 (socket says 172.21.15.107:44436) 2024-04-22T23:26:25.184 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:24.336+0000 7fec7f7fe700 1 -- 172.21.15.107:0/3929140359 learned_addr learned my addr 172.21.15.107:0/3929140359 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:26:25.184 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:24.336+0000 7fec7f7fe700 1 -- 172.21.15.107:0/3929140359 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec740094f0 con 0x7fec80076b00 2024-04-22T23:26:25.184 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:24.336+0000 7fec7f7fe700 1 --2- 172.21.15.107:0/3929140359 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec80076b00 0x7fec80102ee0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fec74011cf0 tx=0x7fec74009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:25.184 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:24.336+0000 7fec67fff700 1 -- 172.21.15.107:0/3929140359 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fec74010800 con 0x7fec80076b00 2024-04-22T23:26:25.185 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:24.336+0000 7fec85af8700 1 -- 172.21.15.107:0/3929140359 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec80101490 con 0x7fec80076b00 2024-04-22T23:26:25.185 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:24.336+0000 7fec67fff700 1 -- 172.21.15.107:0/3929140359 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fec74010e30 con 0x7fec80076b00 2024-04-22T23:26:25.185 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:24.336+0000 7fec67fff700 1 -- 172.21.15.107:0/3929140359 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fec7401ae80 con 0x7fec80076b00 2024-04-22T23:26:25.185 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:24.337+0000 7fec85af8700 1 -- 172.21.15.107:0/3929140359 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec80101980 con 0x7fec80076b00 2024-04-22T23:26:25.185 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:24.337+0000 7fec85af8700 1 -- 172.21.15.107:0/3929140359 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec8004bed0 con 0x7fec80076b00 2024-04-22T23:26:25.185 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:24.337+0000 7fec67fff700 1 -- 172.21.15.107:0/3929140359 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 9) v1 ==== 43389+0+0 (secure 0 0 0) 0x7fec7402d3f0 con 0x7fec80076b00 2024-04-22T23:26:25.185 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:24.338+0000 7fec67fff700 1 --2- 172.21.15.107:0/3929140359 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7fec680367e0 0x7fec68038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:25.185 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:24.338+0000 7fec67fff700 1 -- 172.21.15.107:0/3929140359 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fec7404a150 con 0x7fec80076b00 2024-04-22T23:26:25.185 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:24.338+0000 7fec7effd700 1 -- 172.21.15.107:0/3929140359 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7fec680367e0 msgr2=0x7fec68038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/307474287 2024-04-22T23:26:25.185 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:24.338+0000 7fec7effd700 1 --2- 172.21.15.107:0/3929140359 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7fec680367e0 0x7fec68038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T23:26:25.185 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:24.342+0000 7fec67fff700 1 -- 172.21.15.107:0/3929140359 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fec7402b8e0 con 0x7fec80076b00 2024-04-22T23:26:25.185 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:24.539+0000 7fec7effd700 1 -- 172.21.15.107:0/3929140359 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7fec680367e0 msgr2=0x7fec68038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/307474287 2024-04-22T23:26:25.185 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:24.539+0000 7fec7effd700 1 --2- 172.21.15.107:0/3929140359 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7fec680367e0 0x7fec68038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T23:26:25.185 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:24.624+0000 7fec85af8700 1 -- 172.21.15.107:0/3929140359 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fec80102020 con 0x7fec80076b00 2024-04-22T23:26:25.186 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:24.624+0000 7fec67fff700 1 -- 172.21.15.107:0/3929140359 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+103 (secure 0 0 0) 0x7fec7402b2f0 con 0x7fec80076b00 2024-04-22T23:26:25.186 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:24.626+0000 7fec85af8700 1 -- 172.21.15.107:0/3929140359 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7fec680367e0 msgr2=0x7fec68038c90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T23:26:25.186 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:24.626+0000 7fec85af8700 1 --2- 172.21.15.107:0/3929140359 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7fec680367e0 0x7fec68038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:25.186 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:24.626+0000 7fec85af8700 1 -- 172.21.15.107:0/3929140359 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec80076b00 msgr2=0x7fec80102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:25.186 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:24.626+0000 7fec85af8700 1 --2- 172.21.15.107:0/3929140359 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec80076b00 0x7fec80102ee0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fec74011cf0 tx=0x7fec74009250 comp rx=0 tx=0).stop 2024-04-22T23:26:25.186 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:24.626+0000 7fec85af8700 1 -- 172.21.15.107:0/3929140359 shutdown_connections 2024-04-22T23:26:25.186 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:24.626+0000 7fec85af8700 1 --2- 172.21.15.107:0/3929140359 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7fec680367e0 0x7fec68038c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:25.186 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:24.626+0000 7fec85af8700 1 --2- 172.21.15.107:0/3929140359 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec80076b00 0x7fec80102ee0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:25.186 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:24.626+0000 7fec85af8700 1 -- 172.21.15.107:0/3929140359 >> 172.21.15.107:0/3929140359 conn(0x7fec800fd040 msgr2=0x7fec800fdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:26:25.186 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:24.627+0000 7fec85af8700 1 -- 172.21.15.107:0/3929140359 shutdown_connections 2024-04-22T23:26:25.186 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:24.627+0000 7fec85af8700 1 -- 172.21.15.107:0/3929140359 wait complete. 2024-04-22T23:26:25.186 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for the mgr to restart... 2024-04-22T23:26:25.186 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mgr epoch 9... 2024-04-22T23:26:25.525 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:25 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/3929140359' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-22T23:26:29.863 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:29 smithi107 ceph-mon[26355]: Active manager daemon smithi107.uipxvm restarted 2024-04-22T23:26:29.863 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:29 smithi107 ceph-mon[26355]: Activating manager daemon smithi107.uipxvm 2024-04-22T23:26:29.863 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:29 smithi107 ceph-mon[26355]: osdmap e3: 0 total, 0 up, 0 in 2024-04-22T23:26:29.864 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:29 smithi107 ceph-mon[26355]: mgrmap e10: smithi107.uipxvm(active, starting, since 0.00309177s) 2024-04-22T23:26:29.864 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:29 smithi107 ceph-mon[26355]: from='mgr.14160 172.21.15.107:0/1348717222' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-04-22T23:26:29.864 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:29 smithi107 ceph-mon[26355]: from='mgr.14160 172.21.15.107:0/1348717222' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mgr metadata", "who": "smithi107.uipxvm", "id": "smithi107.uipxvm"}]: dispatch 2024-04-22T23:26:29.864 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:29 smithi107 ceph-mon[26355]: from='mgr.14160 172.21.15.107:0/1348717222' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-22T23:26:29.864 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:29 smithi107 ceph-mon[26355]: from='mgr.14160 172.21.15.107:0/1348717222' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-22T23:26:29.864 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:29 smithi107 ceph-mon[26355]: from='mgr.14160 172.21.15.107:0/1348717222' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-22T23:26:29.864 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:29 smithi107 ceph-mon[26355]: Manager daemon smithi107.uipxvm is now available 2024-04-22T23:26:29.864 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:29 smithi107 ceph-mon[26355]: from='mgr.14160 172.21.15.107:0/1348717222' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:26:29.864 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:29 smithi107 ceph-mon[26355]: from='mgr.14160 172.21.15.107:0/1348717222' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:26:29.864 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:29 smithi107 ceph-mon[26355]: from='mgr.14160 172.21.15.107:0/1348717222' entity='mgr.smithi107.uipxvm' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi107.uipxvm/mirror_snapshot_schedule"}]: dispatch 2024-04-22T23:26:29.864 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:29 smithi107 ceph-mon[26355]: from='mgr.14160 172.21.15.107:0/1348717222' entity='mgr.smithi107.uipxvm' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi107.uipxvm/trash_purge_schedule"}]: dispatch 2024-04-22T23:26:31.566 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-04-22T23:26:31.566 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-22T23:26:31.567 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-22T23:26:31.567 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-04-22T23:26:31.567 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:25.624+0000 7f09ead9d700 1 Processor -- start 2024-04-22T23:26:31.567 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:25.624+0000 7f09ead9d700 1 -- start start 2024-04-22T23:26:31.567 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:25.624+0000 7f09ead9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09e4076b20 0x7f09e41055a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:31.567 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:25.624+0000 7f09ead9d700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e4076f30 con 0x7f09e4076b20 2024-04-22T23:26:31.567 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:25.624+0000 7f09e8b39700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09e4076b20 0x7f09e41055a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:31.567 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:25.624+0000 7f09e8b39700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09e4076b20 0x7f09e41055a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44438/0 (socket says 172.21.15.107:44438) 2024-04-22T23:26:31.567 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:25.624+0000 7f09e8b39700 1 -- 172.21.15.107:0/152547871 learned_addr learned my addr 172.21.15.107:0/152547871 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:26:31.567 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:25.624+0000 7f09e8b39700 1 -- 172.21.15.107:0/152547871 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e4105ae0 con 0x7f09e4076b20 2024-04-22T23:26:31.568 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:25.624+0000 7f09e8b39700 1 --2- 172.21.15.107:0/152547871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09e4076b20 0x7f09e41055a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f09d80098e0 tx=0x7f09d8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da4c2bf79487cc73 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:31.568 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:25.625+0000 7f09e37fe700 1 -- 172.21.15.107:0/152547871 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f09d800b8f0 con 0x7f09e4076b20 2024-04-22T23:26:31.568 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:25.625+0000 7f09e37fe700 1 -- 172.21.15.107:0/152547871 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f09d8011420 con 0x7f09e4076b20 2024-04-22T23:26:31.568 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:25.625+0000 7f09e37fe700 1 -- 172.21.15.107:0/152547871 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f09d8010540 con 0x7f09e4076b20 2024-04-22T23:26:31.568 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:25.625+0000 7f09ead9d700 1 -- 172.21.15.107:0/152547871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09e4076b20 msgr2=0x7f09e41055a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:31.568 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:25.625+0000 7f09ead9d700 1 --2- 172.21.15.107:0/152547871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09e4076b20 0x7f09e41055a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f09d80098e0 tx=0x7f09d8009250 comp rx=0 tx=0).stop 2024-04-22T23:26:31.568 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:25.625+0000 7f09ead9d700 1 -- 172.21.15.107:0/152547871 shutdown_connections 2024-04-22T23:26:31.568 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:25.625+0000 7f09ead9d700 1 --2- 172.21.15.107:0/152547871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09e4076b20 0x7f09e41055a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:31.568 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:25.625+0000 7f09ead9d700 1 -- 172.21.15.107:0/152547871 >> 172.21.15.107:0/152547871 conn(0x7f09e40fd070 msgr2=0x7f09e40ff480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:26:31.568 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:25.625+0000 7f09ead9d700 1 -- 172.21.15.107:0/152547871 shutdown_connections 2024-04-22T23:26:31.568 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:25.625+0000 7f09ead9d700 1 -- 172.21.15.107:0/152547871 wait complete. 2024-04-22T23:26:31.568 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:25.626+0000 7f09ead9d700 1 Processor -- start 2024-04-22T23:26:31.568 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:25.626+0000 7f09ead9d700 1 -- start start 2024-04-22T23:26:31.568 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:25.626+0000 7f09ead9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09e4076b20 0x7f09e41116a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:31.569 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:25.626+0000 7f09ead9d700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e4111be0 con 0x7f09e4076b20 2024-04-22T23:26:31.569 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:25.626+0000 7f09e8b39700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09e4076b20 0x7f09e41116a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:31.569 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:25.626+0000 7f09e8b39700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09e4076b20 0x7f09e41116a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44442/0 (socket says 172.21.15.107:44442) 2024-04-22T23:26:31.569 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:25.626+0000 7f09e8b39700 1 -- 172.21.15.107:0/3138581819 learned_addr learned my addr 172.21.15.107:0/3138581819 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:26:31.569 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:25.627+0000 7f09e8b39700 1 -- 172.21.15.107:0/3138581819 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d80094f0 con 0x7f09e4076b20 2024-04-22T23:26:31.569 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:25.627+0000 7f09e8b39700 1 --2- 172.21.15.107:0/3138581819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09e4076b20 0x7f09e41116a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f09d8011cf0 tx=0x7f09d8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:31.569 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:25.627+0000 7f09e17fa700 1 -- 172.21.15.107:0/3138581819 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f09d8010800 con 0x7f09e4076b20 2024-04-22T23:26:31.569 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:25.627+0000 7f09e17fa700 1 -- 172.21.15.107:0/3138581819 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f09d8010e30 con 0x7f09e4076b20 2024-04-22T23:26:31.569 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:25.627+0000 7f09e17fa700 1 -- 172.21.15.107:0/3138581819 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f09d801ae80 con 0x7f09e4076b20 2024-04-22T23:26:31.569 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:25.627+0000 7f09ead9d700 1 -- 172.21.15.107:0/3138581819 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09e4111de0 con 0x7f09e4076b20 2024-04-22T23:26:31.569 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:25.627+0000 7f09ead9d700 1 -- 172.21.15.107:0/3138581819 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09e4112950 con 0x7f09e4076b20 2024-04-22T23:26:31.569 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:25.628+0000 7f09e17fa700 1 -- 172.21.15.107:0/3138581819 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 9) v1 ==== 43389+0+0 (secure 0 0 0) 0x7f09d802d3f0 con 0x7f09e4076b20 2024-04-22T23:26:31.570 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:25.628+0000 7f09e17fa700 1 --2- 172.21.15.107:0/3138581819 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f09d4036830 0x7f09d4038ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:31.570 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:25.628+0000 7f09ead9d700 1 -- 172.21.15.107:0/3138581819 --> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f09e405f5f0 con 0x7f09d4036830 2024-04-22T23:26:31.570 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:25.628+0000 7f09e17fa700 1 -- 172.21.15.107:0/3138581819 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f09d804a180 con 0x7f09e4076b20 2024-04-22T23:26:31.570 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:25.628+0000 7f09e3fff700 1 -- 172.21.15.107:0/3138581819 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f09d4036830 msgr2=0x7f09d4038ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/307474287 2024-04-22T23:26:31.570 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:25.628+0000 7f09e3fff700 1 --2- 172.21.15.107:0/3138581819 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f09d4036830 0x7f09d4038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T23:26:31.570 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:25.829+0000 7f09e3fff700 1 -- 172.21.15.107:0/3138581819 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f09d4036830 msgr2=0x7f09d4038ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/307474287 2024-04-22T23:26:31.570 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:25.829+0000 7f09e3fff700 1 --2- 172.21.15.107:0/3138581819 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f09d4036830 0x7f09d4038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T23:26:31.570 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:26.230+0000 7f09e3fff700 1 -- 172.21.15.107:0/3138581819 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f09d4036830 msgr2=0x7f09d4038ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/307474287 2024-04-22T23:26:31.570 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:26.230+0000 7f09e3fff700 1 --2- 172.21.15.107:0/3138581819 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f09d4036830 0x7f09d4038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T23:26:31.570 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:27.031+0000 7f09e3fff700 1 -- 172.21.15.107:0/3138581819 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f09d4036830 msgr2=0x7f09d4038ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/307474287 2024-04-22T23:26:31.570 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:27.031+0000 7f09e3fff700 1 --2- 172.21.15.107:0/3138581819 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f09d4036830 0x7f09d4038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T23:26:31.570 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:28.632+0000 7f09e3fff700 1 -- 172.21.15.107:0/3138581819 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f09d4036830 msgr2=0x7f09d4038ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/307474287 2024-04-22T23:26:31.570 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:28.632+0000 7f09e3fff700 1 --2- 172.21.15.107:0/3138581819 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f09d4036830 0x7f09d4038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-22T23:26:31.571 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:29.808+0000 7f09e17fa700 1 -- 172.21.15.107:0/3138581819 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mgrmap(e 10) v1 ==== 43156+0+0 (secure 0 0 0) 0x7f09d801a430 con 0x7f09e4076b20 2024-04-22T23:26:31.571 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:29.808+0000 7f09e17fa700 1 -- 172.21.15.107:0/3138581819 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f09d4036830 msgr2=0x7f09d4038ce0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T23:26:31.571 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:29.808+0000 7f09e17fa700 1 --2- 172.21.15.107:0/3138581819 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f09d4036830 0x7f09d4038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:31.571 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:30.810+0000 7f09e17fa700 1 -- 172.21.15.107:0/3138581819 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 11) v1 ==== 43283+0+0 (secure 0 0 0) 0x7f09d804b7f0 con 0x7f09e4076b20 2024-04-22T23:26:31.571 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:30.810+0000 7f09e17fa700 1 --2- 172.21.15.107:0/3138581819 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f09d4039ff0 0x7f09d403c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:31.571 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:30.810+0000 7f09e17fa700 1 -- 172.21.15.107:0/3138581819 --> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f09e405f5f0 con 0x7f09d4039ff0 2024-04-22T23:26:31.571 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:30.810+0000 7f09e3fff700 1 --2- 172.21.15.107:0/3138581819 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f09d4039ff0 0x7f09d403c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:31.571 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:30.811+0000 7f09e3fff700 1 --2- 172.21.15.107:0/3138581819 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f09d4039ff0 0x7f09d403c3d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f09d00039d0 tx=0x7f09d0009350 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:31.571 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:30.812+0000 7f09e17fa700 1 -- 172.21.15.107:0/3138581819 <== mgr.14160 v2:172.21.15.107:6800/3427816626 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7f09e405f5f0 con 0x7f09d4039ff0 2024-04-22T23:26:31.571 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:30.817+0000 7f09ead9d700 1 -- 172.21.15.107:0/3138581819 --> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f09e4112270 con 0x7f09d4039ff0 2024-04-22T23:26:31.571 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:30.818+0000 7f09e17fa700 1 -- 172.21.15.107:0/3138581819 <== mgr.14160 v2:172.21.15.107:6800/3427816626 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f09e4112270 con 0x7f09d4039ff0 2024-04-22T23:26:31.571 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:30.818+0000 7f09ead9d700 1 -- 172.21.15.107:0/3138581819 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f09d4039ff0 msgr2=0x7f09d403c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:31.571 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:30.818+0000 7f09ead9d700 1 --2- 172.21.15.107:0/3138581819 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f09d4039ff0 0x7f09d403c3d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f09d00039d0 tx=0x7f09d0009350 comp rx=0 tx=0).stop 2024-04-22T23:26:31.571 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:30.818+0000 7f09ead9d700 1 -- 172.21.15.107:0/3138581819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09e4076b20 msgr2=0x7f09e41116a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:31.572 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:30.818+0000 7f09ead9d700 1 --2- 172.21.15.107:0/3138581819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09e4076b20 0x7f09e41116a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f09d8011cf0 tx=0x7f09d8009250 comp rx=0 tx=0).stop 2024-04-22T23:26:31.572 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:30.818+0000 7f09ead9d700 1 -- 172.21.15.107:0/3138581819 shutdown_connections 2024-04-22T23:26:31.572 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:30.818+0000 7f09ead9d700 1 --2- 172.21.15.107:0/3138581819 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f09d4039ff0 0x7f09d403c3d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:31.572 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:30.818+0000 7f09ead9d700 1 --2- 172.21.15.107:0/3138581819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09e4076b20 0x7f09e41116a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:31.572 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:30.818+0000 7f09ead9d700 1 --2- 172.21.15.107:0/3138581819 >> [v2:172.21.15.107:6800/307474287,v1:172.21.15.107:6801/307474287] conn(0x7f09d4036830 0x7f09d4038ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:31.572 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:30.818+0000 7f09ead9d700 1 -- 172.21.15.107:0/3138581819 >> 172.21.15.107:0/3138581819 conn(0x7f09e40fd070 msgr2=0x7f09e40fdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:26:31.572 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:30.818+0000 7f09ead9d700 1 -- 172.21.15.107:0/3138581819 shutdown_connections 2024-04-22T23:26:31.572 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:30.818+0000 7f09ead9d700 1 -- 172.21.15.107:0/3138581819 wait complete. 2024-04-22T23:26:31.572 INFO:teuthology.orchestra.run.smithi107.stdout:mgr epoch 9 is available 2024-04-22T23:26:31.572 INFO:teuthology.orchestra.run.smithi107.stdout:Generating a dashboard self-signed certificate... 2024-04-22T23:26:31.909 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:31 smithi107 ceph-mon[26355]: [22/Apr/2024:23:26:30] ENGINE Bus STARTING 2024-04-22T23:26:31.909 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:31 smithi107 ceph-mon[26355]: [22/Apr/2024:23:26:30] ENGINE Serving on https://172.21.15.107:7150 2024-04-22T23:26:31.910 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:31 smithi107 ceph-mon[26355]: [22/Apr/2024:23:26:30] ENGINE Bus STARTED 2024-04-22T23:26:31.910 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:31 smithi107 ceph-mon[26355]: from='mgr.14160 172.21.15.107:0/1348717222' entity='mgr.smithi107.uipxvm' 2024-04-22T23:26:31.910 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:31 smithi107 ceph-mon[26355]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-22T23:26:31.910 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:31 smithi107 ceph-mon[26355]: mgrmap e11: smithi107.uipxvm(active, since 1.00547s) 2024-04-22T23:26:31.910 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:31 smithi107 ceph-mon[26355]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-22T23:26:31.910 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:31 smithi107 ceph-mon[26355]: from='mgr.14160 172.21.15.107:0/1348717222' entity='mgr.smithi107.uipxvm' 2024-04-22T23:26:32.861 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-22T23:26:32.861 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:32.004+0000 7f159a747700 1 Processor -- start 2024-04-22T23:26:32.861 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:32.004+0000 7f159a747700 1 -- start start 2024-04-22T23:26:32.861 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:32.004+0000 7f159a747700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1594078be0 0x7f1594077280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:32.861 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:32.004+0000 7f159a747700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1594078ff0 con 0x7f1594078be0 2024-04-22T23:26:32.862 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:32.004+0000 7f1593fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1594078be0 0x7f1594077280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:32.862 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:32.004+0000 7f1593fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1594078be0 0x7f1594077280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57690/0 (socket says 172.21.15.107:57690) 2024-04-22T23:26:32.862 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:32.004+0000 7f1593fff700 1 -- 172.21.15.107:0/2893827987 learned_addr learned my addr 172.21.15.107:0/2893827987 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:26:32.862 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:32.004+0000 7f1593fff700 1 -- 172.21.15.107:0/2893827987 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1594079130 con 0x7f1594078be0 2024-04-22T23:26:32.862 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:32.004+0000 7f1593fff700 1 --2- 172.21.15.107:0/2893827987 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1594078be0 0x7f1594077280 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f15840098e0 tx=0x7f1584009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecd97568bdf42eca server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:32.862 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:32.005+0000 7f1592ffd700 1 -- 172.21.15.107:0/2893827987 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f158400b8f0 con 0x7f1594078be0 2024-04-22T23:26:32.862 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:32.005+0000 7f1592ffd700 1 -- 172.21.15.107:0/2893827987 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1584011420 con 0x7f1594078be0 2024-04-22T23:26:32.862 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:32.005+0000 7f1592ffd700 1 -- 172.21.15.107:0/2893827987 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1584010540 con 0x7f1594078be0 2024-04-22T23:26:32.862 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:32.005+0000 7f159a747700 1 -- 172.21.15.107:0/2893827987 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1594078be0 msgr2=0x7f1594077280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:32.862 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:32.005+0000 7f159a747700 1 --2- 172.21.15.107:0/2893827987 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1594078be0 0x7f1594077280 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f15840098e0 tx=0x7f1584009250 comp rx=0 tx=0).stop 2024-04-22T23:26:32.862 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:32.005+0000 7f159a747700 1 -- 172.21.15.107:0/2893827987 shutdown_connections 2024-04-22T23:26:32.862 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:32.005+0000 7f159a747700 1 --2- 172.21.15.107:0/2893827987 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1594078be0 0x7f1594077280 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:32.863 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:32.005+0000 7f159a747700 1 -- 172.21.15.107:0/2893827987 >> 172.21.15.107:0/2893827987 conn(0x7f15940fe0e0 msgr2=0x7f1594100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:26:32.863 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:32.005+0000 7f159a747700 1 -- 172.21.15.107:0/2893827987 shutdown_connections 2024-04-22T23:26:32.863 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:32.005+0000 7f159a747700 1 -- 172.21.15.107:0/2893827987 wait complete. 2024-04-22T23:26:32.863 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:32.006+0000 7f159a747700 1 Processor -- start 2024-04-22T23:26:32.863 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:32.006+0000 7f159a747700 1 -- start start 2024-04-22T23:26:32.863 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:32.006+0000 7f159a747700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1594078be0 0x7f1594113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:32.863 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:32.006+0000 7f159a747700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1594113eb0 con 0x7f1594078be0 2024-04-22T23:26:32.863 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:32.006+0000 7f1593fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1594078be0 0x7f1594113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:32.863 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:32.006+0000 7f1593fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1594078be0 0x7f1594113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57698/0 (socket says 172.21.15.107:57698) 2024-04-22T23:26:32.863 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:32.006+0000 7f1593fff700 1 -- 172.21.15.107:0/3112341294 learned_addr learned my addr 172.21.15.107:0/3112341294 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:26:32.863 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:32.006+0000 7f1593fff700 1 -- 172.21.15.107:0/3112341294 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15840094f0 con 0x7f1594078be0 2024-04-22T23:26:32.863 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:32.007+0000 7f1593fff700 1 --2- 172.21.15.107:0/3112341294 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1594078be0 0x7f1594113970 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1584011cf0 tx=0x7f1584009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:32.863 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:32.007+0000 7f1590ff9700 1 -- 172.21.15.107:0/3112341294 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1584010800 con 0x7f1594078be0 2024-04-22T23:26:32.863 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:32.007+0000 7f1590ff9700 1 -- 172.21.15.107:0/3112341294 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1584010e30 con 0x7f1594078be0 2024-04-22T23:26:32.864 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:32.007+0000 7f1590ff9700 1 -- 172.21.15.107:0/3112341294 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f158401ae80 con 0x7f1594078be0 2024-04-22T23:26:32.864 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:32.007+0000 7f159a747700 1 -- 172.21.15.107:0/3112341294 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15941140b0 con 0x7f1594078be0 2024-04-22T23:26:32.864 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:32.007+0000 7f159a747700 1 -- 172.21.15.107:0/3112341294 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1594114c20 con 0x7f1594078be0 2024-04-22T23:26:32.864 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:32.008+0000 7f1590ff9700 1 -- 172.21.15.107:0/3112341294 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 11) v1 ==== 43283+0+0 (secure 0 0 0) 0x7f1584010960 con 0x7f1594078be0 2024-04-22T23:26:32.864 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:32.008+0000 7f1590ff9700 1 --2- 172.21.15.107:0/3112341294 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f157c0366b0 0x7f157c038b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:32.864 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:32.008+0000 7f159a747700 1 -- 172.21.15.107:0/3112341294 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f159405f5f0 con 0x7f1594078be0 2024-04-22T23:26:32.864 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:32.008+0000 7f1590ff9700 1 -- 172.21.15.107:0/3112341294 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f158404a0d0 con 0x7f1594078be0 2024-04-22T23:26:32.864 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:32.008+0000 7f15937fe700 1 --2- 172.21.15.107:0/3112341294 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f157c0366b0 0x7f157c038b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:32.864 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:32.008+0000 7f15937fe700 1 --2- 172.21.15.107:0/3112341294 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f157c0366b0 0x7f157c038b60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1588009990 tx=0x7f1588006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:32.864 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:32.013+0000 7f1590ff9700 1 -- 172.21.15.107:0/3112341294 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f158402b760 con 0x7f1594078be0 2024-04-22T23:26:32.864 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:32.212+0000 7f159a747700 1 -- 172.21.15.107:0/3112341294 --> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f15940ff4c0 con 0x7f157c0366b0 2024-04-22T23:26:32.864 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:32.273+0000 7f1590ff9700 1 -- 172.21.15.107:0/3112341294 <== mgr.14160 v2:172.21.15.107:6800/3427816626 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f15940ff4c0 con 0x7f157c0366b0 2024-04-22T23:26:32.864 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:32.275+0000 7f159a747700 1 -- 172.21.15.107:0/3112341294 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f157c0366b0 msgr2=0x7f157c038b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:32.865 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:32.275+0000 7f159a747700 1 --2- 172.21.15.107:0/3112341294 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f157c0366b0 0x7f157c038b60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1588009990 tx=0x7f1588006e40 comp rx=0 tx=0).stop 2024-04-22T23:26:32.865 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:32.275+0000 7f159a747700 1 -- 172.21.15.107:0/3112341294 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1594078be0 msgr2=0x7f1594113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:32.865 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:32.275+0000 7f159a747700 1 --2- 172.21.15.107:0/3112341294 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1594078be0 0x7f1594113970 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1584011cf0 tx=0x7f1584009250 comp rx=0 tx=0).stop 2024-04-22T23:26:32.865 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:32.275+0000 7f159a747700 1 -- 172.21.15.107:0/3112341294 shutdown_connections 2024-04-22T23:26:32.865 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:32.275+0000 7f159a747700 1 --2- 172.21.15.107:0/3112341294 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f157c0366b0 0x7f157c038b60 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:32.865 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:32.275+0000 7f159a747700 1 --2- 172.21.15.107:0/3112341294 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1594078be0 0x7f1594113970 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:32.865 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:32.275+0000 7f159a747700 1 -- 172.21.15.107:0/3112341294 >> 172.21.15.107:0/3112341294 conn(0x7f15940fe0e0 msgr2=0x7f15940fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:26:32.865 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:32.275+0000 7f159a747700 1 -- 172.21.15.107:0/3112341294 shutdown_connections 2024-04-22T23:26:32.865 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:32.275+0000 7f159a747700 1 -- 172.21.15.107:0/3112341294 wait complete. 2024-04-22T23:26:32.865 INFO:teuthology.orchestra.run.smithi107.stdout:Creating initial admin user... 2024-04-22T23:26:33.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:33 smithi107 ceph-mon[26355]: from='mgr.14160 172.21.15.107:0/1348717222' entity='mgr.smithi107.uipxvm' 2024-04-22T23:26:33.240 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:33 smithi107 ceph-mon[26355]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:26:33.240 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:33 smithi107 ceph-mon[26355]: from='mgr.14160 172.21.15.107:0/1348717222' entity='mgr.smithi107.uipxvm' 2024-04-22T23:26:33.240 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:33 smithi107 ceph-mon[26355]: from='mgr.14160 172.21.15.107:0/1348717222' entity='mgr.smithi107.uipxvm' 2024-04-22T23:26:33.240 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:33 smithi107 ceph-mon[26355]: mgrmap e12: smithi107.uipxvm(active, since 2s) 2024-04-22T23:26:34.572 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$wOnKhp86zOsg9jAHPX3hk.Rob3ONBsJp7tnIxf.vPSJm4EwX9C1LK", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1713828393, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-22T23:26:34.572 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:33.307+0000 7f4653d12700 1 Processor -- start 2024-04-22T23:26:34.572 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:33.307+0000 7f4653d12700 1 -- start start 2024-04-22T23:26:34.572 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:33.308+0000 7f4653d12700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f464c071280 0x7f464c107970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:34.572 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:33.308+0000 7f4653d12700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f464c071690 con 0x7f464c071280 2024-04-22T23:26:34.572 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:33.308+0000 7f4651aae700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f464c071280 0x7f464c107970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:34.572 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:33.308+0000 7f4651aae700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f464c071280 0x7f464c107970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57714/0 (socket says 172.21.15.107:57714) 2024-04-22T23:26:34.572 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:33.308+0000 7f4651aae700 1 -- 172.21.15.107:0/2258743060 learned_addr learned my addr 172.21.15.107:0/2258743060 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:26:34.572 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:33.308+0000 7f4651aae700 1 -- 172.21.15.107:0/2258743060 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f464c103360 con 0x7f464c071280 2024-04-22T23:26:34.572 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:33.308+0000 7f4651aae700 1 --2- 172.21.15.107:0/2258743060 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f464c071280 0x7f464c107970 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f463c0098e0 tx=0x7f463c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bd56dc582df92db server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:34.572 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:33.308+0000 7f4650aac700 1 -- 172.21.15.107:0/2258743060 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f463c00b860 con 0x7f464c071280 2024-04-22T23:26:34.573 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:33.308+0000 7f4650aac700 1 -- 172.21.15.107:0/2258743060 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f463c00be90 con 0x7f464c071280 2024-04-22T23:26:34.573 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:33.308+0000 7f4650aac700 1 -- 172.21.15.107:0/2258743060 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f463c0103c0 con 0x7f464c071280 2024-04-22T23:26:34.573 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:33.309+0000 7f4653d12700 1 -- 172.21.15.107:0/2258743060 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f464c071280 msgr2=0x7f464c107970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:34.573 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:33.309+0000 7f4653d12700 1 --2- 172.21.15.107:0/2258743060 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f464c071280 0x7f464c107970 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f463c0098e0 tx=0x7f463c009250 comp rx=0 tx=0).stop 2024-04-22T23:26:34.573 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:33.309+0000 7f4653d12700 1 -- 172.21.15.107:0/2258743060 shutdown_connections 2024-04-22T23:26:34.573 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:33.309+0000 7f4653d12700 1 --2- 172.21.15.107:0/2258743060 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f464c071280 0x7f464c107970 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:34.573 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:33.309+0000 7f4653d12700 1 -- 172.21.15.107:0/2258743060 >> 172.21.15.107:0/2258743060 conn(0x7f464c0fcf80 msgr2=0x7f464c0ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:26:34.573 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:33.309+0000 7f4653d12700 1 -- 172.21.15.107:0/2258743060 shutdown_connections 2024-04-22T23:26:34.573 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:33.309+0000 7f4653d12700 1 -- 172.21.15.107:0/2258743060 wait complete. 2024-04-22T23:26:34.573 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:33.309+0000 7f4653d12700 1 Processor -- start 2024-04-22T23:26:34.573 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:33.309+0000 7f4653d12700 1 -- start start 2024-04-22T23:26:34.573 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:33.310+0000 7f4653d12700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f464c071280 0x7f464c113900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:34.573 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:33.310+0000 7f4653d12700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f464c113e40 con 0x7f464c071280 2024-04-22T23:26:34.573 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:33.310+0000 7f4651aae700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f464c071280 0x7f464c113900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:34.574 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:33.310+0000 7f4651aae700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f464c071280 0x7f464c113900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57722/0 (socket says 172.21.15.107:57722) 2024-04-22T23:26:34.574 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:33.310+0000 7f4651aae700 1 -- 172.21.15.107:0/2353394315 learned_addr learned my addr 172.21.15.107:0/2353394315 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:26:34.574 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:33.310+0000 7f4651aae700 1 -- 172.21.15.107:0/2353394315 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f463c0094f0 con 0x7f464c071280 2024-04-22T23:26:34.574 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:33.310+0000 7f4651aae700 1 --2- 172.21.15.107:0/2353394315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f464c071280 0x7f464c113900 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f463c0105b0 tx=0x7f463c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:34.574 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:33.310+0000 7f46427fc700 1 -- 172.21.15.107:0/2353394315 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f463c010840 con 0x7f464c071280 2024-04-22T23:26:34.574 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:33.310+0000 7f46427fc700 1 -- 172.21.15.107:0/2353394315 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f463c010e70 con 0x7f464c071280 2024-04-22T23:26:34.574 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:33.310+0000 7f4653d12700 1 -- 172.21.15.107:0/2353394315 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f464c1140a0 con 0x7f464c071280 2024-04-22T23:26:34.574 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:33.310+0000 7f46427fc700 1 -- 172.21.15.107:0/2353394315 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f463c01ae30 con 0x7f464c071280 2024-04-22T23:26:34.574 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:33.310+0000 7f4653d12700 1 -- 172.21.15.107:0/2353394315 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f464c114bb0 con 0x7f464c071280 2024-04-22T23:26:34.574 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:33.311+0000 7f46427fc700 1 -- 172.21.15.107:0/2353394315 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 12) v1 ==== 43389+0+0 (secure 0 0 0) 0x7f463c02d3f0 con 0x7f464c071280 2024-04-22T23:26:34.574 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:33.311+0000 7f4653d12700 1 -- 172.21.15.107:0/2353394315 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f464c05f5f0 con 0x7f464c071280 2024-04-22T23:26:34.574 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:33.311+0000 7f46427fc700 1 --2- 172.21.15.107:0/2353394315 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f46380367b0 0x7f4638038c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:34.575 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:33.311+0000 7f46427fc700 1 -- 172.21.15.107:0/2353394315 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f463c04a3e0 con 0x7f464c071280 2024-04-22T23:26:34.575 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:33.311+0000 7f46512ad700 1 --2- 172.21.15.107:0/2353394315 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f46380367b0 0x7f4638038c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:34.575 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:33.312+0000 7f46512ad700 1 --2- 172.21.15.107:0/2353394315 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f46380367b0 0x7f4638038c60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4648009990 tx=0x7f4648006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:34.575 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:33.316+0000 7f46427fc700 1 -- 172.21.15.107:0/2353394315 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f463c02b4d0 con 0x7f464c071280 2024-04-22T23:26:34.575 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:33.536+0000 7f4653d12700 1 -- 172.21.15.107:0/2353394315 --> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f464c05f410 con 0x7f46380367b0 2024-04-22T23:26:34.575 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:33.760+0000 7f46427fc700 1 -- 172.21.15.107:0/2353394315 <== mgr.14160 v2:172.21.15.107:6800/3427816626 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f464c05f410 con 0x7f46380367b0 2024-04-22T23:26:34.575 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:33.762+0000 7f4653d12700 1 -- 172.21.15.107:0/2353394315 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f46380367b0 msgr2=0x7f4638038c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:34.575 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:33.762+0000 7f4653d12700 1 --2- 172.21.15.107:0/2353394315 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f46380367b0 0x7f4638038c60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4648009990 tx=0x7f4648006e40 comp rx=0 tx=0).stop 2024-04-22T23:26:34.575 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:33.762+0000 7f4653d12700 1 -- 172.21.15.107:0/2353394315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f464c071280 msgr2=0x7f464c113900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:34.575 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:33.762+0000 7f4653d12700 1 --2- 172.21.15.107:0/2353394315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f464c071280 0x7f464c113900 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f463c0105b0 tx=0x7f463c009250 comp rx=0 tx=0).stop 2024-04-22T23:26:34.575 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:33.763+0000 7f4653d12700 1 -- 172.21.15.107:0/2353394315 shutdown_connections 2024-04-22T23:26:34.575 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:33.763+0000 7f4653d12700 1 --2- 172.21.15.107:0/2353394315 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f46380367b0 0x7f4638038c60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:34.575 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:33.763+0000 7f4653d12700 1 --2- 172.21.15.107:0/2353394315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f464c071280 0x7f464c113900 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:34.575 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:33.763+0000 7f4653d12700 1 -- 172.21.15.107:0/2353394315 >> 172.21.15.107:0/2353394315 conn(0x7f464c0fcf80 msgr2=0x7f464c0fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:26:34.576 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:33.763+0000 7f4653d12700 1 -- 172.21.15.107:0/2353394315 shutdown_connections 2024-04-22T23:26:34.576 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:33.763+0000 7f4653d12700 1 -- 172.21.15.107:0/2353394315 wait complete. 2024-04-22T23:26:34.576 INFO:teuthology.orchestra.run.smithi107.stdout:Fetching dashboard port number... 2024-04-22T23:26:34.941 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:34 smithi107 ceph-mon[26355]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:26:34.941 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:34 smithi107 ceph-mon[26355]: from='mgr.14160 172.21.15.107:0/1348717222' entity='mgr.smithi107.uipxvm' 2024-04-22T23:26:35.989 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:35 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/3015343606' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-22T23:26:36.017 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 8443 2024-04-22T23:26:36.017 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:35.039+0000 7fcf824c9700 1 Processor -- start 2024-04-22T23:26:36.017 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:35.039+0000 7fcf824c9700 1 -- start start 2024-04-22T23:26:36.018 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:35.039+0000 7fcf824c9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf7c101d60 0x7fcf7c102170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:36.018 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:35.039+0000 7fcf824c9700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c1026b0 con 0x7fcf7c101d60 2024-04-22T23:26:36.018 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:35.039+0000 7fcf7bfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf7c101d60 0x7fcf7c102170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:36.018 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:35.039+0000 7fcf7bfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf7c101d60 0x7fcf7c102170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57736/0 (socket says 172.21.15.107:57736) 2024-04-22T23:26:36.018 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:35.039+0000 7fcf7bfff700 1 -- 172.21.15.107:0/2801442072 learned_addr learned my addr 172.21.15.107:0/2801442072 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:26:36.018 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:35.039+0000 7fcf7bfff700 1 -- 172.21.15.107:0/2801442072 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf7c1027f0 con 0x7fcf7c101d60 2024-04-22T23:26:36.018 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:35.039+0000 7fcf7bfff700 1 --2- 172.21.15.107:0/2801442072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf7c101d60 0x7fcf7c102170 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c00bb80 tx=0x7fcf6c00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f6d950968f2bc33 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:36.018 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:35.039+0000 7fcf7affd700 1 -- 172.21.15.107:0/2801442072 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcf6c00dd00 con 0x7fcf7c101d60 2024-04-22T23:26:36.018 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:35.040+0000 7fcf7affd700 1 -- 172.21.15.107:0/2801442072 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcf6c00de60 con 0x7fcf7c101d60 2024-04-22T23:26:36.018 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:35.040+0000 7fcf7affd700 1 -- 172.21.15.107:0/2801442072 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcf6c018920 con 0x7fcf7c101d60 2024-04-22T23:26:36.018 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:35.040+0000 7fcf824c9700 1 -- 172.21.15.107:0/2801442072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf7c101d60 msgr2=0x7fcf7c102170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:36.018 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:35.040+0000 7fcf824c9700 1 --2- 172.21.15.107:0/2801442072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf7c101d60 0x7fcf7c102170 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c00bb80 tx=0x7fcf6c00b4f0 comp rx=0 tx=0).stop 2024-04-22T23:26:36.019 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:35.040+0000 7fcf824c9700 1 -- 172.21.15.107:0/2801442072 shutdown_connections 2024-04-22T23:26:36.019 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:35.040+0000 7fcf824c9700 1 --2- 172.21.15.107:0/2801442072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf7c101d60 0x7fcf7c102170 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:36.019 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:35.040+0000 7fcf824c9700 1 -- 172.21.15.107:0/2801442072 >> 172.21.15.107:0/2801442072 conn(0x7fcf7c0fc770 msgr2=0x7fcf7c0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:26:36.019 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:35.040+0000 7fcf824c9700 1 -- 172.21.15.107:0/2801442072 shutdown_connections 2024-04-22T23:26:36.019 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:35.040+0000 7fcf824c9700 1 -- 172.21.15.107:0/2801442072 wait complete. 2024-04-22T23:26:36.019 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:35.041+0000 7fcf824c9700 1 Processor -- start 2024-04-22T23:26:36.019 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:35.041+0000 7fcf824c9700 1 -- start start 2024-04-22T23:26:36.019 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:35.041+0000 7fcf824c9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf7c101d60 0x7fcf7c10f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:36.019 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:35.041+0000 7fcf824c9700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c10fa60 con 0x7fcf7c101d60 2024-04-22T23:26:36.019 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:35.041+0000 7fcf7bfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf7c101d60 0x7fcf7c10f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:36.019 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:35.041+0000 7fcf7bfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf7c101d60 0x7fcf7c10f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57740/0 (socket says 172.21.15.107:57740) 2024-04-22T23:26:36.019 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:35.041+0000 7fcf7bfff700 1 -- 172.21.15.107:0/3015343606 learned_addr learned my addr 172.21.15.107:0/3015343606 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:26:36.019 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:35.041+0000 7fcf7bfff700 1 -- 172.21.15.107:0/3015343606 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf6c00b790 con 0x7fcf7c101d60 2024-04-22T23:26:36.019 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:35.041+0000 7fcf7bfff700 1 --2- 172.21.15.107:0/3015343606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf7c101d60 0x7fcf7c10f520 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c0106a0 tx=0x7fcf6c00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:36.020 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:35.041+0000 7fcf78ff9700 1 -- 172.21.15.107:0/3015343606 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcf6c010c30 con 0x7fcf7c101d60 2024-04-22T23:26:36.020 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:35.042+0000 7fcf824c9700 1 -- 172.21.15.107:0/3015343606 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf7c10fc60 con 0x7fcf7c101d60 2024-04-22T23:26:36.020 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:35.042+0000 7fcf824c9700 1 -- 172.21.15.107:0/3015343606 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf7c1107d0 con 0x7fcf7c101d60 2024-04-22T23:26:36.020 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:35.042+0000 7fcf78ff9700 1 -- 172.21.15.107:0/3015343606 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcf6c01d050 con 0x7fcf7c101d60 2024-04-22T23:26:36.020 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:35.042+0000 7fcf78ff9700 1 -- 172.21.15.107:0/3015343606 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcf6c02d460 con 0x7fcf7c101d60 2024-04-22T23:26:36.020 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:35.042+0000 7fcf78ff9700 1 -- 172.21.15.107:0/3015343606 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 12) v1 ==== 43389+0+0 (secure 0 0 0) 0x7fcf6c02d5c0 con 0x7fcf7c101d60 2024-04-22T23:26:36.020 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:35.042+0000 7fcf78ff9700 1 --2- 172.21.15.107:0/3015343606 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7fcf640367e0 0x7fcf64038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:36.020 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:35.042+0000 7fcf824c9700 1 -- 172.21.15.107:0/3015343606 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf680052e0 con 0x7fcf7c101d60 2024-04-22T23:26:36.020 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:35.042+0000 7fcf78ff9700 1 -- 172.21.15.107:0/3015343606 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fcf6c014070 con 0x7fcf7c101d60 2024-04-22T23:26:36.020 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:35.043+0000 7fcf7b7fe700 1 --2- 172.21.15.107:0/3015343606 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7fcf640367e0 0x7fcf64038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:36.020 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:35.043+0000 7fcf7b7fe700 1 --2- 172.21.15.107:0/3015343606 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7fcf640367e0 0x7fcf64038c90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcf70009990 tx=0x7fcf70006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:36.020 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:35.047+0000 7fcf78ff9700 1 -- 172.21.15.107:0/3015343606 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fcf6c02d9b0 con 0x7fcf7c101d60 2024-04-22T23:26:36.020 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:35.236+0000 7fcf824c9700 1 -- 172.21.15.107:0/3015343606 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fcf68005b10 con 0x7fcf7c101d60 2024-04-22T23:26:36.020 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:35.236+0000 7fcf78ff9700 1 -- 172.21.15.107:0/3015343606 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7fcf6c021ae0 con 0x7fcf7c101d60 2024-04-22T23:26:36.021 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:35.238+0000 7fcf824c9700 1 -- 172.21.15.107:0/3015343606 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7fcf640367e0 msgr2=0x7fcf64038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:36.021 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:35.238+0000 7fcf824c9700 1 --2- 172.21.15.107:0/3015343606 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7fcf640367e0 0x7fcf64038c90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcf70009990 tx=0x7fcf70006e40 comp rx=0 tx=0).stop 2024-04-22T23:26:36.021 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:35.238+0000 7fcf824c9700 1 -- 172.21.15.107:0/3015343606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf7c101d60 msgr2=0x7fcf7c10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:36.021 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:35.238+0000 7fcf824c9700 1 --2- 172.21.15.107:0/3015343606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf7c101d60 0x7fcf7c10f520 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c0106a0 tx=0x7fcf6c00b4f0 comp rx=0 tx=0).stop 2024-04-22T23:26:36.021 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:35.238+0000 7fcf824c9700 1 -- 172.21.15.107:0/3015343606 shutdown_connections 2024-04-22T23:26:36.021 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:35.238+0000 7fcf824c9700 1 --2- 172.21.15.107:0/3015343606 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7fcf640367e0 0x7fcf64038c90 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:36.021 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:35.238+0000 7fcf824c9700 1 --2- 172.21.15.107:0/3015343606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf7c101d60 0x7fcf7c10f520 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:36.021 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:35.238+0000 7fcf824c9700 1 -- 172.21.15.107:0/3015343606 >> 172.21.15.107:0/3015343606 conn(0x7fcf7c0fc770 msgr2=0x7fcf7c105d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:26:36.021 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:35.239+0000 7fcf824c9700 1 -- 172.21.15.107:0/3015343606 shutdown_connections 2024-04-22T23:26:36.021 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:35.239+0000 7fcf824c9700 1 -- 172.21.15.107:0/3015343606 wait complete. 2024-04-22T23:26:36.025 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-22T23:26:36.025 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout disabled 2024-04-22T23:26:36.033 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-22T23:26:36.033 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-04-22T23:26:36.033 INFO:teuthology.orchestra.run.smithi107.stdout:firewalld.service is not enabled 2024-04-22T23:26:36.033 INFO:teuthology.orchestra.run.smithi107.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-22T23:26:36.034 INFO:teuthology.orchestra.run.smithi107.stdout:Ceph Dashboard is now available at: 2024-04-22T23:26:36.034 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:26:36.035 INFO:teuthology.orchestra.run.smithi107.stdout: URL: https://smithi107:8443/ 2024-04-22T23:26:36.035 INFO:teuthology.orchestra.run.smithi107.stdout: User: admin 2024-04-22T23:26:36.035 INFO:teuthology.orchestra.run.smithi107.stdout: Password: lbqmr6qlig 2024-04-22T23:26:36.035 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:26:36.035 INFO:teuthology.orchestra.run.smithi107.stdout:Saving cluster configuration to /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/config directory 2024-04-22T23:26:36.035 INFO:teuthology.orchestra.run.smithi107.stdout:Enabling autotune for osd_memory_target 2024-04-22T23:26:37.227 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:36.454+0000 7ffb2896b700 1 Processor -- start 2024-04-22T23:26:37.227 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:36.454+0000 7ffb2896b700 1 -- start start 2024-04-22T23:26:37.227 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:36.454+0000 7ffb2896b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb24102930 0x7ffb24102d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:37.227 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:36.454+0000 7ffb2896b700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb24103280 con 0x7ffb24102930 2024-04-22T23:26:37.227 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:36.454+0000 7ffb2259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb24102930 0x7ffb24102d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:37.227 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:36.454+0000 7ffb2259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb24102930 0x7ffb24102d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57756/0 (socket says 172.21.15.107:57756) 2024-04-22T23:26:37.227 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:36.454+0000 7ffb2259c700 1 -- 172.21.15.107:0/58942389 learned_addr learned my addr 172.21.15.107:0/58942389 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:26:37.227 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:36.454+0000 7ffb2259c700 1 -- 172.21.15.107:0/58942389 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb241033c0 con 0x7ffb24102930 2024-04-22T23:26:37.227 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:36.455+0000 7ffb2259c700 1 --2- 172.21.15.107:0/58942389 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb24102930 0x7ffb24102d40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ffb140098e0 tx=0x7ffb14009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db9afdb5caa4ea5c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:37.227 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:36.455+0000 7ffb2159a700 1 -- 172.21.15.107:0/58942389 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffb1400b8f0 con 0x7ffb24102930 2024-04-22T23:26:37.227 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:36.455+0000 7ffb2159a700 1 -- 172.21.15.107:0/58942389 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffb14011420 con 0x7ffb24102930 2024-04-22T23:26:37.227 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:36.455+0000 7ffb2159a700 1 -- 172.21.15.107:0/58942389 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffb14010540 con 0x7ffb24102930 2024-04-22T23:26:37.228 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:36.455+0000 7ffb2896b700 1 -- 172.21.15.107:0/58942389 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb24102930 msgr2=0x7ffb24102d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:37.228 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:36.455+0000 7ffb2896b700 1 --2- 172.21.15.107:0/58942389 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb24102930 0x7ffb24102d40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ffb140098e0 tx=0x7ffb14009250 comp rx=0 tx=0).stop 2024-04-22T23:26:37.228 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:36.455+0000 7ffb2896b700 1 -- 172.21.15.107:0/58942389 shutdown_connections 2024-04-22T23:26:37.228 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:36.455+0000 7ffb2896b700 1 --2- 172.21.15.107:0/58942389 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb24102930 0x7ffb24102d40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:37.228 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:36.455+0000 7ffb2896b700 1 -- 172.21.15.107:0/58942389 >> 172.21.15.107:0/58942389 conn(0x7ffb240fe140 msgr2=0x7ffb24100550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:26:37.228 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:36.456+0000 7ffb2896b700 1 -- 172.21.15.107:0/58942389 shutdown_connections 2024-04-22T23:26:37.228 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:36.456+0000 7ffb2896b700 1 -- 172.21.15.107:0/58942389 wait complete. 2024-04-22T23:26:37.228 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:36.456+0000 7ffb2896b700 1 Processor -- start 2024-04-22T23:26:37.228 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:36.456+0000 7ffb2896b700 1 -- start start 2024-04-22T23:26:37.228 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:36.456+0000 7ffb2896b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb24102930 0x7ffb24111720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:37.228 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:36.456+0000 7ffb2896b700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb24111c60 con 0x7ffb24102930 2024-04-22T23:26:37.228 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:36.456+0000 7ffb2259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb24102930 0x7ffb24111720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:37.228 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:36.456+0000 7ffb2259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb24102930 0x7ffb24111720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57768/0 (socket says 172.21.15.107:57768) 2024-04-22T23:26:37.228 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:36.456+0000 7ffb2259c700 1 -- 172.21.15.107:0/915538977 learned_addr learned my addr 172.21.15.107:0/915538977 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:26:37.228 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:36.457+0000 7ffb2259c700 1 -- 172.21.15.107:0/915538977 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb140094f0 con 0x7ffb24102930 2024-04-22T23:26:37.229 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:36.457+0000 7ffb2259c700 1 --2- 172.21.15.107:0/915538977 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb24102930 0x7ffb24111720 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ffb14006b00 tx=0x7ffb14009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:37.229 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:36.457+0000 7ffb12ffd700 1 -- 172.21.15.107:0/915538977 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffb14010800 con 0x7ffb24102930 2024-04-22T23:26:37.229 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:36.457+0000 7ffb12ffd700 1 -- 172.21.15.107:0/915538977 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffb14010e30 con 0x7ffb24102930 2024-04-22T23:26:37.229 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:36.457+0000 7ffb2896b700 1 -- 172.21.15.107:0/915538977 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb24111e60 con 0x7ffb24102930 2024-04-22T23:26:37.229 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:36.457+0000 7ffb12ffd700 1 -- 172.21.15.107:0/915538977 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffb1401ae40 con 0x7ffb24102930 2024-04-22T23:26:37.229 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:36.457+0000 7ffb2896b700 1 -- 172.21.15.107:0/915538977 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb241129d0 con 0x7ffb24102930 2024-04-22T23:26:37.229 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:36.458+0000 7ffb12ffd700 1 -- 172.21.15.107:0/915538977 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 12) v1 ==== 43389+0+0 (secure 0 0 0) 0x7ffb1402e440 con 0x7ffb24102930 2024-04-22T23:26:37.229 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:36.458+0000 7ffb12ffd700 1 --2- 172.21.15.107:0/915538977 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7ffb0c0367e0 0x7ffb0c038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:37.229 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:36.458+0000 7ffb12ffd700 1 -- 172.21.15.107:0/915538977 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7ffb1404a7f0 con 0x7ffb24102930 2024-04-22T23:26:37.229 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:36.458+0000 7ffb2896b700 1 -- 172.21.15.107:0/915538977 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb2405f5f0 con 0x7ffb24102930 2024-04-22T23:26:37.229 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:36.458+0000 7ffb21d9b700 1 --2- 172.21.15.107:0/915538977 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7ffb0c0367e0 0x7ffb0c038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:37.229 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:36.458+0000 7ffb21d9b700 1 --2- 172.21.15.107:0/915538977 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7ffb0c0367e0 0x7ffb0c038c90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ffb18009990 tx=0x7ffb18006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:37.229 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:36.463+0000 7ffb12ffd700 1 -- 172.21.15.107:0/915538977 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ffb14018920 con 0x7ffb24102930 2024-04-22T23:26:37.229 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:36.648+0000 7ffb2896b700 1 -- 172.21.15.107:0/915538977 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7ffb2404bed0 con 0x7ffb24102930 2024-04-22T23:26:37.230 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:36.648+0000 7ffb12ffd700 1 -- 172.21.15.107:0/915538977 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7ffb1401a6f0 con 0x7ffb24102930 2024-04-22T23:26:37.230 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:36.650+0000 7ffb2896b700 1 -- 172.21.15.107:0/915538977 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7ffb0c0367e0 msgr2=0x7ffb0c038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:37.230 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:36.650+0000 7ffb2896b700 1 --2- 172.21.15.107:0/915538977 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7ffb0c0367e0 0x7ffb0c038c90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ffb18009990 tx=0x7ffb18006e40 comp rx=0 tx=0).stop 2024-04-22T23:26:37.230 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:36.650+0000 7ffb2896b700 1 -- 172.21.15.107:0/915538977 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb24102930 msgr2=0x7ffb24111720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:37.230 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:36.650+0000 7ffb2896b700 1 --2- 172.21.15.107:0/915538977 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb24102930 0x7ffb24111720 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ffb14006b00 tx=0x7ffb14009250 comp rx=0 tx=0).stop 2024-04-22T23:26:37.230 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:36.650+0000 7ffb2896b700 1 -- 172.21.15.107:0/915538977 shutdown_connections 2024-04-22T23:26:37.231 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:36.650+0000 7ffb2896b700 1 --2- 172.21.15.107:0/915538977 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7ffb0c0367e0 0x7ffb0c038c90 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:37.231 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:36.650+0000 7ffb2896b700 1 --2- 172.21.15.107:0/915538977 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb24102930 0x7ffb24111720 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:37.231 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:36.650+0000 7ffb2896b700 1 -- 172.21.15.107:0/915538977 >> 172.21.15.107:0/915538977 conn(0x7ffb240fe140 msgr2=0x7ffb240fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:26:37.231 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:36.650+0000 7ffb2896b700 1 -- 172.21.15.107:0/915538977 shutdown_connections 2024-04-22T23:26:37.231 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:36.650+0000 7ffb2896b700 1 -- 172.21.15.107:0/915538977 wait complete. 2024-04-22T23:26:37.932 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:37 smithi107 ceph-mon[26355]: mgrmap e13: smithi107.uipxvm(active, since 6s) 2024-04-22T23:26:38.659 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:37.629+0000 7ff4af3db700 1 Processor -- start 2024-04-22T23:26:38.659 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:37.629+0000 7ff4af3db700 1 -- start start 2024-04-22T23:26:38.659 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:37.629+0000 7ff4af3db700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4a8102930 0x7ff4a8102d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:38.659 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:37.629+0000 7ff4af3db700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a8103280 con 0x7ff4a8102930 2024-04-22T23:26:38.659 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:37.629+0000 7ff4ad177700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4a8102930 0x7ff4a8102d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:38.659 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:37.629+0000 7ff4ad177700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4a8102930 0x7ff4a8102d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57772/0 (socket says 172.21.15.107:57772) 2024-04-22T23:26:38.660 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:37.629+0000 7ff4ad177700 1 -- 172.21.15.107:0/3667169632 learned_addr learned my addr 172.21.15.107:0/3667169632 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:26:38.660 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:37.630+0000 7ff4ad177700 1 -- 172.21.15.107:0/3667169632 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a81033c0 con 0x7ff4a8102930 2024-04-22T23:26:38.660 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:37.630+0000 7ff4ad177700 1 --2- 172.21.15.107:0/3667169632 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4a8102930 0x7ff4a8102d40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff4a40098e0 tx=0x7ff4a4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8e07846ce72eaf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:38.660 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:37.630+0000 7ff49ffff700 1 -- 172.21.15.107:0/3667169632 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff4a400b8f0 con 0x7ff4a8102930 2024-04-22T23:26:38.660 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:37.630+0000 7ff49ffff700 1 -- 172.21.15.107:0/3667169632 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff4a4011420 con 0x7ff4a8102930 2024-04-22T23:26:38.660 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:37.630+0000 7ff49ffff700 1 -- 172.21.15.107:0/3667169632 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff4a4010540 con 0x7ff4a8102930 2024-04-22T23:26:38.660 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:37.630+0000 7ff4af3db700 1 -- 172.21.15.107:0/3667169632 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4a8102930 msgr2=0x7ff4a8102d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:38.660 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:37.630+0000 7ff4af3db700 1 --2- 172.21.15.107:0/3667169632 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4a8102930 0x7ff4a8102d40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff4a40098e0 tx=0x7ff4a4009250 comp rx=0 tx=0).stop 2024-04-22T23:26:38.660 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:37.631+0000 7ff4af3db700 1 -- 172.21.15.107:0/3667169632 shutdown_connections 2024-04-22T23:26:38.660 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:37.631+0000 7ff4af3db700 1 --2- 172.21.15.107:0/3667169632 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4a8102930 0x7ff4a8102d40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:38.660 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:37.631+0000 7ff4af3db700 1 -- 172.21.15.107:0/3667169632 >> 172.21.15.107:0/3667169632 conn(0x7ff4a80fe140 msgr2=0x7ff4a8100550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:26:38.660 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:37.631+0000 7ff4af3db700 1 -- 172.21.15.107:0/3667169632 shutdown_connections 2024-04-22T23:26:38.660 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:37.631+0000 7ff4af3db700 1 -- 172.21.15.107:0/3667169632 wait complete. 2024-04-22T23:26:38.660 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:37.631+0000 7ff4af3db700 1 Processor -- start 2024-04-22T23:26:38.661 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:37.631+0000 7ff4af3db700 1 -- start start 2024-04-22T23:26:38.661 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:37.631+0000 7ff4af3db700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4a8102930 0x7ff4a8078c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:38.661 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:37.631+0000 7ff4af3db700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a8079190 con 0x7ff4a8102930 2024-04-22T23:26:38.661 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:37.631+0000 7ff4ad177700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4a8102930 0x7ff4a8078c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:38.661 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:37.631+0000 7ff4ad177700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4a8102930 0x7ff4a8078c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57776/0 (socket says 172.21.15.107:57776) 2024-04-22T23:26:38.661 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:37.631+0000 7ff4ad177700 1 -- 172.21.15.107:0/2237601045 learned_addr learned my addr 172.21.15.107:0/2237601045 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:26:38.661 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:37.632+0000 7ff4ad177700 1 -- 172.21.15.107:0/2237601045 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a40094f0 con 0x7ff4a8102930 2024-04-22T23:26:38.661 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:37.632+0000 7ff4ad177700 1 --2- 172.21.15.107:0/2237601045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4a8102930 0x7ff4a8078c50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff4a4006b00 tx=0x7ff4a4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:38.661 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:37.632+0000 7ff49dffb700 1 -- 172.21.15.107:0/2237601045 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff4a4010800 con 0x7ff4a8102930 2024-04-22T23:26:38.661 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:37.632+0000 7ff49dffb700 1 -- 172.21.15.107:0/2237601045 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff4a4010e30 con 0x7ff4a8102930 2024-04-22T23:26:38.661 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:37.632+0000 7ff4af3db700 1 -- 172.21.15.107:0/2237601045 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4a8077200 con 0x7ff4a8102930 2024-04-22T23:26:38.661 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:37.632+0000 7ff49dffb700 1 -- 172.21.15.107:0/2237601045 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff4a401ae40 con 0x7ff4a8102930 2024-04-22T23:26:38.661 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:37.632+0000 7ff4af3db700 1 -- 172.21.15.107:0/2237601045 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4a80776f0 con 0x7ff4a8102930 2024-04-22T23:26:38.661 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:37.633+0000 7ff49dffb700 1 -- 172.21.15.107:0/2237601045 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7ff4a401a6f0 con 0x7ff4a8102930 2024-04-22T23:26:38.662 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:37.633+0000 7ff4af3db700 1 -- 172.21.15.107:0/2237601045 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4a804bed0 con 0x7ff4a8102930 2024-04-22T23:26:38.662 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:37.633+0000 7ff49dffb700 1 --2- 172.21.15.107:0/2237601045 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7ff494036810 0x7ff494038cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:38.662 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:37.633+0000 7ff49dffb700 1 -- 172.21.15.107:0/2237601045 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7ff4a404a840 con 0x7ff4a8102930 2024-04-22T23:26:38.662 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:37.633+0000 7ff4ac976700 1 --2- 172.21.15.107:0/2237601045 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7ff494036810 0x7ff494038cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:38.662 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:37.634+0000 7ff4ac976700 1 --2- 172.21.15.107:0/2237601045 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7ff494036810 0x7ff494038cc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff498009990 tx=0x7ff498006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:38.662 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:37.638+0000 7ff49dffb700 1 -- 172.21.15.107:0/2237601045 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ff4a4018530 con 0x7ff4a8102930 2024-04-22T23:26:38.662 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:37.931+0000 7ff4af3db700 1 -- 172.21.15.107:0/2237601045 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7ff4a8077b60 con 0x7ff4a8102930 2024-04-22T23:26:38.662 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:37.936+0000 7ff49dffb700 1 -- 172.21.15.107:0/2237601045 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v33)=0 set mgr/dashboard/cluster/status v33) v1 ==== 153+0+0 (secure 0 0 0) 0x7ff4a4018350 con 0x7ff4a8102930 2024-04-22T23:26:38.662 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:37.938+0000 7ff4af3db700 1 -- 172.21.15.107:0/2237601045 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7ff494036810 msgr2=0x7ff494038cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:38.662 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:37.938+0000 7ff4af3db700 1 --2- 172.21.15.107:0/2237601045 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7ff494036810 0x7ff494038cc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff498009990 tx=0x7ff498006e40 comp rx=0 tx=0).stop 2024-04-22T23:26:38.662 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:37.938+0000 7ff4af3db700 1 -- 172.21.15.107:0/2237601045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4a8102930 msgr2=0x7ff4a8078c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:38.662 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:37.938+0000 7ff4af3db700 1 --2- 172.21.15.107:0/2237601045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4a8102930 0x7ff4a8078c50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff4a4006b00 tx=0x7ff4a4009250 comp rx=0 tx=0).stop 2024-04-22T23:26:38.662 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:37.938+0000 7ff4af3db700 1 -- 172.21.15.107:0/2237601045 shutdown_connections 2024-04-22T23:26:38.662 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:37.938+0000 7ff4af3db700 1 --2- 172.21.15.107:0/2237601045 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7ff494036810 0x7ff494038cc0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:38.663 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:37.938+0000 7ff4af3db700 1 --2- 172.21.15.107:0/2237601045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4a8102930 0x7ff4a8078c50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:38.663 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:37.938+0000 7ff4af3db700 1 -- 172.21.15.107:0/2237601045 >> 172.21.15.107:0/2237601045 conn(0x7ff4a80fe140 msgr2=0x7ff4a80fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:26:38.663 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:37.938+0000 7ff4af3db700 1 -- 172.21.15.107:0/2237601045 shutdown_connections 2024-04-22T23:26:38.663 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-22T23:26:37.938+0000 7ff4af3db700 1 -- 172.21.15.107:0/2237601045 wait complete. 2024-04-22T23:26:38.663 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-22T23:26:38.663 INFO:teuthology.orchestra.run.smithi107.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-22T23:26:38.663 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:26:38.663 INFO:teuthology.orchestra.run.smithi107.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-22T23:26:38.663 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:26:38.663 INFO:teuthology.orchestra.run.smithi107.stdout:Or, if you are only running a single cluster on this host: 2024-04-22T23:26:38.663 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:26:38.663 INFO:teuthology.orchestra.run.smithi107.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-22T23:26:38.663 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:26:38.663 INFO:teuthology.orchestra.run.smithi107.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-22T23:26:38.663 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:26:38.663 INFO:teuthology.orchestra.run.smithi107.stdout: ceph telemetry on 2024-04-22T23:26:38.664 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:26:38.664 INFO:teuthology.orchestra.run.smithi107.stdout:For more information see: 2024-04-22T23:26:38.664 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:26:38.664 INFO:teuthology.orchestra.run.smithi107.stdout: https://docs.ceph.com/docs/master/mgr/telemetry/ 2024-04-22T23:26:38.664 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:26:38.664 INFO:teuthology.orchestra.run.smithi107.stdout:Bootstrap complete. 2024-04-22T23:26:38.695 INFO:tasks.cephadm:Fetching config... 2024-04-22T23:26:38.695 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-22T23:26:38.695 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-22T23:26:38.711 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-22T23:26:38.711 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-22T23:26:38.711 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-22T23:26:38.769 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-22T23:26:38.769 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-22T23:26:38.769 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/keyring of=/dev/stdout 2024-04-22T23:26:38.834 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-22T23:26:38.834 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-22T23:26:38.834 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-22T23:26:38.889 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-22T23:26:38.889 DEBUG:teuthology.orchestra.run.smithi107:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1WWJuwZTZzGSBHUvFJLoPGJ5EFcXCcshdrIUBKc5qGJzAS2XSOuel0GZnv4ZfjQ8v9flO/XgV+Dv6WLYUbw2CqiEq4kp3dWiauXSh/87wXychQeDCY39g2OQfSokyX2BfKW3cnp62pTsEP5cDP5+elbXFz5/932saD8SAx8R/dsa88XLuVSar4Rt9DIkWHoqwqqYu5hFTtBKbDb1D2gTrTSs6rYXr5+J3qjM1hpjZzF+xw3eg6uimfVFFjmNpvU7tGum3SXtmFrFSfohJcN8p9bXgrBNsBYMNQUkT6dlDF1Sl3Z2Trg64ncGCC0lusUOyEi4Uuh+7W7amYVp7AZSUs+/80EsNGBARvVSVtzpmsA+GXn24sq/EzTumJ91K0lesNX+7rPsLP98KRsVsLACkwWVPD9SctKbx0BUjJfu4vI7QzhS9rLgFgQ6l2Xl/xEL2/PNzN8vtbKSbbDR5shRMhZu9OBW9DXuw8BsdhOg1BNmXZbJwjTi+2BnltAsw9Rk= ceph-8252f6f2-00ff-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-22T23:26:38.964 INFO:teuthology.orchestra.run.smithi107.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1WWJuwZTZzGSBHUvFJLoPGJ5EFcXCcshdrIUBKc5qGJzAS2XSOuel0GZnv4ZfjQ8v9flO/XgV+Dv6WLYUbw2CqiEq4kp3dWiauXSh/87wXychQeDCY39g2OQfSokyX2BfKW3cnp62pTsEP5cDP5+elbXFz5/932saD8SAx8R/dsa88XLuVSar4Rt9DIkWHoqwqqYu5hFTtBKbDb1D2gTrTSs6rYXr5+J3qjM1hpjZzF+xw3eg6uimfVFFjmNpvU7tGum3SXtmFrFSfohJcN8p9bXgrBNsBYMNQUkT6dlDF1Sl3Z2Trg64ncGCC0lusUOyEi4Uuh+7W7amYVp7AZSUs+/80EsNGBARvVSVtzpmsA+GXn24sq/EzTumJ91K0lesNX+7rPsLP98KRsVsLACkwWVPD9SctKbx0BUjJfu4vI7QzhS9rLgFgQ6l2Xl/xEL2/PNzN8vtbKSbbDR5shRMhZu9OBW9DXuw8BsdhOg1BNmXZbJwjTi+2BnltAsw9Rk= ceph-8252f6f2-00ff-11ef-bc93-c7b262605968 2024-04-22T23:26:38.975 DEBUG:teuthology.orchestra.run.smithi157:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1WWJuwZTZzGSBHUvFJLoPGJ5EFcXCcshdrIUBKc5qGJzAS2XSOuel0GZnv4ZfjQ8v9flO/XgV+Dv6WLYUbw2CqiEq4kp3dWiauXSh/87wXychQeDCY39g2OQfSokyX2BfKW3cnp62pTsEP5cDP5+elbXFz5/932saD8SAx8R/dsa88XLuVSar4Rt9DIkWHoqwqqYu5hFTtBKbDb1D2gTrTSs6rYXr5+J3qjM1hpjZzF+xw3eg6uimfVFFjmNpvU7tGum3SXtmFrFSfohJcN8p9bXgrBNsBYMNQUkT6dlDF1Sl3Z2Trg64ncGCC0lusUOyEi4Uuh+7W7amYVp7AZSUs+/80EsNGBARvVSVtzpmsA+GXn24sq/EzTumJ91K0lesNX+7rPsLP98KRsVsLACkwWVPD9SctKbx0BUjJfu4vI7QzhS9rLgFgQ6l2Xl/xEL2/PNzN8vtbKSbbDR5shRMhZu9OBW9DXuw8BsdhOg1BNmXZbJwjTi+2BnltAsw9Rk= ceph-8252f6f2-00ff-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-22T23:26:39.015 INFO:teuthology.orchestra.run.smithi157.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1WWJuwZTZzGSBHUvFJLoPGJ5EFcXCcshdrIUBKc5qGJzAS2XSOuel0GZnv4ZfjQ8v9flO/XgV+Dv6WLYUbw2CqiEq4kp3dWiauXSh/87wXychQeDCY39g2OQfSokyX2BfKW3cnp62pTsEP5cDP5+elbXFz5/932saD8SAx8R/dsa88XLuVSar4Rt9DIkWHoqwqqYu5hFTtBKbDb1D2gTrTSs6rYXr5+J3qjM1hpjZzF+xw3eg6uimfVFFjmNpvU7tGum3SXtmFrFSfohJcN8p9bXgrBNsBYMNQUkT6dlDF1Sl3Z2Trg64ncGCC0lusUOyEi4Uuh+7W7amYVp7AZSUs+/80EsNGBARvVSVtzpmsA+GXn24sq/EzTumJ91K0lesNX+7rPsLP98KRsVsLACkwWVPD9SctKbx0BUjJfu4vI7QzhS9rLgFgQ6l2Xl/xEL2/PNzN8vtbKSbbDR5shRMhZu9OBW9DXuw8BsdhOg1BNmXZbJwjTi+2BnltAsw9Rk= ceph-8252f6f2-00ff-11ef-bc93-c7b262605968 2024-04-22T23:26:39.027 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-22T23:26:39.240 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:38 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/2237601045' entity='client.admin' 2024-04-22T23:26:39.299 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:26:41.740 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:41 smithi107 ceph-mon[26355]: from='mgr.14160 172.21.15.107:0/1348717222' entity='mgr.smithi107.uipxvm' 2024-04-22T23:26:41.740 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:41 smithi107 ceph-mon[26355]: from='mgr.14160 172.21.15.107:0/1348717222' entity='mgr.smithi107.uipxvm' 2024-04-22T23:26:41.740 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:41 smithi107 ceph-mon[26355]: from='mgr.14160 172.21.15.107:0/1348717222' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "who": "osd/host:smithi107", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:26:41.740 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:41 smithi107 ceph-mon[26355]: from='mgr.14160 172.21.15.107:0/1348717222' entity='mgr.smithi107.uipxvm' 2024-04-22T23:26:41.740 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:41 smithi107 ceph-mon[26355]: from='mgr.14160 172.21.15.107:0/1348717222' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi107", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-22T23:26:41.740 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:41 smithi107 ceph-mon[26355]: from='mgr.14160 172.21.15.107:0/1348717222' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi107", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-04-22T23:26:41.740 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:41 smithi107 ceph-mon[26355]: from='mgr.14160 172.21.15.107:0/1348717222' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:26:41.740 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:41 smithi107 ceph-mon[26355]: Deploying daemon crash.smithi107 on smithi107 2024-04-22T23:26:42.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:42.013+0000 7f4c3eebd700 1 -- 172.21.15.107:0/3424005448 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c380fb100 msgr2=0x7f4c380fb510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:42.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:42.013+0000 7f4c3eebd700 1 --2- 172.21.15.107:0/3424005448 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c380fb100 0x7f4c380fb510 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c0049d0 tx=0x7f4c2c00af40 comp rx=0 tx=0).stop 2024-04-22T23:26:42.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:42.013+0000 7f4c3eebd700 1 -- 172.21.15.107:0/3424005448 shutdown_connections 2024-04-22T23:26:42.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:42.013+0000 7f4c3eebd700 1 --2- 172.21.15.107:0/3424005448 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c380fb100 0x7f4c380fb510 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:42.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:42.013+0000 7f4c3eebd700 1 -- 172.21.15.107:0/3424005448 >> 172.21.15.107:0/3424005448 conn(0x7f4c380f6860 msgr2=0x7f4c380f8c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:26:42.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:42.013+0000 7f4c3eebd700 1 -- 172.21.15.107:0/3424005448 shutdown_connections 2024-04-22T23:26:42.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:42.013+0000 7f4c3eebd700 1 -- 172.21.15.107:0/3424005448 wait complete. 2024-04-22T23:26:42.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:42.013+0000 7f4c3eebd700 1 Processor -- start 2024-04-22T23:26:42.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:42.013+0000 7f4c3eebd700 1 -- start start 2024-04-22T23:26:42.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:42.014+0000 7f4c3eebd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c380fb100 0x7f4c3810d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:42.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:42.014+0000 7f4c3eebd700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3810d6b0 con 0x7f4c380fb100 2024-04-22T23:26:42.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:42.014+0000 7f4c3cc59700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c380fb100 0x7f4c3810d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:42.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:42.014+0000 7f4c3cc59700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c380fb100 0x7f4c3810d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57292/0 (socket says 172.21.15.107:57292) 2024-04-22T23:26:42.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:42.014+0000 7f4c3cc59700 1 -- 172.21.15.107:0/3005423248 learned_addr learned my addr 172.21.15.107:0/3005423248 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:26:42.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:42.014+0000 7f4c3cc59700 1 -- 172.21.15.107:0/3005423248 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c2c005040 con 0x7f4c380fb100 2024-04-22T23:26:42.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:42.014+0000 7f4c3cc59700 1 --2- 172.21.15.107:0/3005423248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c380fb100 0x7f4c3810d170 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c004710 tx=0x7f4c2c00af40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:42.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:42.014+0000 7f4c357fa700 1 -- 172.21.15.107:0/3005423248 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4c2c00c8c0 con 0x7f4c380fb100 2024-04-22T23:26:42.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:42.014+0000 7f4c357fa700 1 -- 172.21.15.107:0/3005423248 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4c2c00a8b0 con 0x7f4c380fb100 2024-04-22T23:26:42.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:42.014+0000 7f4c3eebd700 1 -- 172.21.15.107:0/3005423248 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c3810d8b0 con 0x7f4c380fb100 2024-04-22T23:26:42.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:42.014+0000 7f4c357fa700 1 -- 172.21.15.107:0/3005423248 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4c2c014750 con 0x7f4c380fb100 2024-04-22T23:26:42.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:42.014+0000 7f4c3eebd700 1 -- 172.21.15.107:0/3005423248 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c3810e420 con 0x7f4c380fb100 2024-04-22T23:26:42.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:42.015+0000 7f4c357fa700 1 -- 172.21.15.107:0/3005423248 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7f4c2c014980 con 0x7f4c380fb100 2024-04-22T23:26:42.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:42.015+0000 7f4c3eebd700 1 -- 172.21.15.107:0/3005423248 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c38100d90 con 0x7f4c380fb100 2024-04-22T23:26:42.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:42.015+0000 7f4c357fa700 1 --2- 172.21.15.107:0/3005423248 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f4c280368d0 0x7f4c28038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:42.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:42.015+0000 7f4c357fa700 1 -- 172.21.15.107:0/3005423248 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f4c2c04a100 con 0x7f4c380fb100 2024-04-22T23:26:42.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:42.016+0000 7f4c37fff700 1 --2- 172.21.15.107:0/3005423248 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f4c280368d0 0x7f4c28038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:42.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:42.016+0000 7f4c37fff700 1 --2- 172.21.15.107:0/3005423248 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f4c280368d0 0x7f4c28038d80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4c24009990 tx=0x7f4c24006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:42.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:42.020+0000 7f4c357fa700 1 -- 172.21.15.107:0/3005423248 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4c2c012780 con 0x7f4c380fb100 2024-04-22T23:26:42.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:42.213+0000 7f4c3eebd700 1 -- 172.21.15.107:0/3005423248 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f4c3805f5f0 con 0x7f4c380fb100 2024-04-22T23:26:42.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:42.215+0000 7f4c357fa700 1 -- 172.21.15.107:0/3005423248 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f4c2c025360 con 0x7f4c380fb100 2024-04-22T23:26:42.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:42.218+0000 7f4c3eebd700 1 -- 172.21.15.107:0/3005423248 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f4c280368d0 msgr2=0x7f4c28038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:42.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:42.218+0000 7f4c3eebd700 1 --2- 172.21.15.107:0/3005423248 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f4c280368d0 0x7f4c28038d80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4c24009990 tx=0x7f4c24006e40 comp rx=0 tx=0).stop 2024-04-22T23:26:42.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:42.218+0000 7f4c3eebd700 1 -- 172.21.15.107:0/3005423248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c380fb100 msgr2=0x7f4c3810d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:42.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:42.218+0000 7f4c3eebd700 1 --2- 172.21.15.107:0/3005423248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c380fb100 0x7f4c3810d170 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c004710 tx=0x7f4c2c00af40 comp rx=0 tx=0).stop 2024-04-22T23:26:42.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:42.218+0000 7f4c3eebd700 1 -- 172.21.15.107:0/3005423248 shutdown_connections 2024-04-22T23:26:42.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:42.218+0000 7f4c3eebd700 1 --2- 172.21.15.107:0/3005423248 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f4c280368d0 0x7f4c28038d80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:42.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:42.218+0000 7f4c3eebd700 1 --2- 172.21.15.107:0/3005423248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c380fb100 0x7f4c3810d170 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:42.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:42.218+0000 7f4c3eebd700 1 -- 172.21.15.107:0/3005423248 >> 172.21.15.107:0/3005423248 conn(0x7f4c380f6860 msgr2=0x7f4c380f7100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:26:42.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:42.218+0000 7f4c3eebd700 1 -- 172.21.15.107:0/3005423248 shutdown_connections 2024-04-22T23:26:42.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:42.218+0000 7f4c3eebd700 1 -- 172.21.15.107:0/3005423248 wait complete. 2024-04-22T23:26:42.849 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-22T23:26:42.849 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-22T23:26:43.341 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:26:43.467 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:43 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/3005423248' entity='client.admin' 2024-04-22T23:26:45.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:45.968+0000 7f3939824700 1 -- 172.21.15.107:0/815544629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39340fd040 msgr2=0x7f39340fd450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:45.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:45.968+0000 7f3939824700 1 --2- 172.21.15.107:0/815544629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39340fd040 0x7f39340fd450 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f39240048c0 tx=0x7f3924009e30 comp rx=0 tx=0).stop 2024-04-22T23:26:45.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:45.968+0000 7f3939824700 1 -- 172.21.15.107:0/815544629 shutdown_connections 2024-04-22T23:26:45.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:45.968+0000 7f3939824700 1 --2- 172.21.15.107:0/815544629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39340fd040 0x7f39340fd450 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:45.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:45.968+0000 7f3939824700 1 -- 172.21.15.107:0/815544629 >> 172.21.15.107:0/815544629 conn(0x7f39340f70c0 msgr2=0x7f39340f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:26:45.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:45.969+0000 7f3939824700 1 -- 172.21.15.107:0/815544629 shutdown_connections 2024-04-22T23:26:45.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:45.969+0000 7f3939824700 1 -- 172.21.15.107:0/815544629 wait complete. 2024-04-22T23:26:45.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:45.969+0000 7f3939824700 1 Processor -- start 2024-04-22T23:26:45.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:45.969+0000 7f3939824700 1 -- start start 2024-04-22T23:26:45.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:45.969+0000 7f3939824700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39340fd040 0x7f393410d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:45.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:45.969+0000 7f3939824700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f393410d5d0 con 0x7f39340fd040 2024-04-22T23:26:45.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:45.970+0000 7f3932ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39340fd040 0x7f393410d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:45.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:45.970+0000 7f3932ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39340fd040 0x7f393410d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57316/0 (socket says 172.21.15.107:57316) 2024-04-22T23:26:45.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:45.970+0000 7f3932ffd700 1 -- 172.21.15.107:0/772076085 learned_addr learned my addr 172.21.15.107:0/772076085 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:26:45.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:45.970+0000 7f3932ffd700 1 -- 172.21.15.107:0/772076085 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3924005040 con 0x7f39340fd040 2024-04-22T23:26:45.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:45.970+0000 7f3932ffd700 1 --2- 172.21.15.107:0/772076085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39340fd040 0x7f393410d090 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f39240096b0 tx=0x7f3924009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:45.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:45.970+0000 7f391b7fe700 1 -- 172.21.15.107:0/772076085 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f392400bed0 con 0x7f39340fd040 2024-04-22T23:26:45.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:45.970+0000 7f391b7fe700 1 -- 172.21.15.107:0/772076085 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f392400b5c0 con 0x7f39340fd040 2024-04-22T23:26:45.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:45.970+0000 7f3939824700 1 -- 172.21.15.107:0/772076085 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f393410d7d0 con 0x7f39340fd040 2024-04-22T23:26:45.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:45.970+0000 7f391b7fe700 1 -- 172.21.15.107:0/772076085 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f392401c5b0 con 0x7f39340fd040 2024-04-22T23:26:45.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:45.970+0000 7f3939824700 1 -- 172.21.15.107:0/772076085 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f393410e340 con 0x7f39340fd040 2024-04-22T23:26:45.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:45.971+0000 7f391b7fe700 1 -- 172.21.15.107:0/772076085 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7f392401c710 con 0x7f39340fd040 2024-04-22T23:26:45.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:45.972+0000 7f391b7fe700 1 --2- 172.21.15.107:0/772076085 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f391c0368d0 0x7f391c038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:45.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:45.972+0000 7f391b7fe700 1 -- 172.21.15.107:0/772076085 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f39240499c0 con 0x7f39340fd040 2024-04-22T23:26:45.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:45.972+0000 7f3939824700 1 -- 172.21.15.107:0/772076085 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3934101e60 con 0x7f39340fd040 2024-04-22T23:26:45.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:45.972+0000 7f39327fc700 1 --2- 172.21.15.107:0/772076085 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f391c0368d0 0x7f391c038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:45.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:45.972+0000 7f39327fc700 1 --2- 172.21.15.107:0/772076085 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f391c0368d0 0x7f391c038d80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3928009990 tx=0x7f3928006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:45.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:45.977+0000 7f391b7fe700 1 -- 172.21.15.107:0/772076085 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f3924025400 con 0x7f39340fd040 2024-04-22T23:26:46.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:46.175+0000 7f3939824700 1 -- 172.21.15.107:0/772076085 --> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f393404bf40 con 0x7f391c0368d0 2024-04-22T23:26:46.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:46.178+0000 7f391b7fe700 1 -- 172.21.15.107:0/772076085 <== mgr.14160 v2:172.21.15.107:6800/3427816626 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f393404bf40 con 0x7f391c0368d0 2024-04-22T23:26:46.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:46.180+0000 7f3939824700 1 -- 172.21.15.107:0/772076085 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f391c0368d0 msgr2=0x7f391c038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:46.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:46.180+0000 7f3939824700 1 --2- 172.21.15.107:0/772076085 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f391c0368d0 0x7f391c038d80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3928009990 tx=0x7f3928006e40 comp rx=0 tx=0).stop 2024-04-22T23:26:46.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:46.180+0000 7f3939824700 1 -- 172.21.15.107:0/772076085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39340fd040 msgr2=0x7f393410d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:46.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:46.180+0000 7f3939824700 1 --2- 172.21.15.107:0/772076085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39340fd040 0x7f393410d090 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f39240096b0 tx=0x7f3924009e30 comp rx=0 tx=0).stop 2024-04-22T23:26:46.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:46.180+0000 7f3939824700 1 -- 172.21.15.107:0/772076085 shutdown_connections 2024-04-22T23:26:46.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:46.180+0000 7f3939824700 1 --2- 172.21.15.107:0/772076085 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f391c0368d0 0x7f391c038d80 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:46.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:46.180+0000 7f3939824700 1 --2- 172.21.15.107:0/772076085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39340fd040 0x7f393410d090 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:46.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:46.180+0000 7f3939824700 1 -- 172.21.15.107:0/772076085 >> 172.21.15.107:0/772076085 conn(0x7f39340f70c0 msgr2=0x7f3934065910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:26:46.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:46.180+0000 7f3939824700 1 -- 172.21.15.107:0/772076085 shutdown_connections 2024-04-22T23:26:46.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:46.180+0000 7f3939824700 1 -- 172.21.15.107:0/772076085 wait complete. 2024-04-22T23:26:46.778 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi157 2024-04-22T23:26:46.778 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-22T23:26:46.779 DEBUG:teuthology.orchestra.run.smithi157:> dd of=/etc/ceph/ceph.conf 2024-04-22T23:26:46.797 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-22T23:26:46.797 DEBUG:teuthology.orchestra.run.smithi157:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-22T23:26:46.854 INFO:tasks.cephadm:Adding host smithi157 to orchestrator... 2024-04-22T23:26:46.854 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph orch host add smithi157 2024-04-22T23:26:47.115 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:26:47.490 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:47 smithi107 ceph-mon[26355]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:26:47.490 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:47 smithi107 ceph-mon[26355]: from='mgr.14160 172.21.15.107:0/1348717222' entity='mgr.smithi107.uipxvm' 2024-04-22T23:26:47.490 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:47 smithi107 ceph-mon[26355]: from='mgr.14160 172.21.15.107:0/1348717222' entity='mgr.smithi107.uipxvm' 2024-04-22T23:26:47.490 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:47 smithi107 ceph-mon[26355]: from='mgr.14160 172.21.15.107:0/1348717222' entity='mgr.smithi107.uipxvm' 2024-04-22T23:26:47.490 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:47 smithi107 ceph-mon[26355]: from='mgr.14160 172.21.15.107:0/1348717222' entity='mgr.smithi107.uipxvm' 2024-04-22T23:26:47.490 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:47 smithi107 ceph-mon[26355]: from='mgr.14160 172.21.15.107:0/1348717222' entity='mgr.smithi107.uipxvm' 2024-04-22T23:26:48.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:48 smithi107 ceph-mon[26355]: Deploying daemon node-exporter.smithi107 on smithi107 2024-04-22T23:26:48.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:48.822+0000 7fca49e1f700 1 -- 172.21.15.107:0/4207302744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca440669f0 msgr2=0x7fca440ff650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:48.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:48.822+0000 7fca49e1f700 1 --2- 172.21.15.107:0/4207302744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca440669f0 0x7fca440ff650 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fca340048c0 tx=0x7fca34009e30 comp rx=0 tx=0).stop 2024-04-22T23:26:48.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:48.822+0000 7fca49e1f700 1 -- 172.21.15.107:0/4207302744 shutdown_connections 2024-04-22T23:26:48.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:48.822+0000 7fca49e1f700 1 --2- 172.21.15.107:0/4207302744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca440669f0 0x7fca440ff650 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:48.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:48.822+0000 7fca49e1f700 1 -- 172.21.15.107:0/4207302744 >> 172.21.15.107:0/4207302744 conn(0x7fca440f70c0 msgr2=0x7fca440f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:26:48.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:48.823+0000 7fca49e1f700 1 -- 172.21.15.107:0/4207302744 shutdown_connections 2024-04-22T23:26:48.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:48.823+0000 7fca49e1f700 1 -- 172.21.15.107:0/4207302744 wait complete. 2024-04-22T23:26:48.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:48.823+0000 7fca49e1f700 1 Processor -- start 2024-04-22T23:26:48.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:48.823+0000 7fca49e1f700 1 -- start start 2024-04-22T23:26:48.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:48.823+0000 7fca49e1f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca440669f0 0x7fca440fcfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:48.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:48.823+0000 7fca49e1f700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca440fd4e0 con 0x7fca440669f0 2024-04-22T23:26:48.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:48.823+0000 7fca437fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca440669f0 0x7fca440fcfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:48.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:48.823+0000 7fca437fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca440669f0 0x7fca440fcfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57344/0 (socket says 172.21.15.107:57344) 2024-04-22T23:26:48.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:48.823+0000 7fca437fe700 1 -- 172.21.15.107:0/1040052480 learned_addr learned my addr 172.21.15.107:0/1040052480 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:26:48.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:48.824+0000 7fca437fe700 1 -- 172.21.15.107:0/1040052480 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca34005040 con 0x7fca440669f0 2024-04-22T23:26:48.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:48.824+0000 7fca437fe700 1 --2- 172.21.15.107:0/1040052480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca440669f0 0x7fca440fcfa0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fca34009680 tx=0x7fca34009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:48.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:48.824+0000 7fca2bfff700 1 -- 172.21.15.107:0/1040052480 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fca3400bed0 con 0x7fca440669f0 2024-04-22T23:26:48.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:48.824+0000 7fca49e1f700 1 -- 172.21.15.107:0/1040052480 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca440fb5b0 con 0x7fca440669f0 2024-04-22T23:26:48.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:48.824+0000 7fca2bfff700 1 -- 172.21.15.107:0/1040052480 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fca3400b5c0 con 0x7fca440669f0 2024-04-22T23:26:48.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:48.824+0000 7fca2bfff700 1 -- 172.21.15.107:0/1040052480 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fca3401c5b0 con 0x7fca440669f0 2024-04-22T23:26:48.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:48.824+0000 7fca49e1f700 1 -- 172.21.15.107:0/1040052480 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca440fbaa0 con 0x7fca440669f0 2024-04-22T23:26:48.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:48.825+0000 7fca2bfff700 1 -- 172.21.15.107:0/1040052480 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7fca3401c710 con 0x7fca440669f0 2024-04-22T23:26:48.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:48.825+0000 7fca2bfff700 1 --2- 172.21.15.107:0/1040052480 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7fca2c0368d0 0x7fca2c038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:48.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:48.825+0000 7fca2bfff700 1 -- 172.21.15.107:0/1040052480 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fca34013a80 con 0x7fca440669f0 2024-04-22T23:26:48.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:48.825+0000 7fca42ffd700 1 --2- 172.21.15.107:0/1040052480 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7fca2c0368d0 0x7fca2c038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:48.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:48.825+0000 7fca49e1f700 1 -- 172.21.15.107:0/1040052480 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca4405f5f0 con 0x7fca440669f0 2024-04-22T23:26:48.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:48.826+0000 7fca42ffd700 1 --2- 172.21.15.107:0/1040052480 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7fca2c0368d0 0x7fca2c038d80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fca38009990 tx=0x7fca38006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:48.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:48.831+0000 7fca2bfff700 1 -- 172.21.15.107:0/1040052480 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fca34025400 con 0x7fca440669f0 2024-04-22T23:26:49.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:49.064+0000 7fca49e1f700 1 -- 172.21.15.107:0/1040052480 --> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi157", "target": ["mon-mgr", ""]}) v1 -- 0x7fca440fbce0 con 0x7fca2c0368d0 2024-04-22T23:26:49.739 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:49 smithi107 ceph-mon[26355]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi157", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:26:50.952 INFO:teuthology.orchestra.run.smithi107.stdout:Added host 'smithi157' with addr '172.21.15.157' 2024-04-22T23:26:50.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:50.951+0000 7fca2bfff700 1 -- 172.21.15.107:0/1040052480 <== mgr.14160 v2:172.21.15.107:6800/3427816626 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fca440fbce0 con 0x7fca2c0368d0 2024-04-22T23:26:50.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:50.953+0000 7fca49e1f700 1 -- 172.21.15.107:0/1040052480 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7fca2c0368d0 msgr2=0x7fca2c038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:50.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:50.953+0000 7fca49e1f700 1 --2- 172.21.15.107:0/1040052480 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7fca2c0368d0 0x7fca2c038d80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fca38009990 tx=0x7fca38006e40 comp rx=0 tx=0).stop 2024-04-22T23:26:50.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:50.953+0000 7fca49e1f700 1 -- 172.21.15.107:0/1040052480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca440669f0 msgr2=0x7fca440fcfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:50.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:50.953+0000 7fca49e1f700 1 --2- 172.21.15.107:0/1040052480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca440669f0 0x7fca440fcfa0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fca34009680 tx=0x7fca34009e30 comp rx=0 tx=0).stop 2024-04-22T23:26:50.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:50.953+0000 7fca49e1f700 1 -- 172.21.15.107:0/1040052480 shutdown_connections 2024-04-22T23:26:50.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:50.953+0000 7fca49e1f700 1 --2- 172.21.15.107:0/1040052480 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7fca2c0368d0 0x7fca2c038d80 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:50.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:50.953+0000 7fca49e1f700 1 --2- 172.21.15.107:0/1040052480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca440669f0 0x7fca440fcfa0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:50.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:50.953+0000 7fca49e1f700 1 -- 172.21.15.107:0/1040052480 >> 172.21.15.107:0/1040052480 conn(0x7fca440f70c0 msgr2=0x7fca44103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:26:50.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:50.953+0000 7fca49e1f700 1 -- 172.21.15.107:0/1040052480 shutdown_connections 2024-04-22T23:26:50.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:50.953+0000 7fca49e1f700 1 -- 172.21.15.107:0/1040052480 wait complete. 2024-04-22T23:26:51.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:50 smithi107 ceph-mon[26355]: Deploying cephadm binary to smithi157 2024-04-22T23:26:51.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:50 smithi107 ceph-mon[26355]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:26:51.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:50 smithi107 ceph-mon[26355]: from='mgr.14160 172.21.15.107:0/1348717222' entity='mgr.smithi107.uipxvm' 2024-04-22T23:26:51.718 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-22T23:26:52.225 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:26:52.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:51 smithi107 ceph-mon[26355]: from='mgr.14160 172.21.15.107:0/1348717222' entity='mgr.smithi107.uipxvm' 2024-04-22T23:26:52.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:51 smithi107 ceph-mon[26355]: Added host smithi157 2024-04-22T23:26:53.465 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:53 smithi107 ceph-mon[26355]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:26:53.465 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:53 smithi107 ceph-mon[26355]: from='mgr.14160 172.21.15.107:0/1348717222' entity='mgr.smithi107.uipxvm' 2024-04-22T23:26:53.465 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:53 smithi107 ceph-mon[26355]: from='mgr.14160 172.21.15.107:0/1348717222' entity='mgr.smithi107.uipxvm' 2024-04-22T23:26:53.465 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:53 smithi107 ceph-mon[26355]: from='mgr.14160 172.21.15.107:0/1348717222' entity='mgr.smithi107.uipxvm' 2024-04-22T23:26:53.466 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:53 smithi107 ceph-mon[26355]: from='mgr.14160 172.21.15.107:0/1348717222' entity='mgr.smithi107.uipxvm' 2024-04-22T23:26:53.466 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:53 smithi107 ceph-mon[26355]: Deploying daemon alertmanager.smithi107 on smithi107 2024-04-22T23:26:53.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:53.694+0000 7f3702975700 1 -- 172.21.15.107:0/1760013006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36fc066950 msgr2=0x7f36fc0ff720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:53.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:53.694+0000 7f3702975700 1 --2- 172.21.15.107:0/1760013006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36fc066950 0x7f36fc0ff720 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f36ec0048c0 tx=0x7f36ec009e30 comp rx=0 tx=0).stop 2024-04-22T23:26:53.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:53.694+0000 7f3702975700 1 -- 172.21.15.107:0/1760013006 shutdown_connections 2024-04-22T23:26:53.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:53.694+0000 7f3702975700 1 --2- 172.21.15.107:0/1760013006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36fc066950 0x7f36fc0ff720 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:53.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:53.695+0000 7f3702975700 1 -- 172.21.15.107:0/1760013006 >> 172.21.15.107:0/1760013006 conn(0x7f36fc0f70d0 msgr2=0x7f36fc0f9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:26:53.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:53.695+0000 7f3702975700 1 -- 172.21.15.107:0/1760013006 shutdown_connections 2024-04-22T23:26:53.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:53.695+0000 7f3702975700 1 -- 172.21.15.107:0/1760013006 wait complete. 2024-04-22T23:26:53.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:53.695+0000 7f3702975700 1 Processor -- start 2024-04-22T23:26:53.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:53.695+0000 7f3702975700 1 -- start start 2024-04-22T23:26:53.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:53.695+0000 7f3702975700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36fc066950 0x7f36fc10d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:53.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:53.695+0000 7f3702975700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36fc10d680 con 0x7f36fc066950 2024-04-22T23:26:53.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:53.695+0000 7f36fbfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36fc066950 0x7f36fc10d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:53.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:53.695+0000 7f36fbfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36fc066950 0x7f36fc10d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47162/0 (socket says 172.21.15.107:47162) 2024-04-22T23:26:53.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:53.695+0000 7f36fbfff700 1 -- 172.21.15.107:0/331615307 learned_addr learned my addr 172.21.15.107:0/331615307 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:26:53.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:53.696+0000 7f36fbfff700 1 -- 172.21.15.107:0/331615307 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36ec005040 con 0x7f36fc066950 2024-04-22T23:26:53.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:53.696+0000 7f36fbfff700 1 --2- 172.21.15.107:0/331615307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36fc066950 0x7f36fc10d140 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f36ec000c00 tx=0x7f36ec009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:53.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:53.696+0000 7f36f8ff9700 1 -- 172.21.15.107:0/331615307 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f36ec004e60 con 0x7f36fc066950 2024-04-22T23:26:53.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:53.696+0000 7f36f8ff9700 1 -- 172.21.15.107:0/331615307 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f36ec009650 con 0x7f36fc066950 2024-04-22T23:26:53.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:53.696+0000 7f3702975700 1 -- 172.21.15.107:0/331615307 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36fc10d880 con 0x7f36fc066950 2024-04-22T23:26:53.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:53.696+0000 7f36f8ff9700 1 -- 172.21.15.107:0/331615307 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f36ec01c5b0 con 0x7f36fc066950 2024-04-22T23:26:53.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:53.696+0000 7f3702975700 1 -- 172.21.15.107:0/331615307 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36fc10e3f0 con 0x7f36fc066950 2024-04-22T23:26:53.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:53.697+0000 7f3702975700 1 -- 172.21.15.107:0/331615307 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36fc102f70 con 0x7f36fc066950 2024-04-22T23:26:53.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:53.697+0000 7f36f8ff9700 1 -- 172.21.15.107:0/331615307 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7f36ec026440 con 0x7f36fc066950 2024-04-22T23:26:53.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:53.697+0000 7f36f8ff9700 1 --2- 172.21.15.107:0/331615307 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f36e4036880 0x7f36e4038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:26:53.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:53.697+0000 7f36f8ff9700 1 -- 172.21.15.107:0/331615307 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f36ec0493a0 con 0x7f36fc066950 2024-04-22T23:26:53.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:53.697+0000 7f36fb7fe700 1 --2- 172.21.15.107:0/331615307 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f36e4036880 0x7f36e4038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:26:53.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:53.698+0000 7f36fb7fe700 1 --2- 172.21.15.107:0/331615307 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f36e4036880 0x7f36e4038d30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f36f0009990 tx=0x7f36f0006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:26:53.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:53.702+0000 7f36f8ff9700 1 -- 172.21.15.107:0/331615307 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f36ec011550 con 0x7f36fc066950 2024-04-22T23:26:53.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:53.906+0000 7f3702975700 1 -- 172.21.15.107:0/331615307 --> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f36fc10e630 con 0x7f36e4036880 2024-04-22T23:26:53.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:53.907+0000 7f36f8ff9700 1 -- 172.21.15.107:0/331615307 <== mgr.14160 v2:172.21.15.107:6800/3427816626 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+161 (secure 0 0 0) 0x7f36fc10e630 con 0x7f36e4036880 2024-04-22T23:26:53.907 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:26:53.907 INFO:teuthology.orchestra.run.smithi107.stdout:[{"addr": "172.21.15.107", "hostname": "smithi107", "labels": [], "status": ""}, {"addr": "172.21.15.157", "hostname": "smithi157", "labels": [], "status": ""}] 2024-04-22T23:26:53.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:53.909+0000 7f3702975700 1 -- 172.21.15.107:0/331615307 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f36e4036880 msgr2=0x7f36e4038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:53.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:53.909+0000 7f3702975700 1 --2- 172.21.15.107:0/331615307 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f36e4036880 0x7f36e4038d30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f36f0009990 tx=0x7f36f0006e40 comp rx=0 tx=0).stop 2024-04-22T23:26:53.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:53.909+0000 7f3702975700 1 -- 172.21.15.107:0/331615307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36fc066950 msgr2=0x7f36fc10d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:26:53.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:53.909+0000 7f3702975700 1 --2- 172.21.15.107:0/331615307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36fc066950 0x7f36fc10d140 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f36ec000c00 tx=0x7f36ec009e30 comp rx=0 tx=0).stop 2024-04-22T23:26:53.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:53.909+0000 7f3702975700 1 -- 172.21.15.107:0/331615307 shutdown_connections 2024-04-22T23:26:53.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:53.909+0000 7f3702975700 1 --2- 172.21.15.107:0/331615307 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f36e4036880 0x7f36e4038d30 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:53.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:53.909+0000 7f3702975700 1 --2- 172.21.15.107:0/331615307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36fc066950 0x7f36fc10d140 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:26:53.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:53.909+0000 7f3702975700 1 -- 172.21.15.107:0/331615307 >> 172.21.15.107:0/331615307 conn(0x7f36fc0f70d0 msgr2=0x7f36fc103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:26:53.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:53.909+0000 7f3702975700 1 -- 172.21.15.107:0/331615307 shutdown_connections 2024-04-22T23:26:53.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:26:53.909+0000 7f3702975700 1 -- 172.21.15.107:0/331615307 wait complete. 2024-04-22T23:26:54.979 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-22T23:26:54.980 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-22T23:26:55.232 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:26:55.489 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:55 smithi107 ceph-mon[26355]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:26:55.489 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:55 smithi107 ceph-mon[26355]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T23:26:55.490 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:55 smithi107 ceph-mon[26355]: from='mgr.14160 172.21.15.107:0/1348717222' entity='mgr.smithi107.uipxvm' 2024-04-22T23:26:57.489 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:57 smithi107 ceph-mon[26355]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:26:59.242 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:26:59 smithi107 ceph-mon[26355]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:27:00.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:27:00.198+0000 7f08616e7700 1 -- 172.21.15.107:0/2564240102 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f085c066590 msgr2=0x7f085c0ff6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:27:00.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:27:00.198+0000 7f08616e7700 1 --2- 172.21.15.107:0/2564240102 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f085c066590 0x7f085c0ff6c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f084c0048c0 tx=0x7f084c009e30 comp rx=0 tx=0).stop 2024-04-22T23:27:00.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:27:00.198+0000 7f08616e7700 1 -- 172.21.15.107:0/2564240102 shutdown_connections 2024-04-22T23:27:00.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:27:00.198+0000 7f08616e7700 1 --2- 172.21.15.107:0/2564240102 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f085c066590 0x7f085c0ff6c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:27:00.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:27:00.198+0000 7f08616e7700 1 -- 172.21.15.107:0/2564240102 >> 172.21.15.107:0/2564240102 conn(0x7f085c0f70d0 msgr2=0x7f085c0f9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:27:00.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:27:00.198+0000 7f08616e7700 1 -- 172.21.15.107:0/2564240102 shutdown_connections 2024-04-22T23:27:00.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:27:00.198+0000 7f08616e7700 1 -- 172.21.15.107:0/2564240102 wait complete. 2024-04-22T23:27:00.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:27:00.199+0000 7f08616e7700 1 Processor -- start 2024-04-22T23:27:00.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:27:00.199+0000 7f08616e7700 1 -- start start 2024-04-22T23:27:00.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:27:00.199+0000 7f08616e7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f085c066590 0x7f085c0fd010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:27:00.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:27:00.199+0000 7f08616e7700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f085c0fd550 con 0x7f085c066590 2024-04-22T23:27:00.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:27:00.199+0000 7f085affd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f085c066590 0x7f085c0fd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:27:00.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:27:00.199+0000 7f085affd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f085c066590 0x7f085c0fd010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47180/0 (socket says 172.21.15.107:47180) 2024-04-22T23:27:00.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:27:00.199+0000 7f085affd700 1 -- 172.21.15.107:0/408706735 learned_addr learned my addr 172.21.15.107:0/408706735 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:27:00.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:27:00.199+0000 7f085affd700 1 -- 172.21.15.107:0/408706735 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f084c005040 con 0x7f085c066590 2024-04-22T23:27:00.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:27:00.200+0000 7f085affd700 1 --2- 172.21.15.107:0/408706735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f085c066590 0x7f085c0fd010 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f084c009680 tx=0x7f084c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:27:00.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:27:00.200+0000 7f0853fff700 1 -- 172.21.15.107:0/408706735 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f084c00bed0 con 0x7f085c066590 2024-04-22T23:27:00.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:27:00.200+0000 7f0853fff700 1 -- 172.21.15.107:0/408706735 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f084c00b5c0 con 0x7f085c066590 2024-04-22T23:27:00.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:27:00.200+0000 7f08616e7700 1 -- 172.21.15.107:0/408706735 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f085c0fb620 con 0x7f085c066590 2024-04-22T23:27:00.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:27:00.200+0000 7f0853fff700 1 -- 172.21.15.107:0/408706735 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f084c01c5b0 con 0x7f085c066590 2024-04-22T23:27:00.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:27:00.200+0000 7f08616e7700 1 -- 172.21.15.107:0/408706735 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f085c0fbb10 con 0x7f085c066590 2024-04-22T23:27:00.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:27:00.201+0000 7f0853fff700 1 -- 172.21.15.107:0/408706735 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7f084c01c710 con 0x7f085c066590 2024-04-22T23:27:00.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:27:00.201+0000 7f08616e7700 1 -- 172.21.15.107:0/408706735 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f085c05f5f0 con 0x7f085c066590 2024-04-22T23:27:00.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:27:00.201+0000 7f0853fff700 1 --2- 172.21.15.107:0/408706735 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f0848036880 0x7f0848038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:27:00.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:27:00.201+0000 7f0853fff700 1 -- 172.21.15.107:0/408706735 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f084c0499b0 con 0x7f085c066590 2024-04-22T23:27:00.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:27:00.201+0000 7f08527fc700 1 --2- 172.21.15.107:0/408706735 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f0848036880 0x7f0848038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:27:00.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:27:00.201+0000 7f08527fc700 1 --2- 172.21.15.107:0/408706735 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f0848036880 0x7f0848038d30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0844009990 tx=0x7f0844006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:27:00.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:27:00.206+0000 7f0853fff700 1 -- 172.21.15.107:0/408706735 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f084c025400 con 0x7f085c066590 2024-04-22T23:27:00.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:27:00.402+0000 7f08616e7700 1 -- 172.21.15.107:0/408706735 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f085c106440 con 0x7f085c066590 2024-04-22T23:27:01.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:27:01.227+0000 7f0853fff700 1 -- 172.21.15.107:0/408706735 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f084c011b60 con 0x7f085c066590 2024-04-22T23:27:01.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:27:01.229+0000 7f08616e7700 1 -- 172.21.15.107:0/408706735 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f0848036880 msgr2=0x7f0848038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:27:01.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:27:01.229+0000 7f08616e7700 1 --2- 172.21.15.107:0/408706735 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f0848036880 0x7f0848038d30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0844009990 tx=0x7f0844006e40 comp rx=0 tx=0).stop 2024-04-22T23:27:01.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:27:01.229+0000 7f08616e7700 1 -- 172.21.15.107:0/408706735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f085c066590 msgr2=0x7f085c0fd010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:27:01.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:27:01.229+0000 7f08616e7700 1 --2- 172.21.15.107:0/408706735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f085c066590 0x7f085c0fd010 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f084c009680 tx=0x7f084c009e30 comp rx=0 tx=0).stop 2024-04-22T23:27:01.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:27:01.229+0000 7f08616e7700 1 -- 172.21.15.107:0/408706735 shutdown_connections 2024-04-22T23:27:01.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:27:01.229+0000 7f08616e7700 1 --2- 172.21.15.107:0/408706735 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f0848036880 0x7f0848038d30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:27:01.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:27:01.229+0000 7f08616e7700 1 --2- 172.21.15.107:0/408706735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f085c066590 0x7f085c0fd010 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:27:01.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:27:01.229+0000 7f08616e7700 1 -- 172.21.15.107:0/408706735 >> 172.21.15.107:0/408706735 conn(0x7f085c0f70d0 msgr2=0x7f085c103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:27:01.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:27:01.229+0000 7f08616e7700 1 -- 172.21.15.107:0/408706735 shutdown_connections 2024-04-22T23:27:01.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:27:01.229+0000 7f08616e7700 1 -- 172.21.15.107:0/408706735 wait complete. 2024-04-22T23:27:01.232 INFO:teuthology.orchestra.run.smithi107.stderr:adjusted tunables profile to default 2024-04-22T23:27:01.268 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:01 smithi107 ceph-mon[26355]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:27:01.268 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:01 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/408706735' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-22T23:27:01.846 INFO:tasks.cephadm:Adding mon.smithi107 on smithi107 2024-04-22T23:27:01.846 INFO:tasks.cephadm:Adding mon.smithi157 on smithi157 2024-04-22T23:27:01.846 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph orch apply mon '2;smithi107:172.21.15.107=smithi107;smithi157:172.21.15.157=smithi157' 2024-04-22T23:27:02.203 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-22T23:27:02.251 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-22T23:27:02.420 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:02 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/408706735' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-22T23:27:02.420 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:02 smithi107 ceph-mon[26355]: osdmap e4: 0 total, 0 up, 0 in 2024-04-22T23:27:03.489 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:03 smithi107 ceph-mon[26355]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:27:03.489 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:03 smithi107 ceph-mon[26355]: from='mgr.14160 172.21.15.107:0/1348717222' entity='mgr.smithi107.uipxvm' 2024-04-22T23:27:03.489 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:03 smithi107 ceph-mon[26355]: from='mgr.14160 172.21.15.107:0/1348717222' entity='mgr.smithi107.uipxvm' 2024-04-22T23:27:03.490 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:03 smithi107 ceph-mon[26355]: from='mgr.14160 172.21.15.107:0/1348717222' entity='mgr.smithi107.uipxvm' 2024-04-22T23:27:03.490 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:03 smithi107 ceph-mon[26355]: from='mgr.14160 172.21.15.107:0/1348717222' entity='mgr.smithi107.uipxvm' 2024-04-22T23:27:03.490 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:03 smithi107 ceph-mon[26355]: from='mgr.14160 172.21.15.107:0/1348717222' entity='mgr.smithi107.uipxvm' 2024-04-22T23:27:03.490 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:03 smithi107 ceph-mon[26355]: from='mgr.14160 172.21.15.107:0/1348717222' entity='mgr.smithi107.uipxvm' 2024-04-22T23:27:03.490 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:03 smithi107 ceph-mon[26355]: from='mgr.14160 172.21.15.107:0/1348717222' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-04-22T23:27:03.490 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:03 smithi107 ceph-mon[26355]: from='mgr.14160 172.21.15.107:0/1348717222' entity='mgr.smithi107.uipxvm' 2024-04-22T23:27:03.779 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:03.776+0000 7f422ab38700 1 -- 172.21.15.157:0/709315473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42240fca60 msgr2=0x7f42240fce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:27:03.780 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:03.776+0000 7f422ab38700 1 --2- 172.21.15.157:0/709315473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42240fca60 0x7f42240fce70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f42180048c0 tx=0x7f4218009e30 comp rx=0 tx=0).stop 2024-04-22T23:27:03.780 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:03.776+0000 7f422ab38700 1 -- 172.21.15.157:0/709315473 shutdown_connections 2024-04-22T23:27:03.780 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:03.776+0000 7f422ab38700 1 --2- 172.21.15.157:0/709315473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42240fca60 0x7f42240fce70 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:27:03.780 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:03.777+0000 7f422ab38700 1 -- 172.21.15.157:0/709315473 >> 172.21.15.157:0/709315473 conn(0x7f42240f81c0 msgr2=0x7f42240fa5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:27:03.780 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:03.777+0000 7f422ab38700 1 -- 172.21.15.157:0/709315473 shutdown_connections 2024-04-22T23:27:03.780 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:03.777+0000 7f422ab38700 1 -- 172.21.15.157:0/709315473 wait complete. 2024-04-22T23:27:03.780 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:03.777+0000 7f422ab38700 1 Processor -- start 2024-04-22T23:27:03.780 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:03.777+0000 7f422ab38700 1 -- start start 2024-04-22T23:27:03.780 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:03.777+0000 7f422ab38700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42240fca60 0x7f422410d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:27:03.780 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:03.777+0000 7f422ab38700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422410d5f0 con 0x7f42240fca60 2024-04-22T23:27:03.781 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:03.778+0000 7f42288d4700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42240fca60 0x7f422410d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:27:03.781 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:03.778+0000 7f42288d4700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42240fca60 0x7f422410d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.157:36916/0 (socket says 172.21.15.157:36916) 2024-04-22T23:27:03.781 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:03.778+0000 7f42288d4700 1 -- 172.21.15.157:0/3680012796 learned_addr learned my addr 172.21.15.157:0/3680012796 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-22T23:27:03.781 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:03.778+0000 7f42288d4700 1 -- 172.21.15.157:0/3680012796 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4218005040 con 0x7f42240fca60 2024-04-22T23:27:03.781 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:03.778+0000 7f42288d4700 1 --2- 172.21.15.157:0/3680012796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42240fca60 0x7f422410d0b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f42180096b0 tx=0x7f4218009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:27:03.781 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:03.778+0000 7f42217fa700 1 -- 172.21.15.157:0/3680012796 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f421800bed0 con 0x7f42240fca60 2024-04-22T23:27:03.781 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:03.778+0000 7f42217fa700 1 -- 172.21.15.157:0/3680012796 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f421800b5c0 con 0x7f42240fca60 2024-04-22T23:27:03.781 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:03.778+0000 7f422ab38700 1 -- 172.21.15.157:0/3680012796 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f422410d7f0 con 0x7f42240fca60 2024-04-22T23:27:03.781 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:03.778+0000 7f42217fa700 1 -- 172.21.15.157:0/3680012796 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f421801c5b0 con 0x7f42240fca60 2024-04-22T23:27:03.781 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:03.778+0000 7f422ab38700 1 -- 172.21.15.157:0/3680012796 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f422410e360 con 0x7f42240fca60 2024-04-22T23:27:03.782 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:03.779+0000 7f42217fa700 1 -- 172.21.15.157:0/3680012796 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7f421801c820 con 0x7f42240fca60 2024-04-22T23:27:03.782 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:03.779+0000 7f422ab38700 1 -- 172.21.15.157:0/3680012796 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4224102f70 con 0x7f42240fca60 2024-04-22T23:27:03.782 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:03.779+0000 7f42217fa700 1 --2- 172.21.15.157:0/3680012796 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f4214036880 0x7f4214038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:27:03.782 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:03.779+0000 7f42217fa700 1 -- 172.21.15.157:0/3680012796 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f4218049b20 con 0x7f42240fca60 2024-04-22T23:27:03.782 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:03.780+0000 7f4223fff700 1 --2- 172.21.15.157:0/3680012796 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f4214036880 0x7f4214038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:27:03.782 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:03.780+0000 7f4223fff700 1 --2- 172.21.15.157:0/3680012796 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f4214036880 0x7f4214038d30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4210009990 tx=0x7f4210006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:27:03.785 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:03.784+0000 7f42217fa700 1 -- 172.21.15.157:0/3680012796 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4218023300 con 0x7f42240fca60 2024-04-22T23:27:03.981 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:03.979+0000 7f422ab38700 1 -- 172.21.15.157:0/3680012796 --> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi107:172.21.15.107=smithi107;smithi157:172.21.15.157=smithi157", "target": ["mon-mgr", ""]}) v1 -- 0x7f422410e650 con 0x7f4214036880 2024-04-22T23:27:03.986 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:03.985+0000 7f42217fa700 1 -- 172.21.15.157:0/3680012796 <== mgr.14160 v2:172.21.15.107:6800/3427816626 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f422410e650 con 0x7f4214036880 2024-04-22T23:27:03.986 INFO:teuthology.orchestra.run.smithi157.stdout:Scheduled mon update... 2024-04-22T23:27:03.988 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:03.987+0000 7f422ab38700 1 -- 172.21.15.157:0/3680012796 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f4214036880 msgr2=0x7f4214038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:27:03.988 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:03.987+0000 7f422ab38700 1 --2- 172.21.15.157:0/3680012796 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f4214036880 0x7f4214038d30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4210009990 tx=0x7f4210006e40 comp rx=0 tx=0).stop 2024-04-22T23:27:03.988 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:03.987+0000 7f422ab38700 1 -- 172.21.15.157:0/3680012796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42240fca60 msgr2=0x7f422410d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:27:03.988 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:03.987+0000 7f422ab38700 1 --2- 172.21.15.157:0/3680012796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42240fca60 0x7f422410d0b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f42180096b0 tx=0x7f4218009e30 comp rx=0 tx=0).stop 2024-04-22T23:27:03.988 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:03.987+0000 7f422ab38700 1 -- 172.21.15.157:0/3680012796 shutdown_connections 2024-04-22T23:27:03.988 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:03.987+0000 7f422ab38700 1 --2- 172.21.15.157:0/3680012796 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f4214036880 0x7f4214038d30 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:27:03.988 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:03.987+0000 7f422ab38700 1 --2- 172.21.15.157:0/3680012796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42240fca60 0x7f422410d0b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:27:03.988 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:03.987+0000 7f422ab38700 1 -- 172.21.15.157:0/3680012796 >> 172.21.15.157:0/3680012796 conn(0x7f42240f81c0 msgr2=0x7f4224103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:27:03.988 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:03.987+0000 7f422ab38700 1 -- 172.21.15.157:0/3680012796 shutdown_connections 2024-04-22T23:27:03.988 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:03.987+0000 7f422ab38700 1 -- 172.21.15.157:0/3680012796 wait complete. 2024-04-22T23:27:04.489 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:04 smithi107 ceph-mon[26355]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-04-22T23:27:04.489 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:04 smithi107 ceph-mon[26355]: Deploying daemon grafana.smithi107 on smithi107 2024-04-22T23:27:04.489 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:04 smithi107 ceph-mon[26355]: from='mgr.14160 172.21.15.107:0/1348717222' entity='mgr.smithi107.uipxvm' 2024-04-22T23:27:04.581 DEBUG:teuthology.orchestra.run.smithi157:mon.smithi157> sudo journalctl -f -n 0 -u ceph-8252f6f2-00ff-11ef-bc93-c7b262605968@mon.smithi157.service 2024-04-22T23:27:04.583 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-22T23:27:04.584 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-22T23:27:04.858 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-22T23:27:04.906 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-22T23:27:05.489 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:05 smithi107 ceph-mon[26355]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:27:05.489 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:05 smithi107 ceph-mon[26355]: from='client.14192 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi107:172.21.15.107=smithi107;smithi157:172.21.15.157=smithi157", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:27:05.489 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:05 smithi107 ceph-mon[26355]: Saving service mon spec with placement smithi107:172.21.15.107=smithi107;smithi157:172.21.15.157=smithi157;count:2 2024-04-22T23:27:05.489 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:05 smithi107 ceph-mon[26355]: from='mgr.14160 172.21.15.107:0/1348717222' entity='mgr.smithi107.uipxvm' 2024-04-22T23:27:06.220 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:06.218+0000 7f2c93138700 1 -- 172.21.15.157:0/2859860349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c8c0fca80 msgr2=0x7f2c8c0fce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:27:06.220 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:06.218+0000 7f2c93138700 1 --2- 172.21.15.157:0/2859860349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c8c0fca80 0x7f2c8c0fce90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2c800048c0 tx=0x7f2c80009e30 comp rx=0 tx=0).stop 2024-04-22T23:27:06.220 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:06.218+0000 7f2c93138700 1 -- 172.21.15.157:0/2859860349 shutdown_connections 2024-04-22T23:27:06.220 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:06.218+0000 7f2c93138700 1 --2- 172.21.15.157:0/2859860349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c8c0fca80 0x7f2c8c0fce90 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:27:06.221 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:06.218+0000 7f2c93138700 1 -- 172.21.15.157:0/2859860349 >> 172.21.15.157:0/2859860349 conn(0x7f2c8c0f81e0 msgr2=0x7f2c8c0fa610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:27:06.221 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:06.218+0000 7f2c93138700 1 -- 172.21.15.157:0/2859860349 shutdown_connections 2024-04-22T23:27:06.221 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:06.218+0000 7f2c93138700 1 -- 172.21.15.157:0/2859860349 wait complete. 2024-04-22T23:27:06.221 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:06.218+0000 7f2c93138700 1 Processor -- start 2024-04-22T23:27:06.221 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:06.219+0000 7f2c93138700 1 -- start start 2024-04-22T23:27:06.221 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:06.219+0000 7f2c93138700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c8c0fca80 0x7f2c8c10d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:27:06.221 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:06.219+0000 7f2c93138700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8c10d5f0 con 0x7f2c8c0fca80 2024-04-22T23:27:06.221 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:06.219+0000 7f2c90ed4700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c8c0fca80 0x7f2c8c10d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:27:06.221 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:06.219+0000 7f2c90ed4700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c8c0fca80 0x7f2c8c10d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.157:36936/0 (socket says 172.21.15.157:36936) 2024-04-22T23:27:06.221 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:06.219+0000 7f2c90ed4700 1 -- 172.21.15.157:0/3848207946 learned_addr learned my addr 172.21.15.157:0/3848207946 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-22T23:27:06.221 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:06.219+0000 7f2c90ed4700 1 -- 172.21.15.157:0/3848207946 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c80005040 con 0x7f2c8c0fca80 2024-04-22T23:27:06.223 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:06.219+0000 7f2c90ed4700 1 --2- 172.21.15.157:0/3848207946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c8c0fca80 0x7f2c8c10d0b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2c800096b0 tx=0x7f2c80009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:27:06.223 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:06.220+0000 7f2c897fa700 1 -- 172.21.15.157:0/3848207946 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2c8000bed0 con 0x7f2c8c0fca80 2024-04-22T23:27:06.223 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:06.220+0000 7f2c93138700 1 -- 172.21.15.157:0/3848207946 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c8c10d7f0 con 0x7f2c8c0fca80 2024-04-22T23:27:06.223 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:06.220+0000 7f2c897fa700 1 -- 172.21.15.157:0/3848207946 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2c8000b5c0 con 0x7f2c8c0fca80 2024-04-22T23:27:06.223 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:06.220+0000 7f2c897fa700 1 -- 172.21.15.157:0/3848207946 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2c8001c5b0 con 0x7f2c8c0fca80 2024-04-22T23:27:06.223 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:06.220+0000 7f2c93138700 1 -- 172.21.15.157:0/3848207946 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c8c10e360 con 0x7f2c8c0fca80 2024-04-22T23:27:06.223 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:06.221+0000 7f2c897fa700 1 -- 172.21.15.157:0/3848207946 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7f2c80013420 con 0x7f2c8c0fca80 2024-04-22T23:27:06.223 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:06.221+0000 7f2c93138700 1 -- 172.21.15.157:0/3848207946 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c8c102f90 con 0x7f2c8c0fca80 2024-04-22T23:27:06.223 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:06.221+0000 7f2c897fa700 1 --2- 172.21.15.157:0/3848207946 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f2c7c036880 0x7f2c7c038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:27:06.223 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:06.221+0000 7f2c897fa700 1 -- 172.21.15.157:0/3848207946 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f2c80049a00 con 0x7f2c8c0fca80 2024-04-22T23:27:06.223 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:06.221+0000 7f2c8bfff700 1 --2- 172.21.15.157:0/3848207946 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f2c7c036880 0x7f2c7c038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:27:06.223 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:06.221+0000 7f2c8bfff700 1 --2- 172.21.15.157:0/3848207946 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f2c7c036880 0x7f2c7c038d30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2c78009990 tx=0x7f2c78006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:27:06.227 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:06.226+0000 7f2c897fa700 1 -- 172.21.15.157:0/3848207946 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f2c80013a80 con 0x7f2c8c0fca80 2024-04-22T23:27:06.486 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:06.485+0000 7f2c93138700 1 -- 172.21.15.157:0/3848207946 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f2c8c10ed70 con 0x7f2c8c0fca80 2024-04-22T23:27:06.487 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:06.486+0000 7f2c897fa700 1 -- 172.21.15.157:0/3848207946 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f2c80011370 con 0x7f2c8c0fca80 2024-04-22T23:27:06.487 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:27:06.487 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":1,"fsid":"8252f6f2-00ff-11ef-bc93-c7b262605968","modified":"2024-04-22T23:25:22.712092Z","created":"2024-04-22T23:25:22.712092Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi107","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-22T23:27:06.489 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:06.488+0000 7f2c93138700 1 -- 172.21.15.157:0/3848207946 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f2c7c036880 msgr2=0x7f2c7c038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:27:06.489 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:06.488+0000 7f2c93138700 1 --2- 172.21.15.157:0/3848207946 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f2c7c036880 0x7f2c7c038d30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2c78009990 tx=0x7f2c78006e40 comp rx=0 tx=0).stop 2024-04-22T23:27:06.489 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:06.488+0000 7f2c93138700 1 -- 172.21.15.157:0/3848207946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c8c0fca80 msgr2=0x7f2c8c10d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:27:06.489 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:06.488+0000 7f2c93138700 1 --2- 172.21.15.157:0/3848207946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c8c0fca80 0x7f2c8c10d0b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2c800096b0 tx=0x7f2c80009e30 comp rx=0 tx=0).stop 2024-04-22T23:27:06.489 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:06.488+0000 7f2c93138700 1 -- 172.21.15.157:0/3848207946 shutdown_connections 2024-04-22T23:27:06.489 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:06.488+0000 7f2c93138700 1 --2- 172.21.15.157:0/3848207946 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f2c7c036880 0x7f2c7c038d30 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:27:06.489 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:06.488+0000 7f2c93138700 1 --2- 172.21.15.157:0/3848207946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c8c0fca80 0x7f2c8c10d0b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:27:06.489 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:06.488+0000 7f2c93138700 1 -- 172.21.15.157:0/3848207946 >> 172.21.15.157:0/3848207946 conn(0x7f2c8c0f81e0 msgr2=0x7f2c8c103bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:27:06.489 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:06.488+0000 7f2c93138700 1 -- 172.21.15.157:0/3848207946 shutdown_connections 2024-04-22T23:27:06.489 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:06.488+0000 7f2c93138700 1 -- 172.21.15.157:0/3848207946 wait complete. 2024-04-22T23:27:06.490 INFO:teuthology.orchestra.run.smithi157.stderr:dumped monmap epoch 1 2024-04-22T23:27:07.489 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:07 smithi107 ceph-mon[26355]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:27:07.489 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:07 smithi107 ceph-mon[26355]: from='client.? 172.21.15.157:0/3848207946' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T23:27:08.061 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-22T23:27:08.062 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-22T23:27:08.311 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-22T23:27:08.368 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-22T23:27:09.489 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:09 smithi107 ceph-mon[26355]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:27:09.886 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:09.885+0000 7febfb022700 1 -- 172.21.15.157:0/1710058496 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7febf4066a30 msgr2=0x7febf40ff650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:27:09.886 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:09.885+0000 7febfb022700 1 --2- 172.21.15.157:0/1710058496 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7febf4066a30 0x7febf40ff650 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7febe80048c0 tx=0x7febe8009e30 comp rx=0 tx=0).stop 2024-04-22T23:27:09.886 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:09.885+0000 7febfb022700 1 -- 172.21.15.157:0/1710058496 shutdown_connections 2024-04-22T23:27:09.886 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:09.885+0000 7febfb022700 1 --2- 172.21.15.157:0/1710058496 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7febf4066a30 0x7febf40ff650 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:27:09.886 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:09.885+0000 7febfb022700 1 -- 172.21.15.157:0/1710058496 >> 172.21.15.157:0/1710058496 conn(0x7febf40f70c0 msgr2=0x7febf40f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:27:09.886 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:09.885+0000 7febfb022700 1 -- 172.21.15.157:0/1710058496 shutdown_connections 2024-04-22T23:27:09.886 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:09.885+0000 7febfb022700 1 -- 172.21.15.157:0/1710058496 wait complete. 2024-04-22T23:27:09.886 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:09.885+0000 7febfb022700 1 Processor -- start 2024-04-22T23:27:09.887 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:09.885+0000 7febfb022700 1 -- start start 2024-04-22T23:27:09.887 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:09.886+0000 7febfb022700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7febf4066a30 0x7febf410d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:27:09.887 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:09.886+0000 7febfb022700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf410d610 con 0x7febf4066a30 2024-04-22T23:27:09.887 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:09.886+0000 7febf8dbe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7febf4066a30 0x7febf410d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:27:09.887 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:09.886+0000 7febf8dbe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7febf4066a30 0x7febf410d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.157:36952/0 (socket says 172.21.15.157:36952) 2024-04-22T23:27:09.887 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:09.886+0000 7febf8dbe700 1 -- 172.21.15.157:0/1861201365 learned_addr learned my addr 172.21.15.157:0/1861201365 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-22T23:27:09.887 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:09.886+0000 7febf8dbe700 1 -- 172.21.15.157:0/1861201365 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe8005040 con 0x7febf4066a30 2024-04-22T23:27:09.887 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:09.886+0000 7febf8dbe700 1 --2- 172.21.15.157:0/1861201365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7febf4066a30 0x7febf410d0d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7febe8009680 tx=0x7febe8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:27:09.888 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:09.886+0000 7febf17fa700 1 -- 172.21.15.157:0/1861201365 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7febe800bed0 con 0x7febf4066a30 2024-04-22T23:27:09.888 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:09.886+0000 7febfb022700 1 -- 172.21.15.157:0/1861201365 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febf410d810 con 0x7febf4066a30 2024-04-22T23:27:09.888 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:09.886+0000 7febf17fa700 1 -- 172.21.15.157:0/1861201365 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7febe800b5c0 con 0x7febf4066a30 2024-04-22T23:27:09.888 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:09.886+0000 7febf17fa700 1 -- 172.21.15.157:0/1861201365 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7febe801c5b0 con 0x7febf4066a30 2024-04-22T23:27:09.888 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:09.886+0000 7febfb022700 1 -- 172.21.15.157:0/1861201365 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febf410e380 con 0x7febf4066a30 2024-04-22T23:27:09.889 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:09.887+0000 7febf17fa700 1 -- 172.21.15.157:0/1861201365 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7febe801c770 con 0x7febf4066a30 2024-04-22T23:27:09.889 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:09.887+0000 7febf17fa700 1 --2- 172.21.15.157:0/1861201365 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7febe4036880 0x7febe4038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:27:09.889 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:09.887+0000 7febf17fa700 1 -- 172.21.15.157:0/1861201365 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7febe8049a70 con 0x7febf4066a30 2024-04-22T23:27:09.889 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:09.888+0000 7febf3fff700 1 --2- 172.21.15.157:0/1861201365 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7febe4036880 0x7febe4038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:27:09.889 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:09.888+0000 7febfb022700 1 -- 172.21.15.157:0/1861201365 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febf4102f60 con 0x7febf4066a30 2024-04-22T23:27:09.889 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:09.888+0000 7febf3fff700 1 --2- 172.21.15.157:0/1861201365 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7febe4036880 0x7febe4038d30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7febe0009990 tx=0x7febe0006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:27:09.892 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:09.893+0000 7febf17fa700 1 -- 172.21.15.157:0/1861201365 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7febe8025400 con 0x7febf4066a30 2024-04-22T23:27:10.150 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:10.150+0000 7febfb022700 1 -- 172.21.15.157:0/1861201365 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7febf405f5f0 con 0x7febf4066a30 2024-04-22T23:27:10.151 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:10.151+0000 7febf17fa700 1 -- 172.21.15.157:0/1861201365 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7febf405f5f0 con 0x7febf4066a30 2024-04-22T23:27:10.151 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:27:10.151 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":1,"fsid":"8252f6f2-00ff-11ef-bc93-c7b262605968","modified":"2024-04-22T23:25:22.712092Z","created":"2024-04-22T23:25:22.712092Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi107","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-22T23:27:10.153 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:10.153+0000 7febfb022700 1 -- 172.21.15.157:0/1861201365 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7febe4036880 msgr2=0x7febe4038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:27:10.153 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:10.153+0000 7febfb022700 1 --2- 172.21.15.157:0/1861201365 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7febe4036880 0x7febe4038d30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7febe0009990 tx=0x7febe0006e40 comp rx=0 tx=0).stop 2024-04-22T23:27:10.153 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:10.153+0000 7febfb022700 1 -- 172.21.15.157:0/1861201365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7febf4066a30 msgr2=0x7febf410d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:27:10.154 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:10.153+0000 7febfb022700 1 --2- 172.21.15.157:0/1861201365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7febf4066a30 0x7febf410d0d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7febe8009680 tx=0x7febe8009e30 comp rx=0 tx=0).stop 2024-04-22T23:27:10.154 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:10.153+0000 7febfb022700 1 -- 172.21.15.157:0/1861201365 shutdown_connections 2024-04-22T23:27:10.154 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:10.153+0000 7febfb022700 1 --2- 172.21.15.157:0/1861201365 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7febe4036880 0x7febe4038d30 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:27:10.154 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:10.153+0000 7febfb022700 1 --2- 172.21.15.157:0/1861201365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7febf4066a30 0x7febf410d0d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:27:10.154 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:10.153+0000 7febfb022700 1 -- 172.21.15.157:0/1861201365 >> 172.21.15.157:0/1861201365 conn(0x7febf40f70c0 msgr2=0x7febf4103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:27:10.154 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:10.153+0000 7febfb022700 1 -- 172.21.15.157:0/1861201365 shutdown_connections 2024-04-22T23:27:10.154 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:10.153+0000 7febfb022700 1 -- 172.21.15.157:0/1861201365 wait complete. 2024-04-22T23:27:10.154 INFO:teuthology.orchestra.run.smithi157.stderr:dumped monmap epoch 1 2024-04-22T23:27:10.739 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:10 smithi107 ceph-mon[26355]: from='client.? 172.21.15.157:0/1861201365' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T23:27:11.717 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-22T23:27:11.717 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-22T23:27:11.739 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:11 smithi107 ceph-mon[26355]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:27:11.954 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-22T23:27:12.002 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-22T23:27:13.354 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:13 smithi107 ceph-mon[26355]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:27:13.545 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:13.543+0000 7f5b6373e700 1 -- 172.21.15.157:0/1964494727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b5c0fb970 msgr2=0x7f5b5c0fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:27:13.545 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:13.543+0000 7f5b6373e700 1 --2- 172.21.15.157:0/1964494727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b5c0fb970 0x7f5b5c0fbd80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5b540048c0 tx=0x7f5b54009e30 comp rx=0 tx=0).stop 2024-04-22T23:27:13.545 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:13.543+0000 7f5b6373e700 1 -- 172.21.15.157:0/1964494727 shutdown_connections 2024-04-22T23:27:13.545 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:13.543+0000 7f5b6373e700 1 --2- 172.21.15.157:0/1964494727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b5c0fb970 0x7f5b5c0fbd80 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:27:13.545 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:13.543+0000 7f5b6373e700 1 -- 172.21.15.157:0/1964494727 >> 172.21.15.157:0/1964494727 conn(0x7f5b5c0f70d0 msgr2=0x7f5b5c0f9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:27:13.545 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:13.543+0000 7f5b6373e700 1 -- 172.21.15.157:0/1964494727 shutdown_connections 2024-04-22T23:27:13.545 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:13.543+0000 7f5b6373e700 1 -- 172.21.15.157:0/1964494727 wait complete. 2024-04-22T23:27:13.546 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:13.544+0000 7f5b6373e700 1 Processor -- start 2024-04-22T23:27:13.546 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:13.544+0000 7f5b6373e700 1 -- start start 2024-04-22T23:27:13.546 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:13.544+0000 7f5b6373e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b5c0fb970 0x7f5b5c065c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:27:13.546 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:13.544+0000 7f5b6373e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5c10fd80 con 0x7f5b5c0fb970 2024-04-22T23:27:13.546 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:13.544+0000 7f5b614da700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b5c0fb970 0x7f5b5c065c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:27:13.546 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:13.544+0000 7f5b614da700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b5c0fb970 0x7f5b5c065c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.157:40966/0 (socket says 172.21.15.157:40966) 2024-04-22T23:27:13.546 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:13.544+0000 7f5b614da700 1 -- 172.21.15.157:0/3460985381 learned_addr learned my addr 172.21.15.157:0/3460985381 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-22T23:27:13.546 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:13.544+0000 7f5b614da700 1 -- 172.21.15.157:0/3460985381 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b54005040 con 0x7f5b5c0fb970 2024-04-22T23:27:13.546 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:13.544+0000 7f5b614da700 1 --2- 172.21.15.157:0/3460985381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b5c0fb970 0x7f5b5c065c40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5b540096b0 tx=0x7f5b54009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:27:13.546 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:13.545+0000 7f5b51ffb700 1 -- 172.21.15.157:0/3460985381 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5b5400bed0 con 0x7f5b5c0fb970 2024-04-22T23:27:13.546 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:13.545+0000 7f5b51ffb700 1 -- 172.21.15.157:0/3460985381 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5b5400b5c0 con 0x7f5b5c0fb970 2024-04-22T23:27:13.546 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:13.545+0000 7f5b6373e700 1 -- 172.21.15.157:0/3460985381 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b5c10ff20 con 0x7f5b5c0fb970 2024-04-22T23:27:13.546 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:13.545+0000 7f5b51ffb700 1 -- 172.21.15.157:0/3460985381 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5b5401c5b0 con 0x7f5b5c0fb970 2024-04-22T23:27:13.546 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:13.545+0000 7f5b6373e700 1 -- 172.21.15.157:0/3460985381 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b5c1103b0 con 0x7f5b5c0fb970 2024-04-22T23:27:13.547 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:13.546+0000 7f5b51ffb700 1 -- 172.21.15.157:0/3460985381 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7f5b54013420 con 0x7f5b5c0fb970 2024-04-22T23:27:13.548 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:13.546+0000 7f5b6373e700 1 -- 172.21.15.157:0/3460985381 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b5c105930 con 0x7f5b5c0fb970 2024-04-22T23:27:13.548 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:13.546+0000 7f5b51ffb700 1 --2- 172.21.15.157:0/3460985381 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f5b480368d0 0x7f5b48038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:27:13.548 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:13.546+0000 7f5b51ffb700 1 -- 172.21.15.157:0/3460985381 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f5b54049a10 con 0x7f5b5c0fb970 2024-04-22T23:27:13.548 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:13.546+0000 7f5b60cd9700 1 --2- 172.21.15.157:0/3460985381 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f5b480368d0 0x7f5b48038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:27:13.548 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:13.546+0000 7f5b60cd9700 1 --2- 172.21.15.157:0/3460985381 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f5b480368d0 0x7f5b48038d80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c009990 tx=0x7f5b4c006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:27:13.551 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:13.551+0000 7f5b51ffb700 1 -- 172.21.15.157:0/3460985381 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f5b54025400 con 0x7f5b5c0fb970 2024-04-22T23:27:13.811 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:13.811+0000 7f5b6373e700 1 -- 172.21.15.157:0/3460985381 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f5b5c05f5f0 con 0x7f5b5c0fb970 2024-04-22T23:27:13.811 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:13.811+0000 7f5b51ffb700 1 -- 172.21.15.157:0/3460985381 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f5b54004d30 con 0x7f5b5c0fb970 2024-04-22T23:27:13.812 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:27:13.812 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":1,"fsid":"8252f6f2-00ff-11ef-bc93-c7b262605968","modified":"2024-04-22T23:25:22.712092Z","created":"2024-04-22T23:25:22.712092Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi107","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-22T23:27:13.813 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:13.813+0000 7f5b6373e700 1 -- 172.21.15.157:0/3460985381 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f5b480368d0 msgr2=0x7f5b48038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:27:13.813 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:13.813+0000 7f5b6373e700 1 --2- 172.21.15.157:0/3460985381 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f5b480368d0 0x7f5b48038d80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c009990 tx=0x7f5b4c006e40 comp rx=0 tx=0).stop 2024-04-22T23:27:13.813 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:13.813+0000 7f5b6373e700 1 -- 172.21.15.157:0/3460985381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b5c0fb970 msgr2=0x7f5b5c065c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:27:13.814 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:13.813+0000 7f5b6373e700 1 --2- 172.21.15.157:0/3460985381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b5c0fb970 0x7f5b5c065c40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5b540096b0 tx=0x7f5b54009e30 comp rx=0 tx=0).stop 2024-04-22T23:27:13.814 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:13.813+0000 7f5b6373e700 1 -- 172.21.15.157:0/3460985381 shutdown_connections 2024-04-22T23:27:13.814 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:13.813+0000 7f5b6373e700 1 --2- 172.21.15.157:0/3460985381 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f5b480368d0 0x7f5b48038d80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:27:13.814 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:13.813+0000 7f5b6373e700 1 --2- 172.21.15.157:0/3460985381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b5c0fb970 0x7f5b5c065c40 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:27:13.814 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:13.813+0000 7f5b6373e700 1 -- 172.21.15.157:0/3460985381 >> 172.21.15.157:0/3460985381 conn(0x7f5b5c0f70d0 msgr2=0x7f5b5c102ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:27:13.814 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:13.813+0000 7f5b6373e700 1 -- 172.21.15.157:0/3460985381 shutdown_connections 2024-04-22T23:27:13.814 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:13.813+0000 7f5b6373e700 1 -- 172.21.15.157:0/3460985381 wait complete. 2024-04-22T23:27:13.814 INFO:teuthology.orchestra.run.smithi157.stderr:dumped monmap epoch 1 2024-04-22T23:27:14.427 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:14 smithi107 ceph-mon[26355]: from='client.? 172.21.15.157:0/3460985381' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T23:27:15.346 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-22T23:27:15.347 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-22T23:27:15.489 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:15 smithi107 ceph-mon[26355]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:27:15.597 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-22T23:27:15.645 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-22T23:27:17.088 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:17.088+0000 7f4f43e2f700 1 -- 172.21.15.157:0/3324470607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f3c0fe0b0 msgr2=0x7f4f3c0fe4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:27:17.088 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:17.088+0000 7f4f43e2f700 1 --2- 172.21.15.157:0/3324470607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f3c0fe0b0 0x7f4f3c0fe4c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4f380048c0 tx=0x7f4f38009e30 comp rx=0 tx=0).stop 2024-04-22T23:27:17.088 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:17.088+0000 7f4f43e2f700 1 -- 172.21.15.157:0/3324470607 shutdown_connections 2024-04-22T23:27:17.088 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:17.088+0000 7f4f43e2f700 1 --2- 172.21.15.157:0/3324470607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f3c0fe0b0 0x7f4f3c0fe4c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:27:17.089 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:17.088+0000 7f4f43e2f700 1 -- 172.21.15.157:0/3324470607 >> 172.21.15.157:0/3324470607 conn(0x7f4f3c0f6850 msgr2=0x7f4f3c0f8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:27:17.089 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:17.088+0000 7f4f43e2f700 1 -- 172.21.15.157:0/3324470607 shutdown_connections 2024-04-22T23:27:17.089 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:17.088+0000 7f4f43e2f700 1 -- 172.21.15.157:0/3324470607 wait complete. 2024-04-22T23:27:17.089 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:17.088+0000 7f4f43e2f700 1 Processor -- start 2024-04-22T23:27:17.089 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:17.088+0000 7f4f43e2f700 1 -- start start 2024-04-22T23:27:17.090 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:17.089+0000 7f4f43e2f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f3c0fe0b0 0x7f4f3c10d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:27:17.091 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:17.089+0000 7f4f43e2f700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3c10d600 con 0x7f4f3c0fe0b0 2024-04-22T23:27:17.091 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:17.089+0000 7f4f41bcb700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f3c0fe0b0 0x7f4f3c10d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:27:17.091 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:17.089+0000 7f4f41bcb700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f3c0fe0b0 0x7f4f3c10d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.157:40994/0 (socket says 172.21.15.157:40994) 2024-04-22T23:27:17.091 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:17.089+0000 7f4f41bcb700 1 -- 172.21.15.157:0/1757702794 learned_addr learned my addr 172.21.15.157:0/1757702794 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-22T23:27:17.091 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:17.089+0000 7f4f41bcb700 1 -- 172.21.15.157:0/1757702794 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f38005040 con 0x7f4f3c0fe0b0 2024-04-22T23:27:17.091 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:17.089+0000 7f4f41bcb700 1 --2- 172.21.15.157:0/1757702794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f3c0fe0b0 0x7f4f3c10d0c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4f380096b0 tx=0x7f4f38009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:27:17.091 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:17.089+0000 7f4f327fc700 1 -- 172.21.15.157:0/1757702794 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4f3800bed0 con 0x7f4f3c0fe0b0 2024-04-22T23:27:17.091 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:17.090+0000 7f4f327fc700 1 -- 172.21.15.157:0/1757702794 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4f3800b5c0 con 0x7f4f3c0fe0b0 2024-04-22T23:27:17.091 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:17.090+0000 7f4f43e2f700 1 -- 172.21.15.157:0/1757702794 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f3c10d800 con 0x7f4f3c0fe0b0 2024-04-22T23:27:17.091 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:17.090+0000 7f4f327fc700 1 -- 172.21.15.157:0/1757702794 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4f3801c5b0 con 0x7f4f3c0fe0b0 2024-04-22T23:27:17.091 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:17.090+0000 7f4f43e2f700 1 -- 172.21.15.157:0/1757702794 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f3c10e370 con 0x7f4f3c0fe0b0 2024-04-22T23:27:17.091 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:17.090+0000 7f4f327fc700 1 -- 172.21.15.157:0/1757702794 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7f4f38013420 con 0x7f4f3c0fe0b0 2024-04-22T23:27:17.092 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:17.090+0000 7f4f43e2f700 1 -- 172.21.15.157:0/1757702794 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f3c102f80 con 0x7f4f3c0fe0b0 2024-04-22T23:27:17.092 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:17.091+0000 7f4f327fc700 1 --2- 172.21.15.157:0/1757702794 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f4f28036880 0x7f4f28038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:27:17.092 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:17.091+0000 7f4f327fc700 1 -- 172.21.15.157:0/1757702794 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f4f38049b10 con 0x7f4f3c0fe0b0 2024-04-22T23:27:17.092 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:17.091+0000 7f4f413ca700 1 --2- 172.21.15.157:0/1757702794 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f4f28036880 0x7f4f28038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:27:17.092 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:17.091+0000 7f4f413ca700 1 --2- 172.21.15.157:0/1757702794 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f4f28036880 0x7f4f28038d30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c009990 tx=0x7f4f2c006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:27:17.100 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:17.100+0000 7f4f327fc700 1 -- 172.21.15.157:0/1757702794 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4f38025400 con 0x7f4f3c0fe0b0 2024-04-22T23:27:17.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:17 smithi107 ceph-mon[26355]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:27:17.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:17 smithi107 ceph-mon[26355]: from='mgr.14160 172.21.15.107:0/1348717222' entity='mgr.smithi107.uipxvm' 2024-04-22T23:27:17.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:17 smithi107 ceph-mon[26355]: from='mgr.14160 172.21.15.107:0/1348717222' entity='mgr.smithi107.uipxvm' 2024-04-22T23:27:17.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:17 smithi107 ceph-mon[26355]: from='mgr.14160 172.21.15.107:0/1348717222' entity='mgr.smithi107.uipxvm' 2024-04-22T23:27:17.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:17 smithi107 ceph-mon[26355]: from='mgr.14160 172.21.15.107:0/1348717222' entity='mgr.smithi107.uipxvm' 2024-04-22T23:27:17.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:17 smithi107 ceph-mon[26355]: from='mgr.14160 172.21.15.107:0/1348717222' entity='mgr.smithi107.uipxvm' 2024-04-22T23:27:17.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:17 smithi107 ceph-mon[26355]: from='mgr.14160 172.21.15.107:0/1348717222' entity='mgr.smithi107.uipxvm' 2024-04-22T23:27:17.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:17 smithi107 ceph-mon[26355]: Deploying daemon prometheus.smithi107 on smithi107 2024-04-22T23:27:17.363 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:17.363+0000 7f4f43e2f700 1 -- 172.21.15.157:0/1757702794 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f4f3c10ed80 con 0x7f4f3c0fe0b0 2024-04-22T23:27:17.363 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:17.363+0000 7f4f327fc700 1 -- 172.21.15.157:0/1757702794 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f4f38004d30 con 0x7f4f3c0fe0b0 2024-04-22T23:27:17.363 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:27:17.363 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":1,"fsid":"8252f6f2-00ff-11ef-bc93-c7b262605968","modified":"2024-04-22T23:25:22.712092Z","created":"2024-04-22T23:25:22.712092Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi107","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-22T23:27:17.365 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:17.365+0000 7f4f43e2f700 1 -- 172.21.15.157:0/1757702794 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f4f28036880 msgr2=0x7f4f28038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:27:17.365 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:17.365+0000 7f4f43e2f700 1 --2- 172.21.15.157:0/1757702794 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f4f28036880 0x7f4f28038d30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c009990 tx=0x7f4f2c006e40 comp rx=0 tx=0).stop 2024-04-22T23:27:17.365 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:17.365+0000 7f4f43e2f700 1 -- 172.21.15.157:0/1757702794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f3c0fe0b0 msgr2=0x7f4f3c10d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:27:17.365 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:17.365+0000 7f4f43e2f700 1 --2- 172.21.15.157:0/1757702794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f3c0fe0b0 0x7f4f3c10d0c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4f380096b0 tx=0x7f4f38009e30 comp rx=0 tx=0).stop 2024-04-22T23:27:17.365 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:17.365+0000 7f4f43e2f700 1 -- 172.21.15.157:0/1757702794 shutdown_connections 2024-04-22T23:27:17.366 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:17.365+0000 7f4f43e2f700 1 --2- 172.21.15.157:0/1757702794 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f4f28036880 0x7f4f28038d30 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:27:17.366 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:17.365+0000 7f4f43e2f700 1 --2- 172.21.15.157:0/1757702794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f3c0fe0b0 0x7f4f3c10d0c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:27:17.366 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:17.365+0000 7f4f43e2f700 1 -- 172.21.15.157:0/1757702794 >> 172.21.15.157:0/1757702794 conn(0x7f4f3c0f6850 msgr2=0x7f4f3c103bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:27:17.366 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:17.365+0000 7f4f43e2f700 1 -- 172.21.15.157:0/1757702794 shutdown_connections 2024-04-22T23:27:17.366 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:17.365+0000 7f4f43e2f700 1 -- 172.21.15.157:0/1757702794 wait complete. 2024-04-22T23:27:17.366 INFO:teuthology.orchestra.run.smithi157.stderr:dumped monmap epoch 1 2024-04-22T23:27:18.489 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:18 smithi107 ceph-mon[26355]: from='client.? 172.21.15.157:0/1757702794' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T23:27:18.909 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-22T23:27:18.910 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-22T23:27:19.158 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-22T23:27:19.205 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-22T23:27:19.489 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:19 smithi107 ceph-mon[26355]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:27:20.615 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:20.614+0000 7f9dd815b700 1 -- 172.21.15.157:0/2002740554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9dd00fe0b0 msgr2=0x7f9dd00fe4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:27:20.615 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:20.614+0000 7f9dd815b700 1 --2- 172.21.15.157:0/2002740554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9dd00fe0b0 0x7f9dd00fe4c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9dc00048c0 tx=0x7f9dc0009e30 comp rx=0 tx=0).stop 2024-04-22T23:27:20.615 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:20.614+0000 7f9dd815b700 1 -- 172.21.15.157:0/2002740554 shutdown_connections 2024-04-22T23:27:20.615 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:20.614+0000 7f9dd815b700 1 --2- 172.21.15.157:0/2002740554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9dd00fe0b0 0x7f9dd00fe4c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:27:20.615 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:20.614+0000 7f9dd815b700 1 -- 172.21.15.157:0/2002740554 >> 172.21.15.157:0/2002740554 conn(0x7f9dd00f6850 msgr2=0x7f9dd00f8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:27:20.615 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:20.614+0000 7f9dd815b700 1 -- 172.21.15.157:0/2002740554 shutdown_connections 2024-04-22T23:27:20.615 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:20.614+0000 7f9dd815b700 1 -- 172.21.15.157:0/2002740554 wait complete. 2024-04-22T23:27:20.615 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:20.615+0000 7f9dd815b700 1 Processor -- start 2024-04-22T23:27:20.616 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:20.615+0000 7f9dd815b700 1 -- start start 2024-04-22T23:27:20.616 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:20.615+0000 7f9dd815b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9dd00fe0b0 0x7f9dd010d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:27:20.616 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:20.615+0000 7f9dd815b700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd010d600 con 0x7f9dd00fe0b0 2024-04-22T23:27:20.617 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:20.616+0000 7f9dd5ef7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9dd00fe0b0 0x7f9dd010d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:27:20.617 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:20.616+0000 7f9dd5ef7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9dd00fe0b0 0x7f9dd010d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.157:41020/0 (socket says 172.21.15.157:41020) 2024-04-22T23:27:20.617 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:20.616+0000 7f9dd5ef7700 1 -- 172.21.15.157:0/47622868 learned_addr learned my addr 172.21.15.157:0/47622868 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-22T23:27:20.618 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:20.616+0000 7f9dd5ef7700 1 -- 172.21.15.157:0/47622868 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc0005040 con 0x7f9dd00fe0b0 2024-04-22T23:27:20.618 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:20.616+0000 7f9dd5ef7700 1 --2- 172.21.15.157:0/47622868 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9dd00fe0b0 0x7f9dd010d0c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9dc00096b0 tx=0x7f9dc0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:27:20.618 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:20.616+0000 7f9dc67fc700 1 -- 172.21.15.157:0/47622868 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9dc000bed0 con 0x7f9dd00fe0b0 2024-04-22T23:27:20.618 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:20.616+0000 7f9dc67fc700 1 -- 172.21.15.157:0/47622868 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9dc000b5c0 con 0x7f9dd00fe0b0 2024-04-22T23:27:20.618 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:20.616+0000 7f9dc67fc700 1 -- 172.21.15.157:0/47622868 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9dc001c5b0 con 0x7f9dd00fe0b0 2024-04-22T23:27:20.618 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:20.616+0000 7f9dd815b700 1 -- 172.21.15.157:0/47622868 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dd010d800 con 0x7f9dd00fe0b0 2024-04-22T23:27:20.618 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:20.616+0000 7f9dd815b700 1 -- 172.21.15.157:0/47622868 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dd010e370 con 0x7f9dd00fe0b0 2024-04-22T23:27:20.618 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:20.617+0000 7f9dc67fc700 1 -- 172.21.15.157:0/47622868 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7f9dc0013420 con 0x7f9dd00fe0b0 2024-04-22T23:27:20.618 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:20.617+0000 7f9dd815b700 1 -- 172.21.15.157:0/47622868 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dd0102f80 con 0x7f9dd00fe0b0 2024-04-22T23:27:20.618 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:20.617+0000 7f9dc67fc700 1 --2- 172.21.15.157:0/47622868 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f9dbc0368d0 0x7f9dbc038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:27:20.618 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:20.617+0000 7f9dc67fc700 1 -- 172.21.15.157:0/47622868 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f9dc0049a80 con 0x7f9dd00fe0b0 2024-04-22T23:27:20.618 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:20.618+0000 7f9dd56f6700 1 --2- 172.21.15.157:0/47622868 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f9dbc0368d0 0x7f9dbc038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:27:20.619 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:20.618+0000 7f9dd56f6700 1 --2- 172.21.15.157:0/47622868 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f9dbc0368d0 0x7f9dbc038d80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc009990 tx=0x7f9dcc006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:27:20.622 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:20.622+0000 7f9dc67fc700 1 -- 172.21.15.157:0/47622868 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f9dc0025400 con 0x7f9dd00fe0b0 2024-04-22T23:27:20.884 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:20.884+0000 7f9dd815b700 1 -- 172.21.15.157:0/47622868 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9dd010ed80 con 0x7f9dd00fe0b0 2024-04-22T23:27:20.884 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:20.884+0000 7f9dc67fc700 1 -- 172.21.15.157:0/47622868 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f9dc0004d30 con 0x7f9dd00fe0b0 2024-04-22T23:27:20.884 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:27:20.884 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":1,"fsid":"8252f6f2-00ff-11ef-bc93-c7b262605968","modified":"2024-04-22T23:25:22.712092Z","created":"2024-04-22T23:25:22.712092Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi107","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-22T23:27:20.886 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:20.886+0000 7f9dd815b700 1 -- 172.21.15.157:0/47622868 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f9dbc0368d0 msgr2=0x7f9dbc038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:27:20.886 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:20.886+0000 7f9dd815b700 1 --2- 172.21.15.157:0/47622868 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f9dbc0368d0 0x7f9dbc038d80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc009990 tx=0x7f9dcc006e40 comp rx=0 tx=0).stop 2024-04-22T23:27:20.886 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:20.886+0000 7f9dd815b700 1 -- 172.21.15.157:0/47622868 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9dd00fe0b0 msgr2=0x7f9dd010d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:27:20.887 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:20.886+0000 7f9dd815b700 1 --2- 172.21.15.157:0/47622868 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9dd00fe0b0 0x7f9dd010d0c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9dc00096b0 tx=0x7f9dc0009e30 comp rx=0 tx=0).stop 2024-04-22T23:27:20.887 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:20.886+0000 7f9dd815b700 1 -- 172.21.15.157:0/47622868 shutdown_connections 2024-04-22T23:27:20.887 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:20.886+0000 7f9dd815b700 1 --2- 172.21.15.157:0/47622868 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f9dbc0368d0 0x7f9dbc038d80 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:27:20.887 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:20.886+0000 7f9dd815b700 1 --2- 172.21.15.157:0/47622868 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9dd00fe0b0 0x7f9dd010d0c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:27:20.887 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:20.886+0000 7f9dd815b700 1 -- 172.21.15.157:0/47622868 >> 172.21.15.157:0/47622868 conn(0x7f9dd00f6850 msgr2=0x7f9dd0103bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:27:20.887 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:20.886+0000 7f9dd815b700 1 -- 172.21.15.157:0/47622868 shutdown_connections 2024-04-22T23:27:20.887 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:20.886+0000 7f9dd815b700 1 -- 172.21.15.157:0/47622868 wait complete. 2024-04-22T23:27:20.887 INFO:teuthology.orchestra.run.smithi157.stderr:dumped monmap epoch 1 2024-04-22T23:27:21.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:20 smithi107 ceph-mon[26355]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:27:21.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:20 smithi107 ceph-mon[26355]: from='mgr.14160 172.21.15.107:0/1348717222' entity='mgr.smithi107.uipxvm' 2024-04-22T23:27:22.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:21 smithi107 ceph-mon[26355]: from='client.? 172.21.15.157:0/47622868' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T23:27:22.461 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-22T23:27:22.462 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-22T23:27:22.714 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-22T23:27:22.762 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-22T23:27:23.130 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:22 smithi107 ceph-mon[26355]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:27:24.173 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:24.172+0000 7f0af8d07700 1 -- 172.21.15.157:0/3699760900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0af40fb960 msgr2=0x7f0af40fbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:27:24.173 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:24.172+0000 7f0af8d07700 1 --2- 172.21.15.157:0/3699760900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0af40fb960 0x7f0af40fbd70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0ae40048c0 tx=0x7f0ae4009e30 comp rx=0 tx=0).stop 2024-04-22T23:27:24.173 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:24.172+0000 7f0af8d07700 1 -- 172.21.15.157:0/3699760900 shutdown_connections 2024-04-22T23:27:24.173 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:24.172+0000 7f0af8d07700 1 --2- 172.21.15.157:0/3699760900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0af40fb960 0x7f0af40fbd70 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:27:24.173 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:24.172+0000 7f0af8d07700 1 -- 172.21.15.157:0/3699760900 >> 172.21.15.157:0/3699760900 conn(0x7f0af40f70c0 msgr2=0x7f0af40f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:27:24.173 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:24.172+0000 7f0af8d07700 1 -- 172.21.15.157:0/3699760900 shutdown_connections 2024-04-22T23:27:24.174 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:24.172+0000 7f0af8d07700 1 -- 172.21.15.157:0/3699760900 wait complete. 2024-04-22T23:27:24.174 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:24.173+0000 7f0af8d07700 1 Processor -- start 2024-04-22T23:27:24.174 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:24.173+0000 7f0af8d07700 1 -- start start 2024-04-22T23:27:24.174 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:24.173+0000 7f0af8d07700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0af40fb960 0x7f0af410d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:27:24.174 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:24.173+0000 7f0af8d07700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af410d600 con 0x7f0af40fb960 2024-04-22T23:27:24.174 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:24.173+0000 7f0af259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0af40fb960 0x7f0af410d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:27:24.174 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:24.173+0000 7f0af259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0af40fb960 0x7f0af410d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.157:55982/0 (socket says 172.21.15.157:55982) 2024-04-22T23:27:24.174 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:24.173+0000 7f0af259c700 1 -- 172.21.15.157:0/1861148537 learned_addr learned my addr 172.21.15.157:0/1861148537 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-22T23:27:24.174 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:24.173+0000 7f0af259c700 1 -- 172.21.15.157:0/1861148537 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae4005040 con 0x7f0af40fb960 2024-04-22T23:27:24.174 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:24.173+0000 7f0af259c700 1 --2- 172.21.15.157:0/1861148537 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0af40fb960 0x7f0af410d0c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4009680 tx=0x7f0ae4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:27:24.175 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:24.174+0000 7f0ae2ffd700 1 -- 172.21.15.157:0/1861148537 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0ae400b5c0 con 0x7f0af40fb960 2024-04-22T23:27:24.176 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:24.174+0000 7f0af8d07700 1 -- 172.21.15.157:0/1861148537 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0af410d800 con 0x7f0af40fb960 2024-04-22T23:27:24.176 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:24.174+0000 7f0ae2ffd700 1 -- 172.21.15.157:0/1861148537 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0ae400be70 con 0x7f0af40fb960 2024-04-22T23:27:24.176 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:24.174+0000 7f0ae2ffd700 1 -- 172.21.15.157:0/1861148537 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0ae40135a0 con 0x7f0af40fb960 2024-04-22T23:27:24.176 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:24.174+0000 7f0af8d07700 1 -- 172.21.15.157:0/1861148537 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0af410e370 con 0x7f0af40fb960 2024-04-22T23:27:24.176 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:24.174+0000 7f0ae2ffd700 1 -- 172.21.15.157:0/1861148537 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7f0ae401c430 con 0x7f0af40fb960 2024-04-22T23:27:24.176 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:24.175+0000 7f0af8d07700 1 -- 172.21.15.157:0/1861148537 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0af4101e70 con 0x7f0af40fb960 2024-04-22T23:27:24.176 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:24.175+0000 7f0ae2ffd700 1 --2- 172.21.15.157:0/1861148537 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f0adc036880 0x7f0adc038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:27:24.176 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:24.175+0000 7f0ae2ffd700 1 -- 172.21.15.157:0/1861148537 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f0ae4049460 con 0x7f0af40fb960 2024-04-22T23:27:24.176 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:24.175+0000 7f0af1d9b700 1 --2- 172.21.15.157:0/1861148537 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f0adc036880 0x7f0adc038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:27:24.176 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:24.175+0000 7f0af1d9b700 1 --2- 172.21.15.157:0/1861148537 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f0adc036880 0x7f0adc038d30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8009990 tx=0x7f0ae8006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:27:24.179 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:24.180+0000 7f0ae2ffd700 1 -- 172.21.15.157:0/1861148537 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0ae401c7d0 con 0x7f0af40fb960 2024-04-22T23:27:24.439 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:24.439+0000 7f0af8d07700 1 -- 172.21.15.157:0/1861148537 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0af410e5b0 con 0x7f0af40fb960 2024-04-22T23:27:24.440 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:24.440+0000 7f0ae2ffd700 1 -- 172.21.15.157:0/1861148537 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f0ae4021070 con 0x7f0af40fb960 2024-04-22T23:27:24.440 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:27:24.440 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":1,"fsid":"8252f6f2-00ff-11ef-bc93-c7b262605968","modified":"2024-04-22T23:25:22.712092Z","created":"2024-04-22T23:25:22.712092Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi107","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-22T23:27:24.442 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:24.442+0000 7f0af8d07700 1 -- 172.21.15.157:0/1861148537 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f0adc036880 msgr2=0x7f0adc038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:27:24.442 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:24.442+0000 7f0af8d07700 1 --2- 172.21.15.157:0/1861148537 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f0adc036880 0x7f0adc038d30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8009990 tx=0x7f0ae8006e40 comp rx=0 tx=0).stop 2024-04-22T23:27:24.442 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:24.442+0000 7f0af8d07700 1 -- 172.21.15.157:0/1861148537 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0af40fb960 msgr2=0x7f0af410d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:27:24.442 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:24.442+0000 7f0af8d07700 1 --2- 172.21.15.157:0/1861148537 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0af40fb960 0x7f0af410d0c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4009680 tx=0x7f0ae4009e30 comp rx=0 tx=0).stop 2024-04-22T23:27:24.442 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:24.442+0000 7f0af8d07700 1 -- 172.21.15.157:0/1861148537 shutdown_connections 2024-04-22T23:27:24.442 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:24.442+0000 7f0af8d07700 1 --2- 172.21.15.157:0/1861148537 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f0adc036880 0x7f0adc038d30 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:27:24.442 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:24.442+0000 7f0af8d07700 1 --2- 172.21.15.157:0/1861148537 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0af40fb960 0x7f0af410d0c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:27:24.442 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:24.442+0000 7f0af8d07700 1 -- 172.21.15.157:0/1861148537 >> 172.21.15.157:0/1861148537 conn(0x7f0af40f70c0 msgr2=0x7f0af4065d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:27:24.443 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:24.442+0000 7f0af8d07700 1 -- 172.21.15.157:0/1861148537 shutdown_connections 2024-04-22T23:27:24.443 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:24.442+0000 7f0af8d07700 1 -- 172.21.15.157:0/1861148537 wait complete. 2024-04-22T23:27:24.443 INFO:teuthology.orchestra.run.smithi157.stderr:dumped monmap epoch 1 2024-04-22T23:27:25.133 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:24 smithi107 ceph-mon[26355]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:27:25.133 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:24 smithi107 ceph-mon[26355]: from='client.? 172.21.15.157:0/1861148537' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T23:27:25.967 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-22T23:27:25.967 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-22T23:27:26.228 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-22T23:27:26.274 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-22T23:27:27.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:26 smithi107 ceph-mon[26355]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:27:27.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:26 smithi107 ceph-mon[26355]: from='mgr.14160 172.21.15.107:0/1348717222' entity='mgr.smithi107.uipxvm' 2024-04-22T23:27:27.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:26 smithi107 ceph-mon[26355]: from='mgr.14160 172.21.15.107:0/1348717222' entity='mgr.smithi107.uipxvm' 2024-04-22T23:27:27.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:26 smithi107 ceph-mon[26355]: from='mgr.14160 172.21.15.107:0/1348717222' entity='mgr.smithi107.uipxvm' 2024-04-22T23:27:27.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:26 smithi107 ceph-mon[26355]: from='mgr.14160 172.21.15.107:0/1348717222' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-04-22T23:27:27.616 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:27.615+0000 7f207c5ee700 1 -- 172.21.15.157:0/1426876618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20740fca60 msgr2=0x7f20740fce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:27:27.616 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:27.615+0000 7f207c5ee700 1 --2- 172.21.15.157:0/1426876618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20740fca60 0x7f20740fce70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f20700048c0 tx=0x7f2070009e30 comp rx=0 tx=0).stop 2024-04-22T23:27:27.616 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:27.615+0000 7f207c5ee700 1 -- 172.21.15.157:0/1426876618 shutdown_connections 2024-04-22T23:27:27.616 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:27.615+0000 7f207c5ee700 1 --2- 172.21.15.157:0/1426876618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20740fca60 0x7f20740fce70 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:27:27.616 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:27.615+0000 7f207c5ee700 1 -- 172.21.15.157:0/1426876618 >> 172.21.15.157:0/1426876618 conn(0x7f20740f81c0 msgr2=0x7f20740fa5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:27:27.616 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:27.615+0000 7f207c5ee700 1 -- 172.21.15.157:0/1426876618 shutdown_connections 2024-04-22T23:27:27.616 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:27.615+0000 7f207c5ee700 1 -- 172.21.15.157:0/1426876618 wait complete. 2024-04-22T23:27:27.616 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:27.615+0000 7f207c5ee700 1 Processor -- start 2024-04-22T23:27:27.616 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:27.615+0000 7f207c5ee700 1 -- start start 2024-04-22T23:27:27.616 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:27.615+0000 7f207c5ee700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20740fca60 0x7f207410d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:27:27.617 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:27.615+0000 7f207c5ee700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207410d5f0 con 0x7f20740fca60 2024-04-22T23:27:27.617 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:27.616+0000 7f207a38a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20740fca60 0x7f207410d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:27:27.617 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:27.616+0000 7f207a38a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20740fca60 0x7f207410d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.157:55992/0 (socket says 172.21.15.157:55992) 2024-04-22T23:27:27.617 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:27.616+0000 7f207a38a700 1 -- 172.21.15.157:0/3939225078 learned_addr learned my addr 172.21.15.157:0/3939225078 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-22T23:27:27.617 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:27.616+0000 7f207a38a700 1 -- 172.21.15.157:0/3939225078 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2070005040 con 0x7f20740fca60 2024-04-22T23:27:27.617 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:27.616+0000 7f207a38a700 1 --2- 172.21.15.157:0/3939225078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20740fca60 0x7f207410d0b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f20700096b0 tx=0x7f2070009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:27:27.617 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:27.616+0000 7f206affd700 1 -- 172.21.15.157:0/3939225078 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f207000bed0 con 0x7f20740fca60 2024-04-22T23:27:27.617 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:27.616+0000 7f206affd700 1 -- 172.21.15.157:0/3939225078 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f207000b5c0 con 0x7f20740fca60 2024-04-22T23:27:27.617 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:27.616+0000 7f207c5ee700 1 -- 172.21.15.157:0/3939225078 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f207410d7f0 con 0x7f20740fca60 2024-04-22T23:27:27.617 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:27.616+0000 7f206affd700 1 -- 172.21.15.157:0/3939225078 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f207001c5b0 con 0x7f20740fca60 2024-04-22T23:27:27.617 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:27.616+0000 7f207c5ee700 1 -- 172.21.15.157:0/3939225078 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f207410e360 con 0x7f20740fca60 2024-04-22T23:27:27.618 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:27.617+0000 7f206affd700 1 -- 172.21.15.157:0/3939225078 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 14) v1 ==== 43447+0+0 (secure 0 0 0) 0x7f2070013420 con 0x7f20740fca60 2024-04-22T23:27:27.618 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:27.617+0000 7f207c5ee700 1 -- 172.21.15.157:0/3939225078 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2074102f70 con 0x7f20740fca60 2024-04-22T23:27:27.618 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:27.617+0000 7f206affd700 1 --2- 172.21.15.157:0/3939225078 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f20600368d0 0x7f2060038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:27:27.619 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:27.617+0000 7f206affd700 1 -- 172.21.15.157:0/3939225078 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f2070049ab0 con 0x7f20740fca60 2024-04-22T23:27:27.619 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:27.617+0000 7f2079b89700 1 -- 172.21.15.157:0/3939225078 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f20600368d0 msgr2=0x7f2060038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/3427816626 2024-04-22T23:27:27.619 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:27.617+0000 7f2079b89700 1 --2- 172.21.15.157:0/3939225078 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f20600368d0 0x7f2060038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T23:27:27.622 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:27.622+0000 7f206affd700 1 -- 172.21.15.157:0/3939225078 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f2070025400 con 0x7f20740fca60 2024-04-22T23:27:27.818 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:27.818+0000 7f2079b89700 1 -- 172.21.15.157:0/3939225078 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f20600368d0 msgr2=0x7f2060038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/3427816626 2024-04-22T23:27:27.818 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:27.818+0000 7f2079b89700 1 --2- 172.21.15.157:0/3939225078 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f20600368d0 0x7f2060038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T23:27:27.883 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:27.883+0000 7f207c5ee700 1 -- 172.21.15.157:0/3939225078 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f207410ed70 con 0x7f20740fca60 2024-04-22T23:27:27.883 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:27.884+0000 7f206affd700 1 -- 172.21.15.157:0/3939225078 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f2070004d30 con 0x7f20740fca60 2024-04-22T23:27:27.884 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:27:27.884 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":1,"fsid":"8252f6f2-00ff-11ef-bc93-c7b262605968","modified":"2024-04-22T23:25:22.712092Z","created":"2024-04-22T23:25:22.712092Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi107","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-22T23:27:27.885 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:27.886+0000 7f207c5ee700 1 -- 172.21.15.157:0/3939225078 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f20600368d0 msgr2=0x7f2060038d80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T23:27:27.885 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:27.886+0000 7f207c5ee700 1 --2- 172.21.15.157:0/3939225078 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f20600368d0 0x7f2060038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:27:27.886 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:27.886+0000 7f207c5ee700 1 -- 172.21.15.157:0/3939225078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20740fca60 msgr2=0x7f207410d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:27:27.886 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:27.886+0000 7f207c5ee700 1 --2- 172.21.15.157:0/3939225078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20740fca60 0x7f207410d0b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f20700096b0 tx=0x7f2070009e30 comp rx=0 tx=0).stop 2024-04-22T23:27:27.886 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:27.886+0000 7f207c5ee700 1 -- 172.21.15.157:0/3939225078 shutdown_connections 2024-04-22T23:27:27.886 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:27.886+0000 7f207c5ee700 1 --2- 172.21.15.157:0/3939225078 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7f20600368d0 0x7f2060038d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:27:27.886 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:27.886+0000 7f207c5ee700 1 --2- 172.21.15.157:0/3939225078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20740fca60 0x7f207410d0b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:27:27.886 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:27.886+0000 7f207c5ee700 1 -- 172.21.15.157:0/3939225078 >> 172.21.15.157:0/3939225078 conn(0x7f20740f81c0 msgr2=0x7f2074103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:27:27.886 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:27.886+0000 7f207c5ee700 1 -- 172.21.15.157:0/3939225078 shutdown_connections 2024-04-22T23:27:27.886 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:27.886+0000 7f207c5ee700 1 -- 172.21.15.157:0/3939225078 wait complete. 2024-04-22T23:27:27.887 INFO:teuthology.orchestra.run.smithi157.stderr:dumped monmap epoch 1 2024-04-22T23:27:28.717 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:28 smithi107 ceph-mon[26355]: from='mgr.14160 172.21.15.107:0/1348717222' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-04-22T23:27:28.717 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:28 smithi107 ceph-mon[26355]: mgrmap e14: smithi107.uipxvm(active, since 57s) 2024-04-22T23:27:28.717 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:28 smithi107 ceph-mon[26355]: from='client.? 172.21.15.157:0/3939225078' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T23:27:29.503 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-22T23:27:29.504 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-22T23:27:29.755 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-22T23:27:29.803 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-22T23:27:31.249 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:31.248+0000 7ff0ff708700 1 -- 172.21.15.157:0/649661216 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0f80fe0b0 msgr2=0x7ff0f80fe4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:27:31.249 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:31.248+0000 7ff0ff708700 1 --2- 172.21.15.157:0/649661216 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0f80fe0b0 0x7ff0f80fe4c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff0e80048c0 tx=0x7ff0e8009e30 comp rx=0 tx=0).stop 2024-04-22T23:27:31.250 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:31.249+0000 7ff0ff708700 1 -- 172.21.15.157:0/649661216 shutdown_connections 2024-04-22T23:27:31.250 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:31.249+0000 7ff0ff708700 1 --2- 172.21.15.157:0/649661216 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0f80fe0b0 0x7ff0f80fe4c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:27:31.250 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:31.249+0000 7ff0ff708700 1 -- 172.21.15.157:0/649661216 >> 172.21.15.157:0/649661216 conn(0x7ff0f80f6850 msgr2=0x7ff0f80f8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:27:31.250 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:31.249+0000 7ff0ff708700 1 -- 172.21.15.157:0/649661216 shutdown_connections 2024-04-22T23:27:31.250 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:31.249+0000 7ff0ff708700 1 -- 172.21.15.157:0/649661216 wait complete. 2024-04-22T23:27:31.250 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:31.249+0000 7ff0ff708700 1 Processor -- start 2024-04-22T23:27:31.250 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:31.249+0000 7ff0ff708700 1 -- start start 2024-04-22T23:27:31.250 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:31.249+0000 7ff0ff708700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0f80fe0b0 0x7ff0f810d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:27:31.250 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:31.249+0000 7ff0ff708700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f810d630 con 0x7ff0f80fe0b0 2024-04-22T23:27:31.252 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:31.249+0000 7ff0fd4a4700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0f80fe0b0 0x7ff0f810d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:27:31.252 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:31.249+0000 7ff0fd4a4700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0f80fe0b0 0x7ff0f810d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.157:56014/0 (socket says 172.21.15.157:56014) 2024-04-22T23:27:31.252 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:31.249+0000 7ff0fd4a4700 1 -- 172.21.15.157:0/3602815613 learned_addr learned my addr 172.21.15.157:0/3602815613 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-22T23:27:31.252 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:31.250+0000 7ff0fd4a4700 1 -- 172.21.15.157:0/3602815613 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e8005040 con 0x7ff0f80fe0b0 2024-04-22T23:27:31.252 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:31.250+0000 7ff0fd4a4700 1 --2- 172.21.15.157:0/3602815613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0f80fe0b0 0x7ff0f810d0f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8009680 tx=0x7ff0e8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:27:31.252 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:31.250+0000 7ff0edffb700 1 -- 172.21.15.157:0/3602815613 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff0e800b5c0 con 0x7ff0f80fe0b0 2024-04-22T23:27:31.252 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:31.250+0000 7ff0edffb700 1 -- 172.21.15.157:0/3602815613 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff0e800be70 con 0x7ff0f80fe0b0 2024-04-22T23:27:31.252 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:31.250+0000 7ff0ff708700 1 -- 172.21.15.157:0/3602815613 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0f810d830 con 0x7ff0f80fe0b0 2024-04-22T23:27:31.252 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:31.250+0000 7ff0edffb700 1 -- 172.21.15.157:0/3602815613 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff0e80135a0 con 0x7ff0f80fe0b0 2024-04-22T23:27:31.252 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:31.250+0000 7ff0ff708700 1 -- 172.21.15.157:0/3602815613 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0f810e3a0 con 0x7ff0f80fe0b0 2024-04-22T23:27:31.252 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:31.251+0000 7ff0edffb700 1 -- 172.21.15.157:0/3602815613 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 14) v1 ==== 43447+0+0 (secure 0 0 0) 0x7ff0e801c430 con 0x7ff0f80fe0b0 2024-04-22T23:27:31.252 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:31.251+0000 7ff0ff708700 1 -- 172.21.15.157:0/3602815613 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0f8102f80 con 0x7ff0f80fe0b0 2024-04-22T23:27:31.253 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:31.251+0000 7ff0edffb700 1 --2- 172.21.15.157:0/3602815613 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7ff0e40368d0 0x7ff0e4038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:27:31.253 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:31.251+0000 7ff0edffb700 1 -- 172.21.15.157:0/3602815613 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7ff0e80494d0 con 0x7ff0f80fe0b0 2024-04-22T23:27:31.253 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:31.251+0000 7ff0fcca3700 1 -- 172.21.15.157:0/3602815613 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7ff0e40368d0 msgr2=0x7ff0e4038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/3427816626 2024-04-22T23:27:31.253 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:31.251+0000 7ff0fcca3700 1 --2- 172.21.15.157:0/3602815613 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7ff0e40368d0 0x7ff0e4038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T23:27:31.256 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:31.256+0000 7ff0edffb700 1 -- 172.21.15.157:0/3602815613 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ff0e8004d30 con 0x7ff0f80fe0b0 2024-04-22T23:27:31.452 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:31.452+0000 7ff0fcca3700 1 -- 172.21.15.157:0/3602815613 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7ff0e40368d0 msgr2=0x7ff0e4038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/3427816626 2024-04-22T23:27:31.452 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:31.452+0000 7ff0fcca3700 1 --2- 172.21.15.157:0/3602815613 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7ff0e40368d0 0x7ff0e4038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T23:27:31.519 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:31.519+0000 7ff0ff708700 1 -- 172.21.15.157:0/3602815613 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff0f810edb0 con 0x7ff0f80fe0b0 2024-04-22T23:27:31.520 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:31.520+0000 7ff0edffb700 1 -- 172.21.15.157:0/3602815613 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7ff0e8021070 con 0x7ff0f80fe0b0 2024-04-22T23:27:31.520 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:27:31.520 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":1,"fsid":"8252f6f2-00ff-11ef-bc93-c7b262605968","modified":"2024-04-22T23:25:22.712092Z","created":"2024-04-22T23:25:22.712092Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi107","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-22T23:27:31.521 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:31.522+0000 7ff0ff708700 1 -- 172.21.15.157:0/3602815613 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7ff0e40368d0 msgr2=0x7ff0e4038d80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T23:27:31.522 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:31.522+0000 7ff0ff708700 1 --2- 172.21.15.157:0/3602815613 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7ff0e40368d0 0x7ff0e4038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:27:31.522 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:31.522+0000 7ff0ff708700 1 -- 172.21.15.157:0/3602815613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0f80fe0b0 msgr2=0x7ff0f810d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:27:31.522 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:31.522+0000 7ff0ff708700 1 --2- 172.21.15.157:0/3602815613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0f80fe0b0 0x7ff0f810d0f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8009680 tx=0x7ff0e8009e30 comp rx=0 tx=0).stop 2024-04-22T23:27:31.522 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:31.522+0000 7ff0ff708700 1 -- 172.21.15.157:0/3602815613 shutdown_connections 2024-04-22T23:27:31.522 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:31.522+0000 7ff0ff708700 1 --2- 172.21.15.157:0/3602815613 >> [v2:172.21.15.107:6800/3427816626,v1:172.21.15.107:6801/3427816626] conn(0x7ff0e40368d0 0x7ff0e4038d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:27:31.522 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:31.522+0000 7ff0ff708700 1 --2- 172.21.15.157:0/3602815613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0f80fe0b0 0x7ff0f810d0f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:27:31.522 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:31.522+0000 7ff0ff708700 1 -- 172.21.15.157:0/3602815613 >> 172.21.15.157:0/3602815613 conn(0x7ff0f80f6850 msgr2=0x7ff0f80f7500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:27:31.522 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:31.522+0000 7ff0ff708700 1 -- 172.21.15.157:0/3602815613 shutdown_connections 2024-04-22T23:27:31.522 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:31.522+0000 7ff0ff708700 1 -- 172.21.15.157:0/3602815613 wait complete. 2024-04-22T23:27:31.523 INFO:teuthology.orchestra.run.smithi157.stderr:dumped monmap epoch 1 2024-04-22T23:27:31.720 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:31 smithi107 ceph-mon[26355]: from='client.? 172.21.15.157:0/3602815613' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T23:27:33.131 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-22T23:27:33.132 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-22T23:27:33.381 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-22T23:27:33.429 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-22T23:27:33.989 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:33 smithi107 ceph-mon[26355]: Active manager daemon smithi107.uipxvm restarted 2024-04-22T23:27:33.989 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:33 smithi107 ceph-mon[26355]: Activating manager daemon smithi107.uipxvm 2024-04-22T23:27:33.989 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:33 smithi107 ceph-mon[26355]: osdmap e5: 0 total, 0 up, 0 in 2024-04-22T23:27:33.989 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:33 smithi107 ceph-mon[26355]: mgrmap e15: smithi107.uipxvm(active, starting, since 0.00536404s) 2024-04-22T23:27:33.989 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:33 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-04-22T23:27:33.989 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:33 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mgr metadata", "who": "smithi107.uipxvm", "id": "smithi107.uipxvm"}]: dispatch 2024-04-22T23:27:33.989 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:33 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-22T23:27:33.990 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:33 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-22T23:27:33.990 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:33 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-22T23:27:33.990 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:33 smithi107 ceph-mon[26355]: Manager daemon smithi107.uipxvm is now available 2024-04-22T23:27:33.990 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:33 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:27:33.990 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:33 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:27:33.990 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:33 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:27:33.990 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:33 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:27:34.964 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:34.963+0000 7f3fbc008700 1 -- 172.21.15.157:0/619865333 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fb40665f0 msgr2=0x7f3fb40ff660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:27:34.964 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:34.963+0000 7f3fbc008700 1 --2- 172.21.15.157:0/619865333 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fb40665f0 0x7f3fb40ff660 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3fb00048c0 tx=0x7f3fb0009e30 comp rx=0 tx=0).stop 2024-04-22T23:27:34.964 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:34.963+0000 7f3fbc008700 1 -- 172.21.15.157:0/619865333 shutdown_connections 2024-04-22T23:27:34.964 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:34.963+0000 7f3fbc008700 1 --2- 172.21.15.157:0/619865333 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fb40665f0 0x7f3fb40ff660 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:27:34.964 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:34.963+0000 7f3fbc008700 1 -- 172.21.15.157:0/619865333 >> 172.21.15.157:0/619865333 conn(0x7f3fb40f70d0 msgr2=0x7f3fb40f9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:27:34.965 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:34.963+0000 7f3fbc008700 1 -- 172.21.15.157:0/619865333 shutdown_connections 2024-04-22T23:27:34.965 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:34.963+0000 7f3fbc008700 1 -- 172.21.15.157:0/619865333 wait complete. 2024-04-22T23:27:34.965 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:34.964+0000 7f3fbc008700 1 Processor -- start 2024-04-22T23:27:34.965 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:34.964+0000 7f3fbc008700 1 -- start start 2024-04-22T23:27:34.965 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:34.964+0000 7f3fbc008700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fb40665f0 0x7f3fb40fcfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:27:34.965 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:34.964+0000 7f3fbc008700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb40fd4f0 con 0x7f3fb40665f0 2024-04-22T23:27:34.965 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:34.964+0000 7f3fb9da4700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fb40665f0 0x7f3fb40fcfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:27:34.965 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:34.964+0000 7f3fb9da4700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fb40665f0 0x7f3fb40fcfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.157:46614/0 (socket says 172.21.15.157:46614) 2024-04-22T23:27:34.965 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:34.964+0000 7f3fb9da4700 1 -- 172.21.15.157:0/2793046604 learned_addr learned my addr 172.21.15.157:0/2793046604 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-22T23:27:34.965 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:34.965+0000 7f3fb9da4700 1 -- 172.21.15.157:0/2793046604 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb0005040 con 0x7f3fb40665f0 2024-04-22T23:27:34.966 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:34.965+0000 7f3fb9da4700 1 --2- 172.21.15.157:0/2793046604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fb40665f0 0x7f3fb40fcfb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0009680 tx=0x7f3fb0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:27:34.967 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:34.965+0000 7f3faa7fc700 1 -- 172.21.15.157:0/2793046604 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3fb000bed0 con 0x7f3fb40665f0 2024-04-22T23:27:34.967 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:34.965+0000 7f3faa7fc700 1 -- 172.21.15.157:0/2793046604 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3fb000b5c0 con 0x7f3fb40665f0 2024-04-22T23:27:34.967 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:34.965+0000 7f3fbc008700 1 -- 172.21.15.157:0/2793046604 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fb40fb5c0 con 0x7f3fb40665f0 2024-04-22T23:27:34.967 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:34.965+0000 7f3faa7fc700 1 -- 172.21.15.157:0/2793046604 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3fb001c5b0 con 0x7f3fb40665f0 2024-04-22T23:27:34.967 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:34.965+0000 7f3fbc008700 1 -- 172.21.15.157:0/2793046604 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fb40fbab0 con 0x7f3fb40665f0 2024-04-22T23:27:34.967 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:34.966+0000 7f3faa7fc700 1 -- 172.21.15.157:0/2793046604 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 43297+0+0 (secure 0 0 0) 0x7f3fb001c710 con 0x7f3fb40665f0 2024-04-22T23:27:34.967 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:34.966+0000 7f3fbc008700 1 -- 172.21.15.157:0/2793046604 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fb405f5f0 con 0x7f3fb40665f0 2024-04-22T23:27:34.967 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:34.966+0000 7f3faa7fc700 1 --2- 172.21.15.157:0/2793046604 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f3fa0036750 0x7f3fa0038c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:27:34.967 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:34.966+0000 7f3faa7fc700 1 -- 172.21.15.157:0/2793046604 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f3fb0049af0 con 0x7f3fb40665f0 2024-04-22T23:27:34.967 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:34.966+0000 7f3fb95a3700 1 --2- 172.21.15.157:0/2793046604 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f3fa0036750 0x7f3fa0038c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:27:34.967 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:34.967+0000 7f3fb95a3700 1 --2- 172.21.15.157:0/2793046604 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f3fa0036750 0x7f3fa0038c00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4009990 tx=0x7f3fa4006e40 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:27:34.971 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:34.971+0000 7f3faa7fc700 1 -- 172.21.15.157:0/2793046604 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f3fb0025400 con 0x7f3fb40665f0 2024-04-22T23:27:35.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:34 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi107.uipxvm/mirror_snapshot_schedule"}]: dispatch 2024-04-22T23:27:35.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:34 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi107.uipxvm/trash_purge_schedule"}]: dispatch 2024-04-22T23:27:35.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:34 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:27:35.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:34 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:27:35.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:34 smithi107 ceph-mon[26355]: mgrmap e16: smithi107.uipxvm(active, since 1.00538s) 2024-04-22T23:27:35.247 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:35.248+0000 7f3fbc008700 1 -- 172.21.15.157:0/2793046604 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f3fb4106440 con 0x7f3fb40665f0 2024-04-22T23:27:35.248 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:35.248+0000 7f3faa7fc700 1 -- 172.21.15.157:0/2793046604 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f3fb0021e60 con 0x7f3fb40665f0 2024-04-22T23:27:35.248 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:27:35.248 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":1,"fsid":"8252f6f2-00ff-11ef-bc93-c7b262605968","modified":"2024-04-22T23:25:22.712092Z","created":"2024-04-22T23:25:22.712092Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi107","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-22T23:27:35.250 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:35.250+0000 7f3fbc008700 1 -- 172.21.15.157:0/2793046604 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f3fa0036750 msgr2=0x7f3fa0038c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:27:35.250 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:35.250+0000 7f3fbc008700 1 --2- 172.21.15.157:0/2793046604 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f3fa0036750 0x7f3fa0038c00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4009990 tx=0x7f3fa4006e40 comp rx=0 tx=0).stop 2024-04-22T23:27:35.250 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:35.250+0000 7f3fbc008700 1 -- 172.21.15.157:0/2793046604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fb40665f0 msgr2=0x7f3fb40fcfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:27:35.250 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:35.250+0000 7f3fbc008700 1 --2- 172.21.15.157:0/2793046604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fb40665f0 0x7f3fb40fcfb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0009680 tx=0x7f3fb0009e30 comp rx=0 tx=0).stop 2024-04-22T23:27:35.250 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:35.250+0000 7f3fbc008700 1 -- 172.21.15.157:0/2793046604 shutdown_connections 2024-04-22T23:27:35.251 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:35.250+0000 7f3fbc008700 1 --2- 172.21.15.157:0/2793046604 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f3fa0036750 0x7f3fa0038c00 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:27:35.251 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:35.250+0000 7f3fbc008700 1 --2- 172.21.15.157:0/2793046604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fb40665f0 0x7f3fb40fcfb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:27:35.251 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:35.250+0000 7f3fbc008700 1 -- 172.21.15.157:0/2793046604 >> 172.21.15.157:0/2793046604 conn(0x7f3fb40f70d0 msgr2=0x7f3fb4103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:27:35.251 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:35.250+0000 7f3fbc008700 1 -- 172.21.15.157:0/2793046604 shutdown_connections 2024-04-22T23:27:35.251 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:35.251+0000 7f3fbc008700 1 -- 172.21.15.157:0/2793046604 wait complete. 2024-04-22T23:27:35.251 INFO:teuthology.orchestra.run.smithi157.stderr:dumped monmap epoch 1 2024-04-22T23:27:36.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:36 smithi107 ceph-mon[26355]: [22/Apr/2024:23:27:34] ENGINE Bus STARTING 2024-04-22T23:27:36.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:36 smithi107 ceph-mon[26355]: [22/Apr/2024:23:27:34] ENGINE Serving on https://172.21.15.107:7150 2024-04-22T23:27:36.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:36 smithi107 ceph-mon[26355]: [22/Apr/2024:23:27:34] ENGINE Bus STARTED 2024-04-22T23:27:36.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:36 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:27:36.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:36 smithi107 ceph-mon[26355]: from='client.? 172.21.15.157:0/2793046604' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T23:27:36.784 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-22T23:27:36.785 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-22T23:27:37.214 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-22T23:27:37.402 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:37 smithi107 ceph-mon[26355]: mgrmap e17: smithi107.uipxvm(active, since 2s) 2024-04-22T23:27:37.402 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:37 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:27:37.627 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-22T23:27:39.159 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:39.157+0000 7f2948afa700 1 -- 172.21.15.157:0/3890330282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29440fb970 msgr2=0x7f29440fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:27:39.159 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:39.157+0000 7f2948afa700 1 --2- 172.21.15.157:0/3890330282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29440fb970 0x7f29440fbd80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f29380048c0 tx=0x7f2938009e30 comp rx=0 tx=0).stop 2024-04-22T23:27:39.159 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:39.157+0000 7f2948afa700 1 -- 172.21.15.157:0/3890330282 shutdown_connections 2024-04-22T23:27:39.159 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:39.157+0000 7f2948afa700 1 --2- 172.21.15.157:0/3890330282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29440fb970 0x7f29440fbd80 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:27:39.159 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:39.158+0000 7f2948afa700 1 -- 172.21.15.157:0/3890330282 >> 172.21.15.157:0/3890330282 conn(0x7f29440f70d0 msgr2=0x7f29440f9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:27:39.159 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:39.158+0000 7f2948afa700 1 -- 172.21.15.157:0/3890330282 shutdown_connections 2024-04-22T23:27:39.159 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:39.158+0000 7f2948afa700 1 -- 172.21.15.157:0/3890330282 wait complete. 2024-04-22T23:27:39.159 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:39.158+0000 7f2948afa700 1 Processor -- start 2024-04-22T23:27:39.159 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:39.158+0000 7f2948afa700 1 -- start start 2024-04-22T23:27:39.159 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:39.158+0000 7f2948afa700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29440fb970 0x7f2944065c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:27:39.160 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:39.158+0000 7f2948afa700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f294410fd80 con 0x7f29440fb970 2024-04-22T23:27:39.160 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:39.158+0000 7f294259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29440fb970 0x7f2944065c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:27:39.160 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:39.158+0000 7f294259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29440fb970 0x7f2944065c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.157:46624/0 (socket says 172.21.15.157:46624) 2024-04-22T23:27:39.160 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:39.158+0000 7f294259c700 1 -- 172.21.15.157:0/2365680062 learned_addr learned my addr 172.21.15.157:0/2365680062 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-22T23:27:39.160 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:39.159+0000 7f294259c700 1 -- 172.21.15.157:0/2365680062 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2938005040 con 0x7f29440fb970 2024-04-22T23:27:39.160 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:39.159+0000 7f294259c700 1 --2- 172.21.15.157:0/2365680062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29440fb970 0x7f2944065c40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f29380096b0 tx=0x7f2938009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:27:39.160 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:39.159+0000 7f2932ffd700 1 -- 172.21.15.157:0/2365680062 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f293800bed0 con 0x7f29440fb970 2024-04-22T23:27:39.160 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:39.159+0000 7f2948afa700 1 -- 172.21.15.157:0/2365680062 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f294410ff20 con 0x7f29440fb970 2024-04-22T23:27:39.160 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:39.159+0000 7f2932ffd700 1 -- 172.21.15.157:0/2365680062 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f293800b5c0 con 0x7f29440fb970 2024-04-22T23:27:39.160 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:39.159+0000 7f2932ffd700 1 -- 172.21.15.157:0/2365680062 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f293801c5b0 con 0x7f29440fb970 2024-04-22T23:27:39.160 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:39.159+0000 7f2948afa700 1 -- 172.21.15.157:0/2365680062 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29441103b0 con 0x7f29440fb970 2024-04-22T23:27:39.161 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:39.160+0000 7f2932ffd700 1 -- 172.21.15.157:0/2365680062 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 43491+0+0 (secure 0 0 0) 0x7f2938013420 con 0x7f29440fb970 2024-04-22T23:27:39.161 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:39.160+0000 7f2948afa700 1 -- 172.21.15.157:0/2365680062 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2944105930 con 0x7f29440fb970 2024-04-22T23:27:39.161 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:39.160+0000 7f2932ffd700 1 --2- 172.21.15.157:0/2365680062 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f292c036970 0x7f292c038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:27:39.161 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:39.160+0000 7f2932ffd700 1 -- 172.21.15.157:0/2365680062 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f2938049be0 con 0x7f29440fb970 2024-04-22T23:27:39.161 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:39.160+0000 7f2941d9b700 1 --2- 172.21.15.157:0/2365680062 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f292c036970 0x7f292c038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:27:39.161 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:39.161+0000 7f2941d9b700 1 --2- 172.21.15.157:0/2365680062 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f292c036970 0x7f292c038e20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2934009990 tx=0x7f2934006e40 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:27:39.165 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:39.165+0000 7f2932ffd700 1 -- 172.21.15.157:0/2365680062 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f2938013a80 con 0x7f29440fb970 2024-04-22T23:27:39.433 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:39.434+0000 7f2948afa700 1 -- 172.21.15.157:0/2365680062 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f294405f5f0 con 0x7f29440fb970 2024-04-22T23:27:39.434 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:39.434+0000 7f2932ffd700 1 -- 172.21.15.157:0/2365680062 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f2938011370 con 0x7f29440fb970 2024-04-22T23:27:39.434 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:27:39.434 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":1,"fsid":"8252f6f2-00ff-11ef-bc93-c7b262605968","modified":"2024-04-22T23:25:22.712092Z","created":"2024-04-22T23:25:22.712092Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi107","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-22T23:27:39.436 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:39.436+0000 7f2948afa700 1 -- 172.21.15.157:0/2365680062 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f292c036970 msgr2=0x7f292c038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:27:39.436 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:39.436+0000 7f2948afa700 1 --2- 172.21.15.157:0/2365680062 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f292c036970 0x7f292c038e20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2934009990 tx=0x7f2934006e40 comp rx=0 tx=0).stop 2024-04-22T23:27:39.436 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:39.436+0000 7f2948afa700 1 -- 172.21.15.157:0/2365680062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29440fb970 msgr2=0x7f2944065c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:27:39.436 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:39.436+0000 7f2948afa700 1 --2- 172.21.15.157:0/2365680062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29440fb970 0x7f2944065c40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f29380096b0 tx=0x7f2938009e30 comp rx=0 tx=0).stop 2024-04-22T23:27:39.437 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:39.437+0000 7f2948afa700 1 -- 172.21.15.157:0/2365680062 shutdown_connections 2024-04-22T23:27:39.437 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:39.437+0000 7f2948afa700 1 --2- 172.21.15.157:0/2365680062 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f292c036970 0x7f292c038e20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:27:39.437 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:39.437+0000 7f2948afa700 1 --2- 172.21.15.157:0/2365680062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29440fb970 0x7f2944065c40 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:27:39.437 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:39.437+0000 7f2948afa700 1 -- 172.21.15.157:0/2365680062 >> 172.21.15.157:0/2365680062 conn(0x7f29440f70d0 msgr2=0x7f2944102ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:27:39.437 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:39.437+0000 7f2948afa700 1 -- 172.21.15.157:0/2365680062 shutdown_connections 2024-04-22T23:27:39.437 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:39.437+0000 7f2948afa700 1 -- 172.21.15.157:0/2365680062 wait complete. 2024-04-22T23:27:39.437 INFO:teuthology.orchestra.run.smithi157.stderr:dumped monmap epoch 1 2024-04-22T23:27:39.739 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:39 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:27:39.739 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:39 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:27:39.739 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:39 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:27:39.739 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:39 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:27:39.739 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:39 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "who": "osd/host:smithi107", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:27:40.738 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:40 smithi107 ceph-mon[26355]: from='client.? 172.21.15.157:0/2365680062' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T23:27:41.239 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-22T23:27:41.239 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-22T23:27:41.486 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T23:27:42.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:41 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:27:42.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:41 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:27:42.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:41 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:27:42.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:41 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:27:42.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:41 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "who": "osd/host:smithi157", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:27:42.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:41 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:27:42.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:41 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:27:42.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:41 smithi107 ceph-mon[26355]: Updating smithi107:/etc/ceph/ceph.conf 2024-04-22T23:27:42.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:41 smithi107 ceph-mon[26355]: Updating smithi157:/etc/ceph/ceph.conf 2024-04-22T23:27:42.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:41 smithi107 ceph-mon[26355]: Updating smithi157:/var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T23:27:42.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:41 smithi107 ceph-mon[26355]: Updating smithi107:/var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T23:27:42.240 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:41 smithi107 ceph-mon[26355]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2024-04-22T23:27:42.240 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:41 smithi107 ceph-mon[26355]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-04-22T23:27:42.240 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:41 smithi107 ceph-mon[26355]: Updating smithi157:/var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T23:27:42.240 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:41 smithi107 ceph-mon[26355]: Updating smithi107:/var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T23:27:42.240 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:41 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:27:42.240 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:41 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:27:42.240 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:41 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:27:42.240 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:41 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:27:42.240 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:41 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:27:42.240 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:41 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi157", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-22T23:27:42.240 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:41 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi157", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-04-22T23:27:42.240 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:41 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:27:42.985 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:42.984+0000 7f4ec52a6700 1 -- 172.21.15.157:0/93417236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ec0101a10 msgr2=0x7f4ec0101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:27:42.985 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:42.984+0000 7f4ec52a6700 1 --2- 172.21.15.157:0/93417236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ec0101a10 0x7f4ec0101de0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4eb00048c0 tx=0x7f4eb0009e30 comp rx=0 tx=0).stop 2024-04-22T23:27:42.985 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:42.984+0000 7f4ec52a6700 1 -- 172.21.15.157:0/93417236 shutdown_connections 2024-04-22T23:27:42.985 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:42.984+0000 7f4ec52a6700 1 --2- 172.21.15.157:0/93417236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ec0101a10 0x7f4ec0101de0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:27:42.985 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:42.984+0000 7f4ec52a6700 1 -- 172.21.15.157:0/93417236 >> 172.21.15.157:0/93417236 conn(0x7f4ec00f7530 msgr2=0x7f4ec00f9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:27:42.985 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:42.984+0000 7f4ec52a6700 1 -- 172.21.15.157:0/93417236 shutdown_connections 2024-04-22T23:27:42.985 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:42.984+0000 7f4ec52a6700 1 -- 172.21.15.157:0/93417236 wait complete. 2024-04-22T23:27:42.985 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:42.984+0000 7f4ec52a6700 1 Processor -- start 2024-04-22T23:27:42.985 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:42.984+0000 7f4ec52a6700 1 -- start start 2024-04-22T23:27:42.985 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:42.985+0000 7f4ec52a6700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ec0101a10 0x7f4ec0113200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:27:42.985 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:42.985+0000 7f4ec52a6700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec0113740 con 0x7f4ec0101a10 2024-04-22T23:27:42.985 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:42.985+0000 7f4ebeffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ec0101a10 0x7f4ec0113200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:27:42.986 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:42.985+0000 7f4ebeffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ec0101a10 0x7f4ec0113200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.157:37814/0 (socket says 172.21.15.157:37814) 2024-04-22T23:27:42.986 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:42.985+0000 7f4ebeffd700 1 -- 172.21.15.157:0/907802042 learned_addr learned my addr 172.21.15.157:0/907802042 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-22T23:27:42.986 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:42.985+0000 7f4ebeffd700 1 -- 172.21.15.157:0/907802042 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb0005040 con 0x7f4ec0101a10 2024-04-22T23:27:42.986 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:42.985+0000 7f4ebeffd700 1 --2- 172.21.15.157:0/907802042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ec0101a10 0x7f4ec0113200 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4eb000b820 tx=0x7f4eb0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:27:42.987 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:42.986+0000 7f4ea77fe700 1 -- 172.21.15.157:0/907802042 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4eb00093c0 con 0x7f4ec0101a10 2024-04-22T23:27:42.987 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:42.986+0000 7f4ec52a6700 1 -- 172.21.15.157:0/907802042 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ec0113940 con 0x7f4ec0101a10 2024-04-22T23:27:42.987 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:42.986+0000 7f4ea77fe700 1 -- 172.21.15.157:0/907802042 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4eb0013420 con 0x7f4ec0101a10 2024-04-22T23:27:42.987 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:42.986+0000 7f4ea77fe700 1 -- 172.21.15.157:0/907802042 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4eb001c5b0 con 0x7f4ec0101a10 2024-04-22T23:27:42.987 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:42.986+0000 7f4ec52a6700 1 -- 172.21.15.157:0/907802042 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ec0113db0 con 0x7f4ec0101a10 2024-04-22T23:27:42.987 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:42.986+0000 7f4ea77fe700 1 -- 172.21.15.157:0/907802042 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 43491+0+0 (secure 0 0 0) 0x7f4eb0009520 con 0x7f4ec0101a10 2024-04-22T23:27:42.987 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:42.986+0000 7f4ec52a6700 1 -- 172.21.15.157:0/907802042 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ec00fd110 con 0x7f4ec0101a10 2024-04-22T23:27:42.988 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:42.987+0000 7f4ea77fe700 1 --2- 172.21.15.157:0/907802042 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f4ea8036970 0x7f4ea8038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:27:42.988 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:42.987+0000 7f4ea77fe700 1 -- 172.21.15.157:0/907802042 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f4eb0049aa0 con 0x7f4ec0101a10 2024-04-22T23:27:42.988 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:42.987+0000 7f4ebe7fc700 1 --2- 172.21.15.157:0/907802042 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f4ea8036970 0x7f4ea8038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:27:42.988 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:42.987+0000 7f4ebe7fc700 1 --2- 172.21.15.157:0/907802042 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f4ea8036970 0x7f4ea8038e20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4009990 tx=0x7f4eb4006e40 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:27:42.991 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:42.992+0000 7f4ea77fe700 1 -- 172.21.15.157:0/907802042 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4eb0013a90 con 0x7f4ec0101a10 2024-04-22T23:27:43.238 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:42 smithi107 ceph-mon[26355]: Deploying daemon crash.smithi157 on smithi157 2024-04-22T23:27:43.281 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:43.281+0000 7f4ec52a6700 1 -- 172.21.15.157:0/907802042 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f4ec005f5f0 con 0x7f4ec0101a10 2024-04-22T23:27:43.281 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:43.282+0000 7f4ea77fe700 1 -- 172.21.15.157:0/907802042 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f4eb0011370 con 0x7f4ec0101a10 2024-04-22T23:27:43.281 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:27:43.282 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":1,"fsid":"8252f6f2-00ff-11ef-bc93-c7b262605968","modified":"2024-04-22T23:25:22.712092Z","created":"2024-04-22T23:25:22.712092Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi107","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-22T23:27:43.283 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:43.284+0000 7f4ec52a6700 1 -- 172.21.15.157:0/907802042 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f4ea8036970 msgr2=0x7f4ea8038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:27:43.283 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:43.284+0000 7f4ec52a6700 1 --2- 172.21.15.157:0/907802042 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f4ea8036970 0x7f4ea8038e20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4009990 tx=0x7f4eb4006e40 comp rx=0 tx=0).stop 2024-04-22T23:27:43.284 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:43.284+0000 7f4ec52a6700 1 -- 172.21.15.157:0/907802042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ec0101a10 msgr2=0x7f4ec0113200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:27:43.284 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:43.284+0000 7f4ec52a6700 1 --2- 172.21.15.157:0/907802042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ec0101a10 0x7f4ec0113200 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4eb000b820 tx=0x7f4eb0009e30 comp rx=0 tx=0).stop 2024-04-22T23:27:43.284 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:43.284+0000 7f4ec52a6700 1 -- 172.21.15.157:0/907802042 shutdown_connections 2024-04-22T23:27:43.284 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:43.284+0000 7f4ec52a6700 1 --2- 172.21.15.157:0/907802042 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f4ea8036970 0x7f4ea8038e20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:27:43.284 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:43.284+0000 7f4ec52a6700 1 --2- 172.21.15.157:0/907802042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ec0101a10 0x7f4ec0113200 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:27:43.284 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:43.284+0000 7f4ec52a6700 1 -- 172.21.15.157:0/907802042 >> 172.21.15.157:0/907802042 conn(0x7f4ec00f7530 msgr2=0x7f4ec0065e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:27:43.284 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:43.284+0000 7f4ec52a6700 1 -- 172.21.15.157:0/907802042 shutdown_connections 2024-04-22T23:27:43.284 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:43.284+0000 7f4ec52a6700 1 -- 172.21.15.157:0/907802042 wait complete. 2024-04-22T23:27:43.284 INFO:teuthology.orchestra.run.smithi157.stderr:dumped monmap epoch 1 2024-04-22T23:27:44.238 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:43 smithi107 ceph-mon[26355]: from='client.? 172.21.15.157:0/907802042' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T23:27:45.674 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-22T23:27:45.675 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-22T23:27:45.937 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T23:27:46.942 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:46 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:27:46.942 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:46 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:27:46.942 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:46 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:27:46.942 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:46 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:27:46.942 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:46 smithi107 ceph-mon[26355]: Deploying daemon node-exporter.smithi157 on smithi157 2024-04-22T23:27:47.511 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:47.511+0000 7f4718752700 1 -- 172.21.15.157:0/861715360 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4710102be0 msgr2=0x7f4710102fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:27:47.512 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:47.511+0000 7f4718752700 1 --2- 172.21.15.157:0/861715360 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4710102be0 0x7f4710102fb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f470c0048c0 tx=0x7f470c009e30 comp rx=0 tx=0).stop 2024-04-22T23:27:47.512 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:47.512+0000 7f4718752700 1 -- 172.21.15.157:0/861715360 shutdown_connections 2024-04-22T23:27:47.512 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:47.512+0000 7f4718752700 1 --2- 172.21.15.157:0/861715360 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4710102be0 0x7f4710102fb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:27:47.512 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:47.512+0000 7f4718752700 1 -- 172.21.15.157:0/861715360 >> 172.21.15.157:0/861715360 conn(0x7f47100f85f0 msgr2=0x7f47100faa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:27:47.512 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:47.512+0000 7f4718752700 1 -- 172.21.15.157:0/861715360 shutdown_connections 2024-04-22T23:27:47.512 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:47.512+0000 7f4718752700 1 -- 172.21.15.157:0/861715360 wait complete. 2024-04-22T23:27:47.514 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:47.513+0000 7f4718752700 1 Processor -- start 2024-04-22T23:27:47.514 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:47.513+0000 7f4718752700 1 -- start start 2024-04-22T23:27:47.514 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:47.513+0000 7f4718752700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4710102be0 0x7f4710113260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:27:47.514 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:47.513+0000 7f4718752700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47101137a0 con 0x7f4710102be0 2024-04-22T23:27:47.514 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:47.513+0000 7f47164ee700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4710102be0 0x7f4710113260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:27:47.514 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:47.513+0000 7f47164ee700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4710102be0 0x7f4710113260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.157:37846/0 (socket says 172.21.15.157:37846) 2024-04-22T23:27:47.514 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:47.513+0000 7f47164ee700 1 -- 172.21.15.157:0/929347497 learned_addr learned my addr 172.21.15.157:0/929347497 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-22T23:27:47.514 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:47.514+0000 7f47164ee700 1 -- 172.21.15.157:0/929347497 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f470c005040 con 0x7f4710102be0 2024-04-22T23:27:47.515 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:47.514+0000 7f47164ee700 1 --2- 172.21.15.157:0/929347497 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4710102be0 0x7f4710113260 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f470c00b820 tx=0x7f470c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:27:47.515 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:47.514+0000 7f4706ffd700 1 -- 172.21.15.157:0/929347497 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f470c0093c0 con 0x7f4710102be0 2024-04-22T23:27:47.515 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:47.514+0000 7f4706ffd700 1 -- 172.21.15.157:0/929347497 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f470c013420 con 0x7f4710102be0 2024-04-22T23:27:47.515 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:47.514+0000 7f4718752700 1 -- 172.21.15.157:0/929347497 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47101139a0 con 0x7f4710102be0 2024-04-22T23:27:47.515 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:47.514+0000 7f4706ffd700 1 -- 172.21.15.157:0/929347497 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f470c01c5b0 con 0x7f4710102be0 2024-04-22T23:27:47.515 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:47.514+0000 7f4718752700 1 -- 172.21.15.157:0/929347497 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4710113e90 con 0x7f4710102be0 2024-04-22T23:27:47.516 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:47.515+0000 7f4706ffd700 1 -- 172.21.15.157:0/929347497 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 43491+0+0 (secure 0 0 0) 0x7f470c01c7f0 con 0x7f4710102be0 2024-04-22T23:27:47.516 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:47.515+0000 7f4706ffd700 1 --2- 172.21.15.157:0/929347497 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f46fc036970 0x7f46fc038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:27:47.516 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:47.515+0000 7f4718752700 1 -- 172.21.15.157:0/929347497 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f471005f5f0 con 0x7f4710102be0 2024-04-22T23:27:47.516 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:47.515+0000 7f4706ffd700 1 -- 172.21.15.157:0/929347497 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f470c049b90 con 0x7f4710102be0 2024-04-22T23:27:47.516 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:47.515+0000 7f4715ced700 1 --2- 172.21.15.157:0/929347497 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f46fc036970 0x7f46fc038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:27:47.516 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:47.516+0000 7f4715ced700 1 --2- 172.21.15.157:0/929347497 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f46fc036970 0x7f46fc038e20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4700009990 tx=0x7f4700006e40 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:27:47.519 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:47.520+0000 7f4706ffd700 1 -- 172.21.15.157:0/929347497 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f470c023300 con 0x7f4710102be0 2024-04-22T23:27:47.801 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:47.801+0000 7f4718752700 1 -- 172.21.15.157:0/929347497 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f471010e8b0 con 0x7f4710102be0 2024-04-22T23:27:47.801 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:47.802+0000 7f4706ffd700 1 -- 172.21.15.157:0/929347497 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f470c013a90 con 0x7f4710102be0 2024-04-22T23:27:47.802 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:27:47.802 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":1,"fsid":"8252f6f2-00ff-11ef-bc93-c7b262605968","modified":"2024-04-22T23:25:22.712092Z","created":"2024-04-22T23:25:22.712092Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi107","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-22T23:27:47.804 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:47.804+0000 7f4718752700 1 -- 172.21.15.157:0/929347497 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f46fc036970 msgr2=0x7f46fc038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:27:47.804 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:47.804+0000 7f4718752700 1 --2- 172.21.15.157:0/929347497 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f46fc036970 0x7f46fc038e20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4700009990 tx=0x7f4700006e40 comp rx=0 tx=0).stop 2024-04-22T23:27:47.804 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:47.804+0000 7f4718752700 1 -- 172.21.15.157:0/929347497 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4710102be0 msgr2=0x7f4710113260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:27:47.804 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:47.804+0000 7f4718752700 1 --2- 172.21.15.157:0/929347497 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4710102be0 0x7f4710113260 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f470c00b820 tx=0x7f470c009e30 comp rx=0 tx=0).stop 2024-04-22T23:27:47.804 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:47.804+0000 7f4718752700 1 -- 172.21.15.157:0/929347497 shutdown_connections 2024-04-22T23:27:47.804 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:47.804+0000 7f4718752700 1 --2- 172.21.15.157:0/929347497 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f46fc036970 0x7f46fc038e20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:27:47.804 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:47.804+0000 7f4718752700 1 --2- 172.21.15.157:0/929347497 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4710102be0 0x7f4710113260 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:27:47.804 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:47.804+0000 7f4718752700 1 -- 172.21.15.157:0/929347497 >> 172.21.15.157:0/929347497 conn(0x7f47100f85f0 msgr2=0x7f47100f9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:27:47.804 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:47.804+0000 7f4718752700 1 -- 172.21.15.157:0/929347497 shutdown_connections 2024-04-22T23:27:47.804 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:47.804+0000 7f4718752700 1 -- 172.21.15.157:0/929347497 wait complete. 2024-04-22T23:27:47.804 INFO:teuthology.orchestra.run.smithi157.stderr:dumped monmap epoch 1 2024-04-22T23:27:48.129 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:47 smithi107 ceph-mon[26355]: from='client.? 172.21.15.157:0/929347497' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T23:27:49.897 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-22T23:27:49.897 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-22T23:27:50.238 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:49 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:27:50.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:49 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:27:50.444 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T23:27:51.738 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:51 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:27:51.739 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:51 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:27:51.739 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:51 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:27:51.739 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:51 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:27:51.739 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:51 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi157.tuylis", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T23:27:51.739 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:51 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi157.tuylis", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-22T23:27:51.739 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:51 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T23:27:51.739 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:51 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:27:51.739 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:51 smithi107 ceph-mon[26355]: Deploying daemon mgr.smithi157.tuylis on smithi157 2024-04-22T23:27:51.854 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:51.854+0000 7fb4953f4700 1 -- 172.21.15.157:0/2347237318 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb490102b00 msgr2=0x7fb490102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:27:51.854 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:51.854+0000 7fb4953f4700 1 --2- 172.21.15.157:0/2347237318 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb490102b00 0x7fb490102ed0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb4800048c0 tx=0x7fb480009e30 comp rx=0 tx=0).stop 2024-04-22T23:27:51.854 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:51.854+0000 7fb4953f4700 1 -- 172.21.15.157:0/2347237318 shutdown_connections 2024-04-22T23:27:51.854 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:51.854+0000 7fb4953f4700 1 --2- 172.21.15.157:0/2347237318 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb490102b00 0x7fb490102ed0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:27:51.855 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:51.854+0000 7fb4953f4700 1 -- 172.21.15.157:0/2347237318 >> 172.21.15.157:0/2347237318 conn(0x7fb4900f8600 msgr2=0x7fb4900faa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:27:51.855 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:51.854+0000 7fb4953f4700 1 -- 172.21.15.157:0/2347237318 shutdown_connections 2024-04-22T23:27:51.855 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:51.854+0000 7fb4953f4700 1 -- 172.21.15.157:0/2347237318 wait complete. 2024-04-22T23:27:51.855 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:51.855+0000 7fb4953f4700 1 Processor -- start 2024-04-22T23:27:51.855 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:51.855+0000 7fb4953f4700 1 -- start start 2024-04-22T23:27:51.855 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:51.855+0000 7fb4953f4700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb490102b00 0x7fb4900702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:27:51.855 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:51.855+0000 7fb4953f4700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb490070800 con 0x7fb490102b00 2024-04-22T23:27:51.856 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:51.855+0000 7fb48effd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb490102b00 0x7fb4900702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:27:51.857 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:51.855+0000 7fb48effd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb490102b00 0x7fb4900702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.157:37862/0 (socket says 172.21.15.157:37862) 2024-04-22T23:27:51.857 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:51.855+0000 7fb48effd700 1 -- 172.21.15.157:0/3487378415 learned_addr learned my addr 172.21.15.157:0/3487378415 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-22T23:27:51.857 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:51.856+0000 7fb48effd700 1 -- 172.21.15.157:0/3487378415 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb480005040 con 0x7fb490102b00 2024-04-22T23:27:51.857 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:51.856+0000 7fb48effd700 1 --2- 172.21.15.157:0/3487378415 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb490102b00 0x7fb4900702c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb48000b820 tx=0x7fb480009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:27:51.857 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:51.856+0000 7fb4777fe700 1 -- 172.21.15.157:0/3487378415 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb4800093c0 con 0x7fb490102b00 2024-04-22T23:27:51.857 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:51.856+0000 7fb4953f4700 1 -- 172.21.15.157:0/3487378415 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb490070a00 con 0x7fb490102b00 2024-04-22T23:27:51.857 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:51.856+0000 7fb4953f4700 1 -- 172.21.15.157:0/3487378415 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb490070ef0 con 0x7fb490102b00 2024-04-22T23:27:51.857 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:51.856+0000 7fb4777fe700 1 -- 172.21.15.157:0/3487378415 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb480013420 con 0x7fb490102b00 2024-04-22T23:27:51.857 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:51.856+0000 7fb4777fe700 1 -- 172.21.15.157:0/3487378415 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb4800255c0 con 0x7fb490102b00 2024-04-22T23:27:51.857 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:51.857+0000 7fb4953f4700 1 -- 172.21.15.157:0/3487378415 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb47c0052b0 con 0x7fb490102b00 2024-04-22T23:27:51.857 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:51.857+0000 7fb4777fe700 1 -- 172.21.15.157:0/3487378415 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 43491+0+0 (secure 0 0 0) 0x7fb480009520 con 0x7fb490102b00 2024-04-22T23:27:51.857 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:51.857+0000 7fb4777fe700 1 --2- 172.21.15.157:0/3487378415 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fb4780369c0 0x7fb478038e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:27:51.857 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:51.857+0000 7fb4777fe700 1 -- 172.21.15.157:0/3487378415 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fb480049b90 con 0x7fb490102b00 2024-04-22T23:27:51.858 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:51.857+0000 7fb48e7fc700 1 --2- 172.21.15.157:0/3487378415 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fb4780369c0 0x7fb478038e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:27:51.858 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:51.858+0000 7fb48e7fc700 1 --2- 172.21.15.157:0/3487378415 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fb4780369c0 0x7fb478038e70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb484009990 tx=0x7fb484006e40 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:27:51.862 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:51.862+0000 7fb4777fe700 1 -- 172.21.15.157:0/3487378415 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb480013a70 con 0x7fb490102b00 2024-04-22T23:27:52.141 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:52.142+0000 7fb4953f4700 1 -- 172.21.15.157:0/3487378415 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb47c005560 con 0x7fb490102b00 2024-04-22T23:27:52.142 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:52.143+0000 7fb4777fe700 1 -- 172.21.15.157:0/3487378415 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7fb480049e60 con 0x7fb490102b00 2024-04-22T23:27:52.142 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:27:52.142 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":1,"fsid":"8252f6f2-00ff-11ef-bc93-c7b262605968","modified":"2024-04-22T23:25:22.712092Z","created":"2024-04-22T23:25:22.712092Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi107","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-22T23:27:52.145 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:52.145+0000 7fb4953f4700 1 -- 172.21.15.157:0/3487378415 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fb4780369c0 msgr2=0x7fb478038e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:27:52.145 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:52.145+0000 7fb4953f4700 1 --2- 172.21.15.157:0/3487378415 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fb4780369c0 0x7fb478038e70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb484009990 tx=0x7fb484006e40 comp rx=0 tx=0).stop 2024-04-22T23:27:52.145 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:52.145+0000 7fb4953f4700 1 -- 172.21.15.157:0/3487378415 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb490102b00 msgr2=0x7fb4900702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:27:52.145 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:52.145+0000 7fb4953f4700 1 --2- 172.21.15.157:0/3487378415 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb490102b00 0x7fb4900702c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb48000b820 tx=0x7fb480009e30 comp rx=0 tx=0).stop 2024-04-22T23:27:52.145 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:52.145+0000 7fb4953f4700 1 -- 172.21.15.157:0/3487378415 shutdown_connections 2024-04-22T23:27:52.145 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:52.145+0000 7fb4953f4700 1 --2- 172.21.15.157:0/3487378415 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fb4780369c0 0x7fb478038e70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:27:52.145 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:52.145+0000 7fb4953f4700 1 --2- 172.21.15.157:0/3487378415 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb490102b00 0x7fb4900702c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:27:52.145 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:52.145+0000 7fb4953f4700 1 -- 172.21.15.157:0/3487378415 >> 172.21.15.157:0/3487378415 conn(0x7fb4900f8600 msgr2=0x7fb4900f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:27:52.146 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:52.145+0000 7fb4953f4700 1 -- 172.21.15.157:0/3487378415 shutdown_connections 2024-04-22T23:27:52.146 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:52.146+0000 7fb4953f4700 1 -- 172.21.15.157:0/3487378415 wait complete. 2024-04-22T23:27:52.146 INFO:teuthology.orchestra.run.smithi157.stderr:dumped monmap epoch 1 2024-04-22T23:27:52.738 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:52 smithi107 ceph-mon[26355]: from='client.? 172.21.15.157:0/3487378415' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T23:27:54.619 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-22T23:27:54.620 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-22T23:27:55.238 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:54 smithi107 ceph-mon[26355]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:27:55.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:54 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:27:55.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:54 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:27:55.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:54 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:27:55.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:54 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:27:55.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:54 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:27:55.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:54 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T23:27:55.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:54 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:27:55.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:54 smithi107 ceph-mon[26355]: Deploying daemon mon.smithi157 on smithi157 2024-04-22T23:27:56.514 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi157/config 2024-04-22T23:27:57.238 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:27:56 smithi107 ceph-mon[26355]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:27:58.035 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:58.034+0000 7faa99af4700 1 -- 172.21.15.157:0/2330517332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa94102bb0 msgr2=0x7faa94102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:27:58.035 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:58.034+0000 7faa99af4700 1 --2- 172.21.15.157:0/2330517332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa94102bb0 0x7faa94102f80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7faa840048c0 tx=0x7faa84009e30 comp rx=0 tx=0).stop 2024-04-22T23:27:58.035 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:58.034+0000 7faa99af4700 1 -- 172.21.15.157:0/2330517332 shutdown_connections 2024-04-22T23:27:58.035 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:58.034+0000 7faa99af4700 1 --2- 172.21.15.157:0/2330517332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa94102bb0 0x7faa94102f80 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:27:58.036 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:58.034+0000 7faa99af4700 1 -- 172.21.15.157:0/2330517332 >> 172.21.15.157:0/2330517332 conn(0x7faa940f8720 msgr2=0x7faa940fab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:27:58.036 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:58.034+0000 7faa99af4700 1 -- 172.21.15.157:0/2330517332 shutdown_connections 2024-04-22T23:27:58.036 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:58.034+0000 7faa99af4700 1 -- 172.21.15.157:0/2330517332 wait complete. 2024-04-22T23:27:58.036 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:58.034+0000 7faa99af4700 1 Processor -- start 2024-04-22T23:27:58.036 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:58.035+0000 7faa99af4700 1 -- start start 2024-04-22T23:27:58.036 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:58.035+0000 7faa99af4700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa94102bb0 0x7faa94112160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:27:58.036 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:58.035+0000 7faa99af4700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa941126a0 con 0x7faa94102bb0 2024-04-22T23:27:58.037 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:58.035+0000 7faa937fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa94102bb0 0x7faa94112160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:27:58.037 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:58.036+0000 7faa937fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa94102bb0 0x7faa94112160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.157:48738/0 (socket says 172.21.15.157:48738) 2024-04-22T23:27:58.037 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:58.036+0000 7faa937fe700 1 -- 172.21.15.157:0/2609859810 learned_addr learned my addr 172.21.15.157:0/2609859810 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-22T23:27:58.037 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:58.036+0000 7faa937fe700 1 -- 172.21.15.157:0/2609859810 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa84005040 con 0x7faa94102bb0 2024-04-22T23:27:58.037 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:58.036+0000 7faa937fe700 1 --2- 172.21.15.157:0/2609859810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa94102bb0 0x7faa94112160 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7faa8400b820 tx=0x7faa84009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:27:58.037 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:58.036+0000 7faa7bfff700 1 -- 172.21.15.157:0/2609859810 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faa840093c0 con 0x7faa94102bb0 2024-04-22T23:27:58.037 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:58.036+0000 7faa99af4700 1 -- 172.21.15.157:0/2609859810 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa94112840 con 0x7faa94102bb0 2024-04-22T23:27:58.037 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:58.036+0000 7faa7bfff700 1 -- 172.21.15.157:0/2609859810 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faa84013420 con 0x7faa94102bb0 2024-04-22T23:27:58.038 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:58.036+0000 7faa7bfff700 1 -- 172.21.15.157:0/2609859810 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faa8401c5b0 con 0x7faa94102bb0 2024-04-22T23:27:58.038 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:58.036+0000 7faa99af4700 1 -- 172.21.15.157:0/2609859810 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa94112d30 con 0x7faa94102bb0 2024-04-22T23:27:58.038 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:58.037+0000 7faa99af4700 1 -- 172.21.15.157:0/2609859810 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa940fe300 con 0x7faa94102bb0 2024-04-22T23:27:58.038 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:58.037+0000 7faa7bfff700 1 -- 172.21.15.157:0/2609859810 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 43491+0+0 (secure 0 0 0) 0x7faa8401c7f0 con 0x7faa94102bb0 2024-04-22T23:27:58.038 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:58.037+0000 7faa7bfff700 1 --2- 172.21.15.157:0/2609859810 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7faa7c036970 0x7faa7c038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:27:58.039 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:58.037+0000 7faa7bfff700 1 -- 172.21.15.157:0/2609859810 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7faa84049d90 con 0x7faa94102bb0 2024-04-22T23:27:58.039 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:58.038+0000 7faa92ffd700 1 --2- 172.21.15.157:0/2609859810 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7faa7c036970 0x7faa7c038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:27:58.039 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:58.038+0000 7faa92ffd700 1 --2- 172.21.15.157:0/2609859810 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7faa7c036970 0x7faa7c038e20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7faa88009990 tx=0x7faa88006e40 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:27:58.043 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:58.043+0000 7faa7bfff700 1 -- 172.21.15.157:0/2609859810 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7faa84023300 con 0x7faa94102bb0 2024-04-22T23:27:58.343 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:58.342+0000 7faa99af4700 1 -- 172.21.15.157:0/2609859810 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7faa9405f5f0 con 0x7faa94102bb0 2024-04-22T23:27:58.343 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:58.342+0000 7faa7bfff700 1 -- 172.21.15.157:0/2609859810 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7faa84011b60 con 0x7faa94102bb0 2024-04-22T23:27:58.343 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:27:58.343 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":1,"fsid":"8252f6f2-00ff-11ef-bc93-c7b262605968","modified":"2024-04-22T23:25:22.712092Z","created":"2024-04-22T23:25:22.712092Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi107","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-22T23:27:58.346 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:58.346+0000 7faa99af4700 1 -- 172.21.15.157:0/2609859810 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7faa7c036970 msgr2=0x7faa7c038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:27:58.346 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:58.346+0000 7faa99af4700 1 --2- 172.21.15.157:0/2609859810 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7faa7c036970 0x7faa7c038e20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7faa88009990 tx=0x7faa88006e40 comp rx=0 tx=0).stop 2024-04-22T23:27:58.346 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:58.346+0000 7faa99af4700 1 -- 172.21.15.157:0/2609859810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa94102bb0 msgr2=0x7faa94112160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:27:58.346 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:58.346+0000 7faa99af4700 1 --2- 172.21.15.157:0/2609859810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa94102bb0 0x7faa94112160 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7faa8400b820 tx=0x7faa84009e30 comp rx=0 tx=0).stop 2024-04-22T23:27:58.346 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:58.346+0000 7faa99af4700 1 -- 172.21.15.157:0/2609859810 shutdown_connections 2024-04-22T23:27:58.347 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:58.346+0000 7faa99af4700 1 --2- 172.21.15.157:0/2609859810 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7faa7c036970 0x7faa7c038e20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:27:58.347 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:58.346+0000 7faa99af4700 1 --2- 172.21.15.157:0/2609859810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa94102bb0 0x7faa94112160 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:27:58.347 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:58.346+0000 7faa99af4700 1 -- 172.21.15.157:0/2609859810 >> 172.21.15.157:0/2609859810 conn(0x7faa940f8720 msgr2=0x7faa940f9e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:27:58.347 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:58.346+0000 7faa99af4700 1 -- 172.21.15.157:0/2609859810 shutdown_connections 2024-04-22T23:27:58.347 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:27:58.346+0000 7faa99af4700 1 -- 172.21.15.157:0/2609859810 wait complete. 2024-04-22T23:27:58.347 INFO:teuthology.orchestra.run.smithi157.stderr:dumped monmap epoch 1 2024-04-22T23:27:58.749 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:27:58 smithi157 bash[33839]: 0747a85008d9f4a7781d82c00da641ad5c340e116ad04926a3459889ec06a0c5 2024-04-22T23:27:59.084 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:27:58 smithi157 systemd[1]: Started Ceph mon.smithi157 for 8252f6f2-00ff-11ef-bc93-c7b262605968. 2024-04-22T23:28:00.321 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-22T23:28:00.321 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-22T23:28:00.582 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi157/config 2024-04-22T23:28:04.137 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:03 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-04-22T23:28:04.137 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:03 smithi157 ceph-mon[33877]: mon.smithi107 calling monitor election 2024-04-22T23:28:04.137 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:03 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mon metadata", "id": "smithi157"}]: dispatch 2024-04-22T23:28:04.137 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:03 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mon metadata", "id": "smithi157"}]: dispatch 2024-04-22T23:28:04.137 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:03 smithi157 ceph-mon[33877]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:28:04.137 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:03 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mon metadata", "id": "smithi157"}]: dispatch 2024-04-22T23:28:04.137 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:03 smithi157 ceph-mon[33877]: mon.smithi157 calling monitor election 2024-04-22T23:28:04.137 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:03 smithi157 ceph-mon[33877]: from='mgr.? 172.21.15.157:0/876507945' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi157.tuylis/crt"}]: dispatch 2024-04-22T23:28:04.137 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:03 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mon metadata", "id": "smithi157"}]: dispatch 2024-04-22T23:28:04.137 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:03 smithi157 ceph-mon[33877]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:28:04.137 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:03 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mon metadata", "id": "smithi157"}]: dispatch 2024-04-22T23:28:04.137 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:03 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mon metadata", "id": "smithi157"}]: dispatch 2024-04-22T23:28:04.137 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:03 smithi157 ceph-mon[33877]: mon.smithi107 is new leader, mons smithi107,smithi157 in quorum (ranks 0,1) 2024-04-22T23:28:04.138 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:03 smithi157 ceph-mon[33877]: monmap e2: 2 mons at {smithi107=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0],smithi157=[v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-22T23:28:04.138 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:03 smithi157 ceph-mon[33877]: fsmap 2024-04-22T23:28:04.138 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:03 smithi157 ceph-mon[33877]: osdmap e5: 0 total, 0 up, 0 in 2024-04-22T23:28:04.138 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:03 smithi157 ceph-mon[33877]: mgrmap e17: smithi107.uipxvm(active, since 29s) 2024-04-22T23:28:04.138 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:03 smithi157 ceph-mon[33877]: overall HEALTH_OK 2024-04-22T23:28:04.138 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:03 smithi157 ceph-mon[33877]: Standby manager daemon smithi157.tuylis started 2024-04-22T23:28:04.138 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:03 smithi157 ceph-mon[33877]: from='mgr.? 172.21.15.157:0/876507945' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T23:28:04.138 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:03 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:04.138 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:03 smithi157 ceph-mon[33877]: from='mgr.? 172.21.15.157:0/876507945' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi157.tuylis/key"}]: dispatch 2024-04-22T23:28:04.138 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:03 smithi157 ceph-mon[33877]: from='mgr.? 172.21.15.157:0/876507945' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T23:28:04.138 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:03 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:04.138 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:03 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:04.138 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:03 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:04.138 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:03 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:04.138 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:03 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:04.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:03 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-04-22T23:28:04.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:03 smithi107 ceph-mon[26355]: mon.smithi107 calling monitor election 2024-04-22T23:28:04.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:03 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mon metadata", "id": "smithi157"}]: dispatch 2024-04-22T23:28:04.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:03 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mon metadata", "id": "smithi157"}]: dispatch 2024-04-22T23:28:04.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:03 smithi107 ceph-mon[26355]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:28:04.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:03 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mon metadata", "id": "smithi157"}]: dispatch 2024-04-22T23:28:04.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:03 smithi107 ceph-mon[26355]: mon.smithi157 calling monitor election 2024-04-22T23:28:04.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:03 smithi107 ceph-mon[26355]: from='mgr.? 172.21.15.157:0/876507945' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi157.tuylis/crt"}]: dispatch 2024-04-22T23:28:04.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:03 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mon metadata", "id": "smithi157"}]: dispatch 2024-04-22T23:28:04.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:03 smithi107 ceph-mon[26355]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:28:04.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:03 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mon metadata", "id": "smithi157"}]: dispatch 2024-04-22T23:28:04.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:03 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mon metadata", "id": "smithi157"}]: dispatch 2024-04-22T23:28:04.240 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:03 smithi107 ceph-mon[26355]: mon.smithi107 is new leader, mons smithi107,smithi157 in quorum (ranks 0,1) 2024-04-22T23:28:04.240 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:03 smithi107 ceph-mon[26355]: monmap e2: 2 mons at {smithi107=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0],smithi157=[v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-22T23:28:04.240 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:03 smithi107 ceph-mon[26355]: fsmap 2024-04-22T23:28:04.240 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:03 smithi107 ceph-mon[26355]: osdmap e5: 0 total, 0 up, 0 in 2024-04-22T23:28:04.240 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:03 smithi107 ceph-mon[26355]: mgrmap e17: smithi107.uipxvm(active, since 29s) 2024-04-22T23:28:04.240 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:03 smithi107 ceph-mon[26355]: overall HEALTH_OK 2024-04-22T23:28:04.240 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:03 smithi107 ceph-mon[26355]: Standby manager daemon smithi157.tuylis started 2024-04-22T23:28:04.240 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:03 smithi107 ceph-mon[26355]: from='mgr.? 172.21.15.157:0/876507945' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T23:28:04.240 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:03 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:04.240 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:03 smithi107 ceph-mon[26355]: from='mgr.? 172.21.15.157:0/876507945' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi157.tuylis/key"}]: dispatch 2024-04-22T23:28:04.240 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:03 smithi107 ceph-mon[26355]: from='mgr.? 172.21.15.157:0/876507945' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T23:28:04.240 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:03 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:04.240 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:03 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:04.240 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:03 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:04.240 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:03 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:04.240 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:03 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:05.091 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:28:05.090+0000 7efcfc767700 1 -- 172.21.15.157:0/685216917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efce0003580 msgr2=0x7efce0005a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:28:05.091 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:28:05.090+0000 7efcfc767700 1 --2- 172.21.15.157:0/685216917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efce0003580 0x7efce0005a00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7efcf40fcaf0 tx=0x7efcec009040 comp rx=0 tx=0).stop 2024-04-22T23:28:05.091 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:28:05.090+0000 7efcfc767700 1 -- 172.21.15.157:0/685216917 shutdown_connections 2024-04-22T23:28:05.091 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:28:05.090+0000 7efcfc767700 1 --2- 172.21.15.157:0/685216917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efce0003580 0x7efce0005a00 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:28:05.091 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:28:05.090+0000 7efcfc767700 1 -- 172.21.15.157:0/685216917 >> 172.21.15.157:0/685216917 conn(0x7efcf40f8720 msgr2=0x7efcf40fab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:28:05.091 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:28:05.090+0000 7efcfc767700 1 -- 172.21.15.157:0/685216917 shutdown_connections 2024-04-22T23:28:05.092 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:28:05.090+0000 7efcfc767700 1 -- 172.21.15.157:0/685216917 wait complete. 2024-04-22T23:28:05.092 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:04 smithi157 ceph-mon[33877]: mgrmap e18: smithi107.uipxvm(active, since 29s), standbys: smithi157.tuylis 2024-04-22T23:28:05.092 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:04 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mgr metadata", "who": "smithi157.tuylis", "id": "smithi157.tuylis"}]: dispatch 2024-04-22T23:28:05.092 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:04 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:28:05.092 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:04 smithi157 ceph-mon[33877]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:28:05.092 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:04 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:28:05.092 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:04 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mon metadata", "id": "smithi157"}]: dispatch 2024-04-22T23:28:05.098 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:28:05.091+0000 7efcfc767700 1 Processor -- start 2024-04-22T23:28:05.098 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:28:05.091+0000 7efcfc767700 1 -- start start 2024-04-22T23:28:05.098 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:28:05.091+0000 7efcfc767700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efce0003580 0x7efcf4113350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:28:05.098 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:28:05.091+0000 7efcfc767700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcf4113890 0x7efcf410d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:28:05.099 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:28:05.091+0000 7efcfc767700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf4113ea0 con 0x7efcf4113890 2024-04-22T23:28:05.099 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:28:05.091+0000 7efcfc767700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf4114010 con 0x7efce0003580 2024-04-22T23:28:05.099 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:28:05.091+0000 7efcfa503700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efce0003580 0x7efcf4113350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:28:05.099 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:28:05.091+0000 7efcfa503700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efce0003580 0x7efcf4113350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:58514/0 (socket says 172.21.15.157:58514) 2024-04-22T23:28:05.099 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:28:05.091+0000 7efcfa503700 1 -- 172.21.15.157:0/3934322537 learned_addr learned my addr 172.21.15.157:0/3934322537 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-22T23:28:05.099 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:28:05.091+0000 7efcfa503700 1 -- 172.21.15.157:0/3934322537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efce0003580 msgr2=0x7efcf4113350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-22T23:28:05.099 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:28:05.091+0000 7efcfa503700 1 -- 172.21.15.157:0/3934322537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efce0003580 msgr2=0x7efcf4113350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-22T23:28:05.099 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:28:05.091+0000 7efcfa503700 1 --2- 172.21.15.157:0/3934322537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efce0003580 0x7efcf4113350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-22T23:28:05.099 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:28:05.091+0000 7efcfa503700 1 --2- 172.21.15.157:0/3934322537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efce0003580 0x7efcf4113350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T23:28:05.099 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:28:05.092+0000 7efcf9d02700 1 --2- 172.21.15.157:0/3934322537 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcf4113890 0x7efcf410d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:28:05.099 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:28:05.092+0000 7efcf9d02700 1 -- 172.21.15.157:0/3934322537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efce0003580 msgr2=0x7efcf4113350 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T23:28:05.099 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:28:05.092+0000 7efcf9d02700 1 --2- 172.21.15.157:0/3934322537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efce0003580 0x7efcf4113350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:28:05.099 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:28:05.092+0000 7efcf9d02700 1 -- 172.21.15.157:0/3934322537 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcec0079e0 con 0x7efcf4113890 2024-04-22T23:28:05.099 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:28:05.092+0000 7efcf9d02700 1 --2- 172.21.15.157:0/3934322537 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcf4113890 0x7efcf410d2f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efce400b790 tx=0x7efce40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:28:05.100 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:28:05.092+0000 7efceaffd700 1 -- 172.21.15.157:0/3934322537 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efce4011810 con 0x7efcf4113890 2024-04-22T23:28:05.100 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:28:05.092+0000 7efceaffd700 1 -- 172.21.15.157:0/3934322537 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efce4011e40 con 0x7efcf4113890 2024-04-22T23:28:05.100 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:28:05.092+0000 7efcfc767700 1 -- 172.21.15.157:0/3934322537 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcf410d950 con 0x7efcf4113890 2024-04-22T23:28:05.100 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:28:05.092+0000 7efceaffd700 1 -- 172.21.15.157:0/3934322537 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efce400f2e0 con 0x7efcf4113890 2024-04-22T23:28:05.100 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:28:05.092+0000 7efcfc767700 1 -- 172.21.15.157:0/3934322537 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcf410de90 con 0x7efcf4113890 2024-04-22T23:28:05.100 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:28:05.093+0000 7efcfc767700 1 -- 172.21.15.157:0/3934322537 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcf410e160 con 0x7efcf4113890 2024-04-22T23:28:05.100 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:28:05.094+0000 7efceaffd700 1 -- 172.21.15.157:0/3934322537 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 86575+0+0 (secure 0 0 0) 0x7efce4010380 con 0x7efcf4113890 2024-04-22T23:28:05.100 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:28:05.094+0000 7efceaffd700 1 --2- 172.21.15.157:0/3934322537 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7efcd8069320 0x7efcd806b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:28:05.100 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:28:05.094+0000 7efceaffd700 1 -- 172.21.15.157:0/3934322537 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7efce4086360 con 0x7efcf4113890 2024-04-22T23:28:05.100 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:28:05.094+0000 7efcfa503700 1 --2- 172.21.15.157:0/3934322537 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7efcd8069320 0x7efcd806b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:28:05.100 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:28:05.095+0000 7efcfa503700 1 --2- 172.21.15.157:0/3934322537 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7efcd8069320 0x7efcd806b7d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7efcec0139e0 tx=0x7efcec009040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:28:05.100 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:28:05.100+0000 7efceaffd700 1 -- 172.21.15.157:0/3934322537 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7efcf410e160 con 0x7efcf4113890 2024-04-22T23:28:05.238 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:04 smithi107 ceph-mon[26355]: mgrmap e18: smithi107.uipxvm(active, since 29s), standbys: smithi157.tuylis 2024-04-22T23:28:05.238 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:04 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mgr metadata", "who": "smithi157.tuylis", "id": "smithi157.tuylis"}]: dispatch 2024-04-22T23:28:05.238 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:04 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:28:05.238 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:04 smithi107 ceph-mon[26355]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:28:05.238 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:04 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:28:05.239 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:04 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mon metadata", "id": "smithi157"}]: dispatch 2024-04-22T23:28:05.376 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:28:05.377+0000 7efcfc767700 1 -- 172.21.15.157:0/3934322537 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7efcf410e160 con 0x7efcf4113890 2024-04-22T23:28:05.377 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:28:05.378+0000 7efceaffd700 1 -- 172.21.15.157:0/3934322537 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1021 (secure 0 0 0) 0x7efcf410e160 con 0x7efcf4113890 2024-04-22T23:28:05.377 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-04-22T23:28:05.377 INFO:teuthology.orchestra.run.smithi157.stdout:{"epoch":2,"fsid":"8252f6f2-00ff-11ef-bc93-c7b262605968","modified":"2024-04-22T23:27:58.719644Z","created":"2024-04-22T23:25:22.712092Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi107","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi157","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:3300","nonce":0},{"type":"v1","addr":"172.21.15.157:6789","nonce":0}]},"addr":"172.21.15.157:6789/0","public_addr":"172.21.15.157:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-22T23:28:05.380 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:28:05.380+0000 7efcfc767700 1 -- 172.21.15.157:0/3934322537 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7efcd8069320 msgr2=0x7efcd806b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:28:05.380 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:28:05.380+0000 7efcfc767700 1 --2- 172.21.15.157:0/3934322537 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7efcd8069320 0x7efcd806b7d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7efcec0139e0 tx=0x7efcec009040 comp rx=0 tx=0).stop 2024-04-22T23:28:05.380 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:28:05.380+0000 7efcfc767700 1 -- 172.21.15.157:0/3934322537 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcf4113890 msgr2=0x7efcf410d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:28:05.380 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:28:05.380+0000 7efcfc767700 1 --2- 172.21.15.157:0/3934322537 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcf4113890 0x7efcf410d2f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efce400b790 tx=0x7efce40091a0 comp rx=0 tx=0).stop 2024-04-22T23:28:05.380 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:28:05.380+0000 7efcfc767700 1 -- 172.21.15.157:0/3934322537 shutdown_connections 2024-04-22T23:28:05.380 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:28:05.380+0000 7efcfc767700 1 --2- 172.21.15.157:0/3934322537 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7efcd8069320 0x7efcd806b7d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:28:05.380 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:28:05.380+0000 7efcfc767700 1 --2- 172.21.15.157:0/3934322537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efce0003580 0x7efcf4113350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:28:05.380 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:28:05.380+0000 7efcfc767700 1 --2- 172.21.15.157:0/3934322537 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcf4113890 0x7efcf410d2f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:28:05.380 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:28:05.380+0000 7efcfc767700 1 -- 172.21.15.157:0/3934322537 >> 172.21.15.157:0/3934322537 conn(0x7efcf40f8720 msgr2=0x7efcf40fa150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:28:05.380 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:28:05.380+0000 7efcfc767700 1 -- 172.21.15.157:0/3934322537 shutdown_connections 2024-04-22T23:28:05.381 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:28:05.380+0000 7efcfc767700 1 -- 172.21.15.157:0/3934322537 wait complete. 2024-04-22T23:28:05.381 INFO:teuthology.orchestra.run.smithi157.stderr:dumped monmap epoch 2 2024-04-22T23:28:06.136 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:05 smithi157 ceph-mon[33877]: from='client.? 172.21.15.157:0/3934322537' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T23:28:06.184 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-22T23:28:06.185 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-22T23:28:06.211 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:05 smithi107 ceph-mon[26355]: from='client.? 172.21.15.157:0/3934322537' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T23:28:06.504 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:28:07.488 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:07 smithi107 ceph-mon[26355]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:28:07.489 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:07 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:07.489 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:07 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:07.489 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:07 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:28:07.489 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:07 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:28:07.489 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:07 smithi107 ceph-mon[26355]: Updating smithi107:/etc/ceph/ceph.conf 2024-04-22T23:28:07.489 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:07 smithi107 ceph-mon[26355]: Updating smithi157:/etc/ceph/ceph.conf 2024-04-22T23:28:07.489 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:07 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:07.489 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:07 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:07.489 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:07 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:07.489 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:07 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:07.489 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:07 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:07.489 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:07 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T23:28:07.489 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:07 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-22T23:28:07.490 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:07 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:28:07.636 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:07 smithi157 ceph-mon[33877]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:28:07.637 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:07 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:07.637 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:07 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:07.637 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:07 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:28:07.637 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:07 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:28:07.637 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:07 smithi157 ceph-mon[33877]: Updating smithi107:/etc/ceph/ceph.conf 2024-04-22T23:28:07.637 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:07 smithi157 ceph-mon[33877]: Updating smithi157:/etc/ceph/ceph.conf 2024-04-22T23:28:07.637 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:07 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:07.637 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:07 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:07.637 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:07 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:07.637 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:07 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:07.637 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:07 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:07.637 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:07 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T23:28:07.637 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:07 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-22T23:28:07.637 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:07 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:28:08.636 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:08 smithi157 ceph-mon[33877]: Updating smithi107:/var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T23:28:08.636 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:08 smithi157 ceph-mon[33877]: Updating smithi157:/var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T23:28:08.636 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:08 smithi157 ceph-mon[33877]: Reconfiguring mon.smithi107 (unknown last config time)... 2024-04-22T23:28:08.636 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:08 smithi157 ceph-mon[33877]: Reconfiguring daemon mon.smithi107 on smithi107 2024-04-22T23:28:08.664 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:08 smithi107 ceph-mon[26355]: Updating smithi107:/var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T23:28:08.664 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:08 smithi107 ceph-mon[26355]: Updating smithi157:/var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T23:28:08.664 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:08 smithi107 ceph-mon[26355]: Reconfiguring mon.smithi107 (unknown last config time)... 2024-04-22T23:28:08.664 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:08 smithi107 ceph-mon[26355]: Reconfiguring daemon mon.smithi107 on smithi107 2024-04-22T23:28:09.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:09.193+0000 7f1921228700 1 -- 172.21.15.107:0/2725627189 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f191c102e20 msgr2=0x7f191c1031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:28:09.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:09.193+0000 7f1921228700 1 --2- 172.21.15.107:0/2725627189 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f191c102e20 0x7f191c1031f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f190c004940 tx=0x7f190c009e30 comp rx=0 tx=0).stop 2024-04-22T23:28:09.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:09.193+0000 7f1921228700 1 -- 172.21.15.107:0/2725627189 shutdown_connections 2024-04-22T23:28:09.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:09.193+0000 7f1921228700 1 --2- 172.21.15.107:0/2725627189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f191c1037c0 0x7f191c0fce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:28:09.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:09.193+0000 7f1921228700 1 --2- 172.21.15.107:0/2725627189 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f191c102e20 0x7f191c1031f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:28:09.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:09.193+0000 7f1921228700 1 -- 172.21.15.107:0/2725627189 >> 172.21.15.107:0/2725627189 conn(0x7f191c0f89b0 msgr2=0x7f191c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:28:09.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:09.193+0000 7f1921228700 1 -- 172.21.15.107:0/2725627189 shutdown_connections 2024-04-22T23:28:09.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:09.193+0000 7f1921228700 1 -- 172.21.15.107:0/2725627189 wait complete. 2024-04-22T23:28:09.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:09.193+0000 7f1921228700 1 Processor -- start 2024-04-22T23:28:09.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:09.193+0000 7f1921228700 1 -- start start 2024-04-22T23:28:09.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:09.194+0000 7f1921228700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f191c102e20 0x7f191c113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:28:09.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:09.194+0000 7f1921228700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f191c1037c0 0x7f191c113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:28:09.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:09.194+0000 7f1921228700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f191c114590 con 0x7f191c102e20 2024-04-22T23:28:09.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:09.194+0000 7f1921228700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f191c10db00 con 0x7f191c1037c0 2024-04-22T23:28:09.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:09.194+0000 7f191ad9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f191c102e20 0x7f191c113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:28:09.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:09.194+0000 7f191ad9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f191c102e20 0x7f191c113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42078/0 (socket says 172.21.15.107:42078) 2024-04-22T23:28:09.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:09.194+0000 7f191ad9d700 1 -- 172.21.15.107:0/655984178 learned_addr learned my addr 172.21.15.107:0/655984178 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:28:09.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:09.194+0000 7f191259c700 1 --2- 172.21.15.107:0/655984178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f191c1037c0 0x7f191c113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:28:09.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:09.194+0000 7f191ad9d700 1 -- 172.21.15.107:0/655984178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f191c1037c0 msgr2=0x7f191c113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:28:09.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:09.194+0000 7f191ad9d700 1 --2- 172.21.15.107:0/655984178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f191c1037c0 0x7f191c113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:28:09.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:09.194+0000 7f191ad9d700 1 -- 172.21.15.107:0/655984178 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f190c005040 con 0x7f191c102e20 2024-04-22T23:28:09.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:09.194+0000 7f191ad9d700 1 --2- 172.21.15.107:0/655984178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f191c102e20 0x7f191c113970 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f190c009180 tx=0x7f190c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:28:09.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:09.194+0000 7f1913fff700 1 -- 172.21.15.107:0/655984178 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f190c00e040 con 0x7f191c102e20 2024-04-22T23:28:09.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:09.194+0000 7f1913fff700 1 -- 172.21.15.107:0/655984178 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f190c013420 con 0x7f191c102e20 2024-04-22T23:28:09.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:09.194+0000 7f1921228700 1 -- 172.21.15.107:0/655984178 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f191c10dd80 con 0x7f191c102e20 2024-04-22T23:28:09.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:09.194+0000 7f1913fff700 1 -- 172.21.15.107:0/655984178 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f190c01c630 con 0x7f191c102e20 2024-04-22T23:28:09.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:09.194+0000 7f1921228700 1 -- 172.21.15.107:0/655984178 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f191c10e340 con 0x7f191c102e20 2024-04-22T23:28:09.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:09.195+0000 7f1921228700 1 -- 172.21.15.107:0/655984178 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f191c0fe570 con 0x7f191c102e20 2024-04-22T23:28:09.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:09.196+0000 7f1913fff700 1 -- 172.21.15.107:0/655984178 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 86575+0+0 (secure 0 0 0) 0x7f190c01c870 con 0x7f191c102e20 2024-04-22T23:28:09.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:09.196+0000 7f1913fff700 1 --2- 172.21.15.107:0/655984178 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f1908069210 0x7f190806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:28:09.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:09.196+0000 7f1913fff700 1 -- 172.21.15.107:0/655984178 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f190c087260 con 0x7f191c102e20 2024-04-22T23:28:09.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:09.196+0000 7f191259c700 1 --2- 172.21.15.107:0/655984178 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f1908069210 0x7f190806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:28:09.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:09.196+0000 7f191259c700 1 --2- 172.21.15.107:0/655984178 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f1908069210 0x7f190806b6c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f191c10ecc0 tx=0x7f19040091a0 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:28:09.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:09.201+0000 7f1913fff700 1 -- 172.21.15.107:0/655984178 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f190c058100 con 0x7f191c102e20 2024-04-22T23:28:09.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:09.390+0000 7f1921228700 1 -- 172.21.15.107:0/655984178 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f191c109f90 con 0x7f191c102e20 2024-04-22T23:28:09.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:09.391+0000 7f1913fff700 1 -- 172.21.15.107:0/655984178 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v10) v1 ==== 76+0+227 (secure 0 0 0) 0x7f190c086a10 con 0x7f191c102e20 2024-04-22T23:28:09.391 INFO:teuthology.orchestra.run.smithi107.stdout:# minimal ceph.conf for 8252f6f2-00ff-11ef-bc93-c7b262605968 2024-04-22T23:28:09.391 INFO:teuthology.orchestra.run.smithi107.stdout:[global] 2024-04-22T23:28:09.391 INFO:teuthology.orchestra.run.smithi107.stdout: fsid = 8252f6f2-00ff-11ef-bc93-c7b262605968 2024-04-22T23:28:09.391 INFO:teuthology.orchestra.run.smithi107.stdout: mon_host = [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] 2024-04-22T23:28:09.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:09.393+0000 7f1921228700 1 -- 172.21.15.107:0/655984178 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f1908069210 msgr2=0x7f190806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:28:09.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:09.393+0000 7f1921228700 1 --2- 172.21.15.107:0/655984178 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f1908069210 0x7f190806b6c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f191c10ecc0 tx=0x7f19040091a0 comp rx=0 tx=0).stop 2024-04-22T23:28:09.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:09.393+0000 7f1921228700 1 -- 172.21.15.107:0/655984178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f191c102e20 msgr2=0x7f191c113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:28:09.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:09.393+0000 7f1921228700 1 --2- 172.21.15.107:0/655984178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f191c102e20 0x7f191c113970 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f190c009180 tx=0x7f190c009e30 comp rx=0 tx=0).stop 2024-04-22T23:28:09.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:09.393+0000 7f1921228700 1 -- 172.21.15.107:0/655984178 shutdown_connections 2024-04-22T23:28:09.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:09.393+0000 7f1921228700 1 --2- 172.21.15.107:0/655984178 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f1908069210 0x7f190806b6c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:28:09.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:09.393+0000 7f1921228700 1 --2- 172.21.15.107:0/655984178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f191c102e20 0x7f191c113970 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:28:09.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:09.393+0000 7f1921228700 1 --2- 172.21.15.107:0/655984178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f191c1037c0 0x7f191c113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:28:09.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:09.393+0000 7f1921228700 1 -- 172.21.15.107:0/655984178 >> 172.21.15.107:0/655984178 conn(0x7f191c0f89b0 msgr2=0x7f191c0f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:28:09.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:09.393+0000 7f1921228700 1 -- 172.21.15.107:0/655984178 shutdown_connections 2024-04-22T23:28:09.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:09.393+0000 7f1921228700 1 -- 172.21.15.107:0/655984178 wait complete. 2024-04-22T23:28:09.669 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:09 smithi107 ceph-mon[26355]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:28:09.670 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:09 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:09.670 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:09 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:09.670 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:09 smithi107 ceph-mon[26355]: Reconfiguring mgr.smithi107.uipxvm (unknown last config time)... 2024-04-22T23:28:09.670 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:09 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi107.uipxvm", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T23:28:09.670 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:09 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T23:28:09.670 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:09 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:28:09.670 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:09 smithi107 ceph-mon[26355]: Reconfiguring daemon mgr.smithi107.uipxvm on smithi107 2024-04-22T23:28:09.670 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:09 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:09.670 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:09 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/655984178' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:28:09.802 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:09 smithi157 ceph-mon[33877]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:28:09.802 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:09 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:09.802 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:09 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:09.802 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:09 smithi157 ceph-mon[33877]: Reconfiguring mgr.smithi107.uipxvm (unknown last config time)... 2024-04-22T23:28:09.803 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:09 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi107.uipxvm", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T23:28:09.803 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:09 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T23:28:09.803 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:09 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:28:09.803 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:09 smithi157 ceph-mon[33877]: Reconfiguring daemon mgr.smithi107.uipxvm on smithi107 2024-04-22T23:28:09.803 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:09 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:09.803 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:09 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/655984178' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:28:10.427 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-22T23:28:10.428 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-22T23:28:10.428 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/ceph.conf 2024-04-22T23:28:10.495 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-22T23:28:10.495 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-22T23:28:10.559 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-22T23:28:10.559 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/ceph/ceph.conf 2024-04-22T23:28:10.585 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-22T23:28:10.586 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-22T23:28:10.649 INFO:tasks.cephadm:Deploying OSDs... 2024-04-22T23:28:10.649 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-22T23:28:10.649 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/scratch_devs of=/dev/stdout 2024-04-22T23:28:10.664 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-22T23:28:10.665 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_1 2024-04-22T23:28:10.719 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-22T23:28:10.719 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T23:28:10.720 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-22T23:28:10.720 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T23:28:10.720 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T23:28:10.720 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-04-22 23:27:25.769494982 +0000 2024-04-22T23:28:10.720 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-04-22 23:22:02.657079725 +0000 2024-04-22T23:28:10.720 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-04-22 23:22:02.657079725 +0000 2024-04-22T23:28:10.720 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-04-22 23:22:02.657079725 +0000 2024-04-22T23:28:10.720 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-22T23:28:10.782 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-04-22T23:28:10.782 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-04-22T23:28:10.782 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000211676 s, 2.4 MB/s 2024-04-22T23:28:10.784 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-22T23:28:10.839 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_2 2024-04-22T23:28:10.893 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-22T23:28:10.893 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T23:28:10.893 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-22T23:28:10.893 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T23:28:10.893 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T23:28:10.894 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-04-22 23:27:25.769494982 +0000 2024-04-22T23:28:10.894 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-04-22 23:22:03.022073756 +0000 2024-04-22T23:28:10.894 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-04-22 23:22:03.022073756 +0000 2024-04-22T23:28:10.894 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-04-22 23:22:03.022073756 +0000 2024-04-22T23:28:10.894 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-22T23:28:10.956 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-04-22T23:28:10.956 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-04-22T23:28:10.956 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000213259 s, 2.4 MB/s 2024-04-22T23:28:10.957 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-22T23:28:11.013 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_3 2024-04-22T23:28:11.068 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-22T23:28:11.068 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T23:28:11.068 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-22T23:28:11.068 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T23:28:11.068 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T23:28:11.068 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-04-22 23:27:25.769494982 +0000 2024-04-22T23:28:11.068 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-04-22 23:22:03.371068048 +0000 2024-04-22T23:28:11.068 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-04-22 23:22:03.371068048 +0000 2024-04-22T23:28:11.068 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-04-22 23:22:03.371068048 +0000 2024-04-22T23:28:11.069 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-22T23:28:11.132 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-04-22T23:28:11.132 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-04-22T23:28:11.132 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000246188 s, 2.1 MB/s 2024-04-22T23:28:11.133 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-22T23:28:11.189 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_4 2024-04-22T23:28:11.244 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-22T23:28:11.244 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T23:28:11.244 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-22T23:28:11.244 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T23:28:11.244 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T23:28:11.244 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-04-22 23:27:25.770494966 +0000 2024-04-22T23:28:11.244 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-04-22 23:22:03.700062668 +0000 2024-04-22T23:28:11.244 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-04-22 23:22:03.700062668 +0000 2024-04-22T23:28:11.244 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-04-22 23:22:03.700062668 +0000 2024-04-22T23:28:11.245 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-22T23:28:11.306 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-04-22T23:28:11.306 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-04-22T23:28:11.306 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000224024 s, 2.3 MB/s 2024-04-22T23:28:11.307 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-22T23:28:11.363 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-22T23:28:11.363 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/scratch_devs of=/dev/stdout 2024-04-22T23:28:11.380 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-22T23:28:11.380 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_1 2024-04-22T23:28:11.435 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-22T23:28:11.435 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T23:28:11.435 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-04-22T23:28:11.435 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T23:28:11.435 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T23:28:11.435 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-04-22 23:27:56.801001329 +0000 2024-04-22T23:28:11.435 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-04-22 23:22:02.643019055 +0000 2024-04-22T23:28:11.436 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-04-22 23:22:02.643019055 +0000 2024-04-22T23:28:11.436 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-04-22 23:22:02.643019055 +0000 2024-04-22T23:28:11.436 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-22T23:28:11.497 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-04-22T23:28:11.498 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-04-22T23:28:11.498 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000117477 s, 4.4 MB/s 2024-04-22T23:28:11.499 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-22T23:28:11.554 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_2 2024-04-22T23:28:11.597 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:11 smithi107 ceph-mon[26355]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:28:11.597 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:11 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:11.597 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:11 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:11.597 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:11 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi107", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-22T23:28:11.597 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:11 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:28:11.610 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-22T23:28:11.610 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T23:28:11.610 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-22T23:28:11.610 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T23:28:11.610 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T23:28:11.610 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-04-22 23:27:56.802001311 +0000 2024-04-22T23:28:11.610 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-04-22 23:22:02.974013072 +0000 2024-04-22T23:28:11.610 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-04-22 23:22:02.974013072 +0000 2024-04-22T23:28:11.610 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-04-22 23:22:02.974013072 +0000 2024-04-22T23:28:11.611 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-22T23:28:11.670 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:11 smithi157 ceph-mon[33877]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:28:11.670 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:11 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:11.670 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:11 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:11.670 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:11 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi107", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-22T23:28:11.670 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:11 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:28:11.672 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-04-22T23:28:11.672 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-04-22T23:28:11.672 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 8.3591e-05 s, 6.1 MB/s 2024-04-22T23:28:11.674 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-22T23:28:11.729 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_3 2024-04-22T23:28:11.785 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-22T23:28:11.785 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T23:28:11.785 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-22T23:28:11.785 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T23:28:11.785 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T23:28:11.785 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-04-22 23:27:56.802001311 +0000 2024-04-22T23:28:11.785 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-04-22 23:22:03.300007179 +0000 2024-04-22T23:28:11.785 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-04-22 23:22:03.300007179 +0000 2024-04-22T23:28:11.785 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-04-22 23:22:03.300007179 +0000 2024-04-22T23:28:11.786 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-22T23:28:11.846 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-04-22T23:28:11.847 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-04-22T23:28:11.847 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000105177 s, 4.9 MB/s 2024-04-22T23:28:11.848 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-22T23:28:11.903 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_4 2024-04-22T23:28:11.959 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-22T23:28:11.959 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T23:28:11.959 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-04-22T23:28:11.959 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T23:28:11.959 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T23:28:11.959 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-04-22 23:27:56.802001311 +0000 2024-04-22T23:28:11.959 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-04-22 23:22:03.628001250 +0000 2024-04-22T23:28:11.959 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-04-22 23:22:03.628001250 +0000 2024-04-22T23:28:11.959 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-04-22 23:22:03.628001250 +0000 2024-04-22T23:28:11.960 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-22T23:28:12.020 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-04-22T23:28:12.021 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-04-22T23:28:12.021 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000111761 s, 4.6 MB/s 2024-04-22T23:28:12.022 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-22T23:28:12.077 INFO:tasks.cephadm:Deploying osd.0 on smithi107 with /dev/vg_nvme/lv_4... 2024-04-22T23:28:12.077 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-22T23:28:12.349 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:28:12.728 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:12 smithi107 ceph-mon[26355]: Reconfiguring crash.smithi107 (monmap changed)... 2024-04-22T23:28:12.729 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:12 smithi107 ceph-mon[26355]: Reconfiguring daemon crash.smithi107 on smithi107 2024-04-22T23:28:12.729 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:12 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:12.729 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:12 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:12.886 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:12 smithi157 ceph-mon[33877]: Reconfiguring crash.smithi107 (monmap changed)... 2024-04-22T23:28:12.886 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:12 smithi157 ceph-mon[33877]: Reconfiguring daemon crash.smithi107 on smithi107 2024-04-22T23:28:12.886 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:12 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:12.886 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:12 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:13.438 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:13 smithi107 ceph-mon[26355]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:28:13.438 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:13 smithi107 ceph-mon[26355]: Reconfiguring alertmanager.smithi107 (dependencies changed)... 2024-04-22T23:28:13.438 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:13 smithi107 ceph-mon[26355]: Reconfiguring daemon alertmanager.smithi107 on smithi107 2024-04-22T23:28:13.886 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:13 smithi157 ceph-mon[33877]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:28:13.886 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:13 smithi157 ceph-mon[33877]: Reconfiguring alertmanager.smithi107 (dependencies changed)... 2024-04-22T23:28:13.886 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:13 smithi157 ceph-mon[33877]: Reconfiguring daemon alertmanager.smithi107 on smithi107 2024-04-22T23:28:15.738 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:15 smithi107 ceph-mon[26355]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:28:15.886 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:15 smithi157 ceph-mon[33877]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:28:16.287 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-22T23:28:16.287 INFO:teuthology.orchestra.run.smithi107.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-22T23:28:16.288 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10+0 records in 2024-04-22T23:28:16.288 INFO:teuthology.orchestra.run.smithi107.stderr:10+0 records out 2024-04-22T23:28:16.288 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113999 s, 920 MB/s 2024-04-22T23:28:16.288 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping successful for: 2024-04-22T23:28:17.054 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi107:vg_nvme/lv_4 2024-04-22T23:28:17.610 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:17 smithi107 ceph-mon[26355]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:28:17.886 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:17 smithi157 ceph-mon[33877]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:28:18.115 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:28:19.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:19.613+0000 7f27ddb5f700 1 -- 172.21.15.107:0/2358915557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27d80fe050 msgr2=0x7f27d80fe4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:28:19.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:19.613+0000 7f27ddb5f700 1 --2- 172.21.15.107:0/2358915557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27d80fe050 0x7f27d80fe4c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f27cc004940 tx=0x7f27cc009e30 comp rx=0 tx=0).stop 2024-04-22T23:28:19.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:19.613+0000 7f27ddb5f700 1 -- 172.21.15.107:0/2358915557 shutdown_connections 2024-04-22T23:28:19.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:19.613+0000 7f27ddb5f700 1 --2- 172.21.15.107:0/2358915557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27d80fe050 0x7f27d80fe4c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:28:19.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:19.613+0000 7f27ddb5f700 1 --2- 172.21.15.107:0/2358915557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27d80fce50 0x7f27d80fd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:28:19.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:19.613+0000 7f27ddb5f700 1 -- 172.21.15.107:0/2358915557 >> 172.21.15.107:0/2358915557 conn(0x7f27d80f8580 msgr2=0x7f27d80fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:28:19.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:19.613+0000 7f27ddb5f700 1 -- 172.21.15.107:0/2358915557 shutdown_connections 2024-04-22T23:28:19.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:19.613+0000 7f27ddb5f700 1 -- 172.21.15.107:0/2358915557 wait complete. 2024-04-22T23:28:19.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:19.614+0000 7f27ddb5f700 1 Processor -- start 2024-04-22T23:28:19.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:19.614+0000 7f27ddb5f700 1 -- start start 2024-04-22T23:28:19.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:19.614+0000 7f27ddb5f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27d80fce50 0x7f27d810da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:28:19.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:19.614+0000 7f27ddb5f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27d80fe050 0x7f27d810dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:28:19.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:19.614+0000 7f27ddb5f700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d810e660 con 0x7f27d80fe050 2024-04-22T23:28:19.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:19.614+0000 7f27ddb5f700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d810ee50 con 0x7f27d80fce50 2024-04-22T23:28:19.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:19.614+0000 7f27d6ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27d80fe050 0x7f27d810dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:28:19.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:19.614+0000 7f27d6ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27d80fe050 0x7f27d810dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51828/0 (socket says 172.21.15.107:51828) 2024-04-22T23:28:19.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:19.614+0000 7f27d6ffd700 1 -- 172.21.15.107:0/2552550631 learned_addr learned my addr 172.21.15.107:0/2552550631 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:28:19.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:19.614+0000 7f27d77fe700 1 --2- 172.21.15.107:0/2552550631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27d80fce50 0x7f27d810da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:28:19.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:19.614+0000 7f27d6ffd700 1 -- 172.21.15.107:0/2552550631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27d80fce50 msgr2=0x7f27d810da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:28:19.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:19.614+0000 7f27d6ffd700 1 --2- 172.21.15.107:0/2552550631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27d80fce50 0x7f27d810da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:28:19.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:19.614+0000 7f27d6ffd700 1 -- 172.21.15.107:0/2552550631 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27cc005040 con 0x7f27d80fe050 2024-04-22T23:28:19.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:19.615+0000 7f27d6ffd700 1 --2- 172.21.15.107:0/2552550631 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27d80fe050 0x7f27d810dfd0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f27cc00b3b0 tx=0x7f27cc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:28:19.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:19.615+0000 7f27bffff700 1 -- 172.21.15.107:0/2552550631 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27cc00e040 con 0x7f27d80fe050 2024-04-22T23:28:19.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:19.615+0000 7f27bffff700 1 -- 172.21.15.107:0/2552550631 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f27cc00bcc0 con 0x7f27d80fe050 2024-04-22T23:28:19.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:19.615+0000 7f27bffff700 1 -- 172.21.15.107:0/2552550631 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27cc01c630 con 0x7f27d80fe050 2024-04-22T23:28:19.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:19.615+0000 7f27ddb5f700 1 -- 172.21.15.107:0/2552550631 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27d810f0d0 con 0x7f27d80fe050 2024-04-22T23:28:19.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:19.615+0000 7f27ddb5f700 1 -- 172.21.15.107:0/2552550631 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27d810f690 con 0x7f27d80fe050 2024-04-22T23:28:19.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:19.616+0000 7f27ddb5f700 1 -- 172.21.15.107:0/2552550631 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27d805f5f0 con 0x7f27d80fe050 2024-04-22T23:28:19.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:19.616+0000 7f27bffff700 1 -- 172.21.15.107:0/2552550631 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 86575+0+0 (secure 0 0 0) 0x7f27cc013980 con 0x7f27d80fe050 2024-04-22T23:28:19.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:19.617+0000 7f27bffff700 1 --2- 172.21.15.107:0/2552550631 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f27c0069210 0x7f27c006b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:28:19.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:19.617+0000 7f27d77fe700 1 --2- 172.21.15.107:0/2552550631 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f27c0069210 0x7f27c006b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:28:19.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:19.617+0000 7f27bffff700 1 -- 172.21.15.107:0/2552550631 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f27cc086ea0 con 0x7f27d80fe050 2024-04-22T23:28:19.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:19.617+0000 7f27d77fe700 1 --2- 172.21.15.107:0/2552550631 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f27c0069210 0x7f27c006b6c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f27c8006fd0 tx=0x7f27c8008040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:28:19.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:19.621+0000 7f27bffff700 1 -- 172.21.15.107:0/2552550631 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f27cc057cc0 con 0x7f27d80fe050 2024-04-22T23:28:19.738 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:19 smithi107 ceph-mon[26355]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:28:19.738 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:19 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:28:19.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:19.827+0000 7f27ddb5f700 1 -- 172.21.15.107:0/2552550631 --> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f27d810f920 con 0x7f27c0069210 2024-04-22T23:28:19.886 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:19 smithi157 ceph-mon[33877]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:28:19.886 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:19 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:28:20.446 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:20 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T23:28:20.446 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:20 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T23:28:20.446 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:20 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:28:20.446 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:20 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:20.446 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:20 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:20.636 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:20 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T23:28:20.636 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:20 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T23:28:20.636 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:20 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:28:20.636 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:20 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:20.636 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:20 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:21.605 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:21 smithi107 ceph-mon[26355]: from='client.14242 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:28:21.605 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:21 smithi107 ceph-mon[26355]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:28:21.605 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:21 smithi107 ceph-mon[26355]: Reconfiguring grafana.smithi107 (dependencies changed)... 2024-04-22T23:28:21.606 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:21 smithi107 ceph-mon[26355]: Reconfiguring daemon grafana.smithi107 on smithi107 2024-04-22T23:28:21.886 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:21 smithi157 ceph-mon[33877]: from='client.14242 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:28:21.886 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:21 smithi157 ceph-mon[33877]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:28:21.886 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:21 smithi157 ceph-mon[33877]: Reconfiguring grafana.smithi107 (dependencies changed)... 2024-04-22T23:28:21.886 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:21 smithi157 ceph-mon[33877]: Reconfiguring daemon grafana.smithi107 on smithi107 2024-04-22T23:28:23.616 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:23 smithi107 ceph-mon[26355]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:28:23.886 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:23 smithi157 ceph-mon[33877]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:28:25.488 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:25 smithi107 ceph-mon[26355]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:28:25.488 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:25 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:25.488 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:25 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:25.886 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:25 smithi157 ceph-mon[33877]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:28:25.886 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:25 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:25.886 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:25 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:26.724 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:26 smithi107 ceph-mon[26355]: Reconfiguring prometheus.smithi107 (dependencies changed)... 2024-04-22T23:28:26.724 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:26 smithi107 ceph-mon[26355]: Reconfiguring daemon prometheus.smithi107 on smithi107 2024-04-22T23:28:26.724 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:26 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/526778212' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65404290-734a-4901-bbb4-a87d3d846b06"}]: dispatch 2024-04-22T23:28:26.724 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:26 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/526778212' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "65404290-734a-4901-bbb4-a87d3d846b06"}]': finished 2024-04-22T23:28:26.724 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:26 smithi107 ceph-mon[26355]: osdmap e6: 1 total, 0 up, 1 in 2024-04-22T23:28:26.724 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:26 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T23:28:26.886 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:26 smithi157 ceph-mon[33877]: Reconfiguring prometheus.smithi107 (dependencies changed)... 2024-04-22T23:28:26.886 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:26 smithi157 ceph-mon[33877]: Reconfiguring daemon prometheus.smithi107 on smithi107 2024-04-22T23:28:26.886 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:26 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/526778212' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65404290-734a-4901-bbb4-a87d3d846b06"}]: dispatch 2024-04-22T23:28:26.887 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:26 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/526778212' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "65404290-734a-4901-bbb4-a87d3d846b06"}]': finished 2024-04-22T23:28:26.887 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:26 smithi157 ceph-mon[33877]: osdmap e6: 1 total, 0 up, 1 in 2024-04-22T23:28:26.887 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:26 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T23:28:27.738 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:27 smithi107 ceph-mon[26355]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:28:27.738 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:27 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/4006050036' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T23:28:27.886 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:27 smithi157 ceph-mon[33877]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:28:27.886 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:27 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/4006050036' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T23:28:29.738 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:29 smithi107 ceph-mon[26355]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:28:29.886 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:29 smithi157 ceph-mon[33877]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:28:31.738 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:31 smithi107 ceph-mon[26355]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:28:31.886 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:31 smithi157 ceph-mon[33877]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:28:33.738 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:33 smithi107 ceph-mon[26355]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:28:33.886 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:33 smithi157 ceph-mon[33877]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:28:34.700 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:34 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:28:34.886 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:34 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:28:35.738 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:35 smithi107 ceph-mon[26355]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:28:35.886 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:35 smithi157 ceph-mon[33877]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:28:37.488 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:37 smithi107 ceph-mon[26355]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:28:37.488 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:37 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:37.488 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:37 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:37.488 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:37 smithi107 ceph-mon[26355]: Reconfiguring crash.smithi157 (monmap changed)... 2024-04-22T23:28:37.489 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:37 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi157", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-22T23:28:37.489 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:37 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:28:37.489 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:37 smithi107 ceph-mon[26355]: Reconfiguring daemon crash.smithi157 on smithi157 2024-04-22T23:28:37.584 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:37 smithi157 ceph-mon[33877]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:28:37.584 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:37 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:37.584 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:37 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:37.584 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:37 smithi157 ceph-mon[33877]: Reconfiguring crash.smithi157 (monmap changed)... 2024-04-22T23:28:37.584 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:37 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi157", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-22T23:28:37.584 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:37 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:28:37.584 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:37 smithi157 ceph-mon[33877]: Reconfiguring daemon crash.smithi157 on smithi157 2024-04-22T23:28:38.981 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:38 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:38.981 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:38 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:38.981 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:38 smithi157 ceph-mon[33877]: Reconfiguring mgr.smithi157.tuylis (monmap changed)... 2024-04-22T23:28:38.981 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:38 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi157.tuylis", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T23:28:38.981 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:38 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T23:28:38.981 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:38 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:28:38.982 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:38 smithi157 ceph-mon[33877]: Reconfiguring daemon mgr.smithi157.tuylis on smithi157 2024-04-22T23:28:38.982 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:38 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-22T23:28:38.982 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:38 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:28:38.982 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:38 smithi157 ceph-mon[33877]: Deploying daemon osd.0 on smithi107 2024-04-22T23:28:38.982 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:38 smithi157 ceph-mon[33877]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:28:38.988 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:38 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:38.988 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:38 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:38.988 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:38 smithi107 ceph-mon[26355]: Reconfiguring mgr.smithi157.tuylis (monmap changed)... 2024-04-22T23:28:38.988 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:38 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi157.tuylis", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T23:28:38.988 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:38 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T23:28:38.988 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:38 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:28:38.988 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:38 smithi107 ceph-mon[26355]: Reconfiguring daemon mgr.smithi157.tuylis on smithi157 2024-04-22T23:28:38.988 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:38 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-22T23:28:38.988 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:38 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:28:38.988 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:38 smithi107 ceph-mon[26355]: Deploying daemon osd.0 on smithi107 2024-04-22T23:28:38.989 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:38 smithi107 ceph-mon[26355]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:28:40.295 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:40 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:40.295 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:40 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:40.295 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:40 smithi157 ceph-mon[33877]: Reconfiguring mon.smithi157 (monmap changed)... 2024-04-22T23:28:40.296 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:40 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T23:28:40.296 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:40 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-22T23:28:40.296 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:40 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:28:40.296 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:40 smithi157 ceph-mon[33877]: Reconfiguring daemon mon.smithi157 on smithi157 2024-04-22T23:28:40.304 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:40 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:40.304 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:40 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:40.304 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:40 smithi107 ceph-mon[26355]: Reconfiguring mon.smithi157 (monmap changed)... 2024-04-22T23:28:40.304 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:40 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T23:28:40.305 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:40 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-22T23:28:40.305 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:40 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:28:40.305 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:40 smithi107 ceph-mon[26355]: Reconfiguring daemon mon.smithi157 on smithi157 2024-04-22T23:28:41.603 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:41 smithi157 ceph-mon[33877]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:28:41.603 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:41 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:41.603 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:41 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:41.603 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:41 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-22T23:28:41.603 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:41 smithi157 ceph-mon[33877]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-22T23:28:41.603 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:41 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi107:9093"}]: dispatch 2024-04-22T23:28:41.603 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:41 smithi157 ceph-mon[33877]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi107:9093"}]: dispatch 2024-04-22T23:28:41.603 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:41 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:41.603 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:41 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-22T23:28:41.603 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:41 smithi157 ceph-mon[33877]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-22T23:28:41.603 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:41 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi107:3000"}]: dispatch 2024-04-22T23:28:41.603 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:41 smithi157 ceph-mon[33877]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi107:3000"}]: dispatch 2024-04-22T23:28:41.604 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:41 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:41.604 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:41 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-22T23:28:41.604 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:41 smithi157 ceph-mon[33877]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-22T23:28:41.604 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:41 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi107:9095"}]: dispatch 2024-04-22T23:28:41.604 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:41 smithi157 ceph-mon[33877]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi107:9095"}]: dispatch 2024-04-22T23:28:41.604 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:41 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:41.604 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:41 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:28:41.632 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:41 smithi107 ceph-mon[26355]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:28:41.632 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:41 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:41.632 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:41 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:41.632 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:41 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-22T23:28:41.632 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:41 smithi107 ceph-mon[26355]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-22T23:28:41.632 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:41 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi107:9093"}]: dispatch 2024-04-22T23:28:41.632 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:41 smithi107 ceph-mon[26355]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi107:9093"}]: dispatch 2024-04-22T23:28:41.633 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:41 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:41.633 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:41 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-22T23:28:41.633 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:41 smithi107 ceph-mon[26355]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-22T23:28:41.633 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:41 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi107:3000"}]: dispatch 2024-04-22T23:28:41.633 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:41 smithi107 ceph-mon[26355]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi107:3000"}]: dispatch 2024-04-22T23:28:41.633 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:41 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:41.633 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:41 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-22T23:28:41.633 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:41 smithi107 ceph-mon[26355]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-22T23:28:41.633 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:41 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi107:9095"}]: dispatch 2024-04-22T23:28:41.633 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:41 smithi107 ceph-mon[26355]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi107:9095"}]: dispatch 2024-04-22T23:28:41.633 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:41 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:41.633 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:41 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:28:43.479 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:43 smithi107 ceph-mon[26355]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:28:43.479 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:43 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:43.479 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:43 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:43.636 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:43 smithi157 ceph-mon[33877]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:28:43.636 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:43 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:43.636 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:43 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:44.987 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:44 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:44.987 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:44 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:44.987 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:44 smithi107 ceph-mon[26355]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:28:45.136 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:44 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:45.136 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:44 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:45.136 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:44 smithi157 ceph-mon[33877]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:28:47.191 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:47 smithi107 ceph-mon[26355]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:28:47.191 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:47 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:47.191 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:47 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:47.261 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:47 smithi157 ceph-mon[33877]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:28:47.261 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:47 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:47.261 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:47 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:48.199 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:48 smithi107 ceph-mon[26355]: from='osd.0 [v2:172.21.15.107:6802/4253936450,v1:172.21.15.107:6803/4253936450]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-22T23:28:48.200 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:48 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:28:48.200 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:48 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:28:48.200 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:48 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:48.200 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:48 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:28:48.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:48.201+0000 7f27bffff700 1 -- 172.21.15.107:0/2552550631 <== mgr.14206 v2:172.21.15.107:6800/2516490111 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f27d810f920 con 0x7f27c0069210 2024-04-22T23:28:48.200 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 0 on host 'smithi107' 2024-04-22T23:28:48.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:48.203+0000 7f27ddb5f700 1 -- 172.21.15.107:0/2552550631 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f27c0069210 msgr2=0x7f27c006b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:28:48.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:48.203+0000 7f27ddb5f700 1 --2- 172.21.15.107:0/2552550631 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f27c0069210 0x7f27c006b6c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f27c8006fd0 tx=0x7f27c8008040 comp rx=0 tx=0).stop 2024-04-22T23:28:48.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:48.203+0000 7f27ddb5f700 1 -- 172.21.15.107:0/2552550631 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27d80fe050 msgr2=0x7f27d810dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:28:48.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:48.203+0000 7f27ddb5f700 1 --2- 172.21.15.107:0/2552550631 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27d80fe050 0x7f27d810dfd0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f27cc00b3b0 tx=0x7f27cc009e30 comp rx=0 tx=0).stop 2024-04-22T23:28:48.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:48.203+0000 7f27ddb5f700 1 -- 172.21.15.107:0/2552550631 shutdown_connections 2024-04-22T23:28:48.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:48.203+0000 7f27ddb5f700 1 --2- 172.21.15.107:0/2552550631 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f27c0069210 0x7f27c006b6c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:28:48.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:48.203+0000 7f27ddb5f700 1 --2- 172.21.15.107:0/2552550631 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27d80fce50 0x7f27d810da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:28:48.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:48.203+0000 7f27ddb5f700 1 --2- 172.21.15.107:0/2552550631 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27d80fe050 0x7f27d810dfd0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:28:48.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:48.203+0000 7f27ddb5f700 1 -- 172.21.15.107:0/2552550631 >> 172.21.15.107:0/2552550631 conn(0x7f27d80f8580 msgr2=0x7f27d80f9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:28:48.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:48.203+0000 7f27ddb5f700 1 -- 172.21.15.107:0/2552550631 shutdown_connections 2024-04-22T23:28:48.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:48.203+0000 7f27ddb5f700 1 -- 172.21.15.107:0/2552550631 wait complete. 2024-04-22T23:28:48.386 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:48 smithi157 ceph-mon[33877]: from='osd.0 [v2:172.21.15.107:6802/4253936450,v1:172.21.15.107:6803/4253936450]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-22T23:28:48.386 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:48 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:28:48.386 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:48 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:28:48.386 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:48 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:48.386 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:48 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:28:48.864 DEBUG:teuthology.orchestra.run.smithi107:osd.0> sudo journalctl -f -n 0 -u ceph-8252f6f2-00ff-11ef-bc93-c7b262605968@osd.0.service 2024-04-22T23:28:48.867 INFO:tasks.cephadm:Deploying osd.1 on smithi107 with /dev/vg_nvme/lv_3... 2024-04-22T23:28:48.867 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-22T23:28:49.238 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:49 smithi107 ceph-mon[26355]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:28:49.238 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:49 smithi107 ceph-mon[26355]: from='osd.0 [v2:172.21.15.107:6802/4253936450,v1:172.21.15.107:6803/4253936450]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-22T23:28:49.238 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:49 smithi107 ceph-mon[26355]: osdmap e7: 1 total, 0 up, 1 in 2024-04-22T23:28:49.238 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:49 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T23:28:49.238 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:49 smithi107 ceph-mon[26355]: from='osd.0 [v2:172.21.15.107:6802/4253936450,v1:172.21.15.107:6803/4253936450]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-04-22T23:28:49.238 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:49 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:49.238 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:49 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:49.238 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:49 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:28:49.238 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:28:49 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-0[44710]: 2024-04-22T23:28:49.013+0000 7f5ec7f98700 -1 osd.0 0 waiting for initial osdmap 2024-04-22T23:28:49.239 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:28:49 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-0[44710]: 2024-04-22T23:28:49.017+0000 7f5ec3594700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T23:28:49.386 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:49 smithi157 ceph-mon[33877]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:28:49.386 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:49 smithi157 ceph-mon[33877]: from='osd.0 [v2:172.21.15.107:6802/4253936450,v1:172.21.15.107:6803/4253936450]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-22T23:28:49.386 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:49 smithi157 ceph-mon[33877]: osdmap e7: 1 total, 0 up, 1 in 2024-04-22T23:28:49.386 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:49 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T23:28:49.386 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:49 smithi157 ceph-mon[33877]: from='osd.0 [v2:172.21.15.107:6802/4253936450,v1:172.21.15.107:6803/4253936450]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-04-22T23:28:49.386 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:49 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:49.386 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:49 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:49.386 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:49 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:28:49.724 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:28:50.121 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:50 smithi107 ceph-mon[26355]: from='osd.0 [v2:172.21.15.107:6802/4253936450,v1:172.21.15.107:6803/4253936450]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-04-22T23:28:50.122 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:50 smithi107 ceph-mon[26355]: osdmap e8: 1 total, 0 up, 1 in 2024-04-22T23:28:50.122 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:50 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T23:28:50.122 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:50 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T23:28:50.122 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:50 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:50.122 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:50 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:50.386 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:50 smithi157 ceph-mon[33877]: from='osd.0 [v2:172.21.15.107:6802/4253936450,v1:172.21.15.107:6803/4253936450]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-04-22T23:28:50.386 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:50 smithi157 ceph-mon[33877]: osdmap e8: 1 total, 0 up, 1 in 2024-04-22T23:28:50.386 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:50 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T23:28:50.386 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:50 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T23:28:50.386 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:50 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:50.386 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:50 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:51.176 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:51 smithi107 ceph-mon[26355]: purged_snaps scrub starts 2024-04-22T23:28:51.176 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:51 smithi107 ceph-mon[26355]: purged_snaps scrub ok 2024-04-22T23:28:51.176 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:51 smithi107 ceph-mon[26355]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:28:51.176 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:51 smithi107 ceph-mon[26355]: osd.0 [v2:172.21.15.107:6802/4253936450,v1:172.21.15.107:6803/4253936450] boot 2024-04-22T23:28:51.176 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:51 smithi107 ceph-mon[26355]: osdmap e9: 1 total, 1 up, 1 in 2024-04-22T23:28:51.177 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:51 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T23:28:51.386 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:51 smithi157 ceph-mon[33877]: purged_snaps scrub starts 2024-04-22T23:28:51.386 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:51 smithi157 ceph-mon[33877]: purged_snaps scrub ok 2024-04-22T23:28:51.386 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:51 smithi157 ceph-mon[33877]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T23:28:51.386 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:51 smithi157 ceph-mon[33877]: osd.0 [v2:172.21.15.107:6802/4253936450,v1:172.21.15.107:6803/4253936450] boot 2024-04-22T23:28:51.386 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:51 smithi157 ceph-mon[33877]: osdmap e9: 1 total, 1 up, 1 in 2024-04-22T23:28:51.387 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:51 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T23:28:51.934 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-22T23:28:51.934 INFO:teuthology.orchestra.run.smithi107.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-22T23:28:51.934 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10+0 records in 2024-04-22T23:28:51.934 INFO:teuthology.orchestra.run.smithi107.stderr:10+0 records out 2024-04-22T23:28:51.934 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.012599 s, 832 MB/s 2024-04-22T23:28:51.934 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping successful for: 2024-04-22T23:28:52.210 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:52 smithi107 ceph-mon[26355]: osdmap e10: 1 total, 1 up, 1 in 2024-04-22T23:28:52.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:52 smithi157 ceph-mon[33877]: osdmap e10: 1 total, 1 up, 1 in 2024-04-22T23:28:53.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:53 smithi157 ceph-mon[33877]: pgmap v37: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:28:53.429 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:53 smithi107 ceph-mon[26355]: pgmap v37: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:28:53.456 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi107:vg_nvme/lv_3 2024-04-22T23:28:53.801 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:28:55.386 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:55 smithi157 ceph-mon[33877]: pgmap v38: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:28:55.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:55 smithi107 ceph-mon[26355]: pgmap v38: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:28:56.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:56.357+0000 7fed6a6b3700 1 -- 172.21.15.107:0/586121228 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed640feda0 msgr2=0x7fed640faee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:28:56.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:56.357+0000 7fed6a6b3700 1 --2- 172.21.15.107:0/586121228 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed640feda0 0x7fed640faee0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fed54004940 tx=0x7fed54009e30 comp rx=0 tx=0).stop 2024-04-22T23:28:56.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:56.357+0000 7fed6a6b3700 1 -- 172.21.15.107:0/586121228 shutdown_connections 2024-04-22T23:28:56.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:56.357+0000 7fed6a6b3700 1 --2- 172.21.15.107:0/586121228 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed640feda0 0x7fed640faee0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:28:56.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:56.357+0000 7fed6a6b3700 1 --2- 172.21.15.107:0/586121228 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed640fe450 0x7fed640fe860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:28:56.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:56.357+0000 7fed6a6b3700 1 -- 172.21.15.107:0/586121228 >> 172.21.15.107:0/586121228 conn(0x7fed640f6c10 msgr2=0x7fed640f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:28:56.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:56.357+0000 7fed6a6b3700 1 -- 172.21.15.107:0/586121228 shutdown_connections 2024-04-22T23:28:56.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:56.357+0000 7fed6a6b3700 1 -- 172.21.15.107:0/586121228 wait complete. 2024-04-22T23:28:56.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:56.358+0000 7fed6a6b3700 1 Processor -- start 2024-04-22T23:28:56.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:56.358+0000 7fed6a6b3700 1 -- start start 2024-04-22T23:28:56.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:56.358+0000 7fed6a6b3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed640fe450 0x7fed6410d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:28:56.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:56.358+0000 7fed6a6b3700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed640feda0 0x7fed6410deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:28:56.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:56.358+0000 7fed6a6b3700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6410e4b0 con 0x7fed640fe450 2024-04-22T23:28:56.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:56.358+0000 7fed6a6b3700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6410e620 con 0x7fed640feda0 2024-04-22T23:28:56.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:56.358+0000 7fed63fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed640fe450 0x7fed6410d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:28:56.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:56.358+0000 7fed63fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed640fe450 0x7fed6410d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53638/0 (socket says 172.21.15.107:53638) 2024-04-22T23:28:56.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:56.358+0000 7fed63fff700 1 -- 172.21.15.107:0/3902145062 learned_addr learned my addr 172.21.15.107:0/3902145062 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:28:56.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:56.359+0000 7fed637fe700 1 --2- 172.21.15.107:0/3902145062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed640feda0 0x7fed6410deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:28:56.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:56.359+0000 7fed63fff700 1 -- 172.21.15.107:0/3902145062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed640feda0 msgr2=0x7fed6410deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:28:56.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:56.359+0000 7fed63fff700 1 --2- 172.21.15.107:0/3902145062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed640feda0 0x7fed6410deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:28:56.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:56.359+0000 7fed63fff700 1 -- 172.21.15.107:0/3902145062 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed54005040 con 0x7fed640fe450 2024-04-22T23:28:56.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:56.359+0000 7fed637fe700 1 --2- 172.21.15.107:0/3902145062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed640feda0 0x7fed6410deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T23:28:56.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:56.359+0000 7fed63fff700 1 --2- 172.21.15.107:0/3902145062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed640fe450 0x7fed6410d970 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fed5800c810 tx=0x7fed58004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:28:56.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:56.359+0000 7fed60ff9700 1 -- 172.21.15.107:0/3902145062 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed58012810 con 0x7fed640fe450 2024-04-22T23:28:56.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:56.359+0000 7fed60ff9700 1 -- 172.21.15.107:0/3902145062 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fed58012e40 con 0x7fed640fe450 2024-04-22T23:28:56.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:56.359+0000 7fed6a6b3700 1 -- 172.21.15.107:0/3902145062 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed64113080 con 0x7fed640fe450 2024-04-22T23:28:56.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:56.359+0000 7fed60ff9700 1 -- 172.21.15.107:0/3902145062 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed5800a370 con 0x7fed640fe450 2024-04-22T23:28:56.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:56.359+0000 7fed6a6b3700 1 -- 172.21.15.107:0/3902145062 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed64113640 con 0x7fed640fe450 2024-04-22T23:28:56.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:56.360+0000 7fed6a6b3700 1 -- 172.21.15.107:0/3902145062 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed6404c710 con 0x7fed640fe450 2024-04-22T23:28:56.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:56.361+0000 7fed60ff9700 1 -- 172.21.15.107:0/3902145062 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 86575+0+0 (secure 0 0 0) 0x7fed5800a610 con 0x7fed640fe450 2024-04-22T23:28:56.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:56.361+0000 7fed60ff9700 1 --2- 172.21.15.107:0/3902145062 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fed4c069210 0x7fed4c06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:28:56.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:56.361+0000 7fed60ff9700 1 -- 172.21.15.107:0/3902145062 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1978+0+0 (secure 0 0 0) 0x7fed58086680 con 0x7fed640fe450 2024-04-22T23:28:56.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:56.361+0000 7fed637fe700 1 --2- 172.21.15.107:0/3902145062 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fed4c069210 0x7fed4c06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:28:56.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:56.362+0000 7fed637fe700 1 --2- 172.21.15.107:0/3902145062 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fed4c069210 0x7fed4c06b6c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fed5400b970 tx=0x7fed54009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:28:56.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:56.366+0000 7fed60ff9700 1 -- 172.21.15.107:0/3902145062 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fed580572c0 con 0x7fed640fe450 2024-04-22T23:28:56.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:28:56.566+0000 7fed6a6b3700 1 -- 172.21.15.107:0/3902145062 --> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fed641025c0 con 0x7fed4c069210 2024-04-22T23:28:56.724 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:56 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:56.724 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:56 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:56.886 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:56 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:56.886 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:56 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:28:57.636 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:57 smithi157 ceph-mon[33877]: pgmap v39: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:28:57.636 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:57 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T23:28:57.636 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:57 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T23:28:57.636 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:57 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:28:57.710 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:57 smithi107 ceph-mon[26355]: pgmap v39: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:28:57.711 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:57 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T23:28:57.711 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:57 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T23:28:57.711 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:57 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:28:58.568 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:58 smithi107 ceph-mon[26355]: from='client.14260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:28:58.886 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:58 smithi157 ceph-mon[33877]: from='client.14260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:28:59.737 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:28:59 smithi107 ceph-mon[26355]: pgmap v40: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:28:59.886 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:28:59 smithi157 ceph-mon[33877]: pgmap v40: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:29:01.009 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:00 smithi107 ceph-mon[26355]: Detected new or changed devices on smithi107 2024-04-22T23:29:01.009 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:00 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:01.009 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:00 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:01.009 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:00 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:29:01.009 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:00 smithi107 ceph-mon[26355]: Adjusting osd_memory_target on smithi107 to 12864M 2024-04-22T23:29:01.009 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:00 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:01.009 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:00 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:29:01.009 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:00 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:29:01.009 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:00 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:01.009 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:00 smithi107 ceph-mon[26355]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:29:01.009 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:00 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:29:01.226 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:00 smithi157 ceph-mon[33877]: Detected new or changed devices on smithi107 2024-04-22T23:29:01.226 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:00 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:01.226 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:00 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:01.227 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:00 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:29:01.227 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:00 smithi157 ceph-mon[33877]: Adjusting osd_memory_target on smithi107 to 12864M 2024-04-22T23:29:01.227 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:00 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:01.227 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:00 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:29:01.227 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:00 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:29:01.227 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:00 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:01.227 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:00 smithi157 ceph-mon[33877]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:29:01.227 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:00 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:29:02.237 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:01 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/3936993458' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6df381f7-b907-4d1b-bd08-fdf34e9729f6"}]: dispatch 2024-04-22T23:29:02.238 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:01 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/3936993458' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6df381f7-b907-4d1b-bd08-fdf34e9729f6"}]': finished 2024-04-22T23:29:02.238 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:01 smithi107 ceph-mon[26355]: osdmap e11: 2 total, 1 up, 2 in 2024-04-22T23:29:02.238 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:01 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T23:29:02.238 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:01 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:02.238 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:01 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:02.238 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:01 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/3934215297' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T23:29:02.386 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:01 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/3936993458' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6df381f7-b907-4d1b-bd08-fdf34e9729f6"}]: dispatch 2024-04-22T23:29:02.386 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:01 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/3936993458' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6df381f7-b907-4d1b-bd08-fdf34e9729f6"}]': finished 2024-04-22T23:29:02.386 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:01 smithi157 ceph-mon[33877]: osdmap e11: 2 total, 1 up, 2 in 2024-04-22T23:29:02.386 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:01 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T23:29:02.386 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:01 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:02.386 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:01 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:02.386 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:01 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/3934215297' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T23:29:03.237 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:02 smithi107 ceph-mon[26355]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:29:03.386 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:02 smithi157 ceph-mon[33877]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:29:04.237 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:03 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:29:04.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:03 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:29:05.413 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:05 smithi107 ceph-mon[26355]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:29:05.413 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:05 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:05.413 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:05 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:05.413 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:05 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:29:05.413 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:05 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:29:05.413 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:05 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:05.636 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:05 smithi157 ceph-mon[33877]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:29:05.636 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:05 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:05.636 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:05 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:05.636 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:05 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:29:05.636 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:05 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:29:05.636 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:05 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:07.288 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:07 smithi107 ceph-mon[26355]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:29:07.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:07 smithi157 ceph-mon[33877]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:29:09.488 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:09 smithi107 ceph-mon[26355]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:29:09.488 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:09 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-22T23:29:09.488 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:09 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:29:09.488 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:09 smithi107 ceph-mon[26355]: Deploying daemon osd.1 on smithi107 2024-04-22T23:29:09.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:09 smithi157 ceph-mon[33877]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:29:09.636 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:09 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-22T23:29:09.636 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:09 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:29:09.636 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:09 smithi157 ceph-mon[33877]: Deploying daemon osd.1 on smithi107 2024-04-22T23:29:11.446 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:11 smithi107 ceph-mon[26355]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:29:11.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:11 smithi157 ceph-mon[33877]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:29:13.494 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:13 smithi107 ceph-mon[26355]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:29:13.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:13 smithi157 ceph-mon[33877]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:29:14.867 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:14 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:14.867 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:14 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:29:14.868 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:14 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:14.868 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:14 smithi107 ceph-mon[26355]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:29:14.885 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:14 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:14.885 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:14 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:29:14.885 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:14 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:14.885 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:14 smithi157 ceph-mon[33877]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:29:16.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:16.369+0000 7fed60ff9700 1 -- 172.21.15.107:0/3902145062 <== mgr.14206 v2:172.21.15.107:6800/2516490111 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fed641025c0 con 0x7fed4c069210 2024-04-22T23:29:16.368 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 1 on host 'smithi107' 2024-04-22T23:29:16.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:16.371+0000 7fed6a6b3700 1 -- 172.21.15.107:0/3902145062 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fed4c069210 msgr2=0x7fed4c06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:29:16.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:16.371+0000 7fed6a6b3700 1 --2- 172.21.15.107:0/3902145062 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fed4c069210 0x7fed4c06b6c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fed5400b970 tx=0x7fed54009e30 comp rx=0 tx=0).stop 2024-04-22T23:29:16.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:16.371+0000 7fed6a6b3700 1 -- 172.21.15.107:0/3902145062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed640fe450 msgr2=0x7fed6410d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:29:16.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:16.371+0000 7fed6a6b3700 1 --2- 172.21.15.107:0/3902145062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed640fe450 0x7fed6410d970 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fed5800c810 tx=0x7fed58004470 comp rx=0 tx=0).stop 2024-04-22T23:29:16.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:16.371+0000 7fed6a6b3700 1 -- 172.21.15.107:0/3902145062 shutdown_connections 2024-04-22T23:29:16.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:16.371+0000 7fed6a6b3700 1 --2- 172.21.15.107:0/3902145062 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fed4c069210 0x7fed4c06b6c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:29:16.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:16.371+0000 7fed6a6b3700 1 --2- 172.21.15.107:0/3902145062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed640fe450 0x7fed6410d970 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:29:16.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:16.371+0000 7fed6a6b3700 1 --2- 172.21.15.107:0/3902145062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed640feda0 0x7fed6410deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:29:16.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:16.372+0000 7fed6a6b3700 1 -- 172.21.15.107:0/3902145062 >> 172.21.15.107:0/3902145062 conn(0x7fed640f6c10 msgr2=0x7fed640f8280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:29:16.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:16.372+0000 7fed6a6b3700 1 -- 172.21.15.107:0/3902145062 shutdown_connections 2024-04-22T23:29:16.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:16.372+0000 7fed6a6b3700 1 -- 172.21.15.107:0/3902145062 wait complete. 2024-04-22T23:29:17.350 DEBUG:teuthology.orchestra.run.smithi107:osd.1> sudo journalctl -f -n 0 -u ceph-8252f6f2-00ff-11ef-bc93-c7b262605968@osd.1.service 2024-04-22T23:29:17.352 INFO:tasks.cephadm:Deploying osd.2 on smithi107 with /dev/vg_nvme/lv_2... 2024-04-22T23:29:17.353 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-22T23:29:17.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:17 smithi157 ceph-mon[33877]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:29:17.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:17 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:17.636 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:17 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:17.688 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:17 smithi107 ceph-mon[26355]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:29:17.688 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:17 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:17.688 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:17 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:17.688 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:29:17 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-1[52832]: 2024-04-22T23:29:17.421+0000 7fc519c1d540 -1 osd.1 0 log_to_monitors true 2024-04-22T23:29:18.116 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:29:18.564 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:18 smithi107 ceph-mon[26355]: from='osd.1 [v2:172.21.15.107:6810/335851250,v1:172.21.15.107:6811/335851250]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-22T23:29:18.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:18 smithi157 ceph-mon[33877]: from='osd.1 [v2:172.21.15.107:6810/335851250,v1:172.21.15.107:6811/335851250]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-22T23:29:19.618 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:29:19 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-1[52832]: 2024-04-22T23:29:19.327+0000 7fc5103ef700 -1 osd.1 0 waiting for initial osdmap 2024-04-22T23:29:19.618 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:29:19 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-1[52832]: 2024-04-22T23:29:19.331+0000 7fc50d1ee700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T23:29:19.618 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:19 smithi107 ceph-mon[26355]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:29:19.618 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:19 smithi107 ceph-mon[26355]: from='osd.1 [v2:172.21.15.107:6810/335851250,v1:172.21.15.107:6811/335851250]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-22T23:29:19.618 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:19 smithi107 ceph-mon[26355]: osdmap e12: 2 total, 1 up, 2 in 2024-04-22T23:29:19.619 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:19 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T23:29:19.619 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:19 smithi107 ceph-mon[26355]: from='osd.1 [v2:172.21.15.107:6810/335851250,v1:172.21.15.107:6811/335851250]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-04-22T23:29:19.619 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:19 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:29:19.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:19 smithi157 ceph-mon[33877]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:29:19.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:19 smithi157 ceph-mon[33877]: from='osd.1 [v2:172.21.15.107:6810/335851250,v1:172.21.15.107:6811/335851250]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-22T23:29:19.636 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:19 smithi157 ceph-mon[33877]: osdmap e12: 2 total, 1 up, 2 in 2024-04-22T23:29:19.636 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:19 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T23:29:19.636 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:19 smithi157 ceph-mon[33877]: from='osd.1 [v2:172.21.15.107:6810/335851250,v1:172.21.15.107:6811/335851250]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-04-22T23:29:19.636 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:19 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:29:20.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:20 smithi157 ceph-mon[33877]: from='osd.1 [v2:172.21.15.107:6810/335851250,v1:172.21.15.107:6811/335851250]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-04-22T23:29:20.386 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:20 smithi157 ceph-mon[33877]: osdmap e13: 2 total, 1 up, 2 in 2024-04-22T23:29:20.386 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:20 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T23:29:20.386 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:20 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T23:29:20.386 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:20 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T23:29:20.737 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:20 smithi107 ceph-mon[26355]: from='osd.1 [v2:172.21.15.107:6810/335851250,v1:172.21.15.107:6811/335851250]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-04-22T23:29:20.737 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:20 smithi107 ceph-mon[26355]: osdmap e13: 2 total, 1 up, 2 in 2024-04-22T23:29:20.737 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:20 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T23:29:20.737 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:20 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T23:29:20.738 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:20 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T23:29:21.361 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-22T23:29:21.361 INFO:teuthology.orchestra.run.smithi107.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-22T23:29:21.361 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10+0 records in 2024-04-22T23:29:21.361 INFO:teuthology.orchestra.run.smithi107.stderr:10+0 records out 2024-04-22T23:29:21.361 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.012497 s, 839 MB/s 2024-04-22T23:29:21.362 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping successful for: 2024-04-22T23:29:21.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:21 smithi107 ceph-mon[26355]: purged_snaps scrub starts 2024-04-22T23:29:21.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:21 smithi107 ceph-mon[26355]: purged_snaps scrub ok 2024-04-22T23:29:21.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:21 smithi107 ceph-mon[26355]: pgmap v54: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:29:21.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:21 smithi107 ceph-mon[26355]: osd.1 [v2:172.21.15.107:6810/335851250,v1:172.21.15.107:6811/335851250] boot 2024-04-22T23:29:21.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:21 smithi107 ceph-mon[26355]: osdmap e14: 2 total, 2 up, 2 in 2024-04-22T23:29:21.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:21 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T23:29:21.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:21 smithi157 ceph-mon[33877]: purged_snaps scrub starts 2024-04-22T23:29:21.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:21 smithi157 ceph-mon[33877]: purged_snaps scrub ok 2024-04-22T23:29:21.636 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:21 smithi157 ceph-mon[33877]: pgmap v54: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-22T23:29:21.636 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:21 smithi157 ceph-mon[33877]: osd.1 [v2:172.21.15.107:6810/335851250,v1:172.21.15.107:6811/335851250] boot 2024-04-22T23:29:21.636 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:21 smithi157 ceph-mon[33877]: osdmap e14: 2 total, 2 up, 2 in 2024-04-22T23:29:21.636 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:21 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T23:29:22.335 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi107:vg_nvme/lv_2 2024-04-22T23:29:22.604 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:29:22.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:22 smithi157 ceph-mon[33877]: osdmap e15: 2 total, 2 up, 2 in 2024-04-22T23:29:22.712 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:22 smithi107 ceph-mon[26355]: osdmap e15: 2 total, 2 up, 2 in 2024-04-22T23:29:23.737 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:23 smithi107 ceph-mon[26355]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-22T23:29:23.737 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:23 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:23.737 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:23 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:23.885 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:23 smithi157 ceph-mon[33877]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-22T23:29:23.885 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:23 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:23.886 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:23 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:24.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:24.042+0000 7f17aee9f700 1 -- 172.21.15.107:0/3342356300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17a80fcfb0 msgr2=0x7f17a80fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:29:24.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:24.042+0000 7f17aee9f700 1 --2- 172.21.15.107:0/3342356300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17a80fcfb0 0x7f17a80fd420 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1794004940 tx=0x7f1794009e30 comp rx=0 tx=0).stop 2024-04-22T23:29:24.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:24.042+0000 7f17aee9f700 1 -- 172.21.15.107:0/3342356300 shutdown_connections 2024-04-22T23:29:24.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:24.042+0000 7f17aee9f700 1 --2- 172.21.15.107:0/3342356300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17a80fcfb0 0x7f17a80fd420 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:29:24.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:24.042+0000 7f17aee9f700 1 --2- 172.21.15.107:0/3342356300 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17a80fbd00 0x7f17a80fc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:29:24.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:24.042+0000 7f17aee9f700 1 -- 172.21.15.107:0/3342356300 >> 172.21.15.107:0/3342356300 conn(0x7f17a80f7480 msgr2=0x7f17a80f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:29:24.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:24.042+0000 7f17aee9f700 1 -- 172.21.15.107:0/3342356300 shutdown_connections 2024-04-22T23:29:24.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:24.043+0000 7f17aee9f700 1 -- 172.21.15.107:0/3342356300 wait complete. 2024-04-22T23:29:24.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:24.043+0000 7f17aee9f700 1 Processor -- start 2024-04-22T23:29:24.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:24.043+0000 7f17aee9f700 1 -- start start 2024-04-22T23:29:24.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:24.043+0000 7f17aee9f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17a80fbd00 0x7f17a810d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:29:24.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:24.043+0000 7f17aee9f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17a80fcfb0 0x7f17a810df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:29:24.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:24.043+0000 7f17aee9f700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a810e500 con 0x7f17a80fcfb0 2024-04-22T23:29:24.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:24.043+0000 7f17aee9f700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a810e670 con 0x7f17a80fbd00 2024-04-22T23:29:24.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:24.043+0000 7f17a7fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17a80fcfb0 0x7f17a810df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:29:24.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:24.043+0000 7f17a7fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17a80fcfb0 0x7f17a810df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34180/0 (socket says 172.21.15.107:34180) 2024-04-22T23:29:24.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:24.043+0000 7f17a7fff700 1 -- 172.21.15.107:0/1436213079 learned_addr learned my addr 172.21.15.107:0/1436213079 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:29:24.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:24.044+0000 7f17acc3b700 1 --2- 172.21.15.107:0/1436213079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17a80fbd00 0x7f17a810d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:29:24.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:24.044+0000 7f17a7fff700 1 -- 172.21.15.107:0/1436213079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17a80fbd00 msgr2=0x7f17a810d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:29:24.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:24.044+0000 7f17a7fff700 1 --2- 172.21.15.107:0/1436213079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17a80fbd00 0x7f17a810d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:29:24.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:24.044+0000 7f17a7fff700 1 -- 172.21.15.107:0/1436213079 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1794005040 con 0x7f17a80fcfb0 2024-04-22T23:29:24.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:24.044+0000 7f17acc3b700 1 --2- 172.21.15.107:0/1436213079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17a80fbd00 0x7f17a810d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T23:29:24.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:24.044+0000 7f17a7fff700 1 --2- 172.21.15.107:0/1436213079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17a80fcfb0 0x7f17a810df00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1794009fd0 tx=0x7f1794009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:29:24.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:24.044+0000 7f17a57fa700 1 -- 172.21.15.107:0/1436213079 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f179400e040 con 0x7f17a80fcfb0 2024-04-22T23:29:24.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:24.044+0000 7f17aee9f700 1 -- 172.21.15.107:0/1436213079 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17a8113070 con 0x7f17a80fcfb0 2024-04-22T23:29:24.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:24.044+0000 7f17a57fa700 1 -- 172.21.15.107:0/1436213079 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f179400bcc0 con 0x7f17a80fcfb0 2024-04-22T23:29:24.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:24.044+0000 7f17a57fa700 1 -- 172.21.15.107:0/1436213079 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f179401c630 con 0x7f17a80fcfb0 2024-04-22T23:29:24.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:24.044+0000 7f17aee9f700 1 -- 172.21.15.107:0/1436213079 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17a8113630 con 0x7f17a80fcfb0 2024-04-22T23:29:24.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:24.045+0000 7f17aee9f700 1 -- 172.21.15.107:0/1436213079 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17a8101b40 con 0x7f17a80fcfb0 2024-04-22T23:29:24.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:24.045+0000 7f17a57fa700 1 -- 172.21.15.107:0/1436213079 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 86575+0+0 (secure 0 0 0) 0x7f1794013980 con 0x7f17a80fcfb0 2024-04-22T23:29:24.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:24.046+0000 7f17a57fa700 1 --2- 172.21.15.107:0/1436213079 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f1798069210 0x7f179806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:29:24.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:24.046+0000 7f17a57fa700 1 -- 172.21.15.107:0/1436213079 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2410+0+0 (secure 0 0 0) 0x7f17940877a0 con 0x7f17a80fcfb0 2024-04-22T23:29:24.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:24.046+0000 7f17acc3b700 1 --2- 172.21.15.107:0/1436213079 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f1798069210 0x7f179806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:29:24.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:24.046+0000 7f17acc3b700 1 --2- 172.21.15.107:0/1436213079 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f1798069210 0x7f179806b6c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f179c0041d0 tx=0x7f179c009040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:29:24.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:24.050+0000 7f17a57fa700 1 -- 172.21.15.107:0/1436213079 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f17940581b0 con 0x7f17a80fcfb0 2024-04-22T23:29:24.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:24.257+0000 7f17aee9f700 1 -- 172.21.15.107:0/1436213079 --> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f17a802b390 con 0x7f1798069210 2024-04-22T23:29:24.737 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:24 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T23:29:24.737 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:24 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T23:29:24.737 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:24 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:29:24.885 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:24 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T23:29:24.885 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:24 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T23:29:24.885 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:24 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:29:25.648 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:25 smithi107 ceph-mon[26355]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-22T23:29:25.648 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:25 smithi107 ceph-mon[26355]: from='client.14278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:29:25.885 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:25 smithi157 ceph-mon[33877]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-22T23:29:25.885 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:25 smithi157 ceph-mon[33877]: from='client.14278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:29:27.636 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:27 smithi157 ceph-mon[33877]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-22T23:29:27.636 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:27 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:27.636 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:27 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:27.636 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:27 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:29:27.636 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:27 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:29:27.636 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:27 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:27.636 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:27 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:29:27.636 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:27 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:29:27.636 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:27 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:27.697 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:27 smithi107 ceph-mon[26355]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-22T23:29:27.697 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:27 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:27.697 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:27 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:27.697 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:27 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:29:27.698 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:27 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:29:27.698 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:27 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:27.698 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:27 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:29:27.698 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:27 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:29:27.698 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:27 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:28.737 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:28 smithi107 ceph-mon[26355]: Detected new or changed devices on smithi107 2024-04-22T23:29:28.738 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:28 smithi107 ceph-mon[26355]: Adjusting osd_memory_target on smithi107 to 6432M 2024-04-22T23:29:28.738 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:28 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:29:28.885 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:28 smithi157 ceph-mon[33877]: Detected new or changed devices on smithi107 2024-04-22T23:29:28.885 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:28 smithi157 ceph-mon[33877]: Adjusting osd_memory_target on smithi107 to 6432M 2024-04-22T23:29:28.885 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:28 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:29:29.885 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:29 smithi157 ceph-mon[33877]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-22T23:29:29.885 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:29 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/3722315906' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c5e5c37-c1de-4e64-9c21-eacc145f1412"}]: dispatch 2024-04-22T23:29:29.886 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:29 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/3722315906' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c5e5c37-c1de-4e64-9c21-eacc145f1412"}]': finished 2024-04-22T23:29:29.886 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:29 smithi157 ceph-mon[33877]: osdmap e16: 3 total, 2 up, 3 in 2024-04-22T23:29:29.886 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:29 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T23:29:29.886 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:29 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/3996862258' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T23:29:29.987 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:29 smithi107 ceph-mon[26355]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-22T23:29:29.987 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:29 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/3722315906' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c5e5c37-c1de-4e64-9c21-eacc145f1412"}]: dispatch 2024-04-22T23:29:29.987 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:29 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/3722315906' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c5e5c37-c1de-4e64-9c21-eacc145f1412"}]': finished 2024-04-22T23:29:29.987 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:29 smithi107 ceph-mon[26355]: osdmap e16: 3 total, 2 up, 3 in 2024-04-22T23:29:29.987 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:29 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T23:29:29.987 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:29 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/3996862258' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T23:29:31.758 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:31 smithi107 ceph-mon[26355]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-22T23:29:31.885 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:31 smithi157 ceph-mon[33877]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-22T23:29:33.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:33 smithi157 ceph-mon[33877]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-22T23:29:33.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:33 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:33.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:33 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:33.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:33 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:29:33.386 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:33 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:29:33.386 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:33 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:33.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:33 smithi107 ceph-mon[26355]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-22T23:29:33.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:33 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:33.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:33 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:33.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:33 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:29:33.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:33 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:29:33.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:33 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:34.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:34 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:29:34.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:34 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:29:35.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:35 smithi157 ceph-mon[33877]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-22T23:29:35.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:35 smithi107 ceph-mon[26355]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-22T23:29:36.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:36 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-22T23:29:36.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:36 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:29:36.449 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:36 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-22T23:29:36.449 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:36 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:29:37.264 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:37 smithi107 ceph-mon[26355]: Deploying daemon osd.2 on smithi107 2024-04-22T23:29:37.264 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:37 smithi107 ceph-mon[26355]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-22T23:29:37.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:37 smithi157 ceph-mon[33877]: Deploying daemon osd.2 on smithi107 2024-04-22T23:29:37.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:37 smithi157 ceph-mon[33877]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-22T23:29:39.173 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:39 smithi107 ceph-mon[26355]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-22T23:29:39.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:39 smithi157 ceph-mon[33877]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-22T23:29:41.301 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:41 smithi107 ceph-mon[26355]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-22T23:29:41.301 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:41 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:41.301 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:41 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:29:41.302 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:41 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:41.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:41 smithi157 ceph-mon[33877]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-22T23:29:41.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:41 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:41.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:41 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:29:41.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:41 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:43.237 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:43 smithi107 ceph-mon[26355]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-22T23:29:43.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:43 smithi157 ceph-mon[33877]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-22T23:29:44.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:44.962+0000 7f17a57fa700 1 -- 172.21.15.107:0/1436213079 <== mgr.14206 v2:172.21.15.107:6800/2516490111 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f17a802b390 con 0x7f1798069210 2024-04-22T23:29:44.960 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 2 on host 'smithi107' 2024-04-22T23:29:44.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:44.965+0000 7f17aee9f700 1 -- 172.21.15.107:0/1436213079 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f1798069210 msgr2=0x7f179806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:29:44.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:44.965+0000 7f17aee9f700 1 --2- 172.21.15.107:0/1436213079 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f1798069210 0x7f179806b6c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f179c0041d0 tx=0x7f179c009040 comp rx=0 tx=0).stop 2024-04-22T23:29:44.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:44.965+0000 7f17aee9f700 1 -- 172.21.15.107:0/1436213079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17a80fcfb0 msgr2=0x7f17a810df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:29:44.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:44.965+0000 7f17aee9f700 1 --2- 172.21.15.107:0/1436213079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17a80fcfb0 0x7f17a810df00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1794009fd0 tx=0x7f1794009e30 comp rx=0 tx=0).stop 2024-04-22T23:29:44.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:44.965+0000 7f17aee9f700 1 -- 172.21.15.107:0/1436213079 shutdown_connections 2024-04-22T23:29:44.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:44.965+0000 7f17aee9f700 1 --2- 172.21.15.107:0/1436213079 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f1798069210 0x7f179806b6c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:29:44.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:44.965+0000 7f17aee9f700 1 --2- 172.21.15.107:0/1436213079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17a80fbd00 0x7f17a810d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:29:44.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:44.965+0000 7f17aee9f700 1 --2- 172.21.15.107:0/1436213079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17a80fcfb0 0x7f17a810df00 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:29:44.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:44.965+0000 7f17aee9f700 1 -- 172.21.15.107:0/1436213079 >> 172.21.15.107:0/1436213079 conn(0x7f17a80f7480 msgr2=0x7f17a80668d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:29:44.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:44.965+0000 7f17aee9f700 1 -- 172.21.15.107:0/1436213079 shutdown_connections 2024-04-22T23:29:44.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:29:44.965+0000 7f17aee9f700 1 -- 172.21.15.107:0/1436213079 wait complete. 2024-04-22T23:29:45.237 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:45 smithi107 ceph-mon[26355]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-22T23:29:45.237 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:45 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:45.237 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:45 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:45.237 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:45 smithi107 ceph-mon[26355]: from='osd.2 [v2:172.21.15.107:6818/750102033,v1:172.21.15.107:6819/750102033]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-22T23:29:45.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:45 smithi157 ceph-mon[33877]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-22T23:29:45.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:45 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:45.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:45 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:45.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:45 smithi157 ceph-mon[33877]: from='osd.2 [v2:172.21.15.107:6818/750102033,v1:172.21.15.107:6819/750102033]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-22T23:29:46.158 DEBUG:teuthology.orchestra.run.smithi107:osd.2> sudo journalctl -f -n 0 -u ceph-8252f6f2-00ff-11ef-bc93-c7b262605968@osd.2.service 2024-04-22T23:29:46.160 INFO:tasks.cephadm:Deploying osd.3 on smithi157 with /dev/vg_nvme/lv_4... 2024-04-22T23:29:46.160 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-22T23:29:46.419 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi157/config 2024-04-22T23:29:47.135 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:46 smithi157 ceph-mon[33877]: from='osd.2 [v2:172.21.15.107:6818/750102033,v1:172.21.15.107:6819/750102033]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-22T23:29:47.135 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:46 smithi157 ceph-mon[33877]: osdmap e17: 3 total, 2 up, 3 in 2024-04-22T23:29:47.135 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:46 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T23:29:47.135 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:46 smithi157 ceph-mon[33877]: from='osd.2 [v2:172.21.15.107:6818/750102033,v1:172.21.15.107:6819/750102033]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-04-22T23:29:47.135 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:46 smithi157 ceph-mon[33877]: pgmap v71: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-22T23:29:47.237 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:46 smithi107 ceph-mon[26355]: from='osd.2 [v2:172.21.15.107:6818/750102033,v1:172.21.15.107:6819/750102033]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-22T23:29:47.237 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:46 smithi107 ceph-mon[26355]: osdmap e17: 3 total, 2 up, 3 in 2024-04-22T23:29:47.237 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:46 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T23:29:47.237 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:46 smithi107 ceph-mon[26355]: from='osd.2 [v2:172.21.15.107:6818/750102033,v1:172.21.15.107:6819/750102033]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-04-22T23:29:47.237 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:46 smithi107 ceph-mon[26355]: pgmap v71: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-22T23:29:47.237 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:29:46 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-2[60154]: 2024-04-22T23:29:46.960+0000 7f439b928700 -1 osd.2 0 waiting for initial osdmap 2024-04-22T23:29:47.237 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:29:46 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-2[60154]: 2024-04-22T23:29:46.972+0000 7f4398727700 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T23:29:48.136 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:47 smithi157 ceph-mon[33877]: from='osd.2 [v2:172.21.15.107:6818/750102033,v1:172.21.15.107:6819/750102033]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-04-22T23:29:48.136 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:47 smithi157 ceph-mon[33877]: osdmap e18: 3 total, 2 up, 3 in 2024-04-22T23:29:48.136 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:47 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T23:29:48.136 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:47 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T23:29:48.136 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:47 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:48.136 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:47 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:48.217 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-22T23:29:48.218 INFO:teuthology.orchestra.run.smithi157.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-22T23:29:48.218 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10+0 records in 2024-04-22T23:29:48.218 INFO:teuthology.orchestra.run.smithi157.stderr:10+0 records out 2024-04-22T23:29:48.218 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00846587 s, 1.2 GB/s 2024-04-22T23:29:48.218 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping successful for: 2024-04-22T23:29:48.237 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:47 smithi107 ceph-mon[26355]: from='osd.2 [v2:172.21.15.107:6818/750102033,v1:172.21.15.107:6819/750102033]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-04-22T23:29:48.237 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:47 smithi107 ceph-mon[26355]: osdmap e18: 3 total, 2 up, 3 in 2024-04-22T23:29:48.237 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:47 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T23:29:48.237 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:47 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T23:29:48.237 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:47 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:48.237 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:47 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:48.877 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi157:vg_nvme/lv_4 2024-04-22T23:29:48.987 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:48 smithi107 ceph-mon[26355]: purged_snaps scrub starts 2024-04-22T23:29:48.987 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:48 smithi107 ceph-mon[26355]: purged_snaps scrub ok 2024-04-22T23:29:48.987 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:48 smithi107 ceph-mon[26355]: pgmap v73: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-22T23:29:48.987 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:48 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T23:29:48.987 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:48 smithi107 ceph-mon[26355]: osd.2 [v2:172.21.15.107:6818/750102033,v1:172.21.15.107:6819/750102033] boot 2024-04-22T23:29:48.987 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:48 smithi107 ceph-mon[26355]: osdmap e19: 3 total, 3 up, 3 in 2024-04-22T23:29:48.987 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:48 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T23:29:48.988 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:48 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-04-22T23:29:49.132 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi157/config 2024-04-22T23:29:49.135 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:48 smithi157 ceph-mon[33877]: purged_snaps scrub starts 2024-04-22T23:29:49.135 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:48 smithi157 ceph-mon[33877]: purged_snaps scrub ok 2024-04-22T23:29:49.135 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:48 smithi157 ceph-mon[33877]: pgmap v73: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-22T23:29:49.135 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:48 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T23:29:49.135 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:48 smithi157 ceph-mon[33877]: osd.2 [v2:172.21.15.107:6818/750102033,v1:172.21.15.107:6819/750102033] boot 2024-04-22T23:29:49.135 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:48 smithi157 ceph-mon[33877]: osdmap e19: 3 total, 3 up, 3 in 2024-04-22T23:29:49.135 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:48 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T23:29:49.135 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:48 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-04-22T23:29:50.237 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:49 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-04-22T23:29:50.237 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:49 smithi107 ceph-mon[26355]: osdmap e20: 3 total, 3 up, 3 in 2024-04-22T23:29:50.237 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:49 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-22T23:29:50.237 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:49 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:29:50.264 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:49 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-04-22T23:29:50.265 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:49 smithi157 ceph-mon[33877]: osdmap e20: 3 total, 3 up, 3 in 2024-04-22T23:29:50.265 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:49 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-22T23:29:50.265 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:49 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:29:50.622 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:29:50.623+0000 7fd15e148700 1 -- 172.21.15.157:0/278181381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1580fcfb0 msgr2=0x7fd1580fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:29:50.622 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:29:50.623+0000 7fd15e148700 1 --2- 172.21.15.157:0/278181381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1580fcfb0 0x7fd1580fd420 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd14c004940 tx=0x7fd14c009e30 comp rx=0 tx=0).stop 2024-04-22T23:29:50.622 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:29:50.623+0000 7fd15e148700 1 -- 172.21.15.157:0/278181381 shutdown_connections 2024-04-22T23:29:50.622 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:29:50.623+0000 7fd15e148700 1 --2- 172.21.15.157:0/278181381 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1580fcfb0 0x7fd1580fd420 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:29:50.622 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:29:50.623+0000 7fd15e148700 1 --2- 172.21.15.157:0/278181381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1580fbd00 0x7fd1580fc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:29:50.622 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:29:50.623+0000 7fd15e148700 1 -- 172.21.15.157:0/278181381 >> 172.21.15.157:0/278181381 conn(0x7fd1580f7480 msgr2=0x7fd1580f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:29:50.622 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:29:50.623+0000 7fd15e148700 1 -- 172.21.15.157:0/278181381 shutdown_connections 2024-04-22T23:29:50.622 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:29:50.623+0000 7fd15e148700 1 -- 172.21.15.157:0/278181381 wait complete. 2024-04-22T23:29:50.622 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:29:50.623+0000 7fd15e148700 1 Processor -- start 2024-04-22T23:29:50.624 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:29:50.624+0000 7fd15e148700 1 -- start start 2024-04-22T23:29:50.624 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:29:50.624+0000 7fd15e148700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1580fbd00 0x7fd1580ffba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:29:50.624 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:29:50.624+0000 7fd15e148700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1580fcfb0 0x7fd1580fe2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:29:50.624 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:29:50.624+0000 7fd15e148700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1581001a0 con 0x7fd1580fbd00 2024-04-22T23:29:50.624 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:29:50.624+0000 7fd15e148700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1580fe830 con 0x7fd1580fcfb0 2024-04-22T23:29:50.625 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:29:50.624+0000 7fd156ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1580fcfb0 0x7fd1580fe2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:29:50.625 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:29:50.624+0000 7fd156ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1580fcfb0 0x7fd1580fe2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:44824/0 (socket says 172.21.15.157:44824) 2024-04-22T23:29:50.625 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:29:50.624+0000 7fd156ffd700 1 -- 172.21.15.157:0/2665322893 learned_addr learned my addr 172.21.15.157:0/2665322893 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-22T23:29:50.625 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:29:50.624+0000 7fd1577fe700 1 --2- 172.21.15.157:0/2665322893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1580fbd00 0x7fd1580ffba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:29:50.625 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:29:50.624+0000 7fd156ffd700 1 -- 172.21.15.157:0/2665322893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1580fbd00 msgr2=0x7fd1580ffba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:29:50.625 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:29:50.624+0000 7fd156ffd700 1 --2- 172.21.15.157:0/2665322893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1580fbd00 0x7fd1580ffba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:29:50.625 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:29:50.624+0000 7fd156ffd700 1 -- 172.21.15.157:0/2665322893 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd14c005040 con 0x7fd1580fcfb0 2024-04-22T23:29:50.625 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:29:50.624+0000 7fd156ffd700 1 --2- 172.21.15.157:0/2665322893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1580fcfb0 0x7fd1580fe2f0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fd14c0040d0 tx=0x7fd14c009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:29:50.625 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:29:50.624+0000 7fd13ffff700 1 -- 172.21.15.157:0/2665322893 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd14c021070 con 0x7fd1580fcfb0 2024-04-22T23:29:50.625 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:29:50.624+0000 7fd15e148700 1 -- 172.21.15.157:0/2665322893 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1580feab0 con 0x7fd1580fcfb0 2024-04-22T23:29:50.625 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:29:50.624+0000 7fd13ffff700 1 -- 172.21.15.157:0/2665322893 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd14c013de0 con 0x7fd1580fcfb0 2024-04-22T23:29:50.625 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:29:50.624+0000 7fd13ffff700 1 -- 172.21.15.157:0/2665322893 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd14c00e070 con 0x7fd1580fcfb0 2024-04-22T23:29:50.625 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:29:50.625+0000 7fd15e148700 1 -- 172.21.15.157:0/2665322893 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1580fef90 con 0x7fd1580fcfb0 2024-04-22T23:29:50.625 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:29:50.626+0000 7fd15e148700 1 -- 172.21.15.157:0/2665322893 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd15804c710 con 0x7fd1580fcfb0 2024-04-22T23:29:50.626 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:29:50.626+0000 7fd13ffff700 1 -- 172.21.15.157:0/2665322893 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) v1 ==== 86575+0+0 (secure 0 0 0) 0x7fd14c013910 con 0x7fd1580fcfb0 2024-04-22T23:29:50.626 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:29:50.626+0000 7fd13ffff700 1 --2- 172.21.15.157:0/2665322893 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fd140069210 0x7fd14006b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:29:50.626 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:29:50.626+0000 7fd13ffff700 1 -- 172.21.15.157:0/2665322893 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3228+0+0 (secure 0 0 0) 0x7fd14c087a30 con 0x7fd1580fcfb0 2024-04-22T23:29:50.626 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:29:50.626+0000 7fd1577fe700 1 --2- 172.21.15.157:0/2665322893 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fd140069210 0x7fd14006b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:29:50.626 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:29:50.627+0000 7fd1577fe700 1 --2- 172.21.15.157:0/2665322893 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fd140069210 0x7fd14006b6c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd148009b80 tx=0x7fd14800b040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:29:50.629 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:29:50.631+0000 7fd13ffff700 1 -- 172.21.15.157:0/2665322893 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd14c058190 con 0x7fd1580fcfb0 2024-04-22T23:29:50.693 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:29:50 smithi107 sudo[64368]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-22T23:29:50.693 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:29:50 smithi107 sudo[64368]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-22T23:29:50.693 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:29:50 smithi107 sudo[64368]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-22T23:29:50.693 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:29:50 smithi107 sudo[64368]: pam_unix(sudo:session): session closed for user root 2024-04-22T23:29:50.823 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:29:50.824+0000 7fd15e148700 1 -- 172.21.15.157:0/2665322893 --> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fd1581049a0 con 0x7fd140069210 2024-04-22T23:29:50.969 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:29:50 smithi107 sudo[64373]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-22T23:29:50.969 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:29:50 smithi107 sudo[64373]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-22T23:29:50.970 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:29:50 smithi107 sudo[64373]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-22T23:29:50.970 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:29:50 smithi107 sudo[64373]: pam_unix(sudo:session): session closed for user root 2024-04-22T23:29:51.135 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:50 smithi157 ceph-mon[33877]: pgmap v76: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-04-22T23:29:51.135 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:50 smithi157 ceph-mon[33877]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-22T23:29:51.135 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:50 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-22T23:29:51.135 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:50 smithi157 ceph-mon[33877]: osdmap e21: 3 total, 3 up, 3 in 2024-04-22T23:29:51.135 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:50 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:51.135 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:50 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T23:29:51.135 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:50 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T23:29:51.136 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:50 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:29:51.235 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:29:50 smithi107 sudo[64376]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-22T23:29:51.235 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:29:50 smithi107 sudo[64376]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-22T23:29:51.235 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:29:50 smithi107 sudo[64376]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-22T23:29:51.236 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:29:50 smithi107 sudo[64376]: pam_unix(sudo:session): session closed for user root 2024-04-22T23:29:51.236 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:50 smithi107 ceph-mon[26355]: pgmap v76: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-04-22T23:29:51.236 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:50 smithi107 ceph-mon[26355]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-22T23:29:51.236 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:50 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-22T23:29:51.236 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:50 smithi107 ceph-mon[26355]: osdmap e21: 3 total, 3 up, 3 in 2024-04-22T23:29:51.236 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:50 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:51.236 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:50 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T23:29:51.236 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:50 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T23:29:51.236 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:50 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:29:51.487 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:29:51 smithi107 sudo[64414]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-22T23:29:51.487 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:29:51 smithi107 sudo[64414]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-22T23:29:51.487 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:29:51 smithi107 sudo[64414]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-22T23:29:51.487 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:29:51 smithi107 sudo[64411]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-22T23:29:51.487 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:29:51 smithi107 sudo[64411]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-22T23:29:51.487 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:29:51 smithi107 sudo[64411]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-22T23:29:51.487 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:29:51 smithi107 sudo[64411]: pam_unix(sudo:session): session closed for user root 2024-04-22T23:29:51.976 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:29:51 smithi107 sudo[64414]: pam_unix(sudo:session): session closed for user root 2024-04-22T23:29:51.976 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:29:51 smithi107 sudo[64417]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-22T23:29:51.976 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:29:51 smithi107 sudo[64417]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-22T23:29:51.976 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:29:51 smithi107 sudo[64417]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-22T23:29:51.976 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:29:51 smithi107 sudo[64417]: pam_unix(sudo:session): session closed for user root 2024-04-22T23:29:52.226 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:51 smithi107 sudo[64420]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-22T23:29:52.226 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:51 smithi107 sudo[64420]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-22T23:29:52.227 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:51 smithi107 sudo[64420]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-22T23:29:52.227 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:51 smithi107 sudo[64420]: pam_unix(sudo:session): session closed for user root 2024-04-22T23:29:52.227 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:52 smithi107 sudo[64423]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-22T23:29:52.227 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:52 smithi107 sudo[64423]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-22T23:29:52.455 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:52 smithi157 ceph-mon[33877]: from='client.24133 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:29:52.455 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:52 smithi157 ceph-mon[33877]: osdmap e22: 3 total, 3 up, 3 in 2024-04-22T23:29:52.455 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:52 smithi157 ceph-mon[33877]: Detected new or changed devices on smithi107 2024-04-22T23:29:52.455 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:52 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:52.455 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:52 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:52.455 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:52 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:29:52.455 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:52 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:29:52.455 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:52 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:29:52.456 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:52 smithi157 ceph-mon[33877]: Adjusting osd_memory_target on smithi107 to 4288M 2024-04-22T23:29:52.456 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:52 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:52.456 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:52 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:29:52.456 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:52 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:29:52.456 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:52 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:52.456 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:52 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:29:52.456 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:52 smithi157 ceph-mon[33877]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-22T23:29:52.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:52 smithi107 sudo[64423]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-22T23:29:52.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:52 smithi107 sudo[64423]: pam_unix(sudo:session): session closed for user root 2024-04-22T23:29:52.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:52 smithi107 ceph-mon[26355]: from='client.24133 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:29:52.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:52 smithi107 ceph-mon[26355]: osdmap e22: 3 total, 3 up, 3 in 2024-04-22T23:29:52.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:52 smithi107 ceph-mon[26355]: Detected new or changed devices on smithi107 2024-04-22T23:29:52.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:52 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:52.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:52 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:52.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:52 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:29:52.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:52 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:29:52.488 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:52 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:29:52.488 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:52 smithi107 ceph-mon[26355]: Adjusting osd_memory_target on smithi107 to 4288M 2024-04-22T23:29:52.488 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:52 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:52.488 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:52 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:29:52.488 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:52 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:29:52.488 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:52 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:52.488 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:52 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:29:52.488 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:52 smithi107 ceph-mon[26355]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-22T23:29:52.749 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:52 smithi157 sudo[37331]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-22T23:29:52.749 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:52 smithi157 sudo[37331]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-22T23:29:52.750 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:52 smithi157 sudo[37331]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-22T23:29:52.750 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:52 smithi157 sudo[37331]: pam_unix(sudo:session): session closed for user root 2024-04-22T23:29:53.135 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:52 smithi157 sudo[37339]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-22T23:29:53.135 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:52 smithi157 sudo[37339]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-22T23:29:53.135 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:52 smithi157 sudo[37339]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-22T23:29:53.135 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:52 smithi157 sudo[37339]: pam_unix(sudo:session): session closed for user root 2024-04-22T23:29:53.241 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:29:53.241+0000 7fd13ffff700 1 -- 172.21.15.157:0/2665322893 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fd14c01c690 con 0x7fd1580fcfb0 2024-04-22T23:29:53.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:53 smithi107 ceph-mon[26355]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 872 MiB used, 267 GiB / 268 GiB avail 2024-04-22T23:29:53.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:53 smithi107 ceph-mon[26355]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-22T23:29:53.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:53 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-04-22T23:29:53.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:53 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mon metadata", "id": "smithi157"}]: dispatch 2024-04-22T23:29:53.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:53 smithi107 ceph-mon[26355]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-22T23:29:53.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:53 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-04-22T23:29:53.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:53 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mon metadata", "id": "smithi157"}]: dispatch 2024-04-22T23:29:53.488 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:53 smithi107 ceph-mon[26355]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-22T23:29:53.488 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:53 smithi107 ceph-mon[26355]: Cluster is now healthy 2024-04-22T23:29:53.488 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:53 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:29:53.488 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:53 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:29:53.488 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:53 smithi107 ceph-mon[26355]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-22T23:29:53.488 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:53 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:53.488 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:53 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:53.594 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:53 smithi157 ceph-mon[33877]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 872 MiB used, 267 GiB / 268 GiB avail 2024-04-22T23:29:53.594 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:53 smithi157 ceph-mon[33877]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-22T23:29:53.594 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:53 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-04-22T23:29:53.594 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:53 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mon metadata", "id": "smithi157"}]: dispatch 2024-04-22T23:29:53.594 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:53 smithi157 ceph-mon[33877]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-22T23:29:53.594 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:53 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-04-22T23:29:53.595 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:53 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mon metadata", "id": "smithi157"}]: dispatch 2024-04-22T23:29:53.595 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:53 smithi157 ceph-mon[33877]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-22T23:29:53.595 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:53 smithi157 ceph-mon[33877]: Cluster is now healthy 2024-04-22T23:29:53.595 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:53 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:29:53.595 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:53 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:29:53.595 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:53 smithi157 ceph-mon[33877]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-22T23:29:53.595 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:53 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:53.595 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:53 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:29:54.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:54 smithi157 ceph-mon[33877]: mgrmap e19: smithi107.uipxvm(active, since 2m), standbys: smithi157.tuylis 2024-04-22T23:29:54.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:54 smithi157 ceph-mon[33877]: from='client.? 172.21.15.157:0/994285271' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b7c9c124-9f43-4b5b-b039-c61ae30a416d"}]: dispatch 2024-04-22T23:29:54.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:54 smithi157 ceph-mon[33877]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b7c9c124-9f43-4b5b-b039-c61ae30a416d"}]: dispatch 2024-04-22T23:29:54.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:54 smithi157 ceph-mon[33877]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b7c9c124-9f43-4b5b-b039-c61ae30a416d"}]': finished 2024-04-22T23:29:54.636 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:54 smithi157 ceph-mon[33877]: osdmap e23: 4 total, 3 up, 4 in 2024-04-22T23:29:54.636 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:54 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T23:29:54.636 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:54 smithi157 ceph-mon[33877]: from='client.? 172.21.15.157:0/2406431251' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T23:29:54.737 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:54 smithi107 ceph-mon[26355]: mgrmap e19: smithi107.uipxvm(active, since 2m), standbys: smithi157.tuylis 2024-04-22T23:29:54.737 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:54 smithi107 ceph-mon[26355]: from='client.? 172.21.15.157:0/994285271' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b7c9c124-9f43-4b5b-b039-c61ae30a416d"}]: dispatch 2024-04-22T23:29:54.737 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:54 smithi107 ceph-mon[26355]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b7c9c124-9f43-4b5b-b039-c61ae30a416d"}]: dispatch 2024-04-22T23:29:54.737 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:54 smithi107 ceph-mon[26355]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b7c9c124-9f43-4b5b-b039-c61ae30a416d"}]': finished 2024-04-22T23:29:54.737 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:54 smithi107 ceph-mon[26355]: osdmap e23: 4 total, 3 up, 4 in 2024-04-22T23:29:54.737 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:54 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T23:29:54.737 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:54 smithi107 ceph-mon[26355]: from='client.? 172.21.15.157:0/2406431251' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T23:29:55.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:55 smithi157 ceph-mon[33877]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 872 MiB used, 267 GiB / 268 GiB avail 2024-04-22T23:29:55.737 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:55 smithi107 ceph-mon[26355]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 872 MiB used, 267 GiB / 268 GiB avail 2024-04-22T23:29:57.419 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:57 smithi157 ceph-mon[33877]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-04-22T23:29:57.737 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:57 smithi107 ceph-mon[26355]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-04-22T23:29:59.524 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:29:59 smithi157 ceph-mon[33877]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-04-22T23:29:59.737 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:29:59 smithi107 ceph-mon[26355]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-04-22T23:30:00.540 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:00 smithi157 ceph-mon[33877]: overall HEALTH_OK 2024-04-22T23:30:00.540 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:00 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-22T23:30:00.540 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:00 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:30:00.737 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:00 smithi107 ceph-mon[26355]: overall HEALTH_OK 2024-04-22T23:30:00.737 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:00 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-22T23:30:00.737 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:00 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:30:01.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:01 smithi107 ceph-mon[26355]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-04-22T23:30:01.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:01 smithi107 ceph-mon[26355]: Deploying daemon osd.3 on smithi157 2024-04-22T23:30:01.530 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:01 smithi157 ceph-mon[33877]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-04-22T23:30:01.531 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:01 smithi157 ceph-mon[33877]: Deploying daemon osd.3 on smithi157 2024-04-22T23:30:03.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:03 smithi157 ceph-mon[33877]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-04-22T23:30:03.736 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:03 smithi107 ceph-mon[26355]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-04-22T23:30:04.343 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:04 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:30:04.737 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:04 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:30:05.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:05 smithi157 ceph-mon[33877]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.7 KiB/s rd, 73 KiB/s wr, 5 op/s 2024-04-22T23:30:05.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:05 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:30:05.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:05 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:30:05.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:05 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:30:05.737 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:05 smithi107 ceph-mon[26355]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.7 KiB/s rd, 73 KiB/s wr, 5 op/s 2024-04-22T23:30:05.737 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:05 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:30:05.737 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:05 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:30:05.737 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:05 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:30:07.626 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:07 smithi157 ceph-mon[33877]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-04-22T23:30:07.736 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:07 smithi107 ceph-mon[26355]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-04-22T23:30:09.088 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:09.090+0000 7fd13ffff700 1 -- 172.21.15.157:0/2665322893 <== mgr.14206 v2:172.21.15.107:6800/2516490111 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd14800f040 con 0x7fd140069210 2024-04-22T23:30:09.088 INFO:teuthology.orchestra.run.smithi157.stdout:Created osd(s) 3 on host 'smithi157' 2024-04-22T23:30:09.090 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:09.092+0000 7fd15e148700 1 -- 172.21.15.157:0/2665322893 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fd140069210 msgr2=0x7fd14006b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:30:09.091 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:09.092+0000 7fd15e148700 1 --2- 172.21.15.157:0/2665322893 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fd140069210 0x7fd14006b6c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd148009b80 tx=0x7fd14800b040 comp rx=0 tx=0).stop 2024-04-22T23:30:09.091 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:09.092+0000 7fd15e148700 1 -- 172.21.15.157:0/2665322893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1580fcfb0 msgr2=0x7fd1580fe2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:30:09.091 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:09.092+0000 7fd15e148700 1 --2- 172.21.15.157:0/2665322893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1580fcfb0 0x7fd1580fe2f0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fd14c0040d0 tx=0x7fd14c009e30 comp rx=0 tx=0).stop 2024-04-22T23:30:09.091 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:09.092+0000 7fd15e148700 1 -- 172.21.15.157:0/2665322893 shutdown_connections 2024-04-22T23:30:09.091 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:09.092+0000 7fd15e148700 1 --2- 172.21.15.157:0/2665322893 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fd140069210 0x7fd14006b6c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:30:09.092 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:09.092+0000 7fd15e148700 1 --2- 172.21.15.157:0/2665322893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1580fbd00 0x7fd1580ffba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:30:09.092 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:09.092+0000 7fd15e148700 1 --2- 172.21.15.157:0/2665322893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd1580fcfb0 0x7fd1580fe2f0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:30:09.092 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:09.092+0000 7fd15e148700 1 -- 172.21.15.157:0/2665322893 >> 172.21.15.157:0/2665322893 conn(0x7fd1580f7480 msgr2=0x7fd158066810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:30:09.092 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:09.093+0000 7fd15e148700 1 -- 172.21.15.157:0/2665322893 shutdown_connections 2024-04-22T23:30:09.092 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:09.093+0000 7fd15e148700 1 -- 172.21.15.157:0/2665322893 wait complete. 2024-04-22T23:30:09.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:09 smithi157 ceph-mon[33877]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-04-22T23:30:09.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:09 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:30:09.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:09 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:30:09.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:09 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:30:09.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:09 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:30:09.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:09 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:30:09.386 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:09 smithi157 ceph-mon[33877]: from='osd.3 [v2:172.21.15.157:6800/811815812,v1:172.21.15.157:6801/811815812]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-22T23:30:09.386 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:09 smithi157 ceph-mon[33877]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-22T23:30:09.386 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:09 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:30:09.386 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:09 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:30:09.386 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:09 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:30:09.737 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:09 smithi107 ceph-mon[26355]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-04-22T23:30:09.737 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:09 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:30:09.737 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:09 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:30:09.737 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:09 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:30:09.737 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:09 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:30:09.738 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:09 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:30:09.738 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:09 smithi107 ceph-mon[26355]: from='osd.3 [v2:172.21.15.157:6800/811815812,v1:172.21.15.157:6801/811815812]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-22T23:30:09.738 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:09 smithi107 ceph-mon[26355]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-22T23:30:09.738 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:09 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:30:09.738 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:09 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:30:09.738 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:09 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:30:09.759 DEBUG:teuthology.orchestra.run.smithi157:osd.3> sudo journalctl -f -n 0 -u ceph-8252f6f2-00ff-11ef-bc93-c7b262605968@osd.3.service 2024-04-22T23:30:09.762 INFO:tasks.cephadm:Deploying osd.4 on smithi157 with /dev/vg_nvme/lv_3... 2024-04-22T23:30:09.763 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-22T23:30:10.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:10 smithi157 ceph-mon[33877]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-22T23:30:10.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:10 smithi157 ceph-mon[33877]: osdmap e24: 4 total, 3 up, 4 in 2024-04-22T23:30:10.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:10 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T23:30:10.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:10 smithi157 ceph-mon[33877]: from='osd.3 [v2:172.21.15.157:6800/811815812,v1:172.21.15.157:6801/811815812]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-22T23:30:10.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:10 smithi157 ceph-mon[33877]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-22T23:30:10.636 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:30:10 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-3[39457]: 2024-04-22T23:30:10.312+0000 7fd0d5ea2700 -1 osd.3 0 waiting for initial osdmap 2024-04-22T23:30:10.636 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:30:10 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-3[39457]: 2024-04-22T23:30:10.317+0000 7fd0d0c9d700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T23:30:10.737 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:10 smithi107 ceph-mon[26355]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-22T23:30:10.737 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:10 smithi107 ceph-mon[26355]: osdmap e24: 4 total, 3 up, 4 in 2024-04-22T23:30:10.737 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:10 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T23:30:10.737 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:10 smithi107 ceph-mon[26355]: from='osd.3 [v2:172.21.15.157:6800/811815812,v1:172.21.15.157:6801/811815812]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-22T23:30:10.737 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:10 smithi107 ceph-mon[26355]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-22T23:30:10.867 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi157/config 2024-04-22T23:30:11.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:11 smithi157 ceph-mon[33877]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-22T23:30:11.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:11 smithi157 ceph-mon[33877]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-04-22T23:30:11.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:11 smithi157 ceph-mon[33877]: osdmap e25: 4 total, 3 up, 4 in 2024-04-22T23:30:11.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:11 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T23:30:11.386 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:11 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T23:30:11.386 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:11 smithi157 ceph-mon[33877]: osd.3 [v2:172.21.15.157:6800/811815812,v1:172.21.15.157:6801/811815812] boot 2024-04-22T23:30:11.386 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:11 smithi157 ceph-mon[33877]: osdmap e26: 4 total, 4 up, 4 in 2024-04-22T23:30:11.386 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:11 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T23:30:11.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:11 smithi107 ceph-mon[26355]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-22T23:30:11.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:11 smithi107 ceph-mon[26355]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-04-22T23:30:11.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:11 smithi107 ceph-mon[26355]: osdmap e25: 4 total, 3 up, 4 in 2024-04-22T23:30:11.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:11 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T23:30:11.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:11 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T23:30:11.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:11 smithi107 ceph-mon[26355]: osd.3 [v2:172.21.15.157:6800/811815812,v1:172.21.15.157:6801/811815812] boot 2024-04-22T23:30:11.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:11 smithi107 ceph-mon[26355]: osdmap e26: 4 total, 4 up, 4 in 2024-04-22T23:30:11.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:11 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T23:30:12.454 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:12 smithi157 ceph-mon[33877]: purged_snaps scrub starts 2024-04-22T23:30:12.455 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:12 smithi157 ceph-mon[33877]: purged_snaps scrub ok 2024-04-22T23:30:12.455 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:12 smithi157 ceph-mon[33877]: osdmap e27: 4 total, 4 up, 4 in 2024-04-22T23:30:12.737 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:12 smithi107 ceph-mon[26355]: purged_snaps scrub starts 2024-04-22T23:30:12.737 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:12 smithi107 ceph-mon[26355]: purged_snaps scrub ok 2024-04-22T23:30:12.737 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:12 smithi107 ceph-mon[26355]: osdmap e27: 4 total, 4 up, 4 in 2024-04-22T23:30:13.773 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-22T23:30:13.773 INFO:teuthology.orchestra.run.smithi157.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-22T23:30:13.773 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10+0 records in 2024-04-22T23:30:13.773 INFO:teuthology.orchestra.run.smithi157.stderr:10+0 records out 2024-04-22T23:30:13.773 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00953088 s, 1.1 GB/s 2024-04-22T23:30:13.773 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping successful for: 2024-04-22T23:30:13.794 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:13 smithi157 ceph-mon[33877]: pgmap v93: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-22T23:30:13.794 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:13 smithi157 ceph-mon[33877]: Detected new or changed devices on smithi157 2024-04-22T23:30:13.794 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:13 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:30:13.794 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:13 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:30:13.794 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:13 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:30:13.794 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:13 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:30:13.794 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:13 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:30:13.795 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:13 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:30:13.795 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:13 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:30:13.795 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:13 smithi157 ceph-mon[33877]: osdmap e28: 4 total, 4 up, 4 in 2024-04-22T23:30:13.987 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:13 smithi107 ceph-mon[26355]: pgmap v93: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-22T23:30:13.987 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:13 smithi107 ceph-mon[26355]: Detected new or changed devices on smithi157 2024-04-22T23:30:13.987 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:13 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:30:13.987 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:13 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:30:13.987 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:13 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:30:13.987 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:13 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:30:13.987 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:13 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:30:13.987 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:13 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:30:13.987 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:13 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:30:13.987 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:13 smithi107 ceph-mon[26355]: osdmap e28: 4 total, 4 up, 4 in 2024-04-22T23:30:14.422 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi157:vg_nvme/lv_3 2024-04-22T23:30:14.673 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi157/config 2024-04-22T23:30:14.795 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:14 smithi157 ceph-mon[33877]: Adjusting osd_memory_target on smithi157 to 15936M 2024-04-22T23:30:14.796 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:14 smithi157 ceph-mon[33877]: osdmap e29: 4 total, 4 up, 4 in 2024-04-22T23:30:14.986 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:14 smithi107 ceph-mon[26355]: Adjusting osd_memory_target on smithi157 to 15936M 2024-04-22T23:30:14.987 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:14 smithi107 ceph-mon[26355]: osdmap e29: 4 total, 4 up, 4 in 2024-04-22T23:30:15.711 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:15 smithi157 ceph-mon[33877]: pgmap v96: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-22T23:30:15.711 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:15 smithi157 ceph-mon[33877]: osdmap e30: 4 total, 4 up, 4 in 2024-04-22T23:30:15.986 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:15 smithi107 ceph-mon[26355]: pgmap v96: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-22T23:30:15.987 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:15 smithi107 ceph-mon[26355]: osdmap e30: 4 total, 4 up, 4 in 2024-04-22T23:30:16.184 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:16.184+0000 7f608fafa700 1 -- 172.21.15.157:0/1412661628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60880fcc40 msgr2=0x7f60880fd0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:30:16.184 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:16.184+0000 7f608fafa700 1 --2- 172.21.15.157:0/1412661628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60880fcc40 0x7f60880fd0b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6084004940 tx=0x7f6084009e30 comp rx=0 tx=0).stop 2024-04-22T23:30:16.184 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:16.184+0000 7f608fafa700 1 -- 172.21.15.157:0/1412661628 shutdown_connections 2024-04-22T23:30:16.184 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:16.184+0000 7f608fafa700 1 --2- 172.21.15.157:0/1412661628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60880fcc40 0x7f60880fd0b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:30:16.184 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:16.184+0000 7f608fafa700 1 --2- 172.21.15.157:0/1412661628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60880fc2f0 0x7f60880fc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:30:16.184 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:16.184+0000 7f608fafa700 1 -- 172.21.15.157:0/1412661628 >> 172.21.15.157:0/1412661628 conn(0x7f60880f6be0 msgr2=0x7f60880f9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:30:16.184 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:16.184+0000 7f608fafa700 1 -- 172.21.15.157:0/1412661628 shutdown_connections 2024-04-22T23:30:16.184 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:16.184+0000 7f608fafa700 1 -- 172.21.15.157:0/1412661628 wait complete. 2024-04-22T23:30:16.184 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:16.185+0000 7f608fafa700 1 Processor -- start 2024-04-22T23:30:16.184 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:16.185+0000 7f608fafa700 1 -- start start 2024-04-22T23:30:16.184 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:16.185+0000 7f608fafa700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60880fc2f0 0x7f608810c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:30:16.185 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:16.185+0000 7f608fafa700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60880fcc40 0x7f608810cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:30:16.185 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:16.185+0000 7f608fafa700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f608810d350 con 0x7f60880fc2f0 2024-04-22T23:30:16.185 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:16.185+0000 7f608fafa700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f608810d4c0 con 0x7f60880fcc40 2024-04-22T23:30:16.185 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:16.185+0000 7f608d095700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60880fcc40 0x7f608810cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:30:16.185 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:16.185+0000 7f608d095700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60880fcc40 0x7f608810cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:53394/0 (socket says 172.21.15.157:53394) 2024-04-22T23:30:16.185 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:16.185+0000 7f608d095700 1 -- 172.21.15.157:0/2616422828 learned_addr learned my addr 172.21.15.157:0/2616422828 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-22T23:30:16.185 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:16.185+0000 7f608d896700 1 --2- 172.21.15.157:0/2616422828 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60880fc2f0 0x7f608810c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:30:16.185 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:16.185+0000 7f608d095700 1 -- 172.21.15.157:0/2616422828 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60880fc2f0 msgr2=0x7f608810c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:30:16.185 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:16.185+0000 7f608d095700 1 --2- 172.21.15.157:0/2616422828 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60880fc2f0 0x7f608810c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:30:16.185 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:16.185+0000 7f608d095700 1 -- 172.21.15.157:0/2616422828 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6084005040 con 0x7f60880fcc40 2024-04-22T23:30:16.185 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:16.186+0000 7f608d095700 1 --2- 172.21.15.157:0/2616422828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60880fcc40 0x7f608810cd50 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f608400b3b0 tx=0x7f6084009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:30:16.185 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:16.186+0000 7f607a7fc700 1 -- 172.21.15.157:0/2616422828 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f608400e040 con 0x7f60880fcc40 2024-04-22T23:30:16.185 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:16.186+0000 7f608fafa700 1 -- 172.21.15.157:0/2616422828 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6088111ec0 con 0x7f60880fcc40 2024-04-22T23:30:16.185 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:16.186+0000 7f607a7fc700 1 -- 172.21.15.157:0/2616422828 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f608400bcc0 con 0x7f60880fcc40 2024-04-22T23:30:16.186 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:16.186+0000 7f607a7fc700 1 -- 172.21.15.157:0/2616422828 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f608401c630 con 0x7f60880fcc40 2024-04-22T23:30:16.186 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:16.186+0000 7f608fafa700 1 -- 172.21.15.157:0/2616422828 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60880faf90 con 0x7f60880fcc40 2024-04-22T23:30:16.186 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:16.187+0000 7f608fafa700 1 -- 172.21.15.157:0/2616422828 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f608804c710 con 0x7f60880fcc40 2024-04-22T23:30:16.187 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:16.187+0000 7f607a7fc700 1 -- 172.21.15.157:0/2616422828 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f6084013980 con 0x7f60880fcc40 2024-04-22T23:30:16.187 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:16.188+0000 7f607a7fc700 1 --2- 172.21.15.157:0/2616422828 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f607406b4b0 0x7f607406d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:30:16.187 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:16.188+0000 7f607a7fc700 1 -- 172.21.15.157:0/2616422828 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3770+0+0 (secure 0 0 0) 0x7f6084087910 con 0x7f60880fcc40 2024-04-22T23:30:16.187 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:16.188+0000 7f608d896700 1 --2- 172.21.15.157:0/2616422828 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f607406b4b0 0x7f607406d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:30:16.187 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:16.188+0000 7f608d896700 1 --2- 172.21.15.157:0/2616422828 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f607406b4b0 0x7f607406d960 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6088066040 tx=0x7f607c009040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:30:16.190 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:16.192+0000 7f607a7fc700 1 -- 172.21.15.157:0/2616422828 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f6084057cf0 con 0x7f60880fcc40 2024-04-22T23:30:16.384 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:16.386+0000 7f608fafa700 1 -- 172.21.15.157:0/2616422828 --> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f60881013a0 con 0x7f607406b4b0 2024-04-22T23:30:16.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:16 smithi157 ceph-mon[33877]: pgmap v99: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-22T23:30:16.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:16 smithi157 ceph-mon[33877]: from='client.24153 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:30:16.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:16 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T23:30:16.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:16 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T23:30:16.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:16 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:30:16.888 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:16 smithi107 ceph-mon[26355]: pgmap v99: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-22T23:30:16.888 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:16 smithi107 ceph-mon[26355]: from='client.24153 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:30:16.888 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:16 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T23:30:16.888 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:16 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T23:30:16.888 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:16 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:30:19.386 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:19 smithi157 ceph-mon[33877]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-04-22T23:30:19.386 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:19 smithi157 ceph-mon[33877]: from='client.? 172.21.15.157:0/3566121561' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "34a0cb61-1c2b-4515-9d74-8a3ad3f15fbb"}]: dispatch 2024-04-22T23:30:19.386 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:19 smithi157 ceph-mon[33877]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "34a0cb61-1c2b-4515-9d74-8a3ad3f15fbb"}]: dispatch 2024-04-22T23:30:19.386 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:19 smithi157 ceph-mon[33877]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "34a0cb61-1c2b-4515-9d74-8a3ad3f15fbb"}]': finished 2024-04-22T23:30:19.386 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:19 smithi157 ceph-mon[33877]: osdmap e31: 5 total, 4 up, 5 in 2024-04-22T23:30:19.386 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:19 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T23:30:19.386 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:19 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:30:19.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:19 smithi107 ceph-mon[26355]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-04-22T23:30:19.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:19 smithi107 ceph-mon[26355]: from='client.? 172.21.15.157:0/3566121561' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "34a0cb61-1c2b-4515-9d74-8a3ad3f15fbb"}]: dispatch 2024-04-22T23:30:19.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:19 smithi107 ceph-mon[26355]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "34a0cb61-1c2b-4515-9d74-8a3ad3f15fbb"}]: dispatch 2024-04-22T23:30:19.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:19 smithi107 ceph-mon[26355]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "34a0cb61-1c2b-4515-9d74-8a3ad3f15fbb"}]': finished 2024-04-22T23:30:19.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:19 smithi107 ceph-mon[26355]: osdmap e31: 5 total, 4 up, 5 in 2024-04-22T23:30:19.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:19 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T23:30:19.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:19 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:30:20.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:20 smithi157 ceph-mon[33877]: from='client.? 172.21.15.157:0/3996196381' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T23:30:20.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:20 smithi107 ceph-mon[26355]: from='client.? 172.21.15.157:0/3996196381' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T23:30:21.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:21 smithi157 ceph-mon[33877]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-22T23:30:21.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:21 smithi107 ceph-mon[26355]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-22T23:30:23.104 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:23 smithi157 ceph-mon[33877]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-04-22T23:30:23.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:23 smithi107 ceph-mon[26355]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-04-22T23:30:25.384 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:25 smithi157 ceph-mon[33877]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-22T23:30:25.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:25 smithi107 ceph-mon[26355]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-22T23:30:26.291 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:26 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-22T23:30:26.292 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:26 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:30:26.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:26 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-22T23:30:26.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:26 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:30:27.340 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:27 smithi157 ceph-mon[33877]: Deploying daemon osd.4 on smithi157 2024-04-22T23:30:27.341 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:27 smithi157 ceph-mon[33877]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-22T23:30:27.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:27 smithi107 ceph-mon[26355]: Deploying daemon osd.4 on smithi157 2024-04-22T23:30:27.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:27 smithi107 ceph-mon[26355]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-22T23:30:29.135 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:29 smithi157 ceph-mon[33877]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-22T23:30:29.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:29 smithi107 ceph-mon[26355]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-22T23:30:31.384 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:31 smithi157 ceph-mon[33877]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-22T23:30:31.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:31 smithi107 ceph-mon[26355]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-22T23:30:32.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:32 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:30:32.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:32 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:30:32.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:32 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:30:32.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:32 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:30:32.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:32 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:30:32.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:32 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:30:33.319 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:33 smithi157 ceph-mon[33877]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-22T23:30:33.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:33 smithi107 ceph-mon[26355]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-22T23:30:34.098 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:34.100+0000 7f607a7fc700 1 -- 172.21.15.157:0/2616422828 <== mgr.14206 v2:172.21.15.107:6800/2516490111 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f60881013a0 con 0x7f607406b4b0 2024-04-22T23:30:34.099 INFO:teuthology.orchestra.run.smithi157.stdout:Created osd(s) 4 on host 'smithi157' 2024-04-22T23:30:34.100 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:34.103+0000 7f608fafa700 1 -- 172.21.15.157:0/2616422828 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f607406b4b0 msgr2=0x7f607406d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:30:34.100 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:34.103+0000 7f608fafa700 1 --2- 172.21.15.157:0/2616422828 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f607406b4b0 0x7f607406d960 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6088066040 tx=0x7f607c009040 comp rx=0 tx=0).stop 2024-04-22T23:30:34.101 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:34.103+0000 7f608fafa700 1 -- 172.21.15.157:0/2616422828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60880fcc40 msgr2=0x7f608810cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:30:34.101 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:34.103+0000 7f608fafa700 1 --2- 172.21.15.157:0/2616422828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60880fcc40 0x7f608810cd50 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f608400b3b0 tx=0x7f6084009e30 comp rx=0 tx=0).stop 2024-04-22T23:30:34.101 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:34.103+0000 7f608fafa700 1 -- 172.21.15.157:0/2616422828 shutdown_connections 2024-04-22T23:30:34.101 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:34.103+0000 7f608fafa700 1 --2- 172.21.15.157:0/2616422828 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f607406b4b0 0x7f607406d960 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:30:34.101 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:34.103+0000 7f608fafa700 1 --2- 172.21.15.157:0/2616422828 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60880fc2f0 0x7f608810c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:30:34.101 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:34.103+0000 7f608fafa700 1 --2- 172.21.15.157:0/2616422828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60880fcc40 0x7f608810cd50 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:30:34.101 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:34.103+0000 7f608fafa700 1 -- 172.21.15.157:0/2616422828 >> 172.21.15.157:0/2616422828 conn(0x7f60880f6be0 msgr2=0x7f60880f8240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:30:34.101 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:34.103+0000 7f608fafa700 1 -- 172.21.15.157:0/2616422828 shutdown_connections 2024-04-22T23:30:34.101 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:34.103+0000 7f608fafa700 1 -- 172.21.15.157:0/2616422828 wait complete. 2024-04-22T23:30:34.384 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:34 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:30:34.384 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:34 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:30:34.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:34 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:30:34.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:34 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:30:34.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:34 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:30:34.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:34 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:30:34.777 DEBUG:teuthology.orchestra.run.smithi157:osd.4> sudo journalctl -f -n 0 -u ceph-8252f6f2-00ff-11ef-bc93-c7b262605968@osd.4.service 2024-04-22T23:30:34.780 INFO:tasks.cephadm:Deploying osd.5 on smithi157 with /dev/vg_nvme/lv_2... 2024-04-22T23:30:34.780 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-22T23:30:35.040 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:30:34 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4[46184]: 2024-04-22T23:30:34.927+0000 7f1dce30e540 -1 osd.4 0 log_to_monitors true 2024-04-22T23:30:35.384 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:35 smithi157 ceph-mon[33877]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-22T23:30:35.384 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:35 smithi157 ceph-mon[33877]: from='osd.4 [v2:172.21.15.157:6808/2372899121,v1:172.21.15.157:6809/2372899121]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-22T23:30:35.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:35 smithi157 ceph-mon[33877]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-22T23:30:35.460 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi157/config 2024-04-22T23:30:35.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:35 smithi107 ceph-mon[26355]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-22T23:30:35.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:35 smithi107 ceph-mon[26355]: from='osd.4 [v2:172.21.15.157:6808/2372899121,v1:172.21.15.157:6809/2372899121]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-22T23:30:35.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:35 smithi107 ceph-mon[26355]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-22T23:30:36.262 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:36 smithi157 ceph-mon[33877]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-22T23:30:36.262 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:36 smithi157 ceph-mon[33877]: osdmap e32: 5 total, 4 up, 5 in 2024-04-22T23:30:36.262 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:36 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T23:30:36.262 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:36 smithi157 ceph-mon[33877]: from='osd.4 [v2:172.21.15.157:6808/2372899121,v1:172.21.15.157:6809/2372899121]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-22T23:30:36.262 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:36 smithi157 ceph-mon[33877]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-22T23:30:36.265 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:30:36 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4[46184]: 2024-04-22T23:30:36.116+0000 7f1dc4ae0700 -1 osd.4 0 waiting for initial osdmap 2024-04-22T23:30:36.265 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:30:36 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4[46184]: 2024-04-22T23:30:36.124+0000 7f1dbf8db700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T23:30:36.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:36 smithi107 ceph-mon[26355]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-22T23:30:36.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:36 smithi107 ceph-mon[26355]: osdmap e32: 5 total, 4 up, 5 in 2024-04-22T23:30:36.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:36 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T23:30:36.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:36 smithi107 ceph-mon[26355]: from='osd.4 [v2:172.21.15.157:6808/2372899121,v1:172.21.15.157:6809/2372899121]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-22T23:30:36.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:36 smithi107 ceph-mon[26355]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-22T23:30:37.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:37 smithi157 ceph-mon[33877]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-22T23:30:37.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:37 smithi157 ceph-mon[33877]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-04-22T23:30:37.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:37 smithi157 ceph-mon[33877]: osdmap e33: 5 total, 4 up, 5 in 2024-04-22T23:30:37.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:37 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T23:30:37.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:37 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T23:30:37.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:37 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:30:37.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:37 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:30:37.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:37 smithi157 ceph-mon[33877]: osd.4 [v2:172.21.15.157:6808/2372899121,v1:172.21.15.157:6809/2372899121] boot 2024-04-22T23:30:37.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:37 smithi157 ceph-mon[33877]: osdmap e34: 5 total, 5 up, 5 in 2024-04-22T23:30:37.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:37 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T23:30:37.396 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-22T23:30:37.396 INFO:teuthology.orchestra.run.smithi157.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-22T23:30:37.397 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 10+0 records in 2024-04-22T23:30:37.397 INFO:teuthology.orchestra.run.smithi157.stderr:10+0 records out 2024-04-22T23:30:37.397 INFO:teuthology.orchestra.run.smithi157.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0127093 s, 825 MB/s 2024-04-22T23:30:37.397 INFO:teuthology.orchestra.run.smithi157.stderr: stderr: 2024-04-22T23:30:37.397 INFO:teuthology.orchestra.run.smithi157.stderr:--> Zapping successful for: 2024-04-22T23:30:37.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:37 smithi107 ceph-mon[26355]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-22T23:30:37.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:37 smithi107 ceph-mon[26355]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-04-22T23:30:37.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:37 smithi107 ceph-mon[26355]: osdmap e33: 5 total, 4 up, 5 in 2024-04-22T23:30:37.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:37 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T23:30:37.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:37 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T23:30:37.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:37 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:30:37.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:37 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:30:37.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:37 smithi107 ceph-mon[26355]: osd.4 [v2:172.21.15.157:6808/2372899121,v1:172.21.15.157:6809/2372899121] boot 2024-04-22T23:30:37.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:37 smithi107 ceph-mon[26355]: osdmap e34: 5 total, 5 up, 5 in 2024-04-22T23:30:37.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:37 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T23:30:38.359 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:38 smithi157 ceph-mon[33877]: purged_snaps scrub starts 2024-04-22T23:30:38.359 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:38 smithi157 ceph-mon[33877]: purged_snaps scrub ok 2024-04-22T23:30:38.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:38 smithi107 ceph-mon[26355]: purged_snaps scrub starts 2024-04-22T23:30:38.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:38 smithi107 ceph-mon[26355]: purged_snaps scrub ok 2024-04-22T23:30:39.124 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi157:vg_nvme/lv_2 2024-04-22T23:30:39.384 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:39 smithi157 ceph-mon[33877]: pgmap v114: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-22T23:30:39.384 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:39 smithi157 ceph-mon[33877]: osdmap e35: 5 total, 5 up, 5 in 2024-04-22T23:30:39.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:39 smithi107 ceph-mon[26355]: pgmap v114: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-22T23:30:39.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:39 smithi107 ceph-mon[26355]: osdmap e35: 5 total, 5 up, 5 in 2024-04-22T23:30:39.658 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi157/config 2024-04-22T23:30:41.384 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:41 smithi157 ceph-mon[33877]: pgmap v116: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-22T23:30:41.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:41 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:30:41.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:41 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:30:41.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:41 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:30:41.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:41 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:30:41.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:41 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:30:41.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:41 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:30:41.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:41 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:30:41.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:41 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:30:41.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:41 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:30:41.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:41 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:30:41.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:41 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:30:41.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:41 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:30:41.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:41 smithi107 ceph-mon[26355]: pgmap v116: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-22T23:30:41.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:41 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:30:41.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:41 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:30:41.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:41 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:30:41.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:41 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:30:41.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:41 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:30:41.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:41 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:30:41.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:41 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:30:41.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:41 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:30:41.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:41 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:30:41.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:41 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:30:41.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:41 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:30:41.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:41 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:30:41.832 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:41.833+0000 7f3781b29700 1 -- 172.21.15.157:0/2597795100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f377c0fbd40 msgr2=0x7f377c0fc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:30:41.832 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:41.833+0000 7f3781b29700 1 --2- 172.21.15.157:0/2597795100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f377c0fbd40 0x7f377c0fc150 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3770004940 tx=0x7f3770009e30 comp rx=0 tx=0).stop 2024-04-22T23:30:41.833 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:41.833+0000 7f3781b29700 1 -- 172.21.15.157:0/2597795100 shutdown_connections 2024-04-22T23:30:41.833 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:41.833+0000 7f3781b29700 1 --2- 172.21.15.157:0/2597795100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f377c0fcff0 0x7f377c0fd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:30:41.833 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:41.833+0000 7f3781b29700 1 --2- 172.21.15.157:0/2597795100 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f377c0fbd40 0x7f377c0fc150 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:30:41.833 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:41.833+0000 7f3781b29700 1 -- 172.21.15.157:0/2597795100 >> 172.21.15.157:0/2597795100 conn(0x7f377c0f74c0 msgr2=0x7f377c0f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:30:41.833 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:41.833+0000 7f3781b29700 1 -- 172.21.15.157:0/2597795100 shutdown_connections 2024-04-22T23:30:41.833 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:41.833+0000 7f3781b29700 1 -- 172.21.15.157:0/2597795100 wait complete. 2024-04-22T23:30:41.833 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:41.834+0000 7f3781b29700 1 Processor -- start 2024-04-22T23:30:41.833 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:41.834+0000 7f3781b29700 1 -- start start 2024-04-22T23:30:41.833 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:41.834+0000 7f3781b29700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f377c0fbd40 0x7f377c110740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:30:41.833 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:41.834+0000 7f3781b29700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f377c0fcff0 0x7f377c112c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:30:41.833 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:41.834+0000 7f3781b29700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377c0663c0 con 0x7f377c0fbd40 2024-04-22T23:30:41.833 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:41.834+0000 7f3781b29700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377c066500 con 0x7f377c0fcff0 2024-04-22T23:30:41.833 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:41.834+0000 7f377affd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f377c0fcff0 0x7f377c112c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:30:41.834 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:41.834+0000 7f377affd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f377c0fcff0 0x7f377c112c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:52640/0 (socket says 172.21.15.157:52640) 2024-04-22T23:30:41.834 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:41.834+0000 7f377affd700 1 -- 172.21.15.157:0/521712493 learned_addr learned my addr 172.21.15.157:0/521712493 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-22T23:30:41.834 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:41.834+0000 7f377b7fe700 1 --2- 172.21.15.157:0/521712493 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f377c0fbd40 0x7f377c110740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:30:41.834 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:41.834+0000 7f377affd700 1 -- 172.21.15.157:0/521712493 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f377c0fbd40 msgr2=0x7f377c110740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:30:41.834 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:41.834+0000 7f377affd700 1 --2- 172.21.15.157:0/521712493 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f377c0fbd40 0x7f377c110740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:30:41.834 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:41.835+0000 7f377affd700 1 -- 172.21.15.157:0/521712493 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3770005040 con 0x7f377c0fcff0 2024-04-22T23:30:41.834 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:41.835+0000 7f377affd700 1 --2- 172.21.15.157:0/521712493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f377c0fcff0 0x7f377c112c90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f376c00b790 tx=0x7f376c009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:30:41.834 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:41.835+0000 7f3763fff700 1 -- 172.21.15.157:0/521712493 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f376c00f770 con 0x7f377c0fcff0 2024-04-22T23:30:41.834 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:41.835+0000 7f3781b29700 1 -- 172.21.15.157:0/521712493 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f377c113290 con 0x7f377c0fcff0 2024-04-22T23:30:41.834 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:41.835+0000 7f3781b29700 1 -- 172.21.15.157:0/521712493 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f377c113850 con 0x7f377c0fcff0 2024-04-22T23:30:41.834 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:41.835+0000 7f3763fff700 1 -- 172.21.15.157:0/521712493 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f376c00fda0 con 0x7f377c0fcff0 2024-04-22T23:30:41.834 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:41.835+0000 7f3763fff700 1 -- 172.21.15.157:0/521712493 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f376c00d310 con 0x7f377c0fcff0 2024-04-22T23:30:41.835 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:41.836+0000 7f3781b29700 1 -- 172.21.15.157:0/521712493 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37680052e0 con 0x7f377c0fcff0 2024-04-22T23:30:41.835 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:41.836+0000 7f3763fff700 1 -- 172.21.15.157:0/521712493 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f376c00d5b0 con 0x7f377c0fcff0 2024-04-22T23:30:41.835 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:41.837+0000 7f3763fff700 1 --2- 172.21.15.157:0/521712493 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f37640692a0 0x7f376406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:30:41.835 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:41.837+0000 7f3763fff700 1 -- 172.21.15.157:0/521712493 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4202+0+0 (secure 0 0 0) 0x7f376c086b40 con 0x7f377c0fcff0 2024-04-22T23:30:41.836 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:41.837+0000 7f377b7fe700 1 --2- 172.21.15.157:0/521712493 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f37640692a0 0x7f376406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:30:41.836 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:41.837+0000 7f377b7fe700 1 --2- 172.21.15.157:0/521712493 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f37640692a0 0x7f376406b750 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f377000b970 tx=0x7f3770009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:30:41.838 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:41.841+0000 7f3763fff700 1 -- 172.21.15.157:0/521712493 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f376c053550 con 0x7f377c0fcff0 2024-04-22T23:30:42.029 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:30:42.032+0000 7f3781b29700 1 -- 172.21.15.157:0/521712493 --> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f3768000bf0 con 0x7f37640692a0 2024-04-22T23:30:42.384 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:42 smithi157 ceph-mon[33877]: Detected new or changed devices on smithi157 2024-04-22T23:30:42.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:42 smithi157 ceph-mon[33877]: Adjusting osd_memory_target on smithi157 to 7968M 2024-04-22T23:30:42.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:42 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T23:30:42.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:42 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T23:30:42.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:42 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:30:42.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:42 smithi107 ceph-mon[26355]: Detected new or changed devices on smithi157 2024-04-22T23:30:42.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:42 smithi107 ceph-mon[26355]: Adjusting osd_memory_target on smithi157 to 7968M 2024-04-22T23:30:42.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:42 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T23:30:42.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:42 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T23:30:42.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:42 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:30:43.384 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:43 smithi157 ceph-mon[33877]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-22T23:30:43.384 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:43 smithi157 ceph-mon[33877]: from='client.24171 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:30:43.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:43 smithi107 ceph-mon[26355]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-22T23:30:43.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:43 smithi107 ceph-mon[26355]: from='client.24171 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi157:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:30:45.384 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:45 smithi157 ceph-mon[33877]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-22T23:30:45.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:45 smithi157 ceph-mon[33877]: from='client.? 172.21.15.157:0/2405092235' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "00934a7f-7e88-45cf-a2cc-e8a94bceaab0"}]: dispatch 2024-04-22T23:30:45.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:45 smithi157 ceph-mon[33877]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "00934a7f-7e88-45cf-a2cc-e8a94bceaab0"}]: dispatch 2024-04-22T23:30:45.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:45 smithi157 ceph-mon[33877]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "00934a7f-7e88-45cf-a2cc-e8a94bceaab0"}]': finished 2024-04-22T23:30:45.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:45 smithi157 ceph-mon[33877]: osdmap e36: 6 total, 5 up, 6 in 2024-04-22T23:30:45.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:45 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T23:30:45.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:45 smithi157 ceph-mon[33877]: from='client.? 172.21.15.157:0/165674518' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T23:30:45.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:45 smithi107 ceph-mon[26355]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-22T23:30:45.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:45 smithi107 ceph-mon[26355]: from='client.? 172.21.15.157:0/2405092235' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "00934a7f-7e88-45cf-a2cc-e8a94bceaab0"}]: dispatch 2024-04-22T23:30:45.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:45 smithi107 ceph-mon[26355]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "00934a7f-7e88-45cf-a2cc-e8a94bceaab0"}]: dispatch 2024-04-22T23:30:45.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:45 smithi107 ceph-mon[26355]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "00934a7f-7e88-45cf-a2cc-e8a94bceaab0"}]': finished 2024-04-22T23:30:45.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:45 smithi107 ceph-mon[26355]: osdmap e36: 6 total, 5 up, 6 in 2024-04-22T23:30:45.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:45 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T23:30:45.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:45 smithi107 ceph-mon[26355]: from='client.? 172.21.15.157:0/165674518' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T23:30:47.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:47 smithi107 ceph-mon[26355]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-22T23:30:47.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:47 smithi157 ceph-mon[33877]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-22T23:30:49.384 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:49 smithi157 ceph-mon[33877]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-22T23:30:49.384 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:49 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:30:49.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:49 smithi107 ceph-mon[26355]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-22T23:30:49.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:49 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:30:51.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:51 smithi157 ceph-mon[33877]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-22T23:30:51.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:51 smithi107 ceph-mon[26355]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-22T23:30:52.384 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:52 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-22T23:30:52.384 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:52 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:30:52.384 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:52 smithi157 ceph-mon[33877]: Deploying daemon osd.5 on smithi157 2024-04-22T23:30:52.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:52 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-22T23:30:52.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:52 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:30:52.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:52 smithi107 ceph-mon[26355]: Deploying daemon osd.5 on smithi157 2024-04-22T23:30:53.359 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:53 smithi157 ceph-mon[33877]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-22T23:30:53.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:53 smithi107 ceph-mon[26355]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-22T23:30:55.175 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:55 smithi157 ceph-mon[33877]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-22T23:30:55.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:55 smithi107 ceph-mon[26355]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-22T23:30:57.411 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:57 smithi157 ceph-mon[33877]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-22T23:30:57.412 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:57 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:30:57.412 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:57 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:30:57.412 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:57 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:30:57.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:57 smithi107 ceph-mon[26355]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-22T23:30:57.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:57 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:30:57.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:57 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:30:57.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:57 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:30:59.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:30:59 smithi107 ceph-mon[26355]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-22T23:30:59.574 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:30:59 smithi157 ceph-mon[33877]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-22T23:31:00.944 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:00.947+0000 7f3763fff700 1 -- 172.21.15.157:0/521712493 <== mgr.14206 v2:172.21.15.107:6800/2516490111 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f3768000bf0 con 0x7f37640692a0 2024-04-22T23:31:00.944 INFO:teuthology.orchestra.run.smithi157.stdout:Created osd(s) 5 on host 'smithi157' 2024-04-22T23:31:00.946 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:00.949+0000 7f3781b29700 1 -- 172.21.15.157:0/521712493 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f37640692a0 msgr2=0x7f376406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:00.946 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:00.949+0000 7f3781b29700 1 --2- 172.21.15.157:0/521712493 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f37640692a0 0x7f376406b750 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f377000b970 tx=0x7f3770009e30 comp rx=0 tx=0).stop 2024-04-22T23:31:00.946 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:00.949+0000 7f3781b29700 1 -- 172.21.15.157:0/521712493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f377c0fcff0 msgr2=0x7f377c112c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:00.946 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:00.949+0000 7f3781b29700 1 --2- 172.21.15.157:0/521712493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f377c0fcff0 0x7f377c112c90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f376c00b790 tx=0x7f376c009250 comp rx=0 tx=0).stop 2024-04-22T23:31:00.946 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:00.949+0000 7f3781b29700 1 -- 172.21.15.157:0/521712493 shutdown_connections 2024-04-22T23:31:00.946 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:00.949+0000 7f3781b29700 1 --2- 172.21.15.157:0/521712493 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f37640692a0 0x7f376406b750 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:00.946 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:00.949+0000 7f3781b29700 1 --2- 172.21.15.157:0/521712493 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f377c0fbd40 0x7f377c110740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:00.947 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:00.949+0000 7f3781b29700 1 --2- 172.21.15.157:0/521712493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f377c0fcff0 0x7f377c112c90 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:00.947 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:00.949+0000 7f3781b29700 1 -- 172.21.15.157:0/521712493 >> 172.21.15.157:0/521712493 conn(0x7f377c0f74c0 msgr2=0x7f377c0f8ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:31:00.947 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:00.949+0000 7f3781b29700 1 -- 172.21.15.157:0/521712493 shutdown_connections 2024-04-22T23:31:00.947 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:00.949+0000 7f3781b29700 1 -- 172.21.15.157:0/521712493 wait complete. 2024-04-22T23:31:01.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:01 smithi107 ceph-mon[26355]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-22T23:31:01.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:01 smithi107 ceph-mon[26355]: from='osd.5 [v2:172.21.15.157:6816/2125627920,v1:172.21.15.157:6817/2125627920]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-22T23:31:01.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:01 smithi107 ceph-mon[26355]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-22T23:31:01.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:01 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:31:01.487 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:01 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:31:01.547 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:01 smithi157 ceph-mon[33877]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-22T23:31:01.547 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:01 smithi157 ceph-mon[33877]: from='osd.5 [v2:172.21.15.157:6816/2125627920,v1:172.21.15.157:6817/2125627920]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-22T23:31:01.547 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:01 smithi157 ceph-mon[33877]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-22T23:31:01.547 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:01 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:31:01.548 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:01 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:31:01.573 DEBUG:teuthology.orchestra.run.smithi157:osd.5> sudo journalctl -f -n 0 -u ceph-8252f6f2-00ff-11ef-bc93-c7b262605968@osd.5.service 2024-04-22T23:31:01.575 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-04-22T23:31:01.576 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-22T23:31:01.845 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:31:02.236 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:02 smithi107 ceph-mon[26355]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-22T23:31:02.237 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:02 smithi107 ceph-mon[26355]: osdmap e37: 6 total, 5 up, 6 in 2024-04-22T23:31:02.237 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:02 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T23:31:02.237 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:02 smithi107 ceph-mon[26355]: from='osd.5 [v2:172.21.15.157:6816/2125627920,v1:172.21.15.157:6817/2125627920]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-22T23:31:02.237 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:02 smithi107 ceph-mon[26355]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-22T23:31:02.237 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:02 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:31:02.237 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:02 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:31:02.514 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:02 smithi157 ceph-mon[33877]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-22T23:31:02.514 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:02 smithi157 ceph-mon[33877]: osdmap e37: 6 total, 5 up, 6 in 2024-04-22T23:31:02.514 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:02 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T23:31:02.514 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:02 smithi157 ceph-mon[33877]: from='osd.5 [v2:172.21.15.157:6816/2125627920,v1:172.21.15.157:6817/2125627920]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-22T23:31:02.514 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:02 smithi157 ceph-mon[33877]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-22T23:31:02.514 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:02 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:31:02.514 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:02 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:31:02.515 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:31:02 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5[52684]: 2024-04-22T23:31:02.173+0000 7f96df22b700 -1 osd.5 0 waiting for initial osdmap 2024-04-22T23:31:02.515 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:31:02 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5[52684]: 2024-04-22T23:31:02.186+0000 7f96da026700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T23:31:03.304 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:03 smithi107 ceph-mon[26355]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-22T23:31:03.304 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:03 smithi107 ceph-mon[26355]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-04-22T23:31:03.304 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:03 smithi107 ceph-mon[26355]: osdmap e38: 6 total, 5 up, 6 in 2024-04-22T23:31:03.305 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:03 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T23:31:03.305 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:03 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T23:31:03.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:03.498+0000 7f061639d700 1 -- 172.21.15.107:0/4172922927 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06100fcc20 msgr2=0x7f06100fd030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:03.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:03.498+0000 7f061639d700 1 --2- 172.21.15.107:0/4172922927 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06100fcc20 0x7f06100fd030 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f060c004940 tx=0x7f060c009e30 comp rx=0 tx=0).stop 2024-04-22T23:31:03.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:03.498+0000 7f061639d700 1 -- 172.21.15.107:0/4172922927 shutdown_connections 2024-04-22T23:31:03.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:03.498+0000 7f061639d700 1 --2- 172.21.15.107:0/4172922927 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06100fded0 0x7f06100fe340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:03.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:03.498+0000 7f061639d700 1 --2- 172.21.15.107:0/4172922927 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06100fcc20 0x7f06100fd030 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:03.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:03.498+0000 7f061639d700 1 -- 172.21.15.107:0/4172922927 >> 172.21.15.107:0/4172922927 conn(0x7f06100f83a0 msgr2=0x7f06100fa7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:31:03.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:03.498+0000 7f061639d700 1 -- 172.21.15.107:0/4172922927 shutdown_connections 2024-04-22T23:31:03.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:03.498+0000 7f061639d700 1 -- 172.21.15.107:0/4172922927 wait complete. 2024-04-22T23:31:03.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:03.499+0000 7f061639d700 1 Processor -- start 2024-04-22T23:31:03.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:03.499+0000 7f061639d700 1 -- start start 2024-04-22T23:31:03.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:03.499+0000 7f061639d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06100fcc20 0x7f061010d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:03.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:03.499+0000 7f061639d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06100fded0 0x7f061010dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:03.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:03.499+0000 7f061639d700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061010e390 con 0x7f06100fded0 2024-04-22T23:31:03.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:03.499+0000 7f061639d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061010eb70 con 0x7f06100fcc20 2024-04-22T23:31:03.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:03.499+0000 7f0614b9a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06100fded0 0x7f061010dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:03.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:03.499+0000 7f0614b9a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06100fded0 0x7f061010dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42984/0 (socket says 172.21.15.107:42984) 2024-04-22T23:31:03.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:03.499+0000 7f0614b9a700 1 -- 172.21.15.107:0/3567380437 learned_addr learned my addr 172.21.15.107:0/3567380437 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:31:03.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:03.499+0000 7f061539b700 1 --2- 172.21.15.107:0/3567380437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06100fcc20 0x7f061010d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:03.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:03.499+0000 7f0614b9a700 1 -- 172.21.15.107:0/3567380437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06100fcc20 msgr2=0x7f061010d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:03.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:03.499+0000 7f0614b9a700 1 --2- 172.21.15.107:0/3567380437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06100fcc20 0x7f061010d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:03.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:03.499+0000 7f0614b9a700 1 -- 172.21.15.107:0/3567380437 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f060c005040 con 0x7f06100fded0 2024-04-22T23:31:03.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:03.500+0000 7f0614b9a700 1 --2- 172.21.15.107:0/3567380437 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06100fded0 0x7f061010dd90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0600002a50 tx=0x7f06000046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:31:03.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:03.500+0000 7f0605ffb700 1 -- 172.21.15.107:0/3567380437 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0600007cc0 con 0x7f06100fded0 2024-04-22T23:31:03.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:03.500+0000 7f0605ffb700 1 -- 172.21.15.107:0/3567380437 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f060000f040 con 0x7f06100fded0 2024-04-22T23:31:03.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:03.500+0000 7f0605ffb700 1 -- 172.21.15.107:0/3567380437 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f060000da30 con 0x7f06100fded0 2024-04-22T23:31:03.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:03.500+0000 7f061639d700 1 -- 172.21.15.107:0/3567380437 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f061010ee50 con 0x7f06100fded0 2024-04-22T23:31:03.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:03.500+0000 7f061639d700 1 -- 172.21.15.107:0/3567380437 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f061010f390 con 0x7f06100fded0 2024-04-22T23:31:03.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:03.501+0000 7f061639d700 1 -- 172.21.15.107:0/3567380437 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06100fc020 con 0x7f06100fded0 2024-04-22T23:31:03.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:03.501+0000 7f0605ffb700 1 -- 172.21.15.107:0/3567380437 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f060000db90 con 0x7f06100fded0 2024-04-22T23:31:03.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:03.502+0000 7f0605ffb700 1 --2- 172.21.15.107:0/3567380437 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f05fc0692a0 0x7f05fc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:03.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:03.502+0000 7f0605ffb700 1 -- 172.21.15.107:0/3567380437 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f0600086b30 con 0x7f06100fded0 2024-04-22T23:31:03.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:03.502+0000 7f061539b700 1 --2- 172.21.15.107:0/3567380437 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f05fc0692a0 0x7f05fc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:03.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:03.502+0000 7f061539b700 1 --2- 172.21.15.107:0/3567380437 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f05fc0692a0 0x7f05fc06b750 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f060c00b970 tx=0x7f060c009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:31:03.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:03.506+0000 7f0605ffb700 1 -- 172.21.15.107:0/3567380437 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0600056c30 con 0x7f06100fded0 2024-04-22T23:31:03.530 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:03 smithi157 ceph-mon[33877]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-22T23:31:03.530 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:03 smithi157 ceph-mon[33877]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]': finished 2024-04-22T23:31:03.530 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:03 smithi157 ceph-mon[33877]: osdmap e38: 6 total, 5 up, 6 in 2024-04-22T23:31:03.530 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:03 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T23:31:03.530 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:03 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T23:31:03.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:03.690+0000 7f061639d700 1 -- 172.21.15.107:0/3567380437 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f061010f6a0 con 0x7f06100fded0 2024-04-22T23:31:03.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:03.690+0000 7f0605ffb700 1 -- 172.21.15.107:0/3567380437 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v39) v1 ==== 74+0+130 (secure 0 0 0) 0x7f06000567c0 con 0x7f06100fded0 2024-04-22T23:31:03.689 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:31:03.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:03.693+0000 7f061639d700 1 -- 172.21.15.107:0/3567380437 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f05fc0692a0 msgr2=0x7f05fc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:03.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:03.693+0000 7f061639d700 1 --2- 172.21.15.107:0/3567380437 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f05fc0692a0 0x7f05fc06b750 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f060c00b970 tx=0x7f060c009e30 comp rx=0 tx=0).stop 2024-04-22T23:31:03.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:03.693+0000 7f061639d700 1 -- 172.21.15.107:0/3567380437 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06100fded0 msgr2=0x7f061010dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:03.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:03.693+0000 7f061639d700 1 --2- 172.21.15.107:0/3567380437 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06100fded0 0x7f061010dd90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0600002a50 tx=0x7f06000046b0 comp rx=0 tx=0).stop 2024-04-22T23:31:03.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:03.693+0000 7f061639d700 1 -- 172.21.15.107:0/3567380437 shutdown_connections 2024-04-22T23:31:03.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:03.693+0000 7f061639d700 1 --2- 172.21.15.107:0/3567380437 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f05fc0692a0 0x7f05fc06b750 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:03.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:03.693+0000 7f061639d700 1 --2- 172.21.15.107:0/3567380437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06100fcc20 0x7f061010d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:03.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:03.693+0000 7f061639d700 1 --2- 172.21.15.107:0/3567380437 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06100fded0 0x7f061010dd90 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:03.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:03.693+0000 7f061639d700 1 -- 172.21.15.107:0/3567380437 >> 172.21.15.107:0/3567380437 conn(0x7f06100f83a0 msgr2=0x7f06100f9a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:31:03.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:03.693+0000 7f061639d700 1 -- 172.21.15.107:0/3567380437 shutdown_connections 2024-04-22T23:31:03.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:03.693+0000 7f061639d700 1 -- 172.21.15.107:0/3567380437 wait complete. 2024-04-22T23:31:04.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:04 smithi107 ceph-mon[26355]: purged_snaps scrub starts 2024-04-22T23:31:04.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:04 smithi107 ceph-mon[26355]: purged_snaps scrub ok 2024-04-22T23:31:04.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:04 smithi107 ceph-mon[26355]: osd.5 [v2:172.21.15.157:6816/2125627920,v1:172.21.15.157:6817/2125627920] boot 2024-04-22T23:31:04.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:04 smithi107 ceph-mon[26355]: osdmap e39: 6 total, 6 up, 6 in 2024-04-22T23:31:04.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:04 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T23:31:04.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:04 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/3567380437' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-22T23:31:04.337 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:04 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:31:04.362 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":39,"num_osds":6,"num_up_osds":6,"osd_up_since":1713828663,"num_in_osds":6,"osd_in_since":1713828644,"num_remapped_pgs":0} 2024-04-22T23:31:04.363 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-22T23:31:04.480 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:04 smithi157 ceph-mon[33877]: purged_snaps scrub starts 2024-04-22T23:31:04.480 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:04 smithi157 ceph-mon[33877]: purged_snaps scrub ok 2024-04-22T23:31:04.480 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:04 smithi157 ceph-mon[33877]: osd.5 [v2:172.21.15.157:6816/2125627920,v1:172.21.15.157:6817/2125627920] boot 2024-04-22T23:31:04.480 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:04 smithi157 ceph-mon[33877]: osdmap e39: 6 total, 6 up, 6 in 2024-04-22T23:31:04.481 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:04 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T23:31:04.481 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:04 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/3567380437' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-22T23:31:04.481 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:04 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:31:04.629 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:31:05.236 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:05 smithi107 ceph-mon[26355]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-22T23:31:05.236 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:05 smithi107 ceph-mon[26355]: osdmap e40: 6 total, 6 up, 6 in 2024-04-22T23:31:05.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:05 smithi157 ceph-mon[33877]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-22T23:31:05.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:05 smithi157 ceph-mon[33877]: osdmap e40: 6 total, 6 up, 6 in 2024-04-22T23:31:06.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:06.215+0000 7fe309661700 1 -- 172.21.15.107:0/1163607475 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe304101d90 msgr2=0x7fe304102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:06.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:06.215+0000 7fe309661700 1 --2- 172.21.15.107:0/1163607475 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe304101d90 0x7fe304102160 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc009ce0 tx=0x7fe2fc004d10 comp rx=0 tx=0).stop 2024-04-22T23:31:06.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:06.215+0000 7fe309661700 1 -- 172.21.15.107:0/1163607475 shutdown_connections 2024-04-22T23:31:06.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:06.215+0000 7fe309661700 1 --2- 172.21.15.107:0/1163607475 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe304102730 0x7fe3040fbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:06.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:06.215+0000 7fe309661700 1 --2- 172.21.15.107:0/1163607475 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe304101d90 0x7fe304102160 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:06.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:06.215+0000 7fe309661700 1 -- 172.21.15.107:0/1163607475 >> 172.21.15.107:0/1163607475 conn(0x7fe3040f7900 msgr2=0x7fe3040f9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:31:06.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:06.215+0000 7fe309661700 1 -- 172.21.15.107:0/1163607475 shutdown_connections 2024-04-22T23:31:06.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:06.215+0000 7fe309661700 1 -- 172.21.15.107:0/1163607475 wait complete. 2024-04-22T23:31:06.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:06.216+0000 7fe309661700 1 Processor -- start 2024-04-22T23:31:06.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:06.216+0000 7fe309661700 1 -- start start 2024-04-22T23:31:06.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:06.216+0000 7fe309661700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe304101d90 0x7fe304113ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:06.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:06.216+0000 7fe309661700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe304102730 0x7fe304114010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:06.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:06.216+0000 7fe309661700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe304114660 con 0x7fe304101d90 2024-04-22T23:31:06.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:06.216+0000 7fe309661700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe30410dc40 con 0x7fe304102730 2024-04-22T23:31:06.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:06.216+0000 7fe302ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe304101d90 0x7fe304113ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:06.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:06.216+0000 7fe302ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe304101d90 0x7fe304113ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43006/0 (socket says 172.21.15.107:43006) 2024-04-22T23:31:06.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:06.216+0000 7fe302ffd700 1 -- 172.21.15.107:0/2696114558 learned_addr learned my addr 172.21.15.107:0/2696114558 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:31:06.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:06.216+0000 7fe3027fc700 1 --2- 172.21.15.107:0/2696114558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe304102730 0x7fe304114010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:06.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:06.216+0000 7fe302ffd700 1 -- 172.21.15.107:0/2696114558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe304102730 msgr2=0x7fe304114010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:06.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:06.216+0000 7fe302ffd700 1 --2- 172.21.15.107:0/2696114558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe304102730 0x7fe304114010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:06.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:06.216+0000 7fe302ffd700 1 -- 172.21.15.107:0/2696114558 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2fc009940 con 0x7fe304101d90 2024-04-22T23:31:06.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:06.216+0000 7fe3027fc700 1 --2- 172.21.15.107:0/2696114558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe304102730 0x7fe304114010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T23:31:06.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:06.216+0000 7fe302ffd700 1 --2- 172.21.15.107:0/2696114558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe304101d90 0x7fe304113ad0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc00be90 tx=0x7fe2fc004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:31:06.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:06.216+0000 7fe2eb7fe700 1 -- 172.21.15.107:0/2696114558 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2fc00e040 con 0x7fe304101d90 2024-04-22T23:31:06.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:06.217+0000 7fe2eb7fe700 1 -- 172.21.15.107:0/2696114558 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe2fc008ea0 con 0x7fe304101d90 2024-04-22T23:31:06.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:06.217+0000 7fe2eb7fe700 1 -- 172.21.15.107:0/2696114558 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2fc01c810 con 0x7fe304101d90 2024-04-22T23:31:06.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:06.217+0000 7fe309661700 1 -- 172.21.15.107:0/2696114558 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe30410dec0 con 0x7fe304101d90 2024-04-22T23:31:06.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:06.217+0000 7fe309661700 1 -- 172.21.15.107:0/2696114558 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe30410e480 con 0x7fe304101d90 2024-04-22T23:31:06.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:06.217+0000 7fe309661700 1 -- 172.21.15.107:0/2696114558 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3040fd4e0 con 0x7fe304101d90 2024-04-22T23:31:06.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:06.218+0000 7fe2eb7fe700 1 -- 172.21.15.107:0/2696114558 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fe2fc012c00 con 0x7fe304101d90 2024-04-22T23:31:06.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:06.218+0000 7fe2eb7fe700 1 --2- 172.21.15.107:0/2696114558 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fe2ec0692a0 0x7fe2ec06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:06.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:06.218+0000 7fe2eb7fe700 1 -- 172.21.15.107:0/2696114558 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fe2fc088ad0 con 0x7fe304101d90 2024-04-22T23:31:06.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:06.218+0000 7fe3027fc700 1 --2- 172.21.15.107:0/2696114558 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fe2ec0692a0 0x7fe2ec06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:06.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:06.219+0000 7fe3027fc700 1 --2- 172.21.15.107:0/2696114558 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fe2ec0692a0 0x7fe2ec06b750 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe2f8007fd0 tx=0x7fe2f8009040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:31:06.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:06.223+0000 7fe2eb7fe700 1 -- 172.21.15.107:0/2696114558 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe2fc058b50 con 0x7fe304101d90 2024-04-22T23:31:06.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:06.409+0000 7fe309661700 1 -- 172.21.15.107:0/2696114558 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe30410f120 con 0x7fe304101d90 2024-04-22T23:31:06.408 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:06 smithi107 ceph-mon[26355]: Detected new or changed devices on smithi157 2024-04-22T23:31:06.408 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:06 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:31:06.408 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:06 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:31:06.408 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:06 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:31:06.408 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:06 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:31:06.408 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:06 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:31:06.408 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:06 smithi107 ceph-mon[26355]: Adjusting osd_memory_target on smithi157 to 5312M 2024-04-22T23:31:06.408 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:06 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:31:06.408 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:06 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:31:06.408 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:06 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:31:06.408 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:06 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:31:06.408 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:06 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:31:06.409 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:06 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:31:06.409 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:06 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:31:06.409 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:06 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:31:06.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:06.410+0000 7fe2eb7fe700 1 -- 172.21.15.107:0/2696114558 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11107 (secure 0 0 0) 0x7fe2fc004e10 con 0x7fe304101d90 2024-04-22T23:31:06.410 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:31:06.410 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":40,"fsid":"8252f6f2-00ff-11ef-bc93-c7b262605968","created":"2024-04-22T23:25:25.497133+0000","modified":"2024-04-22T23:31:04.176098+0000","last_up_change":"2024-04-22T23:31:03.170868+0000","last_in_change":"2024-04-22T23:30:44.396843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T23:29:48.249903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"65404290-734a-4901-bbb4-a87d3d846b06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4253936450},{"type":"v1","addr":"172.21.15.107:6803","nonce":4253936450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4253936450},{"type":"v1","addr":"172.21.15.107:6805","nonce":4253936450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":4253936450},{"type":"v1","addr":"172.21.15.107:6809","nonce":4253936450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4253936450},{"type":"v1","addr":"172.21.15.107:6807","nonce":4253936450}]},"public_addr":"172.21.15.107:6803/4253936450","cluster_addr":"172.21.15.107:6805/4253936450","heartbeat_back_addr":"172.21.15.107:6809/4253936450","heartbeat_front_addr":"172.21.15.107:6807/4253936450","state":["exists","up"]},{"osd":1,"uuid":"6df381f7-b907-4d1b-bd08-fdf34e9729f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":335851250},{"type":"v1","addr":"172.21.15.107:6811","nonce":335851250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":335851250},{"type":"v1","addr":"172.21.15.107:6813","nonce":335851250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":335851250},{"type":"v1","addr":"172.21.15.107:6817","nonce":335851250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":335851250},{"type":"v1","addr":"172.21.15.107:6815","nonce":335851250}]},"public_addr":"172.21.15.107:6811/335851250","cluster_addr":"172.21.15.107:6813/335851250","heartbeat_back_addr":"172.21.15.107:6817/335851250","heartbeat_front_addr":"172.21.15.107:6815/335851250","state":["exists","up"]},{"osd":2,"uuid":"2c5e5c37-c1de-4e64-9c21-eacc145f1412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":750102033},{"type":"v1","addr":"172.21.15.107:6819","nonce":750102033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":750102033},{"type":"v1","addr":"172.21.15.107:6821","nonce":750102033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":750102033},{"type":"v1","addr":"172.21.15.107:6825","nonce":750102033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":750102033},{"type":"v1","addr":"172.21.15.107:6823","nonce":750102033}]},"public_addr":"172.21.15.107:6819/750102033","cluster_addr":"172.21.15.107:6821/750102033","heartbeat_back_addr":"172.21.15.107:6825/750102033","heartbeat_front_addr":"172.21.15.107:6823/750102033","state":["exists","up"]},{"osd":3,"uuid":"b7c9c124-9f43-4b5b-b039-c61ae30a416d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":811815812},{"type":"v1","addr":"172.21.15.157:6801","nonce":811815812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":811815812},{"type":"v1","addr":"172.21.15.157:6803","nonce":811815812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":811815812},{"type":"v1","addr":"172.21.15.157:6807","nonce":811815812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":811815812},{"type":"v1","addr":"172.21.15.157:6805","nonce":811815812}]},"public_addr":"172.21.15.157:6801/811815812","cluster_addr":"172.21.15.157:6803/811815812","heartbeat_back_addr":"172.21.15.157:6807/811815812","heartbeat_front_addr":"172.21.15.157:6805/811815812","state":["exists","up"]},{"osd":4,"uuid":"34a0cb61-1c2b-4515-9d74-8a3ad3f15fbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2372899121},{"type":"v1","addr":"172.21.15.157:6809","nonce":2372899121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2372899121},{"type":"v1","addr":"172.21.15.157:6811","nonce":2372899121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2372899121},{"type":"v1","addr":"172.21.15.157:6815","nonce":2372899121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2372899121},{"type":"v1","addr":"172.21.15.157:6813","nonce":2372899121}]},"public_addr":"172.21.15.157:6809/2372899121","cluster_addr":"172.21.15.157:6811/2372899121","heartbeat_back_addr":"172.21.15.157:6815/2372899121","heartbeat_front_addr":"172.21.15.157:6813/2372899121","state":["exists","up"]},{"osd":5,"uuid":"00934a7f-7e88-45cf-a2cc-e8a94bceaab0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2125627920},{"type":"v1","addr":"172.21.15.157:6817","nonce":2125627920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2125627920},{"type":"v1","addr":"172.21.15.157:6819","nonce":2125627920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2125627920},{"type":"v1","addr":"172.21.15.157:6823","nonce":2125627920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2125627920},{"type":"v1","addr":"172.21.15.157:6821","nonce":2125627920}]},"public_addr":"172.21.15.157:6817/2125627920","cluster_addr":"172.21.15.157:6819/2125627920","heartbeat_back_addr":"172.21.15.157:6823/2125627920","heartbeat_front_addr":"172.21.15.157:6821/2125627920","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:28:48.161289+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:29:18.447013+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:29:45.948165+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:30:09.825103+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:30:35.976009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:31:01.729305+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:6801/3427816626":"2024-04-23T23:27:33.931895+0000","172.21.15.107:6800/3427816626":"2024-04-23T23:27:33.931895+0000","172.21.15.107:6801/307474287":"2024-04-23T23:26:29.807373+0000","172.21.15.107:6800/307474287":"2024-04-23T23:26:29.807373+0000","172.21.15.107:6800/467262774":"2024-04-23T23:25:58.816560+0000","172.21.15.107:0/1029440409":"2024-04-23T23:26:29.807373+0000","172.21.15.107:0/3810721822":"2024-04-23T23:25:58.816560+0000","172.21.15.107:0/1205692115":"2024-04-23T23:27:33.931895+0000","172.21.15.107:0/3608791958":"2024-04-23T23:27:33.931895+0000","172.21.15.107:6801/467262774":"2024-04-23T23:25:58.816560+0000","172.21.15.107:0/3377956937":"2024-04-23T23:25:58.816560+0000","172.21.15.107:0/1734120985":"2024-04-23T23:27:33.931895+0000","172.21.15.107:0/3920905056":"2024-04-23T23:25:58.816560+0000","172.21.15.107:0/2444490678":"2024-04-23T23:26:29.807373+0000","172.21.15.107:0/663073386":"2024-04-23T23:26:29.807373+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T23:31:06.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:06.412+0000 7fe309661700 1 -- 172.21.15.107:0/2696114558 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fe2ec0692a0 msgr2=0x7fe2ec06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:06.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:06.412+0000 7fe309661700 1 --2- 172.21.15.107:0/2696114558 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fe2ec0692a0 0x7fe2ec06b750 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe2f8007fd0 tx=0x7fe2f8009040 comp rx=0 tx=0).stop 2024-04-22T23:31:06.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:06.412+0000 7fe309661700 1 -- 172.21.15.107:0/2696114558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe304101d90 msgr2=0x7fe304113ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:06.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:06.412+0000 7fe309661700 1 --2- 172.21.15.107:0/2696114558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe304101d90 0x7fe304113ad0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc00be90 tx=0x7fe2fc004d10 comp rx=0 tx=0).stop 2024-04-22T23:31:06.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:06.412+0000 7fe309661700 1 -- 172.21.15.107:0/2696114558 shutdown_connections 2024-04-22T23:31:06.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:06.412+0000 7fe309661700 1 --2- 172.21.15.107:0/2696114558 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fe2ec0692a0 0x7fe2ec06b750 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:06.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:06.412+0000 7fe309661700 1 --2- 172.21.15.107:0/2696114558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe304101d90 0x7fe304113ad0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:06.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:06.412+0000 7fe309661700 1 --2- 172.21.15.107:0/2696114558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe304102730 0x7fe304114010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:06.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:06.412+0000 7fe309661700 1 -- 172.21.15.107:0/2696114558 >> 172.21.15.107:0/2696114558 conn(0x7fe3040f7900 msgr2=0x7fe304105720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:31:06.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:06.412+0000 7fe309661700 1 -- 172.21.15.107:0/2696114558 shutdown_connections 2024-04-22T23:31:06.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:06.412+0000 7fe309661700 1 -- 172.21.15.107:0/2696114558 wait complete. 2024-04-22T23:31:06.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:06 smithi157 ceph-mon[33877]: Detected new or changed devices on smithi157 2024-04-22T23:31:06.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:06 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:31:06.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:06 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:31:06.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:06 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:31:06.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:06 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:31:06.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:06 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:31:06.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:06 smithi157 ceph-mon[33877]: Adjusting osd_memory_target on smithi157 to 5312M 2024-04-22T23:31:06.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:06 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:31:06.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:06 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:31:06.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:06 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:31:06.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:06 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:31:06.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:06 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:31:06.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:06 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:31:06.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:06 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:31:06.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:06 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:31:06.934 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T23:29:48.249903+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-04-22T23:31:06.934 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-22T23:31:07.187 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:31:07.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:07 smithi157 ceph-mon[33877]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:31:07.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:07 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/2696114558' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T23:31:07.642 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:07 smithi107 ceph-mon[26355]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:31:07.643 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:07 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/2696114558' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T23:31:08.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:08.773+0000 7f1cff605700 1 -- 172.21.15.107:0/1214411370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1cf81037b0 msgr2=0x7f1cf80fce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:08.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:08.773+0000 7f1cff605700 1 --2- 172.21.15.107:0/1214411370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1cf81037b0 0x7f1cf80fce20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4004940 tx=0x7f1cf4009e30 comp rx=0 tx=0).stop 2024-04-22T23:31:08.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:08.773+0000 7f1cff605700 1 -- 172.21.15.107:0/1214411370 shutdown_connections 2024-04-22T23:31:08.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:08.773+0000 7f1cff605700 1 --2- 172.21.15.107:0/1214411370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1cf81037b0 0x7f1cf80fce20 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:08.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:08.773+0000 7f1cff605700 1 --2- 172.21.15.107:0/1214411370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cf8102e10 0x7f1cf81031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:08.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:08.773+0000 7f1cff605700 1 -- 172.21.15.107:0/1214411370 >> 172.21.15.107:0/1214411370 conn(0x7f1cf80f89a0 msgr2=0x7f1cf80fadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:31:08.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:08.773+0000 7f1cff605700 1 -- 172.21.15.107:0/1214411370 shutdown_connections 2024-04-22T23:31:08.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:08.773+0000 7f1cff605700 1 -- 172.21.15.107:0/1214411370 wait complete. 2024-04-22T23:31:08.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:08.773+0000 7f1cff605700 1 Processor -- start 2024-04-22T23:31:08.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:08.774+0000 7f1cff605700 1 -- start start 2024-04-22T23:31:08.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:08.774+0000 7f1cff605700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cf8102e10 0x7f1cf8113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:08.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:08.774+0000 7f1cff605700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1cf81037b0 0x7f1cf81140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:08.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:08.774+0000 7f1cff605700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf8114710 con 0x7f1cf81037b0 2024-04-22T23:31:08.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:08.774+0000 7f1cff605700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf810db00 con 0x7f1cf8102e10 2024-04-22T23:31:08.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:08.774+0000 7f1cfcba0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1cf81037b0 0x7f1cf81140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:08.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:08.774+0000 7f1cfcba0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1cf81037b0 0x7f1cf81140c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43020/0 (socket says 172.21.15.107:43020) 2024-04-22T23:31:08.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:08.774+0000 7f1cfcba0700 1 -- 172.21.15.107:0/3614843174 learned_addr learned my addr 172.21.15.107:0/3614843174 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:31:08.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:08.774+0000 7f1cfd3a1700 1 --2- 172.21.15.107:0/3614843174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cf8102e10 0x7f1cf8113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:08.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:08.774+0000 7f1cfcba0700 1 -- 172.21.15.107:0/3614843174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cf8102e10 msgr2=0x7f1cf8113b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:08.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:08.774+0000 7f1cfcba0700 1 --2- 172.21.15.107:0/3614843174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cf8102e10 0x7f1cf8113b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:08.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:08.774+0000 7f1cfcba0700 1 -- 172.21.15.107:0/3614843174 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf4005040 con 0x7f1cf81037b0 2024-04-22T23:31:08.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:08.774+0000 7f1cfcba0700 1 --2- 172.21.15.107:0/3614843174 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1cf81037b0 0x7f1cf81140c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4009180 tx=0x7f1cf4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:31:08.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:08.774+0000 7f1cedffb700 1 -- 172.21.15.107:0/3614843174 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1cf400e040 con 0x7f1cf81037b0 2024-04-22T23:31:08.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:08.774+0000 7f1cedffb700 1 -- 172.21.15.107:0/3614843174 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1cf400bbc0 con 0x7f1cf81037b0 2024-04-22T23:31:08.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:08.774+0000 7f1cff605700 1 -- 172.21.15.107:0/3614843174 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cf810dd80 con 0x7f1cf81037b0 2024-04-22T23:31:08.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:08.774+0000 7f1cedffb700 1 -- 172.21.15.107:0/3614843174 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1cf401c3f0 con 0x7f1cf81037b0 2024-04-22T23:31:08.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:08.775+0000 7f1cff605700 1 -- 172.21.15.107:0/3614843174 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cf810e340 con 0x7f1cf81037b0 2024-04-22T23:31:08.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:08.776+0000 7f1cff605700 1 -- 172.21.15.107:0/3614843174 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cf81052e0 con 0x7f1cf81037b0 2024-04-22T23:31:08.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:08.776+0000 7f1cedffb700 1 -- 172.21.15.107:0/3614843174 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f1cf4013420 con 0x7f1cf81037b0 2024-04-22T23:31:08.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:08.776+0000 7f1cedffb700 1 --2- 172.21.15.107:0/3614843174 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f1ce40692a0 0x7f1ce406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:08.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:08.776+0000 7f1cedffb700 1 -- 172.21.15.107:0/3614843174 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f1cf40881e0 con 0x7f1cf81037b0 2024-04-22T23:31:08.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:08.777+0000 7f1cfd3a1700 1 --2- 172.21.15.107:0/3614843174 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f1ce40692a0 0x7f1ce406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:08.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:08.777+0000 7f1cfd3a1700 1 --2- 172.21.15.107:0/3614843174 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f1ce40692a0 0x7f1ce406b750 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8007fd0 tx=0x7f1ce8009040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:31:08.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:08.781+0000 7f1cedffb700 1 -- 172.21.15.107:0/3614843174 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f1cf40582e0 con 0x7f1cf81037b0 2024-04-22T23:31:08.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:08.964+0000 7f1cff605700 1 -- 172.21.15.107:0/3614843174 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1cf810f1d0 con 0x7f1cf81037b0 2024-04-22T23:31:08.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:08.964+0000 7f1cedffb700 1 -- 172.21.15.107:0/3614843174 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1cf4057e70 con 0x7f1cf81037b0 2024-04-22T23:31:08.962 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2024-04-22T23:31:08.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:08.966+0000 7f1cff605700 1 -- 172.21.15.107:0/3614843174 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f1ce40692a0 msgr2=0x7f1ce406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:08.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:08.966+0000 7f1cff605700 1 --2- 172.21.15.107:0/3614843174 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f1ce40692a0 0x7f1ce406b750 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8007fd0 tx=0x7f1ce8009040 comp rx=0 tx=0).stop 2024-04-22T23:31:08.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:08.966+0000 7f1cff605700 1 -- 172.21.15.107:0/3614843174 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1cf81037b0 msgr2=0x7f1cf81140c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:08.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:08.966+0000 7f1cff605700 1 --2- 172.21.15.107:0/3614843174 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1cf81037b0 0x7f1cf81140c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4009180 tx=0x7f1cf4009e30 comp rx=0 tx=0).stop 2024-04-22T23:31:08.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:08.966+0000 7f1cff605700 1 -- 172.21.15.107:0/3614843174 shutdown_connections 2024-04-22T23:31:08.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:08.966+0000 7f1cff605700 1 --2- 172.21.15.107:0/3614843174 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f1ce40692a0 0x7f1ce406b750 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:08.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:08.966+0000 7f1cff605700 1 --2- 172.21.15.107:0/3614843174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cf8102e10 0x7f1cf8113b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:08.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:08.966+0000 7f1cff605700 1 --2- 172.21.15.107:0/3614843174 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1cf81037b0 0x7f1cf81140c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:08.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:08.966+0000 7f1cff605700 1 -- 172.21.15.107:0/3614843174 >> 172.21.15.107:0/3614843174 conn(0x7f1cf80f89a0 msgr2=0x7f1cf81021d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:31:08.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:08.966+0000 7f1cff605700 1 -- 172.21.15.107:0/3614843174 shutdown_connections 2024-04-22T23:31:08.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:08.967+0000 7f1cff605700 1 -- 172.21.15.107:0/3614843174 wait complete. 2024-04-22T23:31:09.593 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:09 smithi107 ceph-mon[26355]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:31:09.593 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:09 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/3614843174' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T23:31:09.618 INFO:tasks.cephadm:Setting up client nodes... 2024-04-22T23:31:09.618 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-22T23:31:09.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:09 smithi157 ceph-mon[33877]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:31:09.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:09 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/3614843174' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T23:31:09.867 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:31:11.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:11.522+0000 7f233c941700 1 -- 172.21.15.107:0/949933599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23380fbd00 msgr2=0x7f23380fc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:11.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:11.522+0000 7f233c941700 1 --2- 172.21.15.107:0/949933599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23380fbd00 0x7f23380fc110 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2328004940 tx=0x7f2328009e30 comp rx=0 tx=0).stop 2024-04-22T23:31:11.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:11.522+0000 7f233c941700 1 -- 172.21.15.107:0/949933599 shutdown_connections 2024-04-22T23:31:11.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:11.522+0000 7f233c941700 1 --2- 172.21.15.107:0/949933599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23380fcfb0 0x7f23380fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:11.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:11.522+0000 7f233c941700 1 --2- 172.21.15.107:0/949933599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23380fbd00 0x7f23380fc110 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:11.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:11.522+0000 7f233c941700 1 -- 172.21.15.107:0/949933599 >> 172.21.15.107:0/949933599 conn(0x7f23380f7460 msgr2=0x7f23380f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:31:11.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:11.522+0000 7f233c941700 1 -- 172.21.15.107:0/949933599 shutdown_connections 2024-04-22T23:31:11.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:11.522+0000 7f233c941700 1 -- 172.21.15.107:0/949933599 wait complete. 2024-04-22T23:31:11.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:11.523+0000 7f233c941700 1 Processor -- start 2024-04-22T23:31:11.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:11.523+0000 7f233c941700 1 -- start start 2024-04-22T23:31:11.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:11.523+0000 7f233c941700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23380fbd00 0x7f233810c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:11.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:11.523+0000 7f233c941700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23380fcfb0 0x7f233810cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:11.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:11.523+0000 7f233c941700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f233810d440 con 0x7f23380fcfb0 2024-04-22T23:31:11.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:11.523+0000 7f233c941700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2338111de0 con 0x7f23380fbd00 2024-04-22T23:31:11.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:11.523+0000 7f2335d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23380fcfb0 0x7f233810cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:11.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:11.523+0000 7f2335d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23380fcfb0 0x7f233810cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58040/0 (socket says 172.21.15.107:58040) 2024-04-22T23:31:11.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:11.523+0000 7f2335d9b700 1 -- 172.21.15.107:0/2868682955 learned_addr learned my addr 172.21.15.107:0/2868682955 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:31:11.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:11.523+0000 7f233659c700 1 --2- 172.21.15.107:0/2868682955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23380fbd00 0x7f233810c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:11.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:11.524+0000 7f2335d9b700 1 -- 172.21.15.107:0/2868682955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23380fbd00 msgr2=0x7f233810c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:11.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:11.524+0000 7f2335d9b700 1 --2- 172.21.15.107:0/2868682955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23380fbd00 0x7f233810c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:11.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:11.524+0000 7f2335d9b700 1 -- 172.21.15.107:0/2868682955 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2328005040 con 0x7f23380fcfb0 2024-04-22T23:31:11.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:11.524+0000 7f233659c700 1 --2- 172.21.15.107:0/2868682955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23380fbd00 0x7f233810c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T23:31:11.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:11.524+0000 7f2335d9b700 1 --2- 172.21.15.107:0/2868682955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23380fcfb0 0x7f233810cdb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f232c00b790 tx=0x7f232c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:31:11.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:11.524+0000 7f2326ffd700 1 -- 172.21.15.107:0/2868682955 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f232c00f770 con 0x7f23380fcfb0 2024-04-22T23:31:11.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:11.524+0000 7f2326ffd700 1 -- 172.21.15.107:0/2868682955 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f232c00fda0 con 0x7f23380fcfb0 2024-04-22T23:31:11.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:11.524+0000 7f233c941700 1 -- 172.21.15.107:0/2868682955 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2338112040 con 0x7f23380fcfb0 2024-04-22T23:31:11.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:11.524+0000 7f2326ffd700 1 -- 172.21.15.107:0/2868682955 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f232c00d310 con 0x7f23380fcfb0 2024-04-22T23:31:11.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:11.524+0000 7f233c941700 1 -- 172.21.15.107:0/2868682955 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2338112580 con 0x7f23380fcfb0 2024-04-22T23:31:11.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:11.525+0000 7f233c941700 1 -- 172.21.15.107:0/2868682955 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f233804c710 con 0x7f23380fcfb0 2024-04-22T23:31:11.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:11.525+0000 7f2326ffd700 1 -- 172.21.15.107:0/2868682955 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f232c00f8d0 con 0x7f23380fcfb0 2024-04-22T23:31:11.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:11.526+0000 7f2326ffd700 1 --2- 172.21.15.107:0/2868682955 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f23200692a0 0x7f232006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:11.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:11.526+0000 7f233659c700 1 --2- 172.21.15.107:0/2868682955 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f23200692a0 0x7f232006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:11.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:11.526+0000 7f2326ffd700 1 -- 172.21.15.107:0/2868682955 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f232c086e80 con 0x7f23380fcfb0 2024-04-22T23:31:11.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:11.526+0000 7f233659c700 1 --2- 172.21.15.107:0/2868682955 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f23200692a0 0x7f232006b750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2328000c00 tx=0x7f2328009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:31:11.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:11.530+0000 7f2326ffd700 1 -- 172.21.15.107:0/2868682955 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f232c056f80 con 0x7f23380fcfb0 2024-04-22T23:31:11.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:11 smithi157 ceph-mon[33877]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:31:11.736 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:11 smithi107 ceph-mon[26355]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:31:11.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:11.795+0000 7f233c941700 1 -- 172.21.15.107:0/2868682955 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f2338104870 con 0x7f23380fcfb0 2024-04-22T23:31:11.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:11.798+0000 7f2326ffd700 1 -- 172.21.15.107:0/2868682955 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v14) v1 ==== 170+0+59 (secure 0 0 0) 0x7f232c020250 con 0x7f23380fcfb0 2024-04-22T23:31:11.797 INFO:teuthology.orchestra.run.smithi107.stdout:[client.0] 2024-04-22T23:31:11.797 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQA/8yZmcCCHLxAAXzM74Zb6DjtQ5mefhCHAYA== 2024-04-22T23:31:11.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:11.800+0000 7f233c941700 1 -- 172.21.15.107:0/2868682955 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f23200692a0 msgr2=0x7f232006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:11.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:11.800+0000 7f233c941700 1 --2- 172.21.15.107:0/2868682955 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f23200692a0 0x7f232006b750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2328000c00 tx=0x7f2328009e30 comp rx=0 tx=0).stop 2024-04-22T23:31:11.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:11.800+0000 7f233c941700 1 -- 172.21.15.107:0/2868682955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23380fcfb0 msgr2=0x7f233810cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:11.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:11.800+0000 7f233c941700 1 --2- 172.21.15.107:0/2868682955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23380fcfb0 0x7f233810cdb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f232c00b790 tx=0x7f232c009250 comp rx=0 tx=0).stop 2024-04-22T23:31:11.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:11.800+0000 7f233c941700 1 -- 172.21.15.107:0/2868682955 shutdown_connections 2024-04-22T23:31:11.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:11.800+0000 7f233c941700 1 --2- 172.21.15.107:0/2868682955 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f23200692a0 0x7f232006b750 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:11.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:11.801+0000 7f233c941700 1 --2- 172.21.15.107:0/2868682955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f23380fbd00 0x7f233810c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:11.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:11.801+0000 7f233c941700 1 --2- 172.21.15.107:0/2868682955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23380fcfb0 0x7f233810cdb0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:11.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:11.801+0000 7f233c941700 1 -- 172.21.15.107:0/2868682955 >> 172.21.15.107:0/2868682955 conn(0x7f23380f7460 msgr2=0x7f23380f8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:31:11.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:11.801+0000 7f233c941700 1 -- 172.21.15.107:0/2868682955 shutdown_connections 2024-04-22T23:31:11.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:11.801+0000 7f233c941700 1 -- 172.21.15.107:0/2868682955 wait complete. 2024-04-22T23:31:12.509 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-22T23:31:12.509 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-22T23:31:12.509 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-22T23:31:12.546 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-22T23:31:12.736 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:12 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/2868682955' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-22T23:31:12.736 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:12 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/2868682955' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-22T23:31:12.813 INFO:teuthology.orchestra.run.smithi157.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi157/config 2024-04-22T23:31:12.884 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:12 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/2868682955' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-22T23:31:12.884 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:12 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/2868682955' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-22T23:31:13.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:13 smithi157 ceph-mon[33877]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:31:13.736 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:13 smithi107 ceph-mon[26355]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:31:14.447 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:14.448+0000 7f25c5253700 1 -- 172.21.15.157:0/3650808896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25c00fbd40 msgr2=0x7f25c00fc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:14.447 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:14.448+0000 7f25c5253700 1 --2- 172.21.15.157:0/3650808896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25c00fbd40 0x7f25c00fc150 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f25b0004940 tx=0x7f25b0009e30 comp rx=0 tx=0).stop 2024-04-22T23:31:14.447 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:14.448+0000 7f25c5253700 1 -- 172.21.15.157:0/3650808896 shutdown_connections 2024-04-22T23:31:14.447 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:14.448+0000 7f25c5253700 1 --2- 172.21.15.157:0/3650808896 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25c00fcff0 0x7f25c00fd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:14.447 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:14.448+0000 7f25c5253700 1 --2- 172.21.15.157:0/3650808896 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25c00fbd40 0x7f25c00fc150 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:14.447 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:14.448+0000 7f25c5253700 1 -- 172.21.15.157:0/3650808896 >> 172.21.15.157:0/3650808896 conn(0x7f25c00f7460 msgr2=0x7f25c00f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:31:14.447 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:14.448+0000 7f25c5253700 1 -- 172.21.15.157:0/3650808896 shutdown_connections 2024-04-22T23:31:14.447 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:14.448+0000 7f25c5253700 1 -- 172.21.15.157:0/3650808896 wait complete. 2024-04-22T23:31:14.447 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:14.449+0000 7f25c5253700 1 Processor -- start 2024-04-22T23:31:14.447 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:14.449+0000 7f25c5253700 1 -- start start 2024-04-22T23:31:14.447 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:14.449+0000 7f25c5253700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25c00fbd40 0x7f25c0066010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:14.447 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:14.449+0000 7f25c5253700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25c00fcff0 0x7f25c0114b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:14.448 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:14.449+0000 7f25c5253700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c010e540 con 0x7f25c00fcff0 2024-04-22T23:31:14.448 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:14.449+0000 7f25c5253700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c01150a0 con 0x7f25c00fbd40 2024-04-22T23:31:14.448 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:14.449+0000 7f25bed9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25c00fbd40 0x7f25c0066010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:14.448 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:14.449+0000 7f25bed9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25c00fbd40 0x7f25c0066010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:52386/0 (socket says 172.21.15.157:52386) 2024-04-22T23:31:14.448 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:14.449+0000 7f25bed9d700 1 -- 172.21.15.157:0/1863495229 learned_addr learned my addr 172.21.15.157:0/1863495229 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-04-22T23:31:14.448 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:14.449+0000 7f25b7fff700 1 --2- 172.21.15.157:0/1863495229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25c00fcff0 0x7f25c0114b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:14.448 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:14.449+0000 7f25bed9d700 1 -- 172.21.15.157:0/1863495229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25c00fcff0 msgr2=0x7f25c0114b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:14.448 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:14.449+0000 7f25bed9d700 1 --2- 172.21.15.157:0/1863495229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25c00fcff0 0x7f25c0114b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:14.448 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:14.449+0000 7f25bed9d700 1 -- 172.21.15.157:0/1863495229 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b0005040 con 0x7f25c00fbd40 2024-04-22T23:31:14.448 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:14.449+0000 7f25b7fff700 1 --2- 172.21.15.157:0/1863495229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25c00fcff0 0x7f25c0114b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T23:31:14.448 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:14.449+0000 7f25bed9d700 1 --2- 172.21.15.157:0/1863495229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25c00fbd40 0x7f25c0066010 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f25b000b3b0 tx=0x7f25b0009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:31:14.448 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:14.449+0000 7f25b77fe700 1 -- 172.21.15.157:0/1863495229 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25b000e040 con 0x7f25c00fbd40 2024-04-22T23:31:14.448 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:14.449+0000 7f25b77fe700 1 -- 172.21.15.157:0/1863495229 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f25b0004290 con 0x7f25c00fbd40 2024-04-22T23:31:14.448 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:14.449+0000 7f25c5253700 1 -- 172.21.15.157:0/1863495229 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25c01152a0 con 0x7f25c00fbd40 2024-04-22T23:31:14.449 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:14.449+0000 7f25b77fe700 1 -- 172.21.15.157:0/1863495229 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25b001c630 con 0x7f25c00fbd40 2024-04-22T23:31:14.449 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:14.450+0000 7f25c5253700 1 -- 172.21.15.157:0/1863495229 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25c0115860 con 0x7f25c00fbd40 2024-04-22T23:31:14.449 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:14.451+0000 7f25c5253700 1 -- 172.21.15.157:0/1863495229 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25c004c710 con 0x7f25c00fbd40 2024-04-22T23:31:14.450 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:14.451+0000 7f25b77fe700 1 -- 172.21.15.157:0/1863495229 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f25b000bbf0 con 0x7f25c00fbd40 2024-04-22T23:31:14.450 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:14.451+0000 7f25b77fe700 1 --2- 172.21.15.157:0/1863495229 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f25ac0692b0 0x7f25ac06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:14.450 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:14.451+0000 7f25b77fe700 1 -- 172.21.15.157:0/1863495229 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f25b0088260 con 0x7f25c00fbd40 2024-04-22T23:31:14.450 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:14.452+0000 7f25b7fff700 1 --2- 172.21.15.157:0/1863495229 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f25ac0692b0 0x7f25ac06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:14.450 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:14.452+0000 7f25b7fff700 1 --2- 172.21.15.157:0/1863495229 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f25ac0692b0 0x7f25ac06b760 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f25a8009b10 tx=0x7f25a8008040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:31:14.452 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:14.456+0000 7f25b77fe700 1 -- 172.21.15.157:0/1863495229 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f25b0058360 con 0x7f25c00fbd40 2024-04-22T23:31:14.711 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:14.714+0000 7f25c5253700 1 -- 172.21.15.157:0/1863495229 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f25c0115cc0 con 0x7f25c00fbd40 2024-04-22T23:31:14.716 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:14.719+0000 7f25b77fe700 1 -- 172.21.15.157:0/1863495229 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v15) v1 ==== 170+0+59 (secure 0 0 0) 0x7f25b0057ef0 con 0x7f25c00fbd40 2024-04-22T23:31:14.716 INFO:teuthology.orchestra.run.smithi157.stdout:[client.1] 2024-04-22T23:31:14.716 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQBC8yZma/qzKhAABuiiwqMO7lGD1qSn1SUEGg== 2024-04-22T23:31:14.718 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:14.721+0000 7f25c5253700 1 -- 172.21.15.157:0/1863495229 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f25ac0692b0 msgr2=0x7f25ac06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:14.718 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:14.721+0000 7f25c5253700 1 --2- 172.21.15.157:0/1863495229 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f25ac0692b0 0x7f25ac06b760 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f25a8009b10 tx=0x7f25a8008040 comp rx=0 tx=0).stop 2024-04-22T23:31:14.718 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:14.721+0000 7f25c5253700 1 -- 172.21.15.157:0/1863495229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25c00fbd40 msgr2=0x7f25c0066010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:14.718 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:14.721+0000 7f25c5253700 1 --2- 172.21.15.157:0/1863495229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25c00fbd40 0x7f25c0066010 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f25b000b3b0 tx=0x7f25b0009e30 comp rx=0 tx=0).stop 2024-04-22T23:31:14.718 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:14.721+0000 7f25c5253700 1 -- 172.21.15.157:0/1863495229 shutdown_connections 2024-04-22T23:31:14.718 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:14.721+0000 7f25c5253700 1 --2- 172.21.15.157:0/1863495229 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f25ac0692b0 0x7f25ac06b760 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:14.719 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:14.721+0000 7f25c5253700 1 --2- 172.21.15.157:0/1863495229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25c00fbd40 0x7f25c0066010 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:14.719 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:14.721+0000 7f25c5253700 1 --2- 172.21.15.157:0/1863495229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25c00fcff0 0x7f25c0114b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:14.719 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:14.721+0000 7f25c5253700 1 -- 172.21.15.157:0/1863495229 >> 172.21.15.157:0/1863495229 conn(0x7f25c00f7460 msgr2=0x7f25c00f8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:31:14.719 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:14.721+0000 7f25c5253700 1 -- 172.21.15.157:0/1863495229 shutdown_connections 2024-04-22T23:31:14.719 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:31:14.721+0000 7f25c5253700 1 -- 172.21.15.157:0/1863495229 wait complete. 2024-04-22T23:31:15.372 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-22T23:31:15.372 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-04-22T23:31:15.372 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-22T23:31:15.408 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-22T23:31:15.408 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-22T23:31:15.408 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-22T23:31:15.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:15 smithi157 ceph-mon[33877]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:31:15.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:15 smithi157 ceph-mon[33877]: from='client.? 172.21.15.157:0/1863495229' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-22T23:31:15.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:15 smithi157 ceph-mon[33877]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-22T23:31:15.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:15 smithi157 ceph-mon[33877]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-22T23:31:15.669 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:31:15.697 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:15 smithi107 ceph-mon[26355]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:31:15.698 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:15 smithi107 ceph-mon[26355]: from='client.? 172.21.15.157:0/1863495229' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-22T23:31:15.698 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:15 smithi107 ceph-mon[26355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-22T23:31:15.698 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:15 smithi107 ceph-mon[26355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-22T23:31:17.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:17.346+0000 7f033c319700 1 -- 172.21.15.107:0/3709393772 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0334101d90 msgr2=0x7f0334102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:17.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:17.346+0000 7f033c319700 1 --2- 172.21.15.107:0/3709393772 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0334101d90 0x7f0334102160 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0330009ce0 tx=0x7f0330004d10 comp rx=0 tx=0).stop 2024-04-22T23:31:17.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:17.346+0000 7f033c319700 1 -- 172.21.15.107:0/3709393772 shutdown_connections 2024-04-22T23:31:17.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:17.346+0000 7f033c319700 1 --2- 172.21.15.107:0/3709393772 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0334102730 0x7f03340fbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:17.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:17.346+0000 7f033c319700 1 --2- 172.21.15.107:0/3709393772 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0334101d90 0x7f0334102160 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:17.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:17.346+0000 7f033c319700 1 -- 172.21.15.107:0/3709393772 >> 172.21.15.107:0/3709393772 conn(0x7f03340f7900 msgr2=0x7f03340f9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:31:17.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:17.346+0000 7f033c319700 1 -- 172.21.15.107:0/3709393772 shutdown_connections 2024-04-22T23:31:17.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:17.346+0000 7f033c319700 1 -- 172.21.15.107:0/3709393772 wait complete. 2024-04-22T23:31:17.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:17.346+0000 7f033c319700 1 Processor -- start 2024-04-22T23:31:17.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:17.346+0000 7f033c319700 1 -- start start 2024-04-22T23:31:17.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:17.347+0000 7f033c319700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0334101d90 0x7f0334113aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:17.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:17.347+0000 7f033c319700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0334102730 0x7f0334113fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:17.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:17.347+0000 7f033c319700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03341146c0 con 0x7f0334101d90 2024-04-22T23:31:17.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:17.347+0000 7f033c319700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033410dc10 con 0x7f0334102730 2024-04-22T23:31:17.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:17.347+0000 7f033a0b5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0334101d90 0x7f0334113aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:17.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:17.347+0000 7f033a0b5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0334101d90 0x7f0334113aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58046/0 (socket says 172.21.15.107:58046) 2024-04-22T23:31:17.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:17.347+0000 7f033a0b5700 1 -- 172.21.15.107:0/1402253667 learned_addr learned my addr 172.21.15.107:0/1402253667 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:31:17.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:17.347+0000 7f03398b4700 1 --2- 172.21.15.107:0/1402253667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0334102730 0x7f0334113fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:17.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:17.347+0000 7f033a0b5700 1 -- 172.21.15.107:0/1402253667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0334102730 msgr2=0x7f0334113fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:17.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:17.347+0000 7f033a0b5700 1 --2- 172.21.15.107:0/1402253667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0334102730 0x7f0334113fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:17.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:17.347+0000 7f033a0b5700 1 -- 172.21.15.107:0/1402253667 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0330009940 con 0x7f0334101d90 2024-04-22T23:31:17.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:17.347+0000 7f03398b4700 1 --2- 172.21.15.107:0/1402253667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0334102730 0x7f0334113fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T23:31:17.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:17.347+0000 7f033a0b5700 1 --2- 172.21.15.107:0/1402253667 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0334101d90 0x7f0334113aa0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f033000be90 tx=0x7f0330004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:31:17.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:17.347+0000 7f032affd700 1 -- 172.21.15.107:0/1402253667 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f033000e040 con 0x7f0334101d90 2024-04-22T23:31:17.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:17.347+0000 7f033c319700 1 -- 172.21.15.107:0/1402253667 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f033410de30 con 0x7f0334101d90 2024-04-22T23:31:17.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:17.347+0000 7f032affd700 1 -- 172.21.15.107:0/1402253667 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0330008ea0 con 0x7f0334101d90 2024-04-22T23:31:17.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:17.347+0000 7f032affd700 1 -- 172.21.15.107:0/1402253667 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f033001c810 con 0x7f0334101d90 2024-04-22T23:31:17.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:17.347+0000 7f033c319700 1 -- 172.21.15.107:0/1402253667 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f033410e3f0 con 0x7f0334101d90 2024-04-22T23:31:17.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:17.348+0000 7f033c319700 1 -- 172.21.15.107:0/1402253667 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03340fd4e0 con 0x7f0334101d90 2024-04-22T23:31:17.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:17.349+0000 7f032affd700 1 -- 172.21.15.107:0/1402253667 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f0330012c00 con 0x7f0334101d90 2024-04-22T23:31:17.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:17.349+0000 7f032affd700 1 --2- 172.21.15.107:0/1402253667 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f03200692a0 0x7f032006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:17.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:17.350+0000 7f03398b4700 1 --2- 172.21.15.107:0/1402253667 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f03200692a0 0x7f032006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:17.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:17.350+0000 7f032affd700 1 -- 172.21.15.107:0/1402253667 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f0330088a30 con 0x7f0334101d90 2024-04-22T23:31:17.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:17.350+0000 7f03398b4700 1 --2- 172.21.15.107:0/1402253667 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f03200692a0 0x7f032006b750 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f033410f340 tx=0x7f0324008040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:31:17.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:17.354+0000 7f032affd700 1 -- 172.21.15.107:0/1402253667 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0330058b30 con 0x7f0334101d90 2024-04-22T23:31:17.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:17.595+0000 7f033c319700 1 -- 172.21.15.107:0/1402253667 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f033404c710 con 0x7f0334101d90 2024-04-22T23:31:17.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:17.598+0000 7f032affd700 1 -- 172.21.15.107:0/1402253667 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+167187 (secure 0 0 0) 0x7f0330004e10 con 0x7f0334101d90 2024-04-22T23:31:17.598 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:31:17.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:17.601+0000 7f033c319700 1 -- 172.21.15.107:0/1402253667 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f03200692a0 msgr2=0x7f032006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:17.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:17.601+0000 7f033c319700 1 --2- 172.21.15.107:0/1402253667 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f03200692a0 0x7f032006b750 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f033410f340 tx=0x7f0324008040 comp rx=0 tx=0).stop 2024-04-22T23:31:17.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:17.601+0000 7f033c319700 1 -- 172.21.15.107:0/1402253667 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0334101d90 msgr2=0x7f0334113aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:17.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:17.601+0000 7f033c319700 1 --2- 172.21.15.107:0/1402253667 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0334101d90 0x7f0334113aa0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f033000be90 tx=0x7f0330004d10 comp rx=0 tx=0).stop 2024-04-22T23:31:17.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:17.601+0000 7f033c319700 1 -- 172.21.15.107:0/1402253667 shutdown_connections 2024-04-22T23:31:17.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:17.601+0000 7f033c319700 1 --2- 172.21.15.107:0/1402253667 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f03200692a0 0x7f032006b750 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:17.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:17.601+0000 7f033c319700 1 --2- 172.21.15.107:0/1402253667 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0334101d90 0x7f0334113aa0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:17.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:17.601+0000 7f033c319700 1 --2- 172.21.15.107:0/1402253667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0334102730 0x7f0334113fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:17.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:17.601+0000 7f033c319700 1 -- 172.21.15.107:0/1402253667 >> 172.21.15.107:0/1402253667 conn(0x7f03340f7900 msgr2=0x7f0334105660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:31:17.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:17.601+0000 7f033c319700 1 -- 172.21.15.107:0/1402253667 shutdown_connections 2024-04-22T23:31:17.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:17.601+0000 7f033c319700 1 -- 172.21.15.107:0/1402253667 wait complete. 2024-04-22T23:31:17.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:17 smithi157 ceph-mon[33877]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:31:17.736 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:17 smithi107 ceph-mon[26355]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:31:18.301 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":19,"flags":0,"active_gid":14206,"active_name":"smithi107.uipxvm","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":2516490111},{"type":"v1","addr":"172.21.15.107:6801","nonce":2516490111}]},"active_addr":"172.21.15.107:6801/2516490111","active_change":"2024-04-22T23:27:33.932006+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":14228,"name":"smithi157.tuylis","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.107:8443/","prometheus":"http://172.21.15.107:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":525119004}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":2659503121}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":1065200386}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":3567543485}]}]} 2024-04-22T23:31:18.305 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-22T23:31:18.306 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-22T23:31:18.306 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-22T23:31:18.570 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:31:18.585 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:18 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/1402253667' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-22T23:31:18.884 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:18 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/1402253667' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-22T23:31:19.673 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:19 smithi107 ceph-mon[26355]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:31:19.674 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:19 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:31:19.884 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:19 smithi157 ceph-mon[33877]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:31:19.884 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:19 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:31:20.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:20.228+0000 7f25031f9700 1 -- 172.21.15.107:0/667232405 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24fc102730 msgr2=0x7f24fc0fbda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:20.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:20.228+0000 7f25031f9700 1 --2- 172.21.15.107:0/667232405 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24fc102730 0x7f24fc0fbda0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f24ec004940 tx=0x7f24ec009e30 comp rx=0 tx=0).stop 2024-04-22T23:31:20.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:20.228+0000 7f25031f9700 1 -- 172.21.15.107:0/667232405 shutdown_connections 2024-04-22T23:31:20.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:20.228+0000 7f25031f9700 1 --2- 172.21.15.107:0/667232405 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24fc102730 0x7f24fc0fbda0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:20.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:20.228+0000 7f25031f9700 1 --2- 172.21.15.107:0/667232405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24fc101d90 0x7f24fc102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:20.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:20.228+0000 7f25031f9700 1 -- 172.21.15.107:0/667232405 >> 172.21.15.107:0/667232405 conn(0x7f24fc0f7900 msgr2=0x7f24fc0f9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:31:20.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:20.228+0000 7f25031f9700 1 -- 172.21.15.107:0/667232405 shutdown_connections 2024-04-22T23:31:20.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:20.228+0000 7f25031f9700 1 -- 172.21.15.107:0/667232405 wait complete. 2024-04-22T23:31:20.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:20.229+0000 7f25031f9700 1 Processor -- start 2024-04-22T23:31:20.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:20.229+0000 7f25031f9700 1 -- start start 2024-04-22T23:31:20.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:20.229+0000 7f25031f9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24fc101d90 0x7f24fc10eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:20.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:20.229+0000 7f25031f9700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24fc102730 0x7f24fc10f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:20.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:20.229+0000 7f25031f9700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24fc067240 con 0x7f24fc101d90 2024-04-22T23:31:20.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:20.229+0000 7f25031f9700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24fc10f5a0 con 0x7f24fc102730 2024-04-22T23:31:20.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:20.230+0000 7f2500f95700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24fc101d90 0x7f24fc10eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:20.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:20.230+0000 7f2500f95700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24fc101d90 0x7f24fc10eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58074/0 (socket says 172.21.15.107:58074) 2024-04-22T23:31:20.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:20.230+0000 7f2500f95700 1 -- 172.21.15.107:0/2249131256 learned_addr learned my addr 172.21.15.107:0/2249131256 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:31:20.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:20.230+0000 7f24fbfff700 1 --2- 172.21.15.107:0/2249131256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24fc102730 0x7f24fc10f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:20.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:20.230+0000 7f2500f95700 1 -- 172.21.15.107:0/2249131256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24fc102730 msgr2=0x7f24fc10f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:20.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:20.230+0000 7f2500f95700 1 --2- 172.21.15.107:0/2249131256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24fc102730 0x7f24fc10f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:20.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:20.230+0000 7f2500f95700 1 -- 172.21.15.107:0/2249131256 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24ec005040 con 0x7f24fc101d90 2024-04-22T23:31:20.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:20.230+0000 7f2500f95700 1 --2- 172.21.15.107:0/2249131256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24fc101d90 0x7f24fc10eb20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f24f4001d80 tx=0x7f24f4009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:31:20.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:20.230+0000 7f24f97fa700 1 -- 172.21.15.107:0/2249131256 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f24f4007d10 con 0x7f24fc101d90 2024-04-22T23:31:20.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:20.230+0000 7f24f97fa700 1 -- 172.21.15.107:0/2249131256 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f24f4001e80 con 0x7f24fc101d90 2024-04-22T23:31:20.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:20.230+0000 7f25031f9700 1 -- 172.21.15.107:0/2249131256 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24fc10f880 con 0x7f24fc101d90 2024-04-22T23:31:20.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:20.230+0000 7f24f97fa700 1 -- 172.21.15.107:0/2249131256 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f24f4014640 con 0x7f24fc101d90 2024-04-22T23:31:20.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:20.230+0000 7f25031f9700 1 -- 172.21.15.107:0/2249131256 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24fc117e80 con 0x7f24fc101d90 2024-04-22T23:31:20.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:20.231+0000 7f25031f9700 1 -- 172.21.15.107:0/2249131256 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24fc104260 con 0x7f24fc101d90 2024-04-22T23:31:20.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:20.232+0000 7f24f97fa700 1 -- 172.21.15.107:0/2249131256 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f24f400e960 con 0x7f24fc101d90 2024-04-22T23:31:20.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:20.232+0000 7f24f97fa700 1 --2- 172.21.15.107:0/2249131256 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f24e80692a0 0x7f24e806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:20.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:20.232+0000 7f24f97fa700 1 -- 172.21.15.107:0/2249131256 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f24f4086ae0 con 0x7f24fc101d90 2024-04-22T23:31:20.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:20.232+0000 7f24fbfff700 1 --2- 172.21.15.107:0/2249131256 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f24e80692a0 0x7f24e806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:20.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:20.232+0000 7f24fbfff700 1 --2- 172.21.15.107:0/2249131256 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f24e80692a0 0x7f24e806b750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f24ec004890 tx=0x7f24ec009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:31:20.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:20.236+0000 7f24f97fa700 1 -- 172.21.15.107:0/2249131256 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f24f404e250 con 0x7f24fc101d90 2024-04-22T23:31:20.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:20.426+0000 7f25031f9700 1 -- 172.21.15.107:0/2249131256 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f24fc0667d0 con 0x7f24fc101d90 2024-04-22T23:31:20.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:20.427+0000 7f24f97fa700 1 -- 172.21.15.107:0/2249131256 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11107 (secure 0 0 0) 0x7f24f4053200 con 0x7f24fc101d90 2024-04-22T23:31:20.424 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:31:20.424 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":40,"fsid":"8252f6f2-00ff-11ef-bc93-c7b262605968","created":"2024-04-22T23:25:25.497133+0000","modified":"2024-04-22T23:31:04.176098+0000","last_up_change":"2024-04-22T23:31:03.170868+0000","last_in_change":"2024-04-22T23:30:44.396843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T23:29:48.249903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"65404290-734a-4901-bbb4-a87d3d846b06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4253936450},{"type":"v1","addr":"172.21.15.107:6803","nonce":4253936450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4253936450},{"type":"v1","addr":"172.21.15.107:6805","nonce":4253936450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":4253936450},{"type":"v1","addr":"172.21.15.107:6809","nonce":4253936450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4253936450},{"type":"v1","addr":"172.21.15.107:6807","nonce":4253936450}]},"public_addr":"172.21.15.107:6803/4253936450","cluster_addr":"172.21.15.107:6805/4253936450","heartbeat_back_addr":"172.21.15.107:6809/4253936450","heartbeat_front_addr":"172.21.15.107:6807/4253936450","state":["exists","up"]},{"osd":1,"uuid":"6df381f7-b907-4d1b-bd08-fdf34e9729f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":335851250},{"type":"v1","addr":"172.21.15.107:6811","nonce":335851250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":335851250},{"type":"v1","addr":"172.21.15.107:6813","nonce":335851250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":335851250},{"type":"v1","addr":"172.21.15.107:6817","nonce":335851250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":335851250},{"type":"v1","addr":"172.21.15.107:6815","nonce":335851250}]},"public_addr":"172.21.15.107:6811/335851250","cluster_addr":"172.21.15.107:6813/335851250","heartbeat_back_addr":"172.21.15.107:6817/335851250","heartbeat_front_addr":"172.21.15.107:6815/335851250","state":["exists","up"]},{"osd":2,"uuid":"2c5e5c37-c1de-4e64-9c21-eacc145f1412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":750102033},{"type":"v1","addr":"172.21.15.107:6819","nonce":750102033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":750102033},{"type":"v1","addr":"172.21.15.107:6821","nonce":750102033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":750102033},{"type":"v1","addr":"172.21.15.107:6825","nonce":750102033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":750102033},{"type":"v1","addr":"172.21.15.107:6823","nonce":750102033}]},"public_addr":"172.21.15.107:6819/750102033","cluster_addr":"172.21.15.107:6821/750102033","heartbeat_back_addr":"172.21.15.107:6825/750102033","heartbeat_front_addr":"172.21.15.107:6823/750102033","state":["exists","up"]},{"osd":3,"uuid":"b7c9c124-9f43-4b5b-b039-c61ae30a416d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":811815812},{"type":"v1","addr":"172.21.15.157:6801","nonce":811815812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":811815812},{"type":"v1","addr":"172.21.15.157:6803","nonce":811815812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":811815812},{"type":"v1","addr":"172.21.15.157:6807","nonce":811815812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":811815812},{"type":"v1","addr":"172.21.15.157:6805","nonce":811815812}]},"public_addr":"172.21.15.157:6801/811815812","cluster_addr":"172.21.15.157:6803/811815812","heartbeat_back_addr":"172.21.15.157:6807/811815812","heartbeat_front_addr":"172.21.15.157:6805/811815812","state":["exists","up"]},{"osd":4,"uuid":"34a0cb61-1c2b-4515-9d74-8a3ad3f15fbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2372899121},{"type":"v1","addr":"172.21.15.157:6809","nonce":2372899121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2372899121},{"type":"v1","addr":"172.21.15.157:6811","nonce":2372899121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2372899121},{"type":"v1","addr":"172.21.15.157:6815","nonce":2372899121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2372899121},{"type":"v1","addr":"172.21.15.157:6813","nonce":2372899121}]},"public_addr":"172.21.15.157:6809/2372899121","cluster_addr":"172.21.15.157:6811/2372899121","heartbeat_back_addr":"172.21.15.157:6815/2372899121","heartbeat_front_addr":"172.21.15.157:6813/2372899121","state":["exists","up"]},{"osd":5,"uuid":"00934a7f-7e88-45cf-a2cc-e8a94bceaab0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2125627920},{"type":"v1","addr":"172.21.15.157:6817","nonce":2125627920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2125627920},{"type":"v1","addr":"172.21.15.157:6819","nonce":2125627920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2125627920},{"type":"v1","addr":"172.21.15.157:6823","nonce":2125627920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2125627920},{"type":"v1","addr":"172.21.15.157:6821","nonce":2125627920}]},"public_addr":"172.21.15.157:6817/2125627920","cluster_addr":"172.21.15.157:6819/2125627920","heartbeat_back_addr":"172.21.15.157:6823/2125627920","heartbeat_front_addr":"172.21.15.157:6821/2125627920","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:28:48.161289+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:29:18.447013+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:29:45.948165+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:30:09.825103+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:30:35.976009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:31:01.729305+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:6801/3427816626":"2024-04-23T23:27:33.931895+0000","172.21.15.107:6800/3427816626":"2024-04-23T23:27:33.931895+0000","172.21.15.107:6801/307474287":"2024-04-23T23:26:29.807373+0000","172.21.15.107:6800/307474287":"2024-04-23T23:26:29.807373+0000","172.21.15.107:6800/467262774":"2024-04-23T23:25:58.816560+0000","172.21.15.107:0/1029440409":"2024-04-23T23:26:29.807373+0000","172.21.15.107:0/3810721822":"2024-04-23T23:25:58.816560+0000","172.21.15.107:0/1205692115":"2024-04-23T23:27:33.931895+0000","172.21.15.107:0/3608791958":"2024-04-23T23:27:33.931895+0000","172.21.15.107:6801/467262774":"2024-04-23T23:25:58.816560+0000","172.21.15.107:0/3377956937":"2024-04-23T23:25:58.816560+0000","172.21.15.107:0/1734120985":"2024-04-23T23:27:33.931895+0000","172.21.15.107:0/3920905056":"2024-04-23T23:25:58.816560+0000","172.21.15.107:0/2444490678":"2024-04-23T23:26:29.807373+0000","172.21.15.107:0/663073386":"2024-04-23T23:26:29.807373+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T23:31:20.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:20.429+0000 7f25031f9700 1 -- 172.21.15.107:0/2249131256 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f24e80692a0 msgr2=0x7f24e806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:20.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:20.429+0000 7f25031f9700 1 --2- 172.21.15.107:0/2249131256 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f24e80692a0 0x7f24e806b750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f24ec004890 tx=0x7f24ec009e30 comp rx=0 tx=0).stop 2024-04-22T23:31:20.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:20.429+0000 7f25031f9700 1 -- 172.21.15.107:0/2249131256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24fc101d90 msgr2=0x7f24fc10eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:20.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:20.429+0000 7f25031f9700 1 --2- 172.21.15.107:0/2249131256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24fc101d90 0x7f24fc10eb20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f24f4001d80 tx=0x7f24f4009ec0 comp rx=0 tx=0).stop 2024-04-22T23:31:20.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:20.429+0000 7f25031f9700 1 -- 172.21.15.107:0/2249131256 shutdown_connections 2024-04-22T23:31:20.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:20.429+0000 7f25031f9700 1 --2- 172.21.15.107:0/2249131256 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f24e80692a0 0x7f24e806b750 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:20.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:20.429+0000 7f25031f9700 1 --2- 172.21.15.107:0/2249131256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24fc101d90 0x7f24fc10eb20 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:20.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:20.429+0000 7f25031f9700 1 --2- 172.21.15.107:0/2249131256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24fc102730 0x7f24fc10f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:20.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:20.429+0000 7f25031f9700 1 -- 172.21.15.107:0/2249131256 >> 172.21.15.107:0/2249131256 conn(0x7f24fc0f7900 msgr2=0x7f24fc101160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:31:20.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:20.429+0000 7f25031f9700 1 -- 172.21.15.107:0/2249131256 shutdown_connections 2024-04-22T23:31:20.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:20.429+0000 7f25031f9700 1 -- 172.21.15.107:0/2249131256 wait complete. 2024-04-22T23:31:21.042 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-22T23:31:21.042 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-22T23:31:21.299 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:31:21.676 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:21 smithi107 ceph-mon[26355]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:31:21.677 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:21 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/2249131256' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T23:31:21.884 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:21 smithi157 ceph-mon[33877]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:31:21.884 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:21 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/2249131256' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T23:31:23.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:23.047+0000 7f98a610f700 1 -- 172.21.15.107:0/1764148925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98a01026d0 msgr2=0x7f98a00fbd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:23.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:23.047+0000 7f98a610f700 1 --2- 172.21.15.107:0/1764148925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98a01026d0 0x7f98a00fbd40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9894004940 tx=0x7f9894009e30 comp rx=0 tx=0).stop 2024-04-22T23:31:23.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:23.047+0000 7f98a610f700 1 -- 172.21.15.107:0/1764148925 shutdown_connections 2024-04-22T23:31:23.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:23.047+0000 7f98a610f700 1 --2- 172.21.15.107:0/1764148925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98a01026d0 0x7f98a00fbd40 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:23.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:23.047+0000 7f98a610f700 1 --2- 172.21.15.107:0/1764148925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98a0101d30 0x7f98a0102100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:23.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:23.047+0000 7f98a610f700 1 -- 172.21.15.107:0/1764148925 >> 172.21.15.107:0/1764148925 conn(0x7f98a00f7880 msgr2=0x7f98a00f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:31:23.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:23.047+0000 7f98a610f700 1 -- 172.21.15.107:0/1764148925 shutdown_connections 2024-04-22T23:31:23.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:23.047+0000 7f98a610f700 1 -- 172.21.15.107:0/1764148925 wait complete. 2024-04-22T23:31:23.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:23.047+0000 7f98a610f700 1 Processor -- start 2024-04-22T23:31:23.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:23.047+0000 7f98a610f700 1 -- start start 2024-04-22T23:31:23.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:23.048+0000 7f98a610f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98a0101d30 0x7f98a010ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:23.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:23.048+0000 7f98a610f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98a01026d0 0x7f98a010f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:23.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:23.048+0000 7f98a610f700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a010f550 con 0x7f98a0101d30 2024-04-22T23:31:23.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:23.048+0000 7f98a610f700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a010f6c0 con 0x7f98a01026d0 2024-04-22T23:31:23.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:23.048+0000 7f989f7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98a0101d30 0x7f98a010ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:23.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:23.048+0000 7f989f7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98a0101d30 0x7f98a010ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42462/0 (socket says 172.21.15.107:42462) 2024-04-22T23:31:23.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:23.048+0000 7f989f7fe700 1 -- 172.21.15.107:0/1305306162 learned_addr learned my addr 172.21.15.107:0/1305306162 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:31:23.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:23.048+0000 7f989effd700 1 --2- 172.21.15.107:0/1305306162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98a01026d0 0x7f98a010f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:23.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:23.048+0000 7f989f7fe700 1 -- 172.21.15.107:0/1305306162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98a01026d0 msgr2=0x7f98a010f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:23.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:23.048+0000 7f989f7fe700 1 --2- 172.21.15.107:0/1305306162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98a01026d0 0x7f98a010f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:23.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:23.048+0000 7f989f7fe700 1 -- 172.21.15.107:0/1305306162 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9894005040 con 0x7f98a0101d30 2024-04-22T23:31:23.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:23.048+0000 7f989f7fe700 1 --2- 172.21.15.107:0/1305306162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98a0101d30 0x7f98a010ead0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f989000eb40 tx=0x7f989000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:31:23.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:23.048+0000 7f9887fff700 1 -- 172.21.15.107:0/1305306162 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9890007cc0 con 0x7f98a0101d30 2024-04-22T23:31:23.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:23.048+0000 7f98a610f700 1 -- 172.21.15.107:0/1305306162 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98a010f940 con 0x7f98a0101d30 2024-04-22T23:31:23.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:23.048+0000 7f9887fff700 1 -- 172.21.15.107:0/1305306162 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f989000f040 con 0x7f98a0101d30 2024-04-22T23:31:23.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:23.048+0000 7f9887fff700 1 -- 172.21.15.107:0/1305306162 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f989000ca10 con 0x7f98a0101d30 2024-04-22T23:31:23.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:23.048+0000 7f98a610f700 1 -- 172.21.15.107:0/1305306162 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98a0117e10 con 0x7f98a0101d30 2024-04-22T23:31:23.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:23.050+0000 7f98a610f700 1 -- 172.21.15.107:0/1305306162 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98a00fd480 con 0x7f98a0101d30 2024-04-22T23:31:23.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:23.050+0000 7f9887fff700 1 -- 172.21.15.107:0/1305306162 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f989000cb70 con 0x7f98a0101d30 2024-04-22T23:31:23.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:23.050+0000 7f9887fff700 1 --2- 172.21.15.107:0/1305306162 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f98880692a0 0x7f988806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:23.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:23.050+0000 7f9887fff700 1 -- 172.21.15.107:0/1305306162 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f9890086830 con 0x7f98a0101d30 2024-04-22T23:31:23.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:23.050+0000 7f989effd700 1 --2- 172.21.15.107:0/1305306162 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f98880692a0 0x7f988806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:23.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:23.051+0000 7f989effd700 1 --2- 172.21.15.107:0/1305306162 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f98880692a0 0x7f988806b750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9894004890 tx=0x7f9894009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:31:23.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:23.054+0000 7f9887fff700 1 -- 172.21.15.107:0/1305306162 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f989004e250 con 0x7f98a0101d30 2024-04-22T23:31:23.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:23.239+0000 7f98a610f700 1 -- 172.21.15.107:0/1305306162 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f98a004c710 con 0x7f98a0101d30 2024-04-22T23:31:23.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:23.240+0000 7f9887fff700 1 -- 172.21.15.107:0/1305306162 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11107 (secure 0 0 0) 0x7f9890019090 con 0x7f98a0101d30 2024-04-22T23:31:23.236 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:31:23.237 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":40,"fsid":"8252f6f2-00ff-11ef-bc93-c7b262605968","created":"2024-04-22T23:25:25.497133+0000","modified":"2024-04-22T23:31:04.176098+0000","last_up_change":"2024-04-22T23:31:03.170868+0000","last_in_change":"2024-04-22T23:30:44.396843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T23:29:48.249903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"65404290-734a-4901-bbb4-a87d3d846b06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4253936450},{"type":"v1","addr":"172.21.15.107:6803","nonce":4253936450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4253936450},{"type":"v1","addr":"172.21.15.107:6805","nonce":4253936450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":4253936450},{"type":"v1","addr":"172.21.15.107:6809","nonce":4253936450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4253936450},{"type":"v1","addr":"172.21.15.107:6807","nonce":4253936450}]},"public_addr":"172.21.15.107:6803/4253936450","cluster_addr":"172.21.15.107:6805/4253936450","heartbeat_back_addr":"172.21.15.107:6809/4253936450","heartbeat_front_addr":"172.21.15.107:6807/4253936450","state":["exists","up"]},{"osd":1,"uuid":"6df381f7-b907-4d1b-bd08-fdf34e9729f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":335851250},{"type":"v1","addr":"172.21.15.107:6811","nonce":335851250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":335851250},{"type":"v1","addr":"172.21.15.107:6813","nonce":335851250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":335851250},{"type":"v1","addr":"172.21.15.107:6817","nonce":335851250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":335851250},{"type":"v1","addr":"172.21.15.107:6815","nonce":335851250}]},"public_addr":"172.21.15.107:6811/335851250","cluster_addr":"172.21.15.107:6813/335851250","heartbeat_back_addr":"172.21.15.107:6817/335851250","heartbeat_front_addr":"172.21.15.107:6815/335851250","state":["exists","up"]},{"osd":2,"uuid":"2c5e5c37-c1de-4e64-9c21-eacc145f1412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":750102033},{"type":"v1","addr":"172.21.15.107:6819","nonce":750102033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":750102033},{"type":"v1","addr":"172.21.15.107:6821","nonce":750102033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":750102033},{"type":"v1","addr":"172.21.15.107:6825","nonce":750102033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":750102033},{"type":"v1","addr":"172.21.15.107:6823","nonce":750102033}]},"public_addr":"172.21.15.107:6819/750102033","cluster_addr":"172.21.15.107:6821/750102033","heartbeat_back_addr":"172.21.15.107:6825/750102033","heartbeat_front_addr":"172.21.15.107:6823/750102033","state":["exists","up"]},{"osd":3,"uuid":"b7c9c124-9f43-4b5b-b039-c61ae30a416d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":811815812},{"type":"v1","addr":"172.21.15.157:6801","nonce":811815812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":811815812},{"type":"v1","addr":"172.21.15.157:6803","nonce":811815812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":811815812},{"type":"v1","addr":"172.21.15.157:6807","nonce":811815812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":811815812},{"type":"v1","addr":"172.21.15.157:6805","nonce":811815812}]},"public_addr":"172.21.15.157:6801/811815812","cluster_addr":"172.21.15.157:6803/811815812","heartbeat_back_addr":"172.21.15.157:6807/811815812","heartbeat_front_addr":"172.21.15.157:6805/811815812","state":["exists","up"]},{"osd":4,"uuid":"34a0cb61-1c2b-4515-9d74-8a3ad3f15fbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2372899121},{"type":"v1","addr":"172.21.15.157:6809","nonce":2372899121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2372899121},{"type":"v1","addr":"172.21.15.157:6811","nonce":2372899121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2372899121},{"type":"v1","addr":"172.21.15.157:6815","nonce":2372899121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2372899121},{"type":"v1","addr":"172.21.15.157:6813","nonce":2372899121}]},"public_addr":"172.21.15.157:6809/2372899121","cluster_addr":"172.21.15.157:6811/2372899121","heartbeat_back_addr":"172.21.15.157:6815/2372899121","heartbeat_front_addr":"172.21.15.157:6813/2372899121","state":["exists","up"]},{"osd":5,"uuid":"00934a7f-7e88-45cf-a2cc-e8a94bceaab0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2125627920},{"type":"v1","addr":"172.21.15.157:6817","nonce":2125627920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2125627920},{"type":"v1","addr":"172.21.15.157:6819","nonce":2125627920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2125627920},{"type":"v1","addr":"172.21.15.157:6823","nonce":2125627920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2125627920},{"type":"v1","addr":"172.21.15.157:6821","nonce":2125627920}]},"public_addr":"172.21.15.157:6817/2125627920","cluster_addr":"172.21.15.157:6819/2125627920","heartbeat_back_addr":"172.21.15.157:6823/2125627920","heartbeat_front_addr":"172.21.15.157:6821/2125627920","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:28:48.161289+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:29:18.447013+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:29:45.948165+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:30:09.825103+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:30:35.976009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:31:01.729305+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:6801/3427816626":"2024-04-23T23:27:33.931895+0000","172.21.15.107:6800/3427816626":"2024-04-23T23:27:33.931895+0000","172.21.15.107:6801/307474287":"2024-04-23T23:26:29.807373+0000","172.21.15.107:6800/307474287":"2024-04-23T23:26:29.807373+0000","172.21.15.107:6800/467262774":"2024-04-23T23:25:58.816560+0000","172.21.15.107:0/1029440409":"2024-04-23T23:26:29.807373+0000","172.21.15.107:0/3810721822":"2024-04-23T23:25:58.816560+0000","172.21.15.107:0/1205692115":"2024-04-23T23:27:33.931895+0000","172.21.15.107:0/3608791958":"2024-04-23T23:27:33.931895+0000","172.21.15.107:6801/467262774":"2024-04-23T23:25:58.816560+0000","172.21.15.107:0/3377956937":"2024-04-23T23:25:58.816560+0000","172.21.15.107:0/1734120985":"2024-04-23T23:27:33.931895+0000","172.21.15.107:0/3920905056":"2024-04-23T23:25:58.816560+0000","172.21.15.107:0/2444490678":"2024-04-23T23:26:29.807373+0000","172.21.15.107:0/663073386":"2024-04-23T23:26:29.807373+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T23:31:23.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:23.242+0000 7f98a610f700 1 -- 172.21.15.107:0/1305306162 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f98880692a0 msgr2=0x7f988806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:23.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:23.242+0000 7f98a610f700 1 --2- 172.21.15.107:0/1305306162 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f98880692a0 0x7f988806b750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9894004890 tx=0x7f9894009e30 comp rx=0 tx=0).stop 2024-04-22T23:31:23.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:23.242+0000 7f98a610f700 1 -- 172.21.15.107:0/1305306162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98a0101d30 msgr2=0x7f98a010ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:23.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:23.242+0000 7f98a610f700 1 --2- 172.21.15.107:0/1305306162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98a0101d30 0x7f98a010ead0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f989000eb40 tx=0x7f989000b040 comp rx=0 tx=0).stop 2024-04-22T23:31:23.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:23.242+0000 7f98a610f700 1 -- 172.21.15.107:0/1305306162 shutdown_connections 2024-04-22T23:31:23.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:23.242+0000 7f98a610f700 1 --2- 172.21.15.107:0/1305306162 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f98880692a0 0x7f988806b750 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:23.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:23.242+0000 7f98a610f700 1 --2- 172.21.15.107:0/1305306162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98a0101d30 0x7f98a010ead0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:23.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:23.242+0000 7f98a610f700 1 --2- 172.21.15.107:0/1305306162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98a01026d0 0x7f98a010f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:23.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:23.242+0000 7f98a610f700 1 -- 172.21.15.107:0/1305306162 >> 172.21.15.107:0/1305306162 conn(0x7f98a00f7880 msgr2=0x7f98a00f8790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:31:23.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:23.242+0000 7f98a610f700 1 -- 172.21.15.107:0/1305306162 shutdown_connections 2024-04-22T23:31:23.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:23.242+0000 7f98a610f700 1 -- 172.21.15.107:0/1305306162 wait complete. 2024-04-22T23:31:23.736 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:23 smithi107 ceph-mon[26355]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:31:23.736 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:23 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/1305306162' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T23:31:23.884 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:23 smithi157 ceph-mon[33877]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:31:23.884 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:23 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/1305306162' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T23:31:23.892 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-22T23:31:23.893 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-22T23:31:23.893 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-22T23:31:23.894 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-22T23:31:23.894 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-22T23:31:23.894 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-22T23:31:24.192 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:31:24.210 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:31:25.728 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:31:25.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:25 smithi107 ceph-mon[26355]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:31:25.884 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:25 smithi157 ceph-mon[33877]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:31:26.699 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:31:26.699 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:31:26.881 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:31:27.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:27.319+0000 7f40073f0700 1 -- 172.21.15.107:0/1543355762 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40000fe830 msgr2=0x7f4000106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:27.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:27.319+0000 7f40073f0700 1 --2- 172.21.15.107:0/1543355762 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40000fe830 0x7f4000106e70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0004940 tx=0x7f3ff0009e30 comp rx=0 tx=0).stop 2024-04-22T23:31:27.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:27.320+0000 7f40073f0700 1 -- 172.21.15.107:0/1543355762 shutdown_connections 2024-04-22T23:31:27.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:27.320+0000 7f40073f0700 1 --2- 172.21.15.107:0/1543355762 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40000fe830 0x7f4000106e70 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:27.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:27.320+0000 7f40073f0700 1 --2- 172.21.15.107:0/1543355762 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40000fde90 0x7f40000fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:27.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:27.320+0000 7f40073f0700 1 -- 172.21.15.107:0/1543355762 >> 172.21.15.107:0/1543355762 conn(0x7f40000f7900 msgr2=0x7f40000f9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:31:27.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:27.320+0000 7f40073f0700 1 -- 172.21.15.107:0/1543355762 shutdown_connections 2024-04-22T23:31:27.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:27.320+0000 7f40073f0700 1 -- 172.21.15.107:0/1543355762 wait complete. 2024-04-22T23:31:27.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:27.320+0000 7f40073f0700 1 Processor -- start 2024-04-22T23:31:27.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:27.320+0000 7f40073f0700 1 -- start start 2024-04-22T23:31:27.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:27.320+0000 7f40073f0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40000fde90 0x7f40000fbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:27.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:27.320+0000 7f40073f0700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40000fe830 0x7f40000fc200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:27.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:27.320+0000 7f40073f0700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40000fdc10 con 0x7f40000fde90 2024-04-22T23:31:27.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:27.320+0000 7f40073f0700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40000fc740 con 0x7f40000fe830 2024-04-22T23:31:27.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:27.320+0000 7f400518c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40000fde90 0x7f40000fbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:27.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:27.320+0000 7f400518c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40000fde90 0x7f40000fbcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42480/0 (socket says 172.21.15.107:42480) 2024-04-22T23:31:27.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:27.320+0000 7f400518c700 1 -- 172.21.15.107:0/378802109 learned_addr learned my addr 172.21.15.107:0/378802109 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:31:27.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:27.321+0000 7f400498b700 1 --2- 172.21.15.107:0/378802109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40000fe830 0x7f40000fc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:27.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:27.321+0000 7f400518c700 1 -- 172.21.15.107:0/378802109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40000fe830 msgr2=0x7f40000fc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:27.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:27.321+0000 7f400518c700 1 --2- 172.21.15.107:0/378802109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40000fe830 0x7f40000fc200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:27.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:27.321+0000 7f400518c700 1 -- 172.21.15.107:0/378802109 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff0005040 con 0x7f40000fde90 2024-04-22T23:31:27.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:27.321+0000 7f400518c700 1 --2- 172.21.15.107:0/378802109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40000fde90 0x7f40000fbcc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc002a50 tx=0x7f3ffc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:31:27.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:27.321+0000 7f3ff5ffb700 1 -- 172.21.15.107:0/378802109 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ffc007cc0 con 0x7f40000fde90 2024-04-22T23:31:27.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:27.321+0000 7f3ff5ffb700 1 -- 172.21.15.107:0/378802109 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3ffc00f040 con 0x7f40000fde90 2024-04-22T23:31:27.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:27.321+0000 7f3ff5ffb700 1 -- 172.21.15.107:0/378802109 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ffc00da30 con 0x7f40000fde90 2024-04-22T23:31:27.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:27.321+0000 7f40073f0700 1 -- 172.21.15.107:0/378802109 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40000fca20 con 0x7f40000fde90 2024-04-22T23:31:27.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:27.321+0000 7f40073f0700 1 -- 172.21.15.107:0/378802109 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f400011a070 con 0x7f40000fde90 2024-04-22T23:31:27.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:27.322+0000 7f40073f0700 1 -- 172.21.15.107:0/378802109 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4000066b60 con 0x7f40000fde90 2024-04-22T23:31:27.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:27.322+0000 7f3ff5ffb700 1 -- 172.21.15.107:0/378802109 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f3ffc00db90 con 0x7f40000fde90 2024-04-22T23:31:27.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:27.323+0000 7f3ff5ffb700 1 --2- 172.21.15.107:0/378802109 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f3fec0692a0 0x7f3fec06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:27.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:27.323+0000 7f3ff5ffb700 1 -- 172.21.15.107:0/378802109 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f3ffc086b30 con 0x7f40000fde90 2024-04-22T23:31:27.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:27.323+0000 7f400498b700 1 --2- 172.21.15.107:0/378802109 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f3fec0692a0 0x7f3fec06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:27.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:27.323+0000 7f3ff5ffb700 1 --2- 172.21.15.107:0/378802109 >> [v2:172.21.15.157:6816/2125627920,v1:172.21.15.157:6817/2125627920] conn(0x7f3fec06ef70 0x7f3fec071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:27.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:27.323+0000 7f3ff5ffb700 1 -- 172.21.15.107:0/378802109 --> [v2:172.21.15.157:6816/2125627920,v1:172.21.15.157:6817/2125627920] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3fec071a30 con 0x7f3fec06ef70 2024-04-22T23:31:27.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:27.323+0000 7f3ff5ffb700 1 -- 172.21.15.107:0/378802109 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ffc08b220 con 0x7f40000fde90 2024-04-22T23:31:27.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:27.323+0000 7f400498b700 1 --2- 172.21.15.107:0/378802109 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f3fec0692a0 0x7f3fec06b750 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f40000fd3f0 tx=0x7f3ff0009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:31:27.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:27.323+0000 7f400598d700 1 --2- 172.21.15.107:0/378802109 >> [v2:172.21.15.157:6816/2125627920,v1:172.21.15.157:6817/2125627920] conn(0x7f3fec06ef70 0x7f3fec071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:27.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:27.324+0000 7f400598d700 1 --2- 172.21.15.107:0/378802109 >> [v2:172.21.15.157:6816/2125627920,v1:172.21.15.157:6817/2125627920] conn(0x7f3fec06ef70 0x7f3fec071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:31:27.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:27.325+0000 7f3ff5ffb700 1 -- 172.21.15.107:0/378802109 <== osd.5 v2:172.21.15.157:6816/2125627920 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f3fec071a30 con 0x7f3fec06ef70 2024-04-22T23:31:27.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:27.340+0000 7f40073f0700 1 -- 172.21.15.107:0/378802109 --> [v2:172.21.15.157:6816/2125627920,v1:172.21.15.157:6817/2125627920] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f400005e640 con 0x7f3fec06ef70 2024-04-22T23:31:27.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:27.340+0000 7f3ff5ffb700 1 -- 172.21.15.107:0/378802109 <== osd.5 v2:172.21.15.157:6816/2125627920 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f400005e640 con 0x7f3fec06ef70 2024-04-22T23:31:27.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:27.340+0000 7f40073f0700 1 -- 172.21.15.107:0/378802109 >> [v2:172.21.15.157:6816/2125627920,v1:172.21.15.157:6817/2125627920] conn(0x7f3fec06ef70 msgr2=0x7f3fec071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:27.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:27.340+0000 7f40073f0700 1 --2- 172.21.15.107:0/378802109 >> [v2:172.21.15.157:6816/2125627920,v1:172.21.15.157:6817/2125627920] conn(0x7f3fec06ef70 0x7f3fec071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:27.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:27.341+0000 7f40073f0700 1 -- 172.21.15.107:0/378802109 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f3fec0692a0 msgr2=0x7f3fec06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:27.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:27.341+0000 7f40073f0700 1 --2- 172.21.15.107:0/378802109 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f3fec0692a0 0x7f3fec06b750 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f40000fd3f0 tx=0x7f3ff0009e30 comp rx=0 tx=0).stop 2024-04-22T23:31:27.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:27.341+0000 7f40073f0700 1 -- 172.21.15.107:0/378802109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40000fde90 msgr2=0x7f40000fbcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:27.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:27.341+0000 7f40073f0700 1 --2- 172.21.15.107:0/378802109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40000fde90 0x7f40000fbcc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc002a50 tx=0x7f3ffc0046b0 comp rx=0 tx=0).stop 2024-04-22T23:31:27.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:27.341+0000 7f40073f0700 1 -- 172.21.15.107:0/378802109 shutdown_connections 2024-04-22T23:31:27.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:27.341+0000 7f40073f0700 1 --2- 172.21.15.107:0/378802109 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f3fec0692a0 0x7f3fec06b750 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:27.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:27.341+0000 7f40073f0700 1 --2- 172.21.15.107:0/378802109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40000fde90 0x7f40000fbcc0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:27.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:27.341+0000 7f40073f0700 1 --2- 172.21.15.107:0/378802109 >> [v2:172.21.15.157:6816/2125627920,v1:172.21.15.157:6817/2125627920] conn(0x7f3fec06ef70 0x7f3fec071380 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:27.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:27.341+0000 7f40073f0700 1 --2- 172.21.15.107:0/378802109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40000fe830 0x7f40000fc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:27.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:27.341+0000 7f40073f0700 1 -- 172.21.15.107:0/378802109 >> 172.21.15.107:0/378802109 conn(0x7f40000f7900 msgr2=0x7f40000ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:31:27.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:27.341+0000 7f40073f0700 1 -- 172.21.15.107:0/378802109 shutdown_connections 2024-04-22T23:31:27.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:27.341+0000 7f40073f0700 1 -- 172.21.15.107:0/378802109 wait complete. 2024-04-22T23:31:27.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:27 smithi107 ceph-mon[26355]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:31:27.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:27 smithi157 ceph-mon[33877]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:31:29.084 INFO:teuthology.orchestra.run.smithi107.stdout:167503724552 2024-04-22T23:31:29.085 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-22T23:31:29.650 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:29 smithi107 ceph-mon[26355]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:31:29.884 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:29 smithi157 ceph-mon[33877]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:31:30.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:30.176+0000 7f0b03587700 1 -- 172.21.15.107:0/14494752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0afc102e90 msgr2=0x7f0afc103260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:30.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:30.176+0000 7f0b03587700 1 --2- 172.21.15.107:0/14494752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0afc102e90 0x7f0afc103260 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0af8004940 tx=0x7f0af8009e30 comp rx=0 tx=0).stop 2024-04-22T23:31:30.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:30.176+0000 7f0b03587700 1 -- 172.21.15.107:0/14494752 shutdown_connections 2024-04-22T23:31:30.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:30.176+0000 7f0b03587700 1 --2- 172.21.15.107:0/14494752 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0afc0fb4a0 0x7f0afc0fb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:30.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:30.176+0000 7f0b03587700 1 --2- 172.21.15.107:0/14494752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0afc102e90 0x7f0afc103260 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:30.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:30.176+0000 7f0b03587700 1 -- 172.21.15.107:0/14494752 >> 172.21.15.107:0/14494752 conn(0x7f0afc0714c0 msgr2=0x7f0afc0718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:31:30.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:30.176+0000 7f0b03587700 1 -- 172.21.15.107:0/14494752 shutdown_connections 2024-04-22T23:31:30.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:30.176+0000 7f0b03587700 1 -- 172.21.15.107:0/14494752 wait complete. 2024-04-22T23:31:30.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:30.177+0000 7f0b03587700 1 Processor -- start 2024-04-22T23:31:30.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:30.177+0000 7f0b03587700 1 -- start start 2024-04-22T23:31:30.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:30.177+0000 7f0b03587700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0afc0fb4a0 0x7f0afc113b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:30.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:30.177+0000 7f0b03587700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0afc102e90 0x7f0afc1140d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:30.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:30.177+0000 7f0b03587700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0afc1147b0 con 0x7f0afc102e90 2024-04-22T23:31:30.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:30.177+0000 7f0b03587700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0afc10db80 con 0x7f0afc0fb4a0 2024-04-22T23:31:30.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:30.177+0000 7f0b00b22700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0afc102e90 0x7f0afc1140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:30.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:30.177+0000 7f0b00b22700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0afc102e90 0x7f0afc1140d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42510/0 (socket says 172.21.15.107:42510) 2024-04-22T23:31:30.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:30.177+0000 7f0b00b22700 1 -- 172.21.15.107:0/475592830 learned_addr learned my addr 172.21.15.107:0/475592830 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:31:30.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:30.177+0000 7f0b01323700 1 --2- 172.21.15.107:0/475592830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0afc0fb4a0 0x7f0afc113b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:30.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:30.178+0000 7f0b00b22700 1 -- 172.21.15.107:0/475592830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0afc0fb4a0 msgr2=0x7f0afc113b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:30.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:30.178+0000 7f0b00b22700 1 --2- 172.21.15.107:0/475592830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0afc0fb4a0 0x7f0afc113b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:30.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:30.178+0000 7f0b00b22700 1 -- 172.21.15.107:0/475592830 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af8005040 con 0x7f0afc102e90 2024-04-22T23:31:30.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:30.178+0000 7f0b00b22700 1 --2- 172.21.15.107:0/475592830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0afc102e90 0x7f0afc1140d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0aec00b790 tx=0x7f0aec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:31:30.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:30.178+0000 7f0af1ffb700 1 -- 172.21.15.107:0/475592830 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0aec00f6f0 con 0x7f0afc102e90 2024-04-22T23:31:30.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:30.178+0000 7f0b03587700 1 -- 172.21.15.107:0/475592830 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0afc10de60 con 0x7f0afc102e90 2024-04-22T23:31:30.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:30.178+0000 7f0af1ffb700 1 -- 172.21.15.107:0/475592830 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0aec00fd20 con 0x7f0afc102e90 2024-04-22T23:31:30.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:30.178+0000 7f0af1ffb700 1 -- 172.21.15.107:0/475592830 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0aec0103a0 con 0x7f0afc102e90 2024-04-22T23:31:30.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:30.178+0000 7f0b03587700 1 -- 172.21.15.107:0/475592830 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0afc10e3a0 con 0x7f0afc102e90 2024-04-22T23:31:30.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:30.179+0000 7f0b03587700 1 -- 172.21.15.107:0/475592830 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0afc10e970 con 0x7f0afc102e90 2024-04-22T23:31:30.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:30.180+0000 7f0af1ffb700 1 -- 172.21.15.107:0/475592830 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f0aec011400 con 0x7f0afc102e90 2024-04-22T23:31:30.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:30.180+0000 7f0af1ffb700 1 --2- 172.21.15.107:0/475592830 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f0ae80692a0 0x7f0ae806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:30.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:30.180+0000 7f0af1ffb700 1 -- 172.21.15.107:0/475592830 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f0aec086b60 con 0x7f0afc102e90 2024-04-22T23:31:30.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:30.180+0000 7f0b01323700 1 --2- 172.21.15.107:0/475592830 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f0ae80692a0 0x7f0ae806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:30.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:30.180+0000 7f0af1ffb700 1 --2- 172.21.15.107:0/475592830 >> [v2:172.21.15.107:6810/335851250,v1:172.21.15.107:6811/335851250] conn(0x7f0ae806ef70 0x7f0ae8071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:30.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:30.180+0000 7f0af1ffb700 1 -- 172.21.15.107:0/475592830 --> [v2:172.21.15.107:6810/335851250,v1:172.21.15.107:6811/335851250] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0ae8071a30 con 0x7f0ae806ef70 2024-04-22T23:31:30.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:30.180+0000 7f0af1ffb700 1 -- 172.21.15.107:0/475592830 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0aec08b220 con 0x7f0afc102e90 2024-04-22T23:31:30.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:30.180+0000 7f0b01b24700 1 --2- 172.21.15.107:0/475592830 >> [v2:172.21.15.107:6810/335851250,v1:172.21.15.107:6811/335851250] conn(0x7f0ae806ef70 0x7f0ae8071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:30.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:30.181+0000 7f0b01323700 1 --2- 172.21.15.107:0/475592830 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f0ae80692a0 0x7f0ae806b750 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0af8004f90 tx=0x7f0af8009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:31:30.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:30.181+0000 7f0b01b24700 1 --2- 172.21.15.107:0/475592830 >> [v2:172.21.15.107:6810/335851250,v1:172.21.15.107:6811/335851250] conn(0x7f0ae806ef70 0x7f0ae8071380 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:31:30.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:30.182+0000 7f0af1ffb700 1 -- 172.21.15.107:0/475592830 <== osd.1 v2:172.21.15.107:6810/335851250 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f0ae8071a30 con 0x7f0ae806ef70 2024-04-22T23:31:30.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:30.197+0000 7f0b03587700 1 -- 172.21.15.107:0/475592830 --> [v2:172.21.15.107:6810/335851250,v1:172.21.15.107:6811/335851250] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0afc05e640 con 0x7f0ae806ef70 2024-04-22T23:31:30.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:30.198+0000 7f0af1ffb700 1 -- 172.21.15.107:0/475592830 <== osd.1 v2:172.21.15.107:6810/335851250 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0afc05e640 con 0x7f0ae806ef70 2024-04-22T23:31:30.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:30.198+0000 7f0b03587700 1 -- 172.21.15.107:0/475592830 >> [v2:172.21.15.107:6810/335851250,v1:172.21.15.107:6811/335851250] conn(0x7f0ae806ef70 msgr2=0x7f0ae8071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:30.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:30.198+0000 7f0b03587700 1 --2- 172.21.15.107:0/475592830 >> [v2:172.21.15.107:6810/335851250,v1:172.21.15.107:6811/335851250] conn(0x7f0ae806ef70 0x7f0ae8071380 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:30.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:30.198+0000 7f0b03587700 1 -- 172.21.15.107:0/475592830 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f0ae80692a0 msgr2=0x7f0ae806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:30.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:30.198+0000 7f0b03587700 1 --2- 172.21.15.107:0/475592830 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f0ae80692a0 0x7f0ae806b750 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0af8004f90 tx=0x7f0af8009e30 comp rx=0 tx=0).stop 2024-04-22T23:31:30.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:30.198+0000 7f0b03587700 1 -- 172.21.15.107:0/475592830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0afc102e90 msgr2=0x7f0afc1140d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:30.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:30.198+0000 7f0b03587700 1 --2- 172.21.15.107:0/475592830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0afc102e90 0x7f0afc1140d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0aec00b790 tx=0x7f0aec009250 comp rx=0 tx=0).stop 2024-04-22T23:31:30.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:30.198+0000 7f0b03587700 1 -- 172.21.15.107:0/475592830 shutdown_connections 2024-04-22T23:31:30.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:30.198+0000 7f0b03587700 1 --2- 172.21.15.107:0/475592830 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f0ae80692a0 0x7f0ae806b750 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:30.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:30.198+0000 7f0b03587700 1 --2- 172.21.15.107:0/475592830 >> [v2:172.21.15.107:6810/335851250,v1:172.21.15.107:6811/335851250] conn(0x7f0ae806ef70 0x7f0ae8071380 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:30.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:30.198+0000 7f0b03587700 1 --2- 172.21.15.107:0/475592830 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0afc0fb4a0 0x7f0afc113b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:30.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:30.198+0000 7f0b03587700 1 --2- 172.21.15.107:0/475592830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0afc102e90 0x7f0afc1140d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:30.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:30.198+0000 7f0b03587700 1 -- 172.21.15.107:0/475592830 >> 172.21.15.107:0/475592830 conn(0x7f0afc0714c0 msgr2=0x7f0afc0fa6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:31:30.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:30.198+0000 7f0b03587700 1 -- 172.21.15.107:0/475592830 shutdown_connections 2024-04-22T23:31:30.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:30.198+0000 7f0b03587700 1 -- 172.21.15.107:0/475592830 wait complete. 2024-04-22T23:31:30.952 INFO:teuthology.orchestra.run.smithi107.stdout:60129542173 2024-04-22T23:31:30.953 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-22T23:31:31.469 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:31 smithi107 ceph-mon[26355]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:31:31.884 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:31 smithi157 ceph-mon[33877]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:31:31.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:31.452+0000 7f246de11700 1 -- 172.21.15.107:0/123849634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2468102f60 msgr2=0x7f2468103330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:31.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:31.452+0000 7f246de11700 1 --2- 172.21.15.107:0/123849634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2468102f60 0x7f2468103330 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2458004940 tx=0x7f2458009e30 comp rx=0 tx=0).stop 2024-04-22T23:31:31.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:31.452+0000 7f246de11700 1 -- 172.21.15.107:0/123849634 shutdown_connections 2024-04-22T23:31:31.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:31.452+0000 7f246de11700 1 --2- 172.21.15.107:0/123849634 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24680fb4d0 0x7f24680fb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:31.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:31.452+0000 7f246de11700 1 --2- 172.21.15.107:0/123849634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2468102f60 0x7f2468103330 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:31.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:31.452+0000 7f246de11700 1 -- 172.21.15.107:0/123849634 >> 172.21.15.107:0/123849634 conn(0x7f24680714c0 msgr2=0x7f24680718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:31:31.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:31.452+0000 7f246de11700 1 -- 172.21.15.107:0/123849634 shutdown_connections 2024-04-22T23:31:31.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:31.452+0000 7f246de11700 1 -- 172.21.15.107:0/123849634 wait complete. 2024-04-22T23:31:31.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:31.452+0000 7f246de11700 1 Processor -- start 2024-04-22T23:31:31.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:31.453+0000 7f246de11700 1 -- start start 2024-04-22T23:31:31.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:31.453+0000 7f246de11700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24680fb4d0 0x7f2468113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:31.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:31.453+0000 7f246de11700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2468102f60 0x7f2468113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:31.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:31.453+0000 7f246de11700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2468114520 con 0x7f24680fb4d0 2024-04-22T23:31:31.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:31.453+0000 7f246de11700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f246810db20 con 0x7f2468102f60 2024-04-22T23:31:31.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:31.453+0000 7f24677fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24680fb4d0 0x7f2468113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:31.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:31.453+0000 7f24677fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24680fb4d0 0x7f2468113990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42516/0 (socket says 172.21.15.107:42516) 2024-04-22T23:31:31.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:31.453+0000 7f24677fe700 1 -- 172.21.15.107:0/2294962120 learned_addr learned my addr 172.21.15.107:0/2294962120 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:31:31.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:31.453+0000 7f2466ffd700 1 --2- 172.21.15.107:0/2294962120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2468102f60 0x7f2468113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:31.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:31.453+0000 7f24677fe700 1 -- 172.21.15.107:0/2294962120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2468102f60 msgr2=0x7f2468113ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:31.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:31.453+0000 7f24677fe700 1 --2- 172.21.15.107:0/2294962120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2468102f60 0x7f2468113ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:31.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:31.453+0000 7f24677fe700 1 -- 172.21.15.107:0/2294962120 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2458005040 con 0x7f24680fb4d0 2024-04-22T23:31:31.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:31.453+0000 7f24677fe700 1 --2- 172.21.15.107:0/2294962120 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24680fb4d0 0x7f2468113990 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f245800b8b0 tx=0x7f2458009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:31:31.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:31.453+0000 7f244ffff700 1 -- 172.21.15.107:0/2294962120 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f245800e040 con 0x7f24680fb4d0 2024-04-22T23:31:31.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:31.453+0000 7f246de11700 1 -- 172.21.15.107:0/2294962120 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f246810dda0 con 0x7f24680fb4d0 2024-04-22T23:31:31.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:31.453+0000 7f244ffff700 1 -- 172.21.15.107:0/2294962120 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2458004290 con 0x7f24680fb4d0 2024-04-22T23:31:31.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:31.453+0000 7f246de11700 1 -- 172.21.15.107:0/2294962120 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f246810e360 con 0x7f24680fb4d0 2024-04-22T23:31:31.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:31.453+0000 7f244ffff700 1 -- 172.21.15.107:0/2294962120 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f245801c3f0 con 0x7f24680fb4d0 2024-04-22T23:31:31.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:31.454+0000 7f246de11700 1 -- 172.21.15.107:0/2294962120 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f246810e950 con 0x7f24680fb4d0 2024-04-22T23:31:31.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:31.455+0000 7f244ffff700 1 -- 172.21.15.107:0/2294962120 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f24580137b0 con 0x7f24680fb4d0 2024-04-22T23:31:31.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:31.455+0000 7f244ffff700 1 --2- 172.21.15.107:0/2294962120 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f24500692a0 0x7f245006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:31.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:31.455+0000 7f244ffff700 1 -- 172.21.15.107:0/2294962120 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f2458088190 con 0x7f24680fb4d0 2024-04-22T23:31:31.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:31.455+0000 7f2466ffd700 1 --2- 172.21.15.107:0/2294962120 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f24500692a0 0x7f245006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:31.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:31.455+0000 7f244ffff700 1 --2- 172.21.15.107:0/2294962120 >> [v2:172.21.15.107:6802/4253936450,v1:172.21.15.107:6803/4253936450] conn(0x7f245006ef70 0x7f2450071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:31.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:31.455+0000 7f244ffff700 1 -- 172.21.15.107:0/2294962120 --> [v2:172.21.15.107:6802/4253936450,v1:172.21.15.107:6803/4253936450] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2450071a30 con 0x7f245006ef70 2024-04-22T23:31:31.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:31.455+0000 7f244ffff700 1 -- 172.21.15.107:0/2294962120 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f245808a780 con 0x7f24680fb4d0 2024-04-22T23:31:31.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:31.455+0000 7f2467fff700 1 --2- 172.21.15.107:0/2294962120 >> [v2:172.21.15.107:6802/4253936450,v1:172.21.15.107:6803/4253936450] conn(0x7f245006ef70 0x7f2450071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:31.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:31.455+0000 7f2466ffd700 1 --2- 172.21.15.107:0/2294962120 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f24500692a0 0x7f245006b750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f245c007fd0 tx=0x7f245c009040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:31:31.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:31.456+0000 7f2467fff700 1 --2- 172.21.15.107:0/2294962120 >> [v2:172.21.15.107:6802/4253936450,v1:172.21.15.107:6803/4253936450] conn(0x7f245006ef70 0x7f2450071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:31:31.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:31.456+0000 7f244ffff700 1 -- 172.21.15.107:0/2294962120 <== osd.0 v2:172.21.15.107:6802/4253936450 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f2450071a30 con 0x7f245006ef70 2024-04-22T23:31:31.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:31.472+0000 7f246de11700 1 -- 172.21.15.107:0/2294962120 --> [v2:172.21.15.107:6802/4253936450,v1:172.21.15.107:6803/4253936450] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f246805e640 con 0x7f245006ef70 2024-04-22T23:31:31.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:31.472+0000 7f244ffff700 1 -- 172.21.15.107:0/2294962120 <== osd.0 v2:172.21.15.107:6802/4253936450 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f246805e640 con 0x7f245006ef70 2024-04-22T23:31:31.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:31.472+0000 7f246de11700 1 -- 172.21.15.107:0/2294962120 >> [v2:172.21.15.107:6802/4253936450,v1:172.21.15.107:6803/4253936450] conn(0x7f245006ef70 msgr2=0x7f2450071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:31.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:31.472+0000 7f246de11700 1 --2- 172.21.15.107:0/2294962120 >> [v2:172.21.15.107:6802/4253936450,v1:172.21.15.107:6803/4253936450] conn(0x7f245006ef70 0x7f2450071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:31.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:31.472+0000 7f246de11700 1 -- 172.21.15.107:0/2294962120 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f24500692a0 msgr2=0x7f245006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:31.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:31.472+0000 7f246de11700 1 --2- 172.21.15.107:0/2294962120 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f24500692a0 0x7f245006b750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f245c007fd0 tx=0x7f245c009040 comp rx=0 tx=0).stop 2024-04-22T23:31:31.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:31.472+0000 7f246de11700 1 -- 172.21.15.107:0/2294962120 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24680fb4d0 msgr2=0x7f2468113990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:31.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:31.472+0000 7f246de11700 1 --2- 172.21.15.107:0/2294962120 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24680fb4d0 0x7f2468113990 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f245800b8b0 tx=0x7f2458009e30 comp rx=0 tx=0).stop 2024-04-22T23:31:31.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:31.472+0000 7f246de11700 1 -- 172.21.15.107:0/2294962120 shutdown_connections 2024-04-22T23:31:31.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:31.472+0000 7f246de11700 1 --2- 172.21.15.107:0/2294962120 >> [v2:172.21.15.107:6802/4253936450,v1:172.21.15.107:6803/4253936450] conn(0x7f245006ef70 0x7f2450071380 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:31.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:31.472+0000 7f246de11700 1 --2- 172.21.15.107:0/2294962120 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f24500692a0 0x7f245006b750 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:31.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:31.472+0000 7f246de11700 1 --2- 172.21.15.107:0/2294962120 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24680fb4d0 0x7f2468113990 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:31.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:31.472+0000 7f246de11700 1 --2- 172.21.15.107:0/2294962120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2468102f60 0x7f2468113ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:31.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:31.472+0000 7f246de11700 1 -- 172.21.15.107:0/2294962120 >> 172.21.15.107:0/2294962120 conn(0x7f24680714c0 msgr2=0x7f24680fa6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:31:31.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:31.473+0000 7f246de11700 1 -- 172.21.15.107:0/2294962120 shutdown_connections 2024-04-22T23:31:31.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:31.473+0000 7f246de11700 1 -- 172.21.15.107:0/2294962120 wait complete. 2024-04-22T23:31:33.116 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:31:33.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:33.147+0000 7fbd789be700 1 -- 172.21.15.107:0/2065909919 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd74102e00 msgr2=0x7fbd741031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:33.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:33.147+0000 7fbd789be700 1 --2- 172.21.15.107:0/2065909919 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd74102e00 0x7fbd741031d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbd64004940 tx=0x7fbd64009e30 comp rx=0 tx=0).stop 2024-04-22T23:31:33.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:33.147+0000 7fbd789be700 1 -- 172.21.15.107:0/2065909919 shutdown_connections 2024-04-22T23:31:33.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:33.147+0000 7fbd789be700 1 --2- 172.21.15.107:0/2065909919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd741037a0 0x7fbd740fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:33.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:33.147+0000 7fbd789be700 1 --2- 172.21.15.107:0/2065909919 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd74102e00 0x7fbd741031d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:33.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:33.147+0000 7fbd789be700 1 -- 172.21.15.107:0/2065909919 >> 172.21.15.107:0/2065909919 conn(0x7fbd740f8950 msgr2=0x7fbd740fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:31:33.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:33.147+0000 7fbd789be700 1 -- 172.21.15.107:0/2065909919 shutdown_connections 2024-04-22T23:31:33.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:33.147+0000 7fbd789be700 1 -- 172.21.15.107:0/2065909919 wait complete. 2024-04-22T23:31:33.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:33.148+0000 7fbd789be700 1 Processor -- start 2024-04-22T23:31:33.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:33.148+0000 7fbd789be700 1 -- start start 2024-04-22T23:31:33.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:33.148+0000 7fbd789be700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd74102e00 0x7fbd74113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:33.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:33.148+0000 7fbd789be700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd741037a0 0x7fbd74113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:33.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:33.148+0000 7fbd789be700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd741145b0 con 0x7fbd741037a0 2024-04-22T23:31:33.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:33.148+0000 7fbd789be700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7410db20 con 0x7fbd74102e00 2024-04-22T23:31:33.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:33.148+0000 7fbd71d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd741037a0 0x7fbd74113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:33.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:33.148+0000 7fbd71d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd741037a0 0x7fbd74113ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33498/0 (socket says 172.21.15.107:33498) 2024-04-22T23:31:33.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:33.148+0000 7fbd71d9b700 1 -- 172.21.15.107:0/1514177900 learned_addr learned my addr 172.21.15.107:0/1514177900 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:31:33.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:33.148+0000 7fbd7259c700 1 --2- 172.21.15.107:0/1514177900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd74102e00 0x7fbd74113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:33.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:33.149+0000 7fbd71d9b700 1 -- 172.21.15.107:0/1514177900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd74102e00 msgr2=0x7fbd74113990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:33.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:33.149+0000 7fbd71d9b700 1 --2- 172.21.15.107:0/1514177900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd74102e00 0x7fbd74113990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:33.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:33.149+0000 7fbd71d9b700 1 -- 172.21.15.107:0/1514177900 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd64005040 con 0x7fbd741037a0 2024-04-22T23:31:33.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:33.149+0000 7fbd71d9b700 1 --2- 172.21.15.107:0/1514177900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd741037a0 0x7fbd74113ed0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbd6800b790 tx=0x7fbd68009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:31:33.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:33.149+0000 7fbd62ffd700 1 -- 172.21.15.107:0/1514177900 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd6800f6f0 con 0x7fbd741037a0 2024-04-22T23:31:33.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:33.149+0000 7fbd62ffd700 1 -- 172.21.15.107:0/1514177900 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbd6800fd20 con 0x7fbd741037a0 2024-04-22T23:31:33.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:33.149+0000 7fbd62ffd700 1 -- 172.21.15.107:0/1514177900 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd680103a0 con 0x7fbd741037a0 2024-04-22T23:31:33.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:33.149+0000 7fbd789be700 1 -- 172.21.15.107:0/1514177900 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd7410de00 con 0x7fbd741037a0 2024-04-22T23:31:33.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:33.149+0000 7fbd789be700 1 -- 172.21.15.107:0/1514177900 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd7410e370 con 0x7fbd741037a0 2024-04-22T23:31:33.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:33.150+0000 7fbd789be700 1 -- 172.21.15.107:0/1514177900 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbd7410e940 con 0x7fbd741037a0 2024-04-22T23:31:33.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:33.150+0000 7fbd62ffd700 1 -- 172.21.15.107:0/1514177900 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fbd68011400 con 0x7fbd741037a0 2024-04-22T23:31:33.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:33.151+0000 7fbd62ffd700 1 --2- 172.21.15.107:0/1514177900 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fbd5c0692a0 0x7fbd5c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:33.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:33.151+0000 7fbd62ffd700 1 -- 172.21.15.107:0/1514177900 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fbd68086d20 con 0x7fbd741037a0 2024-04-22T23:31:33.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:33.151+0000 7fbd7259c700 1 --2- 172.21.15.107:0/1514177900 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fbd5c0692a0 0x7fbd5c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:33.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:33.151+0000 7fbd62ffd700 1 --2- 172.21.15.107:0/1514177900 >> [v2:172.21.15.157:6808/2372899121,v1:172.21.15.157:6809/2372899121] conn(0x7fbd5c06ef70 0x7fbd5c071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:33.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:33.151+0000 7fbd62ffd700 1 -- 172.21.15.107:0/1514177900 --> [v2:172.21.15.157:6808/2372899121,v1:172.21.15.157:6809/2372899121] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbd5c071a30 con 0x7fbd5c06ef70 2024-04-22T23:31:33.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:33.151+0000 7fbd62ffd700 1 -- 172.21.15.107:0/1514177900 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd6808a020 con 0x7fbd741037a0 2024-04-22T23:31:33.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:33.151+0000 7fbd7259c700 1 --2- 172.21.15.107:0/1514177900 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fbd5c0692a0 0x7fbd5c06b750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbd64004f90 tx=0x7fbd64009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:31:33.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:33.151+0000 7fbd72d9d700 1 --2- 172.21.15.107:0/1514177900 >> [v2:172.21.15.157:6808/2372899121,v1:172.21.15.157:6809/2372899121] conn(0x7fbd5c06ef70 0x7fbd5c071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:33.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:33.152+0000 7fbd72d9d700 1 --2- 172.21.15.107:0/1514177900 >> [v2:172.21.15.157:6808/2372899121,v1:172.21.15.157:6809/2372899121] conn(0x7fbd5c06ef70 0x7fbd5c071380 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:31:33.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:33.152+0000 7fbd62ffd700 1 -- 172.21.15.107:0/1514177900 <== osd.4 v2:172.21.15.157:6808/2372899121 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fbd5c071a30 con 0x7fbd5c06ef70 2024-04-22T23:31:33.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:33.167+0000 7fbd789be700 1 -- 172.21.15.107:0/1514177900 --> [v2:172.21.15.157:6808/2372899121,v1:172.21.15.157:6809/2372899121] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbd740fa210 con 0x7fbd5c06ef70 2024-04-22T23:31:33.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:33.167+0000 7fbd62ffd700 1 -- 172.21.15.107:0/1514177900 <== osd.4 v2:172.21.15.157:6808/2372899121 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fbd740fa210 con 0x7fbd5c06ef70 2024-04-22T23:31:33.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:33.168+0000 7fbd789be700 1 -- 172.21.15.107:0/1514177900 >> [v2:172.21.15.157:6808/2372899121,v1:172.21.15.157:6809/2372899121] conn(0x7fbd5c06ef70 msgr2=0x7fbd5c071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:33.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:33.168+0000 7fbd789be700 1 --2- 172.21.15.107:0/1514177900 >> [v2:172.21.15.157:6808/2372899121,v1:172.21.15.157:6809/2372899121] conn(0x7fbd5c06ef70 0x7fbd5c071380 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:33.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:33.168+0000 7fbd789be700 1 -- 172.21.15.107:0/1514177900 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fbd5c0692a0 msgr2=0x7fbd5c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:33.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:33.168+0000 7fbd789be700 1 --2- 172.21.15.107:0/1514177900 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fbd5c0692a0 0x7fbd5c06b750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbd64004f90 tx=0x7fbd64009e30 comp rx=0 tx=0).stop 2024-04-22T23:31:33.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:33.168+0000 7fbd789be700 1 -- 172.21.15.107:0/1514177900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd741037a0 msgr2=0x7fbd74113ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:33.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:33.168+0000 7fbd789be700 1 --2- 172.21.15.107:0/1514177900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd741037a0 0x7fbd74113ed0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbd6800b790 tx=0x7fbd68009220 comp rx=0 tx=0).stop 2024-04-22T23:31:33.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:33.168+0000 7fbd789be700 1 -- 172.21.15.107:0/1514177900 shutdown_connections 2024-04-22T23:31:33.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:33.168+0000 7fbd789be700 1 --2- 172.21.15.107:0/1514177900 >> [v2:172.21.15.157:6808/2372899121,v1:172.21.15.157:6809/2372899121] conn(0x7fbd5c06ef70 0x7fbd5c071380 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:33.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:33.168+0000 7fbd789be700 1 --2- 172.21.15.107:0/1514177900 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fbd5c0692a0 0x7fbd5c06b750 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:33.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:33.168+0000 7fbd789be700 1 --2- 172.21.15.107:0/1514177900 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd74102e00 0x7fbd74113990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:33.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:33.168+0000 7fbd789be700 1 --2- 172.21.15.107:0/1514177900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd741037a0 0x7fbd74113ed0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:33.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:33.168+0000 7fbd789be700 1 -- 172.21.15.107:0/1514177900 >> 172.21.15.107:0/1514177900 conn(0x7fbd740f8950 msgr2=0x7fbd740f98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:31:33.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:33.168+0000 7fbd789be700 1 -- 172.21.15.107:0/1514177900 shutdown_connections 2024-04-22T23:31:33.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:33.168+0000 7fbd789be700 1 -- 172.21.15.107:0/1514177900 wait complete. 2024-04-22T23:31:33.485 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:33 smithi107 ceph-mon[26355]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:31:33.884 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:33 smithi157 ceph-mon[33877]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:31:34.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:34.630+0000 7f45d348d700 1 -- 172.21.15.107:0/4145925978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45cc1026b0 msgr2=0x7f45cc0fbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:34.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:34.630+0000 7f45d348d700 1 --2- 172.21.15.107:0/4145925978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45cc1026b0 0x7f45cc0fbd20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f45c8004940 tx=0x7f45c8009e30 comp rx=0 tx=0).stop 2024-04-22T23:31:34.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:34.630+0000 7f45d348d700 1 -- 172.21.15.107:0/4145925978 shutdown_connections 2024-04-22T23:31:34.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:34.630+0000 7f45d348d700 1 --2- 172.21.15.107:0/4145925978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45cc1026b0 0x7f45cc0fbd20 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:34.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:34.630+0000 7f45d348d700 1 --2- 172.21.15.107:0/4145925978 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45cc101d10 0x7f45cc1020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:34.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:34.630+0000 7f45d348d700 1 -- 172.21.15.107:0/4145925978 >> 172.21.15.107:0/4145925978 conn(0x7f45cc072bc0 msgr2=0x7f45cc072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:31:34.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:34.630+0000 7f45d348d700 1 -- 172.21.15.107:0/4145925978 shutdown_connections 2024-04-22T23:31:34.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:34.630+0000 7f45d348d700 1 -- 172.21.15.107:0/4145925978 wait complete. 2024-04-22T23:31:34.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:34.630+0000 7f45d348d700 1 Processor -- start 2024-04-22T23:31:34.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:34.631+0000 7f45d348d700 1 -- start start 2024-04-22T23:31:34.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:34.631+0000 7f45d348d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45cc101d10 0x7f45cc113b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:34.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:34.631+0000 7f45d348d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45cc1026b0 0x7f45cc1140d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:34.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:34.631+0000 7f45d348d700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45cc114720 con 0x7f45cc1026b0 2024-04-22T23:31:34.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:34.631+0000 7f45d348d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45cc10db10 con 0x7f45cc101d10 2024-04-22T23:31:34.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:34.631+0000 7f45d0a28700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45cc1026b0 0x7f45cc1140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:34.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:34.631+0000 7f45d0a28700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45cc1026b0 0x7f45cc1140d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33516/0 (socket says 172.21.15.107:33516) 2024-04-22T23:31:34.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:34.631+0000 7f45d0a28700 1 -- 172.21.15.107:0/2392749874 learned_addr learned my addr 172.21.15.107:0/2392749874 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:31:34.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:34.631+0000 7f45d1229700 1 --2- 172.21.15.107:0/2392749874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45cc101d10 0x7f45cc113b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:34.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:34.631+0000 7f45d0a28700 1 -- 172.21.15.107:0/2392749874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45cc101d10 msgr2=0x7f45cc113b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:34.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:34.631+0000 7f45d0a28700 1 --2- 172.21.15.107:0/2392749874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45cc101d10 0x7f45cc113b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:34.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:34.631+0000 7f45d0a28700 1 -- 172.21.15.107:0/2392749874 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c8005040 con 0x7f45cc1026b0 2024-04-22T23:31:34.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:34.631+0000 7f45d0a28700 1 --2- 172.21.15.107:0/2392749874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45cc1026b0 0x7f45cc1140d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f45c800b3b0 tx=0x7f45c8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:31:34.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:34.631+0000 7f45c1ffb700 1 -- 172.21.15.107:0/2392749874 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45c800e040 con 0x7f45cc1026b0 2024-04-22T23:31:34.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:34.632+0000 7f45c1ffb700 1 -- 172.21.15.107:0/2392749874 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f45c8009b10 con 0x7f45cc1026b0 2024-04-22T23:31:34.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:34.632+0000 7f45c1ffb700 1 -- 172.21.15.107:0/2392749874 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45c8013420 con 0x7f45cc1026b0 2024-04-22T23:31:34.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:34.632+0000 7f45d348d700 1 -- 172.21.15.107:0/2392749874 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45cc10dd90 con 0x7f45cc1026b0 2024-04-22T23:31:34.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:34.632+0000 7f45d348d700 1 -- 172.21.15.107:0/2392749874 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45cc10e300 con 0x7f45cc1026b0 2024-04-22T23:31:34.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:34.633+0000 7f45d348d700 1 -- 172.21.15.107:0/2392749874 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f45cc10e830 con 0x7f45cc1026b0 2024-04-22T23:31:34.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:34.633+0000 7f45c1ffb700 1 -- 172.21.15.107:0/2392749874 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f45c801c7c0 con 0x7f45cc1026b0 2024-04-22T23:31:34.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:34.634+0000 7f45c1ffb700 1 --2- 172.21.15.107:0/2392749874 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f45b80692a0 0x7f45b806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:34.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:34.634+0000 7f45c1ffb700 1 -- 172.21.15.107:0/2392749874 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f45c8087980 con 0x7f45cc1026b0 2024-04-22T23:31:34.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:34.634+0000 7f45d1229700 1 --2- 172.21.15.107:0/2392749874 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f45b80692a0 0x7f45b806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:34.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:34.634+0000 7f45c1ffb700 1 --2- 172.21.15.107:0/2392749874 >> [v2:172.21.15.157:6800/811815812,v1:172.21.15.157:6801/811815812] conn(0x7f45b806ef70 0x7f45b8071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:34.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:34.634+0000 7f45c1ffb700 1 -- 172.21.15.107:0/2392749874 --> [v2:172.21.15.157:6800/811815812,v1:172.21.15.157:6801/811815812] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f45b8071a30 con 0x7f45b806ef70 2024-04-22T23:31:34.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:34.634+0000 7f45c1ffb700 1 -- 172.21.15.107:0/2392749874 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45c808c220 con 0x7f45cc1026b0 2024-04-22T23:31:34.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:34.634+0000 7f45d1229700 1 --2- 172.21.15.107:0/2392749874 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f45b80692a0 0x7f45b806b750 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f45bc004f40 tx=0x7f45bc00b040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:31:34.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:34.634+0000 7f45d1a2a700 1 --2- 172.21.15.107:0/2392749874 >> [v2:172.21.15.157:6800/811815812,v1:172.21.15.157:6801/811815812] conn(0x7f45b806ef70 0x7f45b8071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:34.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:34.635+0000 7f45d1a2a700 1 --2- 172.21.15.107:0/2392749874 >> [v2:172.21.15.157:6800/811815812,v1:172.21.15.157:6801/811815812] conn(0x7f45b806ef70 0x7f45b8071380 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:31:34.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:34.635+0000 7f45c1ffb700 1 -- 172.21.15.107:0/2392749874 <== osd.3 v2:172.21.15.157:6800/811815812 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f45b8071a30 con 0x7f45b806ef70 2024-04-22T23:31:34.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:34.650+0000 7f45d348d700 1 -- 172.21.15.107:0/2392749874 --> [v2:172.21.15.157:6800/811815812,v1:172.21.15.157:6801/811815812] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f45cc065f20 con 0x7f45b806ef70 2024-04-22T23:31:34.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:34.651+0000 7f45c1ffb700 1 -- 172.21.15.107:0/2392749874 <== osd.3 v2:172.21.15.157:6800/811815812 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f45cc065f20 con 0x7f45b806ef70 2024-04-22T23:31:34.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:34.651+0000 7f45d348d700 1 -- 172.21.15.107:0/2392749874 >> [v2:172.21.15.157:6800/811815812,v1:172.21.15.157:6801/811815812] conn(0x7f45b806ef70 msgr2=0x7f45b8071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:34.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:34.651+0000 7f45d348d700 1 --2- 172.21.15.107:0/2392749874 >> [v2:172.21.15.157:6800/811815812,v1:172.21.15.157:6801/811815812] conn(0x7f45b806ef70 0x7f45b8071380 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:34.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:34.651+0000 7f45d348d700 1 -- 172.21.15.107:0/2392749874 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f45b80692a0 msgr2=0x7f45b806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:34.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:34.651+0000 7f45d348d700 1 --2- 172.21.15.107:0/2392749874 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f45b80692a0 0x7f45b806b750 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f45bc004f40 tx=0x7f45bc00b040 comp rx=0 tx=0).stop 2024-04-22T23:31:34.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:34.651+0000 7f45d348d700 1 -- 172.21.15.107:0/2392749874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45cc1026b0 msgr2=0x7f45cc1140d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:34.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:34.651+0000 7f45d348d700 1 --2- 172.21.15.107:0/2392749874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45cc1026b0 0x7f45cc1140d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f45c800b3b0 tx=0x7f45c8009e30 comp rx=0 tx=0).stop 2024-04-22T23:31:34.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:34.651+0000 7f45d348d700 1 -- 172.21.15.107:0/2392749874 shutdown_connections 2024-04-22T23:31:34.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:34.651+0000 7f45d348d700 1 --2- 172.21.15.107:0/2392749874 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f45b80692a0 0x7f45b806b750 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:34.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:34.651+0000 7f45d348d700 1 --2- 172.21.15.107:0/2392749874 >> [v2:172.21.15.157:6800/811815812,v1:172.21.15.157:6801/811815812] conn(0x7f45b806ef70 0x7f45b8071380 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:34.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:34.651+0000 7f45d348d700 1 --2- 172.21.15.107:0/2392749874 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45cc101d10 0x7f45cc113b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:34.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:34.651+0000 7f45d348d700 1 --2- 172.21.15.107:0/2392749874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45cc1026b0 0x7f45cc1140d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:34.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:34.651+0000 7f45d348d700 1 -- 172.21.15.107:0/2392749874 >> 172.21.15.107:0/2392749874 conn(0x7f45cc072bc0 msgr2=0x7f45cc0655d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:31:34.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:34.651+0000 7f45d348d700 1 -- 172.21.15.107:0/2392749874 shutdown_connections 2024-04-22T23:31:34.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:34.651+0000 7f45d348d700 1 -- 172.21.15.107:0/2392749874 wait complete. 2024-04-22T23:31:34.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:34 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:31:34.884 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:34 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:31:35.161 INFO:teuthology.orchestra.run.smithi107.stdout:38654705698 2024-04-22T23:31:35.162 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-22T23:31:35.594 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:35 smithi107 ceph-mon[26355]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:31:35.673 INFO:teuthology.orchestra.run.smithi107.stdout:146028888078 2024-04-22T23:31:35.673 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-22T23:31:35.884 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:35 smithi157 ceph-mon[33877]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:31:36.811 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:31:37.190 INFO:teuthology.orchestra.run.smithi107.stdout:111669149715 2024-04-22T23:31:37.190 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-22T23:31:37.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:37 smithi157 ceph-mon[33877]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:31:37.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:37 smithi107 ceph-mon[26355]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:31:38.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:38.272+0000 7fa6ca142700 1 -- 172.21.15.107:0/1753323330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6c4101d40 msgr2=0x7fa6c4102110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:38.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:38.272+0000 7fa6ca142700 1 --2- 172.21.15.107:0/1753323330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6c4101d40 0x7fa6c4102110 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4004940 tx=0x7fa6b4009e30 comp rx=0 tx=0).stop 2024-04-22T23:31:38.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:38.272+0000 7fa6ca142700 1 -- 172.21.15.107:0/1753323330 shutdown_connections 2024-04-22T23:31:38.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:38.272+0000 7fa6ca142700 1 --2- 172.21.15.107:0/1753323330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6c41026e0 0x7fa6c40fbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:38.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:38.272+0000 7fa6ca142700 1 --2- 172.21.15.107:0/1753323330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6c4101d40 0x7fa6c4102110 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:38.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:38.272+0000 7fa6ca142700 1 -- 172.21.15.107:0/1753323330 >> 172.21.15.107:0/1753323330 conn(0x7fa6c40f7870 msgr2=0x7fa6c40f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:31:38.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:38.272+0000 7fa6ca142700 1 -- 172.21.15.107:0/1753323330 shutdown_connections 2024-04-22T23:31:38.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:38.272+0000 7fa6ca142700 1 -- 172.21.15.107:0/1753323330 wait complete. 2024-04-22T23:31:38.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:38.273+0000 7fa6ca142700 1 Processor -- start 2024-04-22T23:31:38.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:38.273+0000 7fa6ca142700 1 -- start start 2024-04-22T23:31:38.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:38.273+0000 7fa6ca142700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6c4101d40 0x7fa6c41128d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:38.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:38.273+0000 7fa6ca142700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6c41026e0 0x7fa6c4112e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:38.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:38.273+0000 7fa6ca142700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c4113460 con 0x7fa6c4101d40 2024-04-22T23:31:38.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:38.273+0000 7fa6ca142700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c410ca60 con 0x7fa6c41026e0 2024-04-22T23:31:38.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:38.274+0000 7fa6c37fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6c4101d40 0x7fa6c41128d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:38.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:38.274+0000 7fa6c37fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6c4101d40 0x7fa6c41128d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33534/0 (socket says 172.21.15.107:33534) 2024-04-22T23:31:38.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:38.274+0000 7fa6c37fe700 1 -- 172.21.15.107:0/735811092 learned_addr learned my addr 172.21.15.107:0/735811092 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:31:38.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:38.274+0000 7fa6c2ffd700 1 --2- 172.21.15.107:0/735811092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6c41026e0 0x7fa6c4112e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:38.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:38.274+0000 7fa6c37fe700 1 -- 172.21.15.107:0/735811092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6c41026e0 msgr2=0x7fa6c4112e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:38.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:38.274+0000 7fa6c37fe700 1 --2- 172.21.15.107:0/735811092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6c41026e0 0x7fa6c4112e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:38.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:38.274+0000 7fa6c37fe700 1 -- 172.21.15.107:0/735811092 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b4005040 con 0x7fa6c4101d40 2024-04-22T23:31:38.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:38.274+0000 7fa6c37fe700 1 --2- 172.21.15.107:0/735811092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6c4101d40 0x7fa6c41128d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa6b400b3b0 tx=0x7fa6b4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:31:38.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:38.274+0000 7fa6abfff700 1 -- 172.21.15.107:0/735811092 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6b400e040 con 0x7fa6c4101d40 2024-04-22T23:31:38.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:38.274+0000 7fa6ca142700 1 -- 172.21.15.107:0/735811092 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6c410cce0 con 0x7fa6c4101d40 2024-04-22T23:31:38.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:38.274+0000 7fa6abfff700 1 -- 172.21.15.107:0/735811092 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa6b400bbc0 con 0x7fa6c4101d40 2024-04-22T23:31:38.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:38.274+0000 7fa6abfff700 1 -- 172.21.15.107:0/735811092 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6b401c3f0 con 0x7fa6c4101d40 2024-04-22T23:31:38.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:38.274+0000 7fa6ca142700 1 -- 172.21.15.107:0/735811092 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6c410d2a0 con 0x7fa6c4101d40 2024-04-22T23:31:38.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:38.275+0000 7fa6ca142700 1 -- 172.21.15.107:0/735811092 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa6c410d780 con 0x7fa6c4101d40 2024-04-22T23:31:38.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:38.276+0000 7fa6abfff700 1 -- 172.21.15.107:0/735811092 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fa6b400bd30 con 0x7fa6c4101d40 2024-04-22T23:31:38.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:38.276+0000 7fa6abfff700 1 --2- 172.21.15.107:0/735811092 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fa6ac0692a0 0x7fa6ac06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:38.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:38.276+0000 7fa6abfff700 1 -- 172.21.15.107:0/735811092 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fa6b4088b70 con 0x7fa6c4101d40 2024-04-22T23:31:38.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:38.276+0000 7fa6c2ffd700 1 --2- 172.21.15.107:0/735811092 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fa6ac0692a0 0x7fa6ac06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:38.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:38.276+0000 7fa6abfff700 1 --2- 172.21.15.107:0/735811092 >> [v2:172.21.15.107:6818/750102033,v1:172.21.15.107:6819/750102033] conn(0x7fa6ac06ef70 0x7fa6ac071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:38.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:38.276+0000 7fa6abfff700 1 -- 172.21.15.107:0/735811092 --> [v2:172.21.15.107:6818/750102033,v1:172.21.15.107:6819/750102033] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa6ac071a30 con 0x7fa6ac06ef70 2024-04-22T23:31:38.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:38.276+0000 7fa6abfff700 1 -- 172.21.15.107:0/735811092 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6b4088f50 con 0x7fa6c4101d40 2024-04-22T23:31:38.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:38.276+0000 7fa6c3fff700 1 --2- 172.21.15.107:0/735811092 >> [v2:172.21.15.107:6818/750102033,v1:172.21.15.107:6819/750102033] conn(0x7fa6ac06ef70 0x7fa6ac071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:38.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:38.276+0000 7fa6c2ffd700 1 --2- 172.21.15.107:0/735811092 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fa6ac0692a0 0x7fa6ac06b750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8007fd0 tx=0x7fa6b8009040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:31:38.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:38.277+0000 7fa6c3fff700 1 --2- 172.21.15.107:0/735811092 >> [v2:172.21.15.107:6818/750102033,v1:172.21.15.107:6819/750102033] conn(0x7fa6ac06ef70 0x7fa6ac071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:31:38.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:38.277+0000 7fa6abfff700 1 -- 172.21.15.107:0/735811092 <== osd.2 v2:172.21.15.107:6818/750102033 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fa6ac071a30 con 0x7fa6ac06ef70 2024-04-22T23:31:38.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:38.292+0000 7fa6ca142700 1 -- 172.21.15.107:0/735811092 --> [v2:172.21.15.107:6818/750102033,v1:172.21.15.107:6819/750102033] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa6c40f9190 con 0x7fa6ac06ef70 2024-04-22T23:31:38.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:38.292+0000 7fa6abfff700 1 -- 172.21.15.107:0/735811092 <== osd.2 v2:172.21.15.107:6818/750102033 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa6c40f9190 con 0x7fa6ac06ef70 2024-04-22T23:31:38.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:38.292+0000 7fa6ca142700 1 -- 172.21.15.107:0/735811092 >> [v2:172.21.15.107:6818/750102033,v1:172.21.15.107:6819/750102033] conn(0x7fa6ac06ef70 msgr2=0x7fa6ac071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:38.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:38.292+0000 7fa6ca142700 1 --2- 172.21.15.107:0/735811092 >> [v2:172.21.15.107:6818/750102033,v1:172.21.15.107:6819/750102033] conn(0x7fa6ac06ef70 0x7fa6ac071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:38.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:38.293+0000 7fa6ca142700 1 -- 172.21.15.107:0/735811092 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fa6ac0692a0 msgr2=0x7fa6ac06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:38.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:38.293+0000 7fa6ca142700 1 --2- 172.21.15.107:0/735811092 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fa6ac0692a0 0x7fa6ac06b750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8007fd0 tx=0x7fa6b8009040 comp rx=0 tx=0).stop 2024-04-22T23:31:38.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:38.293+0000 7fa6ca142700 1 -- 172.21.15.107:0/735811092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6c4101d40 msgr2=0x7fa6c41128d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:38.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:38.293+0000 7fa6ca142700 1 --2- 172.21.15.107:0/735811092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6c4101d40 0x7fa6c41128d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa6b400b3b0 tx=0x7fa6b4009e30 comp rx=0 tx=0).stop 2024-04-22T23:31:38.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:38.293+0000 7fa6ca142700 1 -- 172.21.15.107:0/735811092 shutdown_connections 2024-04-22T23:31:38.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:38.293+0000 7fa6ca142700 1 --2- 172.21.15.107:0/735811092 >> [v2:172.21.15.107:6818/750102033,v1:172.21.15.107:6819/750102033] conn(0x7fa6ac06ef70 0x7fa6ac071380 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:38.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:38.293+0000 7fa6ca142700 1 --2- 172.21.15.107:0/735811092 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fa6ac0692a0 0x7fa6ac06b750 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:38.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:38.293+0000 7fa6ca142700 1 --2- 172.21.15.107:0/735811092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6c4101d40 0x7fa6c41128d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:38.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:38.293+0000 7fa6ca142700 1 --2- 172.21.15.107:0/735811092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6c41026e0 0x7fa6c4112e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:38.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:38.293+0000 7fa6ca142700 1 -- 172.21.15.107:0/735811092 >> 172.21.15.107:0/735811092 conn(0x7fa6c40f7870 msgr2=0x7fa6c40f87c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:31:38.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:38.293+0000 7fa6ca142700 1 -- 172.21.15.107:0/735811092 shutdown_connections 2024-04-22T23:31:38.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:38.293+0000 7fa6ca142700 1 -- 172.21.15.107:0/735811092 wait complete. 2024-04-22T23:31:39.637 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:39 smithi107 ceph-mon[26355]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:31:39.706 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:39 smithi157 ceph-mon[33877]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:31:39.855 INFO:teuthology.orchestra.run.smithi107.stdout:81604378649 2024-04-22T23:31:39.856 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-22T23:31:40.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:40.335+0000 7fa78d290700 1 -- 172.21.15.107:0/1022518924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7880fb4a0 msgr2=0x7fa7880fb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:40.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:40.335+0000 7fa78d290700 1 --2- 172.21.15.107:0/1022518924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7880fb4a0 0x7fa7880fb910 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa77c004940 tx=0x7fa77c009e30 comp rx=0 tx=0).stop 2024-04-22T23:31:40.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:40.335+0000 7fa78d290700 1 -- 172.21.15.107:0/1022518924 shutdown_connections 2024-04-22T23:31:40.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:40.335+0000 7fa78d290700 1 --2- 172.21.15.107:0/1022518924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7880fb4a0 0x7fa7880fb910 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:40.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:40.335+0000 7fa78d290700 1 --2- 172.21.15.107:0/1022518924 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa788102e90 0x7fa788103260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:40.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:40.335+0000 7fa78d290700 1 -- 172.21.15.107:0/1022518924 >> 172.21.15.107:0/1022518924 conn(0x7fa7880714c0 msgr2=0x7fa7880718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:31:40.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:40.335+0000 7fa78d290700 1 -- 172.21.15.107:0/1022518924 shutdown_connections 2024-04-22T23:31:40.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:40.335+0000 7fa78d290700 1 -- 172.21.15.107:0/1022518924 wait complete. 2024-04-22T23:31:40.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:40.336+0000 7fa78d290700 1 Processor -- start 2024-04-22T23:31:40.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:40.336+0000 7fa78d290700 1 -- start start 2024-04-22T23:31:40.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:40.336+0000 7fa78d290700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7880fb4a0 0x7fa788113b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:40.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:40.336+0000 7fa78d290700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa788102e90 0x7fa7881140a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:40.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:40.336+0000 7fa78d290700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7881146f0 con 0x7fa7880fb4a0 2024-04-22T23:31:40.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:40.336+0000 7fa78d290700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa78810dae0 con 0x7fa788102e90 2024-04-22T23:31:40.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:40.336+0000 7fa786ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7880fb4a0 0x7fa788113b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:40.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:40.336+0000 7fa786ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7880fb4a0 0x7fa788113b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33548/0 (socket says 172.21.15.107:33548) 2024-04-22T23:31:40.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:40.336+0000 7fa786ffd700 1 -- 172.21.15.107:0/947178123 learned_addr learned my addr 172.21.15.107:0/947178123 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:31:40.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:40.336+0000 7fa7867fc700 1 --2- 172.21.15.107:0/947178123 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa788102e90 0x7fa7881140a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:40.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:40.336+0000 7fa786ffd700 1 -- 172.21.15.107:0/947178123 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa788102e90 msgr2=0x7fa7881140a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:40.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:40.336+0000 7fa786ffd700 1 --2- 172.21.15.107:0/947178123 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa788102e90 0x7fa7881140a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:40.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:40.336+0000 7fa786ffd700 1 -- 172.21.15.107:0/947178123 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa77c005040 con 0x7fa7880fb4a0 2024-04-22T23:31:40.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:40.336+0000 7fa786ffd700 1 --2- 172.21.15.107:0/947178123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7880fb4a0 0x7fa788113b60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa77800eb40 tx=0x7fa77800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:31:40.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:40.336+0000 7fa76f7fe700 1 -- 172.21.15.107:0/947178123 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa778007cc0 con 0x7fa7880fb4a0 2024-04-22T23:31:40.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:40.337+0000 7fa78d290700 1 -- 172.21.15.107:0/947178123 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa78810dd60 con 0x7fa7880fb4a0 2024-04-22T23:31:40.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:40.337+0000 7fa76f7fe700 1 -- 172.21.15.107:0/947178123 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa77800f040 con 0x7fa7880fb4a0 2024-04-22T23:31:40.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:40.337+0000 7fa76f7fe700 1 -- 172.21.15.107:0/947178123 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa77800ca10 con 0x7fa7880fb4a0 2024-04-22T23:31:40.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:40.337+0000 7fa78d290700 1 -- 172.21.15.107:0/947178123 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa78810e270 con 0x7fa7880fb4a0 2024-04-22T23:31:40.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:40.337+0000 7fa78d290700 1 -- 172.21.15.107:0/947178123 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa788100170 con 0x7fa7880fb4a0 2024-04-22T23:31:40.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:40.338+0000 7fa76f7fe700 1 -- 172.21.15.107:0/947178123 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fa77800cc80 con 0x7fa7880fb4a0 2024-04-22T23:31:40.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:40.338+0000 7fa76f7fe700 1 --2- 172.21.15.107:0/947178123 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fa7700692a0 0x7fa77006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:40.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:40.338+0000 7fa76f7fe700 1 -- 172.21.15.107:0/947178123 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fa778086a50 con 0x7fa7880fb4a0 2024-04-22T23:31:40.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:40.338+0000 7fa7867fc700 1 --2- 172.21.15.107:0/947178123 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fa7700692a0 0x7fa77006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:40.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:40.339+0000 7fa7867fc700 1 --2- 172.21.15.107:0/947178123 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fa7700692a0 0x7fa77006b750 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa77c009b30 tx=0x7fa77c009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:31:40.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:40.343+0000 7fa76f7fe700 1 -- 172.21.15.107:0/947178123 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fa77804e250 con 0x7fa7880fb4a0 2024-04-22T23:31:40.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:40.541+0000 7fa78d290700 1 -- 172.21.15.107:0/947178123 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fa78810f1b0 con 0x7fa7880fb4a0 2024-04-22T23:31:40.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:40.542+0000 7fa76f7fe700 1 -- 172.21.15.107:0/947178123 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa778056b80 con 0x7fa7880fb4a0 2024-04-22T23:31:40.538 INFO:teuthology.orchestra.run.smithi107.stdout:167503724555 2024-04-22T23:31:40.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:40.544+0000 7fa78d290700 1 -- 172.21.15.107:0/947178123 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fa7700692a0 msgr2=0x7fa77006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:40.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:40.544+0000 7fa78d290700 1 --2- 172.21.15.107:0/947178123 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fa7700692a0 0x7fa77006b750 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa77c009b30 tx=0x7fa77c009e30 comp rx=0 tx=0).stop 2024-04-22T23:31:40.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:40.544+0000 7fa78d290700 1 -- 172.21.15.107:0/947178123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7880fb4a0 msgr2=0x7fa788113b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:40.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:40.544+0000 7fa78d290700 1 --2- 172.21.15.107:0/947178123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7880fb4a0 0x7fa788113b60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa77800eb40 tx=0x7fa77800b040 comp rx=0 tx=0).stop 2024-04-22T23:31:40.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:40.544+0000 7fa78d290700 1 -- 172.21.15.107:0/947178123 shutdown_connections 2024-04-22T23:31:40.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:40.544+0000 7fa78d290700 1 --2- 172.21.15.107:0/947178123 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fa7700692a0 0x7fa77006b750 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:40.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:40.544+0000 7fa78d290700 1 --2- 172.21.15.107:0/947178123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7880fb4a0 0x7fa788113b60 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:40.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:40.544+0000 7fa78d290700 1 --2- 172.21.15.107:0/947178123 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa788102e90 0x7fa7881140a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:40.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:40.544+0000 7fa78d290700 1 -- 172.21.15.107:0/947178123 >> 172.21.15.107:0/947178123 conn(0x7fa7880714c0 msgr2=0x7fa7880fa7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:31:40.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:40.544+0000 7fa78d290700 1 -- 172.21.15.107:0/947178123 shutdown_connections 2024-04-22T23:31:40.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:40.544+0000 7fa78d290700 1 -- 172.21.15.107:0/947178123 wait complete. 2024-04-22T23:31:41.331 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724552 got 167503724555 for osd.5 2024-04-22T23:31:41.331 DEBUG:teuthology.parallel:result is None 2024-04-22T23:31:41.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:41.526+0000 7f7f8c8c7700 1 -- 172.21.15.107:0/1144872885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f880fde80 msgr2=0x7f7f880fe250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:41.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:41.526+0000 7f7f8c8c7700 1 --2- 172.21.15.107:0/1144872885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f880fde80 0x7f7f880fe250 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7f78004940 tx=0x7f7f78009e30 comp rx=0 tx=0).stop 2024-04-22T23:31:41.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:41.527+0000 7f7f8c8c7700 1 -- 172.21.15.107:0/1144872885 shutdown_connections 2024-04-22T23:31:41.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:41.527+0000 7f7f8c8c7700 1 --2- 172.21.15.107:0/1144872885 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f880fe820 0x7f7f88106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:41.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:41.527+0000 7f7f8c8c7700 1 --2- 172.21.15.107:0/1144872885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f880fde80 0x7f7f880fe250 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:41.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:41.527+0000 7f7f8c8c7700 1 -- 172.21.15.107:0/1144872885 >> 172.21.15.107:0/1144872885 conn(0x7f7f880f7870 msgr2=0x7f7f880f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:31:41.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:41.527+0000 7f7f8c8c7700 1 -- 172.21.15.107:0/1144872885 shutdown_connections 2024-04-22T23:31:41.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:41.527+0000 7f7f8c8c7700 1 -- 172.21.15.107:0/1144872885 wait complete. 2024-04-22T23:31:41.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:41.527+0000 7f7f8c8c7700 1 Processor -- start 2024-04-22T23:31:41.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:41.527+0000 7f7f8c8c7700 1 -- start start 2024-04-22T23:31:41.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:41.527+0000 7f7f8c8c7700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f880fde80 0x7f7f88113910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:41.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:41.527+0000 7f7f8c8c7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f880fe820 0x7f7f88113e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:41.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:41.527+0000 7f7f8c8c7700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f88114530 con 0x7f7f880fe820 2024-04-22T23:31:41.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:41.527+0000 7f7f8c8c7700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8810daa0 con 0x7f7f880fde80 2024-04-22T23:31:41.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:41.527+0000 7f7f85d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f880fe820 0x7f7f88113e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:41.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:41.528+0000 7f7f85d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f880fe820 0x7f7f88113e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39304/0 (socket says 172.21.15.107:39304) 2024-04-22T23:31:41.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:41.528+0000 7f7f85d9b700 1 -- 172.21.15.107:0/1308140297 learned_addr learned my addr 172.21.15.107:0/1308140297 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:31:41.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:41.528+0000 7f7f8659c700 1 --2- 172.21.15.107:0/1308140297 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f880fde80 0x7f7f88113910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:41.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:41.528+0000 7f7f85d9b700 1 -- 172.21.15.107:0/1308140297 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f880fde80 msgr2=0x7f7f88113910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:41.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:41.528+0000 7f7f85d9b700 1 --2- 172.21.15.107:0/1308140297 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f880fde80 0x7f7f88113910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:41.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:41.528+0000 7f7f85d9b700 1 -- 172.21.15.107:0/1308140297 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f78005040 con 0x7f7f880fe820 2024-04-22T23:31:41.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:41.528+0000 7f7f85d9b700 1 --2- 172.21.15.107:0/1308140297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f880fe820 0x7f7f88113e50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c00b790 tx=0x7f7f7c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:31:41.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:41.528+0000 7f7f76ffd700 1 -- 172.21.15.107:0/1308140297 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f7c00f6f0 con 0x7f7f880fe820 2024-04-22T23:31:41.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:41.528+0000 7f7f76ffd700 1 -- 172.21.15.107:0/1308140297 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7f7c00fd20 con 0x7f7f880fe820 2024-04-22T23:31:41.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:41.528+0000 7f7f8c8c7700 1 -- 172.21.15.107:0/1308140297 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f8810dd80 con 0x7f7f880fe820 2024-04-22T23:31:41.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:41.528+0000 7f7f76ffd700 1 -- 172.21.15.107:0/1308140297 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f7c0103a0 con 0x7f7f880fe820 2024-04-22T23:31:41.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:41.528+0000 7f7f8c8c7700 1 -- 172.21.15.107:0/1308140297 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f8810e2c0 con 0x7f7f880fe820 2024-04-22T23:31:41.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:41.529+0000 7f7f8c8c7700 1 -- 172.21.15.107:0/1308140297 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f88066b60 con 0x7f7f880fe820 2024-04-22T23:31:41.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:41.529+0000 7f7f76ffd700 1 -- 172.21.15.107:0/1308140297 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f7f7c011400 con 0x7f7f880fe820 2024-04-22T23:31:41.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:41.530+0000 7f7f76ffd700 1 --2- 172.21.15.107:0/1308140297 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f7f700692a0 0x7f7f7006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:41.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:41.530+0000 7f7f76ffd700 1 -- 172.21.15.107:0/1308140297 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f7f7c086c40 con 0x7f7f880fe820 2024-04-22T23:31:41.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:41.530+0000 7f7f8659c700 1 --2- 172.21.15.107:0/1308140297 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f7f700692a0 0x7f7f7006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:41.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:41.530+0000 7f7f8659c700 1 --2- 172.21.15.107:0/1308140297 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f7f700692a0 0x7f7f7006b750 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7f78004f90 tx=0x7f7f78009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:31:41.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:41.534+0000 7f7f76ffd700 1 -- 172.21.15.107:0/1308140297 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f7f7c056d40 con 0x7f7f880fe820 2024-04-22T23:31:41.604 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:31:41.660 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:31:41.661 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:31:41.662 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:31:41.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:41 smithi107 ceph-mon[26355]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:31:41.736 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:41 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/947178123' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-22T23:31:41.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:41.745+0000 7f7f8c8c7700 1 -- 172.21.15.107:0/1308140297 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7f8804c710 con 0x7f7f880fe820 2024-04-22T23:31:41.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:41.746+0000 7f7f76ffd700 1 -- 172.21.15.107:0/1308140297 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7f7c0568d0 con 0x7f7f880fe820 2024-04-22T23:31:41.742 INFO:teuthology.orchestra.run.smithi107.stdout:60129542175 2024-04-22T23:31:41.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:41.748+0000 7f7f8c8c7700 1 -- 172.21.15.107:0/1308140297 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f7f700692a0 msgr2=0x7f7f7006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:41.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:41.748+0000 7f7f8c8c7700 1 --2- 172.21.15.107:0/1308140297 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f7f700692a0 0x7f7f7006b750 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7f78004f90 tx=0x7f7f78009e30 comp rx=0 tx=0).stop 2024-04-22T23:31:41.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:41.748+0000 7f7f8c8c7700 1 -- 172.21.15.107:0/1308140297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f880fe820 msgr2=0x7f7f88113e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:41.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:41.748+0000 7f7f8c8c7700 1 --2- 172.21.15.107:0/1308140297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f880fe820 0x7f7f88113e50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c00b790 tx=0x7f7f7c009250 comp rx=0 tx=0).stop 2024-04-22T23:31:41.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:41.748+0000 7f7f8c8c7700 1 -- 172.21.15.107:0/1308140297 shutdown_connections 2024-04-22T23:31:41.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:41.748+0000 7f7f8c8c7700 1 --2- 172.21.15.107:0/1308140297 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f7f700692a0 0x7f7f7006b750 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:41.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:41.748+0000 7f7f8c8c7700 1 --2- 172.21.15.107:0/1308140297 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7f880fde80 0x7f7f88113910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:41.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:41.748+0000 7f7f8c8c7700 1 --2- 172.21.15.107:0/1308140297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f880fe820 0x7f7f88113e50 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:41.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:41.748+0000 7f7f8c8c7700 1 -- 172.21.15.107:0/1308140297 >> 172.21.15.107:0/1308140297 conn(0x7f7f880f7870 msgr2=0x7f7f880f8d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:31:41.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:41.748+0000 7f7f8c8c7700 1 -- 172.21.15.107:0/1308140297 shutdown_connections 2024-04-22T23:31:41.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:41.748+0000 7f7f8c8c7700 1 -- 172.21.15.107:0/1308140297 wait complete. 2024-04-22T23:31:41.884 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:41 smithi157 ceph-mon[33877]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:31:41.884 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:41 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/947178123' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-22T23:31:42.673 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:42 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/1308140297' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-22T23:31:42.884 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:42 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/1308140297' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-22T23:31:43.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:43 smithi107 ceph-mon[26355]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:31:43.884 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:43 smithi157 ceph-mon[33877]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:31:45.698 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:45 smithi107 ceph-mon[26355]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:31:45.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:45.800+0000 7fd38fee4700 1 -- 172.21.15.107:0/2277833424 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd388101d40 msgr2=0x7fd388102110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:45.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:45.800+0000 7fd38fee4700 1 --2- 172.21.15.107:0/2277833424 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd388101d40 0x7fd388102110 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd378004940 tx=0x7fd378009e30 comp rx=0 tx=0).stop 2024-04-22T23:31:45.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:45.800+0000 7fd38fee4700 1 -- 172.21.15.107:0/2277833424 shutdown_connections 2024-04-22T23:31:45.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:45.800+0000 7fd38fee4700 1 --2- 172.21.15.107:0/2277833424 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3881026e0 0x7fd3880fbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:45.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:45.800+0000 7fd38fee4700 1 --2- 172.21.15.107:0/2277833424 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd388101d40 0x7fd388102110 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:45.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:45.800+0000 7fd38fee4700 1 -- 172.21.15.107:0/2277833424 >> 172.21.15.107:0/2277833424 conn(0x7fd3880f7870 msgr2=0x7fd3880f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:31:45.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:45.801+0000 7fd38fee4700 1 -- 172.21.15.107:0/2277833424 shutdown_connections 2024-04-22T23:31:45.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:45.801+0000 7fd38fee4700 1 -- 172.21.15.107:0/2277833424 wait complete. 2024-04-22T23:31:45.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:45.801+0000 7fd38fee4700 1 Processor -- start 2024-04-22T23:31:45.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:45.801+0000 7fd38fee4700 1 -- start start 2024-04-22T23:31:45.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:45.801+0000 7fd38fee4700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd388101d40 0x7fd3881139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:45.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:45.801+0000 7fd38fee4700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3881026e0 0x7fd388113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:45.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:45.801+0000 7fd38fee4700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3881145c0 con 0x7fd3881026e0 2024-04-22T23:31:45.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:45.801+0000 7fd38fee4700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38810db30 con 0x7fd388101d40 2024-04-22T23:31:45.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:45.801+0000 7fd38d47f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3881026e0 0x7fd388113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:45.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:45.802+0000 7fd38d47f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3881026e0 0x7fd388113ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39328/0 (socket says 172.21.15.107:39328) 2024-04-22T23:31:45.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:45.802+0000 7fd38d47f700 1 -- 172.21.15.107:0/3585570179 learned_addr learned my addr 172.21.15.107:0/3585570179 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:31:45.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:45.802+0000 7fd38dc80700 1 --2- 172.21.15.107:0/3585570179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd388101d40 0x7fd3881139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:45.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:45.802+0000 7fd38d47f700 1 -- 172.21.15.107:0/3585570179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd388101d40 msgr2=0x7fd3881139a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:45.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:45.802+0000 7fd38d47f700 1 --2- 172.21.15.107:0/3585570179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd388101d40 0x7fd3881139a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:45.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:45.802+0000 7fd38d47f700 1 -- 172.21.15.107:0/3585570179 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd378005040 con 0x7fd3881026e0 2024-04-22T23:31:45.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:45.802+0000 7fd38d47f700 1 --2- 172.21.15.107:0/3585570179 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3881026e0 0x7fd388113ee0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd38400b790 tx=0x7fd384009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:31:45.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:45.802+0000 7fd37e7fc700 1 -- 172.21.15.107:0/3585570179 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd38400f6f0 con 0x7fd3881026e0 2024-04-22T23:31:45.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:45.802+0000 7fd38fee4700 1 -- 172.21.15.107:0/3585570179 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd38810de10 con 0x7fd3881026e0 2024-04-22T23:31:45.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:45.802+0000 7fd37e7fc700 1 -- 172.21.15.107:0/3585570179 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd38400fd20 con 0x7fd3881026e0 2024-04-22T23:31:45.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:45.802+0000 7fd37e7fc700 1 -- 172.21.15.107:0/3585570179 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3840103a0 con 0x7fd3881026e0 2024-04-22T23:31:45.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:45.802+0000 7fd38fee4700 1 -- 172.21.15.107:0/3585570179 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd38810e3d0 con 0x7fd3881026e0 2024-04-22T23:31:45.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:45.803+0000 7fd38fee4700 1 -- 172.21.15.107:0/3585570179 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd388106440 con 0x7fd3881026e0 2024-04-22T23:31:45.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:45.804+0000 7fd37e7fc700 1 -- 172.21.15.107:0/3585570179 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fd384011400 con 0x7fd3881026e0 2024-04-22T23:31:45.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:45.804+0000 7fd37e7fc700 1 --2- 172.21.15.107:0/3585570179 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fd3740692a0 0x7fd37406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:45.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:45.804+0000 7fd37e7fc700 1 -- 172.21.15.107:0/3585570179 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fd384086c30 con 0x7fd3881026e0 2024-04-22T23:31:45.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:45.804+0000 7fd38dc80700 1 --2- 172.21.15.107:0/3585570179 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fd3740692a0 0x7fd37406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:45.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:45.804+0000 7fd38dc80700 1 --2- 172.21.15.107:0/3585570179 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fd3740692a0 0x7fd37406b750 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd378004f90 tx=0x7fd378009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:31:45.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:45.808+0000 7fd37e7fc700 1 -- 172.21.15.107:0/3585570179 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd384056d30 con 0x7fd3881026e0 2024-04-22T23:31:45.885 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:45 smithi157 ceph-mon[33877]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:31:46.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:46.004+0000 7fd38fee4700 1 -- 172.21.15.107:0/3585570179 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd38810eff0 con 0x7fd3881026e0 2024-04-22T23:31:46.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:46.004+0000 7fd37e7fc700 1 -- 172.21.15.107:0/3585570179 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd3840568c0 con 0x7fd3881026e0 2024-04-22T23:31:46.001 INFO:teuthology.orchestra.run.smithi107.stdout:81604378651 2024-04-22T23:31:46.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:46.006+0000 7fd38fee4700 1 -- 172.21.15.107:0/3585570179 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fd3740692a0 msgr2=0x7fd37406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:46.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:46.006+0000 7fd38fee4700 1 --2- 172.21.15.107:0/3585570179 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fd3740692a0 0x7fd37406b750 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd378004f90 tx=0x7fd378009e30 comp rx=0 tx=0).stop 2024-04-22T23:31:46.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:46.006+0000 7fd38fee4700 1 -- 172.21.15.107:0/3585570179 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3881026e0 msgr2=0x7fd388113ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:46.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:46.006+0000 7fd38fee4700 1 --2- 172.21.15.107:0/3585570179 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3881026e0 0x7fd388113ee0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd38400b790 tx=0x7fd384009220 comp rx=0 tx=0).stop 2024-04-22T23:31:46.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:46.006+0000 7fd38fee4700 1 -- 172.21.15.107:0/3585570179 shutdown_connections 2024-04-22T23:31:46.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:46.006+0000 7fd38fee4700 1 --2- 172.21.15.107:0/3585570179 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fd3740692a0 0x7fd37406b750 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:46.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:46.006+0000 7fd38fee4700 1 --2- 172.21.15.107:0/3585570179 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd388101d40 0x7fd3881139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:46.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:46.006+0000 7fd38fee4700 1 --2- 172.21.15.107:0/3585570179 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3881026e0 0x7fd388113ee0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:46.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:46.006+0000 7fd38fee4700 1 -- 172.21.15.107:0/3585570179 >> 172.21.15.107:0/3585570179 conn(0x7fd3880f7870 msgr2=0x7fd388066ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:31:46.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:46.007+0000 7fd38fee4700 1 -- 172.21.15.107:0/3585570179 shutdown_connections 2024-04-22T23:31:46.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:46.007+0000 7fd38fee4700 1 -- 172.21.15.107:0/3585570179 wait complete. 2024-04-22T23:31:46.545 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:46 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/3585570179' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-22T23:31:46.884 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:46 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/3585570179' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-22T23:31:47.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.285+0000 7fd9a8842700 1 -- 172.21.15.107:0/3423357720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9a4101d10 msgr2=0x7fd9a41020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:47.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.285+0000 7fd9a8842700 1 --2- 172.21.15.107:0/3423357720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9a4101d10 0x7fd9a41020e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd994004940 tx=0x7fd994009e30 comp rx=0 tx=0).stop 2024-04-22T23:31:47.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.285+0000 7fd9a8842700 1 -- 172.21.15.107:0/3423357720 shutdown_connections 2024-04-22T23:31:47.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.285+0000 7fd9a8842700 1 --2- 172.21.15.107:0/3423357720 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9a41026b0 0x7fd9a40fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:47.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.285+0000 7fd9a8842700 1 --2- 172.21.15.107:0/3423357720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9a4101d10 0x7fd9a41020e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:47.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.285+0000 7fd9a8842700 1 -- 172.21.15.107:0/3423357720 >> 172.21.15.107:0/3423357720 conn(0x7fd9a4072bc0 msgr2=0x7fd9a4072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:31:47.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.285+0000 7fd9a8842700 1 -- 172.21.15.107:0/3423357720 shutdown_connections 2024-04-22T23:31:47.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.285+0000 7fd9a8842700 1 -- 172.21.15.107:0/3423357720 wait complete. 2024-04-22T23:31:47.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.286+0000 7fd9a8842700 1 Processor -- start 2024-04-22T23:31:47.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.286+0000 7fd9a8842700 1 -- start start 2024-04-22T23:31:47.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.286+0000 7fd9a8842700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9a4101d10 0x7fd9a410ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:47.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.286+0000 7fd9a8842700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9a41026b0 0x7fd9a410f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:47.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.286+0000 7fd9a8842700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a410f740 con 0x7fd9a4101d10 2024-04-22T23:31:47.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.286+0000 7fd9a8842700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a410f880 con 0x7fd9a41026b0 2024-04-22T23:31:47.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.286+0000 7fd9a259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9a4101d10 0x7fd9a410ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:47.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.286+0000 7fd9a259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9a4101d10 0x7fd9a410ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39344/0 (socket says 172.21.15.107:39344) 2024-04-22T23:31:47.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.286+0000 7fd9a259c700 1 -- 172.21.15.107:0/2505069235 learned_addr learned my addr 172.21.15.107:0/2505069235 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:31:47.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.286+0000 7fd9a1d9b700 1 --2- 172.21.15.107:0/2505069235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9a41026b0 0x7fd9a410f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:47.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.286+0000 7fd9a259c700 1 -- 172.21.15.107:0/2505069235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9a41026b0 msgr2=0x7fd9a410f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:47.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.286+0000 7fd9a259c700 1 --2- 172.21.15.107:0/2505069235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9a41026b0 0x7fd9a410f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:47.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.286+0000 7fd9a259c700 1 -- 172.21.15.107:0/2505069235 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd994005040 con 0x7fd9a4101d10 2024-04-22T23:31:47.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.286+0000 7fd9a1d9b700 1 --2- 172.21.15.107:0/2505069235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9a41026b0 0x7fd9a410f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T23:31:47.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.286+0000 7fd9a259c700 1 --2- 172.21.15.107:0/2505069235 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9a4101d10 0x7fd9a410ecc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd99400b8b0 tx=0x7fd994009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:31:47.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.286+0000 7fd992ffd700 1 -- 172.21.15.107:0/2505069235 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd99400e040 con 0x7fd9a4101d10 2024-04-22T23:31:47.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.286+0000 7fd9a8842700 1 -- 172.21.15.107:0/2505069235 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9a4117b70 con 0x7fd9a4101d10 2024-04-22T23:31:47.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.286+0000 7fd992ffd700 1 -- 172.21.15.107:0/2505069235 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd994004290 con 0x7fd9a4101d10 2024-04-22T23:31:47.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.286+0000 7fd992ffd700 1 -- 172.21.15.107:0/2505069235 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd99401c3f0 con 0x7fd9a4101d10 2024-04-22T23:31:47.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.287+0000 7fd9a8842700 1 -- 172.21.15.107:0/2505069235 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9a41180d0 con 0x7fd9a4101d10 2024-04-22T23:31:47.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.287+0000 7fd9a8842700 1 -- 172.21.15.107:0/2505069235 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9a40fd460 con 0x7fd9a4101d10 2024-04-22T23:31:47.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.288+0000 7fd992ffd700 1 -- 172.21.15.107:0/2505069235 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fd9940137b0 con 0x7fd9a4101d10 2024-04-22T23:31:47.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.288+0000 7fd992ffd700 1 --2- 172.21.15.107:0/2505069235 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fd98c0692a0 0x7fd98c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:47.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.288+0000 7fd992ffd700 1 -- 172.21.15.107:0/2505069235 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fd994088190 con 0x7fd9a4101d10 2024-04-22T23:31:47.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.288+0000 7fd9a1d9b700 1 --2- 172.21.15.107:0/2505069235 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fd98c0692a0 0x7fd98c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:47.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.289+0000 7fd9a1d9b700 1 --2- 172.21.15.107:0/2505069235 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fd98c0692a0 0x7fd98c06b750 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd998007fd0 tx=0x7fd998009040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:31:47.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.293+0000 7fd992ffd700 1 -- 172.21.15.107:0/2505069235 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd994054800 con 0x7fd9a4101d10 2024-04-22T23:31:47.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.367+0000 7faebe5a0700 1 -- 172.21.15.107:0/3199653328 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faeb81026b0 msgr2=0x7faeb80fbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:47.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.367+0000 7faebe5a0700 1 --2- 172.21.15.107:0/3199653328 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faeb81026b0 0x7faeb80fbd20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7faea8004940 tx=0x7faea8009e30 comp rx=0 tx=0).stop 2024-04-22T23:31:47.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.367+0000 7faebe5a0700 1 -- 172.21.15.107:0/3199653328 shutdown_connections 2024-04-22T23:31:47.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.367+0000 7faebe5a0700 1 --2- 172.21.15.107:0/3199653328 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faeb81026b0 0x7faeb80fbd20 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:47.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.367+0000 7faebe5a0700 1 --2- 172.21.15.107:0/3199653328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faeb8101d10 0x7faeb81020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:47.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.367+0000 7faebe5a0700 1 -- 172.21.15.107:0/3199653328 >> 172.21.15.107:0/3199653328 conn(0x7faeb8072bc0 msgr2=0x7faeb8072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:31:47.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.367+0000 7faebe5a0700 1 -- 172.21.15.107:0/3199653328 shutdown_connections 2024-04-22T23:31:47.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.367+0000 7faebe5a0700 1 -- 172.21.15.107:0/3199653328 wait complete. 2024-04-22T23:31:47.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.367+0000 7faebe5a0700 1 Processor -- start 2024-04-22T23:31:47.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.368+0000 7faebe5a0700 1 -- start start 2024-04-22T23:31:47.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.368+0000 7faebe5a0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faeb8101d10 0x7faeb8112a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:47.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.368+0000 7faebe5a0700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faeb81026b0 0x7faeb8112f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:47.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.368+0000 7faebe5a0700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb8113590 con 0x7faeb8101d10 2024-04-22T23:31:47.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.368+0000 7faebe5a0700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb810c980 con 0x7faeb81026b0 2024-04-22T23:31:47.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.368+0000 7faeb7fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faeb8101d10 0x7faeb8112a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:47.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.368+0000 7faeb7fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faeb8101d10 0x7faeb8112a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39352/0 (socket says 172.21.15.107:39352) 2024-04-22T23:31:47.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.368+0000 7faeb7fff700 1 -- 172.21.15.107:0/3755060849 learned_addr learned my addr 172.21.15.107:0/3755060849 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:31:47.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.368+0000 7faeb77fe700 1 --2- 172.21.15.107:0/3755060849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faeb81026b0 0x7faeb8112f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:47.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.368+0000 7faeb7fff700 1 -- 172.21.15.107:0/3755060849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faeb81026b0 msgr2=0x7faeb8112f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:47.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.368+0000 7faeb7fff700 1 --2- 172.21.15.107:0/3755060849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faeb81026b0 0x7faeb8112f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:47.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.368+0000 7faeb7fff700 1 -- 172.21.15.107:0/3755060849 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea8005040 con 0x7faeb8101d10 2024-04-22T23:31:47.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.368+0000 7faeb7fff700 1 --2- 172.21.15.107:0/3755060849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faeb8101d10 0x7faeb8112a00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7faeac00eb40 tx=0x7faeac00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:31:47.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.369+0000 7faeb4ff9700 1 -- 172.21.15.107:0/3755060849 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faeac007cc0 con 0x7faeb8101d10 2024-04-22T23:31:47.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.369+0000 7faeb4ff9700 1 -- 172.21.15.107:0/3755060849 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faeac00f040 con 0x7faeb8101d10 2024-04-22T23:31:47.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.369+0000 7faeb4ff9700 1 -- 172.21.15.107:0/3755060849 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faeac009620 con 0x7faeb8101d10 2024-04-22T23:31:47.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.369+0000 7faebe5a0700 1 -- 172.21.15.107:0/3755060849 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faeb810cc00 con 0x7faeb8101d10 2024-04-22T23:31:47.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.369+0000 7faebe5a0700 1 -- 172.21.15.107:0/3755060849 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeb810d110 con 0x7faeb8101d10 2024-04-22T23:31:47.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.370+0000 7faebe5a0700 1 -- 172.21.15.107:0/3755060849 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faeb80fd460 con 0x7faeb8101d10 2024-04-22T23:31:47.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.370+0000 7faeb4ff9700 1 -- 172.21.15.107:0/3755060849 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7faeac009780 con 0x7faeb8101d10 2024-04-22T23:31:47.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.370+0000 7faeb4ff9700 1 --2- 172.21.15.107:0/3755060849 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7faea00692a0 0x7faea006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:47.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.371+0000 7faeb4ff9700 1 -- 172.21.15.107:0/3755060849 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7faeac086b20 con 0x7faeb8101d10 2024-04-22T23:31:47.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.371+0000 7faeb77fe700 1 --2- 172.21.15.107:0/3755060849 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7faea00692a0 0x7faea006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:47.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.371+0000 7faeb77fe700 1 --2- 172.21.15.107:0/3755060849 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7faea00692a0 0x7faea006b750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7faea8009b30 tx=0x7faea8009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:31:47.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.375+0000 7faeb4ff9700 1 -- 172.21.15.107:0/3755060849 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7faeac04e250 con 0x7faeb8101d10 2024-04-22T23:31:47.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.495+0000 7fd9a8842700 1 -- 172.21.15.107:0/2505069235 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd9a405f5f0 con 0x7fd9a4101d10 2024-04-22T23:31:47.492 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:47 smithi107 ceph-mon[26355]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:31:47.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.495+0000 7fd992ffd700 1 -- 172.21.15.107:0/2505069235 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd994057e20 con 0x7fd9a4101d10 2024-04-22T23:31:47.494 INFO:teuthology.orchestra.run.smithi107.stdout:111669149717 2024-04-22T23:31:47.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.497+0000 7fd9a8842700 1 -- 172.21.15.107:0/2505069235 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fd98c0692a0 msgr2=0x7fd98c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:47.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.497+0000 7fd9a8842700 1 --2- 172.21.15.107:0/2505069235 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fd98c0692a0 0x7fd98c06b750 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd998007fd0 tx=0x7fd998009040 comp rx=0 tx=0).stop 2024-04-22T23:31:47.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.497+0000 7fd9a8842700 1 -- 172.21.15.107:0/2505069235 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9a4101d10 msgr2=0x7fd9a410ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:47.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.497+0000 7fd9a8842700 1 --2- 172.21.15.107:0/2505069235 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9a4101d10 0x7fd9a410ecc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd99400b8b0 tx=0x7fd994009e30 comp rx=0 tx=0).stop 2024-04-22T23:31:47.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.497+0000 7fd9a8842700 1 -- 172.21.15.107:0/2505069235 shutdown_connections 2024-04-22T23:31:47.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.497+0000 7fd9a8842700 1 --2- 172.21.15.107:0/2505069235 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fd98c0692a0 0x7fd98c06b750 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:47.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.497+0000 7fd9a8842700 1 --2- 172.21.15.107:0/2505069235 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9a4101d10 0x7fd9a410ecc0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:47.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.497+0000 7fd9a8842700 1 --2- 172.21.15.107:0/2505069235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9a41026b0 0x7fd9a410f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:47.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.497+0000 7fd9a8842700 1 -- 172.21.15.107:0/2505069235 >> 172.21.15.107:0/2505069235 conn(0x7fd9a4072bc0 msgr2=0x7fd9a4104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:31:47.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.497+0000 7fd9a8842700 1 -- 172.21.15.107:0/2505069235 shutdown_connections 2024-04-22T23:31:47.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.497+0000 7fd9a8842700 1 -- 172.21.15.107:0/2505069235 wait complete. 2024-04-22T23:31:47.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.575+0000 7faebe5a0700 1 -- 172.21.15.107:0/3755060849 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7faeb8108df0 con 0x7faeb8101d10 2024-04-22T23:31:47.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.575+0000 7faeb4ff9700 1 -- 172.21.15.107:0/3755060849 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faeac056c50 con 0x7faeb8101d10 2024-04-22T23:31:47.573 INFO:teuthology.orchestra.run.smithi107.stdout:146028888080 2024-04-22T23:31:47.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.577+0000 7faebe5a0700 1 -- 172.21.15.107:0/3755060849 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7faea00692a0 msgr2=0x7faea006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:47.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.577+0000 7faebe5a0700 1 --2- 172.21.15.107:0/3755060849 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7faea00692a0 0x7faea006b750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7faea8009b30 tx=0x7faea8009e30 comp rx=0 tx=0).stop 2024-04-22T23:31:47.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.577+0000 7faebe5a0700 1 -- 172.21.15.107:0/3755060849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faeb8101d10 msgr2=0x7faeb8112a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:47.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.577+0000 7faebe5a0700 1 --2- 172.21.15.107:0/3755060849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faeb8101d10 0x7faeb8112a00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7faeac00eb40 tx=0x7faeac00b040 comp rx=0 tx=0).stop 2024-04-22T23:31:47.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.577+0000 7faebe5a0700 1 -- 172.21.15.107:0/3755060849 shutdown_connections 2024-04-22T23:31:47.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.577+0000 7faebe5a0700 1 --2- 172.21.15.107:0/3755060849 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7faea00692a0 0x7faea006b750 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:47.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.577+0000 7faebe5a0700 1 --2- 172.21.15.107:0/3755060849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faeb8101d10 0x7faeb8112a00 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:47.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.577+0000 7faebe5a0700 1 --2- 172.21.15.107:0/3755060849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faeb81026b0 0x7faeb8112f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:47.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.577+0000 7faebe5a0700 1 -- 172.21.15.107:0/3755060849 >> 172.21.15.107:0/3755060849 conn(0x7faeb8072bc0 msgr2=0x7faeb80fa150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:31:47.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.577+0000 7faebe5a0700 1 -- 172.21.15.107:0/3755060849 shutdown_connections 2024-04-22T23:31:47.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:47.577+0000 7faebe5a0700 1 -- 172.21.15.107:0/3755060849 wait complete. 2024-04-22T23:31:47.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:47 smithi157 ceph-mon[33877]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:31:47.894 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542173 got 60129542175 for osd.1 2024-04-22T23:31:47.895 DEBUG:teuthology.parallel:result is None 2024-04-22T23:31:48.324 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378649 got 81604378651 for osd.2 2024-04-22T23:31:48.324 DEBUG:teuthology.parallel:result is None 2024-04-22T23:31:48.585 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:48 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/2505069235' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-22T23:31:48.586 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:48 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/3755060849' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-22T23:31:48.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:48 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/2505069235' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-22T23:31:48.884 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:48 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/3755060849' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-22T23:31:49.286 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149715 got 111669149717 for osd.3 2024-04-22T23:31:49.286 DEBUG:teuthology.parallel:result is None 2024-04-22T23:31:49.538 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:49 smithi107 ceph-mon[26355]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:31:49.538 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:49 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:31:49.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:49 smithi157 ceph-mon[33877]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:31:49.884 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:49 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:31:49.905 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888078 got 146028888080 for osd.4 2024-04-22T23:31:49.905 DEBUG:teuthology.parallel:result is None 2024-04-22T23:31:50.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:50.413+0000 7f72ce1ea700 1 -- 172.21.15.107:0/3129544525 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72c8070300 msgr2=0x7f72c80706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:50.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:50.413+0000 7f72ce1ea700 1 --2- 172.21.15.107:0/3129544525 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72c8070300 0x7f72c80706d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f72b8004970 tx=0x7f72b8009e30 comp rx=0 tx=0).stop 2024-04-22T23:31:50.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:50.413+0000 7f72ce1ea700 1 -- 172.21.15.107:0/3129544525 shutdown_connections 2024-04-22T23:31:50.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:50.413+0000 7f72ce1ea700 1 --2- 172.21.15.107:0/3129544525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72c8070c10 0x7f72c8109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:50.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:50.413+0000 7f72ce1ea700 1 --2- 172.21.15.107:0/3129544525 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72c8070300 0x7f72c80706d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:50.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:50.413+0000 7f72ce1ea700 1 -- 172.21.15.107:0/3129544525 >> 172.21.15.107:0/3129544525 conn(0x7f72c80f89b0 msgr2=0x7f72c80fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:31:50.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:50.413+0000 7f72ce1ea700 1 -- 172.21.15.107:0/3129544525 shutdown_connections 2024-04-22T23:31:50.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:50.413+0000 7f72ce1ea700 1 -- 172.21.15.107:0/3129544525 wait complete. 2024-04-22T23:31:50.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:50.414+0000 7f72ce1ea700 1 Processor -- start 2024-04-22T23:31:50.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:50.414+0000 7f72ce1ea700 1 -- start start 2024-04-22T23:31:50.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:50.414+0000 7f72ce1ea700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72c8070300 0x7f72c8115bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:50.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:50.414+0000 7f72ce1ea700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72c8070c10 0x7f72c81160f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:50.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:50.414+0000 7f72ce1ea700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c81167d0 con 0x7f72c8070300 2024-04-22T23:31:50.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:50.414+0000 7f72ce1ea700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c810fd20 con 0x7f72c8070c10 2024-04-22T23:31:50.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:50.414+0000 7f72c77fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72c8070300 0x7f72c8115bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:50.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:50.414+0000 7f72c77fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72c8070300 0x7f72c8115bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39368/0 (socket says 172.21.15.107:39368) 2024-04-22T23:31:50.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:50.414+0000 7f72c77fe700 1 -- 172.21.15.107:0/244611135 learned_addr learned my addr 172.21.15.107:0/244611135 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:31:50.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:50.415+0000 7f72c6ffd700 1 --2- 172.21.15.107:0/244611135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72c8070c10 0x7f72c81160f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:50.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:50.415+0000 7f72c77fe700 1 -- 172.21.15.107:0/244611135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72c8070c10 msgr2=0x7f72c81160f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:50.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:50.415+0000 7f72c77fe700 1 --2- 172.21.15.107:0/244611135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72c8070c10 0x7f72c81160f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:50.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:50.415+0000 7f72c77fe700 1 -- 172.21.15.107:0/244611135 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b8005040 con 0x7f72c8070300 2024-04-22T23:31:50.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:50.415+0000 7f72c6ffd700 1 --2- 172.21.15.107:0/244611135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72c8070c10 0x7f72c81160f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T23:31:50.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:50.415+0000 7f72c77fe700 1 --2- 172.21.15.107:0/244611135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72c8070300 0x7f72c8115bb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f72b8004f50 tx=0x7f72b8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:31:50.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:50.415+0000 7f72affff700 1 -- 172.21.15.107:0/244611135 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72b800e040 con 0x7f72c8070300 2024-04-22T23:31:50.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:50.415+0000 7f72affff700 1 -- 172.21.15.107:0/244611135 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f72b8004290 con 0x7f72c8070300 2024-04-22T23:31:50.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:50.415+0000 7f72ce1ea700 1 -- 172.21.15.107:0/244611135 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72c810ff40 con 0x7f72c8070300 2024-04-22T23:31:50.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:50.415+0000 7f72affff700 1 -- 172.21.15.107:0/244611135 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72b801c3f0 con 0x7f72c8070300 2024-04-22T23:31:50.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:50.415+0000 7f72ce1ea700 1 -- 172.21.15.107:0/244611135 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72c8110480 con 0x7f72c8070300 2024-04-22T23:31:50.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:50.416+0000 7f72ce1ea700 1 -- 172.21.15.107:0/244611135 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72c8071a40 con 0x7f72c8070300 2024-04-22T23:31:50.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:50.416+0000 7f72affff700 1 -- 172.21.15.107:0/244611135 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f72b800bc10 con 0x7f72c8070300 2024-04-22T23:31:50.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:50.417+0000 7f72affff700 1 --2- 172.21.15.107:0/244611135 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f72b00692a0 0x7f72b006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:50.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:50.417+0000 7f72affff700 1 -- 172.21.15.107:0/244611135 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f72b8088120 con 0x7f72c8070300 2024-04-22T23:31:50.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:50.417+0000 7f72c6ffd700 1 --2- 172.21.15.107:0/244611135 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f72b00692a0 0x7f72b006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:50.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:50.417+0000 7f72c6ffd700 1 --2- 172.21.15.107:0/244611135 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f72b00692a0 0x7f72b006b750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f72c8111450 tx=0x7f72bc008040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:31:50.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:50.421+0000 7f72affff700 1 -- 172.21.15.107:0/244611135 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f72b80582d0 con 0x7f72c8070300 2024-04-22T23:31:50.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:50.610+0000 7f72ce1ea700 1 -- 172.21.15.107:0/244611135 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f72c804c710 con 0x7f72c8070300 2024-04-22T23:31:50.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:50.610+0000 7f72affff700 1 -- 172.21.15.107:0/244611135 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f72b8057e60 con 0x7f72c8070300 2024-04-22T23:31:50.607 INFO:teuthology.orchestra.run.smithi107.stdout:38654705701 2024-04-22T23:31:50.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:50.612+0000 7f72ce1ea700 1 -- 172.21.15.107:0/244611135 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f72b00692a0 msgr2=0x7f72b006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:50.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:50.612+0000 7f72ce1ea700 1 --2- 172.21.15.107:0/244611135 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f72b00692a0 0x7f72b006b750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f72c8111450 tx=0x7f72bc008040 comp rx=0 tx=0).stop 2024-04-22T23:31:50.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:50.612+0000 7f72ce1ea700 1 -- 172.21.15.107:0/244611135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72c8070300 msgr2=0x7f72c8115bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:50.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:50.612+0000 7f72ce1ea700 1 --2- 172.21.15.107:0/244611135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72c8070300 0x7f72c8115bb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f72b8004f50 tx=0x7f72b8009e30 comp rx=0 tx=0).stop 2024-04-22T23:31:50.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:50.612+0000 7f72ce1ea700 1 -- 172.21.15.107:0/244611135 shutdown_connections 2024-04-22T23:31:50.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:50.612+0000 7f72ce1ea700 1 --2- 172.21.15.107:0/244611135 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f72b00692a0 0x7f72b006b750 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:50.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:50.612+0000 7f72ce1ea700 1 --2- 172.21.15.107:0/244611135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72c8070300 0x7f72c8115bb0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:50.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:50.612+0000 7f72ce1ea700 1 --2- 172.21.15.107:0/244611135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72c8070c10 0x7f72c81160f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:50.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:50.612+0000 7f72ce1ea700 1 -- 172.21.15.107:0/244611135 >> 172.21.15.107:0/244611135 conn(0x7f72c80f89b0 msgr2=0x7f72c80f9dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:31:50.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:50.613+0000 7f72ce1ea700 1 -- 172.21.15.107:0/244611135 shutdown_connections 2024-04-22T23:31:50.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:50.613+0000 7f72ce1ea700 1 -- 172.21.15.107:0/244611135 wait complete. 2024-04-22T23:31:51.356 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705698 got 38654705701 for osd.0 2024-04-22T23:31:51.356 DEBUG:teuthology.parallel:result is None 2024-04-22T23:31:51.356 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-22T23:31:51.356 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-22T23:31:51.612 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:31:51.723 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:51 smithi107 ceph-mon[26355]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:31:51.723 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:51 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/244611135' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-22T23:31:51.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:51 smithi157 ceph-mon[33877]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:31:51.884 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:51 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/244611135' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-22T23:31:53.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:53.191+0000 7f0746ebe700 1 -- 172.21.15.107:0/3186789664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0740102f60 msgr2=0x7f0740103330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:53.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:53.191+0000 7f0746ebe700 1 --2- 172.21.15.107:0/3186789664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0740102f60 0x7f0740103330 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0734004940 tx=0x7f0734009e30 comp rx=0 tx=0).stop 2024-04-22T23:31:53.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:53.191+0000 7f0746ebe700 1 -- 172.21.15.107:0/3186789664 shutdown_connections 2024-04-22T23:31:53.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:53.191+0000 7f0746ebe700 1 --2- 172.21.15.107:0/3186789664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07400fb4d0 0x7f07400fb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:53.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:53.191+0000 7f0746ebe700 1 --2- 172.21.15.107:0/3186789664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0740102f60 0x7f0740103330 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:53.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:53.191+0000 7f0746ebe700 1 -- 172.21.15.107:0/3186789664 >> 172.21.15.107:0/3186789664 conn(0x7f07400714c0 msgr2=0x7f07400718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:31:53.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:53.191+0000 7f0746ebe700 1 -- 172.21.15.107:0/3186789664 shutdown_connections 2024-04-22T23:31:53.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:53.191+0000 7f0746ebe700 1 -- 172.21.15.107:0/3186789664 wait complete. 2024-04-22T23:31:53.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:53.191+0000 7f0746ebe700 1 Processor -- start 2024-04-22T23:31:53.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:53.192+0000 7f0746ebe700 1 -- start start 2024-04-22T23:31:53.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:53.192+0000 7f0746ebe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07400fb4d0 0x7f0740113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:53.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:53.192+0000 7f0746ebe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0740102f60 0x7f0740113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:53.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:53.192+0000 7f0746ebe700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0740114520 con 0x7f0740102f60 2024-04-22T23:31:53.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:53.192+0000 7f0746ebe700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f074010db20 con 0x7f07400fb4d0 2024-04-22T23:31:53.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:53.192+0000 7f073ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0740102f60 0x7f0740113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:53.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:53.192+0000 7f073ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0740102f60 0x7f0740113ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33582/0 (socket says 172.21.15.107:33582) 2024-04-22T23:31:53.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:53.192+0000 7f073ffff700 1 -- 172.21.15.107:0/1170989392 learned_addr learned my addr 172.21.15.107:0/1170989392 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:31:53.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:53.192+0000 7f0744c5a700 1 --2- 172.21.15.107:0/1170989392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07400fb4d0 0x7f0740113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:53.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:53.192+0000 7f073ffff700 1 -- 172.21.15.107:0/1170989392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07400fb4d0 msgr2=0x7f0740113990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:53.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:53.192+0000 7f073ffff700 1 --2- 172.21.15.107:0/1170989392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07400fb4d0 0x7f0740113990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:53.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:53.192+0000 7f073ffff700 1 -- 172.21.15.107:0/1170989392 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0734005040 con 0x7f0740102f60 2024-04-22T23:31:53.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:53.192+0000 7f0744c5a700 1 --2- 172.21.15.107:0/1170989392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07400fb4d0 0x7f0740113990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T23:31:53.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:53.192+0000 7f073ffff700 1 --2- 172.21.15.107:0/1170989392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0740102f60 0x7f0740113ed0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f072c002a50 tx=0x7f072c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:31:53.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:53.193+0000 7f073d7fa700 1 -- 172.21.15.107:0/1170989392 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f072c007cc0 con 0x7f0740102f60 2024-04-22T23:31:53.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:53.193+0000 7f0746ebe700 1 -- 172.21.15.107:0/1170989392 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f074010de00 con 0x7f0740102f60 2024-04-22T23:31:53.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:53.193+0000 7f073d7fa700 1 -- 172.21.15.107:0/1170989392 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f072c00f040 con 0x7f0740102f60 2024-04-22T23:31:53.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:53.193+0000 7f073d7fa700 1 -- 172.21.15.107:0/1170989392 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f072c009640 con 0x7f0740102f60 2024-04-22T23:31:53.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:53.193+0000 7f0746ebe700 1 -- 172.21.15.107:0/1170989392 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f074010e3c0 con 0x7f0740102f60 2024-04-22T23:31:53.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:53.194+0000 7f0746ebe700 1 -- 172.21.15.107:0/1170989392 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0740100200 con 0x7f0740102f60 2024-04-22T23:31:53.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:53.194+0000 7f073d7fa700 1 -- 172.21.15.107:0/1170989392 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f072c0097a0 con 0x7f0740102f60 2024-04-22T23:31:53.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:53.194+0000 7f073d7fa700 1 --2- 172.21.15.107:0/1170989392 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f07300692a0 0x7f073006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:53.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:53.194+0000 7f073d7fa700 1 -- 172.21.15.107:0/1170989392 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f072c086b80 con 0x7f0740102f60 2024-04-22T23:31:53.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:53.195+0000 7f0744c5a700 1 --2- 172.21.15.107:0/1170989392 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f07300692a0 0x7f073006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:53.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:53.195+0000 7f0744c5a700 1 --2- 172.21.15.107:0/1170989392 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f07300692a0 0x7f073006b750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0734004560 tx=0x7f0734009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:31:53.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:53.199+0000 7f073d7fa700 1 -- 172.21.15.107:0/1170989392 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f072c056c80 con 0x7f0740102f60 2024-04-22T23:31:53.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:53.382+0000 7f0746ebe700 1 -- 172.21.15.107:0/1170989392 --> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0740113660 con 0x7f07300692a0 2024-04-22T23:31:53.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:53.384+0000 7f073d7fa700 1 -- 172.21.15.107:0/1170989392 <== mgr.14206 v2:172.21.15.107:6800/2516490111 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+28414 (secure 0 0 0) 0x7f0740113660 con 0x7f07300692a0 2024-04-22T23:31:53.381 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:31:53.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:53.385+0000 7f0746ebe700 1 -- 172.21.15.107:0/1170989392 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f07300692a0 msgr2=0x7f073006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:53.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:53.385+0000 7f0746ebe700 1 --2- 172.21.15.107:0/1170989392 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f07300692a0 0x7f073006b750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0734004560 tx=0x7f0734009e30 comp rx=0 tx=0).stop 2024-04-22T23:31:53.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:53.386+0000 7f0746ebe700 1 -- 172.21.15.107:0/1170989392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0740102f60 msgr2=0x7f0740113ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:53.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:53.386+0000 7f0746ebe700 1 --2- 172.21.15.107:0/1170989392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0740102f60 0x7f0740113ed0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f072c002a50 tx=0x7f072c0046b0 comp rx=0 tx=0).stop 2024-04-22T23:31:53.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:53.386+0000 7f0746ebe700 1 -- 172.21.15.107:0/1170989392 shutdown_connections 2024-04-22T23:31:53.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:53.386+0000 7f0746ebe700 1 --2- 172.21.15.107:0/1170989392 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f07300692a0 0x7f073006b750 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:53.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:53.386+0000 7f0746ebe700 1 --2- 172.21.15.107:0/1170989392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07400fb4d0 0x7f0740113990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:53.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:53.386+0000 7f0746ebe700 1 --2- 172.21.15.107:0/1170989392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0740102f60 0x7f0740113ed0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:53.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:53.386+0000 7f0746ebe700 1 -- 172.21.15.107:0/1170989392 >> 172.21.15.107:0/1170989392 conn(0x7f07400714c0 msgr2=0x7f07400fa650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:31:53.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:53.386+0000 7f0746ebe700 1 -- 172.21.15.107:0/1170989392 shutdown_connections 2024-04-22T23:31:53.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:53.386+0000 7f0746ebe700 1 -- 172.21.15.107:0/1170989392 wait complete. 2024-04-22T23:31:53.383 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-04-22T23:31:53.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:53 smithi107 ceph-mon[26355]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:31:53.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:53 smithi157 ceph-mon[33877]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:31:54.096 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":157,"stamp":"2024-04-22T23:31:51.981107+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":1788028,"kb_used_data":3876,"kb_used_omap":9,"kb_used_meta":1784054,"kb_avail":560682884,"statfs":{"total":575970213888,"available":574139273216,"internally_reserved":0,"allocated":3969024,"data_stored":2606262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10224,"internal_metadata":1826871312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001915"},"pg_stats":[{"pgid":"1.0","version":"22'117","reported_seq":190,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-22T23:31:03.174159+0000","last_change":"2024-04-22T23:30:37.116017+0000","last_active":"2024-04-22T23:31:03.174159+0000","last_peered":"2024-04-22T23:31:03.174159+0000","last_clean":"2024-04-22T23:31:03.174159+0000","last_became_active":"2024-04-22T23:30:37.115554+0000","last_became_peered":"2024-04-22T23:30:37.115554+0000","last_unstale":"2024-04-22T23:31:03.174159+0000","last_undegraded":"2024-04-22T23:31:03.174159+0000","last_fullsized":"2024-04-22T23:31:03.174159+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T23:29:48.966204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T23:29:48.966204+0000","last_clean_scrub_stamp":"2024-04-22T23:29:48.966204+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:48:10.246540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724557,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297712,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447440,"statfs":{"total":95995035648,"available":95690178560,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]}]},{"osd":4,"up_from":34,"seq":146028888081,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297712,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447440,"statfs":{"total":95995035648,"available":95690178560,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 23:31:38 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.51100000000000001}]},{"osd":1,"last update":"Mon Apr 22 23:31:38 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.52400000000000002}]},{"osd":2,"last update":"Mon Apr 22 23:31:38 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.55400000000000005}]},{"osd":3,"last update":"Mon Apr 22 23:31:38 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.49099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]}]},{"osd":3,"up_from":26,"seq":111669149719,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298300,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446852,"statfs":{"total":95995035648,"available":95689576448,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 23:31:15 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.38500000000000001}]},{"osd":1,"last update":"Mon Apr 22 23:31:15 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.42199999999999999}]},{"osd":2,"last update":"Mon Apr 22 23:31:15 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.442}]},{"osd":4,"last update":"Mon Apr 22 23:31:38 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.52100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]}]},{"osd":2,"up_from":19,"seq":81604378652,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297712,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447440,"statfs":{"total":95995035648,"available":95690178560,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 23:30:49 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"last":0.30199999999999999}]},{"osd":1,"last update":"Mon Apr 22 23:30:49 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.32600000000000001}]},{"osd":3,"last update":"Mon Apr 22 23:31:11 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.61599999999999999}]},{"osd":4,"last update":"Mon Apr 22 23:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.55000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]}]},{"osd":0,"up_from":9,"seq":38654705702,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298296,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446856,"statfs":{"total":95995035648,"available":95689580544,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Apr 22 23:31:24 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.33000000000000002,"15min":0.32500000000000001},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.315},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.34599999999999997,"15min":0.34300000000000003},"min":{"1min":0.23699999999999999,"5min":0.221,"15min":0.221},"max":{"1min":0.495,"5min":0.495,"15min":0.495},"last":0.26600000000000001}]},{"osd":2,"last update":"Mon Apr 22 23:30:50 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.28100000000000003}]},{"osd":3,"last update":"Mon Apr 22 23:31:12 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.495}]},{"osd":4,"last update":"Mon Apr 22 23:31:38 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.45000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]}]},{"osd":1,"up_from":14,"seq":60129542178,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298296,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446856,"statfs":{"total":95995035648,"available":95689580544,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 23:31:27 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.44700000000000001,"5min":0.496,"15min":0.496},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.247,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.49299999999999999,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.28599999999999998}]},{"osd":2,"last update":"Mon Apr 22 23:30:50 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.24099999999999999},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"last":0.30499999999999999}]},{"osd":3,"last update":"Mon Apr 22 23:31:11 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.48799999999999999}]},{"osd":4,"last update":"Mon Apr 22 23:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.47199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T23:31:54.097 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-22T23:31:54.351 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:31:54.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:54 smithi107 ceph-mon[26355]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T23:31:54.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:54 smithi157 ceph-mon[33877]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T23:31:55.669 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:55 smithi107 ceph-mon[26355]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:31:55.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:55 smithi157 ceph-mon[33877]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:31:55.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:55.938+0000 7fd899337700 1 -- 172.21.15.107:0/3639029384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8940fde90 msgr2=0x7fd8940fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:55.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:55.938+0000 7fd899337700 1 --2- 172.21.15.107:0/3639029384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8940fde90 0x7fd8940fe260 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd884004940 tx=0x7fd884009e30 comp rx=0 tx=0).stop 2024-04-22T23:31:55.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:55.938+0000 7fd899337700 1 -- 172.21.15.107:0/3639029384 shutdown_connections 2024-04-22T23:31:55.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:55.938+0000 7fd899337700 1 --2- 172.21.15.107:0/3639029384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8940fe830 0x7fd894106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:55.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:55.938+0000 7fd899337700 1 --2- 172.21.15.107:0/3639029384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8940fde90 0x7fd8940fe260 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:55.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:55.938+0000 7fd899337700 1 -- 172.21.15.107:0/3639029384 >> 172.21.15.107:0/3639029384 conn(0x7fd8940f7900 msgr2=0x7fd8940f9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:31:55.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:55.938+0000 7fd899337700 1 -- 172.21.15.107:0/3639029384 shutdown_connections 2024-04-22T23:31:55.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:55.938+0000 7fd899337700 1 -- 172.21.15.107:0/3639029384 wait complete. 2024-04-22T23:31:55.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:55.939+0000 7fd899337700 1 Processor -- start 2024-04-22T23:31:55.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:55.939+0000 7fd899337700 1 -- start start 2024-04-22T23:31:55.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:55.939+0000 7fd899337700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8940fde90 0x7fd894113960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:55.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:55.939+0000 7fd899337700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8940fe830 0x7fd894113ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:55.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:55.939+0000 7fd899337700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8941144f0 con 0x7fd8940fe830 2024-04-22T23:31:55.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:55.939+0000 7fd899337700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd89410daf0 con 0x7fd8940fde90 2024-04-22T23:31:55.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:55.939+0000 7fd8927fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8940fe830 0x7fd894113ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:55.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:55.939+0000 7fd8927fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8940fe830 0x7fd894113ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33590/0 (socket says 172.21.15.107:33590) 2024-04-22T23:31:55.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:55.939+0000 7fd8927fc700 1 -- 172.21.15.107:0/4096748814 learned_addr learned my addr 172.21.15.107:0/4096748814 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:31:55.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:55.939+0000 7fd892ffd700 1 --2- 172.21.15.107:0/4096748814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8940fde90 0x7fd894113960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:55.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:55.940+0000 7fd8927fc700 1 -- 172.21.15.107:0/4096748814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8940fde90 msgr2=0x7fd894113960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:55.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:55.940+0000 7fd8927fc700 1 --2- 172.21.15.107:0/4096748814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8940fde90 0x7fd894113960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:55.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:55.940+0000 7fd8927fc700 1 -- 172.21.15.107:0/4096748814 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd884005040 con 0x7fd8940fe830 2024-04-22T23:31:55.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:55.940+0000 7fd8927fc700 1 --2- 172.21.15.107:0/4096748814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8940fe830 0x7fd894113ea0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd88800c8c0 tx=0x7fd8880044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:31:55.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:55.940+0000 7fd87b7fe700 1 -- 172.21.15.107:0/4096748814 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd88800a760 con 0x7fd8940fe830 2024-04-22T23:31:55.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:55.940+0000 7fd87b7fe700 1 -- 172.21.15.107:0/4096748814 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd88800ad90 con 0x7fd8940fe830 2024-04-22T23:31:55.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:55.940+0000 7fd899337700 1 -- 172.21.15.107:0/4096748814 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd89410ddd0 con 0x7fd8940fe830 2024-04-22T23:31:55.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:55.940+0000 7fd87b7fe700 1 -- 172.21.15.107:0/4096748814 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd8880113a0 con 0x7fd8940fe830 2024-04-22T23:31:55.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:55.940+0000 7fd899337700 1 -- 172.21.15.107:0/4096748814 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd89410e310 con 0x7fd8940fe830 2024-04-22T23:31:55.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:55.941+0000 7fd899337700 1 -- 172.21.15.107:0/4096748814 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd894066b60 con 0x7fd8940fe830 2024-04-22T23:31:55.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:55.941+0000 7fd87b7fe700 1 -- 172.21.15.107:0/4096748814 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fd888012400 con 0x7fd8940fe830 2024-04-22T23:31:55.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:55.942+0000 7fd87b7fe700 1 --2- 172.21.15.107:0/4096748814 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fd87c0692a0 0x7fd87c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:55.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:55.942+0000 7fd87b7fe700 1 -- 172.21.15.107:0/4096748814 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fd888086e70 con 0x7fd8940fe830 2024-04-22T23:31:55.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:55.942+0000 7fd892ffd700 1 --2- 172.21.15.107:0/4096748814 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fd87c0692a0 0x7fd87c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:55.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:55.942+0000 7fd892ffd700 1 --2- 172.21.15.107:0/4096748814 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fd87c0692a0 0x7fd87c06b750 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd884004f90 tx=0x7fd884009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:31:55.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:55.946+0000 7fd87b7fe700 1 -- 172.21.15.107:0/4096748814 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd888056f70 con 0x7fd8940fe830 2024-04-22T23:31:56.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:56.130+0000 7fd899337700 1 -- 172.21.15.107:0/4096748814 --> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd894113630 con 0x7fd87c0692a0 2024-04-22T23:31:56.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:56.131+0000 7fd87b7fe700 1 -- 172.21.15.107:0/4096748814 <== mgr.14206 v2:172.21.15.107:6800/2516490111 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+28399 (secure 0 0 0) 0x7fd894113630 con 0x7fd87c0692a0 2024-04-22T23:31:56.131 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:31:56.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:56.134+0000 7fd899337700 1 -- 172.21.15.107:0/4096748814 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fd87c0692a0 msgr2=0x7fd87c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:56.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:56.134+0000 7fd899337700 1 --2- 172.21.15.107:0/4096748814 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fd87c0692a0 0x7fd87c06b750 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd884004f90 tx=0x7fd884009e30 comp rx=0 tx=0).stop 2024-04-22T23:31:56.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:56.134+0000 7fd899337700 1 -- 172.21.15.107:0/4096748814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8940fe830 msgr2=0x7fd894113ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:56.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:56.134+0000 7fd899337700 1 --2- 172.21.15.107:0/4096748814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8940fe830 0x7fd894113ea0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd88800c8c0 tx=0x7fd8880044c0 comp rx=0 tx=0).stop 2024-04-22T23:31:56.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:56.134+0000 7fd899337700 1 -- 172.21.15.107:0/4096748814 shutdown_connections 2024-04-22T23:31:56.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:56.134+0000 7fd899337700 1 --2- 172.21.15.107:0/4096748814 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fd87c0692a0 0x7fd87c06b750 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:56.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:56.134+0000 7fd899337700 1 --2- 172.21.15.107:0/4096748814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8940fde90 0x7fd894113960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:56.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:56.134+0000 7fd899337700 1 --2- 172.21.15.107:0/4096748814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8940fe830 0x7fd894113ea0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:56.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:56.134+0000 7fd899337700 1 -- 172.21.15.107:0/4096748814 >> 172.21.15.107:0/4096748814 conn(0x7fd8940f7900 msgr2=0x7fd8941036a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:31:56.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:56.134+0000 7fd899337700 1 -- 172.21.15.107:0/4096748814 shutdown_connections 2024-04-22T23:31:56.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:56.134+0000 7fd899337700 1 -- 172.21.15.107:0/4096748814 wait complete. 2024-04-22T23:31:56.132 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-04-22T23:31:56.874 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":159,"stamp":"2024-04-22T23:31:55.981727+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":1788028,"kb_used_data":3876,"kb_used_omap":9,"kb_used_meta":1784054,"kb_avail":560682884,"statfs":{"total":575970213888,"available":574139273216,"internally_reserved":0,"allocated":3969024,"data_stored":2606262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10224,"internal_metadata":1826871312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001834"},"pg_stats":[{"pgid":"1.0","version":"22'117","reported_seq":190,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-22T23:31:03.174159+0000","last_change":"2024-04-22T23:30:37.116017+0000","last_active":"2024-04-22T23:31:03.174159+0000","last_peered":"2024-04-22T23:31:03.174159+0000","last_clean":"2024-04-22T23:31:03.174159+0000","last_became_active":"2024-04-22T23:30:37.115554+0000","last_became_peered":"2024-04-22T23:30:37.115554+0000","last_unstale":"2024-04-22T23:31:03.174159+0000","last_undegraded":"2024-04-22T23:31:03.174159+0000","last_fullsized":"2024-04-22T23:31:03.174159+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T23:29:48.966204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T23:29:48.966204+0000","last_clean_scrub_stamp":"2024-04-22T23:29:48.966204+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:48:10.246540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724558,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297712,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447440,"statfs":{"total":95995035648,"available":95690178560,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]}]},{"osd":4,"up_from":34,"seq":146028888082,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297712,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447440,"statfs":{"total":95995035648,"available":95690178560,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 23:31:38 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.54100000000000004}]},{"osd":1,"last update":"Mon Apr 22 23:31:38 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.52500000000000002}]},{"osd":2,"last update":"Mon Apr 22 23:31:38 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.499}]},{"osd":3,"last update":"Mon Apr 22 23:31:38 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.35399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33800000000000002}]}]},{"osd":3,"up_from":26,"seq":111669149719,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298300,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446852,"statfs":{"total":95995035648,"available":95689576448,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 23:31:15 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.38500000000000001}]},{"osd":1,"last update":"Mon Apr 22 23:31:15 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.42199999999999999}]},{"osd":2,"last update":"Mon Apr 22 23:31:15 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.442}]},{"osd":4,"last update":"Mon Apr 22 23:31:38 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.52100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]}]},{"osd":2,"up_from":19,"seq":81604378653,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297712,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447440,"statfs":{"total":95995035648,"available":95690178560,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 23:31:52 2024","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.35199999999999998,"15min":0.35799999999999998},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.36799999999999999,"15min":0.374},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.22700000000000001}]},{"osd":1,"last update":"Mon Apr 22 23:31:52 2024","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.376,"15min":0.38},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.251},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.36899999999999999,"15min":0.375},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.48799999999999999,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.28299999999999997}]},{"osd":3,"last update":"Mon Apr 22 23:31:11 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.63400000000000001}]},{"osd":4,"last update":"Mon Apr 22 23:31:37 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.54300000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]}]},{"osd":0,"up_from":9,"seq":38654705702,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298296,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446856,"statfs":{"total":95995035648,"available":95689580544,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Apr 22 23:31:24 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.33000000000000002,"15min":0.32500000000000001},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.315},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.34599999999999997,"15min":0.34300000000000003},"min":{"1min":0.23699999999999999,"5min":0.221,"15min":0.221},"max":{"1min":0.495,"5min":0.495,"15min":0.495},"last":0.26600000000000001}]},{"osd":2,"last update":"Mon Apr 22 23:30:50 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.28100000000000003}]},{"osd":3,"last update":"Mon Apr 22 23:31:12 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.495}]},{"osd":4,"last update":"Mon Apr 22 23:31:38 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.45000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]}]},{"osd":1,"up_from":14,"seq":60129542178,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298296,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446856,"statfs":{"total":95995035648,"available":95689580544,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Apr 22 23:31:27 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.44700000000000001,"5min":0.496,"15min":0.496},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"min":{"1min":0.247,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.49299999999999999,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.28599999999999998}]},{"osd":2,"last update":"Mon Apr 22 23:30:50 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.24099999999999999},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"last":0.30499999999999999}]},{"osd":3,"last update":"Mon Apr 22 23:31:11 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.48799999999999999}]},{"osd":4,"last update":"Mon Apr 22 23:31:41 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.47199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T23:31:56.875 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-22T23:31:56.875 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-22T23:31:56.875 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-22T23:31:56.875 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-22T23:31:57.126 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:31:57.569 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:57 smithi107 ceph-mon[26355]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:31:57.570 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:57 smithi107 ceph-mon[26355]: from='client.14416 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T23:31:57.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:57 smithi157 ceph-mon[33877]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:31:57.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:57 smithi157 ceph-mon[33877]: from='client.14416 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T23:31:58.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:58.692+0000 7f1552e4e700 1 -- 172.21.15.107:0/2532540240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f154c1026b0 msgr2=0x7f154c0fbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:58.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:58.692+0000 7f1552e4e700 1 --2- 172.21.15.107:0/2532540240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f154c1026b0 0x7f154c0fbd20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f153c004940 tx=0x7f153c009e30 comp rx=0 tx=0).stop 2024-04-22T23:31:58.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:58.692+0000 7f1552e4e700 1 -- 172.21.15.107:0/2532540240 shutdown_connections 2024-04-22T23:31:58.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:58.692+0000 7f1552e4e700 1 --2- 172.21.15.107:0/2532540240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f154c1026b0 0x7f154c0fbd20 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:58.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:58.692+0000 7f1552e4e700 1 --2- 172.21.15.107:0/2532540240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f154c101d10 0x7f154c1020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:58.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:58.692+0000 7f1552e4e700 1 -- 172.21.15.107:0/2532540240 >> 172.21.15.107:0/2532540240 conn(0x7f154c072bc0 msgr2=0x7f154c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:31:58.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:58.692+0000 7f1552e4e700 1 -- 172.21.15.107:0/2532540240 shutdown_connections 2024-04-22T23:31:58.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:58.692+0000 7f1552e4e700 1 -- 172.21.15.107:0/2532540240 wait complete. 2024-04-22T23:31:58.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:58.693+0000 7f1552e4e700 1 Processor -- start 2024-04-22T23:31:58.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:58.693+0000 7f1552e4e700 1 -- start start 2024-04-22T23:31:58.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:58.693+0000 7f1552e4e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f154c101d10 0x7f154c112ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:58.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:58.693+0000 7f1552e4e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f154c1026b0 0x7f154c112ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:58.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:58.693+0000 7f1552e4e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f154c113640 con 0x7f154c101d10 2024-04-22T23:31:58.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:58.693+0000 7f1552e4e700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f154c10ca30 con 0x7f154c1026b0 2024-04-22T23:31:58.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:58.693+0000 7f1550bea700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f154c101d10 0x7f154c112ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:58.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:58.693+0000 7f1550bea700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f154c101d10 0x7f154c112ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33610/0 (socket says 172.21.15.107:33610) 2024-04-22T23:31:58.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:58.693+0000 7f1550bea700 1 -- 172.21.15.107:0/1466746243 learned_addr learned my addr 172.21.15.107:0/1466746243 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:31:58.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:58.693+0000 7f154bfff700 1 --2- 172.21.15.107:0/1466746243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f154c1026b0 0x7f154c112ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:58.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:58.693+0000 7f1550bea700 1 -- 172.21.15.107:0/1466746243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f154c1026b0 msgr2=0x7f154c112ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:58.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:58.693+0000 7f1550bea700 1 --2- 172.21.15.107:0/1466746243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f154c1026b0 0x7f154c112ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:58.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:58.693+0000 7f1550bea700 1 -- 172.21.15.107:0/1466746243 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f153c005040 con 0x7f154c101d10 2024-04-22T23:31:58.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:58.694+0000 7f1550bea700 1 --2- 172.21.15.107:0/1466746243 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f154c101d10 0x7f154c112ab0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f154400cb10 tx=0x7f1544007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:31:58.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:58.694+0000 7f15497fa700 1 -- 172.21.15.107:0/1466746243 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1544004200 con 0x7f154c101d10 2024-04-22T23:31:58.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:58.694+0000 7f15497fa700 1 -- 172.21.15.107:0/1466746243 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f154400ac70 con 0x7f154c101d10 2024-04-22T23:31:58.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:58.694+0000 7f15497fa700 1 -- 172.21.15.107:0/1466746243 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1544003d30 con 0x7f154c101d10 2024-04-22T23:31:58.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:58.694+0000 7f1552e4e700 1 -- 172.21.15.107:0/1466746243 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f154c10cd10 con 0x7f154c101d10 2024-04-22T23:31:58.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:58.694+0000 7f1552e4e700 1 -- 172.21.15.107:0/1466746243 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f154c10d2a0 con 0x7f154c101d10 2024-04-22T23:31:58.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:58.695+0000 7f1552e4e700 1 -- 172.21.15.107:0/1466746243 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f154c0fd460 con 0x7f154c101d10 2024-04-22T23:31:58.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:58.695+0000 7f15497fa700 1 -- 172.21.15.107:0/1466746243 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f154400a7a0 con 0x7f154c101d10 2024-04-22T23:31:58.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:58.696+0000 7f15497fa700 1 --2- 172.21.15.107:0/1466746243 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f15380692a0 0x7f153806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:31:58.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:58.696+0000 7f154bfff700 1 --2- 172.21.15.107:0/1466746243 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f15380692a0 0x7f153806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:31:58.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:58.696+0000 7f15497fa700 1 -- 172.21.15.107:0/1466746243 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f1544087330 con 0x7f154c101d10 2024-04-22T23:31:58.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:58.696+0000 7f154bfff700 1 --2- 172.21.15.107:0/1466746243 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f15380692a0 0x7f153806b750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f153c004890 tx=0x7f153c009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:31:58.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:58.700+0000 7f15497fa700 1 -- 172.21.15.107:0/1466746243 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f15440573b0 con 0x7f154c101d10 2024-04-22T23:31:58.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:58.925+0000 7f1552e4e700 1 -- 172.21.15.107:0/1466746243 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f154c108e40 con 0x7f154c101d10 2024-04-22T23:31:58.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:58.926+0000 7f15497fa700 1 -- 172.21.15.107:0/1466746243 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f1544004360 con 0x7f154c101d10 2024-04-22T23:31:58.922 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:31:58.922 INFO:teuthology.orchestra.run.smithi107.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-22T23:31:58.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:58.927+0000 7f1552e4e700 1 -- 172.21.15.107:0/1466746243 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f15380692a0 msgr2=0x7f153806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:58.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:58.927+0000 7f1552e4e700 1 --2- 172.21.15.107:0/1466746243 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f15380692a0 0x7f153806b750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f153c004890 tx=0x7f153c009e30 comp rx=0 tx=0).stop 2024-04-22T23:31:58.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:58.927+0000 7f1552e4e700 1 -- 172.21.15.107:0/1466746243 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f154c101d10 msgr2=0x7f154c112ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:31:58.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:58.927+0000 7f1552e4e700 1 --2- 172.21.15.107:0/1466746243 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f154c101d10 0x7f154c112ab0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f154400cb10 tx=0x7f1544007f90 comp rx=0 tx=0).stop 2024-04-22T23:31:58.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:58.928+0000 7f1552e4e700 1 -- 172.21.15.107:0/1466746243 shutdown_connections 2024-04-22T23:31:58.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:58.928+0000 7f1552e4e700 1 --2- 172.21.15.107:0/1466746243 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f15380692a0 0x7f153806b750 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:58.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:58.928+0000 7f1552e4e700 1 --2- 172.21.15.107:0/1466746243 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f154c101d10 0x7f154c112ab0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:58.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:58.928+0000 7f1552e4e700 1 --2- 172.21.15.107:0/1466746243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f154c1026b0 0x7f154c112ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:31:58.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:58.928+0000 7f1552e4e700 1 -- 172.21.15.107:0/1466746243 >> 172.21.15.107:0/1466746243 conn(0x7f154c072bc0 msgr2=0x7f154c0ff8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:31:58.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:58.928+0000 7f1552e4e700 1 -- 172.21.15.107:0/1466746243 shutdown_connections 2024-04-22T23:31:58.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:31:58.928+0000 7f1552e4e700 1 -- 172.21.15.107:0/1466746243 wait complete. 2024-04-22T23:31:59.592 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:59 smithi107 ceph-mon[26355]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:31:59.592 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:31:59 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/1466746243' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-22T23:31:59.618 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-22T23:31:59.618 INFO:tasks.cephadm:Setup complete, yielding 2024-04-22T23:31:59.619 INFO:teuthology.run_tasks:Running task print... 2024-04-22T23:31:59.629 INFO:teuthology.task.print:**** done end installing quincy cephadm ... 2024-04-22T23:31:59.629 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-22T23:31:59.655 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi107.front.sepia.ceph.com 2024-04-22T23:31:59.656 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-04-22T23:31:59.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:59 smithi157 ceph-mon[33877]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:31:59.884 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:31:59 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/1466746243' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-22T23:31:59.906 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:32:01.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:01.675+0000 7f6de6dcd700 1 -- 172.21.15.107:0/248847630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6de00fcff0 msgr2=0x7f6de00fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:01.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:01.675+0000 7f6de6dcd700 1 --2- 172.21.15.107:0/248847630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6de00fcff0 0x7f6de00fd460 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc004940 tx=0x7f6dcc009e30 comp rx=0 tx=0).stop 2024-04-22T23:32:01.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:01.675+0000 7f6de6dcd700 1 -- 172.21.15.107:0/248847630 shutdown_connections 2024-04-22T23:32:01.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:01.675+0000 7f6de6dcd700 1 --2- 172.21.15.107:0/248847630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6de00fcff0 0x7f6de00fd460 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:01.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:01.675+0000 7f6de6dcd700 1 --2- 172.21.15.107:0/248847630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6de00fbd40 0x7f6de00fc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:01.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:01.675+0000 7f6de6dcd700 1 -- 172.21.15.107:0/248847630 >> 172.21.15.107:0/248847630 conn(0x7f6de00f74c0 msgr2=0x7f6de00f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:32:01.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:01.675+0000 7f6de6dcd700 1 -- 172.21.15.107:0/248847630 shutdown_connections 2024-04-22T23:32:01.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:01.675+0000 7f6de6dcd700 1 -- 172.21.15.107:0/248847630 wait complete. 2024-04-22T23:32:01.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:01.675+0000 7f6de6dcd700 1 Processor -- start 2024-04-22T23:32:01.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:01.676+0000 7f6de6dcd700 1 -- start start 2024-04-22T23:32:01.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:01.676+0000 7f6de6dcd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6de00fbd40 0x7f6de00ffe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:01.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:01.676+0000 7f6de6dcd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6de00fcff0 0x7f6de00fe330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:01.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:01.676+0000 7f6de6dcd700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de00fe8c0 con 0x7f6de00fcff0 2024-04-22T23:32:01.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:01.676+0000 7f6de6dcd700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de00fea00 con 0x7f6de00fbd40 2024-04-22T23:32:01.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:01.676+0000 7f6ddffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6de00fcff0 0x7f6de00fe330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:01.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:01.676+0000 7f6ddffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6de00fcff0 0x7f6de00fe330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55026/0 (socket says 172.21.15.107:55026) 2024-04-22T23:32:01.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:01.676+0000 7f6ddffff700 1 -- 172.21.15.107:0/2431831481 learned_addr learned my addr 172.21.15.107:0/2431831481 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:32:01.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:01.676+0000 7f6de4b69700 1 --2- 172.21.15.107:0/2431831481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6de00fbd40 0x7f6de00ffe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:01.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:01.676+0000 7f6ddffff700 1 -- 172.21.15.107:0/2431831481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6de00fbd40 msgr2=0x7f6de00ffe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:01.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:01.676+0000 7f6ddffff700 1 --2- 172.21.15.107:0/2431831481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6de00fbd40 0x7f6de00ffe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:01.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:01.676+0000 7f6ddffff700 1 -- 172.21.15.107:0/2431831481 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dcc005040 con 0x7f6de00fcff0 2024-04-22T23:32:01.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:01.676+0000 7f6ddffff700 1 --2- 172.21.15.107:0/2431831481 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6de00fcff0 0x7f6de00fe330 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc00b3b0 tx=0x7f6dcc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:32:01.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:01.677+0000 7f6ddd7fa700 1 -- 172.21.15.107:0/2431831481 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6dcc00e040 con 0x7f6de00fcff0 2024-04-22T23:32:01.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:01.677+0000 7f6ddd7fa700 1 -- 172.21.15.107:0/2431831481 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6dcc00bbf0 con 0x7f6de00fcff0 2024-04-22T23:32:01.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:01.677+0000 7f6de6dcd700 1 -- 172.21.15.107:0/2431831481 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6de00fec80 con 0x7f6de00fcff0 2024-04-22T23:32:01.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:01.677+0000 7f6ddd7fa700 1 -- 172.21.15.107:0/2431831481 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6dcc01c630 con 0x7f6de00fcff0 2024-04-22T23:32:01.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:01.677+0000 7f6de6dcd700 1 -- 172.21.15.107:0/2431831481 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6de0115ab0 con 0x7f6de00fcff0 2024-04-22T23:32:01.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:01.678+0000 7f6de6dcd700 1 -- 172.21.15.107:0/2431831481 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6de0101b80 con 0x7f6de00fcff0 2024-04-22T23:32:01.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:01.678+0000 7f6ddd7fa700 1 -- 172.21.15.107:0/2431831481 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f6dcc00bd60 con 0x7f6de00fcff0 2024-04-22T23:32:01.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:01.678+0000 7f6ddd7fa700 1 --2- 172.21.15.107:0/2431831481 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f6dd00692a0 0x7f6dd006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:01.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:01.678+0000 7f6ddd7fa700 1 -- 172.21.15.107:0/2431831481 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f6dcc087b10 con 0x7f6de00fcff0 2024-04-22T23:32:01.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:01.678+0000 7f6de4b69700 1 --2- 172.21.15.107:0/2431831481 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f6dd00692a0 0x7f6dd006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:01.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:01.679+0000 7f6de4b69700 1 --2- 172.21.15.107:0/2431831481 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f6dd00692a0 0x7f6dd006b750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6de00ffba0 tx=0x7f6dd400b040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:32:01.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:01.683+0000 7f6ddd7fa700 1 -- 172.21.15.107:0/2431831481 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f6dcc054180 con 0x7f6de00fcff0 2024-04-22T23:32:01.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:01 smithi107 ceph-mon[26355]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:32:01.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:01.866+0000 7f6de6dcd700 1 -- 172.21.15.107:0/2431831481 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) v1 -- 0x7f6de0104890 con 0x7f6de00fcff0 2024-04-22T23:32:01.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:01.871+0000 7f6ddd7fa700 1 -- 172.21.15.107:0/2431831481 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v20)=0 v20) v1 ==== 143+0+0 (secure 0 0 0) 0x7f6dcc0577a0 con 0x7f6de00fcff0 2024-04-22T23:32:01.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:01.873+0000 7f6de6dcd700 1 -- 172.21.15.107:0/2431831481 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f6dd00692a0 msgr2=0x7f6dd006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:01.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:01.873+0000 7f6de6dcd700 1 --2- 172.21.15.107:0/2431831481 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f6dd00692a0 0x7f6dd006b750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6de00ffba0 tx=0x7f6dd400b040 comp rx=0 tx=0).stop 2024-04-22T23:32:01.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:01.873+0000 7f6de6dcd700 1 -- 172.21.15.107:0/2431831481 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6de00fcff0 msgr2=0x7f6de00fe330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:01.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:01.873+0000 7f6de6dcd700 1 --2- 172.21.15.107:0/2431831481 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6de00fcff0 0x7f6de00fe330 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc00b3b0 tx=0x7f6dcc009e30 comp rx=0 tx=0).stop 2024-04-22T23:32:01.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:01.873+0000 7f6de6dcd700 1 -- 172.21.15.107:0/2431831481 shutdown_connections 2024-04-22T23:32:01.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:01.873+0000 7f6de6dcd700 1 --2- 172.21.15.107:0/2431831481 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f6dd00692a0 0x7f6dd006b750 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:01.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:01.873+0000 7f6de6dcd700 1 --2- 172.21.15.107:0/2431831481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6de00fbd40 0x7f6de00ffe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:01.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:01.873+0000 7f6de6dcd700 1 --2- 172.21.15.107:0/2431831481 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6de00fcff0 0x7f6de00fe330 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:01.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:01.873+0000 7f6de6dcd700 1 -- 172.21.15.107:0/2431831481 >> 172.21.15.107:0/2431831481 conn(0x7f6de00f74c0 msgr2=0x7f6de0065c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:32:01.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:01.873+0000 7f6de6dcd700 1 -- 172.21.15.107:0/2431831481 shutdown_connections 2024-04-22T23:32:01.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:01.874+0000 7f6de6dcd700 1 -- 172.21.15.107:0/2431831481 wait complete. 2024-04-22T23:32:01.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:01 smithi157 ceph-mon[33877]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:32:02.611 INFO:teuthology.run_tasks:Running task print... 2024-04-22T23:32:02.620 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-04-22T23:32:02.620 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-22T23:32:02.663 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi107.front.sepia.ceph.com 2024-04-22T23:32:02.664 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-22T23:32:02.916 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:32:03.133 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:02 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/2431831481' entity='client.admin' 2024-04-22T23:32:03.134 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:02 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:32:03.134 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:02 smithi157 ceph-mon[33877]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:32:03.134 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:02 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:32:03.134 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:02 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:32:03.134 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:02 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:32:03.144 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:02 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/2431831481' entity='client.admin' 2024-04-22T23:32:03.144 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:02 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:32:03.144 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:02 smithi107 ceph-mon[26355]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:32:03.144 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:02 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:32:03.144 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:02 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:32:03.144 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:02 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:32:04.298 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:04 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:32:04.383 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:04 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:32:04.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:04.591+0000 7f49cace2700 1 -- 172.21.15.107:0/764723913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49c40fcfa0 msgr2=0x7f49c40fd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:04.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:04.591+0000 7f49cace2700 1 --2- 172.21.15.107:0/764723913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49c40fcfa0 0x7f49c40fd410 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f49b0004940 tx=0x7f49b0009e30 comp rx=0 tx=0).stop 2024-04-22T23:32:04.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:04.591+0000 7f49cace2700 1 -- 172.21.15.107:0/764723913 shutdown_connections 2024-04-22T23:32:04.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:04.591+0000 7f49cace2700 1 --2- 172.21.15.107:0/764723913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49c40fcfa0 0x7f49c40fd410 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:04.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:04.591+0000 7f49cace2700 1 --2- 172.21.15.107:0/764723913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49c40fbcf0 0x7f49c40fc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:04.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:04.591+0000 7f49cace2700 1 -- 172.21.15.107:0/764723913 >> 172.21.15.107:0/764723913 conn(0x7f49c40f7450 msgr2=0x7f49c40f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:32:04.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:04.591+0000 7f49cace2700 1 -- 172.21.15.107:0/764723913 shutdown_connections 2024-04-22T23:32:04.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:04.591+0000 7f49cace2700 1 -- 172.21.15.107:0/764723913 wait complete. 2024-04-22T23:32:04.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:04.591+0000 7f49cace2700 1 Processor -- start 2024-04-22T23:32:04.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:04.592+0000 7f49cace2700 1 -- start start 2024-04-22T23:32:04.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:04.592+0000 7f49cace2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49c40fbcf0 0x7f49c40ffc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:04.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:04.592+0000 7f49cace2700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49c40fcfa0 0x7f49c40fe2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:04.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:04.592+0000 7f49cace2700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c40fe820 con 0x7f49c40fbcf0 2024-04-22T23:32:04.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:04.592+0000 7f49cace2700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c40fe990 con 0x7f49c40fcfa0 2024-04-22T23:32:04.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:04.592+0000 7f49c8a7e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49c40fbcf0 0x7f49c40ffc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:04.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:04.592+0000 7f49c8a7e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49c40fbcf0 0x7f49c40ffc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55040/0 (socket says 172.21.15.107:55040) 2024-04-22T23:32:04.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:04.592+0000 7f49c8a7e700 1 -- 172.21.15.107:0/1872957206 learned_addr learned my addr 172.21.15.107:0/1872957206 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:32:04.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:04.592+0000 7f49c3fff700 1 --2- 172.21.15.107:0/1872957206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49c40fcfa0 0x7f49c40fe2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:04.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:04.592+0000 7f49c8a7e700 1 -- 172.21.15.107:0/1872957206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49c40fcfa0 msgr2=0x7f49c40fe2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:04.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:04.592+0000 7f49c8a7e700 1 --2- 172.21.15.107:0/1872957206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49c40fcfa0 0x7f49c40fe2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:04.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:04.592+0000 7f49c8a7e700 1 -- 172.21.15.107:0/1872957206 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b0005040 con 0x7f49c40fbcf0 2024-04-22T23:32:04.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:04.593+0000 7f49c8a7e700 1 --2- 172.21.15.107:0/1872957206 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49c40fbcf0 0x7f49c40ffc30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f49b800b790 tx=0x7f49b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:32:04.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:04.593+0000 7f49c17fa700 1 -- 172.21.15.107:0/1872957206 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49b800f6f0 con 0x7f49c40fbcf0 2024-04-22T23:32:04.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:04.593+0000 7f49cace2700 1 -- 172.21.15.107:0/1872957206 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49c40fec70 con 0x7f49c40fbcf0 2024-04-22T23:32:04.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:04.593+0000 7f49cace2700 1 -- 172.21.15.107:0/1872957206 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49c4115d90 con 0x7f49c40fbcf0 2024-04-22T23:32:04.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:04.593+0000 7f49c17fa700 1 -- 172.21.15.107:0/1872957206 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f49b800fd20 con 0x7f49c40fbcf0 2024-04-22T23:32:04.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:04.593+0000 7f49c17fa700 1 -- 172.21.15.107:0/1872957206 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49b8011400 con 0x7f49c40fbcf0 2024-04-22T23:32:04.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:04.594+0000 7f49cace2700 1 -- 172.21.15.107:0/1872957206 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49a80052e0 con 0x7f49c40fbcf0 2024-04-22T23:32:04.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:04.594+0000 7f49c17fa700 1 -- 172.21.15.107:0/1872957206 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f49b801a400 con 0x7f49c40fbcf0 2024-04-22T23:32:04.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:04.595+0000 7f49c17fa700 1 --2- 172.21.15.107:0/1872957206 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f49b40692a0 0x7f49b406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:04.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:04.595+0000 7f49c17fa700 1 -- 172.21.15.107:0/1872957206 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f49b8086b30 con 0x7f49c40fbcf0 2024-04-22T23:32:04.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:04.595+0000 7f49c3fff700 1 --2- 172.21.15.107:0/1872957206 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f49b40692a0 0x7f49b406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:04.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:04.595+0000 7f49c3fff700 1 --2- 172.21.15.107:0/1872957206 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f49b40692a0 0x7f49b406b750 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f49b000b970 tx=0x7f49b0009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:32:04.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:04.599+0000 7f49c17fa700 1 -- 172.21.15.107:0/1872957206 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f49b8056cd0 con 0x7f49c40fbcf0 2024-04-22T23:32:04.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:04.798+0000 7f49cace2700 1 -- 172.21.15.107:0/1872957206 --> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f49a8000bf0 con 0x7f49b40692a0 2024-04-22T23:32:04.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:04.799+0000 7f49c17fa700 1 -- 172.21.15.107:0/1872957206 <== mgr.14206 v2:172.21.15.107:6800/2516490111 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f49a8000bf0 con 0x7f49b40692a0 2024-04-22T23:32:04.795 INFO:teuthology.orchestra.run.smithi107.stdout:Backend: cephadm 2024-04-22T23:32:04.796 INFO:teuthology.orchestra.run.smithi107.stdout:Available: Yes 2024-04-22T23:32:04.796 INFO:teuthology.orchestra.run.smithi107.stdout:Paused: No 2024-04-22T23:32:04.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:04.801+0000 7f49cace2700 1 -- 172.21.15.107:0/1872957206 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f49b40692a0 msgr2=0x7f49b406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:04.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:04.801+0000 7f49cace2700 1 --2- 172.21.15.107:0/1872957206 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f49b40692a0 0x7f49b406b750 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f49b000b970 tx=0x7f49b0009e30 comp rx=0 tx=0).stop 2024-04-22T23:32:04.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:04.801+0000 7f49cace2700 1 -- 172.21.15.107:0/1872957206 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49c40fbcf0 msgr2=0x7f49c40ffc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:04.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:04.801+0000 7f49cace2700 1 --2- 172.21.15.107:0/1872957206 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49c40fbcf0 0x7f49c40ffc30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f49b800b790 tx=0x7f49b8009250 comp rx=0 tx=0).stop 2024-04-22T23:32:04.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:04.801+0000 7f49cace2700 1 -- 172.21.15.107:0/1872957206 shutdown_connections 2024-04-22T23:32:04.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:04.801+0000 7f49cace2700 1 --2- 172.21.15.107:0/1872957206 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f49b40692a0 0x7f49b406b750 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:04.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:04.801+0000 7f49cace2700 1 --2- 172.21.15.107:0/1872957206 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49c40fbcf0 0x7f49c40ffc30 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:04.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:04.801+0000 7f49cace2700 1 --2- 172.21.15.107:0/1872957206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f49c40fcfa0 0x7f49c40fe2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:04.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:04.801+0000 7f49cace2700 1 -- 172.21.15.107:0/1872957206 >> 172.21.15.107:0/1872957206 conn(0x7f49c40f7450 msgr2=0x7f49c4066ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:32:04.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:04.801+0000 7f49cace2700 1 -- 172.21.15.107:0/1872957206 shutdown_connections 2024-04-22T23:32:04.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:04.801+0000 7f49cace2700 1 -- 172.21.15.107:0/1872957206 wait complete. 2024-04-22T23:32:05.383 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:05 smithi157 ceph-mon[33877]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:32:05.433 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:05 smithi107 ceph-mon[26355]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:32:05.459 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-22T23:32:05.721 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:32:06.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:06 smithi157 ceph-mon[33877]: from='client.14428 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:32:06.485 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:06 smithi107 ceph-mon[26355]: from='client.14428 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:32:07.155 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:07 smithi107 ceph-mon[26355]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:32:07.383 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:07 smithi157 ceph-mon[33877]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:32:07.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:07.419+0000 7fee4cd4b700 1 -- 172.21.15.107:0/3135979042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee480feda0 msgr2=0x7fee480faee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:07.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:07.419+0000 7fee4cd4b700 1 --2- 172.21.15.107:0/3135979042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee480feda0 0x7fee480faee0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fee3c004940 tx=0x7fee3c009e30 comp rx=0 tx=0).stop 2024-04-22T23:32:07.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:07.419+0000 7fee4cd4b700 1 -- 172.21.15.107:0/3135979042 shutdown_connections 2024-04-22T23:32:07.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:07.419+0000 7fee4cd4b700 1 --2- 172.21.15.107:0/3135979042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee480feda0 0x7fee480faee0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:07.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:07.419+0000 7fee4cd4b700 1 --2- 172.21.15.107:0/3135979042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee480fe450 0x7fee480fe860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:07.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:07.419+0000 7fee4cd4b700 1 -- 172.21.15.107:0/3135979042 >> 172.21.15.107:0/3135979042 conn(0x7fee480f6bf0 msgr2=0x7fee480f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:32:07.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:07.419+0000 7fee4cd4b700 1 -- 172.21.15.107:0/3135979042 shutdown_connections 2024-04-22T23:32:07.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:07.419+0000 7fee4cd4b700 1 -- 172.21.15.107:0/3135979042 wait complete. 2024-04-22T23:32:07.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:07.419+0000 7fee4cd4b700 1 Processor -- start 2024-04-22T23:32:07.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:07.420+0000 7fee4cd4b700 1 -- start start 2024-04-22T23:32:07.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:07.420+0000 7fee4cd4b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee480fe450 0x7fee4810d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:07.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:07.420+0000 7fee4cd4b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee480feda0 0x7fee4810de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:07.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:07.420+0000 7fee4cd4b700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4810e460 con 0x7fee480fe450 2024-04-22T23:32:07.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:07.420+0000 7fee4cd4b700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4810e5d0 con 0x7fee480feda0 2024-04-22T23:32:07.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:07.420+0000 7fee4659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee480fe450 0x7fee4810d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:07.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:07.420+0000 7fee4659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee480fe450 0x7fee4810d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55058/0 (socket says 172.21.15.107:55058) 2024-04-22T23:32:07.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:07.420+0000 7fee4659c700 1 -- 172.21.15.107:0/4141124540 learned_addr learned my addr 172.21.15.107:0/4141124540 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:32:07.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:07.420+0000 7fee45d9b700 1 --2- 172.21.15.107:0/4141124540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee480feda0 0x7fee4810de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:07.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:07.420+0000 7fee4659c700 1 -- 172.21.15.107:0/4141124540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee480feda0 msgr2=0x7fee4810de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:07.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:07.420+0000 7fee4659c700 1 --2- 172.21.15.107:0/4141124540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee480feda0 0x7fee4810de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:07.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:07.420+0000 7fee4659c700 1 -- 172.21.15.107:0/4141124540 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee3c005040 con 0x7fee480fe450 2024-04-22T23:32:07.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:07.420+0000 7fee4659c700 1 --2- 172.21.15.107:0/4141124540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee480fe450 0x7fee4810d920 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fee38002a50 tx=0x7fee380046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:32:07.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:07.421+0000 7fee36ffd700 1 -- 172.21.15.107:0/4141124540 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee38007cc0 con 0x7fee480fe450 2024-04-22T23:32:07.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:07.421+0000 7fee36ffd700 1 -- 172.21.15.107:0/4141124540 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fee3800f040 con 0x7fee480fe450 2024-04-22T23:32:07.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:07.421+0000 7fee4cd4b700 1 -- 172.21.15.107:0/4141124540 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee48112fd0 con 0x7fee480fe450 2024-04-22T23:32:07.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:07.421+0000 7fee36ffd700 1 -- 172.21.15.107:0/4141124540 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee3800da30 con 0x7fee480fe450 2024-04-22T23:32:07.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:07.421+0000 7fee4cd4b700 1 -- 172.21.15.107:0/4141124540 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee48113560 con 0x7fee480fe450 2024-04-22T23:32:07.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:07.422+0000 7fee4cd4b700 1 -- 172.21.15.107:0/4141124540 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee4804c710 con 0x7fee480fe450 2024-04-22T23:32:07.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:07.422+0000 7fee36ffd700 1 -- 172.21.15.107:0/4141124540 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fee3800db90 con 0x7fee480fe450 2024-04-22T23:32:07.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:07.422+0000 7fee36ffd700 1 --2- 172.21.15.107:0/4141124540 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fee300692a0 0x7fee3006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:07.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:07.423+0000 7fee36ffd700 1 -- 172.21.15.107:0/4141124540 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fee38086c10 con 0x7fee480fe450 2024-04-22T23:32:07.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:07.423+0000 7fee45d9b700 1 --2- 172.21.15.107:0/4141124540 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fee300692a0 0x7fee3006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:07.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:07.423+0000 7fee45d9b700 1 --2- 172.21.15.107:0/4141124540 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fee300692a0 0x7fee3006b750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fee3c00b970 tx=0x7fee3c009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:32:07.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:07.427+0000 7fee36ffd700 1 -- 172.21.15.107:0/4141124540 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fee38056c90 con 0x7fee480fe450 2024-04-22T23:32:07.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:07.621+0000 7fee4cd4b700 1 -- 172.21.15.107:0/4141124540 --> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fee48113750 con 0x7fee300692a0 2024-04-22T23:32:07.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:07.629+0000 7fee36ffd700 1 -- 172.21.15.107:0/4141124540 <== mgr.14206 v2:172.21.15.107:6800/2516490111 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2682 (secure 0 0 0) 0x7fee48113750 con 0x7fee300692a0 2024-04-22T23:32:07.627 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-22T23:32:07.627 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (3m) 2m ago 5m 26.7M - 0.25.0 c8568f914cd2 4d90ceb89249 2024-04-22T23:32:07.627 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (5m) 2m ago 5m 6660k - 17.2.7-884-g70d0cb4b cafd8ddac8fa c10611c756e4 2024-04-22T23:32:07.627 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi157 smithi157 running (4m) 65s ago 4m 6669k - 17.2.7-884-g70d0cb4b cafd8ddac8fa 55d2d46294c9 2024-04-22T23:32:07.628 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (3m) 2m ago 4m 80.1M - 9.4.7 954c08fa6188 c2d1f8de1aa7 2024-04-22T23:32:07.628 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.uipxvm smithi107 *:9283 running (6m) 2m ago 6m 470M - 17.2.7-884-g70d0cb4b cafd8ddac8fa db96e8cbbdfb 2024-04-22T23:32:07.628 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi157.tuylis smithi157 *:8443,9283 running (4m) 65s ago 4m 431M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 0c12be10ffaf 2024-04-22T23:32:07.628 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (6m) 2m ago 6m 43.6M 2048M 17.2.7-884-g70d0cb4b cafd8ddac8fa 8314962fa1b9 2024-04-22T23:32:07.628 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi157 smithi157 running (4m) 65s ago 4m 38.8M 2048M 17.2.7-884-g70d0cb4b cafd8ddac8fa 0747a85008d9 2024-04-22T23:32:07.628 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (5m) 2m ago 5m 15.6M - 1.5.0 0da6a335fe13 8a7c595c114f 2024-04-22T23:32:07.628 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi157 smithi157 *:9100 running (4m) 65s ago 4m 16.4M - 1.5.0 0da6a335fe13 940164771fe3 2024-04-22T23:32:07.628 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (3m) 2m ago 3m 36.3M 4288M 17.2.7-884-g70d0cb4b cafd8ddac8fa 18a179b0a618 2024-04-22T23:32:07.628 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (2m) 2m ago 2m 35.6M 4288M 17.2.7-884-g70d0cb4b cafd8ddac8fa c7272eb84ddb 2024-04-22T23:32:07.628 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (2m) 2m ago 2m 13.1M 4288M 17.2.7-884-g70d0cb4b cafd8ddac8fa 9db1788096ad 2024-04-22T23:32:07.628 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi157 running (2m) 65s ago 2m 39.1M 5312M 17.2.7-884-g70d0cb4b cafd8ddac8fa c9fb2d1a13dd 2024-04-22T23:32:07.628 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi157 running (97s) 65s ago 96s 39.0M 5312M 17.2.7-884-g70d0cb4b cafd8ddac8fa 24a16e19951f 2024-04-22T23:32:07.628 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi157 running (71s) 65s ago 70s 10.9M 5312M 17.2.7-884-g70d0cb4b cafd8ddac8fa 4c5b7a7ef8bd 2024-04-22T23:32:07.628 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (3m) 2m ago 4m 35.8M - 2.43.0 a07b618ecd1d 8c5df40caabe 2024-04-22T23:32:07.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:07.632+0000 7fee4cd4b700 1 -- 172.21.15.107:0/4141124540 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fee300692a0 msgr2=0x7fee3006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:07.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:07.632+0000 7fee4cd4b700 1 --2- 172.21.15.107:0/4141124540 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fee300692a0 0x7fee3006b750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fee3c00b970 tx=0x7fee3c009e30 comp rx=0 tx=0).stop 2024-04-22T23:32:07.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:07.632+0000 7fee4cd4b700 1 -- 172.21.15.107:0/4141124540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee480fe450 msgr2=0x7fee4810d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:07.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:07.632+0000 7fee4cd4b700 1 --2- 172.21.15.107:0/4141124540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee480fe450 0x7fee4810d920 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fee38002a50 tx=0x7fee380046b0 comp rx=0 tx=0).stop 2024-04-22T23:32:07.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:07.632+0000 7fee4cd4b700 1 -- 172.21.15.107:0/4141124540 shutdown_connections 2024-04-22T23:32:07.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:07.632+0000 7fee4cd4b700 1 --2- 172.21.15.107:0/4141124540 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fee300692a0 0x7fee3006b750 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:07.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:07.632+0000 7fee4cd4b700 1 --2- 172.21.15.107:0/4141124540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee480fe450 0x7fee4810d920 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:07.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:07.632+0000 7fee4cd4b700 1 --2- 172.21.15.107:0/4141124540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee480feda0 0x7fee4810de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:07.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:07.632+0000 7fee4cd4b700 1 -- 172.21.15.107:0/4141124540 >> 172.21.15.107:0/4141124540 conn(0x7fee480f6bf0 msgr2=0x7fee480f82c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:32:07.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:07.632+0000 7fee4cd4b700 1 -- 172.21.15.107:0/4141124540 shutdown_connections 2024-04-22T23:32:07.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:07.632+0000 7fee4cd4b700 1 -- 172.21.15.107:0/4141124540 wait complete. 2024-04-22T23:32:08.292 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-22T23:32:08.547 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:32:09.235 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:09 smithi107 ceph-mon[26355]: from='client.14432 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:32:09.235 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:09 smithi107 ceph-mon[26355]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:32:09.384 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:09 smithi157 ceph-mon[33877]: from='client.14432 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:32:09.384 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:09 smithi157 ceph-mon[33877]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:32:10.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:10.143+0000 7ff50b9e9700 1 -- 172.21.15.107:0/1136425838 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5040fffc0 msgr2=0x7ff5041023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:10.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:10.143+0000 7ff50b9e9700 1 --2- 172.21.15.107:0/1136425838 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5040fffc0 0x7ff5041023a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4004940 tx=0x7ff4f4009e30 comp rx=0 tx=0).stop 2024-04-22T23:32:10.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:10.143+0000 7ff50b9e9700 1 -- 172.21.15.107:0/1136425838 shutdown_connections 2024-04-22T23:32:10.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:10.143+0000 7ff50b9e9700 1 --2- 172.21.15.107:0/1136425838 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5040fffc0 0x7ff5041023a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:10.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:10.143+0000 7ff50b9e9700 1 --2- 172.21.15.107:0/1136425838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff504066f00 0x7ff5040ffa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:10.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:10.143+0000 7ff50b9e9700 1 -- 172.21.15.107:0/1136425838 >> 172.21.15.107:0/1136425838 conn(0x7ff5040f7490 msgr2=0x7ff5040f98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:32:10.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:10.143+0000 7ff50b9e9700 1 -- 172.21.15.107:0/1136425838 shutdown_connections 2024-04-22T23:32:10.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:10.143+0000 7ff50b9e9700 1 -- 172.21.15.107:0/1136425838 wait complete. 2024-04-22T23:32:10.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:10.144+0000 7ff50b9e9700 1 Processor -- start 2024-04-22T23:32:10.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:10.144+0000 7ff50b9e9700 1 -- start start 2024-04-22T23:32:10.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:10.144+0000 7ff50b9e9700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff504066f00 0x7ff50410dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:10.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:10.144+0000 7ff50b9e9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5040fffc0 0x7ff50410e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:10.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:10.144+0000 7ff50b9e9700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff50410e900 con 0x7ff5040fffc0 2024-04-22T23:32:10.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:10.144+0000 7ff50b9e9700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff504113210 con 0x7ff504066f00 2024-04-22T23:32:10.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:10.144+0000 7ff508f84700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5040fffc0 0x7ff50410e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:10.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:10.144+0000 7ff508f84700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5040fffc0 0x7ff50410e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55082/0 (socket says 172.21.15.107:55082) 2024-04-22T23:32:10.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:10.144+0000 7ff508f84700 1 -- 172.21.15.107:0/1395609793 learned_addr learned my addr 172.21.15.107:0/1395609793 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:32:10.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:10.144+0000 7ff509785700 1 --2- 172.21.15.107:0/1395609793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff504066f00 0x7ff50410dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:10.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:10.144+0000 7ff508f84700 1 -- 172.21.15.107:0/1395609793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff504066f00 msgr2=0x7ff50410dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:10.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:10.144+0000 7ff508f84700 1 --2- 172.21.15.107:0/1395609793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff504066f00 0x7ff50410dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:10.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:10.144+0000 7ff508f84700 1 -- 172.21.15.107:0/1395609793 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f4005040 con 0x7ff5040fffc0 2024-04-22T23:32:10.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:10.144+0000 7ff508f84700 1 --2- 172.21.15.107:0/1395609793 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5040fffc0 0x7ff50410e1f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff4f400b3b0 tx=0x7ff4f4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:32:10.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:10.145+0000 7ff4f9ffb700 1 -- 172.21.15.107:0/1395609793 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff4f400e040 con 0x7ff5040fffc0 2024-04-22T23:32:10.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:10.145+0000 7ff4f9ffb700 1 -- 172.21.15.107:0/1395609793 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff4f400bbf0 con 0x7ff5040fffc0 2024-04-22T23:32:10.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:10.145+0000 7ff50b9e9700 1 -- 172.21.15.107:0/1395609793 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff504113410 con 0x7ff5040fffc0 2024-04-22T23:32:10.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:10.145+0000 7ff4f9ffb700 1 -- 172.21.15.107:0/1395609793 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff4f401c630 con 0x7ff5040fffc0 2024-04-22T23:32:10.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:10.145+0000 7ff50b9e9700 1 -- 172.21.15.107:0/1395609793 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5041139a0 con 0x7ff5040fffc0 2024-04-22T23:32:10.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:10.146+0000 7ff50b9e9700 1 -- 172.21.15.107:0/1395609793 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5041061c0 con 0x7ff5040fffc0 2024-04-22T23:32:10.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:10.146+0000 7ff4f9ffb700 1 -- 172.21.15.107:0/1395609793 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7ff4f400bd60 con 0x7ff5040fffc0 2024-04-22T23:32:10.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:10.147+0000 7ff4f9ffb700 1 --2- 172.21.15.107:0/1395609793 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7ff4f00692a0 0x7ff4f006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:10.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:10.147+0000 7ff4f9ffb700 1 -- 172.21.15.107:0/1395609793 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7ff4f4087a40 con 0x7ff5040fffc0 2024-04-22T23:32:10.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:10.147+0000 7ff509785700 1 --2- 172.21.15.107:0/1395609793 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7ff4f00692a0 0x7ff4f006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:10.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:10.147+0000 7ff509785700 1 --2- 172.21.15.107:0/1395609793 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7ff4f00692a0 0x7ff4f006b750 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff5040feac0 tx=0x7ff5000091a0 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:32:10.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:10.151+0000 7ff4f9ffb700 1 -- 172.21.15.107:0/1395609793 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ff4f4054150 con 0x7ff5040fffc0 2024-04-22T23:32:10.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:10.345+0000 7ff50b9e9700 1 -- 172.21.15.107:0/1395609793 --> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ff504105b60 con 0x7ff4f00692a0 2024-04-22T23:32:10.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:10.349+0000 7ff4f9ffb700 1 -- 172.21.15.107:0/1395609793 <== mgr.14206 v2:172.21.15.107:6800/2516490111 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1179 (secure 0 0 0) 0x7ff50000e6d0 con 0x7ff4f00692a0 2024-04-22T23:32:10.347 INFO:teuthology.orchestra.run.smithi107.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-22T23:32:10.347 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager ?:9093,9094 1/1 2m ago 5m count:1 2024-04-22T23:32:10.347 INFO:teuthology.orchestra.run.smithi107.stdout:crash 2/2 2m ago 5m * 2024-04-22T23:32:10.347 INFO:teuthology.orchestra.run.smithi107.stdout:grafana ?:3000 1/1 2m ago 5m count:1 2024-04-22T23:32:10.347 INFO:teuthology.orchestra.run.smithi107.stdout:mgr 2/2 2m ago 5m count:2 2024-04-22T23:32:10.347 INFO:teuthology.orchestra.run.smithi107.stdout:mon 2/2 2m ago 5m smithi107:172.21.15.107=smithi107;smithi157:172.21.15.157=smithi157;count:2 2024-04-22T23:32:10.347 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter ?:9100 2/2 2m ago 5m * 2024-04-22T23:32:10.347 INFO:teuthology.orchestra.run.smithi107.stdout:osd 6 2m ago - 2024-04-22T23:32:10.347 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus ?:9095 1/1 2m ago 5m count:1 2024-04-22T23:32:10.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:10.351+0000 7ff50b9e9700 1 -- 172.21.15.107:0/1395609793 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7ff4f00692a0 msgr2=0x7ff4f006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:10.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:10.351+0000 7ff50b9e9700 1 --2- 172.21.15.107:0/1395609793 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7ff4f00692a0 0x7ff4f006b750 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff5040feac0 tx=0x7ff5000091a0 comp rx=0 tx=0).stop 2024-04-22T23:32:10.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:10.351+0000 7ff50b9e9700 1 -- 172.21.15.107:0/1395609793 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5040fffc0 msgr2=0x7ff50410e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:10.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:10.351+0000 7ff50b9e9700 1 --2- 172.21.15.107:0/1395609793 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5040fffc0 0x7ff50410e1f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff4f400b3b0 tx=0x7ff4f4009e30 comp rx=0 tx=0).stop 2024-04-22T23:32:10.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:10.351+0000 7ff50b9e9700 1 -- 172.21.15.107:0/1395609793 shutdown_connections 2024-04-22T23:32:10.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:10.351+0000 7ff50b9e9700 1 --2- 172.21.15.107:0/1395609793 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7ff4f00692a0 0x7ff4f006b750 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:10.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:10.351+0000 7ff50b9e9700 1 --2- 172.21.15.107:0/1395609793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff504066f00 0x7ff50410dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:10.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:10.351+0000 7ff50b9e9700 1 --2- 172.21.15.107:0/1395609793 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5040fffc0 0x7ff50410e1f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:10.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:10.351+0000 7ff50b9e9700 1 -- 172.21.15.107:0/1395609793 >> 172.21.15.107:0/1395609793 conn(0x7ff5040f7490 msgr2=0x7ff5040f78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:32:10.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:10.352+0000 7ff50b9e9700 1 -- 172.21.15.107:0/1395609793 shutdown_connections 2024-04-22T23:32:10.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:10.352+0000 7ff50b9e9700 1 -- 172.21.15.107:0/1395609793 wait complete. 2024-04-22T23:32:11.057 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-22T23:32:11.319 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:32:11.383 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:11 smithi157 ceph-mon[33877]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:32:11.383 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:11 smithi157 ceph-mon[33877]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:32:11.443 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:11 smithi107 ceph-mon[26355]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:32:11.443 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:11 smithi107 ceph-mon[26355]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:32:12.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:12.988+0000 7f830072e700 1 -- 172.21.15.107:0/3329839190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82f80fce40 msgr2=0x7f82f80fd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:12.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:12.988+0000 7f830072e700 1 --2- 172.21.15.107:0/3329839190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82f80fce40 0x7f82f80fd250 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f82f4004940 tx=0x7f82f4009e30 comp rx=0 tx=0).stop 2024-04-22T23:32:12.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:12.988+0000 7f830072e700 1 -- 172.21.15.107:0/3329839190 shutdown_connections 2024-04-22T23:32:12.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:12.988+0000 7f830072e700 1 --2- 172.21.15.107:0/3329839190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82f80fe040 0x7f82f80fe4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:12.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:12.988+0000 7f830072e700 1 --2- 172.21.15.107:0/3329839190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82f80fce40 0x7f82f80fd250 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:12.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:12.988+0000 7f830072e700 1 -- 172.21.15.107:0/3329839190 >> 172.21.15.107:0/3329839190 conn(0x7f82f80f8570 msgr2=0x7f82f80fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:32:12.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:12.988+0000 7f830072e700 1 -- 172.21.15.107:0/3329839190 shutdown_connections 2024-04-22T23:32:12.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:12.988+0000 7f830072e700 1 -- 172.21.15.107:0/3329839190 wait complete. 2024-04-22T23:32:12.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:12.989+0000 7f830072e700 1 Processor -- start 2024-04-22T23:32:12.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:12.989+0000 7f830072e700 1 -- start start 2024-04-22T23:32:12.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:12.989+0000 7f830072e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82f80fce40 0x7f82f810da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:12.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:12.989+0000 7f830072e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82f80fe040 0x7f82f810dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:12.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:12.989+0000 7f830072e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f810e5a0 con 0x7f82f80fe040 2024-04-22T23:32:12.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:12.989+0000 7f830072e700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f810edf0 con 0x7f82f80fce40 2024-04-22T23:32:12.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:12.989+0000 7f82fdcc9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82f80fe040 0x7f82f810dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:12.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:12.989+0000 7f82fdcc9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82f80fe040 0x7f82f810dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40200/0 (socket says 172.21.15.107:40200) 2024-04-22T23:32:12.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:12.989+0000 7f82fdcc9700 1 -- 172.21.15.107:0/3663478095 learned_addr learned my addr 172.21.15.107:0/3663478095 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:32:12.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:12.989+0000 7f82fe4ca700 1 --2- 172.21.15.107:0/3663478095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82f80fce40 0x7f82f810da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:12.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:12.989+0000 7f82fdcc9700 1 -- 172.21.15.107:0/3663478095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82f80fce40 msgr2=0x7f82f810da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:12.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:12.989+0000 7f82fdcc9700 1 --2- 172.21.15.107:0/3663478095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82f80fce40 0x7f82f810da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:12.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:12.989+0000 7f82fdcc9700 1 -- 172.21.15.107:0/3663478095 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f4005040 con 0x7f82f80fe040 2024-04-22T23:32:12.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:12.989+0000 7f82fe4ca700 1 --2- 172.21.15.107:0/3663478095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82f80fce40 0x7f82f810da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T23:32:12.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:12.990+0000 7f82fdcc9700 1 --2- 172.21.15.107:0/3663478095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82f80fe040 0x7f82f810dfa0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f82e800c8c0 tx=0x7f82e80044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:32:12.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:12.990+0000 7f82eeffd700 1 -- 172.21.15.107:0/3663478095 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82e800a980 con 0x7f82f80fe040 2024-04-22T23:32:12.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:12.990+0000 7f82eeffd700 1 -- 172.21.15.107:0/3663478095 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f82e800ce60 con 0x7f82f80fe040 2024-04-22T23:32:12.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:12.990+0000 7f830072e700 1 -- 172.21.15.107:0/3663478095 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82f810f0d0 con 0x7f82f80fe040 2024-04-22T23:32:12.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:12.990+0000 7f82eeffd700 1 -- 172.21.15.107:0/3663478095 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82e80115e0 con 0x7f82f80fe040 2024-04-22T23:32:12.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:12.990+0000 7f830072e700 1 -- 172.21.15.107:0/3663478095 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82f810f610 con 0x7f82f80fe040 2024-04-22T23:32:12.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:12.991+0000 7f830072e700 1 -- 172.21.15.107:0/3663478095 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82f805f5f0 con 0x7f82f80fe040 2024-04-22T23:32:12.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:12.991+0000 7f82eeffd700 1 -- 172.21.15.107:0/3663478095 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f82e8012400 con 0x7f82f80fe040 2024-04-22T23:32:12.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:12.991+0000 7f82eeffd700 1 --2- 172.21.15.107:0/3663478095 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f82e40692a0 0x7f82e406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:12.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:12.992+0000 7f82eeffd700 1 -- 172.21.15.107:0/3663478095 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f82e8086f60 con 0x7f82f80fe040 2024-04-22T23:32:12.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:12.992+0000 7f82fe4ca700 1 --2- 172.21.15.107:0/3663478095 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f82e40692a0 0x7f82e406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:12.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:12.992+0000 7f82fe4ca700 1 --2- 172.21.15.107:0/3663478095 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f82e40692a0 0x7f82e406b750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f82f400b3b0 tx=0x7f82f4009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:32:12.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:12.996+0000 7f82eeffd700 1 -- 172.21.15.107:0/3663478095 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f82e8057060 con 0x7f82f80fe040 2024-04-22T23:32:13.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:13.189+0000 7f830072e700 1 -- 172.21.15.107:0/3663478095 --> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f82f810f930 con 0x7f82e40692a0 2024-04-22T23:32:13.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:13.191+0000 7f82eeffd700 1 -- 172.21.15.107:0/3663478095 <== mgr.14206 v2:172.21.15.107:6800/2516490111 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+148 (secure 0 0 0) 0x7f82f810f930 con 0x7f82e40692a0 2024-04-22T23:32:13.188 INFO:teuthology.orchestra.run.smithi107.stdout:HOST ADDR LABELS STATUS 2024-04-22T23:32:13.188 INFO:teuthology.orchestra.run.smithi107.stdout:smithi107 172.21.15.107 2024-04-22T23:32:13.188 INFO:teuthology.orchestra.run.smithi107.stdout:smithi157 172.21.15.157 2024-04-22T23:32:13.188 INFO:teuthology.orchestra.run.smithi107.stdout:2 hosts in cluster 2024-04-22T23:32:13.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:13.193+0000 7f830072e700 1 -- 172.21.15.107:0/3663478095 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f82e40692a0 msgr2=0x7f82e406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:13.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:13.193+0000 7f830072e700 1 --2- 172.21.15.107:0/3663478095 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f82e40692a0 0x7f82e406b750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f82f400b3b0 tx=0x7f82f4009e30 comp rx=0 tx=0).stop 2024-04-22T23:32:13.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:13.193+0000 7f830072e700 1 -- 172.21.15.107:0/3663478095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82f80fe040 msgr2=0x7f82f810dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:13.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:13.193+0000 7f830072e700 1 --2- 172.21.15.107:0/3663478095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82f80fe040 0x7f82f810dfa0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f82e800c8c0 tx=0x7f82e80044c0 comp rx=0 tx=0).stop 2024-04-22T23:32:13.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:13.193+0000 7f830072e700 1 -- 172.21.15.107:0/3663478095 shutdown_connections 2024-04-22T23:32:13.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:13.193+0000 7f830072e700 1 --2- 172.21.15.107:0/3663478095 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f82e40692a0 0x7f82e406b750 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:13.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:13.193+0000 7f830072e700 1 --2- 172.21.15.107:0/3663478095 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82f80fce40 0x7f82f810da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:13.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:13.193+0000 7f830072e700 1 --2- 172.21.15.107:0/3663478095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82f80fe040 0x7f82f810dfa0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:13.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:13.193+0000 7f830072e700 1 -- 172.21.15.107:0/3663478095 >> 172.21.15.107:0/3663478095 conn(0x7f82f80f8570 msgr2=0x7f82f80f9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:32:13.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:13.193+0000 7f830072e700 1 -- 172.21.15.107:0/3663478095 shutdown_connections 2024-04-22T23:32:13.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:13.193+0000 7f830072e700 1 -- 172.21.15.107:0/3663478095 wait complete. 2024-04-22T23:32:13.383 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:13 smithi157 ceph-mon[33877]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:32:13.485 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:13 smithi107 ceph-mon[26355]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:32:13.916 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-22T23:32:14.172 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:32:14.235 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:14 smithi107 ceph-mon[26355]: from='client.14440 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:32:14.383 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:14 smithi157 ceph-mon[33877]: from='client.14440 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:32:15.235 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:15 smithi107 ceph-mon[26355]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:32:15.383 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:15 smithi157 ceph-mon[33877]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:32:15.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:15.877+0000 7f03cfbc8700 1 -- 172.21.15.107:0/3535896382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03c8100030 msgr2=0x7f03c8102410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:15.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:15.877+0000 7f03cfbc8700 1 --2- 172.21.15.107:0/3535896382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03c8100030 0x7f03c8102410 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f03b8004940 tx=0x7f03b8009e30 comp rx=0 tx=0).stop 2024-04-22T23:32:15.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:15.877+0000 7f03cfbc8700 1 -- 172.21.15.107:0/3535896382 shutdown_connections 2024-04-22T23:32:15.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:15.877+0000 7f03cfbc8700 1 --2- 172.21.15.107:0/3535896382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03c8100030 0x7f03c8102410 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:15.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:15.877+0000 7f03cfbc8700 1 --2- 172.21.15.107:0/3535896382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03c8066230 0x7f03c80ffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:15.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:15.877+0000 7f03cfbc8700 1 -- 172.21.15.107:0/3535896382 >> 172.21.15.107:0/3535896382 conn(0x7f03c80f74c0 msgr2=0x7f03c80f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:32:15.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:15.878+0000 7f03cfbc8700 1 -- 172.21.15.107:0/3535896382 shutdown_connections 2024-04-22T23:32:15.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:15.878+0000 7f03cfbc8700 1 -- 172.21.15.107:0/3535896382 wait complete. 2024-04-22T23:32:15.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:15.878+0000 7f03cfbc8700 1 Processor -- start 2024-04-22T23:32:15.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:15.878+0000 7f03cfbc8700 1 -- start start 2024-04-22T23:32:15.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:15.878+0000 7f03cfbc8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03c8066230 0x7f03c810fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:15.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:15.878+0000 7f03cfbc8700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03c8100030 0x7f03c81101a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:15.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:15.878+0000 7f03cfbc8700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c81107a0 con 0x7f03c8066230 2024-04-22T23:32:15.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:15.878+0000 7f03cfbc8700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c8110ff0 con 0x7f03c8100030 2024-04-22T23:32:15.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:15.878+0000 7f03cd964700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03c8066230 0x7f03c810fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:15.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:15.878+0000 7f03cd964700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03c8066230 0x7f03c810fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40226/0 (socket says 172.21.15.107:40226) 2024-04-22T23:32:15.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:15.878+0000 7f03cd964700 1 -- 172.21.15.107:0/1364258682 learned_addr learned my addr 172.21.15.107:0/1364258682 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:32:15.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:15.879+0000 7f03cd964700 1 -- 172.21.15.107:0/1364258682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03c8100030 msgr2=0x7f03c81101a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:15.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:15.879+0000 7f03cd964700 1 --2- 172.21.15.107:0/1364258682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03c8100030 0x7f03c81101a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:15.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:15.879+0000 7f03cd964700 1 -- 172.21.15.107:0/1364258682 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b8005040 con 0x7f03c8066230 2024-04-22T23:32:15.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:15.879+0000 7f03cd964700 1 --2- 172.21.15.107:0/1364258682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03c8066230 0x7f03c810fc60 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f03c400c8b0 tx=0x7f03c4004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:32:15.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:15.879+0000 7f03be7fc700 1 -- 172.21.15.107:0/1364258682 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03c400a760 con 0x7f03c8066230 2024-04-22T23:32:15.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:15.879+0000 7f03be7fc700 1 -- 172.21.15.107:0/1364258682 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f03c400ad90 con 0x7f03c8066230 2024-04-22T23:32:15.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:15.879+0000 7f03be7fc700 1 -- 172.21.15.107:0/1364258682 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03c40113a0 con 0x7f03c8066230 2024-04-22T23:32:15.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:15.879+0000 7f03cfbc8700 1 -- 172.21.15.107:0/1364258682 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03c81112d0 con 0x7f03c8066230 2024-04-22T23:32:15.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:15.879+0000 7f03cfbc8700 1 -- 172.21.15.107:0/1364258682 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03c8111810 con 0x7f03c8066230 2024-04-22T23:32:15.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:15.880+0000 7f03cfbc8700 1 -- 172.21.15.107:0/1364258682 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03c805f5f0 con 0x7f03c8066230 2024-04-22T23:32:15.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:15.880+0000 7f03be7fc700 1 -- 172.21.15.107:0/1364258682 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f03c4012400 con 0x7f03c8066230 2024-04-22T23:32:15.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:15.881+0000 7f03be7fc700 1 --2- 172.21.15.107:0/1364258682 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f03b40692b0 0x7f03b406b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:15.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:15.881+0000 7f03be7fc700 1 -- 172.21.15.107:0/1364258682 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f03c4086ea0 con 0x7f03c8066230 2024-04-22T23:32:15.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:15.881+0000 7f03cd163700 1 --2- 172.21.15.107:0/1364258682 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f03b40692b0 0x7f03b406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:15.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:15.881+0000 7f03cd163700 1 --2- 172.21.15.107:0/1364258682 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f03b40692b0 0x7f03b406b760 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f03b800b970 tx=0x7f03b8009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:32:15.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:15.885+0000 7f03be7fc700 1 -- 172.21.15.107:0/1364258682 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f03c4056fa0 con 0x7f03c8066230 2024-04-22T23:32:16.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:16.079+0000 7f03cfbc8700 1 -- 172.21.15.107:0/1364258682 --> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f03c8111cd0 con 0x7f03b40692b0 2024-04-22T23:32:16.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:16.080+0000 7f03be7fc700 1 -- 172.21.15.107:0/1364258682 <== mgr.14206 v2:172.21.15.107:6800/2516490111 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+399 (secure 0 0 0) 0x7f03c8111cd0 con 0x7f03b40692b0 2024-04-22T23:32:16.078 INFO:teuthology.orchestra.run.smithi107.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-22T23:32:16.078 INFO:teuthology.orchestra.run.smithi107.stdout:smithi107 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400NK400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-22T23:32:16.078 INFO:teuthology.orchestra.run.smithi107.stdout:smithi157 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7476002V480DGN 447G No 70s ago Has a FileSystem, LVM detected 2024-04-22T23:32:16.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:16.083+0000 7f03cfbc8700 1 -- 172.21.15.107:0/1364258682 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f03b40692b0 msgr2=0x7f03b406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:16.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:16.083+0000 7f03cfbc8700 1 --2- 172.21.15.107:0/1364258682 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f03b40692b0 0x7f03b406b760 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f03b800b970 tx=0x7f03b8009e30 comp rx=0 tx=0).stop 2024-04-22T23:32:16.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:16.083+0000 7f03cfbc8700 1 -- 172.21.15.107:0/1364258682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03c8066230 msgr2=0x7f03c810fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:16.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:16.083+0000 7f03cfbc8700 1 --2- 172.21.15.107:0/1364258682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03c8066230 0x7f03c810fc60 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f03c400c8b0 tx=0x7f03c4004470 comp rx=0 tx=0).stop 2024-04-22T23:32:16.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:16.083+0000 7f03cfbc8700 1 -- 172.21.15.107:0/1364258682 shutdown_connections 2024-04-22T23:32:16.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:16.083+0000 7f03cfbc8700 1 --2- 172.21.15.107:0/1364258682 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f03b40692b0 0x7f03b406b760 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:16.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:16.083+0000 7f03cfbc8700 1 --2- 172.21.15.107:0/1364258682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03c8066230 0x7f03c810fc60 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:16.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:16.083+0000 7f03cfbc8700 1 --2- 172.21.15.107:0/1364258682 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03c8100030 0x7f03c81101a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:16.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:16.083+0000 7f03cfbc8700 1 -- 172.21.15.107:0/1364258682 >> 172.21.15.107:0/1364258682 conn(0x7f03c80f74c0 msgr2=0x7f03c80fbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:32:16.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:16.083+0000 7f03cfbc8700 1 -- 172.21.15.107:0/1364258682 shutdown_connections 2024-04-22T23:32:16.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:16.083+0000 7f03cfbc8700 1 -- 172.21.15.107:0/1364258682 wait complete. 2024-04-22T23:32:16.787 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-22T23:32:16.797 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi107.front.sepia.ceph.com 2024-04-22T23:32:16.797 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-04-22T23:32:17.048 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:32:17.384 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:17 smithi157 ceph-mon[33877]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:32:17.384 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:17 smithi157 ceph-mon[33877]: from='client.14444 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:32:17.423 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:17 smithi107 ceph-mon[26355]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:32:17.423 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:17 smithi107 ceph-mon[26355]: from='client.14444 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:32:18.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:18.602+0000 7f8894a10700 1 -- 172.21.15.107:0/4231951334 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88900fe070 msgr2=0x7f88900fe4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:18.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:18.602+0000 7f8894a10700 1 --2- 172.21.15.107:0/4231951334 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88900fe070 0x7f88900fe4e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8880004940 tx=0x7f8880009e30 comp rx=0 tx=0).stop 2024-04-22T23:32:18.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:18.602+0000 7f8894a10700 1 -- 172.21.15.107:0/4231951334 shutdown_connections 2024-04-22T23:32:18.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:18.602+0000 7f8894a10700 1 --2- 172.21.15.107:0/4231951334 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88900fe070 0x7f88900fe4e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:18.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:18.602+0000 7f8894a10700 1 --2- 172.21.15.107:0/4231951334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88900fcf00 0x7f88900fd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:18.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:18.602+0000 7f8894a10700 1 -- 172.21.15.107:0/4231951334 >> 172.21.15.107:0/4231951334 conn(0x7f88900f85e0 msgr2=0x7f88900fa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:32:18.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:18.602+0000 7f8894a10700 1 -- 172.21.15.107:0/4231951334 shutdown_connections 2024-04-22T23:32:18.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:18.602+0000 7f8894a10700 1 -- 172.21.15.107:0/4231951334 wait complete. 2024-04-22T23:32:18.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:18.602+0000 7f8894a10700 1 Processor -- start 2024-04-22T23:32:18.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:18.603+0000 7f8894a10700 1 -- start start 2024-04-22T23:32:18.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:18.603+0000 7f8894a10700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88900fcf00 0x7f8890071e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:18.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:18.603+0000 7f8894a10700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88900fe070 0x7f8890070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:18.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:18.603+0000 7f8894a10700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88900708c0 con 0x7f88900fcf00 2024-04-22T23:32:18.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:18.603+0000 7f8894a10700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8890070a30 con 0x7f88900fe070 2024-04-22T23:32:18.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:18.603+0000 7f888e59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88900fcf00 0x7f8890071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:18.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:18.603+0000 7f888e59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88900fcf00 0x7f8890071e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40252/0 (socket says 172.21.15.107:40252) 2024-04-22T23:32:18.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:18.603+0000 7f888e59c700 1 -- 172.21.15.107:0/4190886115 learned_addr learned my addr 172.21.15.107:0/4190886115 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:32:18.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:18.603+0000 7f888dd9b700 1 --2- 172.21.15.107:0/4190886115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88900fe070 0x7f8890070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:18.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:18.603+0000 7f888e59c700 1 -- 172.21.15.107:0/4190886115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88900fe070 msgr2=0x7f8890070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:18.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:18.603+0000 7f888e59c700 1 --2- 172.21.15.107:0/4190886115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88900fe070 0x7f8890070310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:18.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:18.603+0000 7f888e59c700 1 -- 172.21.15.107:0/4190886115 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8880005040 con 0x7f88900fcf00 2024-04-22T23:32:18.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:18.603+0000 7f888dd9b700 1 --2- 172.21.15.107:0/4190886115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88900fe070 0x7f8890070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T23:32:18.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:18.603+0000 7f888e59c700 1 --2- 172.21.15.107:0/4190886115 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88900fcf00 0x7f8890071e40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8884002a50 tx=0x7f88840046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:32:18.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:18.603+0000 7f887effd700 1 -- 172.21.15.107:0/4190886115 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8884007cc0 con 0x7f88900fcf00 2024-04-22T23:32:18.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:18.603+0000 7f887effd700 1 -- 172.21.15.107:0/4190886115 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f888400f040 con 0x7f88900fcf00 2024-04-22T23:32:18.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:18.603+0000 7f8894a10700 1 -- 172.21.15.107:0/4190886115 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8890070d10 con 0x7f88900fcf00 2024-04-22T23:32:18.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:18.603+0000 7f887effd700 1 -- 172.21.15.107:0/4190886115 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8884009640 con 0x7f88900fcf00 2024-04-22T23:32:18.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:18.603+0000 7f8894a10700 1 -- 172.21.15.107:0/4190886115 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8890115a50 con 0x7f88900fcf00 2024-04-22T23:32:18.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:18.604+0000 7f8894a10700 1 -- 172.21.15.107:0/4190886115 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f889005f5f0 con 0x7f88900fcf00 2024-04-22T23:32:18.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:18.605+0000 7f887effd700 1 -- 172.21.15.107:0/4190886115 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f88840097a0 con 0x7f88900fcf00 2024-04-22T23:32:18.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:18.605+0000 7f887effd700 1 --2- 172.21.15.107:0/4190886115 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f88780692a0 0x7f887806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:18.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:18.605+0000 7f887effd700 1 -- 172.21.15.107:0/4190886115 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f8884086b80 con 0x7f88900fcf00 2024-04-22T23:32:18.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:18.605+0000 7f888dd9b700 1 --2- 172.21.15.107:0/4190886115 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f88780692a0 0x7f887806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:18.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:18.606+0000 7f888dd9b700 1 --2- 172.21.15.107:0/4190886115 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f88780692a0 0x7f887806b750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f888000b3b0 tx=0x7f8880009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:32:18.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:18.610+0000 7f887effd700 1 -- 172.21.15.107:0/4190886115 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f88840531f0 con 0x7f88900fcf00 2024-04-22T23:32:18.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:18.832+0000 7f8894a10700 1 -- 172.21.15.107:0/4190886115 --> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f88901023e0 con 0x7f88780692a0 2024-04-22T23:32:19.235 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:19 smithi107 ceph-mon[26355]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:32:19.383 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:19 smithi157 ceph-mon[33877]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:32:20.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:20.164+0000 7f887effd700 1 -- 172.21.15.107:0/4190886115 <== mgr.14206 v2:172.21.15.107:6800/2516490111 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f88901023e0 con 0x7f88780692a0 2024-04-22T23:32:20.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:20.167+0000 7f8894a10700 1 -- 172.21.15.107:0/4190886115 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f88780692a0 msgr2=0x7f887806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:20.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:20.167+0000 7f8894a10700 1 --2- 172.21.15.107:0/4190886115 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f88780692a0 0x7f887806b750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f888000b3b0 tx=0x7f8880009e30 comp rx=0 tx=0).stop 2024-04-22T23:32:20.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:20.167+0000 7f8894a10700 1 -- 172.21.15.107:0/4190886115 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88900fcf00 msgr2=0x7f8890071e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:20.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:20.167+0000 7f8894a10700 1 --2- 172.21.15.107:0/4190886115 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88900fcf00 0x7f8890071e40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8884002a50 tx=0x7f88840046b0 comp rx=0 tx=0).stop 2024-04-22T23:32:20.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:20.167+0000 7f8894a10700 1 -- 172.21.15.107:0/4190886115 shutdown_connections 2024-04-22T23:32:20.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:20.167+0000 7f8894a10700 1 --2- 172.21.15.107:0/4190886115 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f88780692a0 0x7f887806b750 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:20.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:20.167+0000 7f8894a10700 1 --2- 172.21.15.107:0/4190886115 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88900fcf00 0x7f8890071e40 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:20.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:20.167+0000 7f8894a10700 1 --2- 172.21.15.107:0/4190886115 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88900fe070 0x7f8890070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:20.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:20.167+0000 7f8894a10700 1 -- 172.21.15.107:0/4190886115 >> 172.21.15.107:0/4190886115 conn(0x7f88900f85e0 msgr2=0x7f88900f9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:32:20.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:20.167+0000 7f8894a10700 1 -- 172.21.15.107:0/4190886115 shutdown_connections 2024-04-22T23:32:20.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:20.168+0000 7f8894a10700 1 -- 172.21.15.107:0/4190886115 wait complete. 2024-04-22T23:32:20.383 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:20 smithi157 ceph-mon[33877]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:32:20.383 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:20 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-04-22T23:32:20.383 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:20 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:32:20.384 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:20 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-04-22T23:32:20.384 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:20 smithi157 ceph-mon[33877]: osdmap e41: 6 total, 6 up, 6 in 2024-04-22T23:32:20.384 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:20 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-04-22T23:32:20.466 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:20 smithi107 ceph-mon[26355]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:32:20.466 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:20 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-04-22T23:32:20.466 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:20 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:32:20.466 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:20 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-04-22T23:32:20.466 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:20 smithi107 ceph-mon[26355]: osdmap e41: 6 total, 6 up, 6 in 2024-04-22T23:32:20.466 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:20 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-04-22T23:32:20.466 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:20 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-mon-smithi107[26351]: 2024-04-22T23:32:20.145+0000 7fb419a62700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-22T23:32:20.782 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-22T23:32:21.361 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:21 smithi107 ceph-mon[26355]: pgmap v172: 33 pgs: 5 creating+peering, 1 active+clean, 27 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:32:21.361 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:21 smithi107 ceph-mon[26355]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-22T23:32:21.361 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:21 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-04-22T23:32:21.362 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:21 smithi107 ceph-mon[26355]: osdmap e42: 6 total, 6 up, 6 in 2024-04-22T23:32:21.362 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:21 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-04-22T23:32:21.362 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:21 smithi107 ceph-mon[26355]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-22T23:32:21.362 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:21 smithi107 ceph-mon[26355]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-22T23:32:21.362 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:21 smithi107 ceph-mon[26355]: osdmap e43: 6 total, 6 up, 6 in 2024-04-22T23:32:21.362 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:21 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-04-22T23:32:21.362 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:21 smithi107 ceph-mon[26355]: fsmap cephfs:0 2024-04-22T23:32:21.362 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:21 smithi107 ceph-mon[26355]: Saving service mds.cephfs spec with placement count:4 2024-04-22T23:32:21.362 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:21 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:32:21.362 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:21 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:32:21.362 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:21 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:32:21.362 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:21 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:32:21.362 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:21 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:32:21.363 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:21 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi107.dfvvkw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T23:32:21.363 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:21 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi107.dfvvkw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T23:32:21.363 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:21 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:32:21.383 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:21 smithi157 ceph-mon[33877]: pgmap v172: 33 pgs: 5 creating+peering, 1 active+clean, 27 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:32:21.384 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:21 smithi157 ceph-mon[33877]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-22T23:32:21.384 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:21 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-04-22T23:32:21.384 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:21 smithi157 ceph-mon[33877]: osdmap e42: 6 total, 6 up, 6 in 2024-04-22T23:32:21.384 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:21 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-04-22T23:32:21.384 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:21 smithi157 ceph-mon[33877]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-22T23:32:21.384 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:21 smithi157 ceph-mon[33877]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-22T23:32:21.384 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:21 smithi157 ceph-mon[33877]: osdmap e43: 6 total, 6 up, 6 in 2024-04-22T23:32:21.384 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:21 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-04-22T23:32:21.384 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:21 smithi157 ceph-mon[33877]: fsmap cephfs:0 2024-04-22T23:32:21.384 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:21 smithi157 ceph-mon[33877]: Saving service mds.cephfs spec with placement count:4 2024-04-22T23:32:21.384 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:21 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:32:21.384 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:21 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:32:21.384 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:21 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:32:21.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:21 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:32:21.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:21 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:32:21.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:21 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi107.dfvvkw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T23:32:21.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:21 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi107.dfvvkw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T23:32:21.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:21 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:32:22.092 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:32:22.235 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:22 smithi107 ceph-mon[26355]: Deploying daemon mds.cephfs.smithi107.dfvvkw on smithi107 2024-04-22T23:32:22.235 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:22 smithi107 ceph-mon[26355]: osdmap e44: 6 total, 6 up, 6 in 2024-04-22T23:32:22.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:22 smithi157 ceph-mon[33877]: Deploying daemon mds.cephfs.smithi107.dfvvkw on smithi107 2024-04-22T23:32:22.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:22 smithi157 ceph-mon[33877]: osdmap e44: 6 total, 6 up, 6 in 2024-04-22T23:32:23.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:23 smithi107 ceph-mon[26355]: pgmap v176: 65 pgs: 8 creating+activating, 11 creating+peering, 23 active+clean, 23 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:32:23.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:23 smithi107 ceph-mon[26355]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-22T23:32:23.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:23 smithi107 ceph-mon[26355]: osdmap e45: 6 total, 6 up, 6 in 2024-04-22T23:32:23.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:23 smithi157 ceph-mon[33877]: pgmap v176: 65 pgs: 8 creating+activating, 11 creating+peering, 23 active+clean, 23 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:32:23.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:23 smithi157 ceph-mon[33877]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-22T23:32:23.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:23 smithi157 ceph-mon[33877]: osdmap e45: 6 total, 6 up, 6 in 2024-04-22T23:32:24.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:24.081+0000 7f888bd41700 1 -- 172.21.15.107:0/4234013350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88840fbd40 msgr2=0x7f88840fc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:24.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:24.081+0000 7f888bd41700 1 --2- 172.21.15.107:0/4234013350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88840fbd40 0x7f88840fc150 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8874004940 tx=0x7f8874009e30 comp rx=0 tx=0).stop 2024-04-22T23:32:24.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:24.081+0000 7f888bd41700 1 -- 172.21.15.107:0/4234013350 shutdown_connections 2024-04-22T23:32:24.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:24.081+0000 7f888bd41700 1 --2- 172.21.15.107:0/4234013350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88840fcff0 0x7f88840fd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:24.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:24.081+0000 7f888bd41700 1 --2- 172.21.15.107:0/4234013350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88840fbd40 0x7f88840fc150 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:24.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:24.081+0000 7f888bd41700 1 -- 172.21.15.107:0/4234013350 >> 172.21.15.107:0/4234013350 conn(0x7f88840f74c0 msgr2=0x7f88840f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:32:24.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:24.081+0000 7f888bd41700 1 -- 172.21.15.107:0/4234013350 shutdown_connections 2024-04-22T23:32:24.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:24.081+0000 7f888bd41700 1 -- 172.21.15.107:0/4234013350 wait complete. 2024-04-22T23:32:24.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:24.082+0000 7f888bd41700 1 Processor -- start 2024-04-22T23:32:24.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:24.082+0000 7f888bd41700 1 -- start start 2024-04-22T23:32:24.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:24.082+0000 7f888bd41700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88840fbd40 0x7f8884112b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:24.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:24.082+0000 7f888bd41700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88840fcff0 0x7f88841150b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:24.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:24.082+0000 7f888bd41700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8884066250 con 0x7f88840fbd40 2024-04-22T23:32:24.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:24.082+0000 7f888bd41700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8884066390 con 0x7f88840fcff0 2024-04-22T23:32:24.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:24.082+0000 7f8889add700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88840fbd40 0x7f8884112b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:24.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:24.082+0000 7f8889add700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88840fbd40 0x7f8884112b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60092/0 (socket says 172.21.15.107:60092) 2024-04-22T23:32:24.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:24.082+0000 7f8889add700 1 -- 172.21.15.107:0/366665994 learned_addr learned my addr 172.21.15.107:0/366665994 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:32:24.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:24.082+0000 7f8889add700 1 -- 172.21.15.107:0/366665994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88840fcff0 msgr2=0x7f88841150b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:24.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:24.082+0000 7f8889add700 1 --2- 172.21.15.107:0/366665994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88840fcff0 0x7f88841150b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:24.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:24.082+0000 7f8889add700 1 -- 172.21.15.107:0/366665994 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8874005040 con 0x7f88840fbd40 2024-04-22T23:32:24.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:24.083+0000 7f8889add700 1 --2- 172.21.15.107:0/366665994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88840fbd40 0x7f8884112b60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f887400b3b0 tx=0x7f8874009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:32:24.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:24.083+0000 7f887a7fc700 1 -- 172.21.15.107:0/366665994 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f887400e040 con 0x7f88840fbd40 2024-04-22T23:32:24.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:24.083+0000 7f887a7fc700 1 -- 172.21.15.107:0/366665994 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f887400bcc0 con 0x7f88840fbd40 2024-04-22T23:32:24.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:24.083+0000 7f888bd41700 1 -- 172.21.15.107:0/366665994 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88841155f0 con 0x7f88840fbd40 2024-04-22T23:32:24.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:24.083+0000 7f887a7fc700 1 -- 172.21.15.107:0/366665994 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f887401c630 con 0x7f88840fbd40 2024-04-22T23:32:24.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:24.083+0000 7f888bd41700 1 -- 172.21.15.107:0/366665994 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88841159c0 con 0x7f88840fbd40 2024-04-22T23:32:24.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:24.084+0000 7f888bd41700 1 -- 172.21.15.107:0/366665994 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f888405f5f0 con 0x7f88840fbd40 2024-04-22T23:32:24.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:24.084+0000 7f887a7fc700 1 -- 172.21.15.107:0/366665994 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f8874013980 con 0x7f88840fbd40 2024-04-22T23:32:24.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:24.084+0000 7f887a7fc700 1 --2- 172.21.15.107:0/366665994 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f88700692a0 0x7f887006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:24.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:24.085+0000 7f887a7fc700 1 -- 172.21.15.107:0/366665994 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f8874087fb0 con 0x7f88840fbd40 2024-04-22T23:32:24.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:24.085+0000 7f88892dc700 1 --2- 172.21.15.107:0/366665994 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f88700692a0 0x7f887006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:24.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:24.085+0000 7f88892dc700 1 --2- 172.21.15.107:0/366665994 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f88700692a0 0x7f887006b750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8880004d00 tx=0x7f8880009ce0 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:32:24.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:24.089+0000 7f887a7fc700 1 -- 172.21.15.107:0/366665994 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8874054260 con 0x7f88840fbd40 2024-04-22T23:32:24.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:24.322+0000 7f888bd41700 1 -- 172.21.15.107:0/366665994 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f88841047e0 con 0x7f88840fbd40 2024-04-22T23:32:24.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:24.322+0000 7f887a7fc700 1 -- 172.21.15.107:0/366665994 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 2 v2) v1 ==== 75+0+1076 (secure 0 0 0) 0x7f8874057880 con 0x7f88840fbd40 2024-04-22T23:32:24.319 INFO:teuthology.orchestra.run.smithi107.stdout:e2 2024-04-22T23:32:24.319 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T23:32:24.319 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:32:24.319 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-04-22T23:32:24.319 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:32:24.319 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-04-22T23:32:24.319 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-04-22T23:32:24.319 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 2 2024-04-22T23:32:24.320 INFO:teuthology.orchestra.run.smithi107.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-22T23:32:24.320 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-04-22T23:32:20.146130+0000 2024-04-22T23:32:24.320 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-04-22T23:32:20.146191+0000 2024-04-22T23:32:24.320 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-04-22T23:32:24.320 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-04-22T23:32:24.320 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-04-22T23:32:24.320 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-04-22T23:32:24.320 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-04-22T23:32:24.320 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-04-22T23:32:24.320 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-04-22T23:32:24.320 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 0 2024-04-22T23:32:24.320 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:32:24.320 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 1 2024-04-22T23:32:24.320 INFO:teuthology.orchestra.run.smithi107.stdout:in 2024-04-22T23:32:24.321 INFO:teuthology.orchestra.run.smithi107.stdout:up {} 2024-04-22T23:32:24.321 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-04-22T23:32:24.321 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-04-22T23:32:24.321 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-04-22T23:32:24.321 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-04-22T23:32:24.321 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-04-22T23:32:24.321 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data disabled 2024-04-22T23:32:24.321 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-04-22T23:32:24.321 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 0 2024-04-22T23:32:24.321 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:32:24.321 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:32:24.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:24.325+0000 7f888bd41700 1 -- 172.21.15.107:0/366665994 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f88700692a0 msgr2=0x7f887006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:24.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:24.325+0000 7f888bd41700 1 --2- 172.21.15.107:0/366665994 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f88700692a0 0x7f887006b750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8880004d00 tx=0x7f8880009ce0 comp rx=0 tx=0).stop 2024-04-22T23:32:24.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:24.325+0000 7f888bd41700 1 -- 172.21.15.107:0/366665994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88840fbd40 msgr2=0x7f8884112b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:24.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:24.325+0000 7f888bd41700 1 --2- 172.21.15.107:0/366665994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88840fbd40 0x7f8884112b60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f887400b3b0 tx=0x7f8874009e30 comp rx=0 tx=0).stop 2024-04-22T23:32:24.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:24.325+0000 7f888bd41700 1 -- 172.21.15.107:0/366665994 shutdown_connections 2024-04-22T23:32:24.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:24.325+0000 7f888bd41700 1 --2- 172.21.15.107:0/366665994 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f88700692a0 0x7f887006b750 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:24.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:24.325+0000 7f888bd41700 1 --2- 172.21.15.107:0/366665994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88840fbd40 0x7f8884112b60 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:24.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:24.325+0000 7f888bd41700 1 --2- 172.21.15.107:0/366665994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88840fcff0 0x7f88841150b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:24.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:24.325+0000 7f888bd41700 1 -- 172.21.15.107:0/366665994 >> 172.21.15.107:0/366665994 conn(0x7f88840f74c0 msgr2=0x7f88840f8bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:32:24.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:24.325+0000 7f888bd41700 1 -- 172.21.15.107:0/366665994 shutdown_connections 2024-04-22T23:32:24.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:24.325+0000 7f888bd41700 1 -- 172.21.15.107:0/366665994 wait complete. 2024-04-22T23:32:24.324 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 2 2024-04-22T23:32:25.059 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-22T23:32:25.069 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi107.front.sepia.ceph.com 2024-04-22T23:32:25.069 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- bash -c 'ceph fs set cephfs max_mds 2' 2024-04-22T23:32:25.332 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:32:25.455 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:25 smithi107 ceph-mon[26355]: pgmap v178: 65 pgs: 8 creating+activating, 11 creating+peering, 38 active+clean, 8 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:32:25.455 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:25 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/366665994' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:32:25.456 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:25 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:32:25.456 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:25 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:32:25.456 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:25 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:32:25.456 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:25 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi157.rcaris", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T23:32:25.456 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:25 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi157.rcaris", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T23:32:25.456 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:25 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:32:25.456 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:25 smithi107 ceph-mon[26355]: Deploying daemon mds.cephfs.smithi157.rcaris on smithi157 2024-04-22T23:32:25.491 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:25 smithi157 ceph-mon[33877]: pgmap v178: 65 pgs: 8 creating+activating, 11 creating+peering, 38 active+clean, 8 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:32:25.491 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:25 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/366665994' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:32:25.491 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:25 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:32:25.492 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:25 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:32:25.492 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:25 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:32:25.492 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:25 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi157.rcaris", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T23:32:25.492 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:25 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi157.rcaris", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T23:32:25.492 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:25 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:32:25.492 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:25 smithi157 ceph-mon[33877]: Deploying daemon mds.cephfs.smithi157.rcaris on smithi157 2024-04-22T23:32:26.378 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:26 smithi157 ceph-mon[33877]: mds.? [v2:172.21.15.107:6826/4144299942,v1:172.21.15.107:6827/4144299942] up:boot 2024-04-22T23:32:26.378 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:26 smithi157 ceph-mon[33877]: daemon mds.cephfs.smithi107.dfvvkw assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-04-22T23:32:26.379 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:26 smithi157 ceph-mon[33877]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-22T23:32:26.379 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:26 smithi157 ceph-mon[33877]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-22T23:32:26.379 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:26 smithi157 ceph-mon[33877]: Cluster is now healthy 2024-04-22T23:32:26.379 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:26 smithi157 ceph-mon[33877]: fsmap cephfs:0 1 up:standby 2024-04-22T23:32:26.379 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:26 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.dfvvkw"}]: dispatch 2024-04-22T23:32:26.379 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:26 smithi157 ceph-mon[33877]: fsmap cephfs:1 {0=cephfs.smithi107.dfvvkw=up:creating} 2024-04-22T23:32:26.379 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:26 smithi157 ceph-mon[33877]: daemon mds.cephfs.smithi107.dfvvkw is now active in filesystem cephfs as rank 0 2024-04-22T23:32:26.439 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:26 smithi107 ceph-mon[26355]: mds.? [v2:172.21.15.107:6826/4144299942,v1:172.21.15.107:6827/4144299942] up:boot 2024-04-22T23:32:26.439 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:26 smithi107 ceph-mon[26355]: daemon mds.cephfs.smithi107.dfvvkw assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-04-22T23:32:26.439 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:26 smithi107 ceph-mon[26355]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-22T23:32:26.440 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:26 smithi107 ceph-mon[26355]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-22T23:32:26.440 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:26 smithi107 ceph-mon[26355]: Cluster is now healthy 2024-04-22T23:32:26.440 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:26 smithi107 ceph-mon[26355]: fsmap cephfs:0 1 up:standby 2024-04-22T23:32:26.440 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:26 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.dfvvkw"}]: dispatch 2024-04-22T23:32:26.440 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:26 smithi107 ceph-mon[26355]: fsmap cephfs:1 {0=cephfs.smithi107.dfvvkw=up:creating} 2024-04-22T23:32:26.440 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:26 smithi107 ceph-mon[26355]: daemon mds.cephfs.smithi107.dfvvkw is now active in filesystem cephfs as rank 0 2024-04-22T23:32:26.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:26.898+0000 7ff0087f5700 1 -- 172.21.15.107:0/194650848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0000662f0 msgr2=0x7ff0000ffa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:26.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:26.898+0000 7ff0087f5700 1 --2- 172.21.15.107:0/194650848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0000662f0 0x7ff0000ffa30 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7feffc004940 tx=0x7feffc009e30 comp rx=0 tx=0).stop 2024-04-22T23:32:26.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:26.899+0000 7ff0087f5700 1 -- 172.21.15.107:0/194650848 shutdown_connections 2024-04-22T23:32:26.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:26.899+0000 7ff0087f5700 1 --2- 172.21.15.107:0/194650848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0000fff70 0x7ff000102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:26.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:26.899+0000 7ff0087f5700 1 --2- 172.21.15.107:0/194650848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0000662f0 0x7ff0000ffa30 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:26.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:26.899+0000 7ff0087f5700 1 -- 172.21.15.107:0/194650848 >> 172.21.15.107:0/194650848 conn(0x7ff0000f74c0 msgr2=0x7ff0000f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:32:26.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:26.899+0000 7ff0087f5700 1 -- 172.21.15.107:0/194650848 shutdown_connections 2024-04-22T23:32:26.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:26.899+0000 7ff0087f5700 1 -- 172.21.15.107:0/194650848 wait complete. 2024-04-22T23:32:26.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:26.899+0000 7ff0087f5700 1 Processor -- start 2024-04-22T23:32:26.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:26.899+0000 7ff0087f5700 1 -- start start 2024-04-22T23:32:26.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:26.899+0000 7ff0087f5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0000662f0 0x7ff0000fd410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:26.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:26.899+0000 7ff0087f5700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0000fff70 0x7ff0000fb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:26.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:26.899+0000 7ff0087f5700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0000fbec0 con 0x7ff0000662f0 2024-04-22T23:32:26.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:26.899+0000 7ff0087f5700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0000fc000 con 0x7ff0000fff70 2024-04-22T23:32:26.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:26.899+0000 7ff006591700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0000662f0 0x7ff0000fd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:26.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:26.899+0000 7ff006591700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0000662f0 0x7ff0000fd410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60114/0 (socket says 172.21.15.107:60114) 2024-04-22T23:32:26.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:26.899+0000 7ff006591700 1 -- 172.21.15.107:0/2496499609 learned_addr learned my addr 172.21.15.107:0/2496499609 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:32:26.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:26.899+0000 7ff005d90700 1 --2- 172.21.15.107:0/2496499609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0000fff70 0x7ff0000fb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:26.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:26.900+0000 7ff006591700 1 -- 172.21.15.107:0/2496499609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0000fff70 msgr2=0x7ff0000fb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:26.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:26.900+0000 7ff006591700 1 --2- 172.21.15.107:0/2496499609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0000fff70 0x7ff0000fb930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:26.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:26.900+0000 7ff006591700 1 -- 172.21.15.107:0/2496499609 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feffc005040 con 0x7ff0000662f0 2024-04-22T23:32:26.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:26.900+0000 7ff006591700 1 --2- 172.21.15.107:0/2496499609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0000662f0 0x7ff0000fd410 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7feffc000c00 tx=0x7feffc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:32:26.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:26.900+0000 7feff6ffd700 1 -- 172.21.15.107:0/2496499609 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feffc00e030 con 0x7ff0000662f0 2024-04-22T23:32:26.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:26.900+0000 7feff6ffd700 1 -- 172.21.15.107:0/2496499609 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feffc013de0 con 0x7ff0000662f0 2024-04-22T23:32:26.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:26.900+0000 7feff6ffd700 1 -- 172.21.15.107:0/2496499609 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feffc01ce50 con 0x7ff0000662f0 2024-04-22T23:32:26.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:26.900+0000 7ff0087f5700 1 -- 172.21.15.107:0/2496499609 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0000fc280 con 0x7ff0000662f0 2024-04-22T23:32:26.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:26.900+0000 7ff0087f5700 1 -- 172.21.15.107:0/2496499609 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff000115a10 con 0x7ff0000662f0 2024-04-22T23:32:26.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:26.901+0000 7ff0087f5700 1 -- 172.21.15.107:0/2496499609 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff00005f5f0 con 0x7ff0000662f0 2024-04-22T23:32:26.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:26.901+0000 7feff6ffd700 1 -- 172.21.15.107:0/2496499609 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7feffc01c400 con 0x7ff0000662f0 2024-04-22T23:32:26.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:26.902+0000 7feff6ffd700 1 --2- 172.21.15.107:0/2496499609 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fefec069230 0x7fefec06b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:26.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:26.902+0000 7feff6ffd700 1 -- 172.21.15.107:0/2496499609 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7feffc088c70 con 0x7ff0000662f0 2024-04-22T23:32:26.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:26.902+0000 7ff005d90700 1 --2- 172.21.15.107:0/2496499609 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fefec069230 0x7fefec06b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:26.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:26.902+0000 7ff005d90700 1 --2- 172.21.15.107:0/2496499609 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fefec069230 0x7fefec06b6e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff0000fd9f0 tx=0x7feff000b040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:32:26.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:26.906+0000 7feff6ffd700 1 -- 172.21.15.107:0/2496499609 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7feffc054f20 con 0x7ff0000662f0 2024-04-22T23:32:27.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:27.152+0000 7ff0087f5700 1 -- 172.21.15.107:0/2496499609 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) v1 -- 0x7ff0001058b0 con 0x7ff0000662f0 2024-04-22T23:32:27.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:27.174+0000 7feff6ffd700 1 -- 172.21.15.107:0/2496499609 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v6) v1 ==== 105+0+0 (secure 0 0 0) 0x7feffc058540 con 0x7ff0000662f0 2024-04-22T23:32:27.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:27.177+0000 7ff0087f5700 1 -- 172.21.15.107:0/2496499609 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fefec069230 msgr2=0x7fefec06b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:27.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:27.177+0000 7ff0087f5700 1 --2- 172.21.15.107:0/2496499609 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fefec069230 0x7fefec06b6e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff0000fd9f0 tx=0x7feff000b040 comp rx=0 tx=0).stop 2024-04-22T23:32:27.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:27.177+0000 7ff0087f5700 1 -- 172.21.15.107:0/2496499609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0000662f0 msgr2=0x7ff0000fd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:27.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:27.177+0000 7ff0087f5700 1 --2- 172.21.15.107:0/2496499609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0000662f0 0x7ff0000fd410 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7feffc000c00 tx=0x7feffc009e30 comp rx=0 tx=0).stop 2024-04-22T23:32:27.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:27.177+0000 7ff0087f5700 1 -- 172.21.15.107:0/2496499609 shutdown_connections 2024-04-22T23:32:27.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:27.177+0000 7ff0087f5700 1 --2- 172.21.15.107:0/2496499609 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fefec069230 0x7fefec06b6e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:27.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:27.177+0000 7ff0087f5700 1 --2- 172.21.15.107:0/2496499609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0000662f0 0x7ff0000fd410 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:27.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:27.177+0000 7ff0087f5700 1 --2- 172.21.15.107:0/2496499609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0000fff70 0x7ff0000fb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:27.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:27.177+0000 7ff0087f5700 1 -- 172.21.15.107:0/2496499609 >> 172.21.15.107:0/2496499609 conn(0x7ff0000f74c0 msgr2=0x7ff0000f7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:32:27.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:27.177+0000 7ff0087f5700 1 -- 172.21.15.107:0/2496499609 shutdown_connections 2024-04-22T23:32:27.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:27.177+0000 7ff0087f5700 1 -- 172.21.15.107:0/2496499609 wait complete. 2024-04-22T23:32:27.485 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:27 smithi107 ceph-mon[26355]: pgmap v179: 65 pgs: 8 creating+activating, 6 creating+peering, 51 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:32:27.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:27 smithi107 ceph-mon[26355]: mds.? [v2:172.21.15.107:6826/4144299942,v1:172.21.15.107:6827/4144299942] up:active 2024-04-22T23:32:27.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:27 smithi107 ceph-mon[26355]: fsmap cephfs:1 {0=cephfs.smithi107.dfvvkw=up:active} 2024-04-22T23:32:27.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:27 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:32:27.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:27 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:32:27.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:27 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:32:27.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:27 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi107.ktuxjl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T23:32:27.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:27 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi107.ktuxjl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T23:32:27.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:27 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:32:27.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:27 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/2496499609' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-04-22T23:32:27.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:27 smithi157 ceph-mon[33877]: pgmap v179: 65 pgs: 8 creating+activating, 6 creating+peering, 51 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:32:27.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:27 smithi157 ceph-mon[33877]: mds.? [v2:172.21.15.107:6826/4144299942,v1:172.21.15.107:6827/4144299942] up:active 2024-04-22T23:32:27.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:27 smithi157 ceph-mon[33877]: fsmap cephfs:1 {0=cephfs.smithi107.dfvvkw=up:active} 2024-04-22T23:32:27.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:27 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:32:27.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:27 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:32:27.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:27 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:32:27.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:27 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi107.ktuxjl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T23:32:27.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:27 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi107.ktuxjl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T23:32:27.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:27 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:32:27.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:27 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/2496499609' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-04-22T23:32:27.831 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-22T23:32:27.841 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi107.front.sepia.ceph.com 2024-04-22T23:32:27.841 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- bash -c 'ceph fs set cephfs allow_standby_replay false' 2024-04-22T23:32:28.402 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:28 smithi157 ceph-mon[33877]: Deploying daemon mds.cephfs.smithi107.ktuxjl on smithi107 2024-04-22T23:32:28.403 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:28 smithi157 ceph-mon[33877]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-22T23:32:28.403 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:28 smithi157 ceph-mon[33877]: mds.? [v2:172.21.15.157:6824/2779251893,v1:172.21.15.157:6825/2779251893] up:boot 2024-04-22T23:32:28.403 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:28 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/2496499609' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-04-22T23:32:28.403 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:28 smithi157 ceph-mon[33877]: daemon mds.cephfs.smithi157.rcaris assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-22T23:32:28.403 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:28 smithi157 ceph-mon[33877]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-04-22T23:32:28.403 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:28 smithi157 ceph-mon[33877]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-22T23:32:28.403 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:28 smithi157 ceph-mon[33877]: fsmap cephfs:1 {0=cephfs.smithi107.dfvvkw=up:active} 1 up:standby 2024-04-22T23:32:28.403 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:28 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi157.rcaris"}]: dispatch 2024-04-22T23:32:28.403 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:28 smithi157 ceph-mon[33877]: fsmap cephfs:2 {0=cephfs.smithi107.dfvvkw=up:active,1=cephfs.smithi157.rcaris=up:creating} 2024-04-22T23:32:28.403 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:28 smithi157 ceph-mon[33877]: daemon mds.cephfs.smithi157.rcaris is now active in filesystem cephfs as rank 1 2024-04-22T23:32:28.485 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:28 smithi107 ceph-mon[26355]: Deploying daemon mds.cephfs.smithi107.ktuxjl on smithi107 2024-04-22T23:32:28.485 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:28 smithi107 ceph-mon[26355]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-22T23:32:28.485 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:28 smithi107 ceph-mon[26355]: mds.? [v2:172.21.15.157:6824/2779251893,v1:172.21.15.157:6825/2779251893] up:boot 2024-04-22T23:32:28.485 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:28 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/2496499609' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-04-22T23:32:28.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:28 smithi107 ceph-mon[26355]: daemon mds.cephfs.smithi157.rcaris assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-22T23:32:28.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:28 smithi107 ceph-mon[26355]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-04-22T23:32:28.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:28 smithi107 ceph-mon[26355]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-22T23:32:28.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:28 smithi107 ceph-mon[26355]: fsmap cephfs:1 {0=cephfs.smithi107.dfvvkw=up:active} 1 up:standby 2024-04-22T23:32:28.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:28 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi157.rcaris"}]: dispatch 2024-04-22T23:32:28.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:28 smithi107 ceph-mon[26355]: fsmap cephfs:2 {0=cephfs.smithi107.dfvvkw=up:active,1=cephfs.smithi157.rcaris=up:creating} 2024-04-22T23:32:28.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:28 smithi107 ceph-mon[26355]: daemon mds.cephfs.smithi157.rcaris is now active in filesystem cephfs as rank 1 2024-04-22T23:32:29.175 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:32:29.298 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:29 smithi107 ceph-mon[26355]: pgmap v180: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s wr, 3 op/s 2024-04-22T23:32:29.298 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:29 smithi107 ceph-mon[26355]: mds.? [v2:172.21.15.157:6824/2779251893,v1:172.21.15.157:6825/2779251893] up:active 2024-04-22T23:32:29.298 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:29 smithi107 ceph-mon[26355]: fsmap cephfs:2 {0=cephfs.smithi107.dfvvkw=up:active,1=cephfs.smithi157.rcaris=up:active} 2024-04-22T23:32:29.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:29 smithi157 ceph-mon[33877]: pgmap v180: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s wr, 3 op/s 2024-04-22T23:32:29.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:29 smithi157 ceph-mon[33877]: mds.? [v2:172.21.15.157:6824/2779251893,v1:172.21.15.157:6825/2779251893] up:active 2024-04-22T23:32:29.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:29 smithi157 ceph-mon[33877]: fsmap cephfs:2 {0=cephfs.smithi107.dfvvkw=up:active,1=cephfs.smithi157.rcaris=up:active} 2024-04-22T23:32:31.204 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:31 smithi107 ceph-mon[26355]: pgmap v181: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s wr, 6 op/s 2024-04-22T23:32:31.204 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:31 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:32:31.204 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:31 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:32:31.205 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:31 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:32:31.205 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:31 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi157.ufywse", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T23:32:31.205 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:31 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi157.ufywse", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T23:32:31.205 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:31 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:32:31.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:31.311+0000 7ff6a221c700 1 -- 172.21.15.107:0/3442820284 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff69c0fdff0 msgr2=0x7ff69c0fe460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:31.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:31.311+0000 7ff6a221c700 1 --2- 172.21.15.107:0/3442820284 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff69c0fdff0 0x7ff69c0fe460 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff690004940 tx=0x7ff690009e30 comp rx=0 tx=0).stop 2024-04-22T23:32:31.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:31.311+0000 7ff6a221c700 1 -- 172.21.15.107:0/3442820284 shutdown_connections 2024-04-22T23:32:31.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:31.311+0000 7ff6a221c700 1 --2- 172.21.15.107:0/3442820284 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff69c0fdff0 0x7ff69c0fe460 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:31.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:31.311+0000 7ff6a221c700 1 --2- 172.21.15.107:0/3442820284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff69c0fce80 0x7ff69c0fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:31.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:31.311+0000 7ff6a221c700 1 -- 172.21.15.107:0/3442820284 >> 172.21.15.107:0/3442820284 conn(0x7ff69c0f8580 msgr2=0x7ff69c0fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:32:31.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:31.311+0000 7ff6a221c700 1 -- 172.21.15.107:0/3442820284 shutdown_connections 2024-04-22T23:32:31.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:31.311+0000 7ff6a221c700 1 -- 172.21.15.107:0/3442820284 wait complete. 2024-04-22T23:32:31.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:31.311+0000 7ff6a221c700 1 Processor -- start 2024-04-22T23:32:31.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:31.311+0000 7ff6a221c700 1 -- start start 2024-04-22T23:32:31.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:31.312+0000 7ff6a221c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff69c0fce80 0x7ff69c10d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:31.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:31.312+0000 7ff6a221c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff69c0fdff0 0x7ff69c10de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:31.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:31.312+0000 7ff6a221c700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69c10e470 con 0x7ff69c0fdff0 2024-04-22T23:32:31.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:31.312+0000 7ff6a221c700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69c10e5e0 con 0x7ff69c0fce80 2024-04-22T23:32:31.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:31.312+0000 7ff69affd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff69c0fdff0 0x7ff69c10de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:31.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:31.312+0000 7ff69affd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff69c0fdff0 0x7ff69c10de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60150/0 (socket says 172.21.15.107:60150) 2024-04-22T23:32:31.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:31.312+0000 7ff69affd700 1 -- 172.21.15.107:0/3521351893 learned_addr learned my addr 172.21.15.107:0/3521351893 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:32:31.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:31.312+0000 7ff69b7fe700 1 --2- 172.21.15.107:0/3521351893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff69c0fce80 0x7ff69c10d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:31.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:31.312+0000 7ff69affd700 1 -- 172.21.15.107:0/3521351893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff69c0fce80 msgr2=0x7ff69c10d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:31.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:31.312+0000 7ff69affd700 1 --2- 172.21.15.107:0/3521351893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff69c0fce80 0x7ff69c10d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:31.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:31.312+0000 7ff69affd700 1 -- 172.21.15.107:0/3521351893 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff690005040 con 0x7ff69c0fdff0 2024-04-22T23:32:31.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:31.312+0000 7ff69affd700 1 --2- 172.21.15.107:0/3521351893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff69c0fdff0 0x7ff69c10de70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff69000b3b0 tx=0x7ff690009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:32:31.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:31.312+0000 7ff683fff700 1 -- 172.21.15.107:0/3521351893 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff69000e040 con 0x7ff69c0fdff0 2024-04-22T23:32:31.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:31.312+0000 7ff683fff700 1 -- 172.21.15.107:0/3521351893 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff69000bbf0 con 0x7ff69c0fdff0 2024-04-22T23:32:31.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:31.312+0000 7ff683fff700 1 -- 172.21.15.107:0/3521351893 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff69001c630 con 0x7ff69c0fdff0 2024-04-22T23:32:31.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:31.312+0000 7ff6a221c700 1 -- 172.21.15.107:0/3521351893 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff69c112fe0 con 0x7ff69c0fdff0 2024-04-22T23:32:31.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:31.312+0000 7ff6a221c700 1 -- 172.21.15.107:0/3521351893 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff69c113570 con 0x7ff69c0fdff0 2024-04-22T23:32:31.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:31.313+0000 7ff6a221c700 1 -- 172.21.15.107:0/3521351893 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff69c04c710 con 0x7ff69c0fdff0 2024-04-22T23:32:31.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:31.314+0000 7ff683fff700 1 -- 172.21.15.107:0/3521351893 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7ff69000bd60 con 0x7ff69c0fdff0 2024-04-22T23:32:31.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:31.314+0000 7ff683fff700 1 --2- 172.21.15.107:0/3521351893 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7ff6840692a0 0x7ff68406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:31.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:31.314+0000 7ff683fff700 1 -- 172.21.15.107:0/3521351893 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7ff690087d80 con 0x7ff69c0fdff0 2024-04-22T23:32:31.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:31.314+0000 7ff69b7fe700 1 --2- 172.21.15.107:0/3521351893 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7ff6840692a0 0x7ff68406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:31.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:31.315+0000 7ff69b7fe700 1 --2- 172.21.15.107:0/3521351893 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7ff6840692a0 0x7ff68406b750 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff68c004f40 tx=0x7ff68c00b040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:32:31.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:31.319+0000 7ff683fff700 1 -- 172.21.15.107:0/3521351893 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ff690057be0 con 0x7ff69c0fdff0 2024-04-22T23:32:31.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:31.547+0000 7ff6a221c700 1 -- 172.21.15.107:0/3521351893 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7ff69c1138a0 con 0x7ff69c0fdff0 2024-04-22T23:32:31.585 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:31 smithi157 ceph-mon[33877]: pgmap v181: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s wr, 6 op/s 2024-04-22T23:32:31.586 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:31 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:32:31.586 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:31 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:32:31.586 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:31 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:32:31.586 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:31 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi157.ufywse", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T23:32:31.586 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:31 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi157.ufywse", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T23:32:31.586 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:31 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:32:32.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:32.196+0000 7ff683fff700 1 -- 172.21.15.107:0/3521351893 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff690057770 con 0x7ff69c0fdff0 2024-04-22T23:32:32.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:32.198+0000 7ff6a221c700 1 -- 172.21.15.107:0/3521351893 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7ff6840692a0 msgr2=0x7ff68406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:32.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:32.198+0000 7ff6a221c700 1 --2- 172.21.15.107:0/3521351893 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7ff6840692a0 0x7ff68406b750 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff68c004f40 tx=0x7ff68c00b040 comp rx=0 tx=0).stop 2024-04-22T23:32:32.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:32.198+0000 7ff6a221c700 1 -- 172.21.15.107:0/3521351893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff69c0fdff0 msgr2=0x7ff69c10de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:32.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:32.198+0000 7ff6a221c700 1 --2- 172.21.15.107:0/3521351893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff69c0fdff0 0x7ff69c10de70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff69000b3b0 tx=0x7ff690009e30 comp rx=0 tx=0).stop 2024-04-22T23:32:32.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:32.198+0000 7ff6a221c700 1 -- 172.21.15.107:0/3521351893 shutdown_connections 2024-04-22T23:32:32.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:32.198+0000 7ff6a221c700 1 --2- 172.21.15.107:0/3521351893 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7ff6840692a0 0x7ff68406b750 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:32.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:32.198+0000 7ff6a221c700 1 --2- 172.21.15.107:0/3521351893 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff69c0fce80 0x7ff69c10d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:32.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:32.198+0000 7ff6a221c700 1 --2- 172.21.15.107:0/3521351893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff69c0fdff0 0x7ff69c10de70 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:32.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:32.198+0000 7ff6a221c700 1 -- 172.21.15.107:0/3521351893 >> 172.21.15.107:0/3521351893 conn(0x7ff69c0f8580 msgr2=0x7ff69c0f9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:32:32.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:32.198+0000 7ff6a221c700 1 -- 172.21.15.107:0/3521351893 shutdown_connections 2024-04-22T23:32:32.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:32.199+0000 7ff6a221c700 1 -- 172.21.15.107:0/3521351893 wait complete. 2024-04-22T23:32:32.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:32 smithi107 ceph-mon[26355]: Deploying daemon mds.cephfs.smithi157.ufywse on smithi157 2024-04-22T23:32:32.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:32 smithi107 ceph-mon[26355]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-04-22T23:32:32.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:32 smithi107 ceph-mon[26355]: Cluster is now healthy 2024-04-22T23:32:32.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:32 smithi107 ceph-mon[26355]: mds.? [v2:172.21.15.107:6828/3335313253,v1:172.21.15.107:6829/3335313253] up:boot 2024-04-22T23:32:32.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:32 smithi107 ceph-mon[26355]: fsmap cephfs:2 {0=cephfs.smithi107.dfvvkw=up:active,1=cephfs.smithi157.rcaris=up:active} 1 up:standby 2024-04-22T23:32:32.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:32 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.ktuxjl"}]: dispatch 2024-04-22T23:32:32.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:32 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/3521351893' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-04-22T23:32:32.625 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:32 smithi157 ceph-mon[33877]: Deploying daemon mds.cephfs.smithi157.ufywse on smithi157 2024-04-22T23:32:32.625 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:32 smithi157 ceph-mon[33877]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-04-22T23:32:32.625 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:32 smithi157 ceph-mon[33877]: Cluster is now healthy 2024-04-22T23:32:32.625 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:32 smithi157 ceph-mon[33877]: mds.? [v2:172.21.15.107:6828/3335313253,v1:172.21.15.107:6829/3335313253] up:boot 2024-04-22T23:32:32.625 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:32 smithi157 ceph-mon[33877]: fsmap cephfs:2 {0=cephfs.smithi107.dfvvkw=up:active,1=cephfs.smithi157.rcaris=up:active} 1 up:standby 2024-04-22T23:32:32.625 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:32 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.ktuxjl"}]: dispatch 2024-04-22T23:32:32.625 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:32 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/3521351893' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-04-22T23:32:32.780 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-22T23:32:32.791 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi107.front.sepia.ceph.com 2024-04-22T23:32:32.792 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- bash -c 'ceph fs set cephfs inline_data false' 2024-04-22T23:32:33.054 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:32:33.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:33 smithi157 ceph-mon[33877]: pgmap v182: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s wr, 8 op/s 2024-04-22T23:32:33.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:33 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/3521351893' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-04-22T23:32:33.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:33 smithi157 ceph-mon[33877]: fsmap cephfs:2 {0=cephfs.smithi107.dfvvkw=up:active,1=cephfs.smithi157.rcaris=up:active} 1 up:standby 2024-04-22T23:32:33.435 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:33 smithi107 ceph-mon[26355]: pgmap v182: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s wr, 8 op/s 2024-04-22T23:32:33.435 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:33 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/3521351893' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-04-22T23:32:33.435 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:33 smithi107 ceph-mon[26355]: fsmap cephfs:2 {0=cephfs.smithi107.dfvvkw=up:active,1=cephfs.smithi157.rcaris=up:active} 1 up:standby 2024-04-22T23:32:34.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:34.832+0000 7f69a4750700 1 -- 172.21.15.107:0/156214995 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f699c0fbce0 msgr2=0x7f699c0fc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:34.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:34.832+0000 7f69a4750700 1 --2- 172.21.15.107:0/156214995 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f699c0fbce0 0x7f699c0fc0f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6998004940 tx=0x7f6998009e30 comp rx=0 tx=0).stop 2024-04-22T23:32:34.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:34.832+0000 7f69a4750700 1 -- 172.21.15.107:0/156214995 shutdown_connections 2024-04-22T23:32:34.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:34.832+0000 7f69a4750700 1 --2- 172.21.15.107:0/156214995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f699c0fcf90 0x7f699c0fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:34.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:34.832+0000 7f69a4750700 1 --2- 172.21.15.107:0/156214995 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f699c0fbce0 0x7f699c0fc0f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:34.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:34.832+0000 7f69a4750700 1 -- 172.21.15.107:0/156214995 >> 172.21.15.107:0/156214995 conn(0x7f699c0f7440 msgr2=0x7f699c0f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:32:34.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:34.832+0000 7f69a4750700 1 -- 172.21.15.107:0/156214995 shutdown_connections 2024-04-22T23:32:34.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:34.832+0000 7f69a4750700 1 -- 172.21.15.107:0/156214995 wait complete. 2024-04-22T23:32:34.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:34.833+0000 7f69a4750700 1 Processor -- start 2024-04-22T23:32:34.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:34.833+0000 7f69a4750700 1 -- start start 2024-04-22T23:32:34.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:34.833+0000 7f69a4750700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f699c0fbce0 0x7f699c066ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:34.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:34.833+0000 7f69a4750700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f699c0fcf90 0x7f699c112740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:34.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:34.833+0000 7f69a4750700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f699c112c80 con 0x7f699c0fbce0 2024-04-22T23:32:34.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:34.833+0000 7f69a4750700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f699c112dc0 con 0x7f699c0fcf90 2024-04-22T23:32:34.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:34.833+0000 7f69a24ec700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f699c0fbce0 0x7f699c066ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:34.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:34.833+0000 7f69a24ec700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f699c0fbce0 0x7f699c066ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36696/0 (socket says 172.21.15.107:36696) 2024-04-22T23:32:34.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:34.833+0000 7f69a24ec700 1 -- 172.21.15.107:0/155944432 learned_addr learned my addr 172.21.15.107:0/155944432 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:32:34.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:34.833+0000 7f69a1ceb700 1 --2- 172.21.15.107:0/155944432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f699c0fcf90 0x7f699c112740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:34.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:34.833+0000 7f69a24ec700 1 -- 172.21.15.107:0/155944432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f699c0fcf90 msgr2=0x7f699c112740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:34.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:34.833+0000 7f69a24ec700 1 --2- 172.21.15.107:0/155944432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f699c0fcf90 0x7f699c112740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:34.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:34.833+0000 7f69a24ec700 1 -- 172.21.15.107:0/155944432 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6998005040 con 0x7f699c0fbce0 2024-04-22T23:32:34.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:34.833+0000 7f69a24ec700 1 --2- 172.21.15.107:0/155944432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f699c0fbce0 0x7f699c066ca0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f699800b3b0 tx=0x7f6998009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:32:34.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:34.834+0000 7f6992ffd700 1 -- 172.21.15.107:0/155944432 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f699800e040 con 0x7f699c0fbce0 2024-04-22T23:32:34.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:34.834+0000 7f69a4750700 1 -- 172.21.15.107:0/155944432 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f699c112fc0 con 0x7f699c0fbce0 2024-04-22T23:32:34.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:34.834+0000 7f6992ffd700 1 -- 172.21.15.107:0/155944432 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f699800bbf0 con 0x7f699c0fbce0 2024-04-22T23:32:34.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:34.834+0000 7f6992ffd700 1 -- 172.21.15.107:0/155944432 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f699801c630 con 0x7f699c0fbce0 2024-04-22T23:32:34.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:34.834+0000 7f69a4750700 1 -- 172.21.15.107:0/155944432 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f699c1134f0 con 0x7f699c0fbce0 2024-04-22T23:32:34.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:34.835+0000 7f69a4750700 1 -- 172.21.15.107:0/155944432 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f699c04c710 con 0x7f699c0fbce0 2024-04-22T23:32:34.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:34.835+0000 7f6992ffd700 1 -- 172.21.15.107:0/155944432 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f699800bd60 con 0x7f699c0fbce0 2024-04-22T23:32:34.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:34.835+0000 7f6992ffd700 1 --2- 172.21.15.107:0/155944432 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f69880692a0 0x7f698806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:34.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:34.835+0000 7f6992ffd700 1 -- 172.21.15.107:0/155944432 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f6998087e50 con 0x7f699c0fbce0 2024-04-22T23:32:34.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:34.835+0000 7f69a1ceb700 1 --2- 172.21.15.107:0/155944432 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f69880692a0 0x7f698806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:34.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:34.836+0000 7f69a1ceb700 1 --2- 172.21.15.107:0/155944432 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f69880692a0 0x7f698806b750 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f698c004f40 tx=0x7f698c00b040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:32:34.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:34.840+0000 7f6992ffd700 1 -- 172.21.15.107:0/155944432 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f6998057c10 con 0x7f699c0fbce0 2024-04-22T23:32:34.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:34 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:32:34.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:34 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:32:34.884 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:34 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:32:34.884 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:34 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:32:34.884 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:34 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:32:34.884 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:34 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:32:34.884 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:34 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:32:34.884 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:34 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:32:34.884 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:34 smithi157 ceph-mon[33877]: pgmap v183: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s wr, 7 op/s 2024-04-22T23:32:34.884 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:34 smithi157 ceph-mon[33877]: mds.? [v2:172.21.15.157:6826/4072444199,v1:172.21.15.157:6827/4072444199] up:boot 2024-04-22T23:32:34.884 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:34 smithi157 ceph-mon[33877]: fsmap cephfs:2 {0=cephfs.smithi107.dfvvkw=up:active,1=cephfs.smithi157.rcaris=up:active} 2 up:standby 2024-04-22T23:32:34.884 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:34 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi157.ufywse"}]: dispatch 2024-04-22T23:32:34.985 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:34 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:32:34.985 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:34 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:32:34.985 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:34 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:32:34.985 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:34 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:32:34.985 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:34 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:32:34.985 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:34 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:32:34.985 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:34 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:32:34.986 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:34 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:32:34.986 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:34 smithi107 ceph-mon[26355]: pgmap v183: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s wr, 7 op/s 2024-04-22T23:32:34.986 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:34 smithi107 ceph-mon[26355]: mds.? [v2:172.21.15.157:6826/4072444199,v1:172.21.15.157:6827/4072444199] up:boot 2024-04-22T23:32:34.986 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:34 smithi107 ceph-mon[26355]: fsmap cephfs:2 {0=cephfs.smithi107.dfvvkw=up:active,1=cephfs.smithi157.rcaris=up:active} 2 up:standby 2024-04-22T23:32:34.986 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:34 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi157.ufywse"}]: dispatch 2024-04-22T23:32:35.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:35.096+0000 7f69a4750700 1 -- 172.21.15.107:0/155944432 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"} v 0) v1 -- 0x7f699c1047d0 con 0x7f699c0fbce0 2024-04-22T23:32:35.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:35.768+0000 7f6992ffd700 1 -- 172.21.15.107:0/155944432 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]=0 inline data disabled v12) v1 ==== 133+0+0 (secure 0 0 0) 0x7f69980577a0 con 0x7f699c0fbce0 2024-04-22T23:32:35.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:35.771+0000 7f69a4750700 1 -- 172.21.15.107:0/155944432 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f69880692a0 msgr2=0x7f698806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:35.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:35.771+0000 7f69a4750700 1 --2- 172.21.15.107:0/155944432 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f69880692a0 0x7f698806b750 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f698c004f40 tx=0x7f698c00b040 comp rx=0 tx=0).stop 2024-04-22T23:32:35.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:35.771+0000 7f69a4750700 1 -- 172.21.15.107:0/155944432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f699c0fbce0 msgr2=0x7f699c066ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:35.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:35.771+0000 7f69a4750700 1 --2- 172.21.15.107:0/155944432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f699c0fbce0 0x7f699c066ca0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f699800b3b0 tx=0x7f6998009e30 comp rx=0 tx=0).stop 2024-04-22T23:32:35.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:35.772+0000 7f69a4750700 1 -- 172.21.15.107:0/155944432 shutdown_connections 2024-04-22T23:32:35.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:35.772+0000 7f69a4750700 1 --2- 172.21.15.107:0/155944432 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f69880692a0 0x7f698806b750 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:35.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:35.772+0000 7f69a4750700 1 --2- 172.21.15.107:0/155944432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f699c0fbce0 0x7f699c066ca0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:35.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:35.772+0000 7f69a4750700 1 --2- 172.21.15.107:0/155944432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f699c0fcf90 0x7f699c112740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:35.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:35.772+0000 7f69a4750700 1 -- 172.21.15.107:0/155944432 >> 172.21.15.107:0/155944432 conn(0x7f699c0f7440 msgr2=0x7f699c0f8b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:32:35.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:35.772+0000 7f69a4750700 1 -- 172.21.15.107:0/155944432 shutdown_connections 2024-04-22T23:32:35.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:35.772+0000 7f69a4750700 1 -- 172.21.15.107:0/155944432 wait complete. 2024-04-22T23:32:35.770 INFO:teuthology.orchestra.run.smithi107.stderr:inline data disabled 2024-04-22T23:32:36.072 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:35 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/155944432' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-04-22T23:32:36.133 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:35 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/155944432' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-04-22T23:32:36.416 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-22T23:32:36.429 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi107.front.sepia.ceph.com 2024-04-22T23:32:36.429 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-22T23:32:37.081 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:32:37.133 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:36 smithi157 ceph-mon[33877]: mds.? [v2:172.21.15.157:6824/2779251893,v1:172.21.15.157:6825/2779251893] up:active 2024-04-22T23:32:37.133 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:36 smithi157 ceph-mon[33877]: mds.? [v2:172.21.15.107:6828/3335313253,v1:172.21.15.107:6829/3335313253] up:standby 2024-04-22T23:32:37.134 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:36 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/155944432' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-04-22T23:32:37.134 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:36 smithi157 ceph-mon[33877]: Dropping low affinity active daemon mds.cephfs.smithi107.dfvvkw in favor of higher affinity standby. 2024-04-22T23:32:37.134 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:36 smithi157 ceph-mon[33877]: Replacing daemon mds.cephfs.smithi107.dfvvkw as rank 0 with standby daemon mds.cephfs.smithi107.ktuxjl 2024-04-22T23:32:37.134 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:36 smithi157 ceph-mon[33877]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-22T23:32:37.134 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:36 smithi157 ceph-mon[33877]: fsmap cephfs:2 {0=cephfs.smithi107.dfvvkw=up:active,1=cephfs.smithi157.rcaris=up:active} 2 up:standby 2024-04-22T23:32:37.134 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:36 smithi157 ceph-mon[33877]: osdmap e46: 6 total, 6 up, 6 in 2024-04-22T23:32:37.134 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:36 smithi157 ceph-mon[33877]: fsmap cephfs:2/2 {0=cephfs.smithi107.ktuxjl=up:replay,1=cephfs.smithi157.rcaris=up:active} 1 up:standby 2024-04-22T23:32:37.134 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:36 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:32:37.134 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:36 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:32:37.134 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:36 smithi157 ceph-mon[33877]: pgmap v185: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s wr, 8 op/s 2024-04-22T23:32:37.204 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:36 smithi107 ceph-mon[26355]: mds.? [v2:172.21.15.157:6824/2779251893,v1:172.21.15.157:6825/2779251893] up:active 2024-04-22T23:32:37.204 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:36 smithi107 ceph-mon[26355]: mds.? [v2:172.21.15.107:6828/3335313253,v1:172.21.15.107:6829/3335313253] up:standby 2024-04-22T23:32:37.204 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:36 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/155944432' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-04-22T23:32:37.204 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:36 smithi107 ceph-mon[26355]: Dropping low affinity active daemon mds.cephfs.smithi107.dfvvkw in favor of higher affinity standby. 2024-04-22T23:32:37.204 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:36 smithi107 ceph-mon[26355]: Replacing daemon mds.cephfs.smithi107.dfvvkw as rank 0 with standby daemon mds.cephfs.smithi107.ktuxjl 2024-04-22T23:32:37.204 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:36 smithi107 ceph-mon[26355]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-22T23:32:37.204 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:36 smithi107 ceph-mon[26355]: fsmap cephfs:2 {0=cephfs.smithi107.dfvvkw=up:active,1=cephfs.smithi157.rcaris=up:active} 2 up:standby 2024-04-22T23:32:37.204 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:36 smithi107 ceph-mon[26355]: osdmap e46: 6 total, 6 up, 6 in 2024-04-22T23:32:37.204 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:36 smithi107 ceph-mon[26355]: fsmap cephfs:2/2 {0=cephfs.smithi107.ktuxjl=up:replay,1=cephfs.smithi157.rcaris=up:active} 1 up:standby 2024-04-22T23:32:37.204 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:36 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:32:37.205 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:36 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:32:37.205 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:36 smithi107 ceph-mon[26355]: pgmap v185: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s wr, 8 op/s 2024-04-22T23:32:38.133 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:37 smithi157 ceph-mon[33877]: mds.? [v2:172.21.15.107:6828/3335313253,v1:172.21.15.107:6829/3335313253] up:resolve 2024-04-22T23:32:38.134 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:37 smithi157 ceph-mon[33877]: mds.? [v2:172.21.15.107:6826/3548717247,v1:172.21.15.107:6827/3548717247] up:boot 2024-04-22T23:32:38.134 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:37 smithi157 ceph-mon[33877]: fsmap cephfs:2/2 {0=cephfs.smithi107.ktuxjl=up:resolve,1=cephfs.smithi157.rcaris=up:active} 2 up:standby 2024-04-22T23:32:38.134 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:37 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.dfvvkw"}]: dispatch 2024-04-22T23:32:38.134 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:37 smithi157 ceph-mon[33877]: mds.? [v2:172.21.15.107:6828/3335313253,v1:172.21.15.107:6829/3335313253] up:reconnect 2024-04-22T23:32:38.134 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:37 smithi157 ceph-mon[33877]: mds.? [v2:172.21.15.157:6826/4072444199,v1:172.21.15.157:6827/4072444199] up:standby 2024-04-22T23:32:38.134 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:37 smithi157 ceph-mon[33877]: fsmap cephfs:2/2 {0=cephfs.smithi107.ktuxjl=up:reconnect,1=cephfs.smithi157.rcaris=up:active} 2 up:standby 2024-04-22T23:32:38.194 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:37 smithi107 ceph-mon[26355]: mds.? [v2:172.21.15.107:6828/3335313253,v1:172.21.15.107:6829/3335313253] up:resolve 2024-04-22T23:32:38.194 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:37 smithi107 ceph-mon[26355]: mds.? [v2:172.21.15.107:6826/3548717247,v1:172.21.15.107:6827/3548717247] up:boot 2024-04-22T23:32:38.194 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:37 smithi107 ceph-mon[26355]: fsmap cephfs:2/2 {0=cephfs.smithi107.ktuxjl=up:resolve,1=cephfs.smithi157.rcaris=up:active} 2 up:standby 2024-04-22T23:32:38.194 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:37 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.dfvvkw"}]: dispatch 2024-04-22T23:32:38.194 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:37 smithi107 ceph-mon[26355]: mds.? [v2:172.21.15.107:6828/3335313253,v1:172.21.15.107:6829/3335313253] up:reconnect 2024-04-22T23:32:38.194 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:37 smithi107 ceph-mon[26355]: mds.? [v2:172.21.15.157:6826/4072444199,v1:172.21.15.157:6827/4072444199] up:standby 2024-04-22T23:32:38.194 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:37 smithi107 ceph-mon[26355]: fsmap cephfs:2/2 {0=cephfs.smithi107.ktuxjl=up:reconnect,1=cephfs.smithi157.rcaris=up:active} 2 up:standby 2024-04-22T23:32:39.035 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:38 smithi157 ceph-mon[33877]: pgmap v186: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-04-22T23:32:39.035 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:38 smithi157 ceph-mon[33877]: mds.? [v2:172.21.15.107:6828/3335313253,v1:172.21.15.107:6829/3335313253] up:rejoin 2024-04-22T23:32:39.035 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:38 smithi157 ceph-mon[33877]: fsmap cephfs:2/2 {0=cephfs.smithi107.ktuxjl=up:rejoin,1=cephfs.smithi157.rcaris=up:active} 2 up:standby 2024-04-22T23:32:39.036 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:38 smithi157 ceph-mon[33877]: daemon mds.cephfs.smithi107.ktuxjl is now active in filesystem cephfs as rank 0 2024-04-22T23:32:39.235 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:38 smithi107 ceph-mon[26355]: pgmap v186: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-04-22T23:32:39.235 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:38 smithi107 ceph-mon[26355]: mds.? [v2:172.21.15.107:6828/3335313253,v1:172.21.15.107:6829/3335313253] up:rejoin 2024-04-22T23:32:39.236 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:38 smithi107 ceph-mon[26355]: fsmap cephfs:2/2 {0=cephfs.smithi107.ktuxjl=up:rejoin,1=cephfs.smithi157.rcaris=up:active} 2 up:standby 2024-04-22T23:32:39.236 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:38 smithi107 ceph-mon[26355]: daemon mds.cephfs.smithi107.ktuxjl is now active in filesystem cephfs as rank 0 2024-04-22T23:32:39.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:39.478+0000 7f8dd59e4700 1 -- 172.21.15.107:0/406948536 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8dd00fbd00 msgr2=0x7f8dd00fc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:39.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:39.478+0000 7f8dd59e4700 1 --2- 172.21.15.107:0/406948536 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8dd00fbd00 0x7f8dd00fc110 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8dc4004940 tx=0x7f8dc4009e30 comp rx=0 tx=0).stop 2024-04-22T23:32:39.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:39.478+0000 7f8dd59e4700 1 -- 172.21.15.107:0/406948536 shutdown_connections 2024-04-22T23:32:39.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:39.478+0000 7f8dd59e4700 1 --2- 172.21.15.107:0/406948536 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8dd00fcfb0 0x7f8dd00fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:39.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:39.478+0000 7f8dd59e4700 1 --2- 172.21.15.107:0/406948536 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8dd00fbd00 0x7f8dd00fc110 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:39.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:39.478+0000 7f8dd59e4700 1 -- 172.21.15.107:0/406948536 >> 172.21.15.107:0/406948536 conn(0x7f8dd00f7480 msgr2=0x7f8dd00f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:32:39.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:39.478+0000 7f8dd59e4700 1 -- 172.21.15.107:0/406948536 shutdown_connections 2024-04-22T23:32:39.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:39.478+0000 7f8dd59e4700 1 -- 172.21.15.107:0/406948536 wait complete. 2024-04-22T23:32:39.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:39.479+0000 7f8dd59e4700 1 Processor -- start 2024-04-22T23:32:39.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:39.479+0000 7f8dd59e4700 1 -- start start 2024-04-22T23:32:39.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:39.479+0000 7f8dd59e4700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8dd00fbd00 0x7f8dd00ffba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:39.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:39.479+0000 7f8dd59e4700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8dd00fcfb0 0x7f8dd00fe2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:39.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:39.479+0000 7f8dd59e4700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd01001a0 con 0x7f8dd00fcfb0 2024-04-22T23:32:39.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:39.479+0000 7f8dd59e4700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd00fe830 con 0x7f8dd00fbd00 2024-04-22T23:32:39.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:39.479+0000 7f8dce7fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8dd00fcfb0 0x7f8dd00fe2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:39.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:39.479+0000 7f8dce7fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8dd00fcfb0 0x7f8dd00fe2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36746/0 (socket says 172.21.15.107:36746) 2024-04-22T23:32:39.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:39.479+0000 7f8dce7fc700 1 -- 172.21.15.107:0/3995670742 learned_addr learned my addr 172.21.15.107:0/3995670742 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:32:39.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:39.479+0000 7f8dceffd700 1 --2- 172.21.15.107:0/3995670742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8dd00fbd00 0x7f8dd00ffba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:39.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:39.479+0000 7f8dce7fc700 1 -- 172.21.15.107:0/3995670742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8dd00fbd00 msgr2=0x7f8dd00ffba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:39.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:39.479+0000 7f8dce7fc700 1 --2- 172.21.15.107:0/3995670742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8dd00fbd00 0x7f8dd00ffba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:39.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:39.479+0000 7f8dce7fc700 1 -- 172.21.15.107:0/3995670742 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc4005040 con 0x7f8dd00fcfb0 2024-04-22T23:32:39.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:39.479+0000 7f8dce7fc700 1 --2- 172.21.15.107:0/3995670742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8dd00fcfb0 0x7f8dd00fe2f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8dc000eb40 tx=0x7f8dc000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:32:39.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:39.479+0000 7f8db77fe700 1 -- 172.21.15.107:0/3995670742 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8dc0007cc0 con 0x7f8dd00fcfb0 2024-04-22T23:32:39.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:39.480+0000 7f8dd59e4700 1 -- 172.21.15.107:0/3995670742 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dd00feab0 con 0x7f8dd00fcfb0 2024-04-22T23:32:39.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:39.480+0000 7f8db77fe700 1 -- 172.21.15.107:0/3995670742 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8dc000f040 con 0x7f8dd00fcfb0 2024-04-22T23:32:39.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:39.480+0000 7f8db77fe700 1 -- 172.21.15.107:0/3995670742 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8dc000ca10 con 0x7f8dd00fcfb0 2024-04-22T23:32:39.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:39.480+0000 7f8dd59e4700 1 -- 172.21.15.107:0/3995670742 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dd00feff0 con 0x7f8dd00fcfb0 2024-04-22T23:32:39.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:39.480+0000 7f8dd59e4700 1 -- 172.21.15.107:0/3995670742 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dd004c710 con 0x7f8dd00fcfb0 2024-04-22T23:32:39.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:39.481+0000 7f8db77fe700 1 -- 172.21.15.107:0/3995670742 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f8dc000cc80 con 0x7f8dd00fcfb0 2024-04-22T23:32:39.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:39.481+0000 7f8db77fe700 1 --2- 172.21.15.107:0/3995670742 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f8db80692a0 0x7f8db806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:39.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:39.481+0000 7f8db77fe700 1 -- 172.21.15.107:0/3995670742 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f8dc0086e00 con 0x7f8dd00fcfb0 2024-04-22T23:32:39.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:39.481+0000 7f8dceffd700 1 --2- 172.21.15.107:0/3995670742 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f8db80692a0 0x7f8db806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:39.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:39.482+0000 7f8dceffd700 1 --2- 172.21.15.107:0/3995670742 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f8db80692a0 0x7f8db806b750 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8dc40045a0 tx=0x7f8dc4009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:32:39.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:39.486+0000 7f8db77fe700 1 -- 172.21.15.107:0/3995670742 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8dc004e250 con 0x7f8dd00fcfb0 2024-04-22T23:32:39.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:39.733+0000 7f8dd59e4700 1 -- 172.21.15.107:0/3995670742 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8dd002b2a0 con 0x7f8dd00fcfb0 2024-04-22T23:32:39.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:39.733+0000 7f8db77fe700 1 -- 172.21.15.107:0/3995670742 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 16 v16) v1 ==== 76+0+1824 (secure 0 0 0) 0x7f8dc0056ba0 con 0x7f8dd00fcfb0 2024-04-22T23:32:39.729 INFO:teuthology.orchestra.run.smithi107.stdout:e16 2024-04-22T23:32:39.730 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T23:32:39.730 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:32:39.730 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-04-22T23:32:39.730 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:32:39.730 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-04-22T23:32:39.730 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-04-22T23:32:39.730 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 16 2024-04-22T23:32:39.730 INFO:teuthology.orchestra.run.smithi107.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-22T23:32:39.730 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-04-22T23:32:20.146130+0000 2024-04-22T23:32:39.730 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-04-22T23:32:38.773323+0000 2024-04-22T23:32:39.730 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-04-22T23:32:39.730 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-04-22T23:32:39.730 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-04-22T23:32:39.730 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-04-22T23:32:39.731 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-04-22T23:32:39.731 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-04-22T23:32:39.731 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-04-22T23:32:39.731 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 46 2024-04-22T23:32:39.731 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:32:39.731 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 2 2024-04-22T23:32:39.731 INFO:teuthology.orchestra.run.smithi107.stdout:in 0,1 2024-04-22T23:32:39.731 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=14468,1=24265} 2024-04-22T23:32:39.731 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-04-22T23:32:39.731 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-04-22T23:32:39.731 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-04-22T23:32:39.731 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-04-22T23:32:39.731 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-04-22T23:32:39.731 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data disabled 2024-04-22T23:32:39.731 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-04-22T23:32:39.732 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-04-22T23:32:39.732 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.ktuxjl{0:14468} state up:rejoin seq 5 join_fscid=1 addr [v2:172.21.15.107:6828/3335313253,v1:172.21.15.107:6829/3335313253] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:32:39.732 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi157.rcaris{1:24265} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.157:6824/2779251893,v1:172.21.15.157:6825/2779251893] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:32:39.732 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:32:39.732 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:32:39.732 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-04-22T23:32:39.732 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:32:39.732 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.dfvvkw{-1:14484} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/3548717247,v1:172.21.15.107:6827/3548717247] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:32:39.732 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi157.ufywse{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.157:6826/4072444199,v1:172.21.15.157:6827/4072444199] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:32:39.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:39.736+0000 7f8dd59e4700 1 -- 172.21.15.107:0/3995670742 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f8db80692a0 msgr2=0x7f8db806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:39.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:39.736+0000 7f8dd59e4700 1 --2- 172.21.15.107:0/3995670742 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f8db80692a0 0x7f8db806b750 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8dc40045a0 tx=0x7f8dc4009e30 comp rx=0 tx=0).stop 2024-04-22T23:32:39.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:39.736+0000 7f8dd59e4700 1 -- 172.21.15.107:0/3995670742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8dd00fcfb0 msgr2=0x7f8dd00fe2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:39.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:39.736+0000 7f8dd59e4700 1 --2- 172.21.15.107:0/3995670742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8dd00fcfb0 0x7f8dd00fe2f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8dc000eb40 tx=0x7f8dc000b040 comp rx=0 tx=0).stop 2024-04-22T23:32:39.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:39.736+0000 7f8dd59e4700 1 -- 172.21.15.107:0/3995670742 shutdown_connections 2024-04-22T23:32:39.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:39.736+0000 7f8dd59e4700 1 --2- 172.21.15.107:0/3995670742 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f8db80692a0 0x7f8db806b750 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:39.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:39.736+0000 7f8dd59e4700 1 --2- 172.21.15.107:0/3995670742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8dd00fbd00 0x7f8dd00ffba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:39.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:39.736+0000 7f8dd59e4700 1 --2- 172.21.15.107:0/3995670742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8dd00fcfb0 0x7f8dd00fe2f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:39.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:39.736+0000 7f8dd59e4700 1 -- 172.21.15.107:0/3995670742 >> 172.21.15.107:0/3995670742 conn(0x7f8dd00f7480 msgr2=0x7f8dd00668b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:32:39.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:39.736+0000 7f8dd59e4700 1 -- 172.21.15.107:0/3995670742 shutdown_connections 2024-04-22T23:32:39.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:39.736+0000 7f8dd59e4700 1 -- 172.21.15.107:0/3995670742 wait complete. 2024-04-22T23:32:39.734 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 16 2024-04-22T23:32:39.985 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:39 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/3995670742' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:32:39.985 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:39 smithi107 ceph-mon[26355]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-22T23:32:39.985 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:39 smithi107 ceph-mon[26355]: Cluster is now healthy 2024-04-22T23:32:39.985 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:39 smithi107 ceph-mon[26355]: mds.? [v2:172.21.15.107:6828/3335313253,v1:172.21.15.107:6829/3335313253] up:active 2024-04-22T23:32:39.985 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:39 smithi107 ceph-mon[26355]: fsmap cephfs:2 {0=cephfs.smithi107.ktuxjl=up:active,1=cephfs.smithi157.rcaris=up:active} 2 up:standby 2024-04-22T23:32:40.133 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:39 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/3995670742' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:32:40.133 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:39 smithi157 ceph-mon[33877]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-22T23:32:40.133 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:39 smithi157 ceph-mon[33877]: Cluster is now healthy 2024-04-22T23:32:40.133 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:39 smithi157 ceph-mon[33877]: mds.? [v2:172.21.15.107:6828/3335313253,v1:172.21.15.107:6829/3335313253] up:active 2024-04-22T23:32:40.133 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:39 smithi157 ceph-mon[33877]: fsmap cephfs:2 {0=cephfs.smithi107.ktuxjl=up:active,1=cephfs.smithi157.rcaris=up:active} 2 up:standby 2024-04-22T23:32:40.908 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:40 smithi107 ceph-mon[26355]: pgmap v187: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 4 op/s 2024-04-22T23:32:40.933 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-04-22T23:32:41.133 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:40 smithi157 ceph-mon[33877]: pgmap v187: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 4 op/s 2024-04-22T23:32:41.669 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:32:43.163 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:43 smithi107 ceph-mon[26355]: pgmap v188: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-22T23:32:43.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:43.183+0000 7fe37edfd700 1 -- 172.21.15.107:0/3357298520 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3780fbd00 msgr2=0x7fe3780fc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:43.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:43.183+0000 7fe37edfd700 1 --2- 172.21.15.107:0/3357298520 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3780fbd00 0x7fe3780fc110 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe36c004940 tx=0x7fe36c009e30 comp rx=0 tx=0).stop 2024-04-22T23:32:43.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:43.183+0000 7fe37edfd700 1 -- 172.21.15.107:0/3357298520 shutdown_connections 2024-04-22T23:32:43.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:43.183+0000 7fe37edfd700 1 --2- 172.21.15.107:0/3357298520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3780fcfb0 0x7fe3780fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:43.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:43.183+0000 7fe37edfd700 1 --2- 172.21.15.107:0/3357298520 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3780fbd00 0x7fe3780fc110 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:43.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:43.183+0000 7fe37edfd700 1 -- 172.21.15.107:0/3357298520 >> 172.21.15.107:0/3357298520 conn(0x7fe3780f7480 msgr2=0x7fe3780f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:32:43.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:43.184+0000 7fe37edfd700 1 -- 172.21.15.107:0/3357298520 shutdown_connections 2024-04-22T23:32:43.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:43.184+0000 7fe37edfd700 1 -- 172.21.15.107:0/3357298520 wait complete. 2024-04-22T23:32:43.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:43.184+0000 7fe37edfd700 1 Processor -- start 2024-04-22T23:32:43.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:43.184+0000 7fe37edfd700 1 -- start start 2024-04-22T23:32:43.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:43.184+0000 7fe37edfd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3780fbd00 0x7fe378066d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:43.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:43.184+0000 7fe37edfd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3780fcfb0 0x7fe378114bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:43.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:43.184+0000 7fe37edfd700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe37810e5b0 con 0x7fe3780fcfb0 2024-04-22T23:32:43.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:43.184+0000 7fe37edfd700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe378115110 con 0x7fe3780fbd00 2024-04-22T23:32:43.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:43.184+0000 7fe377fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3780fcfb0 0x7fe378114bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:43.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:43.184+0000 7fe377fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3780fcfb0 0x7fe378114bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34860/0 (socket says 172.21.15.107:34860) 2024-04-22T23:32:43.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:43.184+0000 7fe377fff700 1 -- 172.21.15.107:0/3527654734 learned_addr learned my addr 172.21.15.107:0/3527654734 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:32:43.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:43.185+0000 7fe37cb99700 1 --2- 172.21.15.107:0/3527654734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3780fbd00 0x7fe378066d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:43.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:43.185+0000 7fe377fff700 1 -- 172.21.15.107:0/3527654734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3780fbd00 msgr2=0x7fe378066d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:43.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:43.185+0000 7fe377fff700 1 --2- 172.21.15.107:0/3527654734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3780fbd00 0x7fe378066d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:43.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:43.185+0000 7fe377fff700 1 -- 172.21.15.107:0/3527654734 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe36c005040 con 0x7fe3780fcfb0 2024-04-22T23:32:43.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:43.185+0000 7fe377fff700 1 --2- 172.21.15.107:0/3527654734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3780fcfb0 0x7fe378114bd0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe36400b7c0 tx=0x7fe364009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:32:43.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:43.185+0000 7fe3757fa700 1 -- 172.21.15.107:0/3527654734 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe36400f6f0 con 0x7fe3780fcfb0 2024-04-22T23:32:43.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:43.185+0000 7fe37edfd700 1 -- 172.21.15.107:0/3527654734 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe378115370 con 0x7fe3780fcfb0 2024-04-22T23:32:43.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:43.185+0000 7fe3757fa700 1 -- 172.21.15.107:0/3527654734 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe36400fd20 con 0x7fe3780fcfb0 2024-04-22T23:32:43.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:43.185+0000 7fe3757fa700 1 -- 172.21.15.107:0/3527654734 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3640103a0 con 0x7fe3780fcfb0 2024-04-22T23:32:43.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:43.185+0000 7fe37edfd700 1 -- 172.21.15.107:0/3527654734 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe378115930 con 0x7fe3780fcfb0 2024-04-22T23:32:43.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:43.186+0000 7fe37edfd700 1 -- 172.21.15.107:0/3527654734 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe37804c710 con 0x7fe3780fcfb0 2024-04-22T23:32:43.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:43.187+0000 7fe3757fa700 1 -- 172.21.15.107:0/3527654734 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fe364011400 con 0x7fe3780fcfb0 2024-04-22T23:32:43.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:43.187+0000 7fe3757fa700 1 --2- 172.21.15.107:0/3527654734 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fe3680692a0 0x7fe36806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:43.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:43.187+0000 7fe3757fa700 1 -- 172.21.15.107:0/3527654734 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fe364086fd0 con 0x7fe3780fcfb0 2024-04-22T23:32:43.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:43.187+0000 7fe37cb99700 1 --2- 172.21.15.107:0/3527654734 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fe3680692a0 0x7fe36806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:43.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:43.187+0000 7fe37cb99700 1 --2- 172.21.15.107:0/3527654734 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fe3680692a0 0x7fe36806b750 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe36c00b3b0 tx=0x7fe36c009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:32:43.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:43.191+0000 7fe3757fa700 1 -- 172.21.15.107:0/3527654734 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe364056ab0 con 0x7fe3780fcfb0 2024-04-22T23:32:43.383 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:43 smithi157 ceph-mon[33877]: pgmap v188: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-22T23:32:43.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:43.438+0000 7fe37edfd700 1 -- 172.21.15.107:0/3527654734 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe378104870 con 0x7fe3780fcfb0 2024-04-22T23:32:43.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:43.438+0000 7fe3757fa700 1 -- 172.21.15.107:0/3527654734 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4715 (secure 0 0 0) 0x7fe3640568d0 con 0x7fe3780fcfb0 2024-04-22T23:32:43.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:43.440+0000 7fe37edfd700 1 -- 172.21.15.107:0/3527654734 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fe3680692a0 msgr2=0x7fe36806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:43.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:43.440+0000 7fe37edfd700 1 --2- 172.21.15.107:0/3527654734 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fe3680692a0 0x7fe36806b750 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe36c00b3b0 tx=0x7fe36c009e30 comp rx=0 tx=0).stop 2024-04-22T23:32:43.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:43.440+0000 7fe37edfd700 1 -- 172.21.15.107:0/3527654734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3780fcfb0 msgr2=0x7fe378114bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:43.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:43.440+0000 7fe37edfd700 1 --2- 172.21.15.107:0/3527654734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3780fcfb0 0x7fe378114bd0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe36400b7c0 tx=0x7fe364009250 comp rx=0 tx=0).stop 2024-04-22T23:32:43.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:43.441+0000 7fe37edfd700 1 -- 172.21.15.107:0/3527654734 shutdown_connections 2024-04-22T23:32:43.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:43.441+0000 7fe37edfd700 1 --2- 172.21.15.107:0/3527654734 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fe3680692a0 0x7fe36806b750 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:43.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:43.441+0000 7fe37edfd700 1 --2- 172.21.15.107:0/3527654734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3780fbd00 0x7fe378066d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:43.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:43.441+0000 7fe37edfd700 1 --2- 172.21.15.107:0/3527654734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3780fcfb0 0x7fe378114bd0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:43.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:43.441+0000 7fe37edfd700 1 -- 172.21.15.107:0/3527654734 >> 172.21.15.107:0/3527654734 conn(0x7fe3780f7480 msgr2=0x7fe3780f8b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:32:43.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:43.441+0000 7fe37edfd700 1 -- 172.21.15.107:0/3527654734 shutdown_connections 2024-04-22T23:32:43.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:43.441+0000 7fe37edfd700 1 -- 172.21.15.107:0/3527654734 wait complete. 2024-04-22T23:32:43.438 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 17 2024-04-22T23:32:43.448 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-04-22T23:32:44.025 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-04-22T23:32:44.619 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:44 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:32:44.619 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:44 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:32:44.619 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:44 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:32:44.619 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:44 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:32:44.619 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:44 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:32:44.620 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:44 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/3527654734' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T23:32:44.620 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:44 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:32:44.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:44 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:32:44.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:44 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:32:44.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:44 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:32:44.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:44 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:32:44.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:44 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:32:44.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:44 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/3527654734' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T23:32:44.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:44 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:32:44.903 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:32:45.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:45 smithi157 ceph-mon[33877]: pgmap v189: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 5.1 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-22T23:32:45.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:45 smithi107 ceph-mon[26355]: pgmap v189: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 5.1 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-22T23:32:46.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:46.551+0000 7f2cc555e700 1 -- 172.21.15.107:0/4072760138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2cc00fc2e0 msgr2=0x7f2cc00fc6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:46.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:46.551+0000 7f2cc555e700 1 --2- 172.21.15.107:0/4072760138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2cc00fc2e0 0x7f2cc00fc6f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8004940 tx=0x7f2cb8009e30 comp rx=0 tx=0).stop 2024-04-22T23:32:46.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:46.551+0000 7f2cc555e700 1 -- 172.21.15.107:0/4072760138 shutdown_connections 2024-04-22T23:32:46.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:46.551+0000 7f2cc555e700 1 --2- 172.21.15.107:0/4072760138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cc00fcc30 0x7f2cc00fd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:46.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:46.551+0000 7f2cc555e700 1 --2- 172.21.15.107:0/4072760138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2cc00fc2e0 0x7f2cc00fc6f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:46.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:46.551+0000 7f2cc555e700 1 -- 172.21.15.107:0/4072760138 >> 172.21.15.107:0/4072760138 conn(0x7f2cc00f6bd0 msgr2=0x7f2cc00f8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:32:46.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:46.551+0000 7f2cc555e700 1 -- 172.21.15.107:0/4072760138 shutdown_connections 2024-04-22T23:32:46.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:46.551+0000 7f2cc555e700 1 -- 172.21.15.107:0/4072760138 wait complete. 2024-04-22T23:32:46.548 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:46 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:32:46.548 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:46 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:32:46.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:46.552+0000 7f2cc555e700 1 Processor -- start 2024-04-22T23:32:46.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:46.552+0000 7f2cc555e700 1 -- start start 2024-04-22T23:32:46.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:46.552+0000 7f2cc555e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2cc00fc2e0 0x7f2cc010d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:46.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:46.552+0000 7f2cc555e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cc00fcc30 0x7f2cc010dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:46.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:46.552+0000 7f2cc555e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc010e4e0 con 0x7f2cc00fc2e0 2024-04-22T23:32:46.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:46.552+0000 7f2cc555e700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc010ecc0 con 0x7f2cc00fcc30 2024-04-22T23:32:46.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:46.552+0000 7f2cbeffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2cc00fc2e0 0x7f2cc010d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:46.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:46.552+0000 7f2cbeffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2cc00fc2e0 0x7f2cc010d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34872/0 (socket says 172.21.15.107:34872) 2024-04-22T23:32:46.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:46.552+0000 7f2cbeffd700 1 -- 172.21.15.107:0/1040957715 learned_addr learned my addr 172.21.15.107:0/1040957715 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:32:46.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:46.552+0000 7f2cbe7fc700 1 --2- 172.21.15.107:0/1040957715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cc00fcc30 0x7f2cc010dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:46.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:46.552+0000 7f2cbeffd700 1 -- 172.21.15.107:0/1040957715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cc00fcc30 msgr2=0x7f2cc010dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:46.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:46.552+0000 7f2cbeffd700 1 --2- 172.21.15.107:0/1040957715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cc00fcc30 0x7f2cc010dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:46.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:46.552+0000 7f2cbeffd700 1 -- 172.21.15.107:0/1040957715 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb8005040 con 0x7f2cc00fc2e0 2024-04-22T23:32:46.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:46.553+0000 7f2cbeffd700 1 --2- 172.21.15.107:0/1040957715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2cc00fc2e0 0x7f2cc010d9a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8013870 tx=0x7f2cb8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:32:46.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:46.553+0000 7f2ca77fe700 1 -- 172.21.15.107:0/1040957715 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2cb8021070 con 0x7f2cc00fc2e0 2024-04-22T23:32:46.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:46.553+0000 7f2ca77fe700 1 -- 172.21.15.107:0/1040957715 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2cb801c400 con 0x7f2cc00fc2e0 2024-04-22T23:32:46.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:46.553+0000 7f2ca77fe700 1 -- 172.21.15.107:0/1040957715 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2cb800e070 con 0x7f2cc00fc2e0 2024-04-22T23:32:46.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:46.553+0000 7f2cc555e700 1 -- 172.21.15.107:0/1040957715 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cc010ef40 con 0x7f2cc00fc2e0 2024-04-22T23:32:46.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:46.553+0000 7f2cc555e700 1 -- 172.21.15.107:0/1040957715 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cc010f480 con 0x7f2cc00fc2e0 2024-04-22T23:32:46.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:46.554+0000 7f2cc555e700 1 -- 172.21.15.107:0/1040957715 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cc0107350 con 0x7f2cc00fc2e0 2024-04-22T23:32:46.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:46.554+0000 7f2ca77fe700 1 -- 172.21.15.107:0/1040957715 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f2cb801ca20 con 0x7f2cc00fc2e0 2024-04-22T23:32:46.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:46.555+0000 7f2ca77fe700 1 --2- 172.21.15.107:0/1040957715 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f2ca80692a0 0x7f2ca806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:46.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:46.555+0000 7f2ca77fe700 1 -- 172.21.15.107:0/1040957715 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f2cb8088510 con 0x7f2cc00fc2e0 2024-04-22T23:32:46.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:46.555+0000 7f2cbe7fc700 1 --2- 172.21.15.107:0/1040957715 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f2ca80692a0 0x7f2ca806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:46.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:46.555+0000 7f2cbe7fc700 1 --2- 172.21.15.107:0/1040957715 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f2ca80692a0 0x7f2ca806b750 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4004b50 tx=0x7f2cb400b040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:32:46.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:46.559+0000 7f2ca77fe700 1 -- 172.21.15.107:0/1040957715 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f2cb8058320 con 0x7f2cc00fc2e0 2024-04-22T23:32:46.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:46.828+0000 7f2cc555e700 1 -- 172.21.15.107:0/1040957715 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7f2cc01048b0 con 0x7f2cc00fc2e0 2024-04-22T23:32:46.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:46.829+0000 7f2ca77fe700 1 -- 172.21.15.107:0/1040957715 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v17) v1 ==== 78+0+99 (secure 0 0 0) 0x7f2cb8004490 con 0x7f2cc00fc2e0 2024-04-22T23:32:46.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:46.831+0000 7f2cc555e700 1 -- 172.21.15.107:0/1040957715 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f2ca80692a0 msgr2=0x7f2ca806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:46.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:46.831+0000 7f2cc555e700 1 --2- 172.21.15.107:0/1040957715 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f2ca80692a0 0x7f2ca806b750 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4004b50 tx=0x7f2cb400b040 comp rx=0 tx=0).stop 2024-04-22T23:32:46.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:46.831+0000 7f2cc555e700 1 -- 172.21.15.107:0/1040957715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2cc00fc2e0 msgr2=0x7f2cc010d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:46.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:46.831+0000 7f2cc555e700 1 --2- 172.21.15.107:0/1040957715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2cc00fc2e0 0x7f2cc010d9a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8013870 tx=0x7f2cb8009e30 comp rx=0 tx=0).stop 2024-04-22T23:32:46.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:46.831+0000 7f2cc555e700 1 -- 172.21.15.107:0/1040957715 shutdown_connections 2024-04-22T23:32:46.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:46.831+0000 7f2cc555e700 1 --2- 172.21.15.107:0/1040957715 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f2ca80692a0 0x7f2ca806b750 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:46.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:46.831+0000 7f2cc555e700 1 --2- 172.21.15.107:0/1040957715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2cc00fc2e0 0x7f2cc010d9a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:46.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:46.831+0000 7f2cc555e700 1 --2- 172.21.15.107:0/1040957715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2cc00fcc30 0x7f2cc010dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:46.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:46.831+0000 7f2cc555e700 1 -- 172.21.15.107:0/1040957715 >> 172.21.15.107:0/1040957715 conn(0x7f2cc00f6bd0 msgr2=0x7f2cc00f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:32:46.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:46.831+0000 7f2cc555e700 1 -- 172.21.15.107:0/1040957715 shutdown_connections 2024-04-22T23:32:46.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:46.831+0000 7f2cc555e700 1 -- 172.21.15.107:0/1040957715 wait complete. 2024-04-22T23:32:46.837 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-04-22T23:32:46.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:46 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:32:46.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:46 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:32:47.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:47 smithi157 ceph-mon[33877]: pgmap v190: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 5.0 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-22T23:32:47.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:47 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/1040957715' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-04-22T23:32:47.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:47 smithi107 ceph-mon[26355]: pgmap v190: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 5.0 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-22T23:32:47.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:47 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/1040957715' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-04-22T23:32:47.779 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-04-22T23:32:47.792 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T23:32:47.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:47.903+0000 7f0ede91d640 1 -- 172.21.15.107:0/3301586981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ed80fb0a0 msgr2=0x7f0ed80fb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:47.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:47.903+0000 7f0ede91d640 1 --2- 172.21.15.107:0/3301586981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ed80fb0a0 0x7f0ed80fb4b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc0048d0 tx=0x7f0ecc009da0 comp rx=0 tx=0).stop 2024-04-22T23:32:47.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:47.903+0000 7f0ede91d640 1 -- 172.21.15.107:0/3301586981 shutdown_connections 2024-04-22T23:32:47.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:47.903+0000 7f0ede91d640 1 --2- 172.21.15.107:0/3301586981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ed80fc370 0x7f0ed80fc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:47.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:47.903+0000 7f0ede91d640 1 --2- 172.21.15.107:0/3301586981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ed80fb0a0 0x7f0ed80fb4b0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:47.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:47.903+0000 7f0ede91d640 1 -- 172.21.15.107:0/3301586981 >> 172.21.15.107:0/3301586981 conn(0x7f0ed80f6840 msgr2=0x7f0ed80f8c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:32:47.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:47.903+0000 7f0ede91d640 1 -- 172.21.15.107:0/3301586981 shutdown_connections 2024-04-22T23:32:47.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:47.903+0000 7f0ede91d640 1 -- 172.21.15.107:0/3301586981 wait complete. 2024-04-22T23:32:47.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:47.903+0000 7f0ede91d640 1 Processor -- start 2024-04-22T23:32:47.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:47.903+0000 7f0ede91d640 1 -- start start 2024-04-22T23:32:47.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:47.903+0000 7f0ede91d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ed80fb0a0 0x7f0ed810cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:47.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:47.903+0000 7f0ede91d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ed80fc370 0x7f0ed810d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:47.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:47.903+0000 7f0ede91d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed810d860 con 0x7f0ed80fb0a0 2024-04-22T23:32:47.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:47.903+0000 7f0ede91d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed8112220 con 0x7f0ed80fc370 2024-04-22T23:32:47.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:47.904+0000 7f0ed7fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ed80fb0a0 0x7f0ed810cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:47.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:47.904+0000 7f0ed7fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ed80fb0a0 0x7f0ed810cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34884/0 (socket says 172.21.15.107:34884) 2024-04-22T23:32:47.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:47.904+0000 7f0ed7fff640 1 -- 172.21.15.107:0/2195203746 learned_addr learned my addr 172.21.15.107:0/2195203746 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:32:47.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:47.904+0000 7f0ed7fff640 1 -- 172.21.15.107:0/2195203746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ed80fc370 msgr2=0x7f0ed810d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:47.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:47.904+0000 7f0ed7fff640 1 --2- 172.21.15.107:0/2195203746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ed80fc370 0x7f0ed810d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:47.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:47.904+0000 7f0ed7fff640 1 -- 172.21.15.107:0/2195203746 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ecc005040 con 0x7f0ed80fb0a0 2024-04-22T23:32:47.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:47.904+0000 7f0ed7fff640 1 --2- 172.21.15.107:0/2195203746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ed80fb0a0 0x7f0ed810cca0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc007d50 tx=0x7f0ecc009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:32:47.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:47.904+0000 7f0ed4ff9640 1 -- 172.21.15.107:0/2195203746 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ecc009590 con 0x7f0ed80fb0a0 2024-04-22T23:32:47.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:47.904+0000 7f0ed4ff9640 1 -- 172.21.15.107:0/2195203746 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0ecc0096f0 con 0x7f0ed80fb0a0 2024-04-22T23:32:47.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:47.904+0000 7f0ede91d640 1 -- 172.21.15.107:0/2195203746 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ed8112360 con 0x7f0ed80fb0a0 2024-04-22T23:32:47.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:47.904+0000 7f0ed4ff9640 1 -- 172.21.15.107:0/2195203746 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ecc03b6d0 con 0x7f0ed80fb0a0 2024-04-22T23:32:47.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:47.904+0000 7f0ede91d640 1 -- 172.21.15.107:0/2195203746 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ed8112850 con 0x7f0ed80fb0a0 2024-04-22T23:32:47.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:47.905+0000 7f0ede91d640 1 -- 172.21.15.107:0/2195203746 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ed804bcc0 con 0x7f0ed80fb0a0 2024-04-22T23:32:47.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:47.905+0000 7f0ed4ff9640 1 -- 172.21.15.107:0/2195203746 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f0ecc02f070 con 0x7f0ed80fb0a0 2024-04-22T23:32:47.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:47.906+0000 7f0ed4ff9640 1 --2- 172.21.15.107:0/2195203746 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f0ec0069210 0x7f0ec006b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:47.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:47.906+0000 7f0ed77fe640 1 --2- 172.21.15.107:0/2195203746 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f0ec0069210 0x7f0ec006b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:47.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:47.906+0000 7f0ed4ff9640 1 -- 172.21.15.107:0/2195203746 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f0ecc0a9800 con 0x7f0ed80fb0a0 2024-04-22T23:32:47.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:47.906+0000 7f0ed77fe640 1 --2- 172.21.15.107:0/2195203746 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f0ec0069210 0x7f0ec006b6c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0ed810e190 tx=0x7f0ec8007cb0 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:32:47.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:47.910+0000 7f0ed4ff9640 1 -- 172.21.15.107:0/2195203746 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0ecc075ae0 con 0x7f0ed80fb0a0 2024-04-22T23:32:48.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.087+0000 7f0ede91d640 1 -- 172.21.15.107:0/2195203746 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ed8103c10 con 0x7f0ed80fb0a0 2024-04-22T23:32:48.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.088+0000 7f0ed4ff9640 1 -- 172.21.15.107:0/2195203746 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+14424 (secure 0 0 0) 0x7f0ecc079100 con 0x7f0ed80fb0a0 2024-04-22T23:32:48.084 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:32:48.084 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":46,"fsid":"8252f6f2-00ff-11ef-bc93-c7b262605968","created":"2024-04-22T23:25:25.497133+0000","modified":"2024-04-22T23:32:35.769487+0000","last_up_change":"2024-04-22T23:31:03.170868+0000","last_in_change":"2024-04-22T23:30:44.396843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T23:29:48.249903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-22T23:32:19.078061+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-22T23:32:19.378987+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"65404290-734a-4901-bbb4-a87d3d846b06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":4253936450},{"type":"v1","addr":"172.21.15.107:6803","nonce":4253936450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":4253936450},{"type":"v1","addr":"172.21.15.107:6805","nonce":4253936450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":4253936450},{"type":"v1","addr":"172.21.15.107:6809","nonce":4253936450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":4253936450},{"type":"v1","addr":"172.21.15.107:6807","nonce":4253936450}]},"public_addr":"172.21.15.107:6803/4253936450","cluster_addr":"172.21.15.107:6805/4253936450","heartbeat_back_addr":"172.21.15.107:6809/4253936450","heartbeat_front_addr":"172.21.15.107:6807/4253936450","state":["exists","up"]},{"osd":1,"uuid":"6df381f7-b907-4d1b-bd08-fdf34e9729f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":335851250},{"type":"v1","addr":"172.21.15.107:6811","nonce":335851250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":335851250},{"type":"v1","addr":"172.21.15.107:6813","nonce":335851250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":335851250},{"type":"v1","addr":"172.21.15.107:6817","nonce":335851250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":335851250},{"type":"v1","addr":"172.21.15.107:6815","nonce":335851250}]},"public_addr":"172.21.15.107:6811/335851250","cluster_addr":"172.21.15.107:6813/335851250","heartbeat_back_addr":"172.21.15.107:6817/335851250","heartbeat_front_addr":"172.21.15.107:6815/335851250","state":["exists","up"]},{"osd":2,"uuid":"2c5e5c37-c1de-4e64-9c21-eacc145f1412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":750102033},{"type":"v1","addr":"172.21.15.107:6819","nonce":750102033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":750102033},{"type":"v1","addr":"172.21.15.107:6821","nonce":750102033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":750102033},{"type":"v1","addr":"172.21.15.107:6825","nonce":750102033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":750102033},{"type":"v1","addr":"172.21.15.107:6823","nonce":750102033}]},"public_addr":"172.21.15.107:6819/750102033","cluster_addr":"172.21.15.107:6821/750102033","heartbeat_back_addr":"172.21.15.107:6825/750102033","heartbeat_front_addr":"172.21.15.107:6823/750102033","state":["exists","up"]},{"osd":3,"uuid":"b7c9c124-9f43-4b5b-b039-c61ae30a416d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":811815812},{"type":"v1","addr":"172.21.15.157:6801","nonce":811815812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":811815812},{"type":"v1","addr":"172.21.15.157:6803","nonce":811815812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":811815812},{"type":"v1","addr":"172.21.15.157:6807","nonce":811815812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":811815812},{"type":"v1","addr":"172.21.15.157:6805","nonce":811815812}]},"public_addr":"172.21.15.157:6801/811815812","cluster_addr":"172.21.15.157:6803/811815812","heartbeat_back_addr":"172.21.15.157:6807/811815812","heartbeat_front_addr":"172.21.15.157:6805/811815812","state":["exists","up"]},{"osd":4,"uuid":"34a0cb61-1c2b-4515-9d74-8a3ad3f15fbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2372899121},{"type":"v1","addr":"172.21.15.157:6809","nonce":2372899121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2372899121},{"type":"v1","addr":"172.21.15.157:6811","nonce":2372899121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2372899121},{"type":"v1","addr":"172.21.15.157:6815","nonce":2372899121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2372899121},{"type":"v1","addr":"172.21.15.157:6813","nonce":2372899121}]},"public_addr":"172.21.15.157:6809/2372899121","cluster_addr":"172.21.15.157:6811/2372899121","heartbeat_back_addr":"172.21.15.157:6815/2372899121","heartbeat_front_addr":"172.21.15.157:6813/2372899121","state":["exists","up"]},{"osd":5,"uuid":"00934a7f-7e88-45cf-a2cc-e8a94bceaab0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2125627920},{"type":"v1","addr":"172.21.15.157:6817","nonce":2125627920}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":2125627920},{"type":"v1","addr":"172.21.15.157:6819","nonce":2125627920}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":2125627920},{"type":"v1","addr":"172.21.15.157:6823","nonce":2125627920}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":2125627920},{"type":"v1","addr":"172.21.15.157:6821","nonce":2125627920}]},"public_addr":"172.21.15.157:6817/2125627920","cluster_addr":"172.21.15.157:6819/2125627920","heartbeat_back_addr":"172.21.15.157:6823/2125627920","heartbeat_front_addr":"172.21.15.157:6821/2125627920","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:28:48.161289+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:29:18.447013+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:29:45.948165+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:30:09.825103+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:30:35.976009+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:31:01.729305+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:6827/4144299942":"2024-04-23T23:32:35.769448+0000","172.21.15.107:0/663073386":"2024-04-23T23:26:29.807373+0000","172.21.15.107:0/3920905056":"2024-04-23T23:25:58.816560+0000","172.21.15.107:0/1734120985":"2024-04-23T23:27:33.931895+0000","172.21.15.107:6826/4144299942":"2024-04-23T23:32:35.769448+0000","172.21.15.107:0/3377956937":"2024-04-23T23:25:58.816560+0000","172.21.15.107:6801/467262774":"2024-04-23T23:25:58.816560+0000","172.21.15.107:0/3608791958":"2024-04-23T23:27:33.931895+0000","172.21.15.107:0/2444490678":"2024-04-23T23:26:29.807373+0000","172.21.15.107:0/1205692115":"2024-04-23T23:27:33.931895+0000","172.21.15.107:0/3810721822":"2024-04-23T23:25:58.816560+0000","172.21.15.107:0/1029440409":"2024-04-23T23:26:29.807373+0000","172.21.15.107:6800/467262774":"2024-04-23T23:25:58.816560+0000","172.21.15.107:6800/307474287":"2024-04-23T23:26:29.807373+0000","172.21.15.107:6801/307474287":"2024-04-23T23:26:29.807373+0000","172.21.15.107:6800/3427816626":"2024-04-23T23:27:33.931895+0000","172.21.15.107:6801/3427816626":"2024-04-23T23:27:33.931895+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T23:32:48.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.090+0000 7f0ede91d640 1 -- 172.21.15.107:0/2195203746 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f0ec0069210 msgr2=0x7f0ec006b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:48.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.090+0000 7f0ede91d640 1 --2- 172.21.15.107:0/2195203746 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f0ec0069210 0x7f0ec006b6c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0ed810e190 tx=0x7f0ec8007cb0 comp rx=0 tx=0).stop 2024-04-22T23:32:48.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.090+0000 7f0ede91d640 1 -- 172.21.15.107:0/2195203746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ed80fb0a0 msgr2=0x7f0ed810cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:48.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.090+0000 7f0ede91d640 1 --2- 172.21.15.107:0/2195203746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ed80fb0a0 0x7f0ed810cca0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc007d50 tx=0x7f0ecc009da0 comp rx=0 tx=0).stop 2024-04-22T23:32:48.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.090+0000 7f0ede91d640 1 -- 172.21.15.107:0/2195203746 shutdown_connections 2024-04-22T23:32:48.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.090+0000 7f0ede91d640 1 --2- 172.21.15.107:0/2195203746 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f0ec0069210 0x7f0ec006b6c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:48.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.090+0000 7f0ede91d640 1 --2- 172.21.15.107:0/2195203746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ed80fc370 0x7f0ed810d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:48.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.090+0000 7f0ede91d640 1 --2- 172.21.15.107:0/2195203746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ed80fb0a0 0x7f0ed810cca0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:48.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.090+0000 7f0ede91d640 1 -- 172.21.15.107:0/2195203746 >> 172.21.15.107:0/2195203746 conn(0x7f0ed80f6840 msgr2=0x7f0ed8064380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:32:48.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.090+0000 7f0ede91d640 1 -- 172.21.15.107:0/2195203746 shutdown_connections 2024-04-22T23:32:48.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.090+0000 7f0ede91d640 1 -- 172.21.15.107:0/2195203746 wait complete. 2024-04-22T23:32:48.097 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T23:29:48.249903+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-22T23:32:19.078061+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-22T23:32:19.378987+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-22T23:32:48.098 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T23:32:48.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.203+0000 7fb01e59d640 1 -- 172.21.15.107:0/2923238282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0180fc370 msgr2=0x7fb0180fc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:48.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.203+0000 7fb01e59d640 1 --2- 172.21.15.107:0/2923238282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0180fc370 0x7fb0180fc7e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb00c0048d0 tx=0x7fb00c009da0 comp rx=0 tx=0).stop 2024-04-22T23:32:48.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.203+0000 7fb01e59d640 1 -- 172.21.15.107:0/2923238282 shutdown_connections 2024-04-22T23:32:48.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.203+0000 7fb01e59d640 1 --2- 172.21.15.107:0/2923238282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0180fc370 0x7fb0180fc7e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:48.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.203+0000 7fb01e59d640 1 --2- 172.21.15.107:0/2923238282 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0180fb0a0 0x7fb0180fb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:48.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.203+0000 7fb01e59d640 1 -- 172.21.15.107:0/2923238282 >> 172.21.15.107:0/2923238282 conn(0x7fb0180f6840 msgr2=0x7fb0180f8c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:32:48.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.203+0000 7fb01e59d640 1 -- 172.21.15.107:0/2923238282 shutdown_connections 2024-04-22T23:32:48.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.203+0000 7fb01e59d640 1 -- 172.21.15.107:0/2923238282 wait complete. 2024-04-22T23:32:48.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.203+0000 7fb01e59d640 1 Processor -- start 2024-04-22T23:32:48.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.203+0000 7fb01e59d640 1 -- start start 2024-04-22T23:32:48.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.204+0000 7fb01e59d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0180fb0a0 0x7fb018064e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:48.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.204+0000 7fb01e59d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0180fc370 0x7fb018113e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:48.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.204+0000 7fb01e59d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01810d810 con 0x7fb0180fc370 2024-04-22T23:32:48.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.204+0000 7fb01e59d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb018114370 con 0x7fb0180fb0a0 2024-04-22T23:32:48.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.204+0000 7fb0177fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0180fc370 0x7fb018113e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:48.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.204+0000 7fb0177fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0180fc370 0x7fb018113e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34908/0 (socket says 172.21.15.107:34908) 2024-04-22T23:32:48.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.204+0000 7fb0177fe640 1 -- 172.21.15.107:0/740862299 learned_addr learned my addr 172.21.15.107:0/740862299 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:32:48.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.204+0000 7fb017fff640 1 --2- 172.21.15.107:0/740862299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0180fb0a0 0x7fb018064e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:48.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.204+0000 7fb0177fe640 1 -- 172.21.15.107:0/740862299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0180fb0a0 msgr2=0x7fb018064e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:48.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.204+0000 7fb0177fe640 1 --2- 172.21.15.107:0/740862299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0180fb0a0 0x7fb018064e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:48.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.204+0000 7fb0177fe640 1 -- 172.21.15.107:0/740862299 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb00c005040 con 0x7fb0180fc370 2024-04-22T23:32:48.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.204+0000 7fb017fff640 1 --2- 172.21.15.107:0/740862299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0180fb0a0 0x7fb018064e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T23:32:48.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.204+0000 7fb0177fe640 1 --2- 172.21.15.107:0/740862299 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0180fc370 0x7fb018113e30 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb00c007d20 tx=0x7fb00c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:32:48.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.204+0000 7fb014ff9640 1 -- 172.21.15.107:0/740862299 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb00c009590 con 0x7fb0180fc370 2024-04-22T23:32:48.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.204+0000 7fb01e59d640 1 -- 172.21.15.107:0/740862299 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb018114510 con 0x7fb0180fc370 2024-04-22T23:32:48.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.204+0000 7fb014ff9640 1 -- 172.21.15.107:0/740862299 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb00c0096f0 con 0x7fb0180fc370 2024-04-22T23:32:48.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.204+0000 7fb01e59d640 1 -- 172.21.15.107:0/740862299 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb018114a30 con 0x7fb0180fc370 2024-04-22T23:32:48.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.204+0000 7fb014ff9640 1 -- 172.21.15.107:0/740862299 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb00c03b6d0 con 0x7fb0180fc370 2024-04-22T23:32:48.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.205+0000 7fb01e59d640 1 -- 172.21.15.107:0/740862299 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb018103bf0 con 0x7fb0180fc370 2024-04-22T23:32:48.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.206+0000 7fb014ff9640 1 -- 172.21.15.107:0/740862299 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fb00c02f070 con 0x7fb0180fc370 2024-04-22T23:32:48.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.206+0000 7fb014ff9640 1 --2- 172.21.15.107:0/740862299 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fb000069210 0x7fb00006b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:48.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.207+0000 7fb014ff9640 1 -- 172.21.15.107:0/740862299 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fb00c0a9960 con 0x7fb0180fc370 2024-04-22T23:32:48.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.207+0000 7fb017fff640 1 --2- 172.21.15.107:0/740862299 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fb000069210 0x7fb00006b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:48.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.207+0000 7fb017fff640 1 --2- 172.21.15.107:0/740862299 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fb000069210 0x7fb00006b6c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb008004c60 tx=0x7fb008009040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:32:48.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.215+0000 7fb014ff9640 1 -- 172.21.15.107:0/740862299 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb00c075c30 con 0x7fb0180fc370 2024-04-22T23:32:48.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.389+0000 7fb01e59d640 1 -- 172.21.15.107:0/740862299 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb018103d80 con 0x7fb0180fc370 2024-04-22T23:32:48.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.389+0000 7fb014ff9640 1 -- 172.21.15.107:0/740862299 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb018103d80 con 0x7fb0180fc370 2024-04-22T23:32:48.385 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2024-04-22T23:32:48.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.391+0000 7fb01e59d640 1 -- 172.21.15.107:0/740862299 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fb000069210 msgr2=0x7fb00006b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:48.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.391+0000 7fb01e59d640 1 --2- 172.21.15.107:0/740862299 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fb000069210 0x7fb00006b6c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb008004c60 tx=0x7fb008009040 comp rx=0 tx=0).stop 2024-04-22T23:32:48.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.391+0000 7fb01e59d640 1 -- 172.21.15.107:0/740862299 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0180fc370 msgr2=0x7fb018113e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:48.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.391+0000 7fb01e59d640 1 --2- 172.21.15.107:0/740862299 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0180fc370 0x7fb018113e30 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb00c007d20 tx=0x7fb00c009da0 comp rx=0 tx=0).stop 2024-04-22T23:32:48.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.391+0000 7fb01e59d640 1 -- 172.21.15.107:0/740862299 shutdown_connections 2024-04-22T23:32:48.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.391+0000 7fb01e59d640 1 --2- 172.21.15.107:0/740862299 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fb000069210 0x7fb00006b6c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:48.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.391+0000 7fb01e59d640 1 --2- 172.21.15.107:0/740862299 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0180fc370 0x7fb018113e30 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:48.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.391+0000 7fb01e59d640 1 --2- 172.21.15.107:0/740862299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0180fb0a0 0x7fb018064e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:48.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.391+0000 7fb01e59d640 1 -- 172.21.15.107:0/740862299 >> 172.21.15.107:0/740862299 conn(0x7fb0180f6840 msgr2=0x7fb0180ffab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:32:48.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.392+0000 7fb01e59d640 1 -- 172.21.15.107:0/740862299 shutdown_connections 2024-04-22T23:32:48.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.392+0000 7fb01e59d640 1 -- 172.21.15.107:0/740862299 wait complete. 2024-04-22T23:32:48.399 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-22T23:32:48.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.508+0000 7f11d831d640 1 -- 172.21.15.107:0/1789307517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11d00fb6a0 msgr2=0x7f11d00fbab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:48.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.508+0000 7f11d831d640 1 --2- 172.21.15.107:0/1789307517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11d00fb6a0 0x7f11d00fbab0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f11cc004810 tx=0x7f11cc009da0 comp rx=0 tx=0).stop 2024-04-22T23:32:48.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.509+0000 7f11d831d640 1 -- 172.21.15.107:0/1789307517 shutdown_connections 2024-04-22T23:32:48.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.509+0000 7f11d831d640 1 --2- 172.21.15.107:0/1789307517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11d00fbff0 0x7f11d00fc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:48.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.509+0000 7f11d831d640 1 --2- 172.21.15.107:0/1789307517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11d00fb6a0 0x7f11d00fbab0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:48.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.509+0000 7f11d831d640 1 -- 172.21.15.107:0/1789307517 >> 172.21.15.107:0/1789307517 conn(0x7f11d00f5ff0 msgr2=0x7f11d00f8420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:32:48.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.509+0000 7f11d831d640 1 -- 172.21.15.107:0/1789307517 shutdown_connections 2024-04-22T23:32:48.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.509+0000 7f11d831d640 1 -- 172.21.15.107:0/1789307517 wait complete. 2024-04-22T23:32:48.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.510+0000 7f11d831d640 1 Processor -- start 2024-04-22T23:32:48.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.510+0000 7f11d831d640 1 -- start start 2024-04-22T23:32:48.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.510+0000 7f11d831d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11d00fb6a0 0x7f11d00fad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:48.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.510+0000 7f11d831d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11d00fbff0 0x7f11d0111ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:48.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.510+0000 7f11d831d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d01120e0 con 0x7f11d00fb6a0 2024-04-22T23:32:48.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.510+0000 7f11d831d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d0112220 con 0x7f11d00fbff0 2024-04-22T23:32:48.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.510+0000 7f11d60b9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11d00fb6a0 0x7f11d00fad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:48.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.510+0000 7f11d60b9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11d00fb6a0 0x7f11d00fad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34926/0 (socket says 172.21.15.107:34926) 2024-04-22T23:32:48.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.510+0000 7f11d60b9640 1 -- 172.21.15.107:0/1239358663 learned_addr learned my addr 172.21.15.107:0/1239358663 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:32:48.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.510+0000 7f11d58b8640 1 --2- 172.21.15.107:0/1239358663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11d00fbff0 0x7f11d0111ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:48.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.511+0000 7f11d60b9640 1 -- 172.21.15.107:0/1239358663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11d00fbff0 msgr2=0x7f11d0111ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:48.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.511+0000 7f11d60b9640 1 --2- 172.21.15.107:0/1239358663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11d00fbff0 0x7f11d0111ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:48.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.511+0000 7f11d60b9640 1 -- 172.21.15.107:0/1239358663 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11cc005040 con 0x7f11d00fb6a0 2024-04-22T23:32:48.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.511+0000 7f11d58b8640 1 --2- 172.21.15.107:0/1239358663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11d00fbff0 0x7f11d0111ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T23:32:48.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.511+0000 7f11d60b9640 1 --2- 172.21.15.107:0/1239358663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11d00fb6a0 0x7f11d00fad50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f11cc00b020 tx=0x7f11cc009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:32:48.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.511+0000 7f11c6ffd640 1 -- 172.21.15.107:0/1239358663 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11cc0090d0 con 0x7f11d00fb6a0 2024-04-22T23:32:48.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.511+0000 7f11d831d640 1 -- 172.21.15.107:0/1239358663 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11d0112420 con 0x7f11d00fb6a0 2024-04-22T23:32:48.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.511+0000 7f11c6ffd640 1 -- 172.21.15.107:0/1239358663 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f11cc03d8e0 con 0x7f11d00fb6a0 2024-04-22T23:32:48.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.511+0000 7f11c6ffd640 1 -- 172.21.15.107:0/1239358663 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11cc032710 con 0x7f11d00fb6a0 2024-04-22T23:32:48.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.511+0000 7f11d831d640 1 -- 172.21.15.107:0/1239358663 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11d0112860 con 0x7f11d00fb6a0 2024-04-22T23:32:48.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.513+0000 7f11c6ffd640 1 -- 172.21.15.107:0/1239358663 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f11cc03da50 con 0x7f11d00fb6a0 2024-04-22T23:32:48.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.513+0000 7f11d831d640 1 -- 172.21.15.107:0/1239358663 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11d01055a0 con 0x7f11d00fb6a0 2024-04-22T23:32:48.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.513+0000 7f11c6ffd640 1 --2- 172.21.15.107:0/1239358663 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f11bc069210 0x7f11bc06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:48.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.513+0000 7f11c6ffd640 1 -- 172.21.15.107:0/1239358663 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f11cc0a99b0 con 0x7f11d00fb6a0 2024-04-22T23:32:48.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.513+0000 7f11d58b8640 1 --2- 172.21.15.107:0/1239358663 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f11bc069210 0x7f11bc06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:48.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.514+0000 7f11d58b8640 1 --2- 172.21.15.107:0/1239358663 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f11bc069210 0x7f11bc06b6c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f11d00fab00 tx=0x7f11c000b040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:32:48.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.521+0000 7f11c6ffd640 1 -- 172.21.15.107:0/1239358663 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f11cc075c90 con 0x7f11d00fb6a0 2024-04-22T23:32:48.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.700+0000 7f11d831d640 1 -- 172.21.15.107:0/1239358663 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f11d010bc20 con 0x7f11d00fb6a0 2024-04-22T23:32:48.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.701+0000 7f11c6ffd640 1 -- 172.21.15.107:0/1239358663 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v46) v1 ==== 107+0+11 (secure 0 0 0) 0x7f11cc0792b0 con 0x7f11d00fb6a0 2024-04-22T23:32:48.697 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-04-22T23:32:48.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.703+0000 7f11d831d640 1 -- 172.21.15.107:0/1239358663 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f11bc069210 msgr2=0x7f11bc06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:48.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.703+0000 7f11d831d640 1 --2- 172.21.15.107:0/1239358663 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f11bc069210 0x7f11bc06b6c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f11d00fab00 tx=0x7f11c000b040 comp rx=0 tx=0).stop 2024-04-22T23:32:48.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.703+0000 7f11d831d640 1 -- 172.21.15.107:0/1239358663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11d00fb6a0 msgr2=0x7f11d00fad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:48.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.703+0000 7f11d831d640 1 --2- 172.21.15.107:0/1239358663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11d00fb6a0 0x7f11d00fad50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f11cc00b020 tx=0x7f11cc009da0 comp rx=0 tx=0).stop 2024-04-22T23:32:48.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.703+0000 7f11d831d640 1 -- 172.21.15.107:0/1239358663 shutdown_connections 2024-04-22T23:32:48.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.703+0000 7f11d831d640 1 --2- 172.21.15.107:0/1239358663 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f11bc069210 0x7f11bc06b6c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:48.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.703+0000 7f11d831d640 1 --2- 172.21.15.107:0/1239358663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11d00fbff0 0x7f11d0111ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:48.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.703+0000 7f11d831d640 1 --2- 172.21.15.107:0/1239358663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11d00fb6a0 0x7f11d00fad50 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:48.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.703+0000 7f11d831d640 1 -- 172.21.15.107:0/1239358663 >> 172.21.15.107:0/1239358663 conn(0x7f11d00f5ff0 msgr2=0x7f11d00ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:32:48.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.703+0000 7f11d831d640 1 -- 172.21.15.107:0/1239358663 shutdown_connections 2024-04-22T23:32:48.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.703+0000 7f11d831d640 1 -- 172.21.15.107:0/1239358663 wait complete. 2024-04-22T23:32:48.708 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:48 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/2195203746' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T23:32:48.709 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:48 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/740862299' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T23:32:48.710 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-22T23:32:48.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.815+0000 7f8e6ca6e640 1 -- 172.21.15.107:0/754794813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e680fa830 msgr2=0x7f8e680fac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:48.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.815+0000 7f8e6ca6e640 1 --2- 172.21.15.107:0/754794813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e680fa830 0x7f8e680fac40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c0048d0 tx=0x7f8e5c009da0 comp rx=0 tx=0).stop 2024-04-22T23:32:48.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.815+0000 7f8e6ca6e640 1 -- 172.21.15.107:0/754794813 shutdown_connections 2024-04-22T23:32:48.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.815+0000 7f8e6ca6e640 1 --2- 172.21.15.107:0/754794813 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e680fbb00 0x7f8e680fbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:48.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.815+0000 7f8e6ca6e640 1 --2- 172.21.15.107:0/754794813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e680fa830 0x7f8e680fac40 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:48.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.815+0000 7f8e6ca6e640 1 -- 172.21.15.107:0/754794813 >> 172.21.15.107:0/754794813 conn(0x7f8e680f5ff0 msgr2=0x7f8e680f8420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:32:48.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.815+0000 7f8e6ca6e640 1 -- 172.21.15.107:0/754794813 shutdown_connections 2024-04-22T23:32:48.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.815+0000 7f8e6ca6e640 1 -- 172.21.15.107:0/754794813 wait complete. 2024-04-22T23:32:48.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.815+0000 7f8e6ca6e640 1 Processor -- start 2024-04-22T23:32:48.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.815+0000 7f8e6ca6e640 1 -- start start 2024-04-22T23:32:48.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.815+0000 7f8e6ca6e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e680fa830 0x7f8e6810aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:48.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.815+0000 7f8e6ca6e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e680fbb00 0x7f8e6810b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:48.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.815+0000 7f8e6ca6e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6810b680 con 0x7f8e680fa830 2024-04-22T23:32:48.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.815+0000 7f8e6ca6e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e68110080 con 0x7f8e680fbb00 2024-04-22T23:32:48.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.816+0000 7f8e6659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e680fa830 0x7f8e6810aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:48.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.816+0000 7f8e6659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e680fa830 0x7f8e6810aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34946/0 (socket says 172.21.15.107:34946) 2024-04-22T23:32:48.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.816+0000 7f8e6659c640 1 -- 172.21.15.107:0/4029331199 learned_addr learned my addr 172.21.15.107:0/4029331199 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:32:48.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.816+0000 7f8e65d9b640 1 --2- 172.21.15.107:0/4029331199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e680fbb00 0x7f8e6810b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:48.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.816+0000 7f8e6659c640 1 -- 172.21.15.107:0/4029331199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e680fbb00 msgr2=0x7f8e6810b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:48.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.816+0000 7f8e6659c640 1 --2- 172.21.15.107:0/4029331199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e680fbb00 0x7f8e6810b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:48.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.816+0000 7f8e6659c640 1 -- 172.21.15.107:0/4029331199 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e5c005040 con 0x7f8e680fa830 2024-04-22T23:32:48.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.816+0000 7f8e65d9b640 1 --2- 172.21.15.107:0/4029331199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e680fbb00 0x7f8e6810b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T23:32:48.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.816+0000 7f8e6659c640 1 --2- 172.21.15.107:0/4029331199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e680fa830 0x7f8e6810aaf0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c0048a0 tx=0x7f8e5c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:32:48.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.816+0000 7f8e56ffd640 1 -- 172.21.15.107:0/4029331199 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e5c007cd0 con 0x7f8e680fa830 2024-04-22T23:32:48.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.816+0000 7f8e56ffd640 1 -- 172.21.15.107:0/4029331199 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8e5c03d450 con 0x7f8e680fa830 2024-04-22T23:32:48.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.816+0000 7f8e6ca6e640 1 -- 172.21.15.107:0/4029331199 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e681123d0 con 0x7f8e680fa830 2024-04-22T23:32:48.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.816+0000 7f8e56ffd640 1 -- 172.21.15.107:0/4029331199 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e5c03b640 con 0x7f8e680fa830 2024-04-22T23:32:48.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.816+0000 7f8e6ca6e640 1 -- 172.21.15.107:0/4029331199 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e68112870 con 0x7f8e680fa830 2024-04-22T23:32:48.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.817+0000 7f8e6ca6e640 1 -- 172.21.15.107:0/4029331199 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e6804bcc0 con 0x7f8e680fa830 2024-04-22T23:32:48.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.818+0000 7f8e56ffd640 1 -- 172.21.15.107:0/4029331199 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f8e5c02f070 con 0x7f8e680fa830 2024-04-22T23:32:48.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.818+0000 7f8e56ffd640 1 --2- 172.21.15.107:0/4029331199 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f8e50069210 0x7f8e5006b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:48.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.818+0000 7f8e56ffd640 1 -- 172.21.15.107:0/4029331199 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f8e5c0a93c0 con 0x7f8e680fa830 2024-04-22T23:32:48.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.818+0000 7f8e65d9b640 1 --2- 172.21.15.107:0/4029331199 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f8e50069210 0x7f8e5006b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:48.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.818+0000 7f8e65d9b640 1 --2- 172.21.15.107:0/4029331199 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f8e50069210 0x7f8e5006b6c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8e6810bff0 tx=0x7f8e58009da0 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:32:48.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:48.822+0000 7f8e56ffd640 1 -- 172.21.15.107:0/4029331199 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8e5c075620 con 0x7f8e680fa830 2024-04-22T23:32:48.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:48 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/2195203746' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T23:32:48.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:48 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/740862299' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T23:32:49.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.005+0000 7f8e6ca6e640 1 -- 172.21.15.107:0/4029331199 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f8e68102b20 con 0x7f8e680fa830 2024-04-22T23:32:49.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.005+0000 7f8e56ffd640 1 -- 172.21.15.107:0/4029331199 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v46) v1 ==== 107+0+11 (secure 0 0 0) 0x7f8e5c078c40 con 0x7f8e680fa830 2024-04-22T23:32:49.001 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-04-22T23:32:49.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.008+0000 7f8e6ca6e640 1 -- 172.21.15.107:0/4029331199 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f8e50069210 msgr2=0x7f8e5006b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:49.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.008+0000 7f8e6ca6e640 1 --2- 172.21.15.107:0/4029331199 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f8e50069210 0x7f8e5006b6c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8e6810bff0 tx=0x7f8e58009da0 comp rx=0 tx=0).stop 2024-04-22T23:32:49.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.008+0000 7f8e6ca6e640 1 -- 172.21.15.107:0/4029331199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e680fa830 msgr2=0x7f8e6810aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:49.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.008+0000 7f8e6ca6e640 1 --2- 172.21.15.107:0/4029331199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e680fa830 0x7f8e6810aaf0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c0048a0 tx=0x7f8e5c009da0 comp rx=0 tx=0).stop 2024-04-22T23:32:49.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.008+0000 7f8e6ca6e640 1 -- 172.21.15.107:0/4029331199 shutdown_connections 2024-04-22T23:32:49.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.008+0000 7f8e6ca6e640 1 --2- 172.21.15.107:0/4029331199 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f8e50069210 0x7f8e5006b6c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:49.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.008+0000 7f8e6ca6e640 1 --2- 172.21.15.107:0/4029331199 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e680fbb00 0x7f8e6810b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:49.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.008+0000 7f8e6ca6e640 1 --2- 172.21.15.107:0/4029331199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e680fa830 0x7f8e6810aaf0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:49.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.008+0000 7f8e6ca6e640 1 -- 172.21.15.107:0/4029331199 >> 172.21.15.107:0/4029331199 conn(0x7f8e680f5ff0 msgr2=0x7f8e680f7700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:32:49.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.008+0000 7f8e6ca6e640 1 -- 172.21.15.107:0/4029331199 shutdown_connections 2024-04-22T23:32:49.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.009+0000 7f8e6ca6e640 1 -- 172.21.15.107:0/4029331199 wait complete. 2024-04-22T23:32:49.016 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T23:32:49.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.116+0000 7f1b830ae640 1 -- 172.21.15.107:0/511625828 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b7c0fc370 msgr2=0x7f1b7c0fc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:49.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.116+0000 7f1b830ae640 1 --2- 172.21.15.107:0/511625828 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b7c0fc370 0x7f1b7c0fc7e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1b680048d0 tx=0x7f1b68009da0 comp rx=0 tx=0).stop 2024-04-22T23:32:49.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.116+0000 7f1b830ae640 1 -- 172.21.15.107:0/511625828 shutdown_connections 2024-04-22T23:32:49.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.116+0000 7f1b830ae640 1 --2- 172.21.15.107:0/511625828 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b7c0fc370 0x7f1b7c0fc7e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:49.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.116+0000 7f1b830ae640 1 --2- 172.21.15.107:0/511625828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b7c0fb0a0 0x7f1b7c0fb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:49.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.116+0000 7f1b830ae640 1 -- 172.21.15.107:0/511625828 >> 172.21.15.107:0/511625828 conn(0x7f1b7c0f6840 msgr2=0x7f1b7c0f8c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:32:49.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.116+0000 7f1b830ae640 1 -- 172.21.15.107:0/511625828 shutdown_connections 2024-04-22T23:32:49.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.116+0000 7f1b830ae640 1 -- 172.21.15.107:0/511625828 wait complete. 2024-04-22T23:32:49.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.116+0000 7f1b830ae640 1 Processor -- start 2024-04-22T23:32:49.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.116+0000 7f1b830ae640 1 -- start start 2024-04-22T23:32:49.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.117+0000 7f1b830ae640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b7c0fb0a0 0x7f1b7c064dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:49.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.117+0000 7f1b830ae640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b7c0fc370 0x7f1b7c111ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:49.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.117+0000 7f1b830ae640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7c112010 con 0x7f1b7c0fc370 2024-04-22T23:32:49.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.117+0000 7f1b830ae640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7c112150 con 0x7f1b7c0fb0a0 2024-04-22T23:32:49.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.117+0000 7f1b7bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b7c0fc370 0x7f1b7c111ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:49.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.117+0000 7f1b7bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b7c0fc370 0x7f1b7c111ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34972/0 (socket says 172.21.15.107:34972) 2024-04-22T23:32:49.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.117+0000 7f1b7bfff640 1 -- 172.21.15.107:0/380896006 learned_addr learned my addr 172.21.15.107:0/380896006 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:32:49.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.117+0000 7f1b80e4a640 1 --2- 172.21.15.107:0/380896006 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b7c0fb0a0 0x7f1b7c064dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:49.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.117+0000 7f1b7bfff640 1 -- 172.21.15.107:0/380896006 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b7c0fb0a0 msgr2=0x7f1b7c064dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:49.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.117+0000 7f1b7bfff640 1 --2- 172.21.15.107:0/380896006 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b7c0fb0a0 0x7f1b7c064dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:49.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.117+0000 7f1b7bfff640 1 -- 172.21.15.107:0/380896006 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b68005040 con 0x7f1b7c0fc370 2024-04-22T23:32:49.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.117+0000 7f1b7bfff640 1 --2- 172.21.15.107:0/380896006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b7c0fc370 0x7f1b7c111ad0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1b68007d20 tx=0x7f1b68009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:32:49.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.117+0000 7f1b797fa640 1 -- 172.21.15.107:0/380896006 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b68009590 con 0x7f1b7c0fc370 2024-04-22T23:32:49.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.117+0000 7f1b797fa640 1 -- 172.21.15.107:0/380896006 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1b680096f0 con 0x7f1b7c0fc370 2024-04-22T23:32:49.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.117+0000 7f1b830ae640 1 -- 172.21.15.107:0/380896006 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b7c112350 con 0x7f1b7c0fc370 2024-04-22T23:32:49.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.117+0000 7f1b797fa640 1 -- 172.21.15.107:0/380896006 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b6803b6d0 con 0x7f1b7c0fc370 2024-04-22T23:32:49.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.117+0000 7f1b830ae640 1 -- 172.21.15.107:0/380896006 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b7c112840 con 0x7f1b7c0fc370 2024-04-22T23:32:49.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.118+0000 7f1b830ae640 1 -- 172.21.15.107:0/380896006 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b7c105580 con 0x7f1b7c0fc370 2024-04-22T23:32:49.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.119+0000 7f1b797fa640 1 -- 172.21.15.107:0/380896006 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f1b6802f070 con 0x7f1b7c0fc370 2024-04-22T23:32:49.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.119+0000 7f1b797fa640 1 --2- 172.21.15.107:0/380896006 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f1b6c069210 0x7f1b6c06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:49.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.119+0000 7f1b797fa640 1 -- 172.21.15.107:0/380896006 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f1b680a9970 con 0x7f1b7c0fc370 2024-04-22T23:32:49.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.119+0000 7f1b80e4a640 1 --2- 172.21.15.107:0/380896006 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f1b6c069210 0x7f1b6c06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:49.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.120+0000 7f1b80e4a640 1 --2- 172.21.15.107:0/380896006 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f1b6c069210 0x7f1b6c06b6c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1b70004c40 tx=0x7f1b70009040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:32:49.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.124+0000 7f1b797fa640 1 -- 172.21.15.107:0/380896006 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f1b68075bd0 con 0x7f1b7c0fc370 2024-04-22T23:32:49.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.333+0000 7f1b830ae640 1 -- 172.21.15.107:0/380896006 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1b7c1129d0 con 0x7f1b7c0fc370 2024-04-22T23:32:49.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.334+0000 7f1b797fa640 1 -- 172.21.15.107:0/380896006 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4715 (secure 0 0 0) 0x7f1b7c1129d0 con 0x7f1b7c0fc370 2024-04-22T23:32:49.331 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:32:49.331 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14484,"name":"cephfs.smithi107.dfvvkw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6827/3548717247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3548717247},{"type":"v1","addr":"172.21.15.107:6827","nonce":3548717247}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":24277,"name":"cephfs.smithi157.ufywse","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.157:6827/4072444199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":4072444199},{"type":"v1","addr":"172.21.15.157:6827","nonce":4072444199}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T23:32:20.146130+0000","modified":"2024-04-22T23:32:39.774747+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14468,"mds_1":24265},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14468":{"gid":14468,"name":"cephfs.smithi107.ktuxjl","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.107:6829/3335313253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3335313253},{"type":"v1","addr":"172.21.15.107:6829","nonce":3335313253}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24265":{"gid":24265,"name":"cephfs.smithi157.rcaris","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6825/2779251893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2779251893},{"type":"v1","addr":"172.21.15.157:6825","nonce":2779251893}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-04-22T23:32:49.331 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 17 2024-04-22T23:32:49.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.336+0000 7f1b830ae640 1 -- 172.21.15.107:0/380896006 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f1b6c069210 msgr2=0x7f1b6c06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:49.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.336+0000 7f1b830ae640 1 --2- 172.21.15.107:0/380896006 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f1b6c069210 0x7f1b6c06b6c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1b70004c40 tx=0x7f1b70009040 comp rx=0 tx=0).stop 2024-04-22T23:32:49.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.336+0000 7f1b830ae640 1 -- 172.21.15.107:0/380896006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b7c0fc370 msgr2=0x7f1b7c111ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:49.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.336+0000 7f1b830ae640 1 --2- 172.21.15.107:0/380896006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b7c0fc370 0x7f1b7c111ad0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1b68007d20 tx=0x7f1b68009da0 comp rx=0 tx=0).stop 2024-04-22T23:32:49.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.336+0000 7f1b830ae640 1 -- 172.21.15.107:0/380896006 shutdown_connections 2024-04-22T23:32:49.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.336+0000 7f1b830ae640 1 --2- 172.21.15.107:0/380896006 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f1b6c069210 0x7f1b6c06b6c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:49.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.336+0000 7f1b830ae640 1 --2- 172.21.15.107:0/380896006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b7c0fc370 0x7f1b7c111ad0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:49.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.336+0000 7f1b830ae640 1 --2- 172.21.15.107:0/380896006 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b7c0fb0a0 0x7f1b7c064dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:49.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.336+0000 7f1b830ae640 1 -- 172.21.15.107:0/380896006 >> 172.21.15.107:0/380896006 conn(0x7f1b7c0f6840 msgr2=0x7f1b7c0ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:32:49.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.336+0000 7f1b830ae640 1 -- 172.21.15.107:0/380896006 shutdown_connections 2024-04-22T23:32:49.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:49.336+0000 7f1b830ae640 1 -- 172.21.15.107:0/380896006 wait complete. 2024-04-22T23:32:49.346 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 17, 'max_mds': 2, 'flags': 18} 2024-04-22T23:32:49.347 INFO:teuthology.run_tasks:Running task kclient... 2024-04-22T23:32:49.381 INFO:tasks.kclient:Mounting kernel clients... 2024-04-22T23:32:49.381 INFO:tasks.kclient:config is {'client.0': {}, 'client.1': {}} 2024-04-22T23:32:49.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T23:32:49.381 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list 2024-04-22T23:32:49.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T23:32:49.401 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link delete ceph-brx 2024-04-22T23:32:49.466 INFO:teuthology.orchestra.run.smithi107.stderr:Cannot find device "ceph-brx" 2024-04-22T23:32:49.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T23:32:49.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T23:32:49.468 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-04-22T23:32:49.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T23:32:49.488 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link delete ceph-brx 2024-04-22T23:32:49.562 INFO:teuthology.orchestra.run.smithi157.stderr:Cannot find device "ceph-brx" 2024-04-22T23:32:49.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T23:32:49.564 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-04-22T23:32:49.564 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-22T23:32:49.564 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-22T23:32:49.564 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-22T23:32:49.625 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:49 smithi107 ceph-mon[26355]: pgmap v191: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 4.2 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-22T23:32:49.625 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:49 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/1239358663' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-22T23:32:49.625 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:49 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:32:49.625 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:49 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/4029331199' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-22T23:32:49.626 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:49 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/380896006' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T23:32:49.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:49 smithi157 ceph-mon[33877]: pgmap v191: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 4.2 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-22T23:32:49.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:49 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/1239358663' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-22T23:32:49.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:49 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:32:49.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:49 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/4029331199' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-22T23:32:49.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:49 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/380896006' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T23:32:50.258 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:32:51.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:51 smithi107 ceph-mon[26355]: pgmap v192: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-22T23:32:51.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:51 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:32:51.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:51 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:32:51.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:51 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:32:51.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:51 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:32:51.736 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:51 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:32:51.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:51 smithi157 ceph-mon[33877]: pgmap v192: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-22T23:32:51.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:51 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:32:51.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:51 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:32:51.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:51 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:32:51.884 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:51 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:32:51.884 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:51 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:32:51.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:51.970+0000 7f94a121a700 1 -- 172.21.15.107:0/2502883606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f949c101d10 msgr2=0x7f949c1020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:51.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:51.970+0000 7f94a121a700 1 --2- 172.21.15.107:0/2502883606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f949c101d10 0x7f949c1020e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9494009ce0 tx=0x7f9494004d10 comp rx=0 tx=0).stop 2024-04-22T23:32:51.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:51.970+0000 7f94a121a700 1 -- 172.21.15.107:0/2502883606 shutdown_connections 2024-04-22T23:32:51.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:51.970+0000 7f94a121a700 1 --2- 172.21.15.107:0/2502883606 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f949c1026b0 0x7f949c0fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:51.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:51.970+0000 7f94a121a700 1 --2- 172.21.15.107:0/2502883606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f949c101d10 0x7f949c1020e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:51.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:51.970+0000 7f94a121a700 1 -- 172.21.15.107:0/2502883606 >> 172.21.15.107:0/2502883606 conn(0x7f949c072bc0 msgr2=0x7f949c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:32:51.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:51.970+0000 7f94a121a700 1 -- 172.21.15.107:0/2502883606 shutdown_connections 2024-04-22T23:32:51.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:51.970+0000 7f94a121a700 1 -- 172.21.15.107:0/2502883606 wait complete. 2024-04-22T23:32:51.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:51.971+0000 7f94a121a700 1 Processor -- start 2024-04-22T23:32:51.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:51.971+0000 7f94a121a700 1 -- start start 2024-04-22T23:32:51.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:51.971+0000 7f94a121a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f949c101d10 0x7f949c10ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:51.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:51.971+0000 7f94a121a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f949c1026b0 0x7f949c10f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:51.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:51.971+0000 7f94a121a700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949c10f840 con 0x7f949c1026b0 2024-04-22T23:32:51.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:51.971+0000 7f94a121a700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949c10f980 con 0x7f949c101d10 2024-04-22T23:32:51.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:51.971+0000 7f949a59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f949c1026b0 0x7f949c10f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:51.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:51.971+0000 7f949a59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f949c1026b0 0x7f949c10f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42288/0 (socket says 172.21.15.107:42288) 2024-04-22T23:32:51.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:51.971+0000 7f949a59c700 1 -- 172.21.15.107:0/732978372 learned_addr learned my addr 172.21.15.107:0/732978372 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:32:51.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:51.971+0000 7f949ad9d700 1 --2- 172.21.15.107:0/732978372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f949c101d10 0x7f949c10ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:51.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:51.971+0000 7f949a59c700 1 -- 172.21.15.107:0/732978372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f949c101d10 msgr2=0x7f949c10ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:51.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:51.971+0000 7f949a59c700 1 --2- 172.21.15.107:0/732978372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f949c101d10 0x7f949c10ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:51.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:51.971+0000 7f949a59c700 1 -- 172.21.15.107:0/732978372 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9494009940 con 0x7f949c1026b0 2024-04-22T23:32:51.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:51.972+0000 7f949a59c700 1 --2- 172.21.15.107:0/732978372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f949c1026b0 0x7f949c10f1f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f949000c800 tx=0x7f9490004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:32:51.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:51.972+0000 7f94837fe700 1 -- 172.21.15.107:0/732978372 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94900106f0 con 0x7f949c1026b0 2024-04-22T23:32:51.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:51.972+0000 7f94a121a700 1 -- 172.21.15.107:0/732978372 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f949c117d20 con 0x7f949c1026b0 2024-04-22T23:32:51.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:51.972+0000 7f94837fe700 1 -- 172.21.15.107:0/732978372 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9490010d20 con 0x7f949c1026b0 2024-04-22T23:32:51.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:51.972+0000 7f94837fe700 1 -- 172.21.15.107:0/732978372 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94900113a0 con 0x7f949c1026b0 2024-04-22T23:32:51.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:51.972+0000 7f94a121a700 1 -- 172.21.15.107:0/732978372 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f949c118280 con 0x7f949c1026b0 2024-04-22T23:32:51.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:51.973+0000 7f94a121a700 1 -- 172.21.15.107:0/732978372 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f949c0fd460 con 0x7f949c1026b0 2024-04-22T23:32:51.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:51.973+0000 7f94837fe700 1 -- 172.21.15.107:0/732978372 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f9490012400 con 0x7f949c1026b0 2024-04-22T23:32:51.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:51.973+0000 7f94837fe700 1 --2- 172.21.15.107:0/732978372 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f94840692a0 0x7f948406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:51.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:51.974+0000 7f949ad9d700 1 --2- 172.21.15.107:0/732978372 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f94840692a0 0x7f948406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:51.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:51.974+0000 7f94837fe700 1 -- 172.21.15.107:0/732978372 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f9490087160 con 0x7f949c1026b0 2024-04-22T23:32:51.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:51.974+0000 7f949ad9d700 1 --2- 172.21.15.107:0/732978372 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f94840692a0 0x7f948406b750 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f949400b740 tx=0x7f9494004d10 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:32:51.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:51.978+0000 7f94837fe700 1 -- 172.21.15.107:0/732978372 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f9490053440 con 0x7f949c1026b0 2024-04-22T23:32:52.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:52.211+0000 7f94a121a700 1 -- 172.21.15.107:0/732978372 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f949c108e40 con 0x7f949c1026b0 2024-04-22T23:32:52.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:52.211+0000 7f94837fe700 1 -- 172.21.15.107:0/732978372 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+83 (secure 0 0 0) 0x7f9490056a60 con 0x7f949c1026b0 2024-04-22T23:32:52.207 INFO:teuthology.orchestra.run.smithi107.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-04-22T23:32:52.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:52.213+0000 7f94a121a700 1 -- 172.21.15.107:0/732978372 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f94840692a0 msgr2=0x7f948406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:52.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:52.213+0000 7f94a121a700 1 --2- 172.21.15.107:0/732978372 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f94840692a0 0x7f948406b750 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f949400b740 tx=0x7f9494004d10 comp rx=0 tx=0).stop 2024-04-22T23:32:52.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:52.213+0000 7f94a121a700 1 -- 172.21.15.107:0/732978372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f949c1026b0 msgr2=0x7f949c10f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:52.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:52.213+0000 7f94a121a700 1 --2- 172.21.15.107:0/732978372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f949c1026b0 0x7f949c10f1f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f949000c800 tx=0x7f9490004470 comp rx=0 tx=0).stop 2024-04-22T23:32:52.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:52.213+0000 7f94a121a700 1 -- 172.21.15.107:0/732978372 shutdown_connections 2024-04-22T23:32:52.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:52.213+0000 7f94a121a700 1 --2- 172.21.15.107:0/732978372 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f94840692a0 0x7f948406b750 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:52.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:52.213+0000 7f94a121a700 1 --2- 172.21.15.107:0/732978372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f949c101d10 0x7f949c10ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:52.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:52.213+0000 7f94a121a700 1 --2- 172.21.15.107:0/732978372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f949c1026b0 0x7f949c10f1f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:52.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:52.213+0000 7f94a121a700 1 -- 172.21.15.107:0/732978372 >> 172.21.15.107:0/732978372 conn(0x7f949c072bc0 msgr2=0x7f949c104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:32:52.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:52.213+0000 7f94a121a700 1 -- 172.21.15.107:0/732978372 shutdown_connections 2024-04-22T23:32:52.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:52.213+0000 7f94a121a700 1 -- 172.21.15.107:0/732978372 wait complete. 2024-04-22T23:32:52.821 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:52 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/732978372' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-22T23:32:52.846 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-22T23:32:52.846 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-22T23:32:52.846 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi107.front.sepia.ceph.com 2024-04-22T23:32:52.846 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-22T23:32:52.846 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-22T23:32:52.846 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-22T23:32:52.846 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-22T23:32:52.846 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-22T23:32:52.847 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-22T23:32:52.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T23:32:52.847 DEBUG:teuthology.orchestra.run.smithi107:> ip addr 2024-04-22T23:32:52.863 INFO:teuthology.orchestra.run.smithi107.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-22T23:32:52.863 INFO:teuthology.orchestra.run.smithi107.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-22T23:32:52.864 INFO:teuthology.orchestra.run.smithi107.stdout: inet 127.0.0.1/8 scope host lo 2024-04-22T23:32:52.864 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-22T23:32:52.864 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 ::1/128 scope host 2024-04-22T23:32:52.864 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-22T23:32:52.864 INFO:teuthology.orchestra.run.smithi107.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-22T23:32:52.864 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:d8:a2:0a brd ff:ff:ff:ff:ff:ff 2024-04-22T23:32:52.864 INFO:teuthology.orchestra.run.smithi107.stdout: altname enp7s0f0 2024-04-22T23:32:52.864 INFO:teuthology.orchestra.run.smithi107.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-22T23:32:52.864 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:d8:a2:0b brd ff:ff:ff:ff:ff:ff 2024-04-22T23:32:52.864 INFO:teuthology.orchestra.run.smithi107.stdout: altname enp7s0f1 2024-04-22T23:32:52.864 INFO:teuthology.orchestra.run.smithi107.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-22T23:32:52.864 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:8f:47:ea brd ff:ff:ff:ff:ff:ff 2024-04-22T23:32:52.864 INFO:teuthology.orchestra.run.smithi107.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-22T23:32:52.864 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:8f:47:eb brd ff:ff:ff:ff:ff:ff 2024-04-22T23:32:52.865 INFO:teuthology.orchestra.run.smithi107.stdout: inet 172.21.15.107/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-22T23:32:52.865 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft 42250sec preferred_lft 42250sec 2024-04-22T23:32:52.865 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 fe80::5dc3:f4ce:498d:c8c6/64 scope link noprefixroute 2024-04-22T23:32:52.865 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-22T23:32:52.865 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-22T23:32:52.865 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-22T23:32:52.865 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-04-22T23:32:52.865 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link add name ceph-brx type bridge 2024-04-22T23:32:52.866 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip addr flush dev ceph-brx 2024-04-22T23:32:52.866 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set ceph-brx up 2024-04-22T23:32:52.866 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-22T23:32:52.866 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-04-22T23:32:52.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:52 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/732978372' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-22T23:32:52.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T23:32:53.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T23:32:53.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T23:32:53.068 DEBUG:teuthology.orchestra.run.smithi107:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-22T23:32:53.092 INFO:teuthology.orchestra.run.smithi107.stdout:1 2024-04-22T23:32:53.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T23:32:53.093 DEBUG:teuthology.orchestra.run.smithi107:> route 2024-04-22T23:32:53.168 INFO:teuthology.orchestra.run.smithi107.stdout:Kernel IP routing table 2024-04-22T23:32:53.168 INFO:teuthology.orchestra.run.smithi107.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-22T23:32:53.168 INFO:teuthology.orchestra.run.smithi107.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-22T23:32:53.168 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-22T23:32:53.168 INFO:teuthology.orchestra.run.smithi107.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-22T23:32:53.169 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-22T23:32:53.169 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-04-22T23:32:53.169 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-22T23:32:53.169 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-22T23:32:53.169 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-22T23:32:53.169 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-04-22T23:32:53.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T23:32:53.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T23:32:53.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T23:32:53.463 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list 2024-04-22T23:32:53.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T23:32:53.478 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list-id 2024-04-22T23:32:53.533 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-22T23:32:53.533 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-04-22T23:32:53.533 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-22T23:32:53.533 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-22T23:32:53.533 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-04-22T23:32:53.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T23:32:53.628 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:53 smithi107 ceph-mon[26355]: pgmap v193: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-22T23:32:53.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T23:32:53.649 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-22T23:32:53.649 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-22T23:32:53.649 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-04-22T23:32:53.649 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-22T23:32:53.649 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-22T23:32:53.649 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-22T23:32:53.649 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-22T23:32:53.649 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-22T23:32:53.649 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-04-22T23:32:53.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T23:32:53.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T23:32:53.768 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-22T23:32:53.769 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-04-22T23:32:53.769 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set brx.0 up 2024-04-22T23:32:53.769 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set dev brx.0 master ceph-brx 2024-04-22T23:32:53.769 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-04-22T23:32:53.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T23:32:53.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T23:32:53.842 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-22T23:32:53.842 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-22T23:32:53.857 INFO:teuthology.orchestra.run.smithi107.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-22T23:32:53.857 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-22T23:32:53.857 DEBUG:teuthology.orchestra.run.smithi107:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-22T23:32:53.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:53 smithi157 ceph-mon[33877]: pgmap v193: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-22T23:32:53.911 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-22T23:32:53.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T23:32:53.911 DEBUG:teuthology.orchestra.run.smithi107:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-22T23:32:53.992 INFO:teuthology.orchestra.run.smithi107.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-22T23:32:53.992 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-22T23:32:53.992 INFO:teuthology.orchestra.run.smithi107.stdout:invalid new device string format 2024-04-22T23:32:54.090 INFO:teuthology.orchestra.run.smithi107.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-22T23:32:54.090 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-22T23:32:54.090 INFO:teuthology.orchestra.run.smithi107.stdout:invalid new device string format 2024-04-22T23:32:54.090 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: resolved to: "172.21.15.107:3300,172.21.15.157:3300" 2024-04-22T23:32:54.090 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-22T23:32:54.095 INFO:teuthology.orchestra.run.smithi107.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-22T23:32:54.095 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-22T23:32:54.095 INFO:teuthology.orchestra.run.smithi107.stdout:invalid new device string format 2024-04-22T23:32:54.095 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: resolved to: "172.21.15.107:3300,172.21.15.157:3300" 2024-04-22T23:32:54.095 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-22T23:32:54.095 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: resolved to: "172.21.15.107:6789,172.21.15.157:6789" 2024-04-22T23:32:54.095 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.107:6789,172.21.15.157:6789:/ 2024-04-22T23:32:54.095 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8252f6f2-00ff-11ef-bc93-c7b262605968" will pass to kernel 2024-04-22T23:32:54.096 INFO:teuthology.orchestra.run.smithi107.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-22T23:32:54.096 INFO:teuthology.orchestra.run.smithi107.stdout: You just mounted a file system that supports labels which does not 2024-04-22T23:32:54.096 INFO:teuthology.orchestra.run.smithi107.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-22T23:32:54.096 INFO:teuthology.orchestra.run.smithi107.stdout: applications will generate AVC messages and not be allowed access to 2024-04-22T23:32:54.096 INFO:teuthology.orchestra.run.smithi107.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-22T23:32:54.097 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-22T23:32:54.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T23:32:54.098 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-22T23:32:54.163 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-22T23:32:54.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:54 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-22T23:32:54.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T23:32:54.264 INFO:teuthology.orchestra.run.smithi107.stdout:client14522 2024-04-22T23:32:54.264 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-22T23:32:54.264 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-22T23:32:54.284 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-22T23:32:54.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:54 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-22T23:32:54.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T23:32:54.384 INFO:teuthology.orchestra.run.smithi107.stdout:client14522 2024-04-22T23:32:54.384 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-22T23:32:54.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:54 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-22T23:32:54.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T23:32:54.444 INFO:teuthology.orchestra.run.smithi107.stdout:client14522 2024-04-22T23:32:54.444 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968.client14522/status') 2024-04-22T23:32:54.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T23:32:54.505 INFO:teuthology.orchestra.run.smithi107.stdout:instance: client.14522 (3)192.168.0.1:0/3769680709 2024-04-22T23:32:54.505 INFO:teuthology.orchestra.run.smithi107.stdout:blocklisted: false 2024-04-22T23:32:54.506 INFO:teuthology.orchestra.run.smithi107.stderr:0+1 records in 2024-04-22T23:32:54.506 INFO:teuthology.orchestra.run.smithi107.stderr:0+1 records out 2024-04-22T23:32:54.506 INFO:teuthology.orchestra.run.smithi107.stderr:70 bytes copied, 2.1886e-05 s, 3.2 MB/s 2024-04-22T23:32:54.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T23:32:54.513 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-04-22T23:32:54.513 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-22T23:32:54.513 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-22T23:32:54.513 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-22T23:32:54.766 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:32:55.797 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:55 smithi107 ceph-mon[26355]: pgmap v194: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-04-22T23:32:55.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:55 smithi157 ceph-mon[33877]: pgmap v194: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-04-22T23:32:56.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:56.336+0000 7f855df52700 1 -- 172.21.15.107:0/2761450889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8558102e10 msgr2=0x7f85581031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:56.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:56.336+0000 7f855df52700 1 --2- 172.21.15.107:0/2761450889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8558102e10 0x7f85581031e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8548004940 tx=0x7f8548009e30 comp rx=0 tx=0).stop 2024-04-22T23:32:56.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:56.336+0000 7f855df52700 1 -- 172.21.15.107:0/2761450889 shutdown_connections 2024-04-22T23:32:56.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:56.336+0000 7f855df52700 1 --2- 172.21.15.107:0/2761450889 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85581037b0 0x7f85580fce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:56.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:56.336+0000 7f855df52700 1 --2- 172.21.15.107:0/2761450889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8558102e10 0x7f85581031e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:56.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:56.336+0000 7f855df52700 1 -- 172.21.15.107:0/2761450889 >> 172.21.15.107:0/2761450889 conn(0x7f85580f89c0 msgr2=0x7f85580fadd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:32:56.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:56.336+0000 7f855df52700 1 -- 172.21.15.107:0/2761450889 shutdown_connections 2024-04-22T23:32:56.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:56.336+0000 7f855df52700 1 -- 172.21.15.107:0/2761450889 wait complete. 2024-04-22T23:32:56.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:56.336+0000 7f855df52700 1 Processor -- start 2024-04-22T23:32:56.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:56.337+0000 7f855df52700 1 -- start start 2024-04-22T23:32:56.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:56.337+0000 7f855df52700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8558102e10 0x7f85580702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:56.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:56.337+0000 7f855df52700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85581037b0 0x7f8558070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:56.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:56.337+0000 7f855df52700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8558072210 con 0x7f8558102e10 2024-04-22T23:32:56.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:56.337+0000 7f855df52700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8558070d40 con 0x7f85581037b0 2024-04-22T23:32:56.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:56.337+0000 7f85577fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8558102e10 0x7f85580702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:56.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:56.337+0000 7f85577fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8558102e10 0x7f85580702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42308/0 (socket says 172.21.15.107:42308) 2024-04-22T23:32:56.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:56.337+0000 7f85577fe700 1 -- 172.21.15.107:0/423903164 learned_addr learned my addr 172.21.15.107:0/423903164 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:32:56.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:56.337+0000 7f8556ffd700 1 --2- 172.21.15.107:0/423903164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85581037b0 0x7f8558070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:56.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:56.337+0000 7f85577fe700 1 -- 172.21.15.107:0/423903164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85581037b0 msgr2=0x7f8558070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:56.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:56.337+0000 7f85577fe700 1 --2- 172.21.15.107:0/423903164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85581037b0 0x7f8558070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:56.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:56.337+0000 7f85577fe700 1 -- 172.21.15.107:0/423903164 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8548005040 con 0x7f8558102e10 2024-04-22T23:32:56.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:56.337+0000 7f8556ffd700 1 --2- 172.21.15.107:0/423903164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85581037b0 0x7f8558070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T23:32:56.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:56.337+0000 7f85577fe700 1 --2- 172.21.15.107:0/423903164 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8558102e10 0x7f85580702c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f854800b3b0 tx=0x7f8548009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:32:56.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:56.337+0000 7f853ffff700 1 -- 172.21.15.107:0/423903164 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f854800e040 con 0x7f8558102e10 2024-04-22T23:32:56.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:56.337+0000 7f853ffff700 1 -- 172.21.15.107:0/423903164 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f854800bbc0 con 0x7f8558102e10 2024-04-22T23:32:56.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:56.337+0000 7f855df52700 1 -- 172.21.15.107:0/423903164 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8558070fc0 con 0x7f8558102e10 2024-04-22T23:32:56.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:56.337+0000 7f853ffff700 1 -- 172.21.15.107:0/423903164 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f854801c3f0 con 0x7f8558102e10 2024-04-22T23:32:56.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:56.338+0000 7f855df52700 1 -- 172.21.15.107:0/423903164 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f855811a0e0 con 0x7f8558102e10 2024-04-22T23:32:56.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:56.338+0000 7f855df52700 1 -- 172.21.15.107:0/423903164 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8558100120 con 0x7f8558102e10 2024-04-22T23:32:56.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:56.339+0000 7f853ffff700 1 -- 172.21.15.107:0/423903164 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f854800bd30 con 0x7f8558102e10 2024-04-22T23:32:56.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:56.339+0000 7f853ffff700 1 --2- 172.21.15.107:0/423903164 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f85400692f0 0x7f854006b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:32:56.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:56.339+0000 7f853ffff700 1 -- 172.21.15.107:0/423903164 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f8548088520 con 0x7f8558102e10 2024-04-22T23:32:56.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:56.339+0000 7f8556ffd700 1 --2- 172.21.15.107:0/423903164 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f85400692f0 0x7f854006b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:32:56.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:56.340+0000 7f8556ffd700 1 --2- 172.21.15.107:0/423903164 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f85400692f0 0x7f854006b7a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f854c009fd0 tx=0x7f854c00b040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:32:56.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:56.344+0000 7f853ffff700 1 -- 172.21.15.107:0/423903164 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f85480546e0 con 0x7f8558102e10 2024-04-22T23:32:56.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:56.575+0000 7f855df52700 1 -- 172.21.15.107:0/423903164 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8558071860 con 0x7f8558102e10 2024-04-22T23:32:56.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:56.575+0000 7f853ffff700 1 -- 172.21.15.107:0/423903164 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+83 (secure 0 0 0) 0x7f8548057d00 con 0x7f8558102e10 2024-04-22T23:32:56.571 INFO:teuthology.orchestra.run.smithi107.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-04-22T23:32:56.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:56.577+0000 7f855df52700 1 -- 172.21.15.107:0/423903164 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f85400692f0 msgr2=0x7f854006b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:56.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:56.577+0000 7f855df52700 1 --2- 172.21.15.107:0/423903164 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f85400692f0 0x7f854006b7a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f854c009fd0 tx=0x7f854c00b040 comp rx=0 tx=0).stop 2024-04-22T23:32:56.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:56.577+0000 7f855df52700 1 -- 172.21.15.107:0/423903164 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8558102e10 msgr2=0x7f85580702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:32:56.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:56.577+0000 7f855df52700 1 --2- 172.21.15.107:0/423903164 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8558102e10 0x7f85580702c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f854800b3b0 tx=0x7f8548009e30 comp rx=0 tx=0).stop 2024-04-22T23:32:56.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:56.577+0000 7f855df52700 1 -- 172.21.15.107:0/423903164 shutdown_connections 2024-04-22T23:32:56.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:56.577+0000 7f855df52700 1 --2- 172.21.15.107:0/423903164 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f85400692f0 0x7f854006b7a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:56.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:56.577+0000 7f855df52700 1 --2- 172.21.15.107:0/423903164 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8558102e10 0x7f85580702c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:56.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:56.577+0000 7f855df52700 1 --2- 172.21.15.107:0/423903164 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f85581037b0 0x7f8558070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:32:56.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:56.577+0000 7f855df52700 1 -- 172.21.15.107:0/423903164 >> 172.21.15.107:0/423903164 conn(0x7f85580f89c0 msgr2=0x7f85581073b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:32:56.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:56.577+0000 7f855df52700 1 -- 172.21.15.107:0/423903164 shutdown_connections 2024-04-22T23:32:56.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:32:56.577+0000 7f855df52700 1 -- 172.21.15.107:0/423903164 wait complete. 2024-04-22T23:32:57.247 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-22T23:32:57.247 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-22T23:32:57.247 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2024-04-22T23:32:57.247 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-22T23:32:57.247 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-22T23:32:57.247 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-22T23:32:57.247 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-22T23:32:57.247 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-22T23:32:57.247 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-22T23:32:57.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T23:32:57.248 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2024-04-22T23:32:57.266 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-22T23:32:57.267 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-22T23:32:57.267 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-04-22T23:32:57.267 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-22T23:32:57.267 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-04-22T23:32:57.267 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-22T23:32:57.267 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-22T23:32:57.267 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2024-04-22T23:32:57.267 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2024-04-22T23:32:57.267 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-22T23:32:57.267 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2024-04-22T23:32:57.267 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2024-04-22T23:32:57.267 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-22T23:32:57.268 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2024-04-22T23:32:57.268 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-22T23:32:57.268 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2024-04-22T23:32:57.268 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-22T23:32:57.268 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 42276sec preferred_lft 42276sec 2024-04-22T23:32:57.268 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::dbaa:e554:368f:6b07/64 scope link noprefixroute 2024-04-22T23:32:57.268 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-04-22T23:32:57.268 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-22T23:32:57.269 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-22T23:32:57.269 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-22T23:32:57.269 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link add name ceph-brx type bridge 2024-04-22T23:32:57.269 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip addr flush dev ceph-brx 2024-04-22T23:32:57.269 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set ceph-brx up 2024-04-22T23:32:57.269 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-22T23:32:57.269 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-22T23:32:57.363 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:32:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T23:32:57.456 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:32:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T23:32:57.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T23:32:57.463 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-22T23:32:57.487 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-04-22T23:32:57.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T23:32:57.489 DEBUG:teuthology.orchestra.run.smithi157:> route 2024-04-22T23:32:57.562 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2024-04-22T23:32:57.562 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-22T23:32:57.562 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-22T23:32:57.562 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-22T23:32:57.562 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-22T23:32:57.563 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-22T23:32:57.563 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-22T23:32:57.563 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-22T23:32:57.563 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-22T23:32:57.563 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-22T23:32:57.563 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-22T23:32:57.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:57 smithi157 ceph-mon[33877]: pgmap v195: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:32:57.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:57 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/423903164' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-22T23:32:57.652 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:32:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T23:32:57.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:57 smithi107 ceph-mon[26355]: pgmap v195: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:32:57.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:57 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/423903164' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-22T23:32:57.822 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:32:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T23:32:57.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T23:32:57.828 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-04-22T23:32:57.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T23:32:57.883 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list-id 2024-04-22T23:32:57.938 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-22T23:32:57.938 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-22T23:32:57.938 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-22T23:32:57.938 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-22T23:32:57.938 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-22T23:32:58.028 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:32:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T23:32:58.051 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:32:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T23:32:58.057 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-22T23:32:58.057 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-22T23:32:58.058 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-22T23:32:58.058 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-22T23:32:58.058 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-22T23:32:58.058 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-22T23:32:58.058 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-22T23:32:58.058 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-22T23:32:58.058 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-22T23:32:58.107 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:32:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T23:32:58.184 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:32:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T23:32:58.190 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-22T23:32:58.191 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-04-22T23:32:58.191 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2024-04-22T23:32:58.191 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set dev brx.0 master ceph-brx 2024-04-22T23:32:58.191 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-04-22T23:32:58.238 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:32:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T23:32:58.259 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:32:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T23:32:58.265 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-22T23:32:58.266 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-22T23:32:58.280 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-22T23:32:58.280 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-22T23:32:58.281 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-22T23:32:58.335 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-22T23:32:58.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T23:32:58.336 DEBUG:teuthology.orchestra.run.smithi157:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-22T23:32:58.432 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-22T23:32:58.432 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-22T23:32:58.432 INFO:teuthology.orchestra.run.smithi157.stdout:invalid new device string format 2024-04-22T23:32:58.545 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-22T23:32:58.545 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-22T23:32:58.545 INFO:teuthology.orchestra.run.smithi157.stdout:invalid new device string format 2024-04-22T23:32:58.545 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: resolved to: "172.21.15.107:3300,172.21.15.157:3300" 2024-04-22T23:32:58.545 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-22T23:32:58.551 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-22T23:32:58.551 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-22T23:32:58.551 INFO:teuthology.orchestra.run.smithi157.stdout:invalid new device string format 2024-04-22T23:32:58.551 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: resolved to: "172.21.15.107:3300,172.21.15.157:3300" 2024-04-22T23:32:58.551 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-22T23:32:58.551 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: resolved to: "172.21.15.107:6789,172.21.15.157:6789" 2024-04-22T23:32:58.551 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.107:6789,172.21.15.157:6789:/ 2024-04-22T23:32:58.551 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=8252f6f2-00ff-11ef-bc93-c7b262605968" will pass to kernel 2024-04-22T23:32:58.552 INFO:teuthology.orchestra.run.smithi157.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-22T23:32:58.552 INFO:teuthology.orchestra.run.smithi157.stdout: You just mounted a file system that supports labels which does not 2024-04-22T23:32:58.552 INFO:teuthology.orchestra.run.smithi157.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-22T23:32:58.552 INFO:teuthology.orchestra.run.smithi157.stdout: applications will generate AVC messages and not be allowed access to 2024-04-22T23:32:58.552 INFO:teuthology.orchestra.run.smithi157.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-22T23:32:58.554 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-22T23:32:58.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T23:32:58.554 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-22T23:32:58.577 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-22T23:32:58.666 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:32:58 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-22T23:32:58.667 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:32:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T23:32:58.673 INFO:teuthology.orchestra.run.smithi157.stdout:client14528 2024-04-22T23:32:58.673 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-22T23:32:58.673 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-22T23:32:58.690 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-22T23:32:58.778 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:32:58 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-22T23:32:58.780 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:32:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T23:32:58.786 INFO:teuthology.orchestra.run.smithi157.stdout:client14528 2024-04-22T23:32:58.813 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-22T23:32:58.862 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:32:58 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-22T23:32:58.864 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:32:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T23:32:58.870 INFO:teuthology.orchestra.run.smithi157.stdout:client14528 2024-04-22T23:32:58.870 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968.client14528/status') 2024-04-22T23:32:58.918 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:32:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T23:32:58.927 INFO:teuthology.orchestra.run.smithi157.stdout:instance: client.14528 (3)172.21.15.157:0/2901108260 2024-04-22T23:32:58.927 INFO:teuthology.orchestra.run.smithi157.stdout:blocklisted: false 2024-04-22T23:32:58.927 INFO:teuthology.orchestra.run.smithi157.stderr:0+1 records in 2024-04-22T23:32:58.927 INFO:teuthology.orchestra.run.smithi157.stderr:0+1 records out 2024-04-22T23:32:58.928 INFO:teuthology.orchestra.run.smithi157.stderr:72 bytes copied, 2.5451e-05 s, 2.8 MB/s 2024-04-22T23:32:58.928 INFO:teuthology.orchestra.run.smithi157.stderr:2024-04-22T23:32:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T23:32:58.937 INFO:teuthology.run_tasks:Running task print... 2024-04-22T23:32:58.949 INFO:teuthology.task.print:**** done client 2024-04-22T23:32:58.949 INFO:teuthology.run_tasks:Running task parallel... 2024-04-22T23:32:59.003 INFO:teuthology.task.parallel:starting parallel... 2024-04-22T23:32:59.004 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-04-22T23:32:59.004 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-22T23:32:59.005 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi107.front.sepia.ceph.com 2024-04-22T23:32:59.005 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -e sha1=43be020184947e53516056c9931e1ac5bdbbb1a5 -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs false || true' 2024-04-22T23:32:59.006 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-04-22T23:32:59.006 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-04-22T23:32:59.010 INFO:tasks.workunit:Pulling workunits from ref 43be020184947e53516056c9931e1ac5bdbbb1a5 2024-04-22T23:32:59.011 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-22T23:32:59.011 INFO:tasks.workunit:timeout=3h 2024-04-22T23:32:59.011 INFO:tasks.workunit:cleanup=True 2024-04-22T23:32:59.012 DEBUG:teuthology.orchestra.run.smithi107:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-22T23:32:59.032 INFO:teuthology.orchestra.run.smithi107.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-22T23:32:59.032 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-22T23:32:59.033 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 91h/145d Inode: 1 Links: 2 2024-04-22T23:32:59.033 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T23:32:59.033 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-22T23:32:59.033 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-04-22T23:32:59.033 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-04-22 23:32:25.164873657 +0000 2024-04-22T23:32:59.033 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-04-22 23:32:54.164220424 +0000 2024-04-22T23:32:59.033 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-04-22 23:32:25.164873657 +0000 2024-04-22T23:32:59.033 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-22T23:32:59.033 DEBUG:teuthology.orchestra.run.smithi107:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-22T23:32:59.102 DEBUG:teuthology.orchestra.run.smithi157:> stat -- /home/ubuntu/cephtest/mnt.1 2024-04-22T23:32:59.120 INFO:teuthology.orchestra.run.smithi157.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-04-22T23:32:59.120 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 1 Blocks: 0 IO Block: 65536 directory 2024-04-22T23:32:59.120 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 70h/112d Inode: 1 Links: 3 2024-04-22T23:32:59.120 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T23:32:59.120 INFO:teuthology.orchestra.run.smithi157.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-22T23:32:59.120 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-04-22T23:32:59.120 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-04-22 23:32:59.100140849 +0000 2024-04-22T23:32:59.120 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-04-22 23:32:59.100140849 +0000 2024-04-22T23:32:59.120 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-04-22 23:32:25.164873657 +0000 2024-04-22T23:32:59.120 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-04-22T23:32:59.121 DEBUG:teuthology.orchestra.run.smithi157:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-04-22T23:32:59.188 DEBUG:teuthology.orchestra.run.smithi107:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 43be020184947e53516056c9931e1ac5bdbbb1a5 2024-04-22T23:32:59.189 DEBUG:teuthology.orchestra.run.smithi157:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout 43be020184947e53516056c9931e1ac5bdbbb1a5 2024-04-22T23:32:59.204 INFO:tasks.workunit.client.0.smithi107.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-22T23:32:59.244 INFO:tasks.workunit.client.1.smithi157.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-04-22T23:32:59.284 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:32:59.658 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:32:59 smithi107 ceph-mon[26355]: pgmap v196: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-04-22T23:32:59.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:32:59 smithi157 ceph-mon[33877]: pgmap v196: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-04-22T23:33:00.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:00.937+0000 7f0db30dc700 1 -- 172.21.15.107:0/1351331446 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0dac0fce20 msgr2=0x7f0dac0fd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:00.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:00.937+0000 7f0db30dc700 1 --2- 172.21.15.107:0/1351331446 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0dac0fce20 0x7f0dac0fd230 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0da8009ce0 tx=0x7f0da8004d10 comp rx=0 tx=0).stop 2024-04-22T23:33:00.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:00.937+0000 7f0db30dc700 1 -- 172.21.15.107:0/1351331446 shutdown_connections 2024-04-22T23:33:00.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:00.937+0000 7f0db30dc700 1 --2- 172.21.15.107:0/1351331446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dac0fdf90 0x7f0dac0fe400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:00.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:00.937+0000 7f0db30dc700 1 --2- 172.21.15.107:0/1351331446 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0dac0fce20 0x7f0dac0fd230 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:00.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:00.937+0000 7f0db30dc700 1 -- 172.21.15.107:0/1351331446 >> 172.21.15.107:0/1351331446 conn(0x7f0dac0f84e0 msgr2=0x7f0dac0fa910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:33:00.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:00.937+0000 7f0db30dc700 1 -- 172.21.15.107:0/1351331446 shutdown_connections 2024-04-22T23:33:00.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:00.937+0000 7f0db30dc700 1 -- 172.21.15.107:0/1351331446 wait complete. 2024-04-22T23:33:00.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:00.938+0000 7f0db30dc700 1 Processor -- start 2024-04-22T23:33:00.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:00.938+0000 7f0db30dc700 1 -- start start 2024-04-22T23:33:00.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:00.938+0000 7f0db30dc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dac0fce20 0x7f0dac071e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:00.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:00.938+0000 7f0db30dc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0dac0fdf90 0x7f0dac070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:00.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:00.938+0000 7f0db30dc700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dac0708c0 con 0x7f0dac0fdf90 2024-04-22T23:33:00.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:00.938+0000 7f0db30dc700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dac070a30 con 0x7f0dac0fce20 2024-04-22T23:33:00.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:00.938+0000 7f0da3fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0dac0fdf90 0x7f0dac070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:00.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:00.938+0000 7f0da3fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0dac0fdf90 0x7f0dac070310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42328/0 (socket says 172.21.15.107:42328) 2024-04-22T23:33:00.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:00.938+0000 7f0da3fff700 1 -- 172.21.15.107:0/1914492043 learned_addr learned my addr 172.21.15.107:0/1914492043 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:33:00.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:00.938+0000 7f0db0e78700 1 --2- 172.21.15.107:0/1914492043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dac0fce20 0x7f0dac071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:00.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:00.938+0000 7f0da3fff700 1 -- 172.21.15.107:0/1914492043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dac0fce20 msgr2=0x7f0dac071e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:00.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:00.938+0000 7f0da3fff700 1 --2- 172.21.15.107:0/1914492043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dac0fce20 0x7f0dac071e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:00.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:00.938+0000 7f0da3fff700 1 -- 172.21.15.107:0/1914492043 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da8009940 con 0x7f0dac0fdf90 2024-04-22T23:33:00.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:00.938+0000 7f0da3fff700 1 --2- 172.21.15.107:0/1914492043 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0dac0fdf90 0x7f0dac070310 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c002a50 tx=0x7f0d9c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:33:00.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:00.939+0000 7f0da17fa700 1 -- 172.21.15.107:0/1914492043 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d9c007cc0 con 0x7f0dac0fdf90 2024-04-22T23:33:00.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:00.939+0000 7f0da17fa700 1 -- 172.21.15.107:0/1914492043 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0d9c00f040 con 0x7f0dac0fdf90 2024-04-22T23:33:00.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:00.939+0000 7f0db30dc700 1 -- 172.21.15.107:0/1914492043 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dac070d10 con 0x7f0dac0fdf90 2024-04-22T23:33:00.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:00.939+0000 7f0da17fa700 1 -- 172.21.15.107:0/1914492043 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d9c009640 con 0x7f0dac0fdf90 2024-04-22T23:33:00.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:00.939+0000 7f0db30dc700 1 -- 172.21.15.107:0/1914492043 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dac115b60 con 0x7f0dac0fdf90 2024-04-22T23:33:00.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:00.940+0000 7f0db30dc700 1 -- 172.21.15.107:0/1914492043 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dac05f5f0 con 0x7f0dac0fdf90 2024-04-22T23:33:00.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:00.940+0000 7f0da17fa700 1 -- 172.21.15.107:0/1914492043 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f0d9c0097a0 con 0x7f0dac0fdf90 2024-04-22T23:33:00.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:00.941+0000 7f0da17fa700 1 --2- 172.21.15.107:0/1914492043 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f0d940692a0 0x7f0d9406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:00.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:00.941+0000 7f0da17fa700 1 -- 172.21.15.107:0/1914492043 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f0d9c087020 con 0x7f0dac0fdf90 2024-04-22T23:33:00.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:00.941+0000 7f0db0e78700 1 --2- 172.21.15.107:0/1914492043 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f0d940692a0 0x7f0d9406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:00.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:00.941+0000 7f0db0e78700 1 --2- 172.21.15.107:0/1914492043 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f0d940692a0 0x7f0d9406b750 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0da8015040 tx=0x7f0da8004d10 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:33:00.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:00.945+0000 7f0da17fa700 1 -- 172.21.15.107:0/1914492043 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0d9c053280 con 0x7f0dac0fdf90 2024-04-22T23:33:01.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:01.132+0000 7f0db30dc700 1 -- 172.21.15.107:0/1914492043 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) v1 -- 0x7f0dac04bed0 con 0x7f0dac0fdf90 2024-04-22T23:33:01.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:01.133+0000 7f0da17fa700 1 -- 172.21.15.107:0/1914492043 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=-22 unrecognized config option 'mgr/orchestrator/fail_fs' v21)=-22 unrecognized config option 'mgr/orchestrator/fail_fs' v21) v1 ==== 179+0+0 (secure 0 0 0) 0x7f0d9c0568a0 con 0x7f0dac0fdf90 2024-04-22T23:33:01.130 INFO:teuthology.orchestra.run.smithi107.stderr:Error EINVAL: unrecognized config option 'mgr/orchestrator/fail_fs' 2024-04-22T23:33:01.719 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:33:01 smithi107 ceph-mon[26355]: pgmap v197: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-04-22T23:33:01.744 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-22T23:33:01.744 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi107.front.sepia.ceph.com 2024-04-22T23:33:01.745 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -e sha1=43be020184947e53516056c9931e1ac5bdbbb1a5 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-04-22T23:33:01.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:33:01 smithi157 ceph-mon[33877]: pgmap v197: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-04-22T23:33:02.038 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:33:03.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:03.527+0000 7f0ec04d7700 1 -- 172.21.15.107:0/1966136416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0eb80fed30 msgr2=0x7f0eb80fae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:03.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:03.527+0000 7f0ec04d7700 1 --2- 172.21.15.107:0/1966136416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0eb80fed30 0x7f0eb80fae70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8004940 tx=0x7f0ea8009e30 comp rx=0 tx=0).stop 2024-04-22T23:33:03.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:03.527+0000 7f0ec04d7700 1 -- 172.21.15.107:0/1966136416 shutdown_connections 2024-04-22T23:33:03.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:03.527+0000 7f0ec04d7700 1 --2- 172.21.15.107:0/1966136416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0eb80fed30 0x7f0eb80fae70 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:03.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:03.527+0000 7f0ec04d7700 1 --2- 172.21.15.107:0/1966136416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0eb80fe3e0 0x7f0eb80fe7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:03.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:03.527+0000 7f0ec04d7700 1 -- 172.21.15.107:0/1966136416 >> 172.21.15.107:0/1966136416 conn(0x7f0eb80f6ba0 msgr2=0x7f0eb80f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:33:03.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:03.527+0000 7f0ec04d7700 1 -- 172.21.15.107:0/1966136416 shutdown_connections 2024-04-22T23:33:03.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:03.527+0000 7f0ec04d7700 1 -- 172.21.15.107:0/1966136416 wait complete. 2024-04-22T23:33:03.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:03.528+0000 7f0ec04d7700 1 Processor -- start 2024-04-22T23:33:03.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:03.528+0000 7f0ec04d7700 1 -- start start 2024-04-22T23:33:03.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:03.528+0000 7f0ec04d7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0eb80fe3e0 0x7f0eb80fdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:03.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:03.528+0000 7f0ec04d7700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0eb80fed30 0x7f0eb80fc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:03.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:03.528+0000 7f0ec04d7700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb80fc650 con 0x7f0eb80fe3e0 2024-04-22T23:33:03.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:03.528+0000 7f0ec04d7700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb80fc7c0 con 0x7f0eb80fed30 2024-04-22T23:33:03.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:03.528+0000 7f0ebe273700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0eb80fe3e0 0x7f0eb80fdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:03.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:03.528+0000 7f0ebe273700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0eb80fe3e0 0x7f0eb80fdbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46858/0 (socket says 172.21.15.107:46858) 2024-04-22T23:33:03.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:03.528+0000 7f0ebe273700 1 -- 172.21.15.107:0/3248269546 learned_addr learned my addr 172.21.15.107:0/3248269546 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:33:03.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:03.528+0000 7f0ebda72700 1 --2- 172.21.15.107:0/3248269546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0eb80fed30 0x7f0eb80fc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:03.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:03.529+0000 7f0ebe273700 1 -- 172.21.15.107:0/3248269546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0eb80fed30 msgr2=0x7f0eb80fc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:03.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:03.529+0000 7f0ebe273700 1 --2- 172.21.15.107:0/3248269546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0eb80fed30 0x7f0eb80fc0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:03.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:03.529+0000 7f0ebe273700 1 -- 172.21.15.107:0/3248269546 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea8005040 con 0x7f0eb80fe3e0 2024-04-22T23:33:03.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:03.529+0000 7f0ebe273700 1 --2- 172.21.15.107:0/3248269546 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0eb80fe3e0 0x7f0eb80fdbd0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0eb400c8c0 tx=0x7f0eb40044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:33:03.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:03.529+0000 7f0eaeffd700 1 -- 172.21.15.107:0/3248269546 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0eb400a760 con 0x7f0eb80fe3e0 2024-04-22T23:33:03.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:03.529+0000 7f0eaeffd700 1 -- 172.21.15.107:0/3248269546 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0eb400ad90 con 0x7f0eb80fe3e0 2024-04-22T23:33:03.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:03.529+0000 7f0eaeffd700 1 -- 172.21.15.107:0/3248269546 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0eb40113a0 con 0x7f0eb80fe3e0 2024-04-22T23:33:03.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:03.529+0000 7f0ec04d7700 1 -- 172.21.15.107:0/3248269546 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eb80fcaa0 con 0x7f0eb80fe3e0 2024-04-22T23:33:03.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:03.529+0000 7f0ec04d7700 1 -- 172.21.15.107:0/3248269546 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eb81159f0 con 0x7f0eb80fe3e0 2024-04-22T23:33:03.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:03.530+0000 7f0ec04d7700 1 -- 172.21.15.107:0/3248269546 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eb80fa820 con 0x7f0eb80fe3e0 2024-04-22T23:33:03.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:03.530+0000 7f0eaeffd700 1 -- 172.21.15.107:0/3248269546 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f0eb4012400 con 0x7f0eb80fe3e0 2024-04-22T23:33:03.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:03.531+0000 7f0eaeffd700 1 --2- 172.21.15.107:0/3248269546 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f0ea40692a0 0x7f0ea406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:03.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:03.531+0000 7f0eaeffd700 1 -- 172.21.15.107:0/3248269546 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f0eb40871a0 con 0x7f0eb80fe3e0 2024-04-22T23:33:03.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:03.531+0000 7f0ebda72700 1 --2- 172.21.15.107:0/3248269546 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f0ea40692a0 0x7f0ea406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:03.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:03.531+0000 7f0ebda72700 1 --2- 172.21.15.107:0/3248269546 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f0ea40692a0 0x7f0ea406b750 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0ea800b3b0 tx=0x7f0ea8009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:33:03.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:03.535+0000 7f0eaeffd700 1 -- 172.21.15.107:0/3248269546 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0eb4053520 con 0x7f0eb80fe3e0 2024-04-22T23:33:03.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:03.724+0000 7f0ec04d7700 1 -- 172.21.15.107:0/3248269546 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f0eb8105890 con 0x7f0eb80fe3e0 2024-04-22T23:33:03.720 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:33:03 smithi107 ceph-mon[26355]: pgmap v198: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 938 B/s wr, 0 op/s 2024-04-22T23:33:03.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:03.724+0000 7f0eaeffd700 1 -- 172.21.15.107:0/3248269546 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v21)=0 v21) v1 ==== 155+0+0 (secure 0 0 0) 0x7f0eb4056b40 con 0x7f0eb80fe3e0 2024-04-22T23:33:03.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:03.726+0000 7f0ec04d7700 1 -- 172.21.15.107:0/3248269546 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f0ea40692a0 msgr2=0x7f0ea406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:03.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:03.726+0000 7f0ec04d7700 1 --2- 172.21.15.107:0/3248269546 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f0ea40692a0 0x7f0ea406b750 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0ea800b3b0 tx=0x7f0ea8009e30 comp rx=0 tx=0).stop 2024-04-22T23:33:03.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:03.726+0000 7f0ec04d7700 1 -- 172.21.15.107:0/3248269546 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0eb80fe3e0 msgr2=0x7f0eb80fdbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:03.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:03.726+0000 7f0ec04d7700 1 --2- 172.21.15.107:0/3248269546 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0eb80fe3e0 0x7f0eb80fdbd0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0eb400c8c0 tx=0x7f0eb40044c0 comp rx=0 tx=0).stop 2024-04-22T23:33:03.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:03.726+0000 7f0ec04d7700 1 -- 172.21.15.107:0/3248269546 shutdown_connections 2024-04-22T23:33:03.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:03.726+0000 7f0ec04d7700 1 --2- 172.21.15.107:0/3248269546 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f0ea40692a0 0x7f0ea406b750 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:03.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:03.726+0000 7f0ec04d7700 1 --2- 172.21.15.107:0/3248269546 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0eb80fe3e0 0x7f0eb80fdbd0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:03.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:03.726+0000 7f0ec04d7700 1 --2- 172.21.15.107:0/3248269546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0eb80fed30 0x7f0eb80fc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:03.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:03.726+0000 7f0ec04d7700 1 -- 172.21.15.107:0/3248269546 >> 172.21.15.107:0/3248269546 conn(0x7f0eb80f6ba0 msgr2=0x7f0eb80f8240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:33:03.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:03.726+0000 7f0ec04d7700 1 -- 172.21.15.107:0/3248269546 shutdown_connections 2024-04-22T23:33:03.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:03.726+0000 7f0ec04d7700 1 -- 172.21.15.107:0/3248269546 wait complete. 2024-04-22T23:33:03.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:33:03 smithi157 ceph-mon[33877]: pgmap v198: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 938 B/s wr, 0 op/s 2024-04-22T23:33:04.441 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -e sha1=43be020184947e53516056c9931e1ac5bdbbb1a5 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-04-22T23:33:04.696 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:33:04.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:33:04 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:33:04.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:33:04 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:33:05.877 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:33:05 smithi107 ceph-mon[26355]: pgmap v199: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 938 B/s wr, 0 op/s 2024-04-22T23:33:05.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:33:05 smithi157 ceph-mon[33877]: pgmap v199: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 938 B/s wr, 0 op/s 2024-04-22T23:33:06.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:06.542+0000 7f4c987d7700 1 -- 172.21.15.107:0/1785695523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c900662f0 msgr2=0x7f4c900ffa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:06.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:06.542+0000 7f4c987d7700 1 --2- 172.21.15.107:0/1785695523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c900662f0 0x7f4c900ffa30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c004940 tx=0x7f4c8c009e30 comp rx=0 tx=0).stop 2024-04-22T23:33:06.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:06.542+0000 7f4c987d7700 1 -- 172.21.15.107:0/1785695523 shutdown_connections 2024-04-22T23:33:06.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:06.542+0000 7f4c987d7700 1 --2- 172.21.15.107:0/1785695523 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c900fff70 0x7f4c90102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:06.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:06.542+0000 7f4c987d7700 1 --2- 172.21.15.107:0/1785695523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c900662f0 0x7f4c900ffa30 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:06.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:06.542+0000 7f4c987d7700 1 -- 172.21.15.107:0/1785695523 >> 172.21.15.107:0/1785695523 conn(0x7f4c900f74c0 msgr2=0x7f4c900f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:33:06.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:06.542+0000 7f4c987d7700 1 -- 172.21.15.107:0/1785695523 shutdown_connections 2024-04-22T23:33:06.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:06.542+0000 7f4c987d7700 1 -- 172.21.15.107:0/1785695523 wait complete. 2024-04-22T23:33:06.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:06.542+0000 7f4c987d7700 1 Processor -- start 2024-04-22T23:33:06.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:06.542+0000 7f4c987d7700 1 -- start start 2024-04-22T23:33:06.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:06.542+0000 7f4c987d7700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c900662f0 0x7f4c9010da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:06.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:06.542+0000 7f4c987d7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c900fff70 0x7f4c9010df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:06.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:06.542+0000 7f4c987d7700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9010e580 con 0x7f4c900fff70 2024-04-22T23:33:06.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:06.542+0000 7f4c987d7700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9010edd0 con 0x7f4c900662f0 2024-04-22T23:33:06.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:06.543+0000 7f4c95d72700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c900fff70 0x7f4c9010df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:06.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:06.543+0000 7f4c95d72700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c900fff70 0x7f4c9010df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46866/0 (socket says 172.21.15.107:46866) 2024-04-22T23:33:06.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:06.543+0000 7f4c95d72700 1 -- 172.21.15.107:0/3194237462 learned_addr learned my addr 172.21.15.107:0/3194237462 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:33:06.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:06.543+0000 7f4c96573700 1 --2- 172.21.15.107:0/3194237462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c900662f0 0x7f4c9010da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:06.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:06.543+0000 7f4c95d72700 1 -- 172.21.15.107:0/3194237462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c900662f0 msgr2=0x7f4c9010da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:06.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:06.543+0000 7f4c95d72700 1 --2- 172.21.15.107:0/3194237462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c900662f0 0x7f4c9010da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:06.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:06.543+0000 7f4c95d72700 1 -- 172.21.15.107:0/3194237462 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c8c005040 con 0x7f4c900fff70 2024-04-22T23:33:06.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:06.543+0000 7f4c96573700 1 --2- 172.21.15.107:0/3194237462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c900662f0 0x7f4c9010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T23:33:06.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:06.543+0000 7f4c95d72700 1 --2- 172.21.15.107:0/3194237462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c900fff70 0x7f4c9010df80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4c8000c830 tx=0x7f4c80004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:33:06.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:06.543+0000 7f4c86ffd700 1 -- 172.21.15.107:0/3194237462 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c8000a760 con 0x7f4c900fff70 2024-04-22T23:33:06.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:06.543+0000 7f4c86ffd700 1 -- 172.21.15.107:0/3194237462 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4c8000ad90 con 0x7f4c900fff70 2024-04-22T23:33:06.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:06.543+0000 7f4c987d7700 1 -- 172.21.15.107:0/3194237462 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c9010f0b0 con 0x7f4c900fff70 2024-04-22T23:33:06.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:06.543+0000 7f4c86ffd700 1 -- 172.21.15.107:0/3194237462 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c800113a0 con 0x7f4c900fff70 2024-04-22T23:33:06.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:06.543+0000 7f4c987d7700 1 -- 172.21.15.107:0/3194237462 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c9010f5f0 con 0x7f4c900fff70 2024-04-22T23:33:06.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:06.544+0000 7f4c987d7700 1 -- 172.21.15.107:0/3194237462 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c9005f5f0 con 0x7f4c900fff70 2024-04-22T23:33:06.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:06.544+0000 7f4c86ffd700 1 -- 172.21.15.107:0/3194237462 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f4c80012400 con 0x7f4c900fff70 2024-04-22T23:33:06.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:06.545+0000 7f4c86ffd700 1 --2- 172.21.15.107:0/3194237462 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f4c7c0692a0 0x7f4c7c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:06.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:06.545+0000 7f4c86ffd700 1 -- 172.21.15.107:0/3194237462 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f4c80087290 con 0x7f4c900fff70 2024-04-22T23:33:06.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:06.545+0000 7f4c96573700 1 --2- 172.21.15.107:0/3194237462 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f4c7c0692a0 0x7f4c7c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:06.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:06.546+0000 7f4c96573700 1 --2- 172.21.15.107:0/3194237462 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f4c7c0692a0 0x7f4c7c06b750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c00b3b0 tx=0x7f4c8c009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:33:06.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:06.549+0000 7f4c86ffd700 1 -- 172.21.15.107:0/3194237462 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4c80057000 con 0x7f4c900fff70 2024-04-22T23:33:06.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:06.734+0000 7f4c987d7700 1 -- 172.21.15.107:0/3194237462 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f4c9004bed0 con 0x7f4c900fff70 2024-04-22T23:33:06.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:06.734+0000 7f4c86ffd700 1 -- 172.21.15.107:0/3194237462 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v21)=0 v21) v1 ==== 163+0+0 (secure 0 0 0) 0x7f4c80056b90 con 0x7f4c900fff70 2024-04-22T23:33:06.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:06.737+0000 7f4c987d7700 1 -- 172.21.15.107:0/3194237462 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f4c7c0692a0 msgr2=0x7f4c7c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:06.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:06.737+0000 7f4c987d7700 1 --2- 172.21.15.107:0/3194237462 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f4c7c0692a0 0x7f4c7c06b750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c00b3b0 tx=0x7f4c8c009e30 comp rx=0 tx=0).stop 2024-04-22T23:33:06.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:06.737+0000 7f4c987d7700 1 -- 172.21.15.107:0/3194237462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c900fff70 msgr2=0x7f4c9010df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:06.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:06.737+0000 7f4c987d7700 1 --2- 172.21.15.107:0/3194237462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c900fff70 0x7f4c9010df80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4c8000c830 tx=0x7f4c80004470 comp rx=0 tx=0).stop 2024-04-22T23:33:06.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:06.737+0000 7f4c987d7700 1 -- 172.21.15.107:0/3194237462 shutdown_connections 2024-04-22T23:33:06.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:06.737+0000 7f4c987d7700 1 --2- 172.21.15.107:0/3194237462 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f4c7c0692a0 0x7f4c7c06b750 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:06.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:06.737+0000 7f4c987d7700 1 --2- 172.21.15.107:0/3194237462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c900662f0 0x7f4c9010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:06.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:06.737+0000 7f4c987d7700 1 --2- 172.21.15.107:0/3194237462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c900fff70 0x7f4c9010df80 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:06.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:06.737+0000 7f4c987d7700 1 -- 172.21.15.107:0/3194237462 >> 172.21.15.107:0/3194237462 conn(0x7f4c900f74c0 msgr2=0x7f4c900f7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:33:06.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:06.737+0000 7f4c987d7700 1 -- 172.21.15.107:0/3194237462 shutdown_connections 2024-04-22T23:33:06.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:06.737+0000 7f4c987d7700 1 -- 172.21.15.107:0/3194237462 wait complete. 2024-04-22T23:33:07.534 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -e sha1=43be020184947e53516056c9931e1ac5bdbbb1a5 -- bash -c 'ceph config set global log_to_journald false --force' 2024-04-22T23:33:07.801 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:33:07.815 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:33:07 smithi107 ceph-mon[26355]: pgmap v200: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 938 B/s wr, 0 op/s 2024-04-22T23:33:07.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:33:07 smithi157 ceph-mon[33877]: pgmap v200: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 938 B/s wr, 0 op/s 2024-04-22T23:33:09.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:09.420+0000 7fc81c05d700 1 -- 172.21.15.107:0/1423872750 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8140fdff0 msgr2=0x7fc8140fe460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:09.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:09.420+0000 7fc81c05d700 1 --2- 172.21.15.107:0/1423872750 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8140fdff0 0x7fc8140fe460 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc804004940 tx=0x7fc804009e30 comp rx=0 tx=0).stop 2024-04-22T23:33:09.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:09.420+0000 7fc81c05d700 1 -- 172.21.15.107:0/1423872750 shutdown_connections 2024-04-22T23:33:09.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:09.420+0000 7fc81c05d700 1 --2- 172.21.15.107:0/1423872750 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8140fdff0 0x7fc8140fe460 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:09.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:09.420+0000 7fc81c05d700 1 --2- 172.21.15.107:0/1423872750 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8140fce80 0x7fc8140fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:09.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:09.420+0000 7fc81c05d700 1 -- 172.21.15.107:0/1423872750 >> 172.21.15.107:0/1423872750 conn(0x7fc8140f8580 msgr2=0x7fc8140fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:33:09.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:09.420+0000 7fc81c05d700 1 -- 172.21.15.107:0/1423872750 shutdown_connections 2024-04-22T23:33:09.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:09.420+0000 7fc81c05d700 1 -- 172.21.15.107:0/1423872750 wait complete. 2024-04-22T23:33:09.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:09.420+0000 7fc81c05d700 1 Processor -- start 2024-04-22T23:33:09.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:09.420+0000 7fc81c05d700 1 -- start start 2024-04-22T23:33:09.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:09.420+0000 7fc81c05d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8140fce80 0x7fc81410d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:09.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:09.421+0000 7fc81c05d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8140fdff0 0x7fc81410dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:09.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:09.421+0000 7fc81c05d700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc81410e530 con 0x7fc8140fdff0 2024-04-22T23:33:09.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:09.421+0000 7fc81c05d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc814112ed0 con 0x7fc8140fce80 2024-04-22T23:33:09.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:09.421+0000 7fc8195f8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8140fdff0 0x7fc81410dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:09.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:09.421+0000 7fc8195f8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8140fdff0 0x7fc81410dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46884/0 (socket says 172.21.15.107:46884) 2024-04-22T23:33:09.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:09.421+0000 7fc8195f8700 1 -- 172.21.15.107:0/46567637 learned_addr learned my addr 172.21.15.107:0/46567637 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:33:09.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:09.421+0000 7fc819df9700 1 --2- 172.21.15.107:0/46567637 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8140fce80 0x7fc81410d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:09.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:09.421+0000 7fc8195f8700 1 -- 172.21.15.107:0/46567637 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8140fce80 msgr2=0x7fc81410d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:09.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:09.421+0000 7fc8195f8700 1 --2- 172.21.15.107:0/46567637 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8140fce80 0x7fc81410d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:09.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:09.421+0000 7fc8195f8700 1 -- 172.21.15.107:0/46567637 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc804005040 con 0x7fc8140fdff0 2024-04-22T23:33:09.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:09.421+0000 7fc8195f8700 1 --2- 172.21.15.107:0/46567637 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8140fdff0 0x7fc81410dea0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc80400b3b0 tx=0x7fc804009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:33:09.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:09.421+0000 7fc80a7fc700 1 -- 172.21.15.107:0/46567637 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc80400e040 con 0x7fc8140fdff0 2024-04-22T23:33:09.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:09.421+0000 7fc80a7fc700 1 -- 172.21.15.107:0/46567637 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc80400bcc0 con 0x7fc8140fdff0 2024-04-22T23:33:09.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:09.421+0000 7fc81c05d700 1 -- 172.21.15.107:0/46567637 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8141130d0 con 0x7fc8140fdff0 2024-04-22T23:33:09.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:09.421+0000 7fc80a7fc700 1 -- 172.21.15.107:0/46567637 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc80401c630 con 0x7fc8140fdff0 2024-04-22T23:33:09.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:09.421+0000 7fc81c05d700 1 -- 172.21.15.107:0/46567637 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc814113640 con 0x7fc8140fdff0 2024-04-22T23:33:09.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:09.422+0000 7fc81c05d700 1 -- 172.21.15.107:0/46567637 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc81404c710 con 0x7fc8140fdff0 2024-04-22T23:33:09.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:09.423+0000 7fc80a7fc700 1 -- 172.21.15.107:0/46567637 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fc804013980 con 0x7fc8140fdff0 2024-04-22T23:33:09.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:09.423+0000 7fc80a7fc700 1 --2- 172.21.15.107:0/46567637 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fc8000692a0 0x7fc80006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:09.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:09.423+0000 7fc80a7fc700 1 -- 172.21.15.107:0/46567637 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fc804087f40 con 0x7fc8140fdff0 2024-04-22T23:33:09.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:09.423+0000 7fc819df9700 1 --2- 172.21.15.107:0/46567637 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fc8000692a0 0x7fc80006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:09.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:09.423+0000 7fc819df9700 1 --2- 172.21.15.107:0/46567637 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fc8000692a0 0x7fc80006b750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc810006fd0 tx=0x7fc810008040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:33:09.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:09.428+0000 7fc80a7fc700 1 -- 172.21.15.107:0/46567637 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fc8040ba250 con 0x7fc8140fdff0 2024-04-22T23:33:09.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:09.616+0000 7fc81c05d700 1 -- 172.21.15.107:0/46567637 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7fc814113ae0 con 0x7fc8140fdff0 2024-04-22T23:33:09.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:09.616+0000 7fc80a7fc700 1 -- 172.21.15.107:0/46567637 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v21)=0 v21) v1 ==== 135+0+0 (secure 0 0 0) 0x7fc804057cb0 con 0x7fc8140fdff0 2024-04-22T23:33:09.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:09.618+0000 7fc81c05d700 1 -- 172.21.15.107:0/46567637 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fc8000692a0 msgr2=0x7fc80006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:09.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:09.618+0000 7fc81c05d700 1 --2- 172.21.15.107:0/46567637 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fc8000692a0 0x7fc80006b750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc810006fd0 tx=0x7fc810008040 comp rx=0 tx=0).stop 2024-04-22T23:33:09.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:09.618+0000 7fc81c05d700 1 -- 172.21.15.107:0/46567637 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8140fdff0 msgr2=0x7fc81410dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:09.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:09.618+0000 7fc81c05d700 1 --2- 172.21.15.107:0/46567637 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8140fdff0 0x7fc81410dea0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc80400b3b0 tx=0x7fc804009e30 comp rx=0 tx=0).stop 2024-04-22T23:33:09.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:09.618+0000 7fc81c05d700 1 -- 172.21.15.107:0/46567637 shutdown_connections 2024-04-22T23:33:09.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:09.618+0000 7fc81c05d700 1 --2- 172.21.15.107:0/46567637 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fc8000692a0 0x7fc80006b750 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:09.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:09.618+0000 7fc81c05d700 1 --2- 172.21.15.107:0/46567637 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8140fce80 0x7fc81410d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:09.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:09.618+0000 7fc81c05d700 1 --2- 172.21.15.107:0/46567637 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8140fdff0 0x7fc81410dea0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:09.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:09.618+0000 7fc81c05d700 1 -- 172.21.15.107:0/46567637 >> 172.21.15.107:0/46567637 conn(0x7fc8140f8580 msgr2=0x7fc8140f9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:33:09.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:09.619+0000 7fc81c05d700 1 -- 172.21.15.107:0/46567637 shutdown_connections 2024-04-22T23:33:09.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:09.619+0000 7fc81c05d700 1 -- 172.21.15.107:0/46567637 wait complete. 2024-04-22T23:33:09.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:33:09 smithi157 ceph-mon[33877]: pgmap v201: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 938 B/s wr, 0 op/s 2024-04-22T23:33:09.985 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:33:09 smithi107 ceph-mon[26355]: pgmap v201: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 938 B/s wr, 0 op/s 2024-04-22T23:33:10.393 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -e sha1=43be020184947e53516056c9931e1ac5bdbbb1a5 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-04-22T23:33:10.652 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:33:11.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:33:11 smithi107 ceph-mon[26355]: pgmap v202: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-04-22T23:33:11.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:33:11 smithi157 ceph-mon[33877]: pgmap v202: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-04-22T23:33:12.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:12.401+0000 7fa21ddaf700 1 -- 172.21.15.107:0/3842391579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa218066f20 msgr2=0x7fa218101e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:12.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:12.401+0000 7fa21ddaf700 1 --2- 172.21.15.107:0/3842391579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa218066f20 0x7fa218101e50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa208004940 tx=0x7fa208009e30 comp rx=0 tx=0).stop 2024-04-22T23:33:12.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:12.401+0000 7fa21ddaf700 1 -- 172.21.15.107:0/3842391579 shutdown_connections 2024-04-22T23:33:12.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:12.401+0000 7fa21ddaf700 1 --2- 172.21.15.107:0/3842391579 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa218102390 0x7fa218104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:12.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:12.401+0000 7fa21ddaf700 1 --2- 172.21.15.107:0/3842391579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa218066f20 0x7fa218101e50 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:12.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:12.401+0000 7fa21ddaf700 1 -- 172.21.15.107:0/3842391579 >> 172.21.15.107:0/3842391579 conn(0x7fa2180f7480 msgr2=0x7fa2180f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:33:12.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:12.401+0000 7fa21ddaf700 1 -- 172.21.15.107:0/3842391579 shutdown_connections 2024-04-22T23:33:12.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:12.401+0000 7fa21ddaf700 1 -- 172.21.15.107:0/3842391579 wait complete. 2024-04-22T23:33:12.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:12.402+0000 7fa21ddaf700 1 Processor -- start 2024-04-22T23:33:12.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:12.402+0000 7fa21ddaf700 1 -- start start 2024-04-22T23:33:12.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:12.402+0000 7fa21ddaf700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa218066f20 0x7fa21810fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:12.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:12.402+0000 7fa21ddaf700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa218102390 0x7fa218110140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:12.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:12.402+0000 7fa21ddaf700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2181107d0 con 0x7fa218066f20 2024-04-22T23:33:12.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:12.402+0000 7fa21ddaf700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa218115170 con 0x7fa218102390 2024-04-22T23:33:12.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:12.402+0000 7fa2177fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa218066f20 0x7fa21810fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:12.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:12.402+0000 7fa2177fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa218066f20 0x7fa21810fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59512/0 (socket says 172.21.15.107:59512) 2024-04-22T23:33:12.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:12.402+0000 7fa2177fe700 1 -- 172.21.15.107:0/3304047108 learned_addr learned my addr 172.21.15.107:0/3304047108 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:33:12.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:12.403+0000 7fa216ffd700 1 --2- 172.21.15.107:0/3304047108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa218102390 0x7fa218110140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:12.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:12.403+0000 7fa2177fe700 1 -- 172.21.15.107:0/3304047108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa218102390 msgr2=0x7fa218110140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:12.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:12.403+0000 7fa2177fe700 1 --2- 172.21.15.107:0/3304047108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa218102390 0x7fa218110140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:12.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:12.403+0000 7fa2177fe700 1 -- 172.21.15.107:0/3304047108 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa208005040 con 0x7fa218066f20 2024-04-22T23:33:12.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:12.403+0000 7fa2177fe700 1 --2- 172.21.15.107:0/3304047108 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa218066f20 0x7fa21810fc00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa208009fd0 tx=0x7fa208009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:33:12.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:12.403+0000 7fa1fffff700 1 -- 172.21.15.107:0/3304047108 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa20800e040 con 0x7fa218066f20 2024-04-22T23:33:12.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:12.403+0000 7fa21ddaf700 1 -- 172.21.15.107:0/3304047108 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa218115370 con 0x7fa218066f20 2024-04-22T23:33:12.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:12.403+0000 7fa1fffff700 1 -- 172.21.15.107:0/3304047108 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa20800bbf0 con 0x7fa218066f20 2024-04-22T23:33:12.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:12.403+0000 7fa1fffff700 1 -- 172.21.15.107:0/3304047108 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa20801c3f0 con 0x7fa218066f20 2024-04-22T23:33:12.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:12.403+0000 7fa21ddaf700 1 -- 172.21.15.107:0/3304047108 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2181158e0 con 0x7fa218066f20 2024-04-22T23:33:12.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:12.404+0000 7fa21ddaf700 1 -- 172.21.15.107:0/3304047108 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa21804c710 con 0x7fa218066f20 2024-04-22T23:33:12.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:12.404+0000 7fa1fffff700 1 -- 172.21.15.107:0/3304047108 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fa208013420 con 0x7fa218066f20 2024-04-22T23:33:12.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:12.405+0000 7fa1fffff700 1 --2- 172.21.15.107:0/3304047108 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fa2000692a0 0x7fa20006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:12.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:12.405+0000 7fa1fffff700 1 -- 172.21.15.107:0/3304047108 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fa208088530 con 0x7fa218066f20 2024-04-22T23:33:12.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:12.405+0000 7fa216ffd700 1 --2- 172.21.15.107:0/3304047108 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fa2000692a0 0x7fa20006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:12.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:12.405+0000 7fa216ffd700 1 --2- 172.21.15.107:0/3304047108 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fa2000692a0 0x7fa20006b750 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa20c006fd0 tx=0x7fa20c008040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:33:12.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:12.409+0000 7fa1fffff700 1 -- 172.21.15.107:0/3304047108 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fa2080582a0 con 0x7fa218066f20 2024-04-22T23:33:12.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:12.613+0000 7fa21ddaf700 1 -- 172.21.15.107:0/3304047108 --> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:43be020184947e53516056c9931e1ac5bdbbb1a5", "target": ["mon-mgr", ""]}) v1 -- 0x7fa2180f8f70 con 0x7fa2000692a0 2024-04-22T23:33:12.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:12.620+0000 7fa1fffff700 1 -- 172.21.15.107:0/3304047108 <== mgr.14206 v2:172.21.15.107:6800/2516490111 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7fa2180f8f70 con 0x7fa2000692a0 2024-04-22T23:33:12.617 INFO:teuthology.orchestra.run.smithi107.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:43be020184947e53516056c9931e1ac5bdbbb1a5 2024-04-22T23:33:12.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:12.622+0000 7fa21ddaf700 1 -- 172.21.15.107:0/3304047108 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fa2000692a0 msgr2=0x7fa20006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:12.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:12.622+0000 7fa21ddaf700 1 --2- 172.21.15.107:0/3304047108 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fa2000692a0 0x7fa20006b750 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa20c006fd0 tx=0x7fa20c008040 comp rx=0 tx=0).stop 2024-04-22T23:33:12.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:12.622+0000 7fa21ddaf700 1 -- 172.21.15.107:0/3304047108 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa218066f20 msgr2=0x7fa21810fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:12.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:12.622+0000 7fa21ddaf700 1 --2- 172.21.15.107:0/3304047108 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa218066f20 0x7fa21810fc00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa208009fd0 tx=0x7fa208009e30 comp rx=0 tx=0).stop 2024-04-22T23:33:12.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:12.622+0000 7fa21ddaf700 1 -- 172.21.15.107:0/3304047108 shutdown_connections 2024-04-22T23:33:12.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:12.622+0000 7fa21ddaf700 1 --2- 172.21.15.107:0/3304047108 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fa2000692a0 0x7fa20006b750 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:12.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:12.622+0000 7fa21ddaf700 1 --2- 172.21.15.107:0/3304047108 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa218066f20 0x7fa21810fc00 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:12.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:12.622+0000 7fa21ddaf700 1 --2- 172.21.15.107:0/3304047108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa218102390 0x7fa218110140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:12.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:12.622+0000 7fa21ddaf700 1 -- 172.21.15.107:0/3304047108 >> 172.21.15.107:0/3304047108 conn(0x7fa2180f7480 msgr2=0x7fa2180f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:33:12.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:12.622+0000 7fa21ddaf700 1 -- 172.21.15.107:0/3304047108 shutdown_connections 2024-04-22T23:33:12.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:12.622+0000 7fa21ddaf700 1 -- 172.21.15.107:0/3304047108 wait complete. 2024-04-22T23:33:13.428 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-22T23:33:13.428 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi107.front.sepia.ceph.com 2024-04-22T23:33:13.429 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -e sha1=43be020184947e53516056c9931e1ac5bdbbb1a5 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-04-22T23:33:13.697 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:33:13.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:33:13 smithi107 ceph-mon[26355]: pgmap v203: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-04-22T23:33:13.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:33:13 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:33:13.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:33:13 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:33:13.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:33:13 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:33:13.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:33:13 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:33:13.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:33:13 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:33:13.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:33:13 smithi157 ceph-mon[33877]: pgmap v203: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-04-22T23:33:13.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:33:13 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:33:13.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:33:13 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:33:13.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:33:13 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:33:13.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:33:13 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:33:13.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:33:13 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:33:14.816 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:33:14 smithi107 ceph-mon[26355]: from='client.14548 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:43be020184947e53516056c9931e1ac5bdbbb1a5", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:33:14.816 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:33:14 smithi107 ceph-mon[26355]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:43be020184947e53516056c9931e1ac5bdbbb1a5 2024-04-22T23:33:14.816 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:33:14 smithi107 ceph-mon[26355]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:43be020184947e53516056c9931e1ac5bdbbb1a5 2024-04-22T23:33:14.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:33:14 smithi157 ceph-mon[33877]: from='client.14548 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:43be020184947e53516056c9931e1ac5bdbbb1a5", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:33:14.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:33:14 smithi157 ceph-mon[33877]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:43be020184947e53516056c9931e1ac5bdbbb1a5 2024-04-22T23:33:14.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:33:14 smithi157 ceph-mon[33877]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:43be020184947e53516056c9931e1ac5bdbbb1a5 2024-04-22T23:33:15.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.316+0000 7fc1e4964700 1 -- 172.21.15.107:0/3397478140 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1e00fce60 msgr2=0x7fc1e00fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:15.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.316+0000 7fc1e4964700 1 --2- 172.21.15.107:0/3397478140 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1e00fce60 0x7fc1e00fd270 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc1d8009ce0 tx=0x7fc1d8004d10 comp rx=0 tx=0).stop 2024-04-22T23:33:15.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.316+0000 7fc1e4964700 1 -- 172.21.15.107:0/3397478140 shutdown_connections 2024-04-22T23:33:15.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.316+0000 7fc1e4964700 1 --2- 172.21.15.107:0/3397478140 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1e00fe060 0x7fc1e00fe4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:15.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.316+0000 7fc1e4964700 1 --2- 172.21.15.107:0/3397478140 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1e00fce60 0x7fc1e00fd270 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:15.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.316+0000 7fc1e4964700 1 -- 172.21.15.107:0/3397478140 >> 172.21.15.107:0/3397478140 conn(0x7fc1e00f8570 msgr2=0x7fc1e00fa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:33:15.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.316+0000 7fc1e4964700 1 -- 172.21.15.107:0/3397478140 shutdown_connections 2024-04-22T23:33:15.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.316+0000 7fc1e4964700 1 -- 172.21.15.107:0/3397478140 wait complete. 2024-04-22T23:33:15.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.317+0000 7fc1e4964700 1 Processor -- start 2024-04-22T23:33:15.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.317+0000 7fc1e4964700 1 -- start start 2024-04-22T23:33:15.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.317+0000 7fc1e4964700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1e00fce60 0x7fc1e010dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:15.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.317+0000 7fc1e4964700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1e00fe060 0x7fc1e010e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:15.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.317+0000 7fc1e4964700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e010e790 con 0x7fc1e00fce60 2024-04-22T23:33:15.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.317+0000 7fc1e4964700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e010ef80 con 0x7fc1e00fe060 2024-04-22T23:33:15.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.317+0000 7fc1de59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1e00fce60 0x7fc1e010dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:15.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.317+0000 7fc1de59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1e00fce60 0x7fc1e010dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59520/0 (socket says 172.21.15.107:59520) 2024-04-22T23:33:15.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.317+0000 7fc1de59c700 1 -- 172.21.15.107:0/2392450476 learned_addr learned my addr 172.21.15.107:0/2392450476 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:33:15.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.317+0000 7fc1ddd9b700 1 --2- 172.21.15.107:0/2392450476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1e00fe060 0x7fc1e010e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:15.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.317+0000 7fc1de59c700 1 -- 172.21.15.107:0/2392450476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1e00fe060 msgr2=0x7fc1e010e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:15.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.317+0000 7fc1de59c700 1 --2- 172.21.15.107:0/2392450476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1e00fe060 0x7fc1e010e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:15.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.317+0000 7fc1de59c700 1 -- 172.21.15.107:0/2392450476 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1d8009940 con 0x7fc1e00fce60 2024-04-22T23:33:15.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.318+0000 7fc1de59c700 1 --2- 172.21.15.107:0/2392450476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1e00fce60 0x7fc1e010dbc0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc1d8009380 tx=0x7fc1d8004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:33:15.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.318+0000 7fc1ceffd700 1 -- 172.21.15.107:0/2392450476 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1d800e040 con 0x7fc1e00fce60 2024-04-22T23:33:15.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.318+0000 7fc1e4964700 1 -- 172.21.15.107:0/2392450476 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1e010f200 con 0x7fc1e00fce60 2024-04-22T23:33:15.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.318+0000 7fc1ceffd700 1 -- 172.21.15.107:0/2392450476 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc1d80128e0 con 0x7fc1e00fce60 2024-04-22T23:33:15.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.318+0000 7fc1ceffd700 1 -- 172.21.15.107:0/2392450476 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1d801c910 con 0x7fc1e00fce60 2024-04-22T23:33:15.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.318+0000 7fc1e4964700 1 -- 172.21.15.107:0/2392450476 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1e010f7c0 con 0x7fc1e00fce60 2024-04-22T23:33:15.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.319+0000 7fc1e4964700 1 -- 172.21.15.107:0/2392450476 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1e005f5f0 con 0x7fc1e00fce60 2024-04-22T23:33:15.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.319+0000 7fc1ceffd700 1 -- 172.21.15.107:0/2392450476 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fc1d8012a50 con 0x7fc1e00fce60 2024-04-22T23:33:15.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.319+0000 7fc1ceffd700 1 --2- 172.21.15.107:0/2392450476 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fc1c80692a0 0x7fc1c806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:15.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.320+0000 7fc1ceffd700 1 -- 172.21.15.107:0/2392450476 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fc1d8088ea0 con 0x7fc1e00fce60 2024-04-22T23:33:15.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.320+0000 7fc1ddd9b700 1 --2- 172.21.15.107:0/2392450476 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fc1c80692a0 0x7fc1c806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:15.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.320+0000 7fc1ddd9b700 1 --2- 172.21.15.107:0/2392450476 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fc1c80692a0 0x7fc1c806b750 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc1d4009bf0 tx=0x7fc1d40091a0 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:33:15.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.324+0000 7fc1ceffd700 1 -- 172.21.15.107:0/2392450476 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fc1d8058b90 con 0x7fc1e00fce60 2024-04-22T23:33:15.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.525+0000 7fc1e4964700 1 -- 172.21.15.107:0/2392450476 --> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc1e010faa0 con 0x7fc1c80692a0 2024-04-22T23:33:15.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.527+0000 7fc1ceffd700 1 -- 172.21.15.107:0/2392450476 <== mgr.14206 v2:172.21.15.107:6800/2516490111 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7fc1e010faa0 con 0x7fc1c80692a0 2024-04-22T23:33:15.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.528+0000 7fc1e4964700 1 -- 172.21.15.107:0/2392450476 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fc1c80692a0 msgr2=0x7fc1c806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:15.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.528+0000 7fc1e4964700 1 --2- 172.21.15.107:0/2392450476 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fc1c80692a0 0x7fc1c806b750 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc1d4009bf0 tx=0x7fc1d40091a0 comp rx=0 tx=0).stop 2024-04-22T23:33:15.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.528+0000 7fc1e4964700 1 -- 172.21.15.107:0/2392450476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1e00fce60 msgr2=0x7fc1e010dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:15.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.528+0000 7fc1e4964700 1 --2- 172.21.15.107:0/2392450476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1e00fce60 0x7fc1e010dbc0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc1d8009380 tx=0x7fc1d8004d10 comp rx=0 tx=0).stop 2024-04-22T23:33:15.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.529+0000 7fc1e4964700 1 -- 172.21.15.107:0/2392450476 shutdown_connections 2024-04-22T23:33:15.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.529+0000 7fc1e4964700 1 --2- 172.21.15.107:0/2392450476 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fc1c80692a0 0x7fc1c806b750 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:15.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.529+0000 7fc1e4964700 1 --2- 172.21.15.107:0/2392450476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1e00fce60 0x7fc1e010dbc0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:15.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.529+0000 7fc1e4964700 1 --2- 172.21.15.107:0/2392450476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1e00fe060 0x7fc1e010e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:15.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.529+0000 7fc1e4964700 1 -- 172.21.15.107:0/2392450476 >> 172.21.15.107:0/2392450476 conn(0x7fc1e00f8570 msgr2=0x7fc1e00f9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:33:15.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.529+0000 7fc1e4964700 1 -- 172.21.15.107:0/2392450476 shutdown_connections 2024-04-22T23:33:15.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.529+0000 7fc1e4964700 1 -- 172.21.15.107:0/2392450476 wait complete. 2024-04-22T23:33:15.536 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-04-22T23:33:15.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.630+0000 7f71713b0700 1 -- 172.21.15.107:0/2655725923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f716c0fbd40 msgr2=0x7f716c0fc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:15.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.630+0000 7f71713b0700 1 --2- 172.21.15.107:0/2655725923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f716c0fbd40 0x7f716c0fc150 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f715c009ce0 tx=0x7f715c004d10 comp rx=0 tx=0).stop 2024-04-22T23:33:15.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.630+0000 7f71713b0700 1 -- 172.21.15.107:0/2655725923 shutdown_connections 2024-04-22T23:33:15.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.630+0000 7f71713b0700 1 --2- 172.21.15.107:0/2655725923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f716c0fcff0 0x7f716c0fd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:15.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.630+0000 7f71713b0700 1 --2- 172.21.15.107:0/2655725923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f716c0fbd40 0x7f716c0fc150 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:15.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.630+0000 7f71713b0700 1 -- 172.21.15.107:0/2655725923 >> 172.21.15.107:0/2655725923 conn(0x7f716c0f74c0 msgr2=0x7f716c0f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:33:15.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.630+0000 7f71713b0700 1 -- 172.21.15.107:0/2655725923 shutdown_connections 2024-04-22T23:33:15.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.630+0000 7f71713b0700 1 -- 172.21.15.107:0/2655725923 wait complete. 2024-04-22T23:33:15.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.630+0000 7f71713b0700 1 Processor -- start 2024-04-22T23:33:15.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.630+0000 7f71713b0700 1 -- start start 2024-04-22T23:33:15.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.631+0000 7f71713b0700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f716c0fbd40 0x7f716c1108f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:15.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.631+0000 7f71713b0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f716c0fcff0 0x7f716c112e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:15.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.631+0000 7f71713b0700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716c0663a0 con 0x7f716c0fcff0 2024-04-22T23:33:15.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.631+0000 7f71713b0700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716c0664e0 con 0x7f716c0fbd40 2024-04-22T23:33:15.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.631+0000 7f716a7fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f716c0fcff0 0x7f716c112e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:15.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.631+0000 7f716a7fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f716c0fcff0 0x7f716c112e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59536/0 (socket says 172.21.15.107:59536) 2024-04-22T23:33:15.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.631+0000 7f716affd700 1 --2- 172.21.15.107:0/951981737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f716c0fbd40 0x7f716c1108f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:15.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.631+0000 7f716a7fc700 1 -- 172.21.15.107:0/951981737 learned_addr learned my addr 172.21.15.107:0/951981737 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:33:15.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.631+0000 7f716a7fc700 1 -- 172.21.15.107:0/951981737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f716c0fbd40 msgr2=0x7f716c1108f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:15.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.631+0000 7f716a7fc700 1 --2- 172.21.15.107:0/951981737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f716c0fbd40 0x7f716c1108f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:15.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.631+0000 7f716a7fc700 1 -- 172.21.15.107:0/951981737 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f715c009940 con 0x7f716c0fcff0 2024-04-22T23:33:15.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.631+0000 7f716a7fc700 1 --2- 172.21.15.107:0/951981737 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f716c0fcff0 0x7f716c112e40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f716000b790 tx=0x7f7160009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:33:15.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.631+0000 7f71537fe700 1 -- 172.21.15.107:0/951981737 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f716000f6f0 con 0x7f716c0fcff0 2024-04-22T23:33:15.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.631+0000 7f71537fe700 1 -- 172.21.15.107:0/951981737 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f716000fd20 con 0x7f716c0fcff0 2024-04-22T23:33:15.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.631+0000 7f71537fe700 1 -- 172.21.15.107:0/951981737 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71600103a0 con 0x7f716c0fcff0 2024-04-22T23:33:15.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.631+0000 7f71713b0700 1 -- 172.21.15.107:0/951981737 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f716c113440 con 0x7f716c0fcff0 2024-04-22T23:33:15.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.632+0000 7f71713b0700 1 -- 172.21.15.107:0/951981737 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f716c1139b0 con 0x7f716c0fcff0 2024-04-22T23:33:15.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.632+0000 7f71713b0700 1 -- 172.21.15.107:0/951981737 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f716c05f5f0 con 0x7f716c0fcff0 2024-04-22T23:33:15.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.633+0000 7f71537fe700 1 -- 172.21.15.107:0/951981737 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f7160011400 con 0x7f716c0fcff0 2024-04-22T23:33:15.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.633+0000 7f71537fe700 1 --2- 172.21.15.107:0/951981737 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f71540692a0 0x7f715406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:15.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.633+0000 7f71537fe700 1 -- 172.21.15.107:0/951981737 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f7160086fd0 con 0x7f716c0fcff0 2024-04-22T23:33:15.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.633+0000 7f716affd700 1 --2- 172.21.15.107:0/951981737 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f71540692a0 0x7f715406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:15.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.634+0000 7f716affd700 1 --2- 172.21.15.107:0/951981737 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f71540692a0 0x7f715406b750 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f715c009470 tx=0x7f715c004d10 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:33:15.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.638+0000 7f71537fe700 1 -- 172.21.15.107:0/951981737 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f71600532b0 con 0x7f716c0fcff0 2024-04-22T23:33:15.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.841+0000 7f71713b0700 1 -- 172.21.15.107:0/951981737 --> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f716c1012e0 con 0x7f71540692a0 2024-04-22T23:33:15.838 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:33:15 smithi107 ceph-mon[26355]: pgmap v204: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:33:15.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.842+0000 7f71537fe700 1 -- 172.21.15.107:0/951981737 <== mgr.14206 v2:172.21.15.107:6800/2516490111 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f716c1012e0 con 0x7f71540692a0 2024-04-22T23:33:15.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.844+0000 7f71713b0700 1 -- 172.21.15.107:0/951981737 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f71540692a0 msgr2=0x7f715406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:15.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.844+0000 7f71713b0700 1 --2- 172.21.15.107:0/951981737 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f71540692a0 0x7f715406b750 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f715c009470 tx=0x7f715c004d10 comp rx=0 tx=0).stop 2024-04-22T23:33:15.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.844+0000 7f71713b0700 1 -- 172.21.15.107:0/951981737 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f716c0fcff0 msgr2=0x7f716c112e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:15.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.844+0000 7f71713b0700 1 --2- 172.21.15.107:0/951981737 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f716c0fcff0 0x7f716c112e40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f716000b790 tx=0x7f7160009250 comp rx=0 tx=0).stop 2024-04-22T23:33:15.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.845+0000 7f71713b0700 1 -- 172.21.15.107:0/951981737 shutdown_connections 2024-04-22T23:33:15.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.845+0000 7f71713b0700 1 --2- 172.21.15.107:0/951981737 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f71540692a0 0x7f715406b750 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:15.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.845+0000 7f71713b0700 1 --2- 172.21.15.107:0/951981737 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f716c0fbd40 0x7f716c1108f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:15.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.845+0000 7f71713b0700 1 --2- 172.21.15.107:0/951981737 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f716c0fcff0 0x7f716c112e40 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:15.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.845+0000 7f71713b0700 1 -- 172.21.15.107:0/951981737 >> 172.21.15.107:0/951981737 conn(0x7f716c0f74c0 msgr2=0x7f716c0f8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:33:15.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.845+0000 7f71713b0700 1 -- 172.21.15.107:0/951981737 shutdown_connections 2024-04-22T23:33:15.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.845+0000 7f71713b0700 1 -- 172.21.15.107:0/951981737 wait complete. 2024-04-22T23:33:15.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:33:15 smithi157 ceph-mon[33877]: pgmap v204: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:33:15.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.944+0000 7fa39cfd7700 1 -- 172.21.15.107:0/2199326250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3980fdff0 msgr2=0x7fa3980fe460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:15.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.944+0000 7fa39cfd7700 1 --2- 172.21.15.107:0/2199326250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3980fdff0 0x7fa3980fe460 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa38c004940 tx=0x7fa38c009e30 comp rx=0 tx=0).stop 2024-04-22T23:33:15.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.945+0000 7fa39cfd7700 1 -- 172.21.15.107:0/2199326250 shutdown_connections 2024-04-22T23:33:15.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.945+0000 7fa39cfd7700 1 --2- 172.21.15.107:0/2199326250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3980fdff0 0x7fa3980fe460 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:15.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.945+0000 7fa39cfd7700 1 --2- 172.21.15.107:0/2199326250 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3980fce80 0x7fa3980fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:15.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.945+0000 7fa39cfd7700 1 -- 172.21.15.107:0/2199326250 >> 172.21.15.107:0/2199326250 conn(0x7fa3980f8540 msgr2=0x7fa3980fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:33:15.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.945+0000 7fa39cfd7700 1 -- 172.21.15.107:0/2199326250 shutdown_connections 2024-04-22T23:33:15.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.945+0000 7fa39cfd7700 1 -- 172.21.15.107:0/2199326250 wait complete. 2024-04-22T23:33:15.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.945+0000 7fa39cfd7700 1 Processor -- start 2024-04-22T23:33:15.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.945+0000 7fa39cfd7700 1 -- start start 2024-04-22T23:33:15.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.945+0000 7fa39cfd7700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3980fce80 0x7fa39810d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:15.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.945+0000 7fa39cfd7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3980fdff0 0x7fa39810de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:15.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.945+0000 7fa39cfd7700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39810e450 con 0x7fa3980fdff0 2024-04-22T23:33:15.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.945+0000 7fa39cfd7700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39810e5c0 con 0x7fa3980fce80 2024-04-22T23:33:15.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.945+0000 7fa395d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3980fdff0 0x7fa39810de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:15.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.945+0000 7fa395d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3980fdff0 0x7fa39810de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59550/0 (socket says 172.21.15.107:59550) 2024-04-22T23:33:15.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.945+0000 7fa395d9b700 1 -- 172.21.15.107:0/4061425892 learned_addr learned my addr 172.21.15.107:0/4061425892 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:33:15.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.946+0000 7fa39659c700 1 --2- 172.21.15.107:0/4061425892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3980fce80 0x7fa39810d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:15.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.946+0000 7fa395d9b700 1 -- 172.21.15.107:0/4061425892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3980fce80 msgr2=0x7fa39810d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:15.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.946+0000 7fa395d9b700 1 --2- 172.21.15.107:0/4061425892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3980fce80 0x7fa39810d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:15.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.946+0000 7fa395d9b700 1 -- 172.21.15.107:0/4061425892 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa38c005040 con 0x7fa3980fdff0 2024-04-22T23:33:15.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.946+0000 7fa395d9b700 1 --2- 172.21.15.107:0/4061425892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3980fdff0 0x7fa39810de50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa38c00b3b0 tx=0x7fa38c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:33:15.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.946+0000 7fa386ffd700 1 -- 172.21.15.107:0/4061425892 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa38c00e040 con 0x7fa3980fdff0 2024-04-22T23:33:15.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.946+0000 7fa386ffd700 1 -- 172.21.15.107:0/4061425892 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa38c00bbf0 con 0x7fa3980fdff0 2024-04-22T23:33:15.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.946+0000 7fa39cfd7700 1 -- 172.21.15.107:0/4061425892 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa398112fc0 con 0x7fa3980fdff0 2024-04-22T23:33:15.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.946+0000 7fa386ffd700 1 -- 172.21.15.107:0/4061425892 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa38c01c630 con 0x7fa3980fdff0 2024-04-22T23:33:15.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.946+0000 7fa39cfd7700 1 -- 172.21.15.107:0/4061425892 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa398113550 con 0x7fa3980fdff0 2024-04-22T23:33:15.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.947+0000 7fa39cfd7700 1 -- 172.21.15.107:0/4061425892 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa39804c710 con 0x7fa3980fdff0 2024-04-22T23:33:15.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.948+0000 7fa386ffd700 1 -- 172.21.15.107:0/4061425892 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fa38c00bd60 con 0x7fa3980fdff0 2024-04-22T23:33:15.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.948+0000 7fa386ffd700 1 --2- 172.21.15.107:0/4061425892 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fa3800692a0 0x7fa38006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:15.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.948+0000 7fa386ffd700 1 -- 172.21.15.107:0/4061425892 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fa38c087ee0 con 0x7fa3980fdff0 2024-04-22T23:33:15.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.948+0000 7fa39659c700 1 --2- 172.21.15.107:0/4061425892 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fa3800692a0 0x7fa38006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:15.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.949+0000 7fa39659c700 1 --2- 172.21.15.107:0/4061425892 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fa3800692a0 0x7fa38006b750 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa388004f40 tx=0x7fa38800b040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:33:15.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:15.952+0000 7fa386ffd700 1 -- 172.21.15.107:0/4061425892 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fa38c057c50 con 0x7fa3980fdff0 2024-04-22T23:33:16.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.146+0000 7fa39cfd7700 1 -- 172.21.15.107:0/4061425892 --> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa3981023a0 con 0x7fa3800692a0 2024-04-22T23:33:16.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.157+0000 7fa386ffd700 1 -- 172.21.15.107:0/4061425892 <== mgr.14206 v2:172.21.15.107:6800/2516490111 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3366 (secure 0 0 0) 0x7fa3981023a0 con 0x7fa3800692a0 2024-04-22T23:33:16.154 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-22T23:33:16.154 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (4m) 25s ago 6m 25.5M - 0.25.0 c8568f914cd2 4d90ceb89249 2024-04-22T23:33:16.154 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (6m) 25s ago 6m 6660k - 17.2.7-884-g70d0cb4b cafd8ddac8fa c10611c756e4 2024-04-22T23:33:16.154 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi157 smithi157 running (5m) 30s ago 5m 6669k - 17.2.7-884-g70d0cb4b cafd8ddac8fa 55d2d46294c9 2024-04-22T23:33:16.154 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (4m) 25s ago 6m 86.8M - 9.4.7 954c08fa6188 c2d1f8de1aa7 2024-04-22T23:33:16.155 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.dfvvkw smithi107 running (52s) 25s ago 51s 10.4M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 08d3aa8d717d 2024-04-22T23:33:16.155 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.ktuxjl smithi107 running (45s) 25s ago 45s 16.9M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 762f27fe0f30 2024-04-22T23:33:16.155 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi157.rcaris smithi157 running (49s) 30s ago 49s 16.8M - 17.2.7-884-g70d0cb4b cafd8ddac8fa d23366ac085d 2024-04-22T23:33:16.155 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi157.ufywse smithi157 running (42s) 30s ago 42s 11.0M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 8b00d61eb355 2024-04-22T23:33:16.155 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.uipxvm smithi107 *:9283 running (7m) 25s ago 7m 481M - 17.2.7-884-g70d0cb4b cafd8ddac8fa db96e8cbbdfb 2024-04-22T23:33:16.155 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi157.tuylis smithi157 *:8443,9283 running (5m) 30s ago 5m 431M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 0c12be10ffaf 2024-04-22T23:33:16.155 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (7m) 25s ago 7m 49.6M 2048M 17.2.7-884-g70d0cb4b cafd8ddac8fa 8314962fa1b9 2024-04-22T23:33:16.155 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi157 smithi157 running (5m) 30s ago 5m 36.6M 2048M 17.2.7-884-g70d0cb4b cafd8ddac8fa 0747a85008d9 2024-04-22T23:33:16.155 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (6m) 25s ago 6m 16.0M - 1.5.0 0da6a335fe13 8a7c595c114f 2024-04-22T23:33:16.155 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi157 smithi157 *:9100 running (5m) 30s ago 5m 16.1M - 1.5.0 0da6a335fe13 940164771fe3 2024-04-22T23:33:16.155 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (4m) 25s ago 4m 46.6M 4288M 17.2.7-884-g70d0cb4b cafd8ddac8fa 18a179b0a618 2024-04-22T23:33:16.155 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (4m) 25s ago 4m 45.9M 4288M 17.2.7-884-g70d0cb4b cafd8ddac8fa c7272eb84ddb 2024-04-22T23:33:16.155 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (3m) 25s ago 3m 46.2M 4288M 17.2.7-884-g70d0cb4b cafd8ddac8fa 9db1788096ad 2024-04-22T23:33:16.155 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi157 running (3m) 30s ago 3m 43.9M 5312M 17.2.7-884-g70d0cb4b cafd8ddac8fa c9fb2d1a13dd 2024-04-22T23:33:16.155 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi157 running (2m) 30s ago 2m 44.2M 5312M 17.2.7-884-g70d0cb4b cafd8ddac8fa 24a16e19951f 2024-04-22T23:33:16.156 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi157 running (2m) 30s ago 2m 43.1M 5312M 17.2.7-884-g70d0cb4b cafd8ddac8fa 4c5b7a7ef8bd 2024-04-22T23:33:16.156 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (4m) 25s ago 5m 51.2M - 2.43.0 a07b618ecd1d 8c5df40caabe 2024-04-22T23:33:16.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.159+0000 7fa39cfd7700 1 -- 172.21.15.107:0/4061425892 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fa3800692a0 msgr2=0x7fa38006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:16.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.159+0000 7fa39cfd7700 1 --2- 172.21.15.107:0/4061425892 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fa3800692a0 0x7fa38006b750 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa388004f40 tx=0x7fa38800b040 comp rx=0 tx=0).stop 2024-04-22T23:33:16.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.159+0000 7fa39cfd7700 1 -- 172.21.15.107:0/4061425892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3980fdff0 msgr2=0x7fa39810de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:16.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.159+0000 7fa39cfd7700 1 --2- 172.21.15.107:0/4061425892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3980fdff0 0x7fa39810de50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa38c00b3b0 tx=0x7fa38c009e30 comp rx=0 tx=0).stop 2024-04-22T23:33:16.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.159+0000 7fa39cfd7700 1 -- 172.21.15.107:0/4061425892 shutdown_connections 2024-04-22T23:33:16.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.159+0000 7fa39cfd7700 1 --2- 172.21.15.107:0/4061425892 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fa3800692a0 0x7fa38006b750 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:16.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.159+0000 7fa39cfd7700 1 --2- 172.21.15.107:0/4061425892 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3980fce80 0x7fa39810d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:16.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.159+0000 7fa39cfd7700 1 --2- 172.21.15.107:0/4061425892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3980fdff0 0x7fa39810de50 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:16.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.159+0000 7fa39cfd7700 1 -- 172.21.15.107:0/4061425892 >> 172.21.15.107:0/4061425892 conn(0x7fa3980f8540 msgr2=0x7fa3980f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:33:16.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.160+0000 7fa39cfd7700 1 -- 172.21.15.107:0/4061425892 shutdown_connections 2024-04-22T23:33:16.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.160+0000 7fa39cfd7700 1 -- 172.21.15.107:0/4061425892 wait complete. 2024-04-22T23:33:16.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.260+0000 7fe2e07b1700 1 -- 172.21.15.107:0/2305260980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2d80fcff0 msgr2=0x7fe2d80fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:16.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.260+0000 7fe2e07b1700 1 --2- 172.21.15.107:0/2305260980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2d80fcff0 0x7fe2d80fd460 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8004940 tx=0x7fe2c8009e30 comp rx=0 tx=0).stop 2024-04-22T23:33:16.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.260+0000 7fe2e07b1700 1 -- 172.21.15.107:0/2305260980 shutdown_connections 2024-04-22T23:33:16.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.260+0000 7fe2e07b1700 1 --2- 172.21.15.107:0/2305260980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2d80fcff0 0x7fe2d80fd460 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:16.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.260+0000 7fe2e07b1700 1 --2- 172.21.15.107:0/2305260980 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2d80fbd40 0x7fe2d80fc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:16.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.260+0000 7fe2e07b1700 1 -- 172.21.15.107:0/2305260980 >> 172.21.15.107:0/2305260980 conn(0x7fe2d80f74c0 msgr2=0x7fe2d80f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:33:16.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.260+0000 7fe2e07b1700 1 -- 172.21.15.107:0/2305260980 shutdown_connections 2024-04-22T23:33:16.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.260+0000 7fe2e07b1700 1 -- 172.21.15.107:0/2305260980 wait complete. 2024-04-22T23:33:16.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.260+0000 7fe2e07b1700 1 Processor -- start 2024-04-22T23:33:16.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.260+0000 7fe2e07b1700 1 -- start start 2024-04-22T23:33:16.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.260+0000 7fe2e07b1700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2d80fbd40 0x7fe2d80ffe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:16.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.260+0000 7fe2e07b1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2d80fcff0 0x7fe2d80fe330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:16.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.260+0000 7fe2e07b1700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d80fe8c0 con 0x7fe2d80fcff0 2024-04-22T23:33:16.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.260+0000 7fe2e07b1700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d80fea00 con 0x7fe2d80fbd40 2024-04-22T23:33:16.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.260+0000 7fe2ddd4c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2d80fcff0 0x7fe2d80fe330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:16.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.261+0000 7fe2ddd4c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2d80fcff0 0x7fe2d80fe330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59568/0 (socket says 172.21.15.107:59568) 2024-04-22T23:33:16.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.261+0000 7fe2ddd4c700 1 -- 172.21.15.107:0/2153724485 learned_addr learned my addr 172.21.15.107:0/2153724485 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:33:16.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.261+0000 7fe2de54d700 1 --2- 172.21.15.107:0/2153724485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2d80fbd40 0x7fe2d80ffe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:16.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.261+0000 7fe2ddd4c700 1 -- 172.21.15.107:0/2153724485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2d80fbd40 msgr2=0x7fe2d80ffe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:16.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.261+0000 7fe2ddd4c700 1 --2- 172.21.15.107:0/2153724485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2d80fbd40 0x7fe2d80ffe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:16.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.261+0000 7fe2ddd4c700 1 -- 172.21.15.107:0/2153724485 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c8005040 con 0x7fe2d80fcff0 2024-04-22T23:33:16.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.261+0000 7fe2ddd4c700 1 --2- 172.21.15.107:0/2153724485 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2d80fcff0 0x7fe2d80fe330 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe2c800b3b0 tx=0x7fe2c8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:33:16.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.261+0000 7fe2ceffd700 1 -- 172.21.15.107:0/2153724485 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2c800e040 con 0x7fe2d80fcff0 2024-04-22T23:33:16.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.261+0000 7fe2ceffd700 1 -- 172.21.15.107:0/2153724485 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe2c800bbf0 con 0x7fe2d80fcff0 2024-04-22T23:33:16.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.261+0000 7fe2e07b1700 1 -- 172.21.15.107:0/2153724485 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2d80fec80 con 0x7fe2d80fcff0 2024-04-22T23:33:16.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.261+0000 7fe2ceffd700 1 -- 172.21.15.107:0/2153724485 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2c801c630 con 0x7fe2d80fcff0 2024-04-22T23:33:16.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.261+0000 7fe2e07b1700 1 -- 172.21.15.107:0/2153724485 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2d8115ad0 con 0x7fe2d80fcff0 2024-04-22T23:33:16.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.262+0000 7fe2e07b1700 1 -- 172.21.15.107:0/2153724485 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2d8101b80 con 0x7fe2d80fcff0 2024-04-22T23:33:16.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.262+0000 7fe2ceffd700 1 -- 172.21.15.107:0/2153724485 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fe2c800bd60 con 0x7fe2d80fcff0 2024-04-22T23:33:16.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.263+0000 7fe2ceffd700 1 --2- 172.21.15.107:0/2153724485 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fe2c40692a0 0x7fe2c406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:16.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.263+0000 7fe2ceffd700 1 -- 172.21.15.107:0/2153724485 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fe2c8087e70 con 0x7fe2d80fcff0 2024-04-22T23:33:16.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.263+0000 7fe2de54d700 1 --2- 172.21.15.107:0/2153724485 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fe2c40692a0 0x7fe2c406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:16.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.263+0000 7fe2de54d700 1 --2- 172.21.15.107:0/2153724485 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fe2c40692a0 0x7fe2c406b750 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe2d80ffba0 tx=0x7fe2d400b040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:33:16.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.267+0000 7fe2ceffd700 1 -- 172.21.15.107:0/2153724485 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe2c8054170 con 0x7fe2d80fcff0 2024-04-22T23:33:16.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.534+0000 7fe2e07b1700 1 -- 172.21.15.107:0/2153724485 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe2d8104940 con 0x7fe2d80fcff0 2024-04-22T23:33:16.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.536+0000 7fe2ceffd700 1 -- 172.21.15.107:0/2153724485 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7fe2c8057790 con 0x7fe2d80fcff0 2024-04-22T23:33:16.533 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-22T23:33:16.533 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-04-22T23:33:16.533 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 2 2024-04-22T23:33:16.533 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:33:16.533 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-04-22T23:33:16.533 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 2 2024-04-22T23:33:16.533 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:33:16.533 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-04-22T23:33:16.533 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 6 2024-04-22T23:33:16.533 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:33:16.533 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-04-22T23:33:16.533 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 4 2024-04-22T23:33:16.533 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:33:16.534 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-04-22T23:33:16.534 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 14 2024-04-22T23:33:16.534 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T23:33:16.534 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-22T23:33:16.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.538+0000 7fe2e07b1700 1 -- 172.21.15.107:0/2153724485 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fe2c40692a0 msgr2=0x7fe2c406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:16.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.538+0000 7fe2e07b1700 1 --2- 172.21.15.107:0/2153724485 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fe2c40692a0 0x7fe2c406b750 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe2d80ffba0 tx=0x7fe2d400b040 comp rx=0 tx=0).stop 2024-04-22T23:33:16.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.538+0000 7fe2e07b1700 1 -- 172.21.15.107:0/2153724485 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2d80fcff0 msgr2=0x7fe2d80fe330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:16.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.538+0000 7fe2e07b1700 1 --2- 172.21.15.107:0/2153724485 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2d80fcff0 0x7fe2d80fe330 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe2c800b3b0 tx=0x7fe2c8009e30 comp rx=0 tx=0).stop 2024-04-22T23:33:16.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.538+0000 7fe2e07b1700 1 -- 172.21.15.107:0/2153724485 shutdown_connections 2024-04-22T23:33:16.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.538+0000 7fe2e07b1700 1 --2- 172.21.15.107:0/2153724485 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fe2c40692a0 0x7fe2c406b750 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:16.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.538+0000 7fe2e07b1700 1 --2- 172.21.15.107:0/2153724485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2d80fbd40 0x7fe2d80ffe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:16.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.538+0000 7fe2e07b1700 1 --2- 172.21.15.107:0/2153724485 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2d80fcff0 0x7fe2d80fe330 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:16.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.538+0000 7fe2e07b1700 1 -- 172.21.15.107:0/2153724485 >> 172.21.15.107:0/2153724485 conn(0x7fe2d80f74c0 msgr2=0x7fe2d8065c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:33:16.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.538+0000 7fe2e07b1700 1 -- 172.21.15.107:0/2153724485 shutdown_connections 2024-04-22T23:33:16.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.538+0000 7fe2e07b1700 1 -- 172.21.15.107:0/2153724485 wait complete. 2024-04-22T23:33:16.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.638+0000 7f582eee1700 1 -- 172.21.15.107:0/3523075010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58280fdff0 msgr2=0x7f58280fe460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:16.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.638+0000 7f582eee1700 1 --2- 172.21.15.107:0/3523075010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58280fdff0 0x7f58280fe460 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5814004940 tx=0x7f5814009e30 comp rx=0 tx=0).stop 2024-04-22T23:33:16.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.638+0000 7f582eee1700 1 -- 172.21.15.107:0/3523075010 shutdown_connections 2024-04-22T23:33:16.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.638+0000 7f582eee1700 1 --2- 172.21.15.107:0/3523075010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58280fdff0 0x7f58280fe460 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:16.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.638+0000 7f582eee1700 1 --2- 172.21.15.107:0/3523075010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58280fce80 0x7f58280fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:16.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.638+0000 7f582eee1700 1 -- 172.21.15.107:0/3523075010 >> 172.21.15.107:0/3523075010 conn(0x7f58280f8540 msgr2=0x7f58280fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:33:16.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.638+0000 7f582eee1700 1 -- 172.21.15.107:0/3523075010 shutdown_connections 2024-04-22T23:33:16.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.638+0000 7f582eee1700 1 -- 172.21.15.107:0/3523075010 wait complete. 2024-04-22T23:33:16.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.639+0000 7f582eee1700 1 Processor -- start 2024-04-22T23:33:16.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.639+0000 7f582eee1700 1 -- start start 2024-04-22T23:33:16.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.639+0000 7f582eee1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58280fce80 0x7f5828071c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:16.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.639+0000 7f582eee1700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58280fdff0 0x7f5828070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:16.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.639+0000 7f582eee1700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5828072260 con 0x7f58280fce80 2024-04-22T23:33:16.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.639+0000 7f582eee1700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5828070850 con 0x7f58280fdff0 2024-04-22T23:33:16.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.639+0000 7f582cc7d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58280fce80 0x7f5828071c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:16.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.639+0000 7f582cc7d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58280fce80 0x7f5828071c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59598/0 (socket says 172.21.15.107:59598) 2024-04-22T23:33:16.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.639+0000 7f582cc7d700 1 -- 172.21.15.107:0/4254285459 learned_addr learned my addr 172.21.15.107:0/4254285459 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:33:16.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.640+0000 7f5827fff700 1 --2- 172.21.15.107:0/4254285459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58280fdff0 0x7f5828070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:16.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.640+0000 7f582cc7d700 1 -- 172.21.15.107:0/4254285459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58280fdff0 msgr2=0x7f5828070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:16.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.640+0000 7f582cc7d700 1 --2- 172.21.15.107:0/4254285459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58280fdff0 0x7f5828070310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:16.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.640+0000 7f582cc7d700 1 -- 172.21.15.107:0/4254285459 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5814005040 con 0x7f58280fce80 2024-04-22T23:33:16.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.640+0000 7f582cc7d700 1 --2- 172.21.15.107:0/4254285459 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58280fce80 0x7f5828071c60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f581c00c8c0 tx=0x7f581c0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:33:16.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.640+0000 7f58257fa700 1 -- 172.21.15.107:0/4254285459 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f581c00a980 con 0x7f58280fce80 2024-04-22T23:33:16.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.640+0000 7f58257fa700 1 -- 172.21.15.107:0/4254285459 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f581c00ce60 con 0x7f58280fce80 2024-04-22T23:33:16.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.640+0000 7f582eee1700 1 -- 172.21.15.107:0/4254285459 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5828070b00 con 0x7f58280fce80 2024-04-22T23:33:16.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.640+0000 7f58257fa700 1 -- 172.21.15.107:0/4254285459 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f581c0115e0 con 0x7f58280fce80 2024-04-22T23:33:16.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.640+0000 7f582eee1700 1 -- 172.21.15.107:0/4254285459 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58280710c0 con 0x7f58280fce80 2024-04-22T23:33:16.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.641+0000 7f582eee1700 1 -- 172.21.15.107:0/4254285459 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f582804c710 con 0x7f58280fce80 2024-04-22T23:33:16.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.642+0000 7f58257fa700 1 -- 172.21.15.107:0/4254285459 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f581c012400 con 0x7f58280fce80 2024-04-22T23:33:16.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.642+0000 7f58257fa700 1 --2- 172.21.15.107:0/4254285459 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f58180692a0 0x7f581806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:16.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.642+0000 7f58257fa700 1 -- 172.21.15.107:0/4254285459 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f581c0872f0 con 0x7f58280fce80 2024-04-22T23:33:16.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.642+0000 7f5827fff700 1 --2- 172.21.15.107:0/4254285459 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f58180692a0 0x7f581806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:16.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.642+0000 7f5827fff700 1 --2- 172.21.15.107:0/4254285459 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f58180692a0 0x7f581806b750 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f581400b3b0 tx=0x7f5814009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:33:16.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.646+0000 7f58257fa700 1 -- 172.21.15.107:0/4254285459 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f581c057060 con 0x7f58280fce80 2024-04-22T23:33:16.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.876+0000 7f582eee1700 1 -- 172.21.15.107:0/4254285459 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5828115cc0 con 0x7f58280fce80 2024-04-22T23:33:16.874 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:33:16 smithi107 ceph-mon[26355]: from='client.14552 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:33:16.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.877+0000 7f58257fa700 1 -- 172.21.15.107:0/4254285459 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1824 (secure 0 0 0) 0x7f581c0126b0 con 0x7f58280fce80 2024-04-22T23:33:16.874 INFO:teuthology.orchestra.run.smithi107.stdout:e17 2024-04-22T23:33:16.874 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T23:33:16.874 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:33:16.874 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-04-22T23:33:16.874 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:33:16.874 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-04-22T23:33:16.875 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-04-22T23:33:16.875 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 17 2024-04-22T23:33:16.875 INFO:teuthology.orchestra.run.smithi107.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-22T23:33:16.875 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-04-22T23:32:20.146130+0000 2024-04-22T23:33:16.875 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-04-22T23:32:39.774747+0000 2024-04-22T23:33:16.875 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-04-22T23:33:16.875 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-04-22T23:33:16.875 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-04-22T23:33:16.875 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-04-22T23:33:16.875 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-04-22T23:33:16.875 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-04-22T23:33:16.875 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-04-22T23:33:16.875 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 46 2024-04-22T23:33:16.875 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:33:16.875 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 2 2024-04-22T23:33:16.875 INFO:teuthology.orchestra.run.smithi107.stdout:in 0,1 2024-04-22T23:33:16.876 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=14468,1=24265} 2024-04-22T23:33:16.876 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-04-22T23:33:16.876 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-04-22T23:33:16.876 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-04-22T23:33:16.876 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-04-22T23:33:16.876 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-04-22T23:33:16.876 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data disabled 2024-04-22T23:33:16.876 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-04-22T23:33:16.876 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-04-22T23:33:16.876 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.ktuxjl{0:14468} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.107:6828/3335313253,v1:172.21.15.107:6829/3335313253] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:33:16.876 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi157.rcaris{1:24265} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.157:6824/2779251893,v1:172.21.15.157:6825/2779251893] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:33:16.876 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:33:16.876 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:33:16.876 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-04-22T23:33:16.876 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:33:16.877 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.dfvvkw{-1:14484} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/3548717247,v1:172.21.15.107:6827/3548717247] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:33:16.877 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi157.ufywse{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.157:6826/4072444199,v1:172.21.15.157:6827/4072444199] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:33:16.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.879+0000 7f582eee1700 1 -- 172.21.15.107:0/4254285459 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f58180692a0 msgr2=0x7f581806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:16.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.879+0000 7f582eee1700 1 --2- 172.21.15.107:0/4254285459 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f58180692a0 0x7f581806b750 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f581400b3b0 tx=0x7f5814009e30 comp rx=0 tx=0).stop 2024-04-22T23:33:16.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.879+0000 7f582eee1700 1 -- 172.21.15.107:0/4254285459 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58280fce80 msgr2=0x7f5828071c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:16.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.879+0000 7f582eee1700 1 --2- 172.21.15.107:0/4254285459 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58280fce80 0x7f5828071c60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f581c00c8c0 tx=0x7f581c0044c0 comp rx=0 tx=0).stop 2024-04-22T23:33:16.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.879+0000 7f582eee1700 1 -- 172.21.15.107:0/4254285459 shutdown_connections 2024-04-22T23:33:16.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.879+0000 7f582eee1700 1 --2- 172.21.15.107:0/4254285459 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f58180692a0 0x7f581806b750 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:16.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.879+0000 7f582eee1700 1 --2- 172.21.15.107:0/4254285459 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58280fce80 0x7f5828071c60 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:16.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.879+0000 7f582eee1700 1 --2- 172.21.15.107:0/4254285459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58280fdff0 0x7f5828070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:16.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.879+0000 7f582eee1700 1 -- 172.21.15.107:0/4254285459 >> 172.21.15.107:0/4254285459 conn(0x7f58280f8540 msgr2=0x7f58280f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:33:16.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.879+0000 7f582eee1700 1 -- 172.21.15.107:0/4254285459 shutdown_connections 2024-04-22T23:33:16.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.879+0000 7f582eee1700 1 -- 172.21.15.107:0/4254285459 wait complete. 2024-04-22T23:33:16.878 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 17 2024-04-22T23:33:16.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:33:16 smithi157 ceph-mon[33877]: from='client.14552 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:33:16.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.981+0000 7f74761ae700 1 -- 172.21.15.107:0/64097316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74700fcff0 msgr2=0x7f74700fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:16.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.981+0000 7f74761ae700 1 --2- 172.21.15.107:0/64097316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74700fcff0 0x7f74700fd460 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7458004940 tx=0x7f7458009e30 comp rx=0 tx=0).stop 2024-04-22T23:33:16.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.981+0000 7f74761ae700 1 -- 172.21.15.107:0/64097316 shutdown_connections 2024-04-22T23:33:16.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.981+0000 7f74761ae700 1 --2- 172.21.15.107:0/64097316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74700fcff0 0x7f74700fd460 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:16.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.981+0000 7f74761ae700 1 --2- 172.21.15.107:0/64097316 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74700fbd40 0x7f74700fc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:16.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.981+0000 7f74761ae700 1 -- 172.21.15.107:0/64097316 >> 172.21.15.107:0/64097316 conn(0x7f74700f74c0 msgr2=0x7f74700f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:33:16.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.981+0000 7f74761ae700 1 -- 172.21.15.107:0/64097316 shutdown_connections 2024-04-22T23:33:16.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.981+0000 7f74761ae700 1 -- 172.21.15.107:0/64097316 wait complete. 2024-04-22T23:33:16.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.981+0000 7f74761ae700 1 Processor -- start 2024-04-22T23:33:16.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.981+0000 7f74761ae700 1 -- start start 2024-04-22T23:33:16.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.982+0000 7f74761ae700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74700fbd40 0x7f7470112b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:16.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.982+0000 7f74761ae700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74700fcff0 0x7f7470115090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:16.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.982+0000 7f74761ae700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7470066250 con 0x7f74700fcff0 2024-04-22T23:33:16.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.982+0000 7f74761ae700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7470066390 con 0x7f74700fbd40 2024-04-22T23:33:16.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.982+0000 7f7466ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74700fcff0 0x7f7470115090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:16.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.982+0000 7f7466ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74700fcff0 0x7f7470115090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59628/0 (socket says 172.21.15.107:59628) 2024-04-22T23:33:16.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.982+0000 7f7466ffd700 1 -- 172.21.15.107:0/3284667182 learned_addr learned my addr 172.21.15.107:0/3284667182 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:33:16.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.982+0000 7f746f7fe700 1 --2- 172.21.15.107:0/3284667182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74700fbd40 0x7f7470112b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:16.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.983+0000 7f7466ffd700 1 -- 172.21.15.107:0/3284667182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74700fbd40 msgr2=0x7f7470112b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:16.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.983+0000 7f7466ffd700 1 --2- 172.21.15.107:0/3284667182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74700fbd40 0x7f7470112b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:16.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.983+0000 7f7466ffd700 1 -- 172.21.15.107:0/3284667182 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7458005040 con 0x7f74700fcff0 2024-04-22T23:33:16.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.983+0000 7f7466ffd700 1 --2- 172.21.15.107:0/3284667182 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74700fcff0 0x7f7470115090 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f745800b3b0 tx=0x7f7458009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:33:16.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.983+0000 7f746cff9700 1 -- 172.21.15.107:0/3284667182 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f745800e040 con 0x7f74700fcff0 2024-04-22T23:33:16.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.983+0000 7f746cff9700 1 -- 172.21.15.107:0/3284667182 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7458004290 con 0x7f74700fcff0 2024-04-22T23:33:16.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.983+0000 7f74761ae700 1 -- 172.21.15.107:0/3284667182 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74701155d0 con 0x7f74700fcff0 2024-04-22T23:33:16.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.983+0000 7f74761ae700 1 -- 172.21.15.107:0/3284667182 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74701159a0 con 0x7f74700fcff0 2024-04-22T23:33:16.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.983+0000 7f746cff9700 1 -- 172.21.15.107:0/3284667182 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f745801c3f0 con 0x7f74700fcff0 2024-04-22T23:33:16.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.984+0000 7f74761ae700 1 -- 172.21.15.107:0/3284667182 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f747005f5f0 con 0x7f74700fcff0 2024-04-22T23:33:16.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.985+0000 7f746cff9700 1 -- 172.21.15.107:0/3284667182 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f745801c6b0 con 0x7f74700fcff0 2024-04-22T23:33:16.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.985+0000 7f746cff9700 1 --2- 172.21.15.107:0/3284667182 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f745c0692b0 0x7f745c06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:16.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.985+0000 7f746cff9700 1 -- 172.21.15.107:0/3284667182 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f74580884d0 con 0x7f74700fcff0 2024-04-22T23:33:16.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.985+0000 7f746f7fe700 1 --2- 172.21.15.107:0/3284667182 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f745c0692b0 0x7f745c06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:16.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.986+0000 7f746f7fe700 1 --2- 172.21.15.107:0/3284667182 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f745c0692b0 0x7f745c06b760 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7470065fe0 tx=0x7f7460009040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:33:16.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:16.992+0000 7f746cff9700 1 -- 172.21.15.107:0/3284667182 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f7458054890 con 0x7f74700fcff0 2024-04-22T23:33:17.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.216+0000 7f74761ae700 1 -- 172.21.15.107:0/3284667182 --> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7470101350 con 0x7f745c0692b0 2024-04-22T23:33:17.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.217+0000 7f746cff9700 1 -- 172.21.15.107:0/3284667182 <== mgr.14206 v2:172.21.15.107:6800/2516490111 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f7470101350 con 0x7f745c0692b0 2024-04-22T23:33:17.214 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-22T23:33:17.214 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:43be020184947e53516056c9931e1ac5bdbbb1a5", 2024-04-22T23:33:17.214 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-04-22T23:33:17.214 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-22T23:33:17.215 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [], 2024-04-22T23:33:17.215 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "", 2024-04-22T23:33:17.215 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Doing first pull of quay.ceph.io/ceph-ci/ceph:43be020184947e53516056c9931e1ac5bdbbb1a5 image", 2024-04-22T23:33:17.215 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-04-22T23:33:17.215 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-22T23:33:17.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.220+0000 7f74761ae700 1 -- 172.21.15.107:0/3284667182 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f745c0692b0 msgr2=0x7f745c06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:17.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.220+0000 7f74761ae700 1 --2- 172.21.15.107:0/3284667182 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f745c0692b0 0x7f745c06b760 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7470065fe0 tx=0x7f7460009040 comp rx=0 tx=0).stop 2024-04-22T23:33:17.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.220+0000 7f74761ae700 1 -- 172.21.15.107:0/3284667182 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74700fcff0 msgr2=0x7f7470115090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:17.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.220+0000 7f74761ae700 1 --2- 172.21.15.107:0/3284667182 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74700fcff0 0x7f7470115090 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f745800b3b0 tx=0x7f7458009e30 comp rx=0 tx=0).stop 2024-04-22T23:33:17.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.220+0000 7f74761ae700 1 -- 172.21.15.107:0/3284667182 shutdown_connections 2024-04-22T23:33:17.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.220+0000 7f74761ae700 1 --2- 172.21.15.107:0/3284667182 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f745c0692b0 0x7f745c06b760 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:17.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.220+0000 7f74761ae700 1 --2- 172.21.15.107:0/3284667182 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74700fbd40 0x7f7470112b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:17.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.220+0000 7f74761ae700 1 --2- 172.21.15.107:0/3284667182 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74700fcff0 0x7f7470115090 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:17.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.220+0000 7f74761ae700 1 -- 172.21.15.107:0/3284667182 >> 172.21.15.107:0/3284667182 conn(0x7f74700f74c0 msgr2=0x7f74700f8bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:33:17.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.220+0000 7f74761ae700 1 -- 172.21.15.107:0/3284667182 shutdown_connections 2024-04-22T23:33:17.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.220+0000 7f74761ae700 1 -- 172.21.15.107:0/3284667182 wait complete. 2024-04-22T23:33:17.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.323+0000 7f12b6028700 1 -- 172.21.15.107:0/1537629092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12b00fe410 msgr2=0x7f12b00fe820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:17.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.323+0000 7f12b6028700 1 --2- 172.21.15.107:0/1537629092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12b00fe410 0x7f12b00fe820 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f12a0004940 tx=0x7f12a0009e30 comp rx=0 tx=0).stop 2024-04-22T23:33:17.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.323+0000 7f12b6028700 1 -- 172.21.15.107:0/1537629092 shutdown_connections 2024-04-22T23:33:17.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.323+0000 7f12b6028700 1 --2- 172.21.15.107:0/1537629092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12b00fed60 0x7f12b00faea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:17.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.323+0000 7f12b6028700 1 --2- 172.21.15.107:0/1537629092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12b00fe410 0x7f12b00fe820 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:17.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.323+0000 7f12b6028700 1 -- 172.21.15.107:0/1537629092 >> 172.21.15.107:0/1537629092 conn(0x7f12b00f6bd0 msgr2=0x7f12b00f8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:33:17.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.323+0000 7f12b6028700 1 -- 172.21.15.107:0/1537629092 shutdown_connections 2024-04-22T23:33:17.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.323+0000 7f12b6028700 1 -- 172.21.15.107:0/1537629092 wait complete. 2024-04-22T23:33:17.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.324+0000 7f12b6028700 1 Processor -- start 2024-04-22T23:33:17.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.324+0000 7f12b6028700 1 -- start start 2024-04-22T23:33:17.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.324+0000 7f12b6028700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12b00fe410 0x7f12b010fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:17.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.324+0000 7f12b6028700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12b00fed60 0x7f12b01101a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:17.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.324+0000 7f12b6028700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b01107a0 con 0x7f12b00fed60 2024-04-22T23:33:17.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.324+0000 7f12b6028700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b0110ff0 con 0x7f12b00fe410 2024-04-22T23:33:17.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.324+0000 7f12aeffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12b00fed60 0x7f12b01101a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:17.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.324+0000 7f12aeffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12b00fed60 0x7f12b01101a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59644/0 (socket says 172.21.15.107:59644) 2024-04-22T23:33:17.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.324+0000 7f12aeffd700 1 -- 172.21.15.107:0/3681897333 learned_addr learned my addr 172.21.15.107:0/3681897333 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:33:17.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.324+0000 7f12af7fe700 1 --2- 172.21.15.107:0/3681897333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12b00fe410 0x7f12b010fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:17.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.325+0000 7f12aeffd700 1 -- 172.21.15.107:0/3681897333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12b00fe410 msgr2=0x7f12b010fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:17.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.325+0000 7f12aeffd700 1 --2- 172.21.15.107:0/3681897333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12b00fe410 0x7f12b010fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:17.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.325+0000 7f12aeffd700 1 -- 172.21.15.107:0/3681897333 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a0005040 con 0x7f12b00fed60 2024-04-22T23:33:17.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.325+0000 7f12aeffd700 1 --2- 172.21.15.107:0/3681897333 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12b00fed60 0x7f12b01101a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f12a4002a50 tx=0x7f12a40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:33:17.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.325+0000 7f1297fff700 1 -- 172.21.15.107:0/3681897333 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12a4007cc0 con 0x7f12b00fed60 2024-04-22T23:33:17.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.325+0000 7f12b6028700 1 -- 172.21.15.107:0/3681897333 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12b01112d0 con 0x7f12b00fed60 2024-04-22T23:33:17.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.325+0000 7f1297fff700 1 -- 172.21.15.107:0/3681897333 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f12a400f040 con 0x7f12b00fed60 2024-04-22T23:33:17.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.325+0000 7f1297fff700 1 -- 172.21.15.107:0/3681897333 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12a400da30 con 0x7f12b00fed60 2024-04-22T23:33:17.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.325+0000 7f12b6028700 1 -- 172.21.15.107:0/3681897333 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12b0111810 con 0x7f12b00fed60 2024-04-22T23:33:17.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.326+0000 7f12b6028700 1 -- 172.21.15.107:0/3681897333 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12b0101b50 con 0x7f12b00fed60 2024-04-22T23:33:17.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.326+0000 7f1297fff700 1 -- 172.21.15.107:0/3681897333 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f12a400db90 con 0x7f12b00fed60 2024-04-22T23:33:17.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.327+0000 7f1297fff700 1 --2- 172.21.15.107:0/3681897333 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f12980692a0 0x7f129806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:17.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.327+0000 7f1297fff700 1 -- 172.21.15.107:0/3681897333 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f12a4086fa0 con 0x7f12b00fed60 2024-04-22T23:33:17.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.327+0000 7f12af7fe700 1 --2- 172.21.15.107:0/3681897333 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f12980692a0 0x7f129806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:17.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.327+0000 7f12af7fe700 1 --2- 172.21.15.107:0/3681897333 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f12980692a0 0x7f129806b750 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f12a000b970 tx=0x7f12a0009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:33:17.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.331+0000 7f1297fff700 1 -- 172.21.15.107:0/3681897333 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f12a4056c90 con 0x7f12b00fed60 2024-04-22T23:33:17.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.608+0000 7f12b6028700 1 -- 172.21.15.107:0/3681897333 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f12b005f5f0 con 0x7f12b00fed60 2024-04-22T23:33:17.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.609+0000 7f1297fff700 1 -- 172.21.15.107:0/3681897333 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f12a4056820 con 0x7f12b00fed60 2024-04-22T23:33:17.606 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_OK 2024-04-22T23:33:17.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.611+0000 7f12b6028700 1 -- 172.21.15.107:0/3681897333 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f12980692a0 msgr2=0x7f129806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:17.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.611+0000 7f12b6028700 1 --2- 172.21.15.107:0/3681897333 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f12980692a0 0x7f129806b750 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f12a000b970 tx=0x7f12a0009e30 comp rx=0 tx=0).stop 2024-04-22T23:33:17.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.611+0000 7f12b6028700 1 -- 172.21.15.107:0/3681897333 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12b00fed60 msgr2=0x7f12b01101a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:17.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.611+0000 7f12b6028700 1 --2- 172.21.15.107:0/3681897333 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12b00fed60 0x7f12b01101a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f12a4002a50 tx=0x7f12a40046b0 comp rx=0 tx=0).stop 2024-04-22T23:33:17.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.611+0000 7f12b6028700 1 -- 172.21.15.107:0/3681897333 shutdown_connections 2024-04-22T23:33:17.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.611+0000 7f12b6028700 1 --2- 172.21.15.107:0/3681897333 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f12980692a0 0x7f129806b750 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:17.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.611+0000 7f12b6028700 1 --2- 172.21.15.107:0/3681897333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12b00fe410 0x7f12b010fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:17.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.611+0000 7f12b6028700 1 --2- 172.21.15.107:0/3681897333 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12b00fed60 0x7f12b01101a0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:17.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.611+0000 7f12b6028700 1 -- 172.21.15.107:0/3681897333 >> 172.21.15.107:0/3681897333 conn(0x7f12b00f6bd0 msgr2=0x7f12b00f8280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:33:17.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.611+0000 7f12b6028700 1 -- 172.21.15.107:0/3681897333 shutdown_connections 2024-04-22T23:33:17.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:17.611+0000 7f12b6028700 1 -- 172.21.15.107:0/3681897333 wait complete. 2024-04-22T23:33:17.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:33:17 smithi157 ceph-mon[33877]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:33:17.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:33:17 smithi157 ceph-mon[33877]: pgmap v205: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:33:17.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:33:17 smithi157 ceph-mon[33877]: from='client.14560 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:33:17.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:33:17 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/2153724485' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:33:17.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:33:17 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/4254285459' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:33:17.985 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:33:17 smithi107 ceph-mon[26355]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:33:17.985 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:33:17 smithi107 ceph-mon[26355]: pgmap v205: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:33:17.985 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:33:17 smithi107 ceph-mon[26355]: from='client.14560 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:33:17.985 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:33:17 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/2153724485' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:33:17.985 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:33:17 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/4254285459' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:33:18.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:33:18 smithi157 ceph-mon[33877]: from='client.14572 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:33:18.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:33:18 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/3681897333' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:33:18.984 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:33:18 smithi107 ceph-mon[26355]: from='client.14572 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:33:18.985 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:33:18 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/3681897333' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:33:19.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:33:19 smithi157 ceph-mon[33877]: pgmap v206: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:33:19.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:33:19 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:33:19.985 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:33:19 smithi107 ceph-mon[26355]: pgmap v206: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:33:19.985 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:33:19 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:33:21.886 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:33:21 smithi157 ceph-mon[33877]: pgmap v207: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:33:21.984 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:33:21 smithi107 ceph-mon[26355]: pgmap v207: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:33:23.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:33:23 smithi157 ceph-mon[33877]: pgmap v208: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:33:23.984 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:33:23 smithi107 ceph-mon[26355]: pgmap v208: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:33:25.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:33:25 smithi157 ceph-mon[33877]: pgmap v209: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:33:25.984 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:33:25 smithi107 ceph-mon[26355]: pgmap v209: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:33:27.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:33:27 smithi157 ceph-mon[33877]: pgmap v210: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:33:27.984 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:33:27 smithi107 ceph-mon[26355]: pgmap v210: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:33:29.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:33:29 smithi157 ceph-mon[33877]: pgmap v211: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:33:29.984 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:33:29 smithi107 ceph-mon[26355]: pgmap v211: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:33:31.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:33:31 smithi157 ceph-mon[33877]: pgmap v212: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:33:31.984 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:33:31 smithi107 ceph-mon[26355]: pgmap v212: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:33:33.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:33:33 smithi157 ceph-mon[33877]: pgmap v213: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-04-22T23:33:33.984 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:33:33 smithi107 ceph-mon[26355]: pgmap v213: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-04-22T23:33:34.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:33:34 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:33:34.987 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:33:34 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:33:35.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:33:35 smithi157 ceph-mon[33877]: pgmap v214: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-04-22T23:33:35.984 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:33:35 smithi107 ceph-mon[26355]: pgmap v214: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-04-22T23:33:36.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:33:36 smithi157 ceph-mon[33877]: pgmap v215: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-04-22T23:33:36.984 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:33:36 smithi107 ceph-mon[26355]: pgmap v215: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-04-22T23:33:39.383 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:33:39 smithi157 ceph-mon[33877]: pgmap v216: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-04-22T23:33:39.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:33:39 smithi107 ceph-mon[26355]: pgmap v216: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-04-22T23:33:41.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:33:41 smithi157 ceph-mon[33877]: pgmap v217: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-04-22T23:33:41.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:33:41 smithi107 ceph-mon[26355]: pgmap v217: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-04-22T23:33:43.383 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:33:43 smithi157 ceph-mon[33877]: pgmap v218: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-04-22T23:33:43.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:33:43 smithi107 ceph-mon[26355]: pgmap v218: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-04-22T23:33:45.383 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:33:45 smithi157 ceph-mon[33877]: pgmap v219: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-04-22T23:33:45.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:33:45 smithi107 ceph-mon[26355]: pgmap v219: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-04-22T23:33:47.383 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:33:47 smithi157 ceph-mon[33877]: pgmap v220: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-04-22T23:33:47.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:33:47 smithi107 ceph-mon[26355]: pgmap v220: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-04-22T23:33:47.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:47.724+0000 7f9cd67fd700 1 -- 172.21.15.107:0/2242870110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cd00fcff0 msgr2=0x7f9cd00fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:47.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:47.724+0000 7f9cd67fd700 1 --2- 172.21.15.107:0/2242870110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cd00fcff0 0x7f9cd00fd460 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4004940 tx=0x7f9cc4009e30 comp rx=0 tx=0).stop 2024-04-22T23:33:47.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:47.724+0000 7f9cd67fd700 1 -- 172.21.15.107:0/2242870110 shutdown_connections 2024-04-22T23:33:47.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:47.724+0000 7f9cd67fd700 1 --2- 172.21.15.107:0/2242870110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cd00fcff0 0x7f9cd00fd460 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:47.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:47.724+0000 7f9cd67fd700 1 --2- 172.21.15.107:0/2242870110 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cd00fbd40 0x7f9cd00fc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:47.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:47.724+0000 7f9cd67fd700 1 -- 172.21.15.107:0/2242870110 >> 172.21.15.107:0/2242870110 conn(0x7f9cd00f74c0 msgr2=0x7f9cd00f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:33:47.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:47.724+0000 7f9cd67fd700 1 -- 172.21.15.107:0/2242870110 shutdown_connections 2024-04-22T23:33:47.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:47.724+0000 7f9cd67fd700 1 -- 172.21.15.107:0/2242870110 wait complete. 2024-04-22T23:33:47.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:47.725+0000 7f9cd67fd700 1 Processor -- start 2024-04-22T23:33:47.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:47.725+0000 7f9cd67fd700 1 -- start start 2024-04-22T23:33:47.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:47.725+0000 7f9cd67fd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cd00fbd40 0x7f9cd010c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:47.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:47.725+0000 7f9cd67fd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cd00fcff0 0x7f9cd010ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:47.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:47.725+0000 7f9cd67fd700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd010d460 con 0x7f9cd00fcff0 2024-04-22T23:33:47.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:47.725+0000 7f9cd67fd700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd010dcb0 con 0x7f9cd00fbd40 2024-04-22T23:33:47.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:47.725+0000 7f9ccf7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cd00fcff0 0x7f9cd010ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:47.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:47.725+0000 7f9ccf7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cd00fcff0 0x7f9cd010ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49052/0 (socket says 172.21.15.107:49052) 2024-04-22T23:33:47.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:47.725+0000 7f9ccf7fe700 1 -- 172.21.15.107:0/56696805 learned_addr learned my addr 172.21.15.107:0/56696805 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:33:47.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:47.725+0000 7f9ccffff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cd00fbd40 0x7f9cd010c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:47.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:47.726+0000 7f9ccf7fe700 1 -- 172.21.15.107:0/56696805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cd00fbd40 msgr2=0x7f9cd010c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:47.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:47.726+0000 7f9ccf7fe700 1 --2- 172.21.15.107:0/56696805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cd00fbd40 0x7f9cd010c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:47.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:47.726+0000 7f9ccf7fe700 1 -- 172.21.15.107:0/56696805 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cc4005040 con 0x7f9cd00fcff0 2024-04-22T23:33:47.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:47.726+0000 7f9ccffff700 1 --2- 172.21.15.107:0/56696805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cd00fbd40 0x7f9cd010c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T23:33:47.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:47.726+0000 7f9ccf7fe700 1 --2- 172.21.15.107:0/56696805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cd00fcff0 0x7f9cd010ce60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4009fd0 tx=0x7f9cc4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:33:47.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:47.726+0000 7f9cccff9700 1 -- 172.21.15.107:0/56696805 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9cc400e040 con 0x7f9cd00fcff0 2024-04-22T23:33:47.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:47.726+0000 7f9cd67fd700 1 -- 172.21.15.107:0/56696805 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cd010df30 con 0x7f9cd00fcff0 2024-04-22T23:33:47.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:47.726+0000 7f9cccff9700 1 -- 172.21.15.107:0/56696805 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9cc400be00 con 0x7f9cd00fcff0 2024-04-22T23:33:47.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:47.726+0000 7f9cccff9700 1 -- 172.21.15.107:0/56696805 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9cc40137f0 con 0x7f9cd00fcff0 2024-04-22T23:33:47.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:47.726+0000 7f9cd67fd700 1 -- 172.21.15.107:0/56696805 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cd010e470 con 0x7f9cd00fcff0 2024-04-22T23:33:47.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:47.727+0000 7f9cd67fd700 1 -- 172.21.15.107:0/56696805 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cd005f5f0 con 0x7f9cd00fcff0 2024-04-22T23:33:47.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:47.728+0000 7f9cccff9700 1 -- 172.21.15.107:0/56696805 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f9cc4013950 con 0x7f9cd00fcff0 2024-04-22T23:33:47.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:47.728+0000 7f9cccff9700 1 --2- 172.21.15.107:0/56696805 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f9cb80692a0 0x7f9cb806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:47.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:47.728+0000 7f9cccff9700 1 -- 172.21.15.107:0/56696805 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f9cc408aa90 con 0x7f9cd00fcff0 2024-04-22T23:33:47.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:47.728+0000 7f9ccffff700 1 --2- 172.21.15.107:0/56696805 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f9cb80692a0 0x7f9cb806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:47.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:47.729+0000 7f9ccffff700 1 --2- 172.21.15.107:0/56696805 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f9cb80692a0 0x7f9cb806b750 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0010520 tx=0x7f9cc000b040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:33:47.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:47.733+0000 7f9cccff9700 1 -- 172.21.15.107:0/56696805 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f9cc405a8b0 con 0x7f9cd00fcff0 2024-04-22T23:33:47.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:47.984+0000 7f9cd67fd700 1 -- 172.21.15.107:0/56696805 --> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9cd010e8d0 con 0x7f9cb80692a0 2024-04-22T23:33:47.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:47.986+0000 7f9cccff9700 1 -- 172.21.15.107:0/56696805 <== mgr.14206 v2:172.21.15.107:6800/2516490111 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f9cd010e8d0 con 0x7f9cb80692a0 2024-04-22T23:33:47.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:47.988+0000 7f9cd67fd700 1 -- 172.21.15.107:0/56696805 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f9cb80692a0 msgr2=0x7f9cb806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:47.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:47.988+0000 7f9cd67fd700 1 --2- 172.21.15.107:0/56696805 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f9cb80692a0 0x7f9cb806b750 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0010520 tx=0x7f9cc000b040 comp rx=0 tx=0).stop 2024-04-22T23:33:47.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:47.988+0000 7f9cd67fd700 1 -- 172.21.15.107:0/56696805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cd00fcff0 msgr2=0x7f9cd010ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:47.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:47.988+0000 7f9cd67fd700 1 --2- 172.21.15.107:0/56696805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cd00fcff0 0x7f9cd010ce60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4009fd0 tx=0x7f9cc4009e30 comp rx=0 tx=0).stop 2024-04-22T23:33:47.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:47.988+0000 7f9cd67fd700 1 -- 172.21.15.107:0/56696805 shutdown_connections 2024-04-22T23:33:47.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:47.988+0000 7f9cd67fd700 1 --2- 172.21.15.107:0/56696805 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f9cb80692a0 0x7f9cb806b750 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:47.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:47.988+0000 7f9cd67fd700 1 --2- 172.21.15.107:0/56696805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cd00fbd40 0x7f9cd010c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:47.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:47.988+0000 7f9cd67fd700 1 --2- 172.21.15.107:0/56696805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cd00fcff0 0x7f9cd010ce60 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:47.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:47.988+0000 7f9cd67fd700 1 -- 172.21.15.107:0/56696805 >> 172.21.15.107:0/56696805 conn(0x7f9cd00f74c0 msgr2=0x7f9cd00f8bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:33:47.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:47.988+0000 7f9cd67fd700 1 -- 172.21.15.107:0/56696805 shutdown_connections 2024-04-22T23:33:47.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:47.988+0000 7f9cd67fd700 1 -- 172.21.15.107:0/56696805 wait complete. 2024-04-22T23:33:47.994 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-04-22T23:33:48.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.099+0000 7fa23193c700 1 -- 172.21.15.107:0/411196224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa22c066230 msgr2=0x7fa22c0ffaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:48.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.099+0000 7fa23193c700 1 --2- 172.21.15.107:0/411196224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa22c066230 0x7fa22c0ffaf0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa21c004940 tx=0x7fa21c009e30 comp rx=0 tx=0).stop 2024-04-22T23:33:48.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.099+0000 7fa23193c700 1 -- 172.21.15.107:0/411196224 shutdown_connections 2024-04-22T23:33:48.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.099+0000 7fa23193c700 1 --2- 172.21.15.107:0/411196224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa22c100030 0x7fa22c102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:48.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.099+0000 7fa23193c700 1 --2- 172.21.15.107:0/411196224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa22c066230 0x7fa22c0ffaf0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:48.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.099+0000 7fa23193c700 1 -- 172.21.15.107:0/411196224 >> 172.21.15.107:0/411196224 conn(0x7fa22c0f74c0 msgr2=0x7fa22c0f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:33:48.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.099+0000 7fa23193c700 1 -- 172.21.15.107:0/411196224 shutdown_connections 2024-04-22T23:33:48.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.099+0000 7fa23193c700 1 -- 172.21.15.107:0/411196224 wait complete. 2024-04-22T23:33:48.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.100+0000 7fa23193c700 1 Processor -- start 2024-04-22T23:33:48.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.100+0000 7fa23193c700 1 -- start start 2024-04-22T23:33:48.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.100+0000 7fa23193c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa22c066230 0x7fa22c10da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:48.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.100+0000 7fa23193c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa22c100030 0x7fa22c10df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:48.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.100+0000 7fa23193c700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22c10e570 con 0x7fa22c100030 2024-04-22T23:33:48.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.100+0000 7fa23193c700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22c10edc0 con 0x7fa22c066230 2024-04-22T23:33:48.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.101+0000 7fa22a7fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa22c100030 0x7fa22c10df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:48.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.101+0000 7fa22a7fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa22c100030 0x7fa22c10df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49076/0 (socket says 172.21.15.107:49076) 2024-04-22T23:33:48.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.101+0000 7fa22a7fc700 1 -- 172.21.15.107:0/3789981871 learned_addr learned my addr 172.21.15.107:0/3789981871 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:33:48.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.101+0000 7fa22affd700 1 --2- 172.21.15.107:0/3789981871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa22c066230 0x7fa22c10da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:48.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.101+0000 7fa22a7fc700 1 -- 172.21.15.107:0/3789981871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa22c066230 msgr2=0x7fa22c10da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:48.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.101+0000 7fa22a7fc700 1 --2- 172.21.15.107:0/3789981871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa22c066230 0x7fa22c10da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:48.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.101+0000 7fa22a7fc700 1 -- 172.21.15.107:0/3789981871 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa21c005040 con 0x7fa22c100030 2024-04-22T23:33:48.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.101+0000 7fa22a7fc700 1 --2- 172.21.15.107:0/3789981871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa22c100030 0x7fa22c10df70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa22000eb40 tx=0x7fa22000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:33:48.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.101+0000 7fa2137fe700 1 -- 172.21.15.107:0/3789981871 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa220007cc0 con 0x7fa22c100030 2024-04-22T23:33:48.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.101+0000 7fa2137fe700 1 -- 172.21.15.107:0/3789981871 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa22000f040 con 0x7fa22c100030 2024-04-22T23:33:48.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.101+0000 7fa23193c700 1 -- 172.21.15.107:0/3789981871 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa22c10f040 con 0x7fa22c100030 2024-04-22T23:33:48.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.101+0000 7fa2137fe700 1 -- 172.21.15.107:0/3789981871 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa22000c9d0 con 0x7fa22c100030 2024-04-22T23:33:48.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.101+0000 7fa23193c700 1 -- 172.21.15.107:0/3789981871 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa22c10f580 con 0x7fa22c100030 2024-04-22T23:33:48.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.102+0000 7fa23193c700 1 -- 172.21.15.107:0/3789981871 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa22c05f5f0 con 0x7fa22c100030 2024-04-22T23:33:48.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.103+0000 7fa2137fe700 1 -- 172.21.15.107:0/3789981871 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fa22000cb30 con 0x7fa22c100030 2024-04-22T23:33:48.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.103+0000 7fa2137fe700 1 --2- 172.21.15.107:0/3789981871 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fa2140692a0 0x7fa21406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:48.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.103+0000 7fa2137fe700 1 -- 172.21.15.107:0/3789981871 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fa220086b10 con 0x7fa22c100030 2024-04-22T23:33:48.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.103+0000 7fa22affd700 1 --2- 172.21.15.107:0/3789981871 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fa2140692a0 0x7fa21406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:48.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.104+0000 7fa22affd700 1 --2- 172.21.15.107:0/3789981871 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fa2140692a0 0x7fa21406b750 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa21c004890 tx=0x7fa21c009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:33:48.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.108+0000 7fa2137fe700 1 -- 172.21.15.107:0/3789981871 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fa22004e250 con 0x7fa22c100030 2024-04-22T23:33:48.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.341+0000 7fa23193c700 1 -- 172.21.15.107:0/3789981871 --> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa22c10f7f0 con 0x7fa2140692a0 2024-04-22T23:33:48.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.343+0000 7fa2137fe700 1 -- 172.21.15.107:0/3789981871 <== mgr.14206 v2:172.21.15.107:6800/2516490111 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7fa22c10f7f0 con 0x7fa2140692a0 2024-04-22T23:33:48.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.345+0000 7fa23193c700 1 -- 172.21.15.107:0/3789981871 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fa2140692a0 msgr2=0x7fa21406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:48.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.345+0000 7fa23193c700 1 --2- 172.21.15.107:0/3789981871 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fa2140692a0 0x7fa21406b750 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa21c004890 tx=0x7fa21c009e30 comp rx=0 tx=0).stop 2024-04-22T23:33:48.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.345+0000 7fa23193c700 1 -- 172.21.15.107:0/3789981871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa22c100030 msgr2=0x7fa22c10df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:48.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.345+0000 7fa23193c700 1 --2- 172.21.15.107:0/3789981871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa22c100030 0x7fa22c10df70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa22000eb40 tx=0x7fa22000b040 comp rx=0 tx=0).stop 2024-04-22T23:33:48.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.345+0000 7fa23193c700 1 -- 172.21.15.107:0/3789981871 shutdown_connections 2024-04-22T23:33:48.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.345+0000 7fa23193c700 1 --2- 172.21.15.107:0/3789981871 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fa2140692a0 0x7fa21406b750 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:48.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.345+0000 7fa23193c700 1 --2- 172.21.15.107:0/3789981871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa22c066230 0x7fa22c10da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:48.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.345+0000 7fa23193c700 1 --2- 172.21.15.107:0/3789981871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa22c100030 0x7fa22c10df70 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:48.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.345+0000 7fa23193c700 1 -- 172.21.15.107:0/3789981871 >> 172.21.15.107:0/3789981871 conn(0x7fa22c0f74c0 msgr2=0x7fa22c0f7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:33:48.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.345+0000 7fa23193c700 1 -- 172.21.15.107:0/3789981871 shutdown_connections 2024-04-22T23:33:48.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.345+0000 7fa23193c700 1 -- 172.21.15.107:0/3789981871 wait complete. 2024-04-22T23:33:48.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.456+0000 7fe98429a700 1 -- 172.21.15.107:0/4145982084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe97c0fce40 msgr2=0x7fe97c0fd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:48.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.456+0000 7fe98429a700 1 --2- 172.21.15.107:0/4145982084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe97c0fce40 0x7fe97c0fd250 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe978004940 tx=0x7fe978009e30 comp rx=0 tx=0).stop 2024-04-22T23:33:48.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.456+0000 7fe98429a700 1 -- 172.21.15.107:0/4145982084 shutdown_connections 2024-04-22T23:33:48.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.456+0000 7fe98429a700 1 --2- 172.21.15.107:0/4145982084 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe97c0fe040 0x7fe97c0fe4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:48.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.456+0000 7fe98429a700 1 --2- 172.21.15.107:0/4145982084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe97c0fce40 0x7fe97c0fd250 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:48.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.456+0000 7fe98429a700 1 -- 172.21.15.107:0/4145982084 >> 172.21.15.107:0/4145982084 conn(0x7fe97c0f8530 msgr2=0x7fe97c0fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:33:48.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.456+0000 7fe98429a700 1 -- 172.21.15.107:0/4145982084 shutdown_connections 2024-04-22T23:33:48.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.456+0000 7fe98429a700 1 -- 172.21.15.107:0/4145982084 wait complete. 2024-04-22T23:33:48.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.457+0000 7fe98429a700 1 Processor -- start 2024-04-22T23:33:48.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.457+0000 7fe98429a700 1 -- start start 2024-04-22T23:33:48.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.457+0000 7fe98429a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe97c0fce40 0x7fe97c10d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:48.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.457+0000 7fe98429a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe97c0fe040 0x7fe97c10df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:48.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.457+0000 7fe98429a700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe97c10e510 con 0x7fe97c0fe040 2024-04-22T23:33:48.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.457+0000 7fe98429a700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe97c10ed60 con 0x7fe97c0fce40 2024-04-22T23:33:48.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.457+0000 7fe981835700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe97c0fe040 0x7fe97c10df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:48.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.457+0000 7fe981835700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe97c0fe040 0x7fe97c10df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49084/0 (socket says 172.21.15.107:49084) 2024-04-22T23:33:48.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.457+0000 7fe981835700 1 -- 172.21.15.107:0/2948997620 learned_addr learned my addr 172.21.15.107:0/2948997620 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:33:48.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.457+0000 7fe982036700 1 --2- 172.21.15.107:0/2948997620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe97c0fce40 0x7fe97c10d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:48.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.457+0000 7fe981835700 1 -- 172.21.15.107:0/2948997620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe97c0fce40 msgr2=0x7fe97c10d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:48.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.457+0000 7fe981835700 1 --2- 172.21.15.107:0/2948997620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe97c0fce40 0x7fe97c10d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:48.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.457+0000 7fe981835700 1 -- 172.21.15.107:0/2948997620 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe978005040 con 0x7fe97c0fe040 2024-04-22T23:33:48.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.458+0000 7fe981835700 1 --2- 172.21.15.107:0/2948997620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe97c0fe040 0x7fe97c10df10 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe96c00eb40 tx=0x7fe96c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:33:48.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.458+0000 7fe972ffd700 1 -- 172.21.15.107:0/2948997620 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe96c007cc0 con 0x7fe97c0fe040 2024-04-22T23:33:48.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.458+0000 7fe972ffd700 1 -- 172.21.15.107:0/2948997620 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe96c00f040 con 0x7fe97c0fe040 2024-04-22T23:33:48.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.458+0000 7fe98429a700 1 -- 172.21.15.107:0/2948997620 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe97c10efe0 con 0x7fe97c0fe040 2024-04-22T23:33:48.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.458+0000 7fe972ffd700 1 -- 172.21.15.107:0/2948997620 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe96c00ca10 con 0x7fe97c0fe040 2024-04-22T23:33:48.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.458+0000 7fe98429a700 1 -- 172.21.15.107:0/2948997620 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe97c10f520 con 0x7fe97c0fe040 2024-04-22T23:33:48.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.459+0000 7fe98429a700 1 -- 172.21.15.107:0/2948997620 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9600052e0 con 0x7fe97c0fe040 2024-04-22T23:33:48.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.460+0000 7fe972ffd700 1 -- 172.21.15.107:0/2948997620 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fe96c00cb70 con 0x7fe97c0fe040 2024-04-22T23:33:48.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.460+0000 7fe972ffd700 1 --2- 172.21.15.107:0/2948997620 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fe9680692b0 0x7fe96806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:48.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.461+0000 7fe972ffd700 1 -- 172.21.15.107:0/2948997620 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fe96c086c70 con 0x7fe97c0fe040 2024-04-22T23:33:48.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.461+0000 7fe982036700 1 --2- 172.21.15.107:0/2948997620 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fe9680692b0 0x7fe96806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:48.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.461+0000 7fe982036700 1 --2- 172.21.15.107:0/2948997620 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fe9680692b0 0x7fe96806b760 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe978013870 tx=0x7fe978009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:33:48.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.466+0000 7fe972ffd700 1 -- 172.21.15.107:0/2948997620 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe96c04e250 con 0x7fe97c0fe040 2024-04-22T23:33:48.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.698+0000 7fe98429a700 1 -- 172.21.15.107:0/2948997620 --> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe960000bf0 con 0x7fe9680692b0 2024-04-22T23:33:48.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.708+0000 7fe972ffd700 1 -- 172.21.15.107:0/2948997620 <== mgr.14206 v2:172.21.15.107:6800/2516490111 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3366 (secure 0 0 0) 0x7fe960000bf0 con 0x7fe9680692b0 2024-04-22T23:33:48.704 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-22T23:33:48.704 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (5m) 57s ago 6m 25.5M - 0.25.0 c8568f914cd2 4d90ceb89249 2024-04-22T23:33:48.704 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (7m) 57s ago 7m 6660k - 17.2.7-884-g70d0cb4b cafd8ddac8fa c10611c756e4 2024-04-22T23:33:48.704 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi157 smithi157 running (6m) 63s ago 6m 6669k - 17.2.7-884-g70d0cb4b cafd8ddac8fa 55d2d46294c9 2024-04-22T23:33:48.704 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (5m) 57s ago 6m 86.8M - 9.4.7 954c08fa6188 c2d1f8de1aa7 2024-04-22T23:33:48.704 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.dfvvkw smithi107 running (85s) 57s ago 84s 10.4M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 08d3aa8d717d 2024-04-22T23:33:48.704 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.ktuxjl smithi107 running (78s) 57s ago 77s 16.9M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 762f27fe0f30 2024-04-22T23:33:48.705 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi157.rcaris smithi157 running (82s) 63s ago 81s 16.8M - 17.2.7-884-g70d0cb4b cafd8ddac8fa d23366ac085d 2024-04-22T23:33:48.705 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi157.ufywse smithi157 running (75s) 63s ago 75s 11.0M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 8b00d61eb355 2024-04-22T23:33:48.705 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.uipxvm smithi107 *:9283 running (8m) 57s ago 8m 481M - 17.2.7-884-g70d0cb4b cafd8ddac8fa db96e8cbbdfb 2024-04-22T23:33:48.705 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi157.tuylis smithi157 *:8443,9283 running (5m) 63s ago 5m 431M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 0c12be10ffaf 2024-04-22T23:33:48.705 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (8m) 57s ago 8m 49.6M 2048M 17.2.7-884-g70d0cb4b cafd8ddac8fa 8314962fa1b9 2024-04-22T23:33:48.705 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi157 smithi157 running (5m) 63s ago 5m 36.6M 2048M 17.2.7-884-g70d0cb4b cafd8ddac8fa 0747a85008d9 2024-04-22T23:33:48.705 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (6m) 57s ago 6m 16.0M - 1.5.0 0da6a335fe13 8a7c595c114f 2024-04-22T23:33:48.705 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi157 smithi157 *:9100 running (5m) 63s ago 5m 16.1M - 1.5.0 0da6a335fe13 940164771fe3 2024-04-22T23:33:48.705 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (5m) 57s ago 5m 46.6M 4288M 17.2.7-884-g70d0cb4b cafd8ddac8fa 18a179b0a618 2024-04-22T23:33:48.705 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (4m) 57s ago 4m 45.9M 4288M 17.2.7-884-g70d0cb4b cafd8ddac8fa c7272eb84ddb 2024-04-22T23:33:48.705 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (4m) 57s ago 4m 46.2M 4288M 17.2.7-884-g70d0cb4b cafd8ddac8fa 9db1788096ad 2024-04-22T23:33:48.705 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi157 running (3m) 63s ago 3m 43.9M 5312M 17.2.7-884-g70d0cb4b cafd8ddac8fa c9fb2d1a13dd 2024-04-22T23:33:48.705 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi157 running (3m) 63s ago 3m 44.2M 5312M 17.2.7-884-g70d0cb4b cafd8ddac8fa 24a16e19951f 2024-04-22T23:33:48.705 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi157 running (2m) 63s ago 2m 43.1M 5312M 17.2.7-884-g70d0cb4b cafd8ddac8fa 4c5b7a7ef8bd 2024-04-22T23:33:48.706 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (5m) 57s ago 6m 51.2M - 2.43.0 a07b618ecd1d 8c5df40caabe 2024-04-22T23:33:48.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.710+0000 7fe98429a700 1 -- 172.21.15.107:0/2948997620 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fe9680692b0 msgr2=0x7fe96806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:48.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.710+0000 7fe98429a700 1 --2- 172.21.15.107:0/2948997620 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fe9680692b0 0x7fe96806b760 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe978013870 tx=0x7fe978009e30 comp rx=0 tx=0).stop 2024-04-22T23:33:48.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.710+0000 7fe98429a700 1 -- 172.21.15.107:0/2948997620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe97c0fe040 msgr2=0x7fe97c10df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:48.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.710+0000 7fe98429a700 1 --2- 172.21.15.107:0/2948997620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe97c0fe040 0x7fe97c10df10 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe96c00eb40 tx=0x7fe96c00b040 comp rx=0 tx=0).stop 2024-04-22T23:33:48.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.710+0000 7fe98429a700 1 -- 172.21.15.107:0/2948997620 shutdown_connections 2024-04-22T23:33:48.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.710+0000 7fe98429a700 1 --2- 172.21.15.107:0/2948997620 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fe9680692b0 0x7fe96806b760 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:48.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.710+0000 7fe98429a700 1 --2- 172.21.15.107:0/2948997620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe97c0fce40 0x7fe97c10d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:48.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.710+0000 7fe98429a700 1 --2- 172.21.15.107:0/2948997620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe97c0fe040 0x7fe97c10df10 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:48.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.711+0000 7fe98429a700 1 -- 172.21.15.107:0/2948997620 >> 172.21.15.107:0/2948997620 conn(0x7fe97c0f8530 msgr2=0x7fe97c0f9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:33:48.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.711+0000 7fe98429a700 1 -- 172.21.15.107:0/2948997620 shutdown_connections 2024-04-22T23:33:48.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.711+0000 7fe98429a700 1 -- 172.21.15.107:0/2948997620 wait complete. 2024-04-22T23:33:48.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.818+0000 7f929c73d700 1 -- 172.21.15.107:0/305540010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92940fcff0 msgr2=0x7f92940fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:48.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.818+0000 7f929c73d700 1 --2- 172.21.15.107:0/305540010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92940fcff0 0x7f92940fd460 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9284004940 tx=0x7f9284009e30 comp rx=0 tx=0).stop 2024-04-22T23:33:48.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.818+0000 7f929c73d700 1 -- 172.21.15.107:0/305540010 shutdown_connections 2024-04-22T23:33:48.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.818+0000 7f929c73d700 1 --2- 172.21.15.107:0/305540010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92940fcff0 0x7f92940fd460 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:48.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.818+0000 7f929c73d700 1 --2- 172.21.15.107:0/305540010 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92940fbd40 0x7f92940fc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:48.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.818+0000 7f929c73d700 1 -- 172.21.15.107:0/305540010 >> 172.21.15.107:0/305540010 conn(0x7f92940f74c0 msgr2=0x7f92940f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:33:48.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.818+0000 7f929c73d700 1 -- 172.21.15.107:0/305540010 shutdown_connections 2024-04-22T23:33:48.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.818+0000 7f929c73d700 1 -- 172.21.15.107:0/305540010 wait complete. 2024-04-22T23:33:48.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.818+0000 7f929c73d700 1 Processor -- start 2024-04-22T23:33:48.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.819+0000 7f929c73d700 1 -- start start 2024-04-22T23:33:48.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.819+0000 7f929c73d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92940fbd40 0x7f929410d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:48.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.819+0000 7f929c73d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92940fcff0 0x7f929410ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:48.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.819+0000 7f929c73d700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929410e4d0 con 0x7f92940fbd40 2024-04-22T23:33:48.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.819+0000 7f929c73d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929410ed20 con 0x7f92940fcff0 2024-04-22T23:33:48.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.819+0000 7f929a4d9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92940fbd40 0x7f929410d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:48.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.819+0000 7f929a4d9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92940fbd40 0x7f929410d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49106/0 (socket says 172.21.15.107:49106) 2024-04-22T23:33:48.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.819+0000 7f929a4d9700 1 -- 172.21.15.107:0/4174532883 learned_addr learned my addr 172.21.15.107:0/4174532883 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:33:48.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.819+0000 7f9299cd8700 1 --2- 172.21.15.107:0/4174532883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92940fcff0 0x7f929410ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:48.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.819+0000 7f929a4d9700 1 -- 172.21.15.107:0/4174532883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92940fcff0 msgr2=0x7f929410ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:48.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.819+0000 7f929a4d9700 1 --2- 172.21.15.107:0/4174532883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92940fcff0 0x7f929410ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:48.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.819+0000 7f929a4d9700 1 -- 172.21.15.107:0/4174532883 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9284005040 con 0x7f92940fbd40 2024-04-22T23:33:48.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.819+0000 7f9299cd8700 1 --2- 172.21.15.107:0/4174532883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92940fcff0 0x7f929410ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T23:33:48.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.819+0000 7f929a4d9700 1 --2- 172.21.15.107:0/4174532883 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92940fbd40 0x7f929410d990 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f929000eb40 tx=0x7f929000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:33:48.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.819+0000 7f928affd700 1 -- 172.21.15.107:0/4174532883 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9290007cc0 con 0x7f92940fbd40 2024-04-22T23:33:48.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.820+0000 7f928affd700 1 -- 172.21.15.107:0/4174532883 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f929000f040 con 0x7f92940fbd40 2024-04-22T23:33:48.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.820+0000 7f929c73d700 1 -- 172.21.15.107:0/4174532883 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f929410efa0 con 0x7f92940fbd40 2024-04-22T23:33:48.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.820+0000 7f928affd700 1 -- 172.21.15.107:0/4174532883 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9290009620 con 0x7f92940fbd40 2024-04-22T23:33:48.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.820+0000 7f929c73d700 1 -- 172.21.15.107:0/4174532883 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f929410f4e0 con 0x7f92940fbd40 2024-04-22T23:33:48.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.821+0000 7f929c73d700 1 -- 172.21.15.107:0/4174532883 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9294101b80 con 0x7f92940fbd40 2024-04-22T23:33:48.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.821+0000 7f928affd700 1 -- 172.21.15.107:0/4174532883 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f9290009780 con 0x7f92940fbd40 2024-04-22T23:33:48.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.822+0000 7f928affd700 1 --2- 172.21.15.107:0/4174532883 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f92800692a0 0x7f928006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:48.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.822+0000 7f928affd700 1 -- 172.21.15.107:0/4174532883 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f92900870d0 con 0x7f92940fbd40 2024-04-22T23:33:48.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.822+0000 7f9299cd8700 1 --2- 172.21.15.107:0/4174532883 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f92800692a0 0x7f928006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:48.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.822+0000 7f9299cd8700 1 --2- 172.21.15.107:0/4174532883 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f92800692a0 0x7f928006b750 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9284009b30 tx=0x7f9284009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:33:48.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:48.826+0000 7f928affd700 1 -- 172.21.15.107:0/4174532883 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f9290056ef0 con 0x7f92940fbd40 2024-04-22T23:33:49.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.159+0000 7f929c73d700 1 -- 172.21.15.107:0/4174532883 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9294104820 con 0x7f92940fbd40 2024-04-22T23:33:49.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.159+0000 7f928affd700 1 -- 172.21.15.107:0/4174532883 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7f9290056a80 con 0x7f92940fbd40 2024-04-22T23:33:49.155 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-22T23:33:49.155 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-04-22T23:33:49.155 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 2 2024-04-22T23:33:49.155 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:33:49.155 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-04-22T23:33:49.155 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 2 2024-04-22T23:33:49.156 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:33:49.156 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-04-22T23:33:49.156 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 6 2024-04-22T23:33:49.156 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:33:49.156 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-04-22T23:33:49.156 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 4 2024-04-22T23:33:49.156 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:33:49.156 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-04-22T23:33:49.156 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 14 2024-04-22T23:33:49.156 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T23:33:49.156 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-22T23:33:49.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.162+0000 7f929c73d700 1 -- 172.21.15.107:0/4174532883 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f92800692a0 msgr2=0x7f928006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:49.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.162+0000 7f929c73d700 1 --2- 172.21.15.107:0/4174532883 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f92800692a0 0x7f928006b750 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9284009b30 tx=0x7f9284009e30 comp rx=0 tx=0).stop 2024-04-22T23:33:49.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.162+0000 7f929c73d700 1 -- 172.21.15.107:0/4174532883 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92940fbd40 msgr2=0x7f929410d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:49.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.162+0000 7f929c73d700 1 --2- 172.21.15.107:0/4174532883 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92940fbd40 0x7f929410d990 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f929000eb40 tx=0x7f929000b040 comp rx=0 tx=0).stop 2024-04-22T23:33:49.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.162+0000 7f929c73d700 1 -- 172.21.15.107:0/4174532883 shutdown_connections 2024-04-22T23:33:49.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.162+0000 7f929c73d700 1 --2- 172.21.15.107:0/4174532883 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f92800692a0 0x7f928006b750 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:49.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.162+0000 7f929c73d700 1 --2- 172.21.15.107:0/4174532883 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92940fbd40 0x7f929410d990 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:49.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.162+0000 7f929c73d700 1 --2- 172.21.15.107:0/4174532883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92940fcff0 0x7f929410ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:49.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.163+0000 7f929c73d700 1 -- 172.21.15.107:0/4174532883 >> 172.21.15.107:0/4174532883 conn(0x7f92940f74c0 msgr2=0x7f92940f8ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:33:49.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.163+0000 7f929c73d700 1 -- 172.21.15.107:0/4174532883 shutdown_connections 2024-04-22T23:33:49.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.163+0000 7f929c73d700 1 -- 172.21.15.107:0/4174532883 wait complete. 2024-04-22T23:33:49.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.278+0000 7f1f59e2c700 1 -- 172.21.15.107:0/1474113921 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f540fcff0 msgr2=0x7f1f540fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:49.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.278+0000 7f1f59e2c700 1 --2- 172.21.15.107:0/1474113921 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f540fcff0 0x7f1f540fd460 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1f48004940 tx=0x7f1f48009e30 comp rx=0 tx=0).stop 2024-04-22T23:33:49.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.278+0000 7f1f59e2c700 1 -- 172.21.15.107:0/1474113921 shutdown_connections 2024-04-22T23:33:49.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.278+0000 7f1f59e2c700 1 --2- 172.21.15.107:0/1474113921 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f540fcff0 0x7f1f540fd460 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:49.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.278+0000 7f1f59e2c700 1 --2- 172.21.15.107:0/1474113921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f540fbd40 0x7f1f540fc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:49.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.278+0000 7f1f59e2c700 1 -- 172.21.15.107:0/1474113921 >> 172.21.15.107:0/1474113921 conn(0x7f1f540f74c0 msgr2=0x7f1f540f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:33:49.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.278+0000 7f1f59e2c700 1 -- 172.21.15.107:0/1474113921 shutdown_connections 2024-04-22T23:33:49.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.278+0000 7f1f59e2c700 1 -- 172.21.15.107:0/1474113921 wait complete. 2024-04-22T23:33:49.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.279+0000 7f1f59e2c700 1 Processor -- start 2024-04-22T23:33:49.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.279+0000 7f1f59e2c700 1 -- start start 2024-04-22T23:33:49.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.279+0000 7f1f59e2c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f540fbd40 0x7f1f5410c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:49.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.279+0000 7f1f59e2c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f540fcff0 0x7f1f5410ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:49.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.280+0000 7f1f59e2c700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5410d420 con 0x7f1f540fbd40 2024-04-22T23:33:49.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.280+0000 7f1f59e2c700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5410dc20 con 0x7f1f540fcff0 2024-04-22T23:33:49.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.280+0000 7f1f537fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f540fbd40 0x7f1f5410c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:49.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.280+0000 7f1f52ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f540fcff0 0x7f1f5410ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:49.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.280+0000 7f1f537fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f540fbd40 0x7f1f5410c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49126/0 (socket says 172.21.15.107:49126) 2024-04-22T23:33:49.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.280+0000 7f1f52ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f540fcff0 0x7f1f5410ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.107:49604/0 (socket says 172.21.15.107:49604) 2024-04-22T23:33:49.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.280+0000 7f1f537fe700 1 -- 172.21.15.107:0/3641610052 learned_addr learned my addr 172.21.15.107:0/3641610052 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:33:49.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.280+0000 7f1f537fe700 1 -- 172.21.15.107:0/3641610052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f540fcff0 msgr2=0x7f1f5410ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:49.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.280+0000 7f1f537fe700 1 --2- 172.21.15.107:0/3641610052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f540fcff0 0x7f1f5410ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:49.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.280+0000 7f1f537fe700 1 -- 172.21.15.107:0/3641610052 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f48005040 con 0x7f1f540fbd40 2024-04-22T23:33:49.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.280+0000 7f1f52ffd700 1 --2- 172.21.15.107:0/3641610052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f540fcff0 0x7f1f5410ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T23:33:49.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.280+0000 7f1f537fe700 1 --2- 172.21.15.107:0/3641610052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f540fbd40 0x7f1f5410c8e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1f4400eb40 tx=0x7f1f4400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:33:49.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.281+0000 7f1f3bfff700 1 -- 172.21.15.107:0/3641610052 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f44007cc0 con 0x7f1f540fbd40 2024-04-22T23:33:49.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.281+0000 7f1f59e2c700 1 -- 172.21.15.107:0/3641610052 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f5410dea0 con 0x7f1f540fbd40 2024-04-22T23:33:49.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.281+0000 7f1f3bfff700 1 -- 172.21.15.107:0/3641610052 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1f4400f040 con 0x7f1f540fbd40 2024-04-22T23:33:49.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.281+0000 7f1f3bfff700 1 -- 172.21.15.107:0/3641610052 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f4400ca10 con 0x7f1f540fbd40 2024-04-22T23:33:49.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.281+0000 7f1f59e2c700 1 -- 172.21.15.107:0/3641610052 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f5410e3e0 con 0x7f1f540fbd40 2024-04-22T23:33:49.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.282+0000 7f1f59e2c700 1 -- 172.21.15.107:0/3641610052 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f5405f5f0 con 0x7f1f540fbd40 2024-04-22T23:33:49.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.283+0000 7f1f3bfff700 1 -- 172.21.15.107:0/3641610052 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f1f4400cb70 con 0x7f1f540fbd40 2024-04-22T23:33:49.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.284+0000 7f1f3bfff700 1 --2- 172.21.15.107:0/3641610052 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f1f3c0692a0 0x7f1f3c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:49.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.284+0000 7f1f3bfff700 1 -- 172.21.15.107:0/3641610052 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f1f44086f00 con 0x7f1f540fbd40 2024-04-22T23:33:49.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.284+0000 7f1f52ffd700 1 --2- 172.21.15.107:0/3641610052 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f1f3c0692a0 0x7f1f3c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:49.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.284+0000 7f1f52ffd700 1 --2- 172.21.15.107:0/3641610052 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f1f3c0692a0 0x7f1f3c06b750 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1f48009b30 tx=0x7f1f48009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:33:49.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.288+0000 7f1f3bfff700 1 -- 172.21.15.107:0/3641610052 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f1f44056dc0 con 0x7f1f540fbd40 2024-04-22T23:33:49.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:33:49 smithi157 ceph-mon[33877]: from='client.14580 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:33:49.383 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:33:49 smithi157 ceph-mon[33877]: pgmap v221: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-04-22T23:33:49.383 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:33:49 smithi157 ceph-mon[33877]: from='client.14584 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:33:49.383 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:33:49 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:33:49.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:33:49 smithi107 ceph-mon[26355]: from='client.14580 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:33:49.485 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:33:49 smithi107 ceph-mon[26355]: pgmap v221: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-04-22T23:33:49.485 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:33:49 smithi107 ceph-mon[26355]: from='client.14584 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:33:49.485 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:33:49 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:33:49.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.541+0000 7f1f59e2c700 1 -- 172.21.15.107:0/3641610052 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1f541047c0 con 0x7f1f540fbd40 2024-04-22T23:33:49.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.542+0000 7f1f3bfff700 1 -- 172.21.15.107:0/3641610052 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1824 (secure 0 0 0) 0x7f1f44056950 con 0x7f1f540fbd40 2024-04-22T23:33:49.537 INFO:teuthology.orchestra.run.smithi107.stdout:e17 2024-04-22T23:33:49.538 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T23:33:49.538 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:33:49.538 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-04-22T23:33:49.538 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:33:49.538 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-04-22T23:33:49.538 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-04-22T23:33:49.538 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 17 2024-04-22T23:33:49.538 INFO:teuthology.orchestra.run.smithi107.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-22T23:33:49.538 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-04-22T23:32:20.146130+0000 2024-04-22T23:33:49.538 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-04-22T23:32:39.774747+0000 2024-04-22T23:33:49.538 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-04-22T23:33:49.538 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-04-22T23:33:49.539 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-04-22T23:33:49.539 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-04-22T23:33:49.539 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-04-22T23:33:49.539 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-04-22T23:33:49.539 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-04-22T23:33:49.539 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 46 2024-04-22T23:33:49.539 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:33:49.539 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 2 2024-04-22T23:33:49.539 INFO:teuthology.orchestra.run.smithi107.stdout:in 0,1 2024-04-22T23:33:49.539 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=14468,1=24265} 2024-04-22T23:33:49.539 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-04-22T23:33:49.539 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-04-22T23:33:49.539 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-04-22T23:33:49.540 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-04-22T23:33:49.540 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-04-22T23:33:49.540 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data disabled 2024-04-22T23:33:49.540 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-04-22T23:33:49.540 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-04-22T23:33:49.540 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.ktuxjl{0:14468} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.107:6828/3335313253,v1:172.21.15.107:6829/3335313253] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:33:49.540 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi157.rcaris{1:24265} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.157:6824/2779251893,v1:172.21.15.157:6825/2779251893] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:33:49.540 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:33:49.540 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:33:49.540 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-04-22T23:33:49.540 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:33:49.540 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.dfvvkw{-1:14484} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/3548717247,v1:172.21.15.107:6827/3548717247] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:33:49.541 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi157.ufywse{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.157:6826/4072444199,v1:172.21.15.157:6827/4072444199] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:33:49.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.544+0000 7f1f59e2c700 1 -- 172.21.15.107:0/3641610052 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f1f3c0692a0 msgr2=0x7f1f3c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:49.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.544+0000 7f1f59e2c700 1 --2- 172.21.15.107:0/3641610052 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f1f3c0692a0 0x7f1f3c06b750 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1f48009b30 tx=0x7f1f48009e30 comp rx=0 tx=0).stop 2024-04-22T23:33:49.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.544+0000 7f1f59e2c700 1 -- 172.21.15.107:0/3641610052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f540fbd40 msgr2=0x7f1f5410c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:49.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.544+0000 7f1f59e2c700 1 --2- 172.21.15.107:0/3641610052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f540fbd40 0x7f1f5410c8e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1f4400eb40 tx=0x7f1f4400b040 comp rx=0 tx=0).stop 2024-04-22T23:33:49.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.544+0000 7f1f59e2c700 1 -- 172.21.15.107:0/3641610052 shutdown_connections 2024-04-22T23:33:49.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.544+0000 7f1f59e2c700 1 --2- 172.21.15.107:0/3641610052 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f1f3c0692a0 0x7f1f3c06b750 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:49.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.544+0000 7f1f59e2c700 1 --2- 172.21.15.107:0/3641610052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f540fbd40 0x7f1f5410c8e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:49.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.544+0000 7f1f59e2c700 1 --2- 172.21.15.107:0/3641610052 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f540fcff0 0x7f1f5410ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:49.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.544+0000 7f1f59e2c700 1 -- 172.21.15.107:0/3641610052 >> 172.21.15.107:0/3641610052 conn(0x7f1f540f74c0 msgr2=0x7f1f540f8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:33:49.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.544+0000 7f1f59e2c700 1 -- 172.21.15.107:0/3641610052 shutdown_connections 2024-04-22T23:33:49.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.544+0000 7f1f59e2c700 1 -- 172.21.15.107:0/3641610052 wait complete. 2024-04-22T23:33:49.543 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 17 2024-04-22T23:33:49.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.652+0000 7f052a379700 1 -- 172.21.15.107:0/1028240513 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05240fe060 msgr2=0x7f05240fe4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:49.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.652+0000 7f052a379700 1 --2- 172.21.15.107:0/1028240513 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05240fe060 0x7f05240fe4d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0518004940 tx=0x7f0518009e30 comp rx=0 tx=0).stop 2024-04-22T23:33:49.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.652+0000 7f052a379700 1 -- 172.21.15.107:0/1028240513 shutdown_connections 2024-04-22T23:33:49.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.652+0000 7f052a379700 1 --2- 172.21.15.107:0/1028240513 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05240fe060 0x7f05240fe4d0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:49.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.652+0000 7f052a379700 1 --2- 172.21.15.107:0/1028240513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05240fce60 0x7f05240fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:49.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.652+0000 7f052a379700 1 -- 172.21.15.107:0/1028240513 >> 172.21.15.107:0/1028240513 conn(0x7f05240f8570 msgr2=0x7f05240fa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:33:49.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.652+0000 7f052a379700 1 -- 172.21.15.107:0/1028240513 shutdown_connections 2024-04-22T23:33:49.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.652+0000 7f052a379700 1 -- 172.21.15.107:0/1028240513 wait complete. 2024-04-22T23:33:49.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.652+0000 7f052a379700 1 Processor -- start 2024-04-22T23:33:49.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.653+0000 7f052a379700 1 -- start start 2024-04-22T23:33:49.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.653+0000 7f052a379700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05240fce60 0x7f052410d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:49.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.653+0000 7f052a379700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05240fe060 0x7f052410df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:49.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.653+0000 7f052a379700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052410e500 con 0x7f05240fe060 2024-04-22T23:33:49.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.653+0000 7f052a379700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052410ed50 con 0x7f05240fce60 2024-04-22T23:33:49.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.653+0000 7f05237fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05240fe060 0x7f052410df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:49.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.653+0000 7f05237fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05240fe060 0x7f052410df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49142/0 (socket says 172.21.15.107:49142) 2024-04-22T23:33:49.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.653+0000 7f0523fff700 1 --2- 172.21.15.107:0/3405306315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05240fce60 0x7f052410d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:49.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.653+0000 7f05237fe700 1 -- 172.21.15.107:0/3405306315 learned_addr learned my addr 172.21.15.107:0/3405306315 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:33:49.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.653+0000 7f05237fe700 1 -- 172.21.15.107:0/3405306315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05240fce60 msgr2=0x7f052410d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:49.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.653+0000 7f05237fe700 1 --2- 172.21.15.107:0/3405306315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05240fce60 0x7f052410d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:49.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.653+0000 7f05237fe700 1 -- 172.21.15.107:0/3405306315 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0518005040 con 0x7f05240fe060 2024-04-22T23:33:49.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.653+0000 7f0523fff700 1 --2- 172.21.15.107:0/3405306315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05240fce60 0x7f052410d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T23:33:49.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.654+0000 7f05237fe700 1 --2- 172.21.15.107:0/3405306315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05240fe060 0x7f052410df00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0518013870 tx=0x7f0518009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:33:49.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.654+0000 7f0520ff9700 1 -- 172.21.15.107:0/3405306315 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0518021070 con 0x7f05240fe060 2024-04-22T23:33:49.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.654+0000 7f0520ff9700 1 -- 172.21.15.107:0/3405306315 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0518009430 con 0x7f05240fe060 2024-04-22T23:33:49.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.654+0000 7f052a379700 1 -- 172.21.15.107:0/3405306315 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f052410efd0 con 0x7f05240fe060 2024-04-22T23:33:49.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.654+0000 7f0520ff9700 1 -- 172.21.15.107:0/3405306315 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f051800e070 con 0x7f05240fe060 2024-04-22T23:33:49.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.654+0000 7f052a379700 1 -- 172.21.15.107:0/3405306315 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f052410f510 con 0x7f05240fe060 2024-04-22T23:33:49.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.655+0000 7f052a379700 1 -- 172.21.15.107:0/3405306315 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f052405f5f0 con 0x7f05240fe060 2024-04-22T23:33:49.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.656+0000 7f0520ff9700 1 -- 172.21.15.107:0/3405306315 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f051801c750 con 0x7f05240fe060 2024-04-22T23:33:49.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.656+0000 7f0520ff9700 1 --2- 172.21.15.107:0/3405306315 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f050c0692a0 0x7f050c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:49.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.656+0000 7f0520ff9700 1 -- 172.21.15.107:0/3405306315 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f05180883f0 con 0x7f05240fe060 2024-04-22T23:33:49.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.656+0000 7f0523fff700 1 --2- 172.21.15.107:0/3405306315 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f050c0692a0 0x7f050c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:49.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.657+0000 7f0523fff700 1 --2- 172.21.15.107:0/3405306315 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f050c0692a0 0x7f050c06b750 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0514009b80 tx=0x7f051400b040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:33:49.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.661+0000 7f0520ff9700 1 -- 172.21.15.107:0/3405306315 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0518058200 con 0x7f05240fe060 2024-04-22T23:33:49.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.885+0000 7f052a379700 1 -- 172.21.15.107:0/3405306315 --> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f052410f820 con 0x7f050c0692a0 2024-04-22T23:33:49.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.887+0000 7f0520ff9700 1 -- 172.21.15.107:0/3405306315 <== mgr.14206 v2:172.21.15.107:6800/2516490111 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f052410f820 con 0x7f050c0692a0 2024-04-22T23:33:49.882 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-22T23:33:49.882 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:43be020184947e53516056c9931e1ac5bdbbb1a5", 2024-04-22T23:33:49.882 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-04-22T23:33:49.882 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-22T23:33:49.883 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [], 2024-04-22T23:33:49.883 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "", 2024-04-22T23:33:49.883 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Doing first pull of quay.ceph.io/ceph-ci/ceph:43be020184947e53516056c9931e1ac5bdbbb1a5 image", 2024-04-22T23:33:49.883 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-04-22T23:33:49.883 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-22T23:33:49.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.889+0000 7f052a379700 1 -- 172.21.15.107:0/3405306315 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f050c0692a0 msgr2=0x7f050c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:49.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.889+0000 7f052a379700 1 --2- 172.21.15.107:0/3405306315 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f050c0692a0 0x7f050c06b750 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0514009b80 tx=0x7f051400b040 comp rx=0 tx=0).stop 2024-04-22T23:33:49.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.889+0000 7f052a379700 1 -- 172.21.15.107:0/3405306315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05240fe060 msgr2=0x7f052410df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:49.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.889+0000 7f052a379700 1 --2- 172.21.15.107:0/3405306315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05240fe060 0x7f052410df00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0518013870 tx=0x7f0518009e30 comp rx=0 tx=0).stop 2024-04-22T23:33:49.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.889+0000 7f052a379700 1 -- 172.21.15.107:0/3405306315 shutdown_connections 2024-04-22T23:33:49.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.890+0000 7f052a379700 1 --2- 172.21.15.107:0/3405306315 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f050c0692a0 0x7f050c06b750 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:49.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.890+0000 7f052a379700 1 --2- 172.21.15.107:0/3405306315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05240fce60 0x7f052410d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:49.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.890+0000 7f052a379700 1 --2- 172.21.15.107:0/3405306315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05240fe060 0x7f052410df00 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:49.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.890+0000 7f052a379700 1 -- 172.21.15.107:0/3405306315 >> 172.21.15.107:0/3405306315 conn(0x7f05240f8570 msgr2=0x7f05240f9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:33:49.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.890+0000 7f052a379700 1 -- 172.21.15.107:0/3405306315 shutdown_connections 2024-04-22T23:33:49.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:49.890+0000 7f052a379700 1 -- 172.21.15.107:0/3405306315 wait complete. 2024-04-22T23:33:49.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:50.002+0000 7fadefc59700 1 -- 172.21.15.107:0/1534149449 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fade80fce50 msgr2=0x7fade80fd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:50.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:50.002+0000 7fadefc59700 1 --2- 172.21.15.107:0/1534149449 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fade80fce50 0x7fade80fd260 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fade4004940 tx=0x7fade4009e30 comp rx=0 tx=0).stop 2024-04-22T23:33:50.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:50.002+0000 7fadefc59700 1 -- 172.21.15.107:0/1534149449 shutdown_connections 2024-04-22T23:33:50.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:50.002+0000 7fadefc59700 1 --2- 172.21.15.107:0/1534149449 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fade80fe050 0x7fade80fe4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:50.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:50.002+0000 7fadefc59700 1 --2- 172.21.15.107:0/1534149449 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fade80fce50 0x7fade80fd260 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:50.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:50.002+0000 7fadefc59700 1 -- 172.21.15.107:0/1534149449 >> 172.21.15.107:0/1534149449 conn(0x7fade80f8580 msgr2=0x7fade80fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:33:50.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:50.002+0000 7fadefc59700 1 -- 172.21.15.107:0/1534149449 shutdown_connections 2024-04-22T23:33:50.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:50.002+0000 7fadefc59700 1 -- 172.21.15.107:0/1534149449 wait complete. 2024-04-22T23:33:50.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:50.003+0000 7fadefc59700 1 Processor -- start 2024-04-22T23:33:50.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:50.003+0000 7fadefc59700 1 -- start start 2024-04-22T23:33:50.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:50.003+0000 7fadefc59700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fade80fce50 0x7fade810da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:50.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:50.003+0000 7fadefc59700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fade80fe050 0x7fade810df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:50.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:50.003+0000 7fadefc59700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade810e560 con 0x7fade80fe050 2024-04-22T23:33:50.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:50.003+0000 7fadefc59700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade810edb0 con 0x7fade80fce50 2024-04-22T23:33:50.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:50.003+0000 7faded1f4700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fade80fe050 0x7fade810df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:50.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:50.003+0000 7faded1f4700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fade80fe050 0x7fade810df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49152/0 (socket says 172.21.15.107:49152) 2024-04-22T23:33:50.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:50.003+0000 7faded1f4700 1 -- 172.21.15.107:0/372257201 learned_addr learned my addr 172.21.15.107:0/372257201 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:33:50.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:50.003+0000 7faded9f5700 1 --2- 172.21.15.107:0/372257201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fade80fce50 0x7fade810da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:50.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:50.004+0000 7faded1f4700 1 -- 172.21.15.107:0/372257201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fade80fce50 msgr2=0x7fade810da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:50.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:50.004+0000 7faded1f4700 1 --2- 172.21.15.107:0/372257201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fade80fce50 0x7fade810da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:50.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:50.004+0000 7faded1f4700 1 -- 172.21.15.107:0/372257201 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade4005040 con 0x7fade80fe050 2024-04-22T23:33:50.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:50.004+0000 7faded9f5700 1 --2- 172.21.15.107:0/372257201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fade80fce50 0x7fade810da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T23:33:50.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:50.004+0000 7faded1f4700 1 --2- 172.21.15.107:0/372257201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fade80fe050 0x7fade810df60 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fadd8002a80 tx=0x7fadd80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:33:50.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:50.004+0000 7fadde7fc700 1 -- 172.21.15.107:0/372257201 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadd8007cc0 con 0x7fade80fe050 2024-04-22T23:33:50.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:50.004+0000 7fadde7fc700 1 -- 172.21.15.107:0/372257201 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fadd800f040 con 0x7fade80fe050 2024-04-22T23:33:50.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:50.004+0000 7fadde7fc700 1 -- 172.21.15.107:0/372257201 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadd80096f0 con 0x7fade80fe050 2024-04-22T23:33:50.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:50.004+0000 7fadefc59700 1 -- 172.21.15.107:0/372257201 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fade810f090 con 0x7fade80fe050 2024-04-22T23:33:50.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:50.004+0000 7fadefc59700 1 -- 172.21.15.107:0/372257201 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fade810f5d0 con 0x7fade80fe050 2024-04-22T23:33:50.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:50.005+0000 7fadefc59700 1 -- 172.21.15.107:0/372257201 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fade805f5f0 con 0x7fade80fe050 2024-04-22T23:33:50.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:50.006+0000 7fadde7fc700 1 -- 172.21.15.107:0/372257201 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fadd800d690 con 0x7fade80fe050 2024-04-22T23:33:50.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:50.006+0000 7fadde7fc700 1 --2- 172.21.15.107:0/372257201 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fadd40692b0 0x7fadd406b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:33:50.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:50.006+0000 7fadde7fc700 1 -- 172.21.15.107:0/372257201 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fadd80868d0 con 0x7fade80fe050 2024-04-22T23:33:50.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:50.006+0000 7faded9f5700 1 --2- 172.21.15.107:0/372257201 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fadd40692b0 0x7fadd406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:33:50.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:50.007+0000 7faded9f5700 1 --2- 172.21.15.107:0/372257201 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fadd40692b0 0x7fadd406b760 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fade400b3b0 tx=0x7fade4009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:33:50.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:50.011+0000 7fadde7fc700 1 -- 172.21.15.107:0/372257201 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fadd804d250 con 0x7fade80fe050 2024-04-22T23:33:50.234 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:33:50 smithi107 ceph-mon[26355]: from='client.14588 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:33:50.234 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:33:50 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/4174532883' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:33:50.235 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:33:50 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/3641610052' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:33:50.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:50.305+0000 7fadefc59700 1 -- 172.21.15.107:0/372257201 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fade804bed0 con 0x7fade80fe050 2024-04-22T23:33:50.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:50.305+0000 7fadde7fc700 1 -- 172.21.15.107:0/372257201 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fadd8056640 con 0x7fade80fe050 2024-04-22T23:33:50.301 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_OK 2024-04-22T23:33:50.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:50.308+0000 7fadefc59700 1 -- 172.21.15.107:0/372257201 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fadd40692b0 msgr2=0x7fadd406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:50.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:50.308+0000 7fadefc59700 1 --2- 172.21.15.107:0/372257201 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fadd40692b0 0x7fadd406b760 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fade400b3b0 tx=0x7fade4009e30 comp rx=0 tx=0).stop 2024-04-22T23:33:50.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:50.308+0000 7fadefc59700 1 -- 172.21.15.107:0/372257201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fade80fe050 msgr2=0x7fade810df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:33:50.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:50.308+0000 7fadefc59700 1 --2- 172.21.15.107:0/372257201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fade80fe050 0x7fade810df60 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fadd8002a80 tx=0x7fadd80046b0 comp rx=0 tx=0).stop 2024-04-22T23:33:50.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:50.308+0000 7fadefc59700 1 -- 172.21.15.107:0/372257201 shutdown_connections 2024-04-22T23:33:50.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:50.308+0000 7fadefc59700 1 --2- 172.21.15.107:0/372257201 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fadd40692b0 0x7fadd406b760 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:50.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:50.308+0000 7fadefc59700 1 --2- 172.21.15.107:0/372257201 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fade80fce50 0x7fade810da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:50.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:50.308+0000 7fadefc59700 1 --2- 172.21.15.107:0/372257201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fade80fe050 0x7fade810df60 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:33:50.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:50.308+0000 7fadefc59700 1 -- 172.21.15.107:0/372257201 >> 172.21.15.107:0/372257201 conn(0x7fade80f8580 msgr2=0x7fade80f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:33:50.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:50.308+0000 7fadefc59700 1 -- 172.21.15.107:0/372257201 shutdown_connections 2024-04-22T23:33:50.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:33:50.308+0000 7fadefc59700 1 -- 172.21.15.107:0/372257201 wait complete. 2024-04-22T23:33:50.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:33:50 smithi157 ceph-mon[33877]: from='client.14588 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:33:50.383 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:33:50 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/4174532883' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:33:50.383 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:33:50 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/3641610052' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:33:51.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:33:51 smithi157 ceph-mon[33877]: from='client.14600 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:33:51.383 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:33:51 smithi157 ceph-mon[33877]: pgmap v222: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-04-22T23:33:51.383 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:33:51 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/372257201' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:33:51.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:33:51 smithi107 ceph-mon[26355]: from='client.14600 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:33:51.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:33:51 smithi107 ceph-mon[26355]: pgmap v222: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-04-22T23:33:51.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:33:51 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/372257201' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:33:53.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:33:53 smithi157 ceph-mon[33877]: pgmap v223: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-04-22T23:33:53.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:33:53 smithi107 ceph-mon[26355]: pgmap v223: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-04-22T23:33:55.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:33:55 smithi157 ceph-mon[33877]: pgmap v224: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:33:55.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:33:55 smithi107 ceph-mon[26355]: pgmap v224: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:33:57.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:33:57 smithi157 ceph-mon[33877]: pgmap v225: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:33:57.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:33:57 smithi107 ceph-mon[26355]: pgmap v225: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:33:59.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:33:59 smithi157 ceph-mon[33877]: pgmap v226: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:33:59.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:33:59 smithi107 ceph-mon[26355]: pgmap v226: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:34:01.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:34:01 smithi157 ceph-mon[33877]: pgmap v227: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:34:01.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:34:01 smithi107 ceph-mon[26355]: pgmap v227: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:34:03.383 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:34:03 smithi157 ceph-mon[33877]: pgmap v228: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:34:03.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:34:03 smithi107 ceph-mon[26355]: pgmap v228: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:34:04.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:34:04 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:34:04.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:34:04 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:34:05.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:34:05 smithi157 ceph-mon[33877]: pgmap v229: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:34:05.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:34:05 smithi107 ceph-mon[26355]: pgmap v229: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:34:07.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:34:07 smithi157 ceph-mon[33877]: pgmap v230: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:34:07.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:34:07 smithi107 ceph-mon[26355]: pgmap v230: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:34:09.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:34:09 smithi157 ceph-mon[33877]: pgmap v231: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:34:09.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:34:09 smithi107 ceph-mon[26355]: pgmap v231: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:34:11.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:34:11 smithi157 ceph-mon[33877]: pgmap v232: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:34:11.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:34:11 smithi107 ceph-mon[26355]: pgmap v232: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:34:13.383 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:34:13 smithi157 ceph-mon[33877]: pgmap v233: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:34:13.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:34:13 smithi107 ceph-mon[26355]: pgmap v233: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:34:15.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:34:15 smithi157 ceph-mon[33877]: pgmap v234: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:34:15.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:34:15 smithi107 ceph-mon[26355]: pgmap v234: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:34:17.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:34:17 smithi157 ceph-mon[33877]: pgmap v235: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:34:17.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:34:17 smithi107 ceph-mon[26355]: pgmap v235: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:34:19.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:34:19 smithi157 ceph-mon[33877]: pgmap v236: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:34:19.383 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:34:19 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:34:19.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:34:19 smithi107 ceph-mon[26355]: pgmap v236: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:34:19.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:34:19 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:34:20.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.472+0000 7f4ec0ec8700 1 -- 172.21.15.107:0/1346034458 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ebc0fed60 msgr2=0x7f4ebc0faea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:34:20.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.473+0000 7f4ec0ec8700 1 --2- 172.21.15.107:0/1346034458 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ebc0fed60 0x7f4ebc0faea0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0004940 tx=0x7f4eb0009e30 comp rx=0 tx=0).stop 2024-04-22T23:34:20.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.473+0000 7f4ec0ec8700 1 -- 172.21.15.107:0/1346034458 shutdown_connections 2024-04-22T23:34:20.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.473+0000 7f4ec0ec8700 1 --2- 172.21.15.107:0/1346034458 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ebc0fed60 0x7f4ebc0faea0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:20.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.473+0000 7f4ec0ec8700 1 --2- 172.21.15.107:0/1346034458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ebc0fe410 0x7f4ebc0fe820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:20.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.473+0000 7f4ec0ec8700 1 -- 172.21.15.107:0/1346034458 >> 172.21.15.107:0/1346034458 conn(0x7f4ebc0f6bd0 msgr2=0x7f4ebc0f8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:34:20.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.473+0000 7f4ec0ec8700 1 -- 172.21.15.107:0/1346034458 shutdown_connections 2024-04-22T23:34:20.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.473+0000 7f4ec0ec8700 1 -- 172.21.15.107:0/1346034458 wait complete. 2024-04-22T23:34:20.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.474+0000 7f4ec0ec8700 1 Processor -- start 2024-04-22T23:34:20.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.474+0000 7f4ec0ec8700 1 -- start start 2024-04-22T23:34:20.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.475+0000 7f4ec0ec8700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ebc0fe410 0x7f4ebc10d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:34:20.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.475+0000 7f4ec0ec8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ebc0fed60 0x7f4ebc10dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:34:20.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.475+0000 7f4ec0ec8700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ebc10e290 con 0x7f4ebc0fed60 2024-04-22T23:34:20.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.475+0000 7f4ec0ec8700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ebc10eae0 con 0x7f4ebc0fe410 2024-04-22T23:34:20.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.475+0000 7f4eb9d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ebc0fed60 0x7f4ebc10dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:34:20.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.475+0000 7f4eba59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ebc0fe410 0x7f4ebc10d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:34:20.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.475+0000 7f4eba59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ebc0fe410 0x7f4ebc10d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.107:33862/0 (socket says 172.21.15.107:33862) 2024-04-22T23:34:20.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.475+0000 7f4eba59c700 1 -- 172.21.15.107:0/2171209865 learned_addr learned my addr 172.21.15.107:0/2171209865 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:34:20.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.475+0000 7f4eba59c700 1 -- 172.21.15.107:0/2171209865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ebc0fed60 msgr2=0x7f4ebc10dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:34:20.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.475+0000 7f4eba59c700 1 --2- 172.21.15.107:0/2171209865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ebc0fed60 0x7f4ebc10dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:20.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.475+0000 7f4eba59c700 1 -- 172.21.15.107:0/2171209865 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb0005040 con 0x7f4ebc0fe410 2024-04-22T23:34:20.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.475+0000 7f4eb9d9b700 1 --2- 172.21.15.107:0/2171209865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ebc0fed60 0x7f4ebc10dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T23:34:20.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.476+0000 7f4eba59c700 1 --2- 172.21.15.107:0/2171209865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ebc0fe410 0x7f4ebc10d750 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4eac00eb40 tx=0x7f4eac00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:34:20.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.476+0000 7f4eaaffd700 1 -- 172.21.15.107:0/2171209865 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4eac007cc0 con 0x7f4ebc0fe410 2024-04-22T23:34:20.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.476+0000 7f4ec0ec8700 1 -- 172.21.15.107:0/2171209865 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ebc10ed60 con 0x7f4ebc0fe410 2024-04-22T23:34:20.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.476+0000 7f4ec0ec8700 1 -- 172.21.15.107:0/2171209865 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ebc10f2a0 con 0x7f4ebc0fe410 2024-04-22T23:34:20.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.476+0000 7f4eaaffd700 1 -- 172.21.15.107:0/2171209865 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4eac00f040 con 0x7f4ebc0fe410 2024-04-22T23:34:20.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.476+0000 7f4eaaffd700 1 -- 172.21.15.107:0/2171209865 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4eac00ca00 con 0x7f4ebc0fe410 2024-04-22T23:34:20.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.477+0000 7f4ea8ff9700 1 -- 172.21.15.107:0/2171209865 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ea00052b0 con 0x7f4ebc0fe410 2024-04-22T23:34:20.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.478+0000 7f4eaaffd700 1 -- 172.21.15.107:0/2171209865 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f4eac00cb60 con 0x7f4ebc0fe410 2024-04-22T23:34:20.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.478+0000 7f4eaaffd700 1 --2- 172.21.15.107:0/2171209865 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f4ea40692f0 0x7f4ea406b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:34:20.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.478+0000 7f4eaaffd700 1 -- 172.21.15.107:0/2171209865 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f4eac086ce0 con 0x7f4ebc0fe410 2024-04-22T23:34:20.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.479+0000 7f4eb9d9b700 1 --2- 172.21.15.107:0/2171209865 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f4ea40692f0 0x7f4ea406b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:34:20.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.479+0000 7f4eb9d9b700 1 --2- 172.21.15.107:0/2171209865 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f4ea40692f0 0x7f4ea406b7a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0000c00 tx=0x7f4eb0009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:34:20.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.486+0000 7f4eaaffd700 1 -- 172.21.15.107:0/2171209865 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4eac04e250 con 0x7f4ebc0fe410 2024-04-22T23:34:20.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.769+0000 7f4ea8ff9700 1 -- 172.21.15.107:0/2171209865 --> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ea0000bc0 con 0x7f4ea40692f0 2024-04-22T23:34:20.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.771+0000 7f4eaaffd700 1 -- 172.21.15.107:0/2171209865 <== mgr.14206 v2:172.21.15.107:6800/2516490111 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f4ea0000bc0 con 0x7f4ea40692f0 2024-04-22T23:34:20.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.774+0000 7f4ea8ff9700 1 -- 172.21.15.107:0/2171209865 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f4ea40692f0 msgr2=0x7f4ea406b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:34:20.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.774+0000 7f4ea8ff9700 1 --2- 172.21.15.107:0/2171209865 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f4ea40692f0 0x7f4ea406b7a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0000c00 tx=0x7f4eb0009e30 comp rx=0 tx=0).stop 2024-04-22T23:34:20.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.774+0000 7f4ea8ff9700 1 -- 172.21.15.107:0/2171209865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ebc0fe410 msgr2=0x7f4ebc10d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:34:20.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.774+0000 7f4ea8ff9700 1 --2- 172.21.15.107:0/2171209865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ebc0fe410 0x7f4ebc10d750 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4eac00eb40 tx=0x7f4eac00b040 comp rx=0 tx=0).stop 2024-04-22T23:34:20.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.774+0000 7f4ea8ff9700 1 -- 172.21.15.107:0/2171209865 shutdown_connections 2024-04-22T23:34:20.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.774+0000 7f4ea8ff9700 1 --2- 172.21.15.107:0/2171209865 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f4ea40692f0 0x7f4ea406b7a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:20.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.774+0000 7f4ea8ff9700 1 --2- 172.21.15.107:0/2171209865 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ebc0fe410 0x7f4ebc10d750 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:20.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.774+0000 7f4ea8ff9700 1 --2- 172.21.15.107:0/2171209865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ebc0fed60 0x7f4ebc10dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:20.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.774+0000 7f4ea8ff9700 1 -- 172.21.15.107:0/2171209865 >> 172.21.15.107:0/2171209865 conn(0x7f4ebc0f6bd0 msgr2=0x7f4ebc0f7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:34:20.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.774+0000 7f4ea8ff9700 1 -- 172.21.15.107:0/2171209865 shutdown_connections 2024-04-22T23:34:20.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.774+0000 7f4ea8ff9700 1 -- 172.21.15.107:0/2171209865 wait complete. 2024-04-22T23:34:20.785 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-04-22T23:34:20.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.934+0000 7f8fcba5e700 1 -- 172.21.15.107:0/14054646 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fc40fce50 msgr2=0x7f8fc40fd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:34:20.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.934+0000 7f8fcba5e700 1 --2- 172.21.15.107:0/14054646 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fc40fce50 0x7f8fc40fd260 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0004940 tx=0x7f8fc0009e30 comp rx=0 tx=0).stop 2024-04-22T23:34:20.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.934+0000 7f8fcba5e700 1 -- 172.21.15.107:0/14054646 shutdown_connections 2024-04-22T23:34:20.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.934+0000 7f8fcba5e700 1 --2- 172.21.15.107:0/14054646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fc40fe050 0x7f8fc40fe4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:20.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.934+0000 7f8fcba5e700 1 --2- 172.21.15.107:0/14054646 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fc40fce50 0x7f8fc40fd260 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:20.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.934+0000 7f8fcba5e700 1 -- 172.21.15.107:0/14054646 >> 172.21.15.107:0/14054646 conn(0x7f8fc40f8540 msgr2=0x7f8fc40fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:34:20.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.934+0000 7f8fcba5e700 1 -- 172.21.15.107:0/14054646 shutdown_connections 2024-04-22T23:34:20.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.934+0000 7f8fcba5e700 1 -- 172.21.15.107:0/14054646 wait complete. 2024-04-22T23:34:20.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.935+0000 7f8fcba5e700 1 Processor -- start 2024-04-22T23:34:20.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.935+0000 7f8fcba5e700 1 -- start start 2024-04-22T23:34:20.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.935+0000 7f8fcba5e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fc40fce50 0x7f8fc410b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:34:20.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.935+0000 7f8fcba5e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fc40fe050 0x7f8fc410ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:34:20.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.935+0000 7f8fcba5e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc410c090 con 0x7f8fc40fe050 2024-04-22T23:34:20.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.935+0000 7f8fcba5e700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc410c8e0 con 0x7f8fc40fce50 2024-04-22T23:34:20.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.936+0000 7f8fc97fa700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fc40fce50 0x7f8fc410b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:34:20.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.936+0000 7f8fc8ff9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fc40fe050 0x7f8fc410ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:34:20.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.936+0000 7f8fc8ff9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fc40fe050 0x7f8fc410ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55714/0 (socket says 172.21.15.107:55714) 2024-04-22T23:34:20.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.936+0000 7f8fc8ff9700 1 -- 172.21.15.107:0/3213275382 learned_addr learned my addr 172.21.15.107:0/3213275382 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:34:20.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.936+0000 7f8fc8ff9700 1 -- 172.21.15.107:0/3213275382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fc40fce50 msgr2=0x7f8fc410b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:34:20.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.936+0000 7f8fc8ff9700 1 --2- 172.21.15.107:0/3213275382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fc40fce50 0x7f8fc410b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:20.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.936+0000 7f8fc8ff9700 1 -- 172.21.15.107:0/3213275382 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc0005040 con 0x7f8fc40fe050 2024-04-22T23:34:20.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.936+0000 7f8fc8ff9700 1 --2- 172.21.15.107:0/3213275382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fc40fe050 0x7f8fc410ba40 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8fb400eb40 tx=0x7f8fb400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:34:20.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.937+0000 7f8fb9ffb700 1 -- 172.21.15.107:0/3213275382 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8fb4007cc0 con 0x7f8fc40fe050 2024-04-22T23:34:20.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.937+0000 7f8fb9ffb700 1 -- 172.21.15.107:0/3213275382 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8fb400f040 con 0x7f8fc40fe050 2024-04-22T23:34:20.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.937+0000 7f8fb9ffb700 1 -- 172.21.15.107:0/3213275382 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8fb400ca00 con 0x7f8fc40fe050 2024-04-22T23:34:20.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.937+0000 7f8fcba5e700 1 -- 172.21.15.107:0/3213275382 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fc410cb60 con 0x7f8fc40fe050 2024-04-22T23:34:20.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.937+0000 7f8fcba5e700 1 -- 172.21.15.107:0/3213275382 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fc410d0a0 con 0x7f8fc40fe050 2024-04-22T23:34:20.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.939+0000 7f8fcba5e700 1 -- 172.21.15.107:0/3213275382 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fc41019e0 con 0x7f8fc40fe050 2024-04-22T23:34:20.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.940+0000 7f8fb9ffb700 1 -- 172.21.15.107:0/3213275382 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f8fb400cb60 con 0x7f8fc40fe050 2024-04-22T23:34:20.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.940+0000 7f8fb9ffb700 1 --2- 172.21.15.107:0/3213275382 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f8fb00692a0 0x7f8fb006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:34:20.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.940+0000 7f8fb9ffb700 1 -- 172.21.15.107:0/3213275382 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f8fb4086c50 con 0x7f8fc40fe050 2024-04-22T23:34:20.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.940+0000 7f8fc97fa700 1 --2- 172.21.15.107:0/3213275382 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f8fb00692a0 0x7f8fb006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:34:20.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.941+0000 7f8fc97fa700 1 --2- 172.21.15.107:0/3213275382 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f8fb00692a0 0x7f8fb006b750 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8fc00136a0 tx=0x7f8fc0009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:34:20.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:20.949+0000 7f8fb9ffb700 1 -- 172.21.15.107:0/3213275382 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8fb4052e90 con 0x7f8fc40fe050 2024-04-22T23:34:21.112 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:34:21 smithi107 ceph-mon[26355]: pgmap v237: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:34:21.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.224+0000 7f8fcba5e700 1 -- 172.21.15.107:0/3213275382 --> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8fc40008d0 con 0x7f8fb00692a0 2024-04-22T23:34:21.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.226+0000 7f8fb9ffb700 1 -- 172.21.15.107:0/3213275382 <== mgr.14206 v2:172.21.15.107:6800/2516490111 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f8fc40008d0 con 0x7f8fb00692a0 2024-04-22T23:34:21.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.228+0000 7f8fcba5e700 1 -- 172.21.15.107:0/3213275382 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f8fb00692a0 msgr2=0x7f8fb006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:34:21.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.228+0000 7f8fcba5e700 1 --2- 172.21.15.107:0/3213275382 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f8fb00692a0 0x7f8fb006b750 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8fc00136a0 tx=0x7f8fc0009e30 comp rx=0 tx=0).stop 2024-04-22T23:34:21.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.228+0000 7f8fcba5e700 1 -- 172.21.15.107:0/3213275382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fc40fe050 msgr2=0x7f8fc410ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:34:21.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.228+0000 7f8fcba5e700 1 --2- 172.21.15.107:0/3213275382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fc40fe050 0x7f8fc410ba40 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8fb400eb40 tx=0x7f8fb400b040 comp rx=0 tx=0).stop 2024-04-22T23:34:21.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.228+0000 7f8fcba5e700 1 -- 172.21.15.107:0/3213275382 shutdown_connections 2024-04-22T23:34:21.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.228+0000 7f8fcba5e700 1 --2- 172.21.15.107:0/3213275382 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f8fb00692a0 0x7f8fb006b750 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:21.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.228+0000 7f8fcba5e700 1 --2- 172.21.15.107:0/3213275382 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8fc40fce50 0x7f8fc410b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:21.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.228+0000 7f8fcba5e700 1 --2- 172.21.15.107:0/3213275382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fc40fe050 0x7f8fc410ba40 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:21.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.228+0000 7f8fcba5e700 1 -- 172.21.15.107:0/3213275382 >> 172.21.15.107:0/3213275382 conn(0x7f8fc40f8540 msgr2=0x7f8fc40f9b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:34:21.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.228+0000 7f8fcba5e700 1 -- 172.21.15.107:0/3213275382 shutdown_connections 2024-04-22T23:34:21.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.229+0000 7f8fcba5e700 1 -- 172.21.15.107:0/3213275382 wait complete. 2024-04-22T23:34:21.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.348+0000 7f1532ef1700 1 -- 172.21.15.107:0/3968976071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f152c0fbd40 msgr2=0x7f152c0fc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:34:21.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.348+0000 7f1532ef1700 1 --2- 172.21.15.107:0/3968976071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f152c0fbd40 0x7f152c0fc150 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1520004940 tx=0x7f1520009e30 comp rx=0 tx=0).stop 2024-04-22T23:34:21.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.349+0000 7f1532ef1700 1 -- 172.21.15.107:0/3968976071 shutdown_connections 2024-04-22T23:34:21.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.349+0000 7f1532ef1700 1 --2- 172.21.15.107:0/3968976071 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f152c0fcff0 0x7f152c0fd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:21.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.349+0000 7f1532ef1700 1 --2- 172.21.15.107:0/3968976071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f152c0fbd40 0x7f152c0fc150 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:21.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.349+0000 7f1532ef1700 1 -- 172.21.15.107:0/3968976071 >> 172.21.15.107:0/3968976071 conn(0x7f152c0f74c0 msgr2=0x7f152c0f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:34:21.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.349+0000 7f1532ef1700 1 -- 172.21.15.107:0/3968976071 shutdown_connections 2024-04-22T23:34:21.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.349+0000 7f1532ef1700 1 -- 172.21.15.107:0/3968976071 wait complete. 2024-04-22T23:34:21.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.349+0000 7f1532ef1700 1 Processor -- start 2024-04-22T23:34:21.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.349+0000 7f1532ef1700 1 -- start start 2024-04-22T23:34:21.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.350+0000 7f1532ef1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f152c0fbd40 0x7f152c10e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:34:21.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.350+0000 7f1532ef1700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f152c0fcff0 0x7f152c110ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:34:21.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.350+0000 7f1532ef1700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152c066450 con 0x7f152c0fbd40 2024-04-22T23:34:21.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.350+0000 7f1532ef1700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152c066590 con 0x7f152c0fcff0 2024-04-22T23:34:21.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.350+0000 7f1530c8d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f152c0fbd40 0x7f152c10e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:34:21.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.350+0000 7f1530c8d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f152c0fbd40 0x7f152c10e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55718/0 (socket says 172.21.15.107:55718) 2024-04-22T23:34:21.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.350+0000 7f1530c8d700 1 -- 172.21.15.107:0/3914104459 learned_addr learned my addr 172.21.15.107:0/3914104459 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:34:21.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.350+0000 7f152bfff700 1 --2- 172.21.15.107:0/3914104459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f152c0fcff0 0x7f152c110ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:34:21.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.350+0000 7f1530c8d700 1 -- 172.21.15.107:0/3914104459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f152c0fcff0 msgr2=0x7f152c110ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:34:21.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.351+0000 7f1530c8d700 1 --2- 172.21.15.107:0/3914104459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f152c0fcff0 0x7f152c110ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:21.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.351+0000 7f1530c8d700 1 -- 172.21.15.107:0/3914104459 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1520005040 con 0x7f152c0fbd40 2024-04-22T23:34:21.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.351+0000 7f152bfff700 1 --2- 172.21.15.107:0/3914104459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f152c0fcff0 0x7f152c110ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T23:34:21.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.351+0000 7f1530c8d700 1 --2- 172.21.15.107:0/3914104459 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f152c0fbd40 0x7f152c10e570 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1520009040 tx=0x7f1520009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:34:21.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.351+0000 7f15297fa700 1 -- 172.21.15.107:0/3914104459 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f152000e070 con 0x7f152c0fbd40 2024-04-22T23:34:21.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.352+0000 7f1532ef1700 1 -- 172.21.15.107:0/3914104459 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f152c1110f0 con 0x7f152c0fbd40 2024-04-22T23:34:21.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.352+0000 7f15297fa700 1 -- 172.21.15.107:0/3914104459 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1520013420 con 0x7f152c0fbd40 2024-04-22T23:34:21.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.352+0000 7f1532ef1700 1 -- 172.21.15.107:0/3914104459 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f152c111650 con 0x7f152c0fbd40 2024-04-22T23:34:21.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.352+0000 7f15297fa700 1 -- 172.21.15.107:0/3914104459 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f152000e070 con 0x7f152c0fbd40 2024-04-22T23:34:21.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.354+0000 7f1532ef1700 1 -- 172.21.15.107:0/3914104459 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15100052e0 con 0x7f152c0fbd40 2024-04-22T23:34:21.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.355+0000 7f15297fa700 1 -- 172.21.15.107:0/3914104459 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f1520013a50 con 0x7f152c0fbd40 2024-04-22T23:34:21.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.355+0000 7f15297fa700 1 --2- 172.21.15.107:0/3914104459 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f151c069310 0x7f151c06b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:34:21.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.356+0000 7f15297fa700 1 -- 172.21.15.107:0/3914104459 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f152005c400 con 0x7f152c0fbd40 2024-04-22T23:34:21.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.356+0000 7f152bfff700 1 --2- 172.21.15.107:0/3914104459 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f151c069310 0x7f151c06b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:34:21.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.357+0000 7f152bfff700 1 --2- 172.21.15.107:0/3914104459 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f151c069310 0x7f151c06b7c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1518006fd0 tx=0x7f15180090f0 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:34:21.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.363+0000 7f15297fa700 1 -- 172.21.15.107:0/3914104459 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f1520055c80 con 0x7f152c0fbd40 2024-04-22T23:34:21.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:34:21 smithi157 ceph-mon[33877]: pgmap v237: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:34:21.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.766+0000 7f1532ef1700 1 -- 172.21.15.107:0/3914104459 --> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1510000bf0 con 0x7f151c069310 2024-04-22T23:34:21.782 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-22T23:34:21.782 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (6m) 90s ago 7m 25.5M - 0.25.0 c8568f914cd2 4d90ceb89249 2024-04-22T23:34:21.782 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (7m) 90s ago 7m 6660k - 17.2.7-884-g70d0cb4b cafd8ddac8fa c10611c756e4 2024-04-22T23:34:21.782 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi157 smithi157 running (6m) 96s ago 6m 6669k - 17.2.7-884-g70d0cb4b cafd8ddac8fa 55d2d46294c9 2024-04-22T23:34:21.782 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (5m) 90s ago 7m 86.8M - 9.4.7 954c08fa6188 c2d1f8de1aa7 2024-04-22T23:34:21.782 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.dfvvkw smithi107 running (118s) 90s ago 117s 10.4M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 08d3aa8d717d 2024-04-22T23:34:21.783 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.ktuxjl smithi107 running (111s) 90s ago 110s 16.9M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 762f27fe0f30 2024-04-22T23:34:21.783 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi157.rcaris smithi157 running (115s) 96s ago 114s 16.8M - 17.2.7-884-g70d0cb4b cafd8ddac8fa d23366ac085d 2024-04-22T23:34:21.783 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi157.ufywse smithi157 running (108s) 96s ago 108s 11.0M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 8b00d61eb355 2024-04-22T23:34:21.783 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.uipxvm smithi107 *:9283 running (8m) 90s ago 8m 481M - 17.2.7-884-g70d0cb4b cafd8ddac8fa db96e8cbbdfb 2024-04-22T23:34:21.783 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi157.tuylis smithi157 *:8443,9283 running (6m) 96s ago 6m 431M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 0c12be10ffaf 2024-04-22T23:34:21.783 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (8m) 90s ago 8m 49.6M 2048M 17.2.7-884-g70d0cb4b cafd8ddac8fa 8314962fa1b9 2024-04-22T23:34:21.783 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi157 smithi157 running (6m) 96s ago 6m 36.6M 2048M 17.2.7-884-g70d0cb4b cafd8ddac8fa 0747a85008d9 2024-04-22T23:34:21.783 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (7m) 90s ago 7m 16.0M - 1.5.0 0da6a335fe13 8a7c595c114f 2024-04-22T23:34:21.783 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi157 smithi157 *:9100 running (6m) 96s ago 6m 16.1M - 1.5.0 0da6a335fe13 940164771fe3 2024-04-22T23:34:21.783 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (5m) 90s ago 5m 46.6M 4288M 17.2.7-884-g70d0cb4b cafd8ddac8fa 18a179b0a618 2024-04-22T23:34:21.783 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (5m) 90s ago 5m 45.9M 4288M 17.2.7-884-g70d0cb4b cafd8ddac8fa c7272eb84ddb 2024-04-22T23:34:21.783 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (4m) 90s ago 4m 46.2M 4288M 17.2.7-884-g70d0cb4b cafd8ddac8fa 9db1788096ad 2024-04-22T23:34:21.783 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi157 running (4m) 96s ago 4m 43.9M 5312M 17.2.7-884-g70d0cb4b cafd8ddac8fa c9fb2d1a13dd 2024-04-22T23:34:21.783 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi157 running (3m) 96s ago 3m 44.2M 5312M 17.2.7-884-g70d0cb4b cafd8ddac8fa 24a16e19951f 2024-04-22T23:34:21.783 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi157 running (3m) 96s ago 3m 43.1M 5312M 17.2.7-884-g70d0cb4b cafd8ddac8fa 4c5b7a7ef8bd 2024-04-22T23:34:21.784 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (5m) 90s ago 6m 51.2M - 2.43.0 a07b618ecd1d 8c5df40caabe 2024-04-22T23:34:21.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.785+0000 7f15297fa700 1 -- 172.21.15.107:0/3914104459 <== mgr.14206 v2:172.21.15.107:6800/2516490111 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3410 (secure 0 0 0) 0x7f1510000bf0 con 0x7f151c069310 2024-04-22T23:34:21.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.787+0000 7f1532ef1700 1 -- 172.21.15.107:0/3914104459 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f151c069310 msgr2=0x7f151c06b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:34:21.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.787+0000 7f1532ef1700 1 --2- 172.21.15.107:0/3914104459 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f151c069310 0x7f151c06b7c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1518006fd0 tx=0x7f15180090f0 comp rx=0 tx=0).stop 2024-04-22T23:34:21.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.787+0000 7f1532ef1700 1 -- 172.21.15.107:0/3914104459 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f152c0fbd40 msgr2=0x7f152c10e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:34:21.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.787+0000 7f1532ef1700 1 --2- 172.21.15.107:0/3914104459 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f152c0fbd40 0x7f152c10e570 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1520009040 tx=0x7f1520009e30 comp rx=0 tx=0).stop 2024-04-22T23:34:21.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.788+0000 7f1532ef1700 1 -- 172.21.15.107:0/3914104459 shutdown_connections 2024-04-22T23:34:21.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.788+0000 7f1532ef1700 1 --2- 172.21.15.107:0/3914104459 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f151c069310 0x7f151c06b7c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:21.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.788+0000 7f1532ef1700 1 --2- 172.21.15.107:0/3914104459 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f152c0fbd40 0x7f152c10e570 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:21.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.788+0000 7f1532ef1700 1 --2- 172.21.15.107:0/3914104459 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f152c0fcff0 0x7f152c110ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:21.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.788+0000 7f1532ef1700 1 -- 172.21.15.107:0/3914104459 >> 172.21.15.107:0/3914104459 conn(0x7f152c0f74c0 msgr2=0x7f152c0f8bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:34:21.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.788+0000 7f1532ef1700 1 -- 172.21.15.107:0/3914104459 shutdown_connections 2024-04-22T23:34:21.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.788+0000 7f1532ef1700 1 -- 172.21.15.107:0/3914104459 wait complete. 2024-04-22T23:34:21.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.949+0000 7fb719cd6700 1 -- 172.21.15.107:0/373422378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7141046a0 msgr2=0x7fb714104b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:34:21.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.949+0000 7fb719cd6700 1 --2- 172.21.15.107:0/373422378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7141046a0 0x7fb714104b10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb708004940 tx=0x7fb708009e30 comp rx=0 tx=0).stop 2024-04-22T23:34:21.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.949+0000 7fb719cd6700 1 -- 172.21.15.107:0/373422378 shutdown_connections 2024-04-22T23:34:21.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.949+0000 7fb719cd6700 1 --2- 172.21.15.107:0/373422378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7141046a0 0x7fb714104b10 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:21.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.949+0000 7fb719cd6700 1 --2- 172.21.15.107:0/373422378 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7141033f0 0x7fb714103800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:21.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.949+0000 7fb719cd6700 1 -- 172.21.15.107:0/373422378 >> 172.21.15.107:0/373422378 conn(0x7fb714072640 msgr2=0x7fb714074a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:34:21.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.949+0000 7fb719cd6700 1 -- 172.21.15.107:0/373422378 shutdown_connections 2024-04-22T23:34:21.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.950+0000 7fb719cd6700 1 -- 172.21.15.107:0/373422378 wait complete. 2024-04-22T23:34:21.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.950+0000 7fb719cd6700 1 Processor -- start 2024-04-22T23:34:21.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.953+0000 7fb719cd6700 1 -- start start 2024-04-22T23:34:21.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.953+0000 7fb719cd6700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7141033f0 0x7fb71406fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:34:21.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.953+0000 7fb719cd6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7141046a0 0x7fb71406e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:34:21.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.953+0000 7fb719cd6700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb71406e7f0 con 0x7fb7141033f0 2024-04-22T23:34:21.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.953+0000 7fb719cd6700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb71406e960 con 0x7fb7141046a0 2024-04-22T23:34:21.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.953+0000 7fb7137fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7141033f0 0x7fb71406fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:34:21.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.953+0000 7fb7137fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7141033f0 0x7fb71406fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38444/0 (socket says 172.21.15.107:38444) 2024-04-22T23:34:21.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.953+0000 7fb7137fe700 1 -- 172.21.15.107:0/3098985527 learned_addr learned my addr 172.21.15.107:0/3098985527 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:34:21.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.953+0000 7fb712ffd700 1 --2- 172.21.15.107:0/3098985527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7141046a0 0x7fb71406e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:34:21.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.954+0000 7fb7137fe700 1 -- 172.21.15.107:0/3098985527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7141046a0 msgr2=0x7fb71406e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:34:21.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.954+0000 7fb7137fe700 1 --2- 172.21.15.107:0/3098985527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7141046a0 0x7fb71406e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:21.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.954+0000 7fb7137fe700 1 -- 172.21.15.107:0/3098985527 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb708005040 con 0x7fb7141033f0 2024-04-22T23:34:21.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.954+0000 7fb7137fe700 1 --2- 172.21.15.107:0/3098985527 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7141033f0 0x7fb71406fc80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb70400eb40 tx=0x7fb70400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:34:21.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.955+0000 7fb6fbfff700 1 -- 172.21.15.107:0/3098985527 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb704007cc0 con 0x7fb7141033f0 2024-04-22T23:34:21.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.955+0000 7fb719cd6700 1 -- 172.21.15.107:0/3098985527 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb71406ebe0 con 0x7fb7141033f0 2024-04-22T23:34:21.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.955+0000 7fb719cd6700 1 -- 172.21.15.107:0/3098985527 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb714122350 con 0x7fb7141033f0 2024-04-22T23:34:21.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.955+0000 7fb6fbfff700 1 -- 172.21.15.107:0/3098985527 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb70400f040 con 0x7fb7141033f0 2024-04-22T23:34:21.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.955+0000 7fb6fbfff700 1 -- 172.21.15.107:0/3098985527 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb70400ca10 con 0x7fb7141033f0 2024-04-22T23:34:21.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.958+0000 7fb6fbfff700 1 -- 172.21.15.107:0/3098985527 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fb70400cb70 con 0x7fb7141033f0 2024-04-22T23:34:21.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.958+0000 7fb719cd6700 1 -- 172.21.15.107:0/3098985527 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7000052e0 con 0x7fb7141033f0 2024-04-22T23:34:21.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.958+0000 7fb6fbfff700 1 --2- 172.21.15.107:0/3098985527 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fb6fc0692b0 0x7fb6fc06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:34:21.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.958+0000 7fb6fbfff700 1 -- 172.21.15.107:0/3098985527 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fb704086db0 con 0x7fb7141033f0 2024-04-22T23:34:21.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.959+0000 7fb712ffd700 1 --2- 172.21.15.107:0/3098985527 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fb6fc0692b0 0x7fb6fc06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:34:21.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.959+0000 7fb712ffd700 1 --2- 172.21.15.107:0/3098985527 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fb6fc0692b0 0x7fb6fc06b760 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb708013870 tx=0x7fb708009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:34:21.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:21.967+0000 7fb6fbfff700 1 -- 172.21.15.107:0/3098985527 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb70404e250 con 0x7fb7141033f0 2024-04-22T23:34:22.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:34:22 smithi157 ceph-mon[33877]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:34:22.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:34:22 smithi157 ceph-mon[33877]: from='client.14612 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:34:22.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.428+0000 7fb719cd6700 1 -- 172.21.15.107:0/3098985527 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb700005da0 con 0x7fb7141033f0 2024-04-22T23:34:22.424 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:34:22 smithi107 ceph-mon[26355]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:34:22.425 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:34:22 smithi107 ceph-mon[26355]: from='client.14612 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:34:22.425 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-22T23:34:22.425 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-04-22T23:34:22.426 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 2 2024-04-22T23:34:22.426 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:34:22.426 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-04-22T23:34:22.426 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 2 2024-04-22T23:34:22.426 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:34:22.426 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-04-22T23:34:22.426 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 6 2024-04-22T23:34:22.426 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:34:22.426 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-04-22T23:34:22.426 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 4 2024-04-22T23:34:22.426 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:34:22.426 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-04-22T23:34:22.426 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 14 2024-04-22T23:34:22.427 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T23:34:22.427 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-22T23:34:22.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.428+0000 7fb6fbfff700 1 -- 172.21.15.107:0/3098985527 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7fb704056b30 con 0x7fb7141033f0 2024-04-22T23:34:22.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.431+0000 7fb719cd6700 1 -- 172.21.15.107:0/3098985527 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fb6fc0692b0 msgr2=0x7fb6fc06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:34:22.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.431+0000 7fb719cd6700 1 --2- 172.21.15.107:0/3098985527 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fb6fc0692b0 0x7fb6fc06b760 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb708013870 tx=0x7fb708009e30 comp rx=0 tx=0).stop 2024-04-22T23:34:22.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.431+0000 7fb719cd6700 1 -- 172.21.15.107:0/3098985527 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7141033f0 msgr2=0x7fb71406fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:34:22.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.431+0000 7fb719cd6700 1 --2- 172.21.15.107:0/3098985527 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7141033f0 0x7fb71406fc80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb70400eb40 tx=0x7fb70400b040 comp rx=0 tx=0).stop 2024-04-22T23:34:22.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.432+0000 7fb719cd6700 1 -- 172.21.15.107:0/3098985527 shutdown_connections 2024-04-22T23:34:22.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.432+0000 7fb719cd6700 1 --2- 172.21.15.107:0/3098985527 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fb6fc0692b0 0x7fb6fc06b760 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:22.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.432+0000 7fb719cd6700 1 --2- 172.21.15.107:0/3098985527 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7141033f0 0x7fb71406fc80 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:22.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.432+0000 7fb719cd6700 1 --2- 172.21.15.107:0/3098985527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7141046a0 0x7fb71406e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:22.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.432+0000 7fb719cd6700 1 -- 172.21.15.107:0/3098985527 >> 172.21.15.107:0/3098985527 conn(0x7fb714072640 msgr2=0x7fb714074930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:34:22.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.432+0000 7fb719cd6700 1 -- 172.21.15.107:0/3098985527 shutdown_connections 2024-04-22T23:34:22.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.432+0000 7fb719cd6700 1 -- 172.21.15.107:0/3098985527 wait complete. 2024-04-22T23:34:22.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.598+0000 7f1dfa6d9700 1 -- 172.21.15.107:0/13689584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1df40fb490 msgr2=0x7f1df40fb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:34:22.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.598+0000 7f1dfa6d9700 1 --2- 172.21.15.107:0/13689584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1df40fb490 0x7f1df40fb8a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1de4004940 tx=0x7f1de4009e30 comp rx=0 tx=0).stop 2024-04-22T23:34:22.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.599+0000 7f1dfa6d9700 1 -- 172.21.15.107:0/13689584 shutdown_connections 2024-04-22T23:34:22.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.599+0000 7f1dfa6d9700 1 --2- 172.21.15.107:0/13689584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1df40fc740 0x7f1df40fcbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:22.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.599+0000 7f1dfa6d9700 1 --2- 172.21.15.107:0/13689584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1df40fb490 0x7f1df40fb8a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:22.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.599+0000 7f1dfa6d9700 1 -- 172.21.15.107:0/13689584 >> 172.21.15.107:0/13689584 conn(0x7f1df40f6bd0 msgr2=0x7f1df40f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:34:22.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.599+0000 7f1dfa6d9700 1 -- 172.21.15.107:0/13689584 shutdown_connections 2024-04-22T23:34:22.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.599+0000 7f1dfa6d9700 1 -- 172.21.15.107:0/13689584 wait complete. 2024-04-22T23:34:22.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.599+0000 7f1dfa6d9700 1 Processor -- start 2024-04-22T23:34:22.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.600+0000 7f1dfa6d9700 1 -- start start 2024-04-22T23:34:22.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.600+0000 7f1dfa6d9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1df40fc740 0x7f1df410b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:34:22.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.600+0000 7f1dfa6d9700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1df410bd70 0x7f1df4110da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:34:22.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.600+0000 7f1dfa6d9700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df410c230 con 0x7f1df40fc740 2024-04-22T23:34:22.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.600+0000 7f1dfa6d9700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df410c370 con 0x7f1df410bd70 2024-04-22T23:34:22.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.600+0000 7f1df3fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1df40fc740 0x7f1df410b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:34:22.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.600+0000 7f1df37fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1df410bd70 0x7f1df4110da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:34:22.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.600+0000 7f1df37fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1df410bd70 0x7f1df4110da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.107:41186/0 (socket says 172.21.15.107:41186) 2024-04-22T23:34:22.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.600+0000 7f1df37fe700 1 -- 172.21.15.107:0/1660929674 learned_addr learned my addr 172.21.15.107:0/1660929674 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:34:22.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.601+0000 7f1df37fe700 1 -- 172.21.15.107:0/1660929674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1df40fc740 msgr2=0x7f1df410b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:34:22.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.601+0000 7f1df37fe700 1 --2- 172.21.15.107:0/1660929674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1df40fc740 0x7f1df410b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:22.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.601+0000 7f1df37fe700 1 -- 172.21.15.107:0/1660929674 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1de4005040 con 0x7f1df410bd70 2024-04-22T23:34:22.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.601+0000 7f1df37fe700 1 --2- 172.21.15.107:0/1660929674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1df410bd70 0x7f1df4110da0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1de800a6c0 tx=0x7f1de800c040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:34:22.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.602+0000 7f1df0ff9700 1 -- 172.21.15.107:0/1660929674 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1de800ab50 con 0x7f1df410bd70 2024-04-22T23:34:22.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.602+0000 7f1df0ff9700 1 -- 172.21.15.107:0/1660929674 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1de8010040 con 0x7f1df410bd70 2024-04-22T23:34:22.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.602+0000 7f1df0ff9700 1 -- 172.21.15.107:0/1660929674 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1de800e9c0 con 0x7f1df410bd70 2024-04-22T23:34:22.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.602+0000 7f1dfa6d9700 1 -- 172.21.15.107:0/1660929674 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1df41112e0 con 0x7f1df410bd70 2024-04-22T23:34:22.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.602+0000 7f1dfa6d9700 1 -- 172.21.15.107:0/1660929674 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1df4122920 con 0x7f1df410bd70 2024-04-22T23:34:22.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.603+0000 7f1dfa6d9700 1 -- 172.21.15.107:0/1660929674 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1df404c710 con 0x7f1df410bd70 2024-04-22T23:34:22.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.604+0000 7f1df0ff9700 1 -- 172.21.15.107:0/1660929674 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f1de80090d0 con 0x7f1df410bd70 2024-04-22T23:34:22.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.605+0000 7f1df0ff9700 1 --2- 172.21.15.107:0/1660929674 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f1ddc069480 0x7f1ddc06b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:34:22.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.605+0000 7f1df0ff9700 1 -- 172.21.15.107:0/1660929674 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f1de80868a0 con 0x7f1df410bd70 2024-04-22T23:34:22.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.605+0000 7f1df3fff700 1 --2- 172.21.15.107:0/1660929674 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f1ddc069480 0x7f1ddc06b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:34:22.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.606+0000 7f1df3fff700 1 --2- 172.21.15.107:0/1660929674 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f1ddc069480 0x7f1ddc06b930 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1de4000c00 tx=0x7f1de4009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:34:22.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.609+0000 7f1df0ff9700 1 -- 172.21.15.107:0/1660929674 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f1de804e250 con 0x7f1df410bd70 2024-04-22T23:34:22.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.990+0000 7f1dfa6d9700 1 -- 172.21.15.107:0/1660929674 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1df4122c20 con 0x7f1df410bd70 2024-04-22T23:34:22.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.990+0000 7f1df0ff9700 1 -- 172.21.15.107:0/1660929674 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1824 (secure 0 0 0) 0x7f1de80566a0 con 0x7f1df410bd70 2024-04-22T23:34:22.989 INFO:teuthology.orchestra.run.smithi107.stdout:e17 2024-04-22T23:34:22.989 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T23:34:22.989 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:34:22.989 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-04-22T23:34:22.989 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:34:22.989 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-04-22T23:34:22.989 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-04-22T23:34:22.989 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 17 2024-04-22T23:34:22.989 INFO:teuthology.orchestra.run.smithi107.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-22T23:34:22.989 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-04-22T23:32:20.146130+0000 2024-04-22T23:34:22.989 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-04-22T23:32:39.774747+0000 2024-04-22T23:34:22.990 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-04-22T23:34:22.990 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-04-22T23:34:22.990 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-04-22T23:34:22.990 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-04-22T23:34:22.990 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-04-22T23:34:22.990 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-04-22T23:34:22.990 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-04-22T23:34:22.990 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 46 2024-04-22T23:34:22.990 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:34:22.990 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 2 2024-04-22T23:34:22.990 INFO:teuthology.orchestra.run.smithi107.stdout:in 0,1 2024-04-22T23:34:22.990 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=14468,1=24265} 2024-04-22T23:34:22.990 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-04-22T23:34:22.990 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-04-22T23:34:22.991 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-04-22T23:34:22.991 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-04-22T23:34:22.991 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-04-22T23:34:22.991 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data disabled 2024-04-22T23:34:22.991 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-04-22T23:34:22.991 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-04-22T23:34:22.991 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.ktuxjl{0:14468} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.107:6828/3335313253,v1:172.21.15.107:6829/3335313253] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:34:22.991 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi157.rcaris{1:24265} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.157:6824/2779251893,v1:172.21.15.157:6825/2779251893] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:34:22.991 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:34:22.991 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:34:22.991 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-04-22T23:34:22.991 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:34:22.991 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.dfvvkw{-1:14484} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/3548717247,v1:172.21.15.107:6827/3548717247] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:34:22.991 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi157.ufywse{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.157:6826/4072444199,v1:172.21.15.157:6827/4072444199] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:34:22.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.994+0000 7f1dfa6d9700 1 -- 172.21.15.107:0/1660929674 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f1ddc069480 msgr2=0x7f1ddc06b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:34:22.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.994+0000 7f1dfa6d9700 1 --2- 172.21.15.107:0/1660929674 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f1ddc069480 0x7f1ddc06b930 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1de4000c00 tx=0x7f1de4009e30 comp rx=0 tx=0).stop 2024-04-22T23:34:22.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.994+0000 7f1dfa6d9700 1 -- 172.21.15.107:0/1660929674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1df410bd70 msgr2=0x7f1df4110da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:34:22.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.994+0000 7f1dfa6d9700 1 --2- 172.21.15.107:0/1660929674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1df410bd70 0x7f1df4110da0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1de800a6c0 tx=0x7f1de800c040 comp rx=0 tx=0).stop 2024-04-22T23:34:22.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.994+0000 7f1dfa6d9700 1 -- 172.21.15.107:0/1660929674 shutdown_connections 2024-04-22T23:34:22.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.994+0000 7f1dfa6d9700 1 --2- 172.21.15.107:0/1660929674 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f1ddc069480 0x7f1ddc06b930 secure :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1de4000c00 tx=0x7f1de4009e30 comp rx=0 tx=0).stop 2024-04-22T23:34:22.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.994+0000 7f1dfa6d9700 1 --2- 172.21.15.107:0/1660929674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1df40fc740 0x7f1df410b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:22.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.994+0000 7f1dfa6d9700 1 --2- 172.21.15.107:0/1660929674 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1df410bd70 0x7f1df4110da0 secure :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1de800a6c0 tx=0x7f1de800c040 comp rx=0 tx=0).stop 2024-04-22T23:34:22.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.994+0000 7f1dfa6d9700 1 -- 172.21.15.107:0/1660929674 >> 172.21.15.107:0/1660929674 conn(0x7f1df40f6bd0 msgr2=0x7f1df40f8260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:34:22.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.995+0000 7f1dfa6d9700 1 -- 172.21.15.107:0/1660929674 shutdown_connections 2024-04-22T23:34:22.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:22.995+0000 7f1dfa6d9700 1 -- 172.21.15.107:0/1660929674 wait complete. 2024-04-22T23:34:22.993 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 17 2024-04-22T23:34:23.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.149+0000 7f924d436700 1 -- 172.21.15.107:0/3924276363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92480662f0 msgr2=0x7f92480ffa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:34:23.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.149+0000 7f924d436700 1 --2- 172.21.15.107:0/3924276363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92480662f0 0x7f92480ffa30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9238004940 tx=0x7f9238009e30 comp rx=0 tx=0).stop 2024-04-22T23:34:23.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.149+0000 7f924d436700 1 -- 172.21.15.107:0/3924276363 shutdown_connections 2024-04-22T23:34:23.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.149+0000 7f924d436700 1 --2- 172.21.15.107:0/3924276363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92480fff70 0x7f9248102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:23.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.149+0000 7f924d436700 1 --2- 172.21.15.107:0/3924276363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92480662f0 0x7f92480ffa30 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:23.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.149+0000 7f924d436700 1 -- 172.21.15.107:0/3924276363 >> 172.21.15.107:0/3924276363 conn(0x7f92480f74c0 msgr2=0x7f92480f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:34:23.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.149+0000 7f924d436700 1 -- 172.21.15.107:0/3924276363 shutdown_connections 2024-04-22T23:34:23.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.150+0000 7f924d436700 1 -- 172.21.15.107:0/3924276363 wait complete. 2024-04-22T23:34:23.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.150+0000 7f924d436700 1 Processor -- start 2024-04-22T23:34:23.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.150+0000 7f924d436700 1 -- start start 2024-04-22T23:34:23.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.151+0000 7f924d436700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92480fff70 0x7f924806e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:34:23.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.151+0000 7f924d436700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f924806eac0 0x7f924806ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:34:23.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.151+0000 7f92467fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f924806eac0 0x7f924806ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:34:23.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.151+0000 7f92467fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f924806eac0 0x7f924806ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38470/0 (socket says 172.21.15.107:38470) 2024-04-22T23:34:23.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.151+0000 7f9246ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92480fff70 0x7f924806e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:34:23.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.151+0000 7f92467fc700 1 -- 172.21.15.107:0/1341204227 learned_addr learned my addr 172.21.15.107:0/1341204227 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:34:23.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.151+0000 7f924d436700 1 -- 172.21.15.107:0/1341204227 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92480700b0 con 0x7f924806eac0 2024-04-22T23:34:23.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.151+0000 7f924d436700 1 -- 172.21.15.107:0/1341204227 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f924810ad10 con 0x7f92480fff70 2024-04-22T23:34:23.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.151+0000 7f92467fc700 1 -- 172.21.15.107:0/1341204227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92480fff70 msgr2=0x7f924806e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:34:23.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.151+0000 7f92467fc700 1 --2- 172.21.15.107:0/1341204227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92480fff70 0x7f924806e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:23.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.151+0000 7f92467fc700 1 -- 172.21.15.107:0/1341204227 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9238005040 con 0x7f924806eac0 2024-04-22T23:34:23.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.152+0000 7f92467fc700 1 --2- 172.21.15.107:0/1341204227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f924806eac0 0x7f924806ef30 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f923c004fc0 tx=0x7f923c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:34:23.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.152+0000 7f922f7fe700 1 -- 172.21.15.107:0/1341204227 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f923c00e070 con 0x7f924806eac0 2024-04-22T23:34:23.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.153+0000 7f922f7fe700 1 -- 172.21.15.107:0/1341204227 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f923c007ab0 con 0x7f924806eac0 2024-04-22T23:34:23.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.153+0000 7f922f7fe700 1 -- 172.21.15.107:0/1341204227 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f923c009350 con 0x7f924806eac0 2024-04-22T23:34:23.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.154+0000 7f924d436700 1 -- 172.21.15.107:0/1341204227 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92480fb810 con 0x7f924806eac0 2024-04-22T23:34:23.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.154+0000 7f924d436700 1 -- 172.21.15.107:0/1341204227 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92480fbd20 con 0x7f924806eac0 2024-04-22T23:34:23.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.155+0000 7f922d7fa700 1 -- 172.21.15.107:0/1341204227 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92280052b0 con 0x7f924806eac0 2024-04-22T23:34:23.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.158+0000 7f922f7fe700 1 -- 172.21.15.107:0/1341204227 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f923c00bb30 con 0x7f924806eac0 2024-04-22T23:34:23.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.158+0000 7f922f7fe700 1 --2- 172.21.15.107:0/1341204227 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f9230069370 0x7f923006b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:34:23.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.158+0000 7f922f7fe700 1 -- 172.21.15.107:0/1341204227 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f923c08f7f0 con 0x7f924806eac0 2024-04-22T23:34:23.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.158+0000 7f9246ffd700 1 --2- 172.21.15.107:0/1341204227 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f9230069370 0x7f923006b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:34:23.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.159+0000 7f9246ffd700 1 --2- 172.21.15.107:0/1341204227 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f9230069370 0x7f923006b820 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9238009fd0 tx=0x7f9238009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:34:23.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.165+0000 7f922f7fe700 1 -- 172.21.15.107:0/1341204227 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f923c05bab0 con 0x7f924806eac0 2024-04-22T23:34:23.280 INFO:tasks.workunit.client.1.smithi157.stderr:Updating files: 99% (12158/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-04-22T23:34:23.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:34:23 smithi157 ceph-mon[33877]: from='client.14616 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:34:23.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:34:23 smithi157 ceph-mon[33877]: pgmap v238: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:34:23.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:34:23 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/3098985527' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:34:23.383 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:34:23 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/1660929674' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:34:23.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:34:23 smithi107 ceph-mon[26355]: from='client.14616 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:34:23.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:34:23 smithi107 ceph-mon[26355]: pgmap v238: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:34:23.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:34:23 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/3098985527' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:34:23.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:34:23 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/1660929674' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:34:23.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.564+0000 7f922d7fa700 1 -- 172.21.15.107:0/1341204227 --> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9228000bc0 con 0x7f9230069370 2024-04-22T23:34:23.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.566+0000 7f922f7fe700 1 -- 172.21.15.107:0/1341204227 <== mgr.14206 v2:172.21.15.107:6800/2516490111 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f9228000bc0 con 0x7f9230069370 2024-04-22T23:34:23.562 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-22T23:34:23.562 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:43be020184947e53516056c9931e1ac5bdbbb1a5", 2024-04-22T23:34:23.562 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-04-22T23:34:23.562 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-22T23:34:23.562 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [], 2024-04-22T23:34:23.562 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "", 2024-04-22T23:34:23.562 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Doing first pull of quay.ceph.io/ceph-ci/ceph:43be020184947e53516056c9931e1ac5bdbbb1a5 image", 2024-04-22T23:34:23.562 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-04-22T23:34:23.562 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-22T23:34:23.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.568+0000 7f922d7fa700 1 -- 172.21.15.107:0/1341204227 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f9230069370 msgr2=0x7f923006b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:34:23.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.568+0000 7f922d7fa700 1 --2- 172.21.15.107:0/1341204227 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f9230069370 0x7f923006b820 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9238009fd0 tx=0x7f9238009e30 comp rx=0 tx=0).stop 2024-04-22T23:34:23.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.568+0000 7f922d7fa700 1 -- 172.21.15.107:0/1341204227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f924806eac0 msgr2=0x7f924806ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:34:23.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.568+0000 7f922d7fa700 1 --2- 172.21.15.107:0/1341204227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f924806eac0 0x7f924806ef30 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f923c004fc0 tx=0x7f923c00b040 comp rx=0 tx=0).stop 2024-04-22T23:34:23.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.568+0000 7f922d7fa700 1 -- 172.21.15.107:0/1341204227 shutdown_connections 2024-04-22T23:34:23.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.568+0000 7f922d7fa700 1 --2- 172.21.15.107:0/1341204227 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f9230069370 0x7f923006b820 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:23.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.568+0000 7f922d7fa700 1 --2- 172.21.15.107:0/1341204227 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f92480fff70 0x7f924806e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:23.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.568+0000 7f922d7fa700 1 --2- 172.21.15.107:0/1341204227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f924806eac0 0x7f924806ef30 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:23.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.568+0000 7f922d7fa700 1 -- 172.21.15.107:0/1341204227 >> 172.21.15.107:0/1341204227 conn(0x7f92480f74c0 msgr2=0x7f92480f7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:34:23.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.568+0000 7f922d7fa700 1 -- 172.21.15.107:0/1341204227 shutdown_connections 2024-04-22T23:34:23.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.568+0000 7f922d7fa700 1 -- 172.21.15.107:0/1341204227 wait complete. 2024-04-22T23:34:23.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.672+0000 7f2bc76ff700 1 -- 172.21.15.107:0/4007535660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bc00fcf20 msgr2=0x7f2bc00fd330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:34:23.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.672+0000 7f2bc76ff700 1 --2- 172.21.15.107:0/4007535660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bc00fcf20 0x7f2bc00fd330 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc004940 tx=0x7f2bbc009e30 comp rx=0 tx=0).stop 2024-04-22T23:34:23.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.672+0000 7f2bc76ff700 1 -- 172.21.15.107:0/4007535660 shutdown_connections 2024-04-22T23:34:23.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.672+0000 7f2bc76ff700 1 --2- 172.21.15.107:0/4007535660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bc00fe090 0x7f2bc00fe500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:23.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.672+0000 7f2bc76ff700 1 --2- 172.21.15.107:0/4007535660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bc00fcf20 0x7f2bc00fd330 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:23.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.672+0000 7f2bc76ff700 1 -- 172.21.15.107:0/4007535660 >> 172.21.15.107:0/4007535660 conn(0x7f2bc00f85c0 msgr2=0x7f2bc00faa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:34:23.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.672+0000 7f2bc76ff700 1 -- 172.21.15.107:0/4007535660 shutdown_connections 2024-04-22T23:34:23.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.672+0000 7f2bc76ff700 1 -- 172.21.15.107:0/4007535660 wait complete. 2024-04-22T23:34:23.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.673+0000 7f2bc76ff700 1 Processor -- start 2024-04-22T23:34:23.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.673+0000 7f2bc76ff700 1 -- start start 2024-04-22T23:34:23.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.673+0000 7f2bc76ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bc00fcf20 0x7f2bc0071df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:34:23.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.673+0000 7f2bc76ff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bc00fe090 0x7f2bc0070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:34:23.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.673+0000 7f2bc76ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc0111720 con 0x7f2bc00fcf20 2024-04-22T23:34:23.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.673+0000 7f2bc76ff700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc0111860 con 0x7f2bc00fe090 2024-04-22T23:34:23.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.673+0000 7f2bc549b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bc00fcf20 0x7f2bc0071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:34:23.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.673+0000 7f2bc549b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bc00fcf20 0x7f2bc0071df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38476/0 (socket says 172.21.15.107:38476) 2024-04-22T23:34:23.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.673+0000 7f2bc549b700 1 -- 172.21.15.107:0/3349996847 learned_addr learned my addr 172.21.15.107:0/3349996847 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:34:23.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.673+0000 7f2bc4c9a700 1 --2- 172.21.15.107:0/3349996847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bc00fe090 0x7f2bc0070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:34:23.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.673+0000 7f2bc549b700 1 -- 172.21.15.107:0/3349996847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bc00fe090 msgr2=0x7f2bc0070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:34:23.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.673+0000 7f2bc549b700 1 --2- 172.21.15.107:0/3349996847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bc00fe090 0x7f2bc0070310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:23.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.674+0000 7f2bc549b700 1 -- 172.21.15.107:0/3349996847 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bbc005040 con 0x7f2bc00fcf20 2024-04-22T23:34:23.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.674+0000 7f2bc549b700 1 --2- 172.21.15.107:0/3349996847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bc00fcf20 0x7f2bc0071df0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc00b3b0 tx=0x7f2bbc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:34:23.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.674+0000 7f2bb5ffb700 1 -- 172.21.15.107:0/3349996847 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2bbc00e040 con 0x7f2bc00fcf20 2024-04-22T23:34:23.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.674+0000 7f2bc76ff700 1 -- 172.21.15.107:0/3349996847 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bc00708b0 con 0x7f2bc00fcf20 2024-04-22T23:34:23.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.674+0000 7f2bb5ffb700 1 -- 172.21.15.107:0/3349996847 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2bbc00bbf0 con 0x7f2bc00fcf20 2024-04-22T23:34:23.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.674+0000 7f2bb5ffb700 1 -- 172.21.15.107:0/3349996847 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2bbc01c630 con 0x7f2bc00fcf20 2024-04-22T23:34:23.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.674+0000 7f2bc76ff700 1 -- 172.21.15.107:0/3349996847 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bc0070e70 con 0x7f2bc00fcf20 2024-04-22T23:34:23.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.675+0000 7f2bc76ff700 1 -- 172.21.15.107:0/3349996847 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bc0106120 con 0x7f2bc00fcf20 2024-04-22T23:34:23.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.677+0000 7f2bb5ffb700 1 -- 172.21.15.107:0/3349996847 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f2bbc00bd60 con 0x7f2bc00fcf20 2024-04-22T23:34:23.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.677+0000 7f2bb5ffb700 1 --2- 172.21.15.107:0/3349996847 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f2bac0692a0 0x7f2bac06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:34:23.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.677+0000 7f2bb5ffb700 1 -- 172.21.15.107:0/3349996847 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f2bbc087df0 con 0x7f2bc00fcf20 2024-04-22T23:34:23.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.677+0000 7f2bc4c9a700 1 --2- 172.21.15.107:0/3349996847 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f2bac0692a0 0x7f2bac06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:34:23.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.678+0000 7f2bc4c9a700 1 --2- 172.21.15.107:0/3349996847 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f2bac0692a0 0x7f2bac06b750 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2bc00723d0 tx=0x7f2bb000b040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:34:23.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.682+0000 7f2bb5ffb700 1 -- 172.21.15.107:0/3349996847 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f2bbc054170 con 0x7f2bc00fcf20 2024-04-22T23:34:23.790 INFO:tasks.workunit.client.1.smithi157.stderr:Note: switching to '43be020184947e53516056c9931e1ac5bdbbb1a5'. 2024-04-22T23:34:23.790 INFO:tasks.workunit.client.1.smithi157.stderr: 2024-04-22T23:34:23.790 INFO:tasks.workunit.client.1.smithi157.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-22T23:34:23.790 INFO:tasks.workunit.client.1.smithi157.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-22T23:34:23.790 INFO:tasks.workunit.client.1.smithi157.stderr:state without impacting any branches by switching back to a branch. 2024-04-22T23:34:23.790 INFO:tasks.workunit.client.1.smithi157.stderr: 2024-04-22T23:34:23.791 INFO:tasks.workunit.client.1.smithi157.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-22T23:34:23.791 INFO:tasks.workunit.client.1.smithi157.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-22T23:34:23.791 INFO:tasks.workunit.client.1.smithi157.stderr: 2024-04-22T23:34:23.791 INFO:tasks.workunit.client.1.smithi157.stderr: git switch -c 2024-04-22T23:34:23.791 INFO:tasks.workunit.client.1.smithi157.stderr: 2024-04-22T23:34:23.791 INFO:tasks.workunit.client.1.smithi157.stderr:Or undo this operation with: 2024-04-22T23:34:23.791 INFO:tasks.workunit.client.1.smithi157.stderr: 2024-04-22T23:34:23.791 INFO:tasks.workunit.client.1.smithi157.stderr: git switch - 2024-04-22T23:34:23.791 INFO:tasks.workunit.client.1.smithi157.stderr: 2024-04-22T23:34:23.791 INFO:tasks.workunit.client.1.smithi157.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-22T23:34:23.791 INFO:tasks.workunit.client.1.smithi157.stderr: 2024-04-22T23:34:23.791 INFO:tasks.workunit.client.1.smithi157.stderr:HEAD is now at 43be0201849 Merge branch 'host-drain-test-ignore-stray-host' of https://github.com/adk3798/ceph into wip-adk-testing-2024-04-22-1618 2024-04-22T23:34:23.796 DEBUG:teuthology.orchestra.run.smithi157:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-04-22T23:34:23.865 INFO:tasks.workunit.client.1.smithi157.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-22T23:34:23.866 INFO:tasks.workunit.client.1.smithi157.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-04-22T23:34:23.866 INFO:tasks.workunit.client.1.smithi157.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-22T23:34:23.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.981+0000 7f2bc76ff700 1 -- 172.21.15.107:0/3349996847 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f2bc004bed0 con 0x7f2bc00fcf20 2024-04-22T23:34:23.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.981+0000 7f2bb5ffb700 1 -- 172.21.15.107:0/3349996847 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f2bbc057790 con 0x7f2bc00fcf20 2024-04-22T23:34:23.977 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_OK 2024-04-22T23:34:23.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.985+0000 7f2bc76ff700 1 -- 172.21.15.107:0/3349996847 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f2bac0692a0 msgr2=0x7f2bac06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:34:23.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.985+0000 7f2bc76ff700 1 --2- 172.21.15.107:0/3349996847 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f2bac0692a0 0x7f2bac06b750 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2bc00723d0 tx=0x7f2bb000b040 comp rx=0 tx=0).stop 2024-04-22T23:34:23.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.985+0000 7f2bc76ff700 1 -- 172.21.15.107:0/3349996847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bc00fcf20 msgr2=0x7f2bc0071df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:34:23.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.985+0000 7f2bc76ff700 1 --2- 172.21.15.107:0/3349996847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bc00fcf20 0x7f2bc0071df0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc00b3b0 tx=0x7f2bbc009e30 comp rx=0 tx=0).stop 2024-04-22T23:34:23.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.985+0000 7f2bc76ff700 1 -- 172.21.15.107:0/3349996847 shutdown_connections 2024-04-22T23:34:23.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.985+0000 7f2bc76ff700 1 --2- 172.21.15.107:0/3349996847 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f2bac0692a0 0x7f2bac06b750 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:23.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.985+0000 7f2bc76ff700 1 --2- 172.21.15.107:0/3349996847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bc00fcf20 0x7f2bc0071df0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:23.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.985+0000 7f2bc76ff700 1 --2- 172.21.15.107:0/3349996847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bc00fe090 0x7f2bc0070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:23.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.985+0000 7f2bc76ff700 1 -- 172.21.15.107:0/3349996847 >> 172.21.15.107:0/3349996847 conn(0x7f2bc00f85c0 msgr2=0x7f2bc00f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:34:23.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.985+0000 7f2bc76ff700 1 -- 172.21.15.107:0/3349996847 shutdown_connections 2024-04-22T23:34:23.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:23.985+0000 7f2bc76ff700 1 -- 172.21.15.107:0/3349996847 wait complete. 2024-04-22T23:34:24.234 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:34:24 smithi107 ceph-mon[26355]: from='client.14628 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:34:24.234 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:34:24 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/3349996847' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:34:24.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:34:24 smithi157 ceph-mon[33877]: from='client.14628 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:34:24.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:34:24 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/3349996847' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:34:24.504 INFO:tasks.workunit.client.1.smithi157.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-22T23:34:24.559 INFO:tasks.workunit.client.1.smithi157.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-22T23:34:24.599 INFO:tasks.workunit.client.1.smithi157.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-04-22T23:34:24.600 INFO:tasks.workunit.client.1.smithi157.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-04-22T23:34:24.601 INFO:tasks.workunit.client.1.smithi157.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-22T23:34:24.640 INFO:tasks.workunit.client.1.smithi157.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-04-22T23:34:24.643 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-22T23:34:24.643 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-04-22T23:34:24.700 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-04-22T23:34:24.703 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-04-22T23:34:24.704 DEBUG:teuthology.orchestra.run.smithi157:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=43be020184947e53516056c9931e1ac5bdbbb1a5 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-04-22T23:34:24.761 INFO:tasks.workunit.client.1.smithi157.stderr:+ mkdir -p fsstress 2024-04-22T23:34:24.763 INFO:tasks.workunit.client.1.smithi157.stderr:+ pushd fsstress 2024-04-22T23:34:24.764 INFO:tasks.workunit.client.1.smithi157.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-04-22T23:34:24.764 INFO:tasks.workunit.client.1.smithi157.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-04-22T23:34:25.273 INFO:tasks.workunit.client.1.smithi157.stderr:+ tar xzf ltp-full.tgz 2024-04-22T23:34:25.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:34:25 smithi157 ceph-mon[33877]: pgmap v239: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:34:25.477 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:34:25 smithi107 ceph-mon[26355]: pgmap v239: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:34:26.878 INFO:tasks.workunit.client.0.smithi107.stderr:Updating files: 88% (10901/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-04-22T23:34:26.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:34:26 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:34:26.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:34:26 smithi157 ceph-mon[33877]: Upgrade: Target is version 19.0.0-3252-g43be0201 (unknown) 2024-04-22T23:34:26.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:34:26 smithi157 ceph-mon[33877]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, digests ['quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c'] 2024-04-22T23:34:26.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:34:26 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:34:26.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:34:26 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:34:26.984 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:34:26 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:34:26.984 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:34:26 smithi107 ceph-mon[26355]: Upgrade: Target is version 19.0.0-3252-g43be0201 (unknown) 2024-04-22T23:34:26.985 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:34:26 smithi107 ceph-mon[26355]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, digests ['quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c'] 2024-04-22T23:34:26.985 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:34:26 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:34:26.985 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:34:26 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:34:27.740 INFO:tasks.workunit.client.0.smithi107.stderr:Note: switching to '43be020184947e53516056c9931e1ac5bdbbb1a5'. 2024-04-22T23:34:27.740 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-04-22T23:34:27.740 INFO:tasks.workunit.client.0.smithi107.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-22T23:34:27.740 INFO:tasks.workunit.client.0.smithi107.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-22T23:34:27.740 INFO:tasks.workunit.client.0.smithi107.stderr:state without impacting any branches by switching back to a branch. 2024-04-22T23:34:27.740 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-04-22T23:34:27.740 INFO:tasks.workunit.client.0.smithi107.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-22T23:34:27.740 INFO:tasks.workunit.client.0.smithi107.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-22T23:34:27.741 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-04-22T23:34:27.741 INFO:tasks.workunit.client.0.smithi107.stderr: git switch -c 2024-04-22T23:34:27.741 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-04-22T23:34:27.741 INFO:tasks.workunit.client.0.smithi107.stderr:Or undo this operation with: 2024-04-22T23:34:27.741 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-04-22T23:34:27.741 INFO:tasks.workunit.client.0.smithi107.stderr: git switch - 2024-04-22T23:34:27.741 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-04-22T23:34:27.741 INFO:tasks.workunit.client.0.smithi107.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-22T23:34:27.741 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-04-22T23:34:27.741 INFO:tasks.workunit.client.0.smithi107.stderr:HEAD is now at 43be0201849 Merge branch 'host-drain-test-ignore-stray-host' of https://github.com/adk3798/ceph into wip-adk-testing-2024-04-22-1618 2024-04-22T23:34:27.746 DEBUG:teuthology.orchestra.run.smithi107:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-22T23:34:27.778 INFO:tasks.workunit.client.0.smithi107.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-22T23:34:27.780 INFO:tasks.workunit.client.0.smithi107.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-22T23:34:27.780 INFO:tasks.workunit.client.0.smithi107.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-22T23:34:27.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:34:27 smithi157 ceph-mon[33877]: Upgrade: Need to upgrade myself (mgr.smithi107.uipxvm) 2024-04-22T23:34:27.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:34:27 smithi157 ceph-mon[33877]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c on smithi157 2024-04-22T23:34:27.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:34:27 smithi157 ceph-mon[33877]: pgmap v240: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:34:27.984 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:34:27 smithi107 ceph-mon[26355]: Upgrade: Need to upgrade myself (mgr.smithi107.uipxvm) 2024-04-22T23:34:27.984 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:34:27 smithi107 ceph-mon[26355]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c on smithi157 2024-04-22T23:34:27.984 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:34:27 smithi107 ceph-mon[26355]: pgmap v240: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:34:28.418 INFO:tasks.workunit.client.0.smithi107.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-22T23:34:28.472 INFO:tasks.workunit.client.0.smithi107.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-22T23:34:28.519 INFO:tasks.workunit.client.0.smithi107.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-22T23:34:28.520 INFO:tasks.workunit.client.0.smithi107.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-22T23:34:28.520 INFO:tasks.workunit.client.0.smithi107.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-22T23:34:28.559 INFO:tasks.workunit.client.0.smithi107.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-22T23:34:28.562 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-22T23:34:28.562 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-22T23:34:28.632 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-04-22T23:34:28.633 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-04-22T23:34:28.633 DEBUG:teuthology.orchestra.run.smithi107:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=43be020184947e53516056c9931e1ac5bdbbb1a5 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-04-22T23:34:28.695 INFO:tasks.workunit.client.0.smithi107.stderr:+ mkdir -p fsstress 2024-04-22T23:34:28.697 INFO:tasks.workunit.client.0.smithi107.stderr:+ pushd fsstress 2024-04-22T23:34:28.698 INFO:tasks.workunit.client.0.smithi107.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-22T23:34:28.698 INFO:tasks.workunit.client.0.smithi107.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-04-22T23:34:29.403 INFO:tasks.workunit.client.0.smithi107.stderr:+ tar xzf ltp-full.tgz 2024-04-22T23:34:29.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:34:29 smithi157 ceph-mon[33877]: pgmap v241: 65 pgs: 65 active+clean; 2.5 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 174 KiB/s wr, 0 op/s 2024-04-22T23:34:29.984 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:34:29 smithi107 ceph-mon[26355]: pgmap v241: 65 pgs: 65 active+clean; 2.5 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 174 KiB/s wr, 0 op/s 2024-04-22T23:34:30.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:34:30 smithi157 ceph-mon[33877]: pgmap v242: 65 pgs: 65 active+clean; 2.5 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 174 KiB/s wr, 0 op/s 2024-04-22T23:34:30.984 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:34:30 smithi107 ceph-mon[26355]: pgmap v242: 65 pgs: 65 active+clean; 2.5 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 174 KiB/s wr, 0 op/s 2024-04-22T23:34:32.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:34:32 smithi157 ceph-mon[33877]: fsmap cephfs:2 {0=cephfs.smithi107.ktuxjl=up:active,1=cephfs.smithi157.rcaris=up:active} 2 up:standby 2024-04-22T23:34:32.883 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:34:32 smithi157 ceph-mon[33877]: pgmap v243: 65 pgs: 65 active+clean; 35 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 3.0 MiB/s wr, 235 op/s 2024-04-22T23:34:32.984 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:34:32 smithi107 ceph-mon[26355]: fsmap cephfs:2 {0=cephfs.smithi107.ktuxjl=up:active,1=cephfs.smithi157.rcaris=up:active} 2 up:standby 2024-04-22T23:34:32.984 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:34:32 smithi107 ceph-mon[26355]: pgmap v243: 65 pgs: 65 active+clean; 35 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 3.0 MiB/s wr, 235 op/s 2024-04-22T23:34:34.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:34:34 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:34:34.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:34:34 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:34:35.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:34:35 smithi157 ceph-mon[33877]: pgmap v244: 65 pgs: 65 active+clean; 37 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 3.2 MiB/s wr, 278 op/s 2024-04-22T23:34:35.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:34:35 smithi107 ceph-mon[26355]: pgmap v244: 65 pgs: 65 active+clean; 37 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 3.2 MiB/s wr, 278 op/s 2024-04-22T23:34:37.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:34:37 smithi157 ceph-mon[33877]: pgmap v245: 65 pgs: 65 active+clean; 57 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 5.0 MiB/s wr, 477 op/s 2024-04-22T23:34:37.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:34:37 smithi107 ceph-mon[26355]: pgmap v245: 65 pgs: 65 active+clean; 57 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 5.0 MiB/s wr, 477 op/s 2024-04-22T23:34:39.234 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:34:39 smithi107 ceph-mon[26355]: pgmap v246: 65 pgs: 65 active+clean; 79 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 6.9 MiB/s wr, 579 op/s 2024-04-22T23:34:39.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:34:39 smithi157 ceph-mon[33877]: pgmap v246: 65 pgs: 65 active+clean; 79 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 6.9 MiB/s wr, 579 op/s 2024-04-22T23:34:41.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:34:41 smithi157 ceph-mon[33877]: pgmap v247: 65 pgs: 65 active+clean; 82 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 7.0 MiB/s wr, 603 op/s 2024-04-22T23:34:41.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:34:41 smithi107 ceph-mon[26355]: pgmap v247: 65 pgs: 65 active+clean; 82 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 7.0 MiB/s wr, 603 op/s 2024-04-22T23:34:43.269 INFO:tasks.workunit.client.1.smithi157.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-04-22T23:34:43.269 INFO:tasks.workunit.client.1.smithi157.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-04-22T23:34:43.269 INFO:tasks.workunit.client.1.smithi157.stderr:+ make 2024-04-22T23:34:43.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:34:43 smithi157 ceph-mon[33877]: pgmap v248: 65 pgs: 65 active+clean; 100 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 8.5 MiB/s wr, 620 op/s 2024-04-22T23:34:43.417 INFO:tasks.workunit.client.1.smithi157.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-04-22T23:34:43.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:34:43 smithi107 ceph-mon[26355]: pgmap v248: 65 pgs: 65 active+clean; 100 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 8.5 MiB/s wr, 620 op/s 2024-04-22T23:34:43.823 INFO:tasks.workunit.client.1.smithi157.stderr:++ readlink -f fsstress 2024-04-22T23:34:43.824 INFO:tasks.workunit.client.1.smithi157.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-04-22T23:34:43.824 INFO:tasks.workunit.client.1.smithi157.stderr:+ popd 2024-04-22T23:34:43.824 INFO:tasks.workunit.client.1.smithi157.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-04-22T23:34:43.824 INFO:tasks.workunit.client.1.smithi157.stderr:+ popd 2024-04-22T23:34:43.824 INFO:tasks.workunit.client.1.smithi157.stdout:~/cephtest/mnt.1/client.1/tmp 2024-04-22T23:34:43.825 INFO:tasks.workunit.client.1.smithi157.stderr:++ mktemp -d -p . 2024-04-22T23:34:43.826 INFO:tasks.workunit.client.1.smithi157.stderr:+ T=./tmp.3ctMruI8Hs 2024-04-22T23:34:43.826 INFO:tasks.workunit.client.1.smithi157.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.3ctMruI8Hs -l 1 -n 1000 -p 10 -v 2024-04-22T23:34:43.827 INFO:tasks.workunit.client.1.smithi157.stdout:seed = 1714048017 2024-04-22T23:34:43.829 INFO:tasks.workunit.client.1.smithi157.stdout:1/0: write - no filename 2024-04-22T23:34:43.830 INFO:tasks.workunit.client.1.smithi157.stdout:1/1: dwrite - no filename 2024-04-22T23:34:43.830 INFO:tasks.workunit.client.1.smithi157.stdout:1/2: unlink - no file 2024-04-22T23:34:43.830 INFO:tasks.workunit.client.1.smithi157.stdout:1/3: fsync - no filename 2024-04-22T23:34:43.830 INFO:tasks.workunit.client.1.smithi157.stdout:0/0: creat f0 x:0 0 0 2024-04-22T23:34:43.830 INFO:tasks.workunit.client.1.smithi157.stdout:2/0: fsync - no filename 2024-04-22T23:34:43.831 INFO:tasks.workunit.client.1.smithi157.stdout:2/1: write - no filename 2024-04-22T23:34:43.831 INFO:tasks.workunit.client.1.smithi157.stdout:1/4: creat f0 x:0 0 0 2024-04-22T23:34:43.831 INFO:tasks.workunit.client.1.smithi157.stdout:1/5: creat f1 x:0 0 0 2024-04-22T23:34:43.831 INFO:tasks.workunit.client.1.smithi157.stdout:1/6: truncate f0 704402 0 2024-04-22T23:34:43.832 INFO:tasks.workunit.client.1.smithi157.stdout:3/0: readlink - no filename 2024-04-22T23:34:43.832 INFO:tasks.workunit.client.1.smithi157.stdout:3/1: write - no filename 2024-04-22T23:34:43.833 INFO:tasks.workunit.client.1.smithi157.stdout:2/2: creat f0 x:0 0 0 2024-04-22T23:34:43.833 INFO:tasks.workunit.client.1.smithi157.stdout:2/3: stat f0 0 2024-04-22T23:34:43.867 INFO:tasks.workunit.client.1.smithi157.stdout:3/2: mkdir d0 0 2024-04-22T23:34:43.868 INFO:tasks.workunit.client.1.smithi157.stdout:3/3: read - no filename 2024-04-22T23:34:43.868 INFO:tasks.workunit.client.1.smithi157.stdout:3/4: write - no filename 2024-04-22T23:34:43.868 INFO:tasks.workunit.client.1.smithi157.stdout:3/5: getdents d0 0 2024-04-22T23:34:43.873 INFO:tasks.workunit.client.1.smithi157.stdout:2/4: mknod c1 0 2024-04-22T23:34:43.874 INFO:tasks.workunit.client.1.smithi157.stdout:2/5: truncate f0 944343 0 2024-04-22T23:34:43.874 INFO:tasks.workunit.client.1.smithi157.stdout:2/6: chown c1 1887778179 1 2024-04-22T23:34:43.879 INFO:tasks.workunit.client.1.smithi157.stdout:3/6: creat d0/f1 x:0 0 0 2024-04-22T23:34:43.879 INFO:tasks.workunit.client.1.smithi157.stdout:3/7: creat d0/f2 x:0 0 0 2024-04-22T23:34:43.879 INFO:tasks.workunit.client.1.smithi157.stdout:3/8: write d0/f2 [599434,93564] 0 2024-04-22T23:34:43.879 INFO:tasks.workunit.client.1.smithi157.stdout:5/0: chown . 1484155 1 2024-04-22T23:34:43.879 INFO:tasks.workunit.client.1.smithi157.stdout:5/1: rename - no filename 2024-04-22T23:34:43.879 INFO:tasks.workunit.client.1.smithi157.stdout:5/2: stat - no entries 2024-04-22T23:34:43.880 INFO:tasks.workunit.client.1.smithi157.stdout:4/0: symlink l0 0 2024-04-22T23:34:43.880 INFO:tasks.workunit.client.1.smithi157.stdout:4/1: dread - no filename 2024-04-22T23:34:43.883 INFO:tasks.workunit.client.1.smithi157.stdout:2/7: link c1 c2 0 2024-04-22T23:34:43.884 INFO:tasks.workunit.client.1.smithi157.stdout:2/8: creat f3 x:0 0 0 2024-04-22T23:34:43.889 INFO:tasks.workunit.client.1.smithi157.stdout:3/9: mkdir d0/d3 0 2024-04-22T23:34:43.889 INFO:tasks.workunit.client.1.smithi157.stdout:3/10: truncate d0/f2 1033178 0 2024-04-22T23:34:43.889 INFO:tasks.workunit.client.1.smithi157.stdout:3/11: dread - d0/f1 zero size 2024-04-22T23:34:43.889 INFO:tasks.workunit.client.1.smithi157.stdout:5/3: creat f0 x:0 0 0 2024-04-22T23:34:43.889 INFO:tasks.workunit.client.1.smithi157.stdout:5/4: dread - f0 zero size 2024-04-22T23:34:43.891 INFO:tasks.workunit.client.1.smithi157.stdout:6/0: stat - no entries 2024-04-22T23:34:43.891 INFO:tasks.workunit.client.1.smithi157.stdout:6/1: fsync - no filename 2024-04-22T23:34:43.891 INFO:tasks.workunit.client.1.smithi157.stdout:6/2: read - no filename 2024-04-22T23:34:43.891 INFO:tasks.workunit.client.1.smithi157.stdout:6/3: read - no filename 2024-04-22T23:34:43.893 INFO:tasks.workunit.client.1.smithi157.stdout:4/2: creat f1 x:0 0 0 2024-04-22T23:34:43.895 INFO:tasks.workunit.client.1.smithi157.stdout:3/12: symlink d0/d3/l4 0 2024-04-22T23:34:43.896 INFO:tasks.workunit.client.1.smithi157.stdout:5/5: mkdir d1 0 2024-04-22T23:34:43.896 INFO:tasks.workunit.client.1.smithi157.stdout:5/6: write f0 [556018,59764] 0 2024-04-22T23:34:43.896 INFO:tasks.workunit.client.1.smithi157.stdout:5/7: readlink - no filename 2024-04-22T23:34:43.896 INFO:tasks.workunit.client.1.smithi157.stdout:5/8: getdents d1 0 2024-04-22T23:34:43.897 INFO:tasks.workunit.client.1.smithi157.stdout:6/4: creat f0 x:0 0 0 2024-04-22T23:34:43.897 INFO:tasks.workunit.client.1.smithi157.stdout:6/5: rmdir - no directory 2024-04-22T23:34:43.897 INFO:tasks.workunit.client.1.smithi157.stdout:6/6: chown f0 32123684 1 2024-04-22T23:34:43.898 INFO:tasks.workunit.client.1.smithi157.stdout:7/0: write - no filename 2024-04-22T23:34:43.898 INFO:tasks.workunit.client.1.smithi157.stdout:7/1: readlink - no filename 2024-04-22T23:34:43.898 INFO:tasks.workunit.client.1.smithi157.stdout:7/2: write - no filename 2024-04-22T23:34:43.901 INFO:tasks.workunit.client.1.smithi157.stdout:3/13: link d0/f2 d0/d3/f5 0 2024-04-22T23:34:43.901 INFO:tasks.workunit.client.1.smithi157.stdout:5/9: symlink d1/l2 0 2024-04-22T23:34:43.901 INFO:tasks.workunit.client.1.smithi157.stdout:5/10: write f0 [1247226,106252] 0 2024-04-22T23:34:43.901 INFO:tasks.workunit.client.1.smithi157.stdout:6/7: mkdir d1 0 2024-04-22T23:34:43.907 INFO:tasks.workunit.client.1.smithi157.stdout:7/3: mkdir d0 0 2024-04-22T23:34:43.908 INFO:tasks.workunit.client.1.smithi157.stdout:7/4: write - no filename 2024-04-22T23:34:43.910 INFO:tasks.workunit.client.1.smithi157.stdout:8/0: dwrite - no filename 2024-04-22T23:34:43.910 INFO:tasks.workunit.client.1.smithi157.stdout:8/1: chown . 314304 1 2024-04-22T23:34:43.910 INFO:tasks.workunit.client.1.smithi157.stdout:8/2: fsync - no filename 2024-04-22T23:34:43.910 INFO:tasks.workunit.client.1.smithi157.stdout:8/3: write - no filename 2024-04-22T23:34:43.910 INFO:tasks.workunit.client.1.smithi157.stdout:8/4: readlink - no filename 2024-04-22T23:34:43.910 INFO:tasks.workunit.client.1.smithi157.stdout:8/5: dwrite - no filename 2024-04-22T23:34:43.920 INFO:tasks.workunit.client.1.smithi157.stdout:7/5: mknod d0/c1 0 2024-04-22T23:34:43.948 INFO:tasks.workunit.client.1.smithi157.stdout:0/1: dwrite f0 [0,4194304] 0 2024-04-22T23:34:44.091 INFO:tasks.workunit.client.1.smithi157.stdout:2/9: getdents . 0 2024-04-22T23:34:44.124 INFO:tasks.workunit.client.1.smithi157.stdout:7/6: getdents d0 0 2024-04-22T23:34:44.124 INFO:tasks.workunit.client.1.smithi157.stdout:7/7: getdents d0 0 2024-04-22T23:34:44.124 INFO:tasks.workunit.client.1.smithi157.stdout:7/8: truncate - no filename 2024-04-22T23:34:44.124 INFO:tasks.workunit.client.1.smithi157.stdout:7/9: dwrite - no filename 2024-04-22T23:34:44.124 INFO:tasks.workunit.client.1.smithi157.stdout:7/10: dwrite - no filename 2024-04-22T23:34:44.125 INFO:tasks.workunit.client.1.smithi157.stdout:7/11: write - no filename 2024-04-22T23:34:44.584 INFO:tasks.workunit.client.1.smithi157.stdout:3/14: write d0/f2 [91005,100819] 0 2024-04-22T23:34:44.584 INFO:tasks.workunit.client.1.smithi157.stdout:2/10: creat f4 x:0 0 0 2024-04-22T23:34:44.584 INFO:tasks.workunit.client.1.smithi157.stdout:2/11: write f4 [275424,27218] 0 2024-04-22T23:34:44.584 INFO:tasks.workunit.client.1.smithi157.stdout:2/12: stat f4 0 2024-04-22T23:34:44.584 INFO:tasks.workunit.client.1.smithi157.stdout:2/13: chown c2 28919 1 2024-04-22T23:34:44.837 INFO:tasks.workunit.client.1.smithi157.stdout:5/11: write f0 [314788,30510] 0 2024-04-22T23:34:44.852 INFO:tasks.workunit.client.1.smithi157.stdout:4/3: dwrite f1 [0,4194304] 0 2024-04-22T23:34:44.896 INFO:tasks.workunit.client.1.smithi157.stdout:9/0: chown . 5632658 1 2024-04-22T23:34:44.969 INFO:tasks.workunit.client.1.smithi157.stdout:6/8: dwrite f0 [0,4194304] 0 2024-04-22T23:34:44.969 INFO:tasks.workunit.client.1.smithi157.stdout:6/9: fdatasync f0 0 2024-04-22T23:34:44.990 INFO:tasks.workunit.client.1.smithi157.stdout:3/15: dwrite d0/d3/f5 [0,4194304] 0 2024-04-22T23:34:45.065 INFO:tasks.workunit.client.1.smithi157.stdout:2/14: dwrite f4 [0,4194304] 0 2024-04-22T23:34:45.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:34:45 smithi157 ceph-mon[33877]: pgmap v249: 65 pgs: 65 active+clean; 101 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 5.8 MiB/s wr, 411 op/s 2024-04-22T23:34:45.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:34:45 smithi107 ceph-mon[26355]: pgmap v249: 65 pgs: 65 active+clean; 101 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 5.8 MiB/s wr, 411 op/s 2024-04-22T23:34:46.801 INFO:tasks.workunit.client.1.smithi157.stdout:5/12: mknod d1/c3 0 2024-04-22T23:34:46.801 INFO:tasks.workunit.client.1.smithi157.stdout:5/13: write f0 [1470534,45299] 0 2024-04-22T23:34:47.112 INFO:tasks.workunit.client.1.smithi157.stdout:7/12: creat d0/f2 x:0 0 0 2024-04-22T23:34:47.232 INFO:tasks.workunit.client.1.smithi157.stdout:6/10: creat d1/f2 x:0 0 0 2024-04-22T23:34:47.233 INFO:tasks.workunit.client.1.smithi157.stdout:6/11: creat d1/f3 x:0 0 0 2024-04-22T23:34:47.247 INFO:tasks.workunit.client.1.smithi157.stdout:6/12: dwrite d1/f3 [0,4194304] 0 2024-04-22T23:34:47.305 INFO:tasks.workunit.client.1.smithi157.stdout:3/16: rename d0/f2 to d0/d3/f6 0 2024-04-22T23:34:47.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:34:47 smithi157 ceph-mon[33877]: pgmap v250: 65 pgs: 65 active+clean; 118 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 7.1 MiB/s wr, 529 op/s 2024-04-22T23:34:47.426 INFO:tasks.workunit.client.1.smithi157.stdout:2/15: stat c1 0 2024-04-22T23:34:47.426 INFO:tasks.workunit.client.1.smithi157.stdout:2/16: creat f5 x:0 0 0 2024-04-22T23:34:47.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:34:47 smithi107 ceph-mon[26355]: pgmap v250: 65 pgs: 65 active+clean; 118 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 7.1 MiB/s wr, 529 op/s 2024-04-22T23:34:47.578 INFO:tasks.workunit.client.1.smithi157.stdout:5/14: symlink d1/l4 0 2024-04-22T23:34:47.579 INFO:tasks.workunit.client.1.smithi157.stdout:0/2: truncate f0 939645 0 2024-04-22T23:34:47.581 INFO:tasks.workunit.client.1.smithi157.stdout:7/13: symlink d0/l3 0 2024-04-22T23:34:47.581 INFO:tasks.workunit.client.1.smithi157.stdout:7/14: truncate d0/f2 357932 0 2024-04-22T23:34:47.582 INFO:tasks.workunit.client.1.smithi157.stdout:6/13: rename d1/f2 to d1/f4 0 2024-04-22T23:34:47.582 INFO:tasks.workunit.client.1.smithi157.stdout:6/14: chown d1/f4 1 1 2024-04-22T23:34:47.583 INFO:tasks.workunit.client.1.smithi157.stdout:3/17: unlink d0/d3/f5 0 2024-04-22T23:34:47.585 INFO:tasks.workunit.client.1.smithi157.stdout:2/17: rename c2 to c6 0 2024-04-22T23:34:47.586 INFO:tasks.workunit.client.1.smithi157.stdout:5/15: mkdir d1/d5 0 2024-04-22T23:34:47.586 INFO:tasks.workunit.client.1.smithi157.stdout:5/16: chown d1/d5 128210839 1 2024-04-22T23:34:47.586 INFO:tasks.workunit.client.1.smithi157.stdout:5/17: readlink d1/l2 0 2024-04-22T23:34:47.589 INFO:tasks.workunit.client.1.smithi157.stdout:2/18: mkdir d7 0 2024-04-22T23:34:47.589 INFO:tasks.workunit.client.1.smithi157.stdout:5/18: fdatasync f0 0 2024-04-22T23:34:47.591 INFO:tasks.workunit.client.1.smithi157.stdout:2/19: mkdir d7/d8 0 2024-04-22T23:34:47.591 INFO:tasks.workunit.client.1.smithi157.stdout:2/20: chown f4 4710628 1 2024-04-22T23:34:47.592 INFO:tasks.workunit.client.1.smithi157.stdout:2/21: dread f0 [0,4194304] 0 2024-04-22T23:34:47.592 INFO:tasks.workunit.client.1.smithi157.stdout:5/19: rename d1/c3 to d1/d5/c6 0 2024-04-22T23:34:47.592 INFO:tasks.workunit.client.1.smithi157.stdout:5/20: readlink d1/l2 0 2024-04-22T23:34:47.594 INFO:tasks.workunit.client.1.smithi157.stdout:2/22: creat d7/d8/f9 x:0 0 0 2024-04-22T23:34:47.594 INFO:tasks.workunit.client.1.smithi157.stdout:2/23: chown f5 1737330 1 2024-04-22T23:34:47.595 INFO:tasks.workunit.client.1.smithi157.stdout:5/21: mkdir d1/d5/d7 0 2024-04-22T23:34:47.596 INFO:tasks.workunit.client.1.smithi157.stdout:7/15: dwrite d0/f2 [0,4194304] 0 2024-04-22T23:34:47.597 INFO:tasks.workunit.client.1.smithi157.stdout:5/22: dread f0 [0,4194304] 0 2024-04-22T23:34:47.600 INFO:tasks.workunit.client.1.smithi157.stdout:5/23: unlink f0 0 2024-04-22T23:34:47.600 INFO:tasks.workunit.client.1.smithi157.stdout:5/24: chown d1/d5/d7 2400292 1 2024-04-22T23:34:47.600 INFO:tasks.workunit.client.1.smithi157.stdout:5/25: dread - no filename 2024-04-22T23:34:47.600 INFO:tasks.workunit.client.1.smithi157.stdout:5/26: chown d1/d5/d7 6 1 2024-04-22T23:34:47.600 INFO:tasks.workunit.client.1.smithi157.stdout:5/27: readlink d1/l4 0 2024-04-22T23:34:47.600 INFO:tasks.workunit.client.1.smithi157.stdout:5/28: dwrite - no filename 2024-04-22T23:34:47.600 INFO:tasks.workunit.client.1.smithi157.stdout:5/29: write - no filename 2024-04-22T23:34:47.600 INFO:tasks.workunit.client.1.smithi157.stdout:5/30: dread - no filename 2024-04-22T23:34:47.601 INFO:tasks.workunit.client.1.smithi157.stdout:5/31: symlink d1/d5/l8 0 2024-04-22T23:34:47.601 INFO:tasks.workunit.client.1.smithi157.stdout:5/32: creat d1/d5/f9 x:0 0 0 2024-04-22T23:34:47.601 INFO:tasks.workunit.client.1.smithi157.stdout:5/33: dread - d1/d5/f9 zero size 2024-04-22T23:34:47.602 INFO:tasks.workunit.client.1.smithi157.stdout:5/34: symlink d1/d5/d7/la 0 2024-04-22T23:34:47.602 INFO:tasks.workunit.client.1.smithi157.stdout:5/35: dread - d1/d5/f9 zero size 2024-04-22T23:34:47.602 INFO:tasks.workunit.client.1.smithi157.stdout:5/36: write d1/d5/f9 [404598,93577] 0 2024-04-22T23:34:47.603 INFO:tasks.workunit.client.1.smithi157.stdout:5/37: rmdir d1/d5/d7 39 2024-04-22T23:34:47.604 INFO:tasks.workunit.client.1.smithi157.stdout:7/16: dread d0/f2 [0,4194304] 0 2024-04-22T23:34:47.604 INFO:tasks.workunit.client.1.smithi157.stdout:5/38: symlink d1/d5/d7/lb 0 2024-04-22T23:34:47.604 INFO:tasks.workunit.client.1.smithi157.stdout:7/17: creat d0/f4 x:0 0 0 2024-04-22T23:34:47.604 INFO:tasks.workunit.client.1.smithi157.stdout:7/18: chown d0/f4 259 1 2024-04-22T23:34:47.605 INFO:tasks.workunit.client.1.smithi157.stdout:5/39: mkdir d1/d5/dc 0 2024-04-22T23:34:47.605 INFO:tasks.workunit.client.1.smithi157.stdout:5/40: write d1/d5/f9 [804334,30218] 0 2024-04-22T23:34:47.605 INFO:tasks.workunit.client.1.smithi157.stdout:5/41: getdents d1/d5/dc 0 2024-04-22T23:34:47.606 INFO:tasks.workunit.client.1.smithi157.stdout:7/19: link d0/l3 d0/l5 0 2024-04-22T23:34:47.606 INFO:tasks.workunit.client.1.smithi157.stdout:7/20: creat d0/f6 x:0 0 0 2024-04-22T23:34:47.606 INFO:tasks.workunit.client.1.smithi157.stdout:7/21: chown d0/f2 5 1 2024-04-22T23:34:47.607 INFO:tasks.workunit.client.1.smithi157.stdout:5/42: unlink d1/d5/d7/lb 0 2024-04-22T23:34:47.608 INFO:tasks.workunit.client.1.smithi157.stdout:5/43: write d1/d5/f9 [506212,2682] 0 2024-04-22T23:34:47.609 INFO:tasks.workunit.client.1.smithi157.stdout:5/44: mknod d1/cd 0 2024-04-22T23:34:47.610 INFO:tasks.workunit.client.1.smithi157.stdout:7/22: rename d0/f6 to d0/f7 0 2024-04-22T23:34:47.616 INFO:tasks.workunit.client.1.smithi157.stdout:5/45: dread d1/d5/f9 [0,4194304] 0 2024-04-22T23:34:47.617 INFO:tasks.workunit.client.1.smithi157.stdout:5/46: creat d1/fe x:0 0 0 2024-04-22T23:34:47.618 INFO:tasks.workunit.client.1.smithi157.stdout:5/47: link d1/cd d1/d5/d7/cf 0 2024-04-22T23:34:47.619 INFO:tasks.workunit.client.1.smithi157.stdout:5/48: dread - d1/fe zero size 2024-04-22T23:34:47.619 INFO:tasks.workunit.client.1.smithi157.stdout:5/49: mkdir d1/d5/d7/d10 0 2024-04-22T23:34:47.621 INFO:tasks.workunit.client.1.smithi157.stdout:5/50: dread d1/d5/f9 [0,4194304] 0 2024-04-22T23:34:47.622 INFO:tasks.workunit.client.1.smithi157.stdout:5/51: dread d1/d5/f9 [0,4194304] 0 2024-04-22T23:34:47.627 INFO:tasks.workunit.client.1.smithi157.stdout:7/23: dwrite d0/f7 [0,4194304] 0 2024-04-22T23:34:47.633 INFO:tasks.workunit.client.1.smithi157.stdout:7/24: dread d0/f7 [0,4194304] 0 2024-04-22T23:34:47.633 INFO:tasks.workunit.client.1.smithi157.stdout:7/25: creat d0/f8 x:0 0 0 2024-04-22T23:34:47.633 INFO:tasks.workunit.client.1.smithi157.stdout:7/26: fdatasync d0/f4 0 2024-04-22T23:34:47.755 INFO:tasks.workunit.client.1.smithi157.stdout:4/4: dwrite f1 [4194304,4194304] 0 2024-04-22T23:34:47.756 INFO:tasks.workunit.client.1.smithi157.stdout:4/5: symlink l2 0 2024-04-22T23:34:47.780 INFO:tasks.workunit.client.1.smithi157.stdout:4/6: dwrite f1 [4194304,4194304] 0 2024-04-22T23:34:47.780 INFO:tasks.workunit.client.1.smithi157.stdout:4/7: stat l2 0 2024-04-22T23:34:47.781 INFO:tasks.workunit.client.1.smithi157.stdout:4/8: mkdir d3 0 2024-04-22T23:34:47.782 INFO:tasks.workunit.client.1.smithi157.stdout:4/9: mkdir d3/d4 0 2024-04-22T23:34:47.790 INFO:tasks.workunit.client.1.smithi157.stdout:4/10: write f1 [51948,21451] 0 2024-04-22T23:34:47.791 INFO:tasks.workunit.client.1.smithi157.stdout:4/11: rmdir d3 39 2024-04-22T23:34:47.791 INFO:tasks.workunit.client.1.smithi157.stdout:4/12: write f1 [8702373,3214] 0 2024-04-22T23:34:47.800 INFO:tasks.workunit.client.1.smithi157.stdout:4/13: write f1 [8240889,23185] 0 2024-04-22T23:34:47.801 INFO:tasks.workunit.client.1.smithi157.stdout:4/14: creat d3/d4/f5 x:0 0 0 2024-04-22T23:34:47.801 INFO:tasks.workunit.client.1.smithi157.stdout:4/15: write f1 [7397006,115340] 0 2024-04-22T23:34:47.801 INFO:tasks.workunit.client.1.smithi157.stdout:4/16: stat d3/d4/f5 0 2024-04-22T23:34:47.804 INFO:tasks.workunit.client.1.smithi157.stdout:4/17: fsync f1 0 2024-04-22T23:34:47.804 INFO:tasks.workunit.client.1.smithi157.stdout:4/18: fdatasync d3/d4/f5 0 2024-04-22T23:34:47.804 INFO:tasks.workunit.client.1.smithi157.stdout:4/19: dread - d3/d4/f5 zero size 2024-04-22T23:34:47.806 INFO:tasks.workunit.client.1.smithi157.stdout:4/20: rename f1 to d3/d4/f6 0 2024-04-22T23:34:47.807 INFO:tasks.workunit.client.1.smithi157.stdout:4/21: chown d3/d4/f5 773 1 2024-04-22T23:34:47.939 INFO:tasks.workunit.client.1.smithi157.stdout:0/3: truncate f0 1559486 0 2024-04-22T23:34:47.941 INFO:tasks.workunit.client.1.smithi157.stdout:6/15: rmdir d1 39 2024-04-22T23:34:47.941 INFO:tasks.workunit.client.1.smithi157.stdout:6/16: chown d1 451996 1 2024-04-22T23:34:47.944 INFO:tasks.workunit.client.1.smithi157.stdout:3/18: truncate d0/d3/f6 2524449 0 2024-04-22T23:34:47.947 INFO:tasks.workunit.client.1.smithi157.stdout:6/17: dread d1/f3 [0,4194304] 0 2024-04-22T23:34:47.947 INFO:tasks.workunit.client.1.smithi157.stdout:6/18: stat d1/f3 0 2024-04-22T23:34:47.947 INFO:tasks.workunit.client.1.smithi157.stdout:6/19: stat d1 0 2024-04-22T23:34:47.950 INFO:tasks.workunit.client.1.smithi157.stdout:5/52: fsync d1/fe 0 2024-04-22T23:34:47.953 INFO:tasks.workunit.client.1.smithi157.stdout:7/27: fsync d0/f8 0 2024-04-22T23:34:47.954 INFO:tasks.workunit.client.1.smithi157.stdout:6/20: write f0 [2955643,45121] 0 2024-04-22T23:34:47.958 INFO:tasks.workunit.client.1.smithi157.stdout:4/22: truncate d3/d4/f6 8668488 0 2024-04-22T23:34:47.958 INFO:tasks.workunit.client.1.smithi157.stdout:4/23: dread - d3/d4/f5 zero size 2024-04-22T23:34:47.958 INFO:tasks.workunit.client.1.smithi157.stdout:4/24: truncate d3/d4/f5 19113 0 2024-04-22T23:34:47.958 INFO:tasks.workunit.client.1.smithi157.stdout:4/25: chown l0 163 1 2024-04-22T23:34:47.958 INFO:tasks.workunit.client.1.smithi157.stdout:4/26: fdatasync d3/d4/f5 0 2024-04-22T23:34:47.963 INFO:tasks.workunit.client.1.smithi157.stdout:6/21: dread f0 [0,4194304] 0 2024-04-22T23:34:47.963 INFO:tasks.workunit.client.1.smithi157.stdout:4/27: getdents d3 0 2024-04-22T23:34:47.964 INFO:tasks.workunit.client.1.smithi157.stdout:0/4: truncate f0 1534654 0 2024-04-22T23:34:47.964 INFO:tasks.workunit.client.1.smithi157.stdout:6/22: getdents d1 0 2024-04-22T23:34:47.966 INFO:tasks.workunit.client.1.smithi157.stdout:0/5: mknod c1 0 2024-04-22T23:34:47.968 INFO:tasks.workunit.client.1.smithi157.stdout:7/28: dwrite d0/f7 [0,4194304] 0 2024-04-22T23:34:47.968 INFO:tasks.workunit.client.1.smithi157.stdout:6/23: rename d1/f4 to d1/f5 0 2024-04-22T23:34:47.969 INFO:tasks.workunit.client.1.smithi157.stdout:5/53: truncate d1/d5/f9 260774 0 2024-04-22T23:34:47.970 INFO:tasks.workunit.client.1.smithi157.stdout:0/6: symlink l2 0 2024-04-22T23:34:47.971 INFO:tasks.workunit.client.1.smithi157.stdout:7/29: mkdir d0/d9 0 2024-04-22T23:34:47.972 INFO:tasks.workunit.client.1.smithi157.stdout:6/24: rename d1 to d1/d6 22 2024-04-22T23:34:47.974 INFO:tasks.workunit.client.1.smithi157.stdout:4/28: read d3/d4/f6 [1667858,54487] 0 2024-04-22T23:34:47.974 INFO:tasks.workunit.client.1.smithi157.stdout:3/19: dread d0/d3/f6 [0,4194304] 0 2024-04-22T23:34:47.975 INFO:tasks.workunit.client.1.smithi157.stdout:6/25: fsync d1/f5 0 2024-04-22T23:34:47.976 INFO:tasks.workunit.client.1.smithi157.stdout:4/29: link l0 d3/l7 0 2024-04-22T23:34:47.976 INFO:tasks.workunit.client.1.smithi157.stdout:4/30: fdatasync d3/d4/f5 0 2024-04-22T23:34:47.981 INFO:tasks.workunit.client.1.smithi157.stdout:4/31: rename d3/d4/f6 to d3/f8 0 2024-04-22T23:34:47.981 INFO:tasks.workunit.client.1.smithi157.stdout:6/26: truncate d1/f3 3616951 0 2024-04-22T23:34:47.986 INFO:tasks.workunit.client.1.smithi157.stdout:6/27: creat d1/f7 x:0 0 0 2024-04-22T23:34:47.990 INFO:tasks.workunit.client.1.smithi157.stdout:6/28: mknod d1/c8 0 2024-04-22T23:34:47.992 INFO:tasks.workunit.client.1.smithi157.stdout:4/32: dread d3/f8 [4194304,4194304] 0 2024-04-22T23:34:47.992 INFO:tasks.workunit.client.1.smithi157.stdout:4/33: stat d3/f8 0 2024-04-22T23:34:48.000 INFO:tasks.workunit.client.1.smithi157.stdout:4/34: dread d3/f8 [4194304,4194304] 0 2024-04-22T23:34:48.000 INFO:tasks.workunit.client.1.smithi157.stdout:4/35: chown d3/d4/f5 25 1 2024-04-22T23:34:48.001 INFO:tasks.workunit.client.1.smithi157.stdout:4/36: truncate d3/f8 1694846 0 2024-04-22T23:34:48.001 INFO:tasks.workunit.client.1.smithi157.stdout:4/37: creat d3/d4/f9 x:0 0 0 2024-04-22T23:34:48.003 INFO:tasks.workunit.client.1.smithi157.stdout:4/38: mknod d3/d4/ca 0 2024-04-22T23:34:48.003 INFO:tasks.workunit.client.1.smithi157.stdout:4/39: write d3/d4/f9 [1016079,94575] 0 2024-04-22T23:34:48.014 INFO:tasks.workunit.client.1.smithi157.stdout:3/20: dwrite d0/d3/f6 [0,4194304] 0 2024-04-22T23:34:48.014 INFO:tasks.workunit.client.1.smithi157.stdout:5/54: dwrite d1/fe [0,4194304] 0 2024-04-22T23:34:48.014 INFO:tasks.workunit.client.1.smithi157.stdout:0/7: dwrite f0 [0,4194304] 0 2024-04-22T23:34:48.016 INFO:tasks.workunit.client.1.smithi157.stdout:3/21: mkdir d0/d7 0 2024-04-22T23:34:48.016 INFO:tasks.workunit.client.1.smithi157.stdout:6/29: fsync d1/f7 0 2024-04-22T23:34:48.020 INFO:tasks.workunit.client.1.smithi157.stdout:0/8: unlink c1 0 2024-04-22T23:34:48.020 INFO:tasks.workunit.client.1.smithi157.stdout:0/9: creat f3 x:0 0 0 2024-04-22T23:34:48.023 INFO:tasks.workunit.client.1.smithi157.stdout:6/30: getdents d1 0 2024-04-22T23:34:48.023 INFO:tasks.workunit.client.1.smithi157.stdout:6/31: write f0 [839637,120859] 0 2024-04-22T23:34:48.024 INFO:tasks.workunit.client.1.smithi157.stdout:6/32: fsync d1/f7 0 2024-04-22T23:34:48.027 INFO:tasks.workunit.client.1.smithi157.stdout:6/33: rmdir d1 39 2024-04-22T23:34:48.038 INFO:tasks.workunit.client.1.smithi157.stdout:3/22: dread d0/d3/f6 [0,4194304] 0 2024-04-22T23:34:48.038 INFO:tasks.workunit.client.1.smithi157.stdout:3/23: stat d0/d3/f6 0 2024-04-22T23:34:48.043 INFO:tasks.workunit.client.1.smithi157.stdout:3/24: dread d0/d3/f6 [0,4194304] 0 2024-04-22T23:34:48.045 INFO:tasks.workunit.client.1.smithi157.stdout:4/40: dwrite d3/d4/f9 [0,4194304] 0 2024-04-22T23:34:48.045 INFO:tasks.workunit.client.1.smithi157.stdout:5/55: dwrite d1/fe [0,4194304] 0 2024-04-22T23:34:48.047 INFO:tasks.workunit.client.1.smithi157.stdout:4/41: rename l0 to d3/d4/lb 0 2024-04-22T23:34:48.047 INFO:tasks.workunit.client.1.smithi157.stdout:4/42: readlink d3/l7 0 2024-04-22T23:34:48.050 INFO:tasks.workunit.client.1.smithi157.stdout:6/34: dread f0 [0,4194304] 0 2024-04-22T23:34:48.051 INFO:tasks.workunit.client.1.smithi157.stdout:6/35: symlink d1/l9 0 2024-04-22T23:34:48.051 INFO:tasks.workunit.client.1.smithi157.stdout:6/36: dread - d1/f5 zero size 2024-04-22T23:34:48.052 INFO:tasks.workunit.client.1.smithi157.stdout:6/37: creat d1/fa x:0 0 0 2024-04-22T23:34:48.053 INFO:tasks.workunit.client.1.smithi157.stdout:6/38: symlink d1/lb 0 2024-04-22T23:34:48.062 INFO:tasks.workunit.client.1.smithi157.stdout:5/56: dread d1/fe [0,4194304] 0 2024-04-22T23:34:48.063 INFO:tasks.workunit.client.1.smithi157.stdout:5/57: mknod d1/d5/d7/d10/c11 0 2024-04-22T23:34:48.064 INFO:tasks.workunit.client.1.smithi157.stdout:5/58: creat d1/d5/f12 x:0 0 0 2024-04-22T23:34:48.064 INFO:tasks.workunit.client.1.smithi157.stdout:5/59: stat d1/d5/f9 0 2024-04-22T23:34:48.074 INFO:tasks.workunit.client.1.smithi157.stdout:3/25: dwrite d0/f1 [0,4194304] 0 2024-04-22T23:34:48.075 INFO:tasks.workunit.client.1.smithi157.stdout:6/39: dwrite d1/fa [0,4194304] 0 2024-04-22T23:34:48.075 INFO:tasks.workunit.client.1.smithi157.stdout:4/43: dwrite d3/d4/f9 [0,4194304] 0 2024-04-22T23:34:48.075 INFO:tasks.workunit.client.1.smithi157.stdout:4/44: dread d3/d4/f5 [0,4194304] 0 2024-04-22T23:34:48.076 INFO:tasks.workunit.client.1.smithi157.stdout:3/26: truncate d0/d3/f6 480576 0 2024-04-22T23:34:48.076 INFO:tasks.workunit.client.1.smithi157.stdout:3/27: stat d0/d7 0 2024-04-22T23:34:48.077 INFO:tasks.workunit.client.1.smithi157.stdout:6/40: mknod d1/cc 0 2024-04-22T23:34:48.080 INFO:tasks.workunit.client.1.smithi157.stdout:3/28: symlink d0/l8 0 2024-04-22T23:34:48.081 INFO:tasks.workunit.client.1.smithi157.stdout:6/41: mkdir d1/dd 0 2024-04-22T23:34:48.081 INFO:tasks.workunit.client.1.smithi157.stdout:6/42: write f0 [5088300,36741] 0 2024-04-22T23:34:48.082 INFO:tasks.workunit.client.1.smithi157.stdout:5/60: fsync d1/d5/f12 0 2024-04-22T23:34:48.083 INFO:tasks.workunit.client.1.smithi157.stdout:5/61: unlink d1/l2 0 2024-04-22T23:34:48.088 INFO:tasks.workunit.client.1.smithi157.stdout:4/45: write d3/d4/f9 [728045,87909] 0 2024-04-22T23:34:48.088 INFO:tasks.workunit.client.1.smithi157.stdout:6/43: dread d1/fa [0,4194304] 0 2024-04-22T23:34:48.088 INFO:tasks.workunit.client.1.smithi157.stdout:6/44: creat d1/fe x:0 0 0 2024-04-22T23:34:48.105 INFO:tasks.workunit.client.1.smithi157.stdout:5/62: dwrite d1/fe [0,4194304] 0 2024-04-22T23:34:48.117 INFO:tasks.workunit.client.1.smithi157.stdout:4/46: creat d3/fc x:0 0 0 2024-04-22T23:34:48.117 INFO:tasks.workunit.client.1.smithi157.stdout:4/47: dread - d3/fc zero size 2024-04-22T23:34:48.121 INFO:tasks.workunit.client.1.smithi157.stdout:4/48: dread d3/f8 [0,4194304] 0 2024-04-22T23:34:48.121 INFO:tasks.workunit.client.1.smithi157.stdout:4/49: truncate d3/d4/f5 999026 0 2024-04-22T23:34:48.126 INFO:tasks.workunit.client.1.smithi157.stdout:6/45: symlink d1/lf 0 2024-04-22T23:34:48.126 INFO:tasks.workunit.client.1.smithi157.stdout:6/46: write d1/f7 [905341,994] 0 2024-04-22T23:34:48.127 INFO:tasks.workunit.client.1.smithi157.stdout:6/47: fsync f0 0 2024-04-22T23:34:48.128 INFO:tasks.workunit.client.1.smithi157.stdout:6/48: rename d1 to d1/dd/d10 22 2024-04-22T23:34:48.322 INFO:tasks.workunit.client.1.smithi157.stdout:5/63: link d1/d5/l8 d1/d5/d7/d10/l13 0 2024-04-22T23:34:48.322 INFO:tasks.workunit.client.1.smithi157.stdout:5/64: creat d1/d5/f14 x:0 0 0 2024-04-22T23:34:48.330 INFO:tasks.workunit.client.1.smithi157.stdout:5/65: write d1/fe [503720,31521] 0 2024-04-22T23:34:48.330 INFO:tasks.workunit.client.1.smithi157.stdout:5/66: creat d1/d5/f15 x:0 0 0 2024-04-22T23:34:48.366 INFO:tasks.workunit.client.1.smithi157.stdout:4/50: truncate d3/f8 1345280 0 2024-04-22T23:34:48.368 INFO:tasks.workunit.client.1.smithi157.stdout:6/49: creat d1/dd/f11 x:0 0 0 2024-04-22T23:34:48.369 INFO:tasks.workunit.client.1.smithi157.stdout:5/67: symlink d1/d5/dc/l16 0 2024-04-22T23:34:48.371 INFO:tasks.workunit.client.1.smithi157.stdout:6/50: rename d1/fa to d1/f12 0 2024-04-22T23:34:48.371 INFO:tasks.workunit.client.1.smithi157.stdout:6/51: chown d1/cc 250097907 1 2024-04-22T23:34:48.373 INFO:tasks.workunit.client.1.smithi157.stdout:6/52: link d1/cc d1/dd/c13 0 2024-04-22T23:34:48.373 INFO:tasks.workunit.client.1.smithi157.stdout:6/53: chown d1/cc 34 1 2024-04-22T23:34:48.375 INFO:tasks.workunit.client.1.smithi157.stdout:4/51: getdents d3 0 2024-04-22T23:34:48.377 INFO:tasks.workunit.client.1.smithi157.stdout:6/54: dread d1/f3 [0,4194304] 0 2024-04-22T23:34:48.377 INFO:tasks.workunit.client.1.smithi157.stdout:6/55: truncate d1/fe 1004083 0 2024-04-22T23:34:48.377 INFO:tasks.workunit.client.1.smithi157.stdout:6/56: readlink d1/l9 0 2024-04-22T23:34:48.377 INFO:tasks.workunit.client.1.smithi157.stdout:6/57: write d1/fe [372052,73004] 0 2024-04-22T23:34:48.377 INFO:tasks.workunit.client.1.smithi157.stdout:6/58: read f0 [828118,122202] 0 2024-04-22T23:34:48.377 INFO:tasks.workunit.client.1.smithi157.stdout:6/59: chown d1/l9 26709 1 2024-04-22T23:34:48.377 INFO:tasks.workunit.client.1.smithi157.stdout:6/60: readlink d1/lb 0 2024-04-22T23:34:48.392 INFO:tasks.workunit.client.1.smithi157.stdout:6/61: dwrite d1/f12 [0,4194304] 0 2024-04-22T23:34:48.447 INFO:tasks.workunit.client.1.smithi157.stdout:6/62: truncate d1/f12 3394651 0 2024-04-22T23:34:48.448 INFO:tasks.workunit.client.1.smithi157.stdout:6/63: symlink d1/l14 0 2024-04-22T23:34:48.450 INFO:tasks.workunit.client.1.smithi157.stdout:6/64: symlink d1/dd/l15 0 2024-04-22T23:34:48.450 INFO:tasks.workunit.client.1.smithi157.stdout:6/65: stat d1/dd/l15 0 2024-04-22T23:34:48.450 INFO:tasks.workunit.client.1.smithi157.stdout:6/66: chown d1/f7 10 1 2024-04-22T23:34:48.564 INFO:tasks.workunit.client.1.smithi157.stdout:6/67: dread d1/f12 [0,4194304] 0 2024-04-22T23:34:48.565 INFO:tasks.workunit.client.1.smithi157.stdout:6/68: rename d1/l9 to d1/l16 0 2024-04-22T23:34:48.581 INFO:tasks.workunit.client.1.smithi157.stdout:6/69: dwrite f0 [0,4194304] 0 2024-04-22T23:34:48.582 INFO:tasks.workunit.client.1.smithi157.stdout:6/70: write d1/f7 [608888,121208] 0 2024-04-22T23:34:48.582 INFO:tasks.workunit.client.1.smithi157.stdout:6/71: chown d1/f3 20388597 1 2024-04-22T23:34:48.583 INFO:tasks.workunit.client.1.smithi157.stdout:6/72: unlink d1/fe 0 2024-04-22T23:34:48.583 INFO:tasks.workunit.client.1.smithi157.stdout:6/73: chown d1/cc 4163 1 2024-04-22T23:34:48.584 INFO:tasks.workunit.client.1.smithi157.stdout:6/74: truncate d1/f12 1957083 0 2024-04-22T23:34:48.585 INFO:tasks.workunit.client.1.smithi157.stdout:6/75: mkdir d1/d17 0 2024-04-22T23:34:48.586 INFO:tasks.workunit.client.1.smithi157.stdout:6/76: truncate d1/f5 788139 0 2024-04-22T23:34:48.586 INFO:tasks.workunit.client.1.smithi157.stdout:6/77: creat d1/f18 x:0 0 0 2024-04-22T23:34:48.587 INFO:tasks.workunit.client.1.smithi157.stdout:6/78: symlink d1/l19 0 2024-04-22T23:34:48.588 INFO:tasks.workunit.client.1.smithi157.stdout:6/79: write d1/f7 [1293170,80646] 0 2024-04-22T23:34:48.593 INFO:tasks.workunit.client.1.smithi157.stdout:6/80: write f0 [650555,80870] 0 2024-04-22T23:34:48.598 INFO:tasks.workunit.client.1.smithi157.stdout:6/81: dread f0 [0,4194304] 0 2024-04-22T23:34:48.599 INFO:tasks.workunit.client.1.smithi157.stdout:6/82: rename d1/lf to d1/l1a 0 2024-04-22T23:34:48.600 INFO:tasks.workunit.client.1.smithi157.stdout:6/83: symlink d1/l1b 0 2024-04-22T23:34:48.600 INFO:tasks.workunit.client.1.smithi157.stdout:6/84: mknod d1/dd/c1c 0 2024-04-22T23:34:48.601 INFO:tasks.workunit.client.1.smithi157.stdout:6/85: creat d1/dd/f1d x:0 0 0 2024-04-22T23:34:48.605 INFO:tasks.workunit.client.1.smithi157.stdout:6/86: rmdir d1 39 2024-04-22T23:34:48.605 INFO:tasks.workunit.client.1.smithi157.stdout:6/87: dread - d1/dd/f1d zero size 2024-04-22T23:34:48.606 INFO:tasks.workunit.client.1.smithi157.stdout:6/88: unlink d1/dd/f11 0 2024-04-22T23:34:48.606 INFO:tasks.workunit.client.1.smithi157.stdout:6/89: unlink d1/dd/f1d 0 2024-04-22T23:34:48.607 INFO:tasks.workunit.client.1.smithi157.stdout:6/90: truncate d1/f18 547446 0 2024-04-22T23:34:48.608 INFO:tasks.workunit.client.1.smithi157.stdout:6/91: stat d1/l16 0 2024-04-22T23:34:48.609 INFO:tasks.workunit.client.1.smithi157.stdout:6/92: rename d1/l19 to d1/d17/l1e 0 2024-04-22T23:34:48.610 INFO:tasks.workunit.client.1.smithi157.stdout:6/93: rmdir d1/d17 39 2024-04-22T23:34:48.611 INFO:tasks.workunit.client.1.smithi157.stdout:6/94: mknod d1/d17/c1f 0 2024-04-22T23:34:48.611 INFO:tasks.workunit.client.1.smithi157.stdout:6/95: write d1/f5 [1634088,58392] 0 2024-04-22T23:34:48.624 INFO:tasks.workunit.client.1.smithi157.stdout:6/96: dwrite d1/f7 [0,4194304] 0 2024-04-22T23:34:48.639 INFO:tasks.workunit.client.1.smithi157.stdout:6/97: truncate d1/f7 2075579 0 2024-04-22T23:34:48.639 INFO:tasks.workunit.client.1.smithi157.stdout:6/98: chown d1/f7 511 1 2024-04-22T23:34:48.639 INFO:tasks.workunit.client.1.smithi157.stdout:6/99: chown d1/f5 1 1 2024-04-22T23:34:48.655 INFO:tasks.workunit.client.1.smithi157.stdout:6/100: dwrite f0 [4194304,4194304] 0 2024-04-22T23:34:48.657 INFO:tasks.workunit.client.1.smithi157.stdout:6/101: dread d1/f3 [0,4194304] 0 2024-04-22T23:34:48.658 INFO:tasks.workunit.client.1.smithi157.stdout:6/102: rmdir d1 39 2024-04-22T23:34:48.659 INFO:tasks.workunit.client.1.smithi157.stdout:6/103: link d1/dd/c1c d1/dd/c20 0 2024-04-22T23:34:48.660 INFO:tasks.workunit.client.1.smithi157.stdout:6/104: dread d1/f18 [0,4194304] 0 2024-04-22T23:34:48.673 INFO:tasks.workunit.client.1.smithi157.stdout:6/105: dwrite f0 [0,4194304] 0 2024-04-22T23:34:48.673 INFO:tasks.workunit.client.1.smithi157.stdout:6/106: stat d1/d17/c1f 0 2024-04-22T23:34:48.676 INFO:tasks.workunit.client.1.smithi157.stdout:6/107: dread d1/f3 [0,4194304] 0 2024-04-22T23:34:48.678 INFO:tasks.workunit.client.1.smithi157.stdout:6/108: link f0 d1/d17/f21 0 2024-04-22T23:34:48.684 INFO:tasks.workunit.client.1.smithi157.stdout:6/109: write f0 [1995318,91350] 0 2024-04-22T23:34:48.685 INFO:tasks.workunit.client.1.smithi157.stdout:6/110: unlink d1/dd/c13 0 2024-04-22T23:34:48.685 INFO:tasks.workunit.client.1.smithi157.stdout:6/111: creat d1/f22 x:0 0 0 2024-04-22T23:34:48.686 INFO:tasks.workunit.client.1.smithi157.stdout:6/112: symlink d1/l23 0 2024-04-22T23:34:48.691 INFO:tasks.workunit.client.1.smithi157.stdout:6/113: dread d1/d17/f21 [0,4194304] 0 2024-04-22T23:34:48.693 INFO:tasks.workunit.client.1.smithi157.stdout:6/114: dread d1/f3 [0,4194304] 0 2024-04-22T23:34:48.700 INFO:tasks.workunit.client.1.smithi157.stdout:6/115: truncate d1/d17/f21 6262451 0 2024-04-22T23:34:48.701 INFO:tasks.workunit.client.1.smithi157.stdout:6/116: creat d1/d17/f24 x:0 0 0 2024-04-22T23:34:48.702 INFO:tasks.workunit.client.1.smithi157.stdout:6/117: creat d1/f25 x:0 0 0 2024-04-22T23:34:48.702 INFO:tasks.workunit.client.1.smithi157.stdout:6/118: write d1/d17/f24 [271589,34479] 0 2024-04-22T23:34:48.702 INFO:tasks.workunit.client.1.smithi157.stdout:6/119: chown d1/d17 294878630 1 2024-04-22T23:34:48.704 INFO:tasks.workunit.client.1.smithi157.stdout:6/120: link d1/dd/c20 d1/dd/c26 0 2024-04-22T23:34:48.705 INFO:tasks.workunit.client.1.smithi157.stdout:6/121: link d1/f18 d1/d17/f27 0 2024-04-22T23:34:48.705 INFO:tasks.workunit.client.1.smithi157.stdout:6/122: fsync f0 0 2024-04-22T23:34:48.705 INFO:tasks.workunit.client.1.smithi157.stdout:6/123: creat d1/d17/f28 x:0 0 0 2024-04-22T23:34:48.705 INFO:tasks.workunit.client.1.smithi157.stdout:6/124: stat d1/d17/f28 0 2024-04-22T23:34:48.710 INFO:tasks.workunit.client.1.smithi157.stdout:6/125: dread f0 [4194304,4194304] 0 2024-04-22T23:34:48.710 INFO:tasks.workunit.client.1.smithi157.stdout:6/126: creat d1/f29 x:0 0 0 2024-04-22T23:34:48.730 INFO:tasks.workunit.client.1.smithi157.stdout:6/127: dwrite d1/f29 [0,4194304] 0 2024-04-22T23:34:48.732 INFO:tasks.workunit.client.1.smithi157.stdout:6/128: creat d1/dd/f2a x:0 0 0 2024-04-22T23:34:48.733 INFO:tasks.workunit.client.1.smithi157.stdout:6/129: mknod d1/dd/c2b 0 2024-04-22T23:34:48.733 INFO:tasks.workunit.client.1.smithi157.stdout:6/130: creat d1/f2c x:0 0 0 2024-04-22T23:34:48.733 INFO:tasks.workunit.client.1.smithi157.stdout:6/131: chown d1/f2c 5373914 1 2024-04-22T23:34:48.735 INFO:tasks.workunit.client.1.smithi157.stdout:6/132: symlink d1/d17/l2d 0 2024-04-22T23:34:48.735 INFO:tasks.workunit.client.1.smithi157.stdout:6/133: creat d1/dd/f2e x:0 0 0 2024-04-22T23:34:48.736 INFO:tasks.workunit.client.1.smithi157.stdout:6/134: mkdir d1/d2f 0 2024-04-22T23:34:48.737 INFO:tasks.workunit.client.1.smithi157.stdout:6/135: symlink d1/d2f/l30 0 2024-04-22T23:34:48.737 INFO:tasks.workunit.client.1.smithi157.stdout:6/136: creat d1/d17/f31 x:0 0 0 2024-04-22T23:34:48.737 INFO:tasks.workunit.client.1.smithi157.stdout:6/137: creat d1/dd/f32 x:0 0 0 2024-04-22T23:34:48.737 INFO:tasks.workunit.client.1.smithi157.stdout:6/138: fdatasync d1/f3 0 2024-04-22T23:34:48.757 INFO:tasks.workunit.client.1.smithi157.stdout:6/139: dwrite d1/f22 [0,4194304] 0 2024-04-22T23:34:48.757 INFO:tasks.workunit.client.1.smithi157.stdout:6/140: chown d1/d17/l1e 12 1 2024-04-22T23:34:48.757 INFO:tasks.workunit.client.1.smithi157.stdout:6/141: truncate d1/dd/f2e 245991 0 2024-04-22T23:34:48.757 INFO:tasks.workunit.client.1.smithi157.stdout:6/142: chown d1/l16 16153 1 2024-04-22T23:34:48.757 INFO:tasks.workunit.client.1.smithi157.stdout:6/143: creat d1/d17/f33 x:0 0 0 2024-04-22T23:34:48.757 INFO:tasks.workunit.client.1.smithi157.stdout:6/144: creat d1/f34 x:0 0 0 2024-04-22T23:34:48.758 INFO:tasks.workunit.client.1.smithi157.stdout:6/145: chown d1/lb 7063092 1 2024-04-22T23:34:48.763 INFO:tasks.workunit.client.1.smithi157.stdout:6/146: rmdir d1/d17 39 2024-04-22T23:34:48.764 INFO:tasks.workunit.client.1.smithi157.stdout:6/147: mknod d1/c35 0 2024-04-22T23:34:48.765 INFO:tasks.workunit.client.1.smithi157.stdout:6/148: mkdir d1/d36 0 2024-04-22T23:34:48.766 INFO:tasks.workunit.client.1.smithi157.stdout:6/149: creat d1/d36/f37 x:0 0 0 2024-04-22T23:34:48.783 INFO:tasks.workunit.client.1.smithi157.stdout:6/150: dwrite d1/d17/f28 [0,4194304] 0 2024-04-22T23:34:48.783 INFO:tasks.workunit.client.1.smithi157.stdout:6/151: readlink d1/d17/l2d 0 2024-04-22T23:34:48.783 INFO:tasks.workunit.client.1.smithi157.stdout:6/152: fsync d1/f7 0 2024-04-22T23:34:48.785 INFO:tasks.workunit.client.1.smithi157.stdout:6/153: unlink d1/f25 0 2024-04-22T23:34:48.786 INFO:tasks.workunit.client.1.smithi157.stdout:6/154: dread d1/d17/f27 [0,4194304] 0 2024-04-22T23:34:48.786 INFO:tasks.workunit.client.1.smithi157.stdout:6/155: dread - d1/d17/f31 zero size 2024-04-22T23:34:48.786 INFO:tasks.workunit.client.1.smithi157.stdout:6/156: fsync d1/d36/f37 0 2024-04-22T23:34:48.787 INFO:tasks.workunit.client.1.smithi157.stdout:6/157: mkdir d1/d38 0 2024-04-22T23:34:48.788 INFO:tasks.workunit.client.1.smithi157.stdout:6/158: getdents d1/d17 0 2024-04-22T23:34:48.788 INFO:tasks.workunit.client.1.smithi157.stdout:6/159: write d1/d17/f33 [1037248,45115] 0 2024-04-22T23:34:48.789 INFO:tasks.workunit.client.1.smithi157.stdout:6/160: mkdir d1/dd/d39 0 2024-04-22T23:34:48.790 INFO:tasks.workunit.client.1.smithi157.stdout:6/161: unlink d1/l14 0 2024-04-22T23:34:48.791 INFO:tasks.workunit.client.1.smithi157.stdout:6/162: truncate d1/d17/f33 1206671 0 2024-04-22T23:34:48.791 INFO:tasks.workunit.client.1.smithi157.stdout:6/163: truncate d1/f5 2672071 0 2024-04-22T23:34:48.791 INFO:tasks.workunit.client.1.smithi157.stdout:6/164: mknod d1/d2f/c3a 0 2024-04-22T23:34:48.791 INFO:tasks.workunit.client.1.smithi157.stdout:6/165: write d1/f5 [1886648,109431] 0 2024-04-22T23:34:48.796 INFO:tasks.workunit.client.1.smithi157.stdout:6/166: rename d1/dd to d1/d2f/d3b 0 2024-04-22T23:34:48.796 INFO:tasks.workunit.client.1.smithi157.stdout:6/167: truncate d1/d17/f31 924719 0 2024-04-22T23:34:48.797 INFO:tasks.workunit.client.1.smithi157.stdout:6/168: rename d1/d17/l2d to d1/d36/l3c 0 2024-04-22T23:34:48.798 INFO:tasks.workunit.client.1.smithi157.stdout:6/169: symlink d1/d36/l3d 0 2024-04-22T23:34:48.799 INFO:tasks.workunit.client.1.smithi157.stdout:6/170: creat d1/d2f/d3b/f3e x:0 0 0 2024-04-22T23:34:48.802 INFO:tasks.workunit.client.1.smithi157.stdout:6/171: dread d1/f7 [0,4194304] 0 2024-04-22T23:34:48.818 INFO:tasks.workunit.client.1.smithi157.stdout:6/172: dwrite d1/f18 [0,4194304] 0 2024-04-22T23:34:48.821 INFO:tasks.workunit.client.1.smithi157.stdout:6/173: rename f0 to d1/d36/f3f 0 2024-04-22T23:34:48.821 INFO:tasks.workunit.client.1.smithi157.stdout:6/174: fsync d1/d2f/d3b/f32 0 2024-04-22T23:34:48.830 INFO:tasks.workunit.client.1.smithi157.stdout:6/175: read d1/d17/f28 [1660483,99944] 0 2024-04-22T23:34:48.832 INFO:tasks.workunit.client.1.smithi157.stdout:6/176: truncate d1/d17/f27 3796597 0 2024-04-22T23:34:48.834 INFO:tasks.workunit.client.1.smithi157.stdout:6/177: getdents d1 0 2024-04-22T23:34:48.834 INFO:tasks.workunit.client.1.smithi157.stdout:6/178: write d1/f2c [298887,122540] 0 2024-04-22T23:34:48.834 INFO:tasks.workunit.client.1.smithi157.stdout:6/179: fsync d1/f12 0 2024-04-22T23:34:48.836 INFO:tasks.workunit.client.1.smithi157.stdout:6/180: fsync d1/d17/f33 0 2024-04-22T23:34:48.836 INFO:tasks.workunit.client.1.smithi157.stdout:6/181: fsync d1/f3 0 2024-04-22T23:34:48.838 INFO:tasks.workunit.client.1.smithi157.stdout:6/182: creat d1/d2f/f40 x:0 0 0 2024-04-22T23:34:48.839 INFO:tasks.workunit.client.1.smithi157.stdout:6/183: mknod d1/d36/c41 0 2024-04-22T23:34:48.839 INFO:tasks.workunit.client.1.smithi157.stdout:6/184: fdatasync d1/d36/f3f 0 2024-04-22T23:34:48.840 INFO:tasks.workunit.client.1.smithi157.stdout:6/185: link d1/d2f/l30 d1/d38/l42 0 2024-04-22T23:34:48.840 INFO:tasks.workunit.client.1.smithi157.stdout:6/186: chown d1/cc 43 1 2024-04-22T23:34:48.841 INFO:tasks.workunit.client.1.smithi157.stdout:6/187: link d1/f2c d1/d17/f43 0 2024-04-22T23:34:48.842 INFO:tasks.workunit.client.1.smithi157.stdout:6/188: mknod d1/d36/c44 0 2024-04-22T23:34:48.842 INFO:tasks.workunit.client.1.smithi157.stdout:6/189: fsync d1/d17/f21 0 2024-04-22T23:34:48.842 INFO:tasks.workunit.client.1.smithi157.stdout:6/190: getdents d1 0 2024-04-22T23:34:48.843 INFO:tasks.workunit.client.1.smithi157.stdout:6/191: mkdir d1/d38/d45 0 2024-04-22T23:34:48.843 INFO:tasks.workunit.client.1.smithi157.stdout:6/192: dread - d1/d2f/f40 zero size 2024-04-22T23:34:48.843 INFO:tasks.workunit.client.1.smithi157.stdout:6/193: dread - d1/d2f/d3b/f2a zero size 2024-04-22T23:34:48.843 INFO:tasks.workunit.client.1.smithi157.stdout:6/194: dread - d1/d2f/d3b/f2a zero size 2024-04-22T23:34:48.844 INFO:tasks.workunit.client.1.smithi157.stdout:6/195: creat d1/d2f/d3b/d39/f46 x:0 0 0 2024-04-22T23:34:48.846 INFO:tasks.workunit.client.1.smithi157.stdout:6/196: fdatasync d1/d17/f43 0 2024-04-22T23:34:48.846 INFO:tasks.workunit.client.1.smithi157.stdout:6/197: creat d1/d36/f47 x:0 0 0 2024-04-22T23:34:48.847 INFO:tasks.workunit.client.1.smithi157.stdout:6/198: symlink d1/d2f/l48 0 2024-04-22T23:34:48.847 INFO:tasks.workunit.client.1.smithi157.stdout:6/199: chown d1/d38/d45 27 1 2024-04-22T23:34:49.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:34:49 smithi157 ceph-mon[33877]: pgmap v251: 65 pgs: 65 active+clean; 170 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 9.9 MiB/s wr, 769 op/s 2024-04-22T23:34:49.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:34:49 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:34:49.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:34:49 smithi107 ceph-mon[26355]: pgmap v251: 65 pgs: 65 active+clean; 170 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 9.9 MiB/s wr, 769 op/s 2024-04-22T23:34:49.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:34:49 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:34:50.713 INFO:tasks.workunit.client.1.smithi157.stdout:8/6: sync 2024-04-22T23:34:50.713 INFO:tasks.workunit.client.1.smithi157.stdout:9/1: sync 2024-04-22T23:34:50.713 INFO:tasks.workunit.client.1.smithi157.stdout:1/7: sync 2024-04-22T23:34:50.713 INFO:tasks.workunit.client.1.smithi157.stdout:9/2: readlink - no filename 2024-04-22T23:34:50.713 INFO:tasks.workunit.client.1.smithi157.stdout:9/3: dread - no filename 2024-04-22T23:34:50.714 INFO:tasks.workunit.client.1.smithi157.stdout:8/7: creat f0 x:0 0 0 2024-04-22T23:34:50.714 INFO:tasks.workunit.client.1.smithi157.stdout:8/8: write f0 [30238,83807] 0 2024-04-22T23:34:50.714 INFO:tasks.workunit.client.1.smithi157.stdout:8/9: truncate f0 855141 0 2024-04-22T23:34:50.715 INFO:tasks.workunit.client.1.smithi157.stdout:9/4: mknod c0 0 2024-04-22T23:34:50.715 INFO:tasks.workunit.client.1.smithi157.stdout:9/5: fdatasync - no filename 2024-04-22T23:34:50.717 INFO:tasks.workunit.client.1.smithi157.stdout:9/6: unlink c0 0 2024-04-22T23:34:50.717 INFO:tasks.workunit.client.1.smithi157.stdout:9/7: stat - no entries 2024-04-22T23:34:50.717 INFO:tasks.workunit.client.1.smithi157.stdout:9/8: dwrite - no filename 2024-04-22T23:34:50.717 INFO:tasks.workunit.client.1.smithi157.stdout:9/9: unlink - no file 2024-04-22T23:34:50.717 INFO:tasks.workunit.client.1.smithi157.stdout:9/10: write - no filename 2024-04-22T23:34:50.717 INFO:tasks.workunit.client.1.smithi157.stdout:9/11: link - no file 2024-04-22T23:34:50.717 INFO:tasks.workunit.client.1.smithi157.stdout:9/12: readlink - no filename 2024-04-22T23:34:50.717 INFO:tasks.workunit.client.1.smithi157.stdout:9/13: truncate - no filename 2024-04-22T23:34:50.718 INFO:tasks.workunit.client.1.smithi157.stdout:9/14: creat f1 x:0 0 0 2024-04-22T23:34:50.718 INFO:tasks.workunit.client.1.smithi157.stdout:9/15: truncate f1 342847 0 2024-04-22T23:34:50.719 INFO:tasks.workunit.client.1.smithi157.stdout:9/16: unlink f1 0 2024-04-22T23:34:50.719 INFO:tasks.workunit.client.1.smithi157.stdout:9/17: link - no file 2024-04-22T23:34:50.720 INFO:tasks.workunit.client.1.smithi157.stdout:8/10: write f0 [686444,84438] 0 2024-04-22T23:34:50.721 INFO:tasks.workunit.client.1.smithi157.stdout:9/18: symlink l2 0 2024-04-22T23:34:50.721 INFO:tasks.workunit.client.1.smithi157.stdout:9/19: dread - no filename 2024-04-22T23:34:50.722 INFO:tasks.workunit.client.1.smithi157.stdout:9/20: unlink l2 0 2024-04-22T23:34:50.722 INFO:tasks.workunit.client.1.smithi157.stdout:9/21: stat - no entries 2024-04-22T23:34:50.722 INFO:tasks.workunit.client.1.smithi157.stdout:9/22: rename - no filename 2024-04-22T23:34:50.722 INFO:tasks.workunit.client.1.smithi157.stdout:9/23: stat - no entries 2024-04-22T23:34:50.729 INFO:tasks.workunit.client.1.smithi157.stdout:9/24: getdents . 0 2024-04-22T23:34:50.729 INFO:tasks.workunit.client.1.smithi157.stdout:9/25: rmdir - no directory 2024-04-22T23:34:50.729 INFO:tasks.workunit.client.1.smithi157.stdout:9/26: fdatasync - no filename 2024-04-22T23:34:50.730 INFO:tasks.workunit.client.1.smithi157.stdout:9/27: creat f3 x:0 0 0 2024-04-22T23:34:50.731 INFO:tasks.workunit.client.1.smithi157.stdout:9/28: mkdir d4 0 2024-04-22T23:34:50.732 INFO:tasks.workunit.client.1.smithi157.stdout:8/11: dread f0 [0,4194304] 0 2024-04-22T23:34:50.732 INFO:tasks.workunit.client.1.smithi157.stdout:9/29: write f3 [247198,113476] 0 2024-04-22T23:34:50.732 INFO:tasks.workunit.client.1.smithi157.stdout:8/12: rename f0 to f1 0 2024-04-22T23:34:50.732 INFO:tasks.workunit.client.1.smithi157.stdout:8/13: readlink - no filename 2024-04-22T23:34:50.733 INFO:tasks.workunit.client.1.smithi157.stdout:9/30: symlink d4/l5 0 2024-04-22T23:34:50.736 INFO:tasks.workunit.client.1.smithi157.stdout:1/8: dwrite f1 [0,4194304] 0 2024-04-22T23:34:50.745 INFO:tasks.workunit.client.1.smithi157.stdout:9/31: dread f3 [0,4194304] 0 2024-04-22T23:34:50.748 INFO:tasks.workunit.client.1.smithi157.stdout:9/32: dread f3 [0,4194304] 0 2024-04-22T23:34:50.748 INFO:tasks.workunit.client.1.smithi157.stdout:9/33: getdents d4 0 2024-04-22T23:34:50.750 INFO:tasks.workunit.client.1.smithi157.stdout:9/34: symlink d4/l6 0 2024-04-22T23:34:50.755 INFO:tasks.workunit.client.1.smithi157.stdout:9/35: getdents d4 0 2024-04-22T23:34:50.755 INFO:tasks.workunit.client.1.smithi157.stdout:8/14: dwrite f1 [0,4194304] 0 2024-04-22T23:34:50.755 INFO:tasks.workunit.client.1.smithi157.stdout:9/36: write f3 [1347011,92876] 0 2024-04-22T23:34:50.757 INFO:tasks.workunit.client.1.smithi157.stdout:9/37: mknod d4/c7 0 2024-04-22T23:34:50.758 INFO:tasks.workunit.client.1.smithi157.stdout:9/38: creat d4/f8 x:0 0 0 2024-04-22T23:34:50.759 INFO:tasks.workunit.client.1.smithi157.stdout:9/39: chown d4/l5 7913584 1 2024-04-22T23:34:50.761 INFO:tasks.workunit.client.1.smithi157.stdout:1/9: dwrite f1 [0,4194304] 0 2024-04-22T23:34:50.761 INFO:tasks.workunit.client.1.smithi157.stdout:1/10: creat f2 x:0 0 0 2024-04-22T23:34:50.761 INFO:tasks.workunit.client.1.smithi157.stdout:1/11: read - f2 zero size 2024-04-22T23:34:50.761 INFO:tasks.workunit.client.1.smithi157.stdout:9/40: mkdir d4/d9 0 2024-04-22T23:34:50.765 INFO:tasks.workunit.client.1.smithi157.stdout:9/41: symlink d4/la 0 2024-04-22T23:34:50.765 INFO:tasks.workunit.client.1.smithi157.stdout:8/15: write f1 [651198,4808] 0 2024-04-22T23:34:50.765 INFO:tasks.workunit.client.1.smithi157.stdout:9/42: write f3 [1861620,51709] 0 2024-04-22T23:34:50.766 INFO:tasks.workunit.client.1.smithi157.stdout:8/16: mkdir d2 0 2024-04-22T23:34:50.766 INFO:tasks.workunit.client.1.smithi157.stdout:9/43: creat d4/d9/fb x:0 0 0 2024-04-22T23:34:50.767 INFO:tasks.workunit.client.1.smithi157.stdout:9/44: chown d4/d9/fb 9844 1 2024-04-22T23:34:50.772 INFO:tasks.workunit.client.1.smithi157.stdout:1/12: read f1 [204630,6913] 0 2024-04-22T23:34:50.773 INFO:tasks.workunit.client.1.smithi157.stdout:8/17: fsync f1 0 2024-04-22T23:34:50.775 INFO:tasks.workunit.client.1.smithi157.stdout:1/13: rename f0 to f3 0 2024-04-22T23:34:50.786 INFO:tasks.workunit.client.1.smithi157.stdout:9/45: dwrite d4/d9/fb [0,4194304] 0 2024-04-22T23:34:50.786 INFO:tasks.workunit.client.1.smithi157.stdout:9/46: chown d4/l5 1196139220 1 2024-04-22T23:34:50.788 INFO:tasks.workunit.client.1.smithi157.stdout:1/14: write f3 [830425,56062] 0 2024-04-22T23:34:50.790 INFO:tasks.workunit.client.1.smithi157.stdout:1/15: mkdir d4 0 2024-04-22T23:34:50.790 INFO:tasks.workunit.client.1.smithi157.stdout:1/16: readlink - no filename 2024-04-22T23:34:50.790 INFO:tasks.workunit.client.1.smithi157.stdout:1/17: read - f2 zero size 2024-04-22T23:34:50.790 INFO:tasks.workunit.client.1.smithi157.stdout:4/52: sync 2024-04-22T23:34:50.791 INFO:tasks.workunit.client.1.smithi157.stdout:2/24: sync 2024-04-22T23:34:50.791 INFO:tasks.workunit.client.1.smithi157.stdout:3/29: sync 2024-04-22T23:34:50.791 INFO:tasks.workunit.client.1.smithi157.stdout:6/200: sync 2024-04-22T23:34:50.791 INFO:tasks.workunit.client.1.smithi157.stdout:7/30: sync 2024-04-22T23:34:50.791 INFO:tasks.workunit.client.1.smithi157.stdout:2/25: fsync f3 0 2024-04-22T23:34:50.791 INFO:tasks.workunit.client.1.smithi157.stdout:0/10: sync 2024-04-22T23:34:50.791 INFO:tasks.workunit.client.1.smithi157.stdout:4/53: truncate d3/fc 346048 0 2024-04-22T23:34:50.791 INFO:tasks.workunit.client.1.smithi157.stdout:2/26: dread - d7/d8/f9 zero size 2024-04-22T23:34:50.791 INFO:tasks.workunit.client.1.smithi157.stdout:0/11: dread - f3 zero size 2024-04-22T23:34:50.791 INFO:tasks.workunit.client.1.smithi157.stdout:2/27: fdatasync f0 0 2024-04-22T23:34:50.791 INFO:tasks.workunit.client.1.smithi157.stdout:4/54: write d3/d4/f5 [1294210,27898] 0 2024-04-22T23:34:50.791 INFO:tasks.workunit.client.1.smithi157.stdout:4/55: chown d3/l7 496346 1 2024-04-22T23:34:50.791 INFO:tasks.workunit.client.1.smithi157.stdout:5/68: sync 2024-04-22T23:34:50.792 INFO:tasks.workunit.client.1.smithi157.stdout:1/18: unlink f2 0 2024-04-22T23:34:50.795 INFO:tasks.workunit.client.1.smithi157.stdout:6/201: rename d1/d38/d45 to d1/d2f/d3b/d49 0 2024-04-22T23:34:50.795 INFO:tasks.workunit.client.1.smithi157.stdout:6/202: creat d1/d2f/f4a x:0 0 0 2024-04-22T23:34:50.795 INFO:tasks.workunit.client.1.smithi157.stdout:6/203: stat d1/f7 0 2024-04-22T23:34:50.796 INFO:tasks.workunit.client.1.smithi157.stdout:2/28: creat d7/fa x:0 0 0 2024-04-22T23:34:50.796 INFO:tasks.workunit.client.1.smithi157.stdout:2/29: chown f5 182 1 2024-04-22T23:34:50.796 INFO:tasks.workunit.client.1.smithi157.stdout:4/56: symlink d3/ld 0 2024-04-22T23:34:50.797 INFO:tasks.workunit.client.1.smithi157.stdout:4/57: fdatasync d3/fc 0 2024-04-22T23:34:50.797 INFO:tasks.workunit.client.1.smithi157.stdout:4/58: read d3/f8 [92551,55509] 0 2024-04-22T23:34:50.799 INFO:tasks.workunit.client.1.smithi157.stdout:5/69: symlink d1/d5/d7/l17 0 2024-04-22T23:34:50.799 INFO:tasks.workunit.client.1.smithi157.stdout:1/19: creat d4/f5 x:0 0 0 2024-04-22T23:34:50.800 INFO:tasks.workunit.client.1.smithi157.stdout:5/70: dread d1/d5/f9 [0,4194304] 0 2024-04-22T23:34:50.801 INFO:tasks.workunit.client.1.smithi157.stdout:0/12: write f0 [3259703,103188] 0 2024-04-22T23:34:50.801 INFO:tasks.workunit.client.1.smithi157.stdout:6/204: mknod d1/d2f/d3b/d49/c4b 0 2024-04-22T23:34:50.801 INFO:tasks.workunit.client.1.smithi157.stdout:0/13: read - f3 zero size 2024-04-22T23:34:50.801 INFO:tasks.workunit.client.1.smithi157.stdout:0/14: creat f4 x:0 0 0 2024-04-22T23:34:50.801 INFO:tasks.workunit.client.1.smithi157.stdout:0/15: chown f0 111476267 1 2024-04-22T23:34:50.802 INFO:tasks.workunit.client.1.smithi157.stdout:2/30: symlink d7/d8/lb 0 2024-04-22T23:34:50.802 INFO:tasks.workunit.client.1.smithi157.stdout:2/31: chown d7/d8/f9 326985795 1 2024-04-22T23:34:50.802 INFO:tasks.workunit.client.1.smithi157.stdout:2/32: write d7/fa [236333,60036] 0 2024-04-22T23:34:50.802 INFO:tasks.workunit.client.1.smithi157.stdout:2/33: fsync f5 0 2024-04-22T23:34:50.803 INFO:tasks.workunit.client.1.smithi157.stdout:4/59: truncate d3/f8 516983 0 2024-04-22T23:34:50.803 INFO:tasks.workunit.client.1.smithi157.stdout:3/30: dread d0/f1 [0,4194304] 0 2024-04-22T23:34:50.803 INFO:tasks.workunit.client.1.smithi157.stdout:1/20: mknod d4/c6 0 2024-04-22T23:34:50.805 INFO:tasks.workunit.client.1.smithi157.stdout:5/71: mknod d1/c18 0 2024-04-22T23:34:50.805 INFO:tasks.workunit.client.1.smithi157.stdout:5/72: fsync d1/d5/f12 0 2024-04-22T23:34:50.805 INFO:tasks.workunit.client.1.smithi157.stdout:6/205: dread d1/f12 [0,4194304] 0 2024-04-22T23:34:50.807 INFO:tasks.workunit.client.1.smithi157.stdout:3/31: symlink d0/l9 0 2024-04-22T23:34:50.809 INFO:tasks.workunit.client.1.smithi157.stdout:5/73: mknod d1/d5/d7/c19 0 2024-04-22T23:34:50.810 INFO:tasks.workunit.client.1.smithi157.stdout:9/47: dwrite f3 [0,4194304] 0 2024-04-22T23:34:50.811 INFO:tasks.workunit.client.1.smithi157.stdout:0/16: unlink f4 0 2024-04-22T23:34:50.811 INFO:tasks.workunit.client.1.smithi157.stdout:0/17: truncate f3 991123 0 2024-04-22T23:34:50.811 INFO:tasks.workunit.client.1.smithi157.stdout:0/18: fsync f3 0 2024-04-22T23:34:50.811 INFO:tasks.workunit.client.1.smithi157.stdout:0/19: stat l2 0 2024-04-22T23:34:50.811 INFO:tasks.workunit.client.1.smithi157.stdout:0/20: read f0 [2759399,104752] 0 2024-04-22T23:34:50.811 INFO:tasks.workunit.client.1.smithi157.stdout:0/21: fsync f0 0 2024-04-22T23:34:50.811 INFO:tasks.workunit.client.1.smithi157.stdout:0/22: rmdir - no directory 2024-04-22T23:34:50.811 INFO:tasks.workunit.client.1.smithi157.stdout:0/23: creat f5 x:0 0 0 2024-04-22T23:34:50.811 INFO:tasks.workunit.client.1.smithi157.stdout:7/31: dwrite d0/f2 [0,4194304] 0 2024-04-22T23:34:50.812 INFO:tasks.workunit.client.1.smithi157.stdout:3/32: link d0/d3/f6 d0/d7/fa 0 2024-04-22T23:34:50.814 INFO:tasks.workunit.client.1.smithi157.stdout:5/74: symlink d1/d5/dc/l1a 0 2024-04-22T23:34:50.814 INFO:tasks.workunit.client.1.smithi157.stdout:5/75: read d1/fe [3435408,130710] 0 2024-04-22T23:34:50.814 INFO:tasks.workunit.client.1.smithi157.stdout:5/76: read - d1/d5/f12 zero size 2024-04-22T23:34:50.815 INFO:tasks.workunit.client.1.smithi157.stdout:5/77: write d1/d5/f15 [502436,71325] 0 2024-04-22T23:34:50.815 INFO:tasks.workunit.client.1.smithi157.stdout:5/78: chown d1/d5/d7/d10/c11 374 1 2024-04-22T23:34:50.818 INFO:tasks.workunit.client.1.smithi157.stdout:6/206: read d1/f22 [135257,94159] 0 2024-04-22T23:34:50.823 INFO:tasks.workunit.client.1.smithi157.stdout:9/48: creat d4/fc x:0 0 0 2024-04-22T23:34:50.823 INFO:tasks.workunit.client.1.smithi157.stdout:9/49: stat d4/l5 0 2024-04-22T23:34:50.825 INFO:tasks.workunit.client.1.smithi157.stdout:7/32: dwrite d0/f8 [0,4194304] 0 2024-04-22T23:34:50.829 INFO:tasks.workunit.client.1.smithi157.stdout:1/21: rmdir d4 39 2024-04-22T23:34:50.830 INFO:tasks.workunit.client.1.smithi157.stdout:6/207: stat d1/cc 0 2024-04-22T23:34:50.833 INFO:tasks.workunit.client.1.smithi157.stdout:1/22: rename d4/c6 to d4/c7 0 2024-04-22T23:34:50.833 INFO:tasks.workunit.client.1.smithi157.stdout:1/23: stat f1 0 2024-04-22T23:34:50.835 INFO:tasks.workunit.client.1.smithi157.stdout:6/208: mkdir d1/d2f/d3b/d49/d4c 0 2024-04-22T23:34:50.835 INFO:tasks.workunit.client.1.smithi157.stdout:6/209: creat d1/d36/f4d x:0 0 0 2024-04-22T23:34:50.835 INFO:tasks.workunit.client.1.smithi157.stdout:1/24: rmdir d4 39 2024-04-22T23:34:50.837 INFO:tasks.workunit.client.1.smithi157.stdout:1/25: creat d4/f8 x:0 0 0 2024-04-22T23:34:50.837 INFO:tasks.workunit.client.1.smithi157.stdout:1/26: readlink - no filename 2024-04-22T23:34:50.837 INFO:tasks.workunit.client.1.smithi157.stdout:1/27: chown f3 244968 1 2024-04-22T23:34:50.837 INFO:tasks.workunit.client.1.smithi157.stdout:1/28: chown d4/f5 415594 1 2024-04-22T23:34:50.838 INFO:tasks.workunit.client.1.smithi157.stdout:1/29: write f1 [2216143,69327] 0 2024-04-22T23:34:50.838 INFO:tasks.workunit.client.1.smithi157.stdout:6/210: write d1/f12 [884038,52558] 0 2024-04-22T23:34:50.838 INFO:tasks.workunit.client.1.smithi157.stdout:1/30: rename d4/f8 to d4/f9 0 2024-04-22T23:34:50.838 INFO:tasks.workunit.client.1.smithi157.stdout:1/31: dread - d4/f5 zero size 2024-04-22T23:34:50.839 INFO:tasks.workunit.client.1.smithi157.stdout:6/211: mkdir d1/d2f/d3b/d39/d4e 0 2024-04-22T23:34:50.839 INFO:tasks.workunit.client.1.smithi157.stdout:6/212: getdents d1/d2f/d3b 0 2024-04-22T23:34:50.840 INFO:tasks.workunit.client.1.smithi157.stdout:6/213: write d1/d2f/d3b/d39/f46 [836718,108667] 0 2024-04-22T23:34:50.840 INFO:tasks.workunit.client.1.smithi157.stdout:6/214: mknod d1/d38/c4f 0 2024-04-22T23:34:50.841 INFO:tasks.workunit.client.1.smithi157.stdout:0/24: dwrite f5 [0,4194304] 0 2024-04-22T23:34:50.841 INFO:tasks.workunit.client.1.smithi157.stdout:0/25: rmdir - no directory 2024-04-22T23:34:50.841 INFO:tasks.workunit.client.1.smithi157.stdout:6/215: creat d1/d38/f50 x:0 0 0 2024-04-22T23:34:50.842 INFO:tasks.workunit.client.1.smithi157.stdout:1/32: dread f3 [0,4194304] 0 2024-04-22T23:34:50.845 INFO:tasks.workunit.client.1.smithi157.stdout:6/216: rmdir d1/d17 39 2024-04-22T23:34:50.845 INFO:tasks.workunit.client.1.smithi157.stdout:3/33: dwrite d0/d3/f6 [0,4194304] 0 2024-04-22T23:34:50.845 INFO:tasks.workunit.client.1.smithi157.stdout:3/34: stat d0/d3 0 2024-04-22T23:34:50.845 INFO:tasks.workunit.client.1.smithi157.stdout:0/26: symlink l6 0 2024-04-22T23:34:50.845 INFO:tasks.workunit.client.1.smithi157.stdout:7/33: dwrite d0/f2 [0,4194304] 0 2024-04-22T23:34:50.845 INFO:tasks.workunit.client.1.smithi157.stdout:4/60: dread d3/f8 [0,4194304] 0 2024-04-22T23:34:50.845 INFO:tasks.workunit.client.1.smithi157.stdout:4/61: write d3/f8 [1218351,116262] 0 2024-04-22T23:34:50.845 INFO:tasks.workunit.client.1.smithi157.stdout:4/62: creat d3/d4/fe x:0 0 0 2024-04-22T23:34:50.846 INFO:tasks.workunit.client.1.smithi157.stdout:3/35: mknod d0/cb 0 2024-04-22T23:34:50.846 INFO:tasks.workunit.client.1.smithi157.stdout:3/36: stat d0/l8 0 2024-04-22T23:34:50.846 INFO:tasks.workunit.client.1.smithi157.stdout:3/37: fsync d0/d3/f6 0 2024-04-22T23:34:50.848 INFO:tasks.workunit.client.1.smithi157.stdout:0/27: rename l6 to l7 0 2024-04-22T23:34:50.848 INFO:tasks.workunit.client.1.smithi157.stdout:0/28: creat f8 x:0 0 0 2024-04-22T23:34:50.851 INFO:tasks.workunit.client.1.smithi157.stdout:4/63: getdents d3 0 2024-04-22T23:34:50.851 INFO:tasks.workunit.client.1.smithi157.stdout:4/64: creat d3/d4/ff x:0 0 0 2024-04-22T23:34:50.852 INFO:tasks.workunit.client.1.smithi157.stdout:4/65: dread d3/fc [0,4194304] 0 2024-04-22T23:34:50.852 INFO:tasks.workunit.client.1.smithi157.stdout:4/66: chown d3/fc 559 1 2024-04-22T23:34:50.853 INFO:tasks.workunit.client.1.smithi157.stdout:3/38: mknod d0/d3/cc 0 2024-04-22T23:34:50.857 INFO:tasks.workunit.client.1.smithi157.stdout:7/34: rmdir d0 39 2024-04-22T23:34:50.861 INFO:tasks.workunit.client.1.smithi157.stdout:1/33: dwrite d4/f5 [0,4194304] 0 2024-04-22T23:34:50.865 INFO:tasks.workunit.client.1.smithi157.stdout:3/39: write d0/d3/f6 [3543073,79537] 0 2024-04-22T23:34:50.866 INFO:tasks.workunit.client.1.smithi157.stdout:3/40: chown d0/f1 63 1 2024-04-22T23:34:50.869 INFO:tasks.workunit.client.1.smithi157.stdout:7/35: dread d0/f8 [0,4194304] 0 2024-04-22T23:34:50.869 INFO:tasks.workunit.client.1.smithi157.stdout:4/67: dwrite d3/fc [0,4194304] 0 2024-04-22T23:34:50.869 INFO:tasks.workunit.client.1.smithi157.stdout:4/68: creat d3/d4/f10 x:0 0 0 2024-04-22T23:34:50.869 INFO:tasks.workunit.client.1.smithi157.stdout:4/69: fdatasync d3/d4/f5 0 2024-04-22T23:34:50.869 INFO:tasks.workunit.client.1.smithi157.stdout:0/29: getdents . 0 2024-04-22T23:34:50.873 INFO:tasks.workunit.client.1.smithi157.stdout:7/36: truncate d0/f8 4038310 0 2024-04-22T23:34:50.873 INFO:tasks.workunit.client.1.smithi157.stdout:7/37: write d0/f2 [4803859,101165] 0 2024-04-22T23:34:50.876 INFO:tasks.workunit.client.1.smithi157.stdout:7/38: fdatasync d0/f2 0 2024-04-22T23:34:50.877 INFO:tasks.workunit.client.1.smithi157.stdout:4/70: mkdir d3/d11 0 2024-04-22T23:34:50.877 INFO:tasks.workunit.client.1.smithi157.stdout:4/71: chown d3/d4/ca 95255141 1 2024-04-22T23:34:50.877 INFO:tasks.workunit.client.1.smithi157.stdout:7/39: write d0/f4 [349083,74641] 0 2024-04-22T23:34:50.877 INFO:tasks.workunit.client.1.smithi157.stdout:7/40: readlink d0/l3 0 2024-04-22T23:34:50.877 INFO:tasks.workunit.client.1.smithi157.stdout:1/34: truncate d4/f5 2082459 0 2024-04-22T23:34:50.884 INFO:tasks.workunit.client.1.smithi157.stdout:7/41: dread d0/f2 [0,4194304] 0 2024-04-22T23:34:50.886 INFO:tasks.workunit.client.1.smithi157.stdout:7/42: symlink d0/d9/la 0 2024-04-22T23:34:50.886 INFO:tasks.workunit.client.1.smithi157.stdout:7/43: write d0/f4 [1241098,44673] 0 2024-04-22T23:34:50.888 INFO:tasks.workunit.client.1.smithi157.stdout:3/41: dwrite d0/d3/f6 [0,4194304] 0 2024-04-22T23:34:50.888 INFO:tasks.workunit.client.1.smithi157.stdout:3/42: write d0/d7/fa [4729987,94475] 0 2024-04-22T23:34:50.890 INFO:tasks.workunit.client.1.smithi157.stdout:3/43: creat d0/d3/fd x:0 0 0 2024-04-22T23:34:50.890 INFO:tasks.workunit.client.1.smithi157.stdout:1/35: write d4/f5 [525252,48808] 0 2024-04-22T23:34:50.891 INFO:tasks.workunit.client.1.smithi157.stdout:3/44: unlink d0/d3/cc 0 2024-04-22T23:34:50.892 INFO:tasks.workunit.client.1.smithi157.stdout:1/36: mknod d4/ca 0 2024-04-22T23:34:50.893 INFO:tasks.workunit.client.1.smithi157.stdout:3/45: link d0/d7/fa d0/d3/fe 0 2024-04-22T23:34:50.894 INFO:tasks.workunit.client.1.smithi157.stdout:1/37: mkdir d4/db 0 2024-04-22T23:34:50.894 INFO:tasks.workunit.client.1.smithi157.stdout:1/38: getdents d4/db 0 2024-04-22T23:34:50.894 INFO:tasks.workunit.client.1.smithi157.stdout:1/39: write d4/f5 [2367844,117345] 0 2024-04-22T23:34:50.894 INFO:tasks.workunit.client.1.smithi157.stdout:1/40: write d4/f9 [433198,57442] 0 2024-04-22T23:34:50.897 INFO:tasks.workunit.client.1.smithi157.stdout:4/72: dwrite d3/f8 [0,4194304] 0 2024-04-22T23:34:50.897 INFO:tasks.workunit.client.1.smithi157.stdout:4/73: dread - d3/d4/ff zero size 2024-04-22T23:34:50.898 INFO:tasks.workunit.client.1.smithi157.stdout:4/74: mknod d3/d11/c12 0 2024-04-22T23:34:50.901 INFO:tasks.workunit.client.1.smithi157.stdout:1/41: getdents d4 0 2024-04-22T23:34:50.902 INFO:tasks.workunit.client.1.smithi157.stdout:1/42: creat d4/fc x:0 0 0 2024-04-22T23:34:50.904 INFO:tasks.workunit.client.1.smithi157.stdout:1/43: creat d4/db/fd x:0 0 0 2024-04-22T23:34:50.904 INFO:tasks.workunit.client.1.smithi157.stdout:1/44: dread - d4/db/fd zero size 2024-04-22T23:34:50.904 INFO:tasks.workunit.client.1.smithi157.stdout:1/45: stat d4/db 0 2024-04-22T23:34:50.906 INFO:tasks.workunit.client.1.smithi157.stdout:4/75: write d3/fc [1594093,108960] 0 2024-04-22T23:34:50.907 INFO:tasks.workunit.client.1.smithi157.stdout:3/46: dwrite d0/f1 [4194304,4194304] 0 2024-04-22T23:34:50.910 INFO:tasks.workunit.client.1.smithi157.stdout:3/47: link d0/l9 d0/d3/lf 0 2024-04-22T23:34:50.919 INFO:tasks.workunit.client.1.smithi157.stdout:3/48: dread d0/d3/f6 [4194304,4194304] 0 2024-04-22T23:34:50.919 INFO:tasks.workunit.client.1.smithi157.stdout:3/49: creat d0/f10 x:0 0 0 2024-04-22T23:34:50.921 INFO:tasks.workunit.client.1.smithi157.stdout:4/76: write d3/f8 [3221954,103925] 0 2024-04-22T23:34:50.921 INFO:tasks.workunit.client.1.smithi157.stdout:7/44: dwrite d0/f4 [0,4194304] 0 2024-04-22T23:34:50.923 INFO:tasks.workunit.client.1.smithi157.stdout:4/77: creat d3/f13 x:0 0 0 2024-04-22T23:34:50.923 INFO:tasks.workunit.client.1.smithi157.stdout:4/78: write d3/d4/fe [41570,115051] 0 2024-04-22T23:34:50.923 INFO:tasks.workunit.client.1.smithi157.stdout:4/79: stat d3/d4/ff 0 2024-04-22T23:34:50.924 INFO:tasks.workunit.client.1.smithi157.stdout:4/80: mknod d3/d4/c14 0 2024-04-22T23:34:50.925 INFO:tasks.workunit.client.1.smithi157.stdout:1/46: dwrite f1 [0,4194304] 0 2024-04-22T23:34:50.925 INFO:tasks.workunit.client.1.smithi157.stdout:1/47: creat d4/db/fe x:0 0 0 2024-04-22T23:34:50.933 INFO:tasks.workunit.client.1.smithi157.stdout:7/45: dread d0/f4 [0,4194304] 0 2024-04-22T23:34:50.940 INFO:tasks.workunit.client.1.smithi157.stdout:3/50: write d0/d3/fe [724705,20065] 0 2024-04-22T23:34:50.940 INFO:tasks.workunit.client.1.smithi157.stdout:3/51: read d0/d7/fa [2731651,45219] 0 2024-04-22T23:34:50.940 INFO:tasks.workunit.client.1.smithi157.stdout:3/52: write d0/d3/fd [510546,30892] 0 2024-04-22T23:34:50.945 INFO:tasks.workunit.client.1.smithi157.stdout:3/53: unlink d0/l8 0 2024-04-22T23:34:50.950 INFO:tasks.workunit.client.1.smithi157.stdout:4/81: dwrite d3/d4/fe [0,4194304] 0 2024-04-22T23:34:50.953 INFO:tasks.workunit.client.1.smithi157.stdout:1/48: dwrite d4/f5 [0,4194304] 0 2024-04-22T23:34:50.953 INFO:tasks.workunit.client.1.smithi157.stdout:1/49: chown d4/db/fd 7 1 2024-04-22T23:34:50.954 INFO:tasks.workunit.client.1.smithi157.stdout:1/50: chown d4/c7 3603217 1 2024-04-22T23:34:50.956 INFO:tasks.workunit.client.1.smithi157.stdout:1/51: symlink d4/db/lf 0 2024-04-22T23:34:50.956 INFO:tasks.workunit.client.1.smithi157.stdout:1/52: readlink d4/db/lf 0 2024-04-22T23:34:50.957 INFO:tasks.workunit.client.1.smithi157.stdout:1/53: getdents d4 0 2024-04-22T23:34:50.959 INFO:tasks.workunit.client.1.smithi157.stdout:3/54: rmdir d0 39 2024-04-22T23:34:50.961 INFO:tasks.workunit.client.1.smithi157.stdout:3/55: rename d0/f10 to d0/f11 0 2024-04-22T23:34:50.961 INFO:tasks.workunit.client.1.smithi157.stdout:3/56: write d0/d3/fe [2472317,25433] 0 2024-04-22T23:34:50.966 INFO:tasks.workunit.client.1.smithi157.stdout:3/57: dread d0/f1 [0,4194304] 0 2024-04-22T23:34:50.966 INFO:tasks.workunit.client.1.smithi157.stdout:3/58: readlink d0/l9 0 2024-04-22T23:34:50.969 INFO:tasks.workunit.client.1.smithi157.stdout:3/59: fsync d0/d7/fa 0 2024-04-22T23:34:50.971 INFO:tasks.workunit.client.1.smithi157.stdout:1/54: dwrite d4/f5 [0,4194304] 0 2024-04-22T23:34:50.973 INFO:tasks.workunit.client.1.smithi157.stdout:1/55: getdents d4/db 0 2024-04-22T23:34:50.973 INFO:tasks.workunit.client.1.smithi157.stdout:1/56: write d4/f9 [1113785,22114] 0 2024-04-22T23:34:50.973 INFO:tasks.workunit.client.1.smithi157.stdout:1/57: chown f1 1111770064 1 2024-04-22T23:34:50.973 INFO:tasks.workunit.client.1.smithi157.stdout:1/58: readlink d4/db/lf 0 2024-04-22T23:34:50.973 INFO:tasks.workunit.client.1.smithi157.stdout:1/59: symlink d4/l10 0 2024-04-22T23:34:50.975 INFO:tasks.workunit.client.1.smithi157.stdout:1/60: mknod d4/db/c11 0 2024-04-22T23:34:50.975 INFO:tasks.workunit.client.1.smithi157.stdout:1/61: truncate d4/f9 1636190 0 2024-04-22T23:34:50.975 INFO:tasks.workunit.client.1.smithi157.stdout:1/62: rename d4/db to d4/db/d12 22 2024-04-22T23:34:50.975 INFO:tasks.workunit.client.1.smithi157.stdout:1/63: creat d4/db/f13 x:0 0 0 2024-04-22T23:34:50.977 INFO:tasks.workunit.client.1.smithi157.stdout:1/64: rename d4/f9 to d4/db/f14 0 2024-04-22T23:34:50.977 INFO:tasks.workunit.client.1.smithi157.stdout:3/60: dread d0/d3/f6 [0,4194304] 0 2024-04-22T23:34:50.977 INFO:tasks.workunit.client.1.smithi157.stdout:1/65: read d4/db/f14 [604810,51704] 0 2024-04-22T23:34:50.978 INFO:tasks.workunit.client.1.smithi157.stdout:1/66: truncate d4/db/fd 259706 0 2024-04-22T23:34:50.978 INFO:tasks.workunit.client.1.smithi157.stdout:3/61: symlink d0/d3/l12 0 2024-04-22T23:34:50.978 INFO:tasks.workunit.client.1.smithi157.stdout:3/62: truncate d0/d3/fe 5198687 0 2024-04-22T23:34:50.978 INFO:tasks.workunit.client.1.smithi157.stdout:3/63: read d0/d7/fa [1920091,75456] 0 2024-04-22T23:34:50.979 INFO:tasks.workunit.client.1.smithi157.stdout:1/67: mknod d4/db/c15 0 2024-04-22T23:34:50.979 INFO:tasks.workunit.client.1.smithi157.stdout:1/68: write d4/db/f14 [1265014,54520] 0 2024-04-22T23:34:50.979 INFO:tasks.workunit.client.1.smithi157.stdout:3/64: mkdir d0/d3/d13 0 2024-04-22T23:34:50.980 INFO:tasks.workunit.client.1.smithi157.stdout:1/69: rename f3 to d4/f16 0 2024-04-22T23:34:50.981 INFO:tasks.workunit.client.1.smithi157.stdout:3/65: rmdir d0/d3 39 2024-04-22T23:34:50.981 INFO:tasks.workunit.client.1.smithi157.stdout:3/66: write d0/f11 [557301,97252] 0 2024-04-22T23:34:50.982 INFO:tasks.workunit.client.1.smithi157.stdout:1/70: dread d4/f16 [0,4194304] 0 2024-04-22T23:34:50.982 INFO:tasks.workunit.client.1.smithi157.stdout:1/71: write d4/db/fe [678151,20939] 0 2024-04-22T23:34:50.983 INFO:tasks.workunit.client.1.smithi157.stdout:1/72: mkdir d4/d17 0 2024-04-22T23:34:50.992 INFO:tasks.workunit.client.1.smithi157.stdout:1/73: creat d4/f18 x:0 0 0 2024-04-22T23:34:51.000 INFO:tasks.workunit.client.1.smithi157.stdout:3/67: dwrite d0/d3/f6 [0,4194304] 0 2024-04-22T23:34:51.016 INFO:tasks.workunit.client.1.smithi157.stdout:3/68: read d0/d3/f6 [324098,54259] 0 2024-04-22T23:34:51.021 INFO:tasks.workunit.client.1.smithi157.stdout:1/74: dwrite d4/db/f14 [0,4194304] 0 2024-04-22T23:34:51.026 INFO:tasks.workunit.client.1.smithi157.stdout:3/69: creat d0/f14 x:0 0 0 2024-04-22T23:34:51.027 INFO:tasks.workunit.client.1.smithi157.stdout:3/70: rmdir d0/d7 39 2024-04-22T23:34:51.027 INFO:tasks.workunit.client.1.smithi157.stdout:3/71: chown d0/d3/d13 519598913 1 2024-04-22T23:34:51.029 INFO:tasks.workunit.client.1.smithi157.stdout:3/72: rename d0/d7/fa to d0/f15 0 2024-04-22T23:34:51.030 INFO:tasks.workunit.client.1.smithi157.stdout:3/73: mknod d0/d7/c16 0 2024-04-22T23:34:51.032 INFO:tasks.workunit.client.1.smithi157.stdout:3/74: mkdir d0/d7/d17 0 2024-04-22T23:34:51.032 INFO:tasks.workunit.client.1.smithi157.stdout:1/75: dread d4/db/f14 [0,4194304] 0 2024-04-22T23:34:51.033 INFO:tasks.workunit.client.1.smithi157.stdout:1/76: dread - d4/f18 zero size 2024-04-22T23:34:51.033 INFO:tasks.workunit.client.1.smithi157.stdout:3/75: creat d0/d7/d17/f18 x:0 0 0 2024-04-22T23:34:51.038 INFO:tasks.workunit.client.1.smithi157.stdout:1/77: unlink d4/c7 0 2024-04-22T23:34:51.038 INFO:tasks.workunit.client.1.smithi157.stdout:1/78: chown d4/db/fd 21995061 1 2024-04-22T23:34:51.039 INFO:tasks.workunit.client.1.smithi157.stdout:1/79: fdatasync d4/db/f14 0 2024-04-22T23:34:51.039 INFO:tasks.workunit.client.1.smithi157.stdout:1/80: chown d4/db/f14 501 1 2024-04-22T23:34:51.039 INFO:tasks.workunit.client.1.smithi157.stdout:3/76: unlink d0/d7/c16 0 2024-04-22T23:34:51.039 INFO:tasks.workunit.client.1.smithi157.stdout:3/77: chown d0/l9 237 1 2024-04-22T23:34:51.039 INFO:tasks.workunit.client.1.smithi157.stdout:1/81: symlink d4/db/l19 0 2024-04-22T23:34:51.039 INFO:tasks.workunit.client.1.smithi157.stdout:1/82: dread - d4/db/f13 zero size 2024-04-22T23:34:51.046 INFO:tasks.workunit.client.1.smithi157.stdout:1/83: mknod d4/db/c1a 0 2024-04-22T23:34:51.046 INFO:tasks.workunit.client.1.smithi157.stdout:1/84: truncate d4/f18 679331 0 2024-04-22T23:34:51.046 INFO:tasks.workunit.client.1.smithi157.stdout:1/85: stat d4 0 2024-04-22T23:34:51.046 INFO:tasks.workunit.client.1.smithi157.stdout:1/86: fsync d4/f16 0 2024-04-22T23:34:51.046 INFO:tasks.workunit.client.1.smithi157.stdout:1/87: stat d4/db/fe 0 2024-04-22T23:34:51.046 INFO:tasks.workunit.client.1.smithi157.stdout:1/88: chown d4/db/c11 65405 1 2024-04-22T23:34:51.144 INFO:tasks.workunit.client.1.smithi157.stdout:8/18: sync 2024-04-22T23:34:51.144 INFO:tasks.workunit.client.1.smithi157.stdout:5/79: sync 2024-04-22T23:34:51.144 INFO:tasks.workunit.client.1.smithi157.stdout:0/30: sync 2024-04-22T23:34:51.144 INFO:tasks.workunit.client.1.smithi157.stdout:6/217: sync 2024-04-22T23:34:51.144 INFO:tasks.workunit.client.1.smithi157.stdout:2/34: sync 2024-04-22T23:34:51.145 INFO:tasks.workunit.client.1.smithi157.stdout:8/19: chown f1 2042964 1 2024-04-22T23:34:51.145 INFO:tasks.workunit.client.1.smithi157.stdout:9/50: sync 2024-04-22T23:34:51.145 INFO:tasks.workunit.client.1.smithi157.stdout:0/31: rename f0 to f9 0 2024-04-22T23:34:51.146 INFO:tasks.workunit.client.1.smithi157.stdout:6/218: write d1/f7 [93121,96317] 0 2024-04-22T23:34:51.146 INFO:tasks.workunit.client.1.smithi157.stdout:6/219: stat d1/d2f/d3b/d39/f46 0 2024-04-22T23:34:51.147 INFO:tasks.workunit.client.1.smithi157.stdout:2/35: link c1 d7/d8/cc 0 2024-04-22T23:34:51.147 INFO:tasks.workunit.client.1.smithi157.stdout:2/36: read d7/fa [281061,27792] 0 2024-04-22T23:34:51.147 INFO:tasks.workunit.client.1.smithi157.stdout:8/20: truncate f1 2983597 0 2024-04-22T23:34:51.148 INFO:tasks.workunit.client.1.smithi157.stdout:5/80: creat d1/d5/dc/f1b x:0 0 0 2024-04-22T23:34:51.148 INFO:tasks.workunit.client.1.smithi157.stdout:5/81: stat d1/d5/dc/f1b 0 2024-04-22T23:34:51.148 INFO:tasks.workunit.client.1.smithi157.stdout:5/82: fsync d1/d5/f12 0 2024-04-22T23:34:51.148 INFO:tasks.workunit.client.1.smithi157.stdout:2/37: dread d7/fa [0,4194304] 0 2024-04-22T23:34:51.148 INFO:tasks.workunit.client.1.smithi157.stdout:0/32: mknod ca 0 2024-04-22T23:34:51.148 INFO:tasks.workunit.client.1.smithi157.stdout:0/33: chown f9 7132684 1 2024-04-22T23:34:51.149 INFO:tasks.workunit.client.1.smithi157.stdout:6/220: unlink d1/d2f/l30 0 2024-04-22T23:34:51.149 INFO:tasks.workunit.client.1.smithi157.stdout:2/38: dread f0 [0,4194304] 0 2024-04-22T23:34:51.149 INFO:tasks.workunit.client.1.smithi157.stdout:2/39: chown d7/d8 672007 1 2024-04-22T23:34:51.150 INFO:tasks.workunit.client.1.smithi157.stdout:6/221: dread d1/d17/f31 [0,4194304] 0 2024-04-22T23:34:51.150 INFO:tasks.workunit.client.1.smithi157.stdout:6/222: write d1/d17/f24 [1210288,65390] 0 2024-04-22T23:34:51.151 INFO:tasks.workunit.client.1.smithi157.stdout:6/223: dread d1/d2f/d3b/f2e [0,4194304] 0 2024-04-22T23:34:51.151 INFO:tasks.workunit.client.1.smithi157.stdout:6/224: readlink d1/l23 0 2024-04-22T23:34:51.151 INFO:tasks.workunit.client.1.smithi157.stdout:6/225: readlink d1/d36/l3d 0 2024-04-22T23:34:51.151 INFO:tasks.workunit.client.1.smithi157.stdout:6/226: truncate d1/d2f/f40 1042985 0 2024-04-22T23:34:51.153 INFO:tasks.workunit.client.1.smithi157.stdout:5/83: mknod d1/d5/d7/d10/c1c 0 2024-04-22T23:34:51.153 INFO:tasks.workunit.client.1.smithi157.stdout:5/84: creat d1/d5/dc/f1d x:0 0 0 2024-04-22T23:34:51.153 INFO:tasks.workunit.client.1.smithi157.stdout:5/85: dread - d1/d5/dc/f1b zero size 2024-04-22T23:34:51.155 INFO:tasks.workunit.client.1.smithi157.stdout:0/34: symlink lb 0 2024-04-22T23:34:51.155 INFO:tasks.workunit.client.1.smithi157.stdout:2/40: stat c6 0 2024-04-22T23:34:51.160 INFO:tasks.workunit.client.1.smithi157.stdout:5/86: symlink d1/d5/d7/d10/l1e 0 2024-04-22T23:34:51.162 INFO:tasks.workunit.client.1.smithi157.stdout:8/21: rename f1 to d2/f3 0 2024-04-22T23:34:51.163 INFO:tasks.workunit.client.1.smithi157.stdout:0/35: creat fc x:0 0 0 2024-04-22T23:34:51.164 INFO:tasks.workunit.client.1.smithi157.stdout:2/41: mkdir d7/d8/dd 0 2024-04-22T23:34:51.164 INFO:tasks.workunit.client.1.smithi157.stdout:2/42: write d7/d8/f9 [373573,3975] 0 2024-04-22T23:34:51.164 INFO:tasks.workunit.client.1.smithi157.stdout:2/43: chown f3 0 1 2024-04-22T23:34:51.165 INFO:tasks.workunit.client.1.smithi157.stdout:9/51: dwrite d4/d9/fb [0,4194304] 0 2024-04-22T23:34:51.165 INFO:tasks.workunit.client.1.smithi157.stdout:5/87: mkdir d1/d1f 0 2024-04-22T23:34:51.165 INFO:tasks.workunit.client.1.smithi157.stdout:5/88: fsync d1/d5/f15 0 2024-04-22T23:34:51.168 INFO:tasks.workunit.client.1.smithi157.stdout:8/22: mkdir d2/d4 0 2024-04-22T23:34:51.168 INFO:tasks.workunit.client.1.smithi157.stdout:8/23: chown d2 407127 1 2024-04-22T23:34:51.168 INFO:tasks.workunit.client.1.smithi157.stdout:8/24: stat d2/d4 0 2024-04-22T23:34:51.169 INFO:tasks.workunit.client.1.smithi157.stdout:6/227: dwrite d1/d17/f33 [0,4194304] 0 2024-04-22T23:34:51.170 INFO:tasks.workunit.client.1.smithi157.stdout:0/36: rename l2 to ld 0 2024-04-22T23:34:51.170 INFO:tasks.workunit.client.1.smithi157.stdout:0/37: creat fe x:0 0 0 2024-04-22T23:34:51.172 INFO:tasks.workunit.client.1.smithi157.stdout:2/44: mknod d7/d8/ce 0 2024-04-22T23:34:51.177 INFO:tasks.workunit.client.1.smithi157.stdout:8/25: creat d2/f5 x:0 0 0 2024-04-22T23:34:51.186 INFO:tasks.workunit.client.1.smithi157.stdout:0/38: mknod cf 0 2024-04-22T23:34:51.186 INFO:tasks.workunit.client.1.smithi157.stdout:2/45: mkdir d7/d8/df 0 2024-04-22T23:34:51.187 INFO:tasks.workunit.client.1.smithi157.stdout:9/52: symlink d4/ld 0 2024-04-22T23:34:51.188 INFO:tasks.workunit.client.1.smithi157.stdout:8/26: mkdir d2/d6 0 2024-04-22T23:34:51.188 INFO:tasks.workunit.client.1.smithi157.stdout:8/27: fdatasync d2/f5 0 2024-04-22T23:34:51.188 INFO:tasks.workunit.client.1.smithi157.stdout:0/39: mknod c10 0 2024-04-22T23:34:51.188 INFO:tasks.workunit.client.1.smithi157.stdout:0/40: creat f11 x:0 0 0 2024-04-22T23:34:51.189 INFO:tasks.workunit.client.1.smithi157.stdout:2/46: mknod d7/c10 0 2024-04-22T23:34:51.190 INFO:tasks.workunit.client.1.smithi157.stdout:9/53: rename d4/c7 to d4/d9/ce 0 2024-04-22T23:34:51.190 INFO:tasks.workunit.client.1.smithi157.stdout:9/54: write d4/fc [1023166,39662] 0 2024-04-22T23:34:51.191 INFO:tasks.workunit.client.1.smithi157.stdout:8/28: symlink d2/d4/l7 0 2024-04-22T23:34:51.191 INFO:tasks.workunit.client.1.smithi157.stdout:8/29: truncate d2/f5 1036340 0 2024-04-22T23:34:51.192 INFO:tasks.workunit.client.1.smithi157.stdout:5/89: dwrite d1/d5/f12 [0,4194304] 0 2024-04-22T23:34:51.192 INFO:tasks.workunit.client.1.smithi157.stdout:5/90: fsync d1/fe 0 2024-04-22T23:34:51.193 INFO:tasks.workunit.client.1.smithi157.stdout:0/41: symlink l12 0 2024-04-22T23:34:51.193 INFO:tasks.workunit.client.1.smithi157.stdout:0/42: chown f3 147819825 1 2024-04-22T23:34:51.194 INFO:tasks.workunit.client.1.smithi157.stdout:2/47: rename f4 to d7/d8/dd/f11 0 2024-04-22T23:34:51.194 INFO:tasks.workunit.client.1.smithi157.stdout:2/48: write f0 [1199639,31237] 0 2024-04-22T23:34:51.194 INFO:tasks.workunit.client.1.smithi157.stdout:2/49: write d7/d8/f9 [994144,41060] 0 2024-04-22T23:34:51.194 INFO:tasks.workunit.client.1.smithi157.stdout:2/50: readlink d7/d8/lb 0 2024-04-22T23:34:51.195 INFO:tasks.workunit.client.1.smithi157.stdout:6/228: dwrite d1/f5 [0,4194304] 0 2024-04-22T23:34:51.197 INFO:tasks.workunit.client.1.smithi157.stdout:0/43: mknod c13 0 2024-04-22T23:34:51.207 INFO:tasks.workunit.client.1.smithi157.stdout:9/55: dread d4/d9/fb [0,4194304] 0 2024-04-22T23:34:51.207 INFO:tasks.workunit.client.1.smithi157.stdout:8/30: getdents d2 0 2024-04-22T23:34:51.210 INFO:tasks.workunit.client.1.smithi157.stdout:8/31: truncate d2/f3 244966 0 2024-04-22T23:34:51.213 INFO:tasks.workunit.client.1.smithi157.stdout:0/44: write f5 [2528085,40020] 0 2024-04-22T23:34:51.213 INFO:tasks.workunit.client.1.smithi157.stdout:0/45: readlink l12 0 2024-04-22T23:34:51.214 INFO:tasks.workunit.client.1.smithi157.stdout:0/46: write f8 [283194,128293] 0 2024-04-22T23:34:51.214 INFO:tasks.workunit.client.1.smithi157.stdout:0/47: mknod c14 0 2024-04-22T23:34:51.216 INFO:tasks.workunit.client.1.smithi157.stdout:0/48: rename f9 to f15 0 2024-04-22T23:34:51.216 INFO:tasks.workunit.client.1.smithi157.stdout:0/49: write f8 [672614,102476] 0 2024-04-22T23:34:51.216 INFO:tasks.workunit.client.1.smithi157.stdout:0/50: stat fc 0 2024-04-22T23:34:51.216 INFO:tasks.workunit.client.1.smithi157.stdout:0/51: fsync f3 0 2024-04-22T23:34:51.216 INFO:tasks.workunit.client.1.smithi157.stdout:0/52: dread - fc zero size 2024-04-22T23:34:51.217 INFO:tasks.workunit.client.1.smithi157.stdout:5/91: dwrite d1/d5/dc/f1d [0,4194304] 0 2024-04-22T23:34:51.221 INFO:tasks.workunit.client.1.smithi157.stdout:0/53: dread f15 [0,4194304] 0 2024-04-22T23:34:51.222 INFO:tasks.workunit.client.1.smithi157.stdout:0/54: truncate fc 424771 0 2024-04-22T23:34:51.223 INFO:tasks.workunit.client.1.smithi157.stdout:9/56: dread d4/fc [0,4194304] 0 2024-04-22T23:34:51.223 INFO:tasks.workunit.client.1.smithi157.stdout:6/229: dwrite d1/d17/f33 [0,4194304] 0 2024-04-22T23:34:51.223 INFO:tasks.workunit.client.1.smithi157.stdout:0/55: rename ld to l16 0 2024-04-22T23:34:51.223 INFO:tasks.workunit.client.1.smithi157.stdout:0/56: truncate f5 4612707 0 2024-04-22T23:34:51.227 INFO:tasks.workunit.client.1.smithi157.stdout:2/51: write d7/d8/dd/f11 [3861194,33089] 0 2024-04-22T23:34:51.227 INFO:tasks.workunit.client.1.smithi157.stdout:0/57: mkdir d17 0 2024-04-22T23:34:51.228 INFO:tasks.workunit.client.1.smithi157.stdout:9/57: getdents d4 0 2024-04-22T23:34:51.228 INFO:tasks.workunit.client.1.smithi157.stdout:2/52: write f0 [917290,103918] 0 2024-04-22T23:34:51.228 INFO:tasks.workunit.client.1.smithi157.stdout:6/230: getdents d1/d2f 0 2024-04-22T23:34:51.228 INFO:tasks.workunit.client.1.smithi157.stdout:6/231: stat d1/d17 0 2024-04-22T23:34:51.228 INFO:tasks.workunit.client.1.smithi157.stdout:6/232: readlink d1/l1a 0 2024-04-22T23:34:51.229 INFO:tasks.workunit.client.1.smithi157.stdout:0/58: mknod d17/c18 0 2024-04-22T23:34:51.232 INFO:tasks.workunit.client.1.smithi157.stdout:6/233: link d1/d2f/d3b/l15 d1/d2f/d3b/d39/d4e/l51 0 2024-04-22T23:34:51.234 INFO:tasks.workunit.client.1.smithi157.stdout:6/234: creat d1/d2f/d3b/d39/d4e/f52 x:0 0 0 2024-04-22T23:34:51.235 INFO:tasks.workunit.client.1.smithi157.stdout:9/58: rename d4/d9 to d4/df 0 2024-04-22T23:34:51.237 INFO:tasks.workunit.client.1.smithi157.stdout:2/53: dread d7/d8/dd/f11 [0,4194304] 0 2024-04-22T23:34:51.237 INFO:tasks.workunit.client.1.smithi157.stdout:2/54: read f0 [1191568,107488] 0 2024-04-22T23:34:51.242 INFO:tasks.workunit.client.1.smithi157.stdout:2/55: mkdir d7/d12 0 2024-04-22T23:34:51.242 INFO:tasks.workunit.client.1.smithi157.stdout:8/32: dwrite d2/f3 [0,4194304] 0 2024-04-22T23:34:51.242 INFO:tasks.workunit.client.1.smithi157.stdout:8/33: link d2/f5 d2/d6/f8 0 2024-04-22T23:34:51.243 INFO:tasks.workunit.client.1.smithi157.stdout:8/34: mknod d2/d4/c9 0 2024-04-22T23:34:51.243 INFO:tasks.workunit.client.1.smithi157.stdout:8/35: readlink d2/d4/l7 0 2024-04-22T23:34:51.244 INFO:tasks.workunit.client.1.smithi157.stdout:8/36: write d2/d6/f8 [1001932,32100] 0 2024-04-22T23:34:51.244 INFO:tasks.workunit.client.1.smithi157.stdout:8/37: stat d2/d6/f8 0 2024-04-22T23:34:51.244 INFO:tasks.workunit.client.1.smithi157.stdout:9/59: write d4/df/fb [2468543,56502] 0 2024-04-22T23:34:51.246 INFO:tasks.workunit.client.1.smithi157.stdout:9/60: stat d4/l6 0 2024-04-22T23:34:51.246 INFO:tasks.workunit.client.1.smithi157.stdout:6/235: dread d1/d36/f3f [4194304,4194304] 0 2024-04-22T23:34:51.246 INFO:tasks.workunit.client.1.smithi157.stdout:5/92: dwrite d1/d5/f14 [0,4194304] 0 2024-04-22T23:34:51.246 INFO:tasks.workunit.client.1.smithi157.stdout:5/93: chown d1/d5/f14 25363 1 2024-04-22T23:34:51.249 INFO:tasks.workunit.client.1.smithi157.stdout:9/61: dread d4/fc [0,4194304] 0 2024-04-22T23:34:51.249 INFO:tasks.workunit.client.1.smithi157.stdout:6/236: unlink d1/l1a 0 2024-04-22T23:34:51.249 INFO:tasks.workunit.client.1.smithi157.stdout:9/62: read d4/df/fb [1521040,9221] 0 2024-04-22T23:34:51.249 INFO:tasks.workunit.client.1.smithi157.stdout:6/237: dread d1/d2f/d3b/f2e [0,4194304] 0 2024-04-22T23:34:51.251 INFO:tasks.workunit.client.1.smithi157.stdout:0/59: dwrite fc [0,4194304] 0 2024-04-22T23:34:51.251 INFO:tasks.workunit.client.1.smithi157.stdout:5/94: creat d1/d5/d7/f20 x:0 0 0 2024-04-22T23:34:51.254 INFO:tasks.workunit.client.1.smithi157.stdout:9/63: rename d4/l6 to d4/df/l10 0 2024-04-22T23:34:51.256 INFO:tasks.workunit.client.1.smithi157.stdout:6/238: unlink d1/l16 0 2024-04-22T23:34:51.256 INFO:tasks.workunit.client.1.smithi157.stdout:6/239: fsync d1/d17/f43 0 2024-04-22T23:34:51.257 INFO:tasks.workunit.client.1.smithi157.stdout:6/240: write d1/d2f/f40 [263018,89929] 0 2024-04-22T23:34:51.259 INFO:tasks.workunit.client.1.smithi157.stdout:2/56: getdents d7 0 2024-04-22T23:34:51.259 INFO:tasks.workunit.client.1.smithi157.stdout:2/57: dread - f3 zero size 2024-04-22T23:34:51.259 INFO:tasks.workunit.client.1.smithi157.stdout:2/58: write f5 [840737,114530] 0 2024-04-22T23:34:51.262 INFO:tasks.workunit.client.1.smithi157.stdout:8/38: dwrite d2/f3 [0,4194304] 0 2024-04-22T23:34:51.268 INFO:tasks.workunit.client.1.smithi157.stdout:0/60: getdents d17 0 2024-04-22T23:34:51.269 INFO:tasks.workunit.client.1.smithi157.stdout:2/59: unlink f0 0 2024-04-22T23:34:51.270 INFO:tasks.workunit.client.1.smithi157.stdout:2/60: chown f3 9052 1 2024-04-22T23:34:51.270 INFO:tasks.workunit.client.1.smithi157.stdout:9/64: symlink d4/l11 0 2024-04-22T23:34:51.271 INFO:tasks.workunit.client.1.smithi157.stdout:0/61: unlink f15 0 2024-04-22T23:34:51.272 INFO:tasks.workunit.client.1.smithi157.stdout:5/95: dwrite d1/fe [0,4194304] 0 2024-04-22T23:34:51.272 INFO:tasks.workunit.client.1.smithi157.stdout:5/96: fdatasync d1/d5/f15 0 2024-04-22T23:34:51.273 INFO:tasks.workunit.client.1.smithi157.stdout:2/61: rename d7/fa to d7/f13 0 2024-04-22T23:34:51.273 INFO:tasks.workunit.client.1.smithi157.stdout:2/62: write f3 [642277,52557] 0 2024-04-22T23:34:51.273 INFO:tasks.workunit.client.1.smithi157.stdout:2/63: truncate d7/f13 868829 0 2024-04-22T23:34:51.274 INFO:tasks.workunit.client.1.smithi157.stdout:0/62: creat d17/f19 x:0 0 0 2024-04-22T23:34:51.274 INFO:tasks.workunit.client.1.smithi157.stdout:0/63: creat d17/f1a x:0 0 0 2024-04-22T23:34:51.274 INFO:tasks.workunit.client.1.smithi157.stdout:0/64: creat d17/f1b x:0 0 0 2024-04-22T23:34:51.278 INFO:tasks.workunit.client.1.smithi157.stdout:6/241: getdents d1 0 2024-04-22T23:34:51.281 INFO:tasks.workunit.client.1.smithi157.stdout:5/97: truncate d1/d5/f12 2509375 0 2024-04-22T23:34:51.283 INFO:tasks.workunit.client.1.smithi157.stdout:2/64: rename c1 to d7/d12/c14 0 2024-04-22T23:34:51.289 INFO:tasks.workunit.client.1.smithi157.stdout:0/65: rename c10 to d17/c1c 0 2024-04-22T23:34:51.289 INFO:tasks.workunit.client.1.smithi157.stdout:0/66: write f5 [4227730,31725] 0 2024-04-22T23:34:51.290 INFO:tasks.workunit.client.1.smithi157.stdout:2/65: rename d7/d8/ce to d7/d8/c15 0 2024-04-22T23:34:51.291 INFO:tasks.workunit.client.1.smithi157.stdout:2/66: write f3 [1177991,56902] 0 2024-04-22T23:34:51.292 INFO:tasks.workunit.client.1.smithi157.stdout:5/98: dread d1/d5/f14 [0,4194304] 0 2024-04-22T23:34:51.293 INFO:tasks.workunit.client.1.smithi157.stdout:0/67: symlink d17/l1d 0 2024-04-22T23:34:51.295 INFO:tasks.workunit.client.1.smithi157.stdout:2/67: creat d7/d8/df/f16 x:0 0 0 2024-04-22T23:34:51.295 INFO:tasks.workunit.client.1.smithi157.stdout:2/68: chown d7/f13 18 1 2024-04-22T23:34:51.295 INFO:tasks.workunit.client.1.smithi157.stdout:2/69: chown d7/d8 26659033 1 2024-04-22T23:34:51.296 INFO:tasks.workunit.client.1.smithi157.stdout:7/46: sync 2024-04-22T23:34:51.296 INFO:tasks.workunit.client.1.smithi157.stdout:4/82: sync 2024-04-22T23:34:51.296 INFO:tasks.workunit.client.1.smithi157.stdout:4/83: fsync d3/fc 0 2024-04-22T23:34:51.296 INFO:tasks.workunit.client.1.smithi157.stdout:4/84: fdatasync d3/d4/f9 0 2024-04-22T23:34:51.296 INFO:tasks.workunit.client.1.smithi157.stdout:5/99: dread d1/d5/f14 [0,4194304] 0 2024-04-22T23:34:51.296 INFO:tasks.workunit.client.1.smithi157.stdout:9/65: dwrite d4/df/fb [4194304,4194304] 0 2024-04-22T23:34:51.296 INFO:tasks.workunit.client.1.smithi157.stdout:8/39: dwrite d2/d6/f8 [0,4194304] 0 2024-04-22T23:34:51.296 INFO:tasks.workunit.client.1.smithi157.stdout:0/68: unlink l7 0 2024-04-22T23:34:51.298 INFO:tasks.workunit.client.1.smithi157.stdout:4/85: truncate d3/d4/f5 90339 0 2024-04-22T23:34:51.302 INFO:tasks.workunit.client.1.smithi157.stdout:9/66: truncate f3 478830 0 2024-04-22T23:34:51.302 INFO:tasks.workunit.client.1.smithi157.stdout:9/67: dread - d4/f8 zero size 2024-04-22T23:34:51.304 INFO:tasks.workunit.client.1.smithi157.stdout:7/47: write d0/f7 [3369618,106282] 0 2024-04-22T23:34:51.305 INFO:tasks.workunit.client.1.smithi157.stdout:8/40: link d2/d4/c9 d2/d4/ca 0 2024-04-22T23:34:51.305 INFO:tasks.workunit.client.1.smithi157.stdout:1/89: sync 2024-04-22T23:34:51.305 INFO:tasks.workunit.client.1.smithi157.stdout:3/78: sync 2024-04-22T23:34:51.305 INFO:tasks.workunit.client.1.smithi157.stdout:3/79: rename d0 to d0/d19 22 2024-04-22T23:34:51.305 INFO:tasks.workunit.client.1.smithi157.stdout:1/90: creat d4/f1b x:0 0 0 2024-04-22T23:34:51.306 INFO:tasks.workunit.client.1.smithi157.stdout:1/91: stat d4/f16 0 2024-04-22T23:34:51.306 INFO:tasks.workunit.client.1.smithi157.stdout:1/92: fsync d4/f16 0 2024-04-22T23:34:51.306 INFO:tasks.workunit.client.1.smithi157.stdout:1/93: stat d4/db 0 2024-04-22T23:34:51.306 INFO:tasks.workunit.client.1.smithi157.stdout:1/94: chown d4/f1b 158165415 1 2024-04-22T23:34:51.306 INFO:tasks.workunit.client.1.smithi157.stdout:3/80: write d0/d3/f6 [5147209,82830] 0 2024-04-22T23:34:51.309 INFO:tasks.workunit.client.1.smithi157.stdout:9/68: creat d4/df/f12 x:0 0 0 2024-04-22T23:34:51.309 INFO:tasks.workunit.client.1.smithi157.stdout:7/48: rmdir d0/d9 39 2024-04-22T23:34:51.313 INFO:tasks.workunit.client.1.smithi157.stdout:1/95: mknod d4/d17/c1c 0 2024-04-22T23:34:51.313 INFO:tasks.workunit.client.1.smithi157.stdout:8/41: dread d2/d6/f8 [0,4194304] 0 2024-04-22T23:34:51.315 INFO:tasks.workunit.client.1.smithi157.stdout:3/81: symlink d0/d7/d17/l1a 0 2024-04-22T23:34:51.316 INFO:tasks.workunit.client.1.smithi157.stdout:7/49: mkdir d0/d9/db 0 2024-04-22T23:34:51.316 INFO:tasks.workunit.client.1.smithi157.stdout:3/82: dread d0/d3/fd [0,4194304] 0 2024-04-22T23:34:51.317 INFO:tasks.workunit.client.1.smithi157.stdout:1/96: mkdir d4/d17/d1d 0 2024-04-22T23:34:51.318 INFO:tasks.workunit.client.1.smithi157.stdout:8/42: creat d2/d6/fb x:0 0 0 2024-04-22T23:34:51.318 INFO:tasks.workunit.client.1.smithi157.stdout:8/43: chown d2/f5 2883495 1 2024-04-22T23:34:51.319 INFO:tasks.workunit.client.1.smithi157.stdout:3/83: creat d0/d3/f1b x:0 0 0 2024-04-22T23:34:51.320 INFO:tasks.workunit.client.1.smithi157.stdout:7/50: read d0/f8 [2507317,93743] 0 2024-04-22T23:34:51.321 INFO:tasks.workunit.client.1.smithi157.stdout:1/97: rename f1 to d4/db/f1e 0 2024-04-22T23:34:51.321 INFO:tasks.workunit.client.1.smithi157.stdout:1/98: creat d4/f1f x:0 0 0 2024-04-22T23:34:51.321 INFO:tasks.workunit.client.1.smithi157.stdout:1/99: dread - d4/f1f zero size 2024-04-22T23:34:51.323 INFO:tasks.workunit.client.1.smithi157.stdout:5/100: dread d1/d5/f12 [0,4194304] 0 2024-04-22T23:34:51.323 INFO:tasks.workunit.client.1.smithi157.stdout:5/101: fdatasync d1/d5/f15 0 2024-04-22T23:34:51.323 INFO:tasks.workunit.client.1.smithi157.stdout:5/102: stat d1/d5/d7/la 0 2024-04-22T23:34:51.324 INFO:tasks.workunit.client.1.smithi157.stdout:3/84: link d0/f15 d0/f1c 0 2024-04-22T23:34:51.326 INFO:tasks.workunit.client.1.smithi157.stdout:7/51: link d0/f2 d0/d9/fc 0 2024-04-22T23:34:51.329 INFO:tasks.workunit.client.1.smithi157.stdout:1/100: symlink d4/l20 0 2024-04-22T23:34:51.332 INFO:tasks.workunit.client.1.smithi157.stdout:5/103: rename d1/d5/d7/f20 to d1/d5/d7/f21 0 2024-04-22T23:34:51.332 INFO:tasks.workunit.client.1.smithi157.stdout:7/52: truncate d0/f4 1224082 0 2024-04-22T23:34:51.333 INFO:tasks.workunit.client.1.smithi157.stdout:4/86: dwrite d3/f13 [0,4194304] 0 2024-04-22T23:34:51.333 INFO:tasks.workunit.client.1.smithi157.stdout:0/69: dwrite fc [4194304,4194304] 0 2024-04-22T23:34:51.333 INFO:tasks.workunit.client.1.smithi157.stdout:0/70: write d17/f19 [723747,2310] 0 2024-04-22T23:34:51.334 INFO:tasks.workunit.client.1.smithi157.stdout:7/53: symlink d0/ld 0 2024-04-22T23:34:51.334 INFO:tasks.workunit.client.1.smithi157.stdout:9/69: dwrite d4/df/f12 [0,4194304] 0 2024-04-22T23:34:51.335 INFO:tasks.workunit.client.1.smithi157.stdout:7/54: write d0/f8 [2983621,91122] 0 2024-04-22T23:34:51.336 INFO:tasks.workunit.client.1.smithi157.stdout:8/44: truncate d2/d6/f8 1181291 0 2024-04-22T23:34:51.337 INFO:tasks.workunit.client.1.smithi157.stdout:5/104: dread d1/d5/f14 [0,4194304] 0 2024-04-22T23:34:51.337 INFO:tasks.workunit.client.1.smithi157.stdout:5/105: fdatasync d1/d5/dc/f1d 0 2024-04-22T23:34:51.337 INFO:tasks.workunit.client.1.smithi157.stdout:5/106: fdatasync d1/d5/f14 0 2024-04-22T23:34:51.337 INFO:tasks.workunit.client.1.smithi157.stdout:4/87: mknod d3/d11/c15 0 2024-04-22T23:34:51.338 INFO:tasks.workunit.client.1.smithi157.stdout:9/70: read f3 [47821,109214] 0 2024-04-22T23:34:51.339 INFO:tasks.workunit.client.1.smithi157.stdout:2/70: truncate d7/f13 275729 0 2024-04-22T23:34:51.340 INFO:tasks.workunit.client.1.smithi157.stdout:5/107: write d1/d5/f15 [405859,2653] 0 2024-04-22T23:34:51.341 INFO:tasks.workunit.client.1.smithi157.stdout:5/108: chown d1/d5/f15 26181049 1 2024-04-22T23:34:51.341 INFO:tasks.workunit.client.1.smithi157.stdout:0/71: link cf d17/c1e 0 2024-04-22T23:34:51.342 INFO:tasks.workunit.client.1.smithi157.stdout:2/71: dread d7/d8/f9 [0,4194304] 0 2024-04-22T23:34:51.346 INFO:tasks.workunit.client.1.smithi157.stdout:8/45: mkdir d2/d6/dc 0 2024-04-22T23:34:51.348 INFO:tasks.workunit.client.1.smithi157.stdout:9/71: mknod d4/df/c13 0 2024-04-22T23:34:51.348 INFO:tasks.workunit.client.1.smithi157.stdout:3/85: dwrite d0/d3/fd [0,4194304] 0 2024-04-22T23:34:51.349 INFO:tasks.workunit.client.1.smithi157.stdout:4/88: write d3/f13 [40807,56986] 0 2024-04-22T23:34:51.350 INFO:tasks.workunit.client.1.smithi157.stdout:9/72: dread f3 [0,4194304] 0 2024-04-22T23:34:51.350 INFO:tasks.workunit.client.1.smithi157.stdout:9/73: stat d4/df/fb 0 2024-04-22T23:34:51.350 INFO:tasks.workunit.client.1.smithi157.stdout:9/74: readlink d4/df/l10 0 2024-04-22T23:34:51.351 INFO:tasks.workunit.client.1.smithi157.stdout:5/109: unlink d1/d5/d7/l17 0 2024-04-22T23:34:51.358 INFO:tasks.workunit.client.1.smithi157.stdout:1/101: dwrite d4/f5 [0,4194304] 0 2024-04-22T23:34:51.360 INFO:tasks.workunit.client.1.smithi157.stdout:7/55: dwrite d0/f2 [0,4194304] 0 2024-04-22T23:34:51.360 INFO:tasks.workunit.client.1.smithi157.stdout:0/72: unlink f5 0 2024-04-22T23:34:51.361 INFO:tasks.workunit.client.1.smithi157.stdout:0/73: dread f3 [0,4194304] 0 2024-04-22T23:34:51.362 INFO:tasks.workunit.client.1.smithi157.stdout:0/74: write fe [37638,30245] 0 2024-04-22T23:34:51.362 INFO:tasks.workunit.client.1.smithi157.stdout:2/72: dwrite f5 [0,4194304] 0 2024-04-22T23:34:51.365 INFO:tasks.workunit.client.1.smithi157.stdout:9/75: rmdir d4 39 2024-04-22T23:34:51.366 INFO:tasks.workunit.client.1.smithi157.stdout:5/110: mkdir d1/d1f/d22 0 2024-04-22T23:34:51.367 INFO:tasks.workunit.client.1.smithi157.stdout:1/102: symlink d4/d17/l21 0 2024-04-22T23:34:51.369 INFO:tasks.workunit.client.1.smithi157.stdout:4/89: getdents d3/d11 0 2024-04-22T23:34:51.370 INFO:tasks.workunit.client.1.smithi157.stdout:0/75: link f11 d17/f1f 0 2024-04-22T23:34:51.372 INFO:tasks.workunit.client.1.smithi157.stdout:2/73: mknod d7/d8/df/c17 0 2024-04-22T23:34:51.372 INFO:tasks.workunit.client.1.smithi157.stdout:2/74: write d7/d8/dd/f11 [1995755,47238] 0 2024-04-22T23:34:51.373 INFO:tasks.workunit.client.1.smithi157.stdout:3/86: getdents d0/d7 0 2024-04-22T23:34:51.373 INFO:tasks.workunit.client.1.smithi157.stdout:3/87: creat d0/d3/f1d x:0 0 0 2024-04-22T23:34:51.376 INFO:tasks.workunit.client.1.smithi157.stdout:1/103: mknod d4/d17/d1d/c22 0 2024-04-22T23:34:51.376 INFO:tasks.workunit.client.1.smithi157.stdout:1/104: creat d4/f23 x:0 0 0 2024-04-22T23:34:51.376 INFO:tasks.workunit.client.1.smithi157.stdout:1/105: dread - d4/f1b zero size 2024-04-22T23:34:51.376 INFO:tasks.workunit.client.1.smithi157.stdout:1/106: chown d4/db/f1e 630349 1 2024-04-22T23:34:51.376 INFO:tasks.workunit.client.1.smithi157.stdout:1/107: readlink d4/l20 0 2024-04-22T23:34:51.377 INFO:tasks.workunit.client.1.smithi157.stdout:0/76: link f11 d17/f20 0 2024-04-22T23:34:51.379 INFO:tasks.workunit.client.1.smithi157.stdout:5/111: dread d1/d5/f12 [0,4194304] 0 2024-04-22T23:34:51.380 INFO:tasks.workunit.client.1.smithi157.stdout:3/88: link d0/f1c d0/d3/f1e 0 2024-04-22T23:34:51.381 INFO:tasks.workunit.client.1.smithi157.stdout:9/76: symlink d4/l14 0 2024-04-22T23:34:51.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:34:51 smithi157 ceph-mon[33877]: pgmap v252: 65 pgs: 65 active+clean; 212 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 5.4 MiB/s rd, 14 MiB/s wr, 801 op/s 2024-04-22T23:34:51.383 INFO:tasks.workunit.client.1.smithi157.stdout:9/77: dread d4/fc [0,4194304] 0 2024-04-22T23:34:51.383 INFO:tasks.workunit.client.1.smithi157.stdout:9/78: write d4/df/f12 [5170622,99973] 0 2024-04-22T23:34:51.384 INFO:tasks.workunit.client.1.smithi157.stdout:0/77: mknod d17/c21 0 2024-04-22T23:34:51.385 INFO:tasks.workunit.client.1.smithi157.stdout:0/78: chown d17/c21 4053990 1 2024-04-22T23:34:51.386 INFO:tasks.workunit.client.1.smithi157.stdout:2/75: write d7/f13 [161564,9420] 0 2024-04-22T23:34:51.386 INFO:tasks.workunit.client.1.smithi157.stdout:2/76: fsync d7/d8/f9 0 2024-04-22T23:34:51.386 INFO:tasks.workunit.client.1.smithi157.stdout:7/56: dwrite d0/f2 [4194304,4194304] 0 2024-04-22T23:34:51.386 INFO:tasks.workunit.client.1.smithi157.stdout:7/57: chown d0 3818 1 2024-04-22T23:34:51.388 INFO:tasks.workunit.client.1.smithi157.stdout:8/46: dwrite d2/f5 [0,4194304] 0 2024-04-22T23:34:51.388 INFO:tasks.workunit.client.1.smithi157.stdout:8/47: fsync d2/d6/fb 0 2024-04-22T23:34:51.388 INFO:tasks.workunit.client.1.smithi157.stdout:5/112: dread d1/d5/f12 [0,4194304] 0 2024-04-22T23:34:51.388 INFO:tasks.workunit.client.1.smithi157.stdout:0/79: rename d17/f1f to d17/f22 0 2024-04-22T23:34:51.389 INFO:tasks.workunit.client.1.smithi157.stdout:9/79: mkdir d4/d15 0 2024-04-22T23:34:51.391 INFO:tasks.workunit.client.1.smithi157.stdout:3/89: dread d0/d3/fd [0,4194304] 0 2024-04-22T23:34:51.391 INFO:tasks.workunit.client.1.smithi157.stdout:0/80: mkdir d17/d23 0 2024-04-22T23:34:51.391 INFO:tasks.workunit.client.1.smithi157.stdout:0/81: write f8 [764842,76037] 0 2024-04-22T23:34:51.396 INFO:tasks.workunit.client.1.smithi157.stdout:0/82: symlink d17/d23/l24 0 2024-04-22T23:34:51.396 INFO:tasks.workunit.client.1.smithi157.stdout:0/83: creat d17/f25 x:0 0 0 2024-04-22T23:34:51.396 INFO:tasks.workunit.client.1.smithi157.stdout:0/84: chown c14 43 1 2024-04-22T23:34:51.396 INFO:tasks.workunit.client.1.smithi157.stdout:8/48: write d2/d6/f8 [4077249,68459] 0 2024-04-22T23:34:51.396 INFO:tasks.workunit.client.1.smithi157.stdout:4/90: dwrite d3/fc [0,4194304] 0 2024-04-22T23:34:51.396 INFO:tasks.workunit.client.1.smithi157.stdout:4/91: fdatasync d3/d4/ff 0 2024-04-22T23:34:51.396 INFO:tasks.workunit.client.1.smithi157.stdout:4/92: creat d3/d4/f16 x:0 0 0 2024-04-22T23:34:51.396 INFO:tasks.workunit.client.1.smithi157.stdout:4/93: creat d3/d4/f17 x:0 0 0 2024-04-22T23:34:51.396 INFO:tasks.workunit.client.1.smithi157.stdout:4/94: chown l2 5168360 1 2024-04-22T23:34:51.396 INFO:tasks.workunit.client.1.smithi157.stdout:4/95: write d3/f13 [5164307,18499] 0 2024-04-22T23:34:51.396 INFO:tasks.workunit.client.1.smithi157.stdout:4/96: creat d3/d4/f18 x:0 0 0 2024-04-22T23:34:51.397 INFO:tasks.workunit.client.1.smithi157.stdout:9/80: dread d4/df/fb [4194304,4194304] 0 2024-04-22T23:34:51.397 INFO:tasks.workunit.client.1.smithi157.stdout:9/81: chown d4/df/c13 10476805 1 2024-04-22T23:34:51.398 INFO:tasks.workunit.client.1.smithi157.stdout:0/85: symlink d17/l26 0 2024-04-22T23:34:51.399 INFO:tasks.workunit.client.1.smithi157.stdout:0/86: write d17/f20 [1015769,82363] 0 2024-04-22T23:34:51.402 INFO:tasks.workunit.client.1.smithi157.stdout:8/49: rename d2/d6/f8 to d2/d6/dc/fd 0 2024-04-22T23:34:51.402 INFO:tasks.workunit.client.1.smithi157.stdout:8/50: dread - d2/d6/fb zero size 2024-04-22T23:34:51.405 INFO:tasks.workunit.client.1.smithi157.stdout:5/113: dread d1/fe [0,4194304] 0 2024-04-22T23:34:51.405 INFO:tasks.workunit.client.1.smithi157.stdout:5/114: truncate d1/d5/d7/f21 6090 0 2024-04-22T23:34:51.405 INFO:tasks.workunit.client.1.smithi157.stdout:1/108: getdents d4/d17/d1d 0 2024-04-22T23:34:51.409 INFO:tasks.workunit.client.1.smithi157.stdout:2/77: dwrite d7/d8/df/f16 [0,4194304] 0 2024-04-22T23:34:51.410 INFO:tasks.workunit.client.1.smithi157.stdout:7/58: dread d0/f8 [0,4194304] 0 2024-04-22T23:34:51.413 INFO:tasks.workunit.client.1.smithi157.stdout:3/90: dread d0/f1c [0,4194304] 0 2024-04-22T23:34:51.413 INFO:tasks.workunit.client.1.smithi157.stdout:3/91: fdatasync d0/d3/f1d 0 2024-04-22T23:34:51.414 INFO:tasks.workunit.client.1.smithi157.stdout:9/82: dwrite d4/df/fb [4194304,4194304] 0 2024-04-22T23:34:51.418 INFO:tasks.workunit.client.1.smithi157.stdout:7/59: mkdir d0/d9/de 0 2024-04-22T23:34:51.422 INFO:tasks.workunit.client.1.smithi157.stdout:7/60: dread d0/f7 [0,4194304] 0 2024-04-22T23:34:51.422 INFO:tasks.workunit.client.1.smithi157.stdout:7/61: fdatasync d0/f4 0 2024-04-22T23:34:51.422 INFO:tasks.workunit.client.1.smithi157.stdout:1/109: dread d4/f18 [0,4194304] 0 2024-04-22T23:34:51.424 INFO:tasks.workunit.client.1.smithi157.stdout:8/51: rename d2/d6/dc to d2/d6/de 0 2024-04-22T23:34:51.424 INFO:tasks.workunit.client.1.smithi157.stdout:8/52: stat d2/f5 0 2024-04-22T23:34:51.425 INFO:tasks.workunit.client.1.smithi157.stdout:3/92: rename d0/d3/fd to d0/d3/f1f 0 2024-04-22T23:34:51.428 INFO:tasks.workunit.client.1.smithi157.stdout:9/83: link f3 d4/df/f16 0 2024-04-22T23:34:51.428 INFO:tasks.workunit.client.1.smithi157.stdout:9/84: truncate d4/df/f16 1026444 0 2024-04-22T23:34:51.430 INFO:tasks.workunit.client.1.smithi157.stdout:7/62: mkdir d0/d9/db/df 0 2024-04-22T23:34:51.431 INFO:tasks.workunit.client.1.smithi157.stdout:1/110: rename d4/f1f to d4/db/f24 0 2024-04-22T23:34:51.432 INFO:tasks.workunit.client.1.smithi157.stdout:8/53: symlink d2/lf 0 2024-04-22T23:34:51.432 INFO:tasks.workunit.client.1.smithi157.stdout:8/54: read - d2/d6/fb zero size 2024-04-22T23:34:51.432 INFO:tasks.workunit.client.1.smithi157.stdout:8/55: write d2/d6/de/fd [5063983,5025] 0 2024-04-22T23:34:51.433 INFO:tasks.workunit.client.1.smithi157.stdout:3/93: truncate d0/d3/f1e 1327939 0 2024-04-22T23:34:51.434 INFO:tasks.workunit.client.1.smithi157.stdout:2/78: dread d7/d8/df/f16 [0,4194304] 0 2024-04-22T23:34:51.434 INFO:tasks.workunit.client.1.smithi157.stdout:2/79: fsync f3 0 2024-04-22T23:34:51.434 INFO:tasks.workunit.client.1.smithi157.stdout:2/80: write d7/d8/f9 [1002767,63241] 0 2024-04-22T23:34:51.437 INFO:tasks.workunit.client.1.smithi157.stdout:4/97: dwrite d3/f13 [0,4194304] 0 2024-04-22T23:34:51.437 INFO:tasks.workunit.client.1.smithi157.stdout:5/115: dwrite d1/fe [0,4194304] 0 2024-04-22T23:34:51.440 INFO:tasks.workunit.client.1.smithi157.stdout:4/98: creat d3/d11/f19 x:0 0 0 2024-04-22T23:34:51.440 INFO:tasks.workunit.client.1.smithi157.stdout:4/99: creat d3/d11/f1a x:0 0 0 2024-04-22T23:34:51.440 INFO:tasks.workunit.client.1.smithi157.stdout:4/100: stat d3/d4/f9 0 2024-04-22T23:34:51.440 INFO:tasks.workunit.client.1.smithi157.stdout:4/101: truncate d3/d4/f5 318118 0 2024-04-22T23:34:51.440 INFO:tasks.workunit.client.1.smithi157.stdout:4/102: fsync d3/d4/ff 0 2024-04-22T23:34:51.440 INFO:tasks.workunit.client.1.smithi157.stdout:4/103: write d3/d11/f19 [320220,96991] 0 2024-04-22T23:34:51.440 INFO:tasks.workunit.client.1.smithi157.stdout:4/104: chown d3/d11/f19 25046826 1 2024-04-22T23:34:51.440 INFO:tasks.workunit.client.1.smithi157.stdout:5/116: symlink d1/d5/l23 0 2024-04-22T23:34:51.440 INFO:tasks.workunit.client.1.smithi157.stdout:5/117: rename d1/d5/d7 to d1/d5/d7/d10/d24 22 2024-04-22T23:34:51.441 INFO:tasks.workunit.client.1.smithi157.stdout:5/118: read - d1/d5/dc/f1b zero size 2024-04-22T23:34:51.441 INFO:tasks.workunit.client.1.smithi157.stdout:5/119: chown d1/c18 4179 1 2024-04-22T23:34:51.441 INFO:tasks.workunit.client.0.smithi107.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-04-22T23:34:51.442 INFO:tasks.workunit.client.1.smithi157.stdout:4/105: truncate d3/d4/f9 1907910 0 2024-04-22T23:34:51.443 INFO:tasks.workunit.client.1.smithi157.stdout:5/120: truncate d1/d5/dc/f1d 2655215 0 2024-04-22T23:34:51.443 INFO:tasks.workunit.client.1.smithi157.stdout:5/121: dread - d1/d5/dc/f1b zero size 2024-04-22T23:34:51.445 INFO:tasks.workunit.client.1.smithi157.stdout:5/122: symlink d1/d1f/d22/l25 0 2024-04-22T23:34:51.445 INFO:tasks.workunit.client.1.smithi157.stdout:5/123: creat d1/f26 x:0 0 0 2024-04-22T23:34:51.448 INFO:tasks.workunit.client.1.smithi157.stdout:5/124: mkdir d1/d27 0 2024-04-22T23:34:51.449 INFO:tasks.workunit.client.1.smithi157.stdout:7/63: dread d0/f2 [0,4194304] 0 2024-04-22T23:34:51.450 INFO:tasks.workunit.client.1.smithi157.stdout:5/125: creat d1/d1f/f28 x:0 0 0 2024-04-22T23:34:51.451 INFO:tasks.workunit.client.1.smithi157.stdout:8/56: write d2/f3 [1673902,98617] 0 2024-04-22T23:34:51.451 INFO:tasks.workunit.client.1.smithi157.stdout:8/57: chown d2/d4/c9 10347 1 2024-04-22T23:34:51.451 INFO:tasks.workunit.client.1.smithi157.stdout:8/58: stat d2/lf 0 2024-04-22T23:34:51.451 INFO:tasks.workunit.client.1.smithi157.stdout:7/64: mkdir d0/d9/d10 0 2024-04-22T23:34:51.451 INFO:tasks.workunit.client.1.smithi157.stdout:7/65: chown d0/f7 6347 1 2024-04-22T23:34:51.453 INFO:tasks.workunit.client.1.smithi157.stdout:8/59: creat d2/d4/f10 x:0 0 0 2024-04-22T23:34:51.455 INFO:tasks.workunit.client.1.smithi157.stdout:7/66: symlink d0/d9/l11 0 2024-04-22T23:34:51.458 INFO:tasks.workunit.client.0.smithi107.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-22T23:34:51.459 INFO:tasks.workunit.client.1.smithi157.stdout:5/126: write d1/d5/f14 [1012867,31173] 0 2024-04-22T23:34:51.459 INFO:tasks.workunit.client.1.smithi157.stdout:5/127: chown d1/fe 10868536 1 2024-04-22T23:34:51.459 INFO:tasks.workunit.client.0.smithi107.stderr:+ make 2024-04-22T23:34:51.461 INFO:tasks.workunit.client.1.smithi157.stdout:3/94: dwrite d0/f14 [0,4194304] 0 2024-04-22T23:34:51.461 INFO:tasks.workunit.client.1.smithi157.stdout:7/67: write d0/f2 [604726,101623] 0 2024-04-22T23:34:51.461 INFO:tasks.workunit.client.1.smithi157.stdout:3/95: dread - d0/d3/f1b zero size 2024-04-22T23:34:51.462 INFO:tasks.workunit.client.1.smithi157.stdout:9/85: dwrite d4/df/f16 [0,4194304] 0 2024-04-22T23:34:51.462 INFO:tasks.workunit.client.1.smithi157.stdout:2/81: dwrite d7/d8/dd/f11 [0,4194304] 0 2024-04-22T23:34:51.464 INFO:tasks.workunit.client.1.smithi157.stdout:7/68: creat d0/d9/d10/f12 x:0 0 0 2024-04-22T23:34:51.464 INFO:tasks.workunit.client.1.smithi157.stdout:7/69: write d0/d9/d10/f12 [989864,70760] 0 2024-04-22T23:34:51.465 INFO:tasks.workunit.client.1.smithi157.stdout:7/70: write d0/f7 [49196,122975] 0 2024-04-22T23:34:51.466 INFO:tasks.workunit.client.1.smithi157.stdout:1/111: dwrite d4/fc [0,4194304] 0 2024-04-22T23:34:51.466 INFO:tasks.workunit.client.1.smithi157.stdout:7/71: write d0/d9/d10/f12 [200535,58735] 0 2024-04-22T23:34:51.466 INFO:tasks.workunit.client.1.smithi157.stdout:1/112: write d4/db/f24 [82996,76281] 0 2024-04-22T23:34:51.467 INFO:tasks.workunit.client.1.smithi157.stdout:1/113: creat d4/f25 x:0 0 0 2024-04-22T23:34:51.472 INFO:tasks.workunit.client.1.smithi157.stdout:7/72: creat d0/d9/db/f13 x:0 0 0 2024-04-22T23:34:51.472 INFO:tasks.workunit.client.1.smithi157.stdout:7/73: dread - d0/d9/db/f13 zero size 2024-04-22T23:34:51.473 INFO:tasks.workunit.client.1.smithi157.stdout:4/106: dwrite d3/d4/f5 [0,4194304] 0 2024-04-22T23:34:51.473 INFO:tasks.workunit.client.1.smithi157.stdout:4/107: readlink d3/ld 0 2024-04-22T23:34:51.474 INFO:tasks.workunit.client.1.smithi157.stdout:5/128: dwrite d1/d5/dc/f1b [0,4194304] 0 2024-04-22T23:34:51.474 INFO:tasks.workunit.client.1.smithi157.stdout:2/82: write d7/d8/dd/f11 [1053228,37992] 0 2024-04-22T23:34:51.479 INFO:tasks.workunit.client.1.smithi157.stdout:3/96: write d0/d3/f6 [2119299,39468] 0 2024-04-22T23:34:51.481 INFO:tasks.workunit.client.1.smithi157.stdout:4/108: write d3/fc [2548328,3633] 0 2024-04-22T23:34:51.481 INFO:tasks.workunit.client.1.smithi157.stdout:4/109: creat d3/f1b x:0 0 0 2024-04-22T23:34:51.481 INFO:tasks.workunit.client.1.smithi157.stdout:4/110: creat d3/d4/f1c x:0 0 0 2024-04-22T23:34:51.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:34:51 smithi107 ceph-mon[26355]: pgmap v252: 65 pgs: 65 active+clean; 212 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 5.4 MiB/s rd, 14 MiB/s wr, 801 op/s 2024-04-22T23:34:51.489 INFO:tasks.workunit.client.1.smithi157.stdout:7/74: mknod d0/c14 0 2024-04-22T23:34:51.489 INFO:tasks.workunit.client.1.smithi157.stdout:7/75: creat d0/d9/db/f15 x:0 0 0 2024-04-22T23:34:51.492 INFO:tasks.workunit.client.1.smithi157.stdout:7/76: dread d0/f8 [0,4194304] 0 2024-04-22T23:34:51.492 INFO:tasks.workunit.client.1.smithi157.stdout:7/77: write d0/d9/db/f13 [357058,91764] 0 2024-04-22T23:34:51.492 INFO:tasks.workunit.client.1.smithi157.stdout:7/78: stat d0/d9/de 0 2024-04-22T23:34:51.493 INFO:tasks.workunit.client.1.smithi157.stdout:7/79: write d0/d9/db/f15 [79861,11957] 0 2024-04-22T23:34:51.493 INFO:tasks.workunit.client.1.smithi157.stdout:5/129: mkdir d1/d27/d29 0 2024-04-22T23:34:51.494 INFO:tasks.workunit.client.1.smithi157.stdout:9/86: dwrite d4/df/fb [0,4194304] 0 2024-04-22T23:34:51.495 INFO:tasks.workunit.client.1.smithi157.stdout:2/83: creat d7/d12/f18 x:0 0 0 2024-04-22T23:34:51.495 INFO:tasks.workunit.client.1.smithi157.stdout:2/84: rename d7/d8 to d7/d8/d19 22 2024-04-22T23:34:51.495 INFO:tasks.workunit.client.1.smithi157.stdout:2/85: readlink d7/d8/lb 0 2024-04-22T23:34:51.497 INFO:tasks.workunit.client.1.smithi157.stdout:8/60: truncate d2/f3 1150494 0 2024-04-22T23:34:51.497 INFO:tasks.workunit.client.1.smithi157.stdout:1/114: dwrite d4/f25 [0,4194304] 0 2024-04-22T23:34:51.501 INFO:tasks.workunit.client.1.smithi157.stdout:5/130: dread d1/d5/dc/f1b [0,4194304] 0 2024-04-22T23:34:51.502 INFO:tasks.workunit.client.1.smithi157.stdout:9/87: symlink d4/df/l17 0 2024-04-22T23:34:51.508 INFO:tasks.workunit.client.1.smithi157.stdout:9/88: dread d4/fc [0,4194304] 0 2024-04-22T23:34:51.509 INFO:tasks.workunit.client.1.smithi157.stdout:1/115: link d4/f1b d4/f26 0 2024-04-22T23:34:51.511 INFO:tasks.workunit.client.1.smithi157.stdout:8/61: truncate d2/f5 1503573 0 2024-04-22T23:34:51.511 INFO:tasks.workunit.client.1.smithi157.stdout:8/62: readlink d2/d4/l7 0 2024-04-22T23:34:51.513 INFO:tasks.workunit.client.1.smithi157.stdout:9/89: rename d4/la to d4/df/l18 0 2024-04-22T23:34:51.514 INFO:tasks.workunit.client.1.smithi157.stdout:1/116: symlink d4/d17/d1d/l27 0 2024-04-22T23:34:51.514 INFO:tasks.workunit.client.1.smithi157.stdout:1/117: chown d4/db/c1a 3785 1 2024-04-22T23:34:51.519 INFO:tasks.workunit.client.1.smithi157.stdout:2/86: getdents d7/d12 0 2024-04-22T23:34:51.520 INFO:tasks.workunit.client.1.smithi157.stdout:7/80: dwrite d0/d9/fc [4194304,4194304] 0 2024-04-22T23:34:51.520 INFO:tasks.workunit.client.1.smithi157.stdout:2/87: unlink d7/d12/f18 0 2024-04-22T23:34:51.521 INFO:tasks.workunit.client.1.smithi157.stdout:5/131: dwrite d1/d5/f12 [0,4194304] 0 2024-04-22T23:34:51.522 INFO:tasks.workunit.client.1.smithi157.stdout:4/111: getdents d3 0 2024-04-22T23:34:51.522 INFO:tasks.workunit.client.1.smithi157.stdout:4/112: write d3/d4/f18 [18588,128753] 0 2024-04-22T23:34:51.523 INFO:tasks.workunit.client.1.smithi157.stdout:2/88: rename d7/d8/df/c17 to d7/c1a 0 2024-04-22T23:34:51.527 INFO:tasks.workunit.client.1.smithi157.stdout:5/132: rename d1/d1f/f28 to d1/d27/d29/f2a 0 2024-04-22T23:34:51.528 INFO:tasks.workunit.client.1.smithi157.stdout:4/113: getdents d3/d4 0 2024-04-22T23:34:51.529 INFO:tasks.workunit.client.1.smithi157.stdout:4/114: fsync d3/fc 0 2024-04-22T23:34:51.530 INFO:tasks.workunit.client.1.smithi157.stdout:5/133: symlink d1/d1f/l2b 0 2024-04-22T23:34:51.531 INFO:tasks.workunit.client.1.smithi157.stdout:4/115: mkdir d3/d11/d1d 0 2024-04-22T23:34:51.531 INFO:tasks.workunit.client.1.smithi157.stdout:4/116: truncate d3/d4/f16 924048 0 2024-04-22T23:34:51.531 INFO:tasks.workunit.client.1.smithi157.stdout:1/118: fsync d4/f26 0 2024-04-22T23:34:51.532 INFO:tasks.workunit.client.1.smithi157.stdout:4/117: creat d3/d11/d1d/f1e x:0 0 0 2024-04-22T23:34:51.532 INFO:tasks.workunit.client.1.smithi157.stdout:4/118: fdatasync d3/d4/f16 0 2024-04-22T23:34:51.532 INFO:tasks.workunit.client.1.smithi157.stdout:4/119: readlink d3/ld 0 2024-04-22T23:34:51.532 INFO:tasks.workunit.client.1.smithi157.stdout:4/120: dread - d3/d4/f1c zero size 2024-04-22T23:34:51.532 INFO:tasks.workunit.client.1.smithi157.stdout:4/121: write d3/fc [447828,20422] 0 2024-04-22T23:34:51.532 INFO:tasks.workunit.client.1.smithi157.stdout:1/119: write d4/f16 [400844,33813] 0 2024-04-22T23:34:51.532 INFO:tasks.workunit.client.1.smithi157.stdout:1/120: truncate d4/db/f14 4367845 0 2024-04-22T23:34:51.534 INFO:tasks.workunit.client.1.smithi157.stdout:4/122: rename d3/f13 to d3/d11/f1f 0 2024-04-22T23:34:51.534 INFO:tasks.workunit.client.1.smithi157.stdout:4/123: read - d3/d4/ff zero size 2024-04-22T23:34:51.534 INFO:tasks.workunit.client.1.smithi157.stdout:4/124: dread - d3/d4/f10 zero size 2024-04-22T23:34:51.534 INFO:tasks.workunit.client.1.smithi157.stdout:4/125: chown d3/d4/f1c 0 1 2024-04-22T23:34:51.535 INFO:tasks.workunit.client.1.smithi157.stdout:9/90: getdents d4 0 2024-04-22T23:34:51.536 INFO:tasks.workunit.client.1.smithi157.stdout:1/121: creat d4/d17/f28 x:0 0 0 2024-04-22T23:34:51.536 INFO:tasks.workunit.client.1.smithi157.stdout:1/122: write d4/f1b [780409,48090] 0 2024-04-22T23:34:51.536 INFO:tasks.workunit.client.1.smithi157.stdout:1/123: readlink d4/d17/d1d/l27 0 2024-04-22T23:34:51.536 INFO:tasks.workunit.client.1.smithi157.stdout:4/126: truncate d3/d4/fe 1698640 0 2024-04-22T23:34:51.536 INFO:tasks.workunit.client.1.smithi157.stdout:1/124: write d4/f1b [350447,63473] 0 2024-04-22T23:34:51.537 INFO:tasks.workunit.client.1.smithi157.stdout:4/127: unlink d3/ld 0 2024-04-22T23:34:51.539 INFO:tasks.workunit.client.1.smithi157.stdout:4/128: mkdir d3/d4/d20 0 2024-04-22T23:34:51.540 INFO:tasks.workunit.client.1.smithi157.stdout:6/242: sync 2024-04-22T23:34:51.542 INFO:tasks.workunit.client.1.smithi157.stdout:6/243: symlink d1/d2f/d3b/d49/d4c/l53 0 2024-04-22T23:34:51.543 INFO:tasks.workunit.client.1.smithi157.stdout:6/244: dread - d1/d36/f47 zero size 2024-04-22T23:34:51.544 INFO:tasks.workunit.client.1.smithi157.stdout:6/245: mkdir d1/d2f/d54 0 2024-04-22T23:34:51.545 INFO:tasks.workunit.client.1.smithi157.stdout:6/246: rmdir d1/d2f/d3b 39 2024-04-22T23:34:51.546 INFO:tasks.workunit.client.1.smithi157.stdout:9/91: dread d4/df/fb [4194304,4194304] 0 2024-04-22T23:34:51.547 INFO:tasks.workunit.client.1.smithi157.stdout:6/247: link d1/d2f/d3b/d49/d4c/l53 d1/d2f/d3b/d39/l55 0 2024-04-22T23:34:51.547 INFO:tasks.workunit.client.1.smithi157.stdout:6/248: write d1/d2f/f4a [1008168,100689] 0 2024-04-22T23:34:51.547 INFO:tasks.workunit.client.1.smithi157.stdout:9/92: creat d4/d15/f19 x:0 0 0 2024-04-22T23:34:51.547 INFO:tasks.workunit.client.1.smithi157.stdout:9/93: readlink d4/l14 0 2024-04-22T23:34:51.548 INFO:tasks.workunit.client.1.smithi157.stdout:5/134: dwrite d1/f26 [0,4194304] 0 2024-04-22T23:34:51.550 INFO:tasks.workunit.client.1.smithi157.stdout:9/94: dread d4/fc [0,4194304] 0 2024-04-22T23:34:51.550 INFO:tasks.workunit.client.1.smithi157.stdout:9/95: getdents d4 0 2024-04-22T23:34:51.551 INFO:tasks.workunit.client.1.smithi157.stdout:9/96: write d4/f8 [836510,75779] 0 2024-04-22T23:34:51.552 INFO:tasks.workunit.client.1.smithi157.stdout:9/97: write d4/fc [992858,87708] 0 2024-04-22T23:34:51.552 INFO:tasks.workunit.client.1.smithi157.stdout:9/98: chown d4/df/fb 936 1 2024-04-22T23:34:51.552 INFO:tasks.workunit.client.1.smithi157.stdout:9/99: dread - d4/d15/f19 zero size 2024-04-22T23:34:51.552 INFO:tasks.workunit.client.1.smithi157.stdout:9/100: chown d4/df/l17 82811 1 2024-04-22T23:34:51.554 INFO:tasks.workunit.client.1.smithi157.stdout:4/129: rename d3/d4/fe to d3/d4/d20/f21 0 2024-04-22T23:34:51.554 INFO:tasks.workunit.client.1.smithi157.stdout:4/130: chown d3/d11/f19 1760 1 2024-04-22T23:34:51.555 INFO:tasks.workunit.client.1.smithi157.stdout:9/101: mknod d4/d15/c1a 0 2024-04-22T23:34:51.555 INFO:tasks.workunit.client.1.smithi157.stdout:9/102: creat d4/df/f1b x:0 0 0 2024-04-22T23:34:51.555 INFO:tasks.workunit.client.1.smithi157.stdout:9/103: read - d4/d15/f19 zero size 2024-04-22T23:34:51.555 INFO:tasks.workunit.client.1.smithi157.stdout:9/104: readlink d4/l14 0 2024-04-22T23:34:51.556 INFO:tasks.workunit.client.1.smithi157.stdout:5/135: rename d1/d5/l23 to d1/d1f/l2c 0 2024-04-22T23:34:51.556 INFO:tasks.workunit.client.1.smithi157.stdout:5/136: stat d1/d1f/d22/l25 0 2024-04-22T23:34:51.559 INFO:tasks.workunit.client.1.smithi157.stdout:1/125: dwrite d4/f23 [0,4194304] 0 2024-04-22T23:34:51.559 INFO:tasks.workunit.client.1.smithi157.stdout:1/126: fsync d4/db/f1e 0 2024-04-22T23:34:51.559 INFO:tasks.workunit.client.1.smithi157.stdout:1/127: creat d4/f29 x:0 0 0 2024-04-22T23:34:51.559 INFO:tasks.workunit.client.1.smithi157.stdout:1/128: chown d4/f29 16188 1 2024-04-22T23:34:51.569 INFO:tasks.workunit.client.1.smithi157.stdout:6/249: dwrite d1/d36/f3f [0,4194304] 0 2024-04-22T23:34:51.570 INFO:tasks.workunit.client.1.smithi157.stdout:6/250: mkdir d1/d2f/d3b/d49/d4c/d56 0 2024-04-22T23:34:51.570 INFO:tasks.workunit.client.1.smithi157.stdout:6/251: creat d1/d36/f57 x:0 0 0 2024-04-22T23:34:51.571 INFO:tasks.workunit.client.1.smithi157.stdout:6/252: read d1/d17/f28 [186918,6520] 0 2024-04-22T23:34:51.573 INFO:tasks.workunit.client.1.smithi157.stdout:6/253: mknod d1/d17/c58 0 2024-04-22T23:34:51.573 INFO:tasks.workunit.client.1.smithi157.stdout:9/105: write d4/df/f16 [3066258,90651] 0 2024-04-22T23:34:51.576 INFO:tasks.workunit.client.1.smithi157.stdout:4/131: truncate d3/fc 3266096 0 2024-04-22T23:34:51.576 INFO:tasks.workunit.client.1.smithi157.stdout:4/132: fsync d3/d4/f18 0 2024-04-22T23:34:51.576 INFO:tasks.workunit.client.1.smithi157.stdout:4/133: creat d3/d11/f22 x:0 0 0 2024-04-22T23:34:51.577 INFO:tasks.workunit.client.1.smithi157.stdout:4/134: symlink d3/l23 0 2024-04-22T23:34:51.578 INFO:tasks.workunit.client.1.smithi157.stdout:4/135: rename d3/d11/f1a to d3/d11/d1d/f24 0 2024-04-22T23:34:51.580 INFO:tasks.workunit.client.1.smithi157.stdout:4/136: symlink d3/d4/l25 0 2024-04-22T23:34:51.580 INFO:tasks.workunit.client.1.smithi157.stdout:4/137: creat d3/d11/d1d/f26 x:0 0 0 2024-04-22T23:34:51.581 INFO:tasks.workunit.client.1.smithi157.stdout:5/137: dwrite d1/d27/d29/f2a [0,4194304] 0 2024-04-22T23:34:51.582 INFO:tasks.workunit.client.1.smithi157.stdout:6/254: write d1/d36/f3f [4148153,86335] 0 2024-04-22T23:34:51.586 INFO:tasks.workunit.client.1.smithi157.stdout:6/255: creat d1/d2f/d3b/d49/d4c/d56/f59 x:0 0 0 2024-04-22T23:34:51.588 INFO:tasks.workunit.client.1.smithi157.stdout:6/256: dread d1/d17/f24 [0,4194304] 0 2024-04-22T23:34:51.590 INFO:tasks.workunit.client.1.smithi157.stdout:5/138: write d1/f26 [1388346,84084] 0 2024-04-22T23:34:51.596 INFO:tasks.workunit.client.0.smithi107.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-04-22T23:34:51.624 INFO:tasks.workunit.client.1.smithi157.stdout:9/106: dwrite f3 [0,4194304] 0 2024-04-22T23:34:51.624 INFO:tasks.workunit.client.1.smithi157.stdout:4/138: dwrite d3/d4/f5 [0,4194304] 0 2024-04-22T23:34:51.624 INFO:tasks.workunit.client.1.smithi157.stdout:6/257: dwrite d1/d17/f27 [0,4194304] 0 2024-04-22T23:34:51.624 INFO:tasks.workunit.client.1.smithi157.stdout:4/139: truncate d3/d4/f16 998896 0 2024-04-22T23:34:51.624 INFO:tasks.workunit.client.1.smithi157.stdout:5/139: dwrite d1/f26 [0,4194304] 0 2024-04-22T23:34:51.625 INFO:tasks.workunit.client.1.smithi157.stdout:6/258: symlink d1/d17/l5a 0 2024-04-22T23:34:51.625 INFO:tasks.workunit.client.1.smithi157.stdout:6/259: chown d1/l1b 3367 1 2024-04-22T23:34:51.625 INFO:tasks.workunit.client.1.smithi157.stdout:4/140: getdents d3/d4/d20 0 2024-04-22T23:34:51.625 INFO:tasks.workunit.client.1.smithi157.stdout:4/141: fdatasync d3/d11/d1d/f1e 0 2024-04-22T23:34:51.626 INFO:tasks.workunit.client.1.smithi157.stdout:6/260: unlink d1/f29 0 2024-04-22T23:34:51.626 INFO:tasks.workunit.client.1.smithi157.stdout:6/261: write d1/d2f/d3b/d49/d4c/d56/f59 [305625,10761] 0 2024-04-22T23:34:51.627 INFO:tasks.workunit.client.1.smithi157.stdout:6/262: mknod d1/d2f/d3b/d49/d4c/d56/c5b 0 2024-04-22T23:34:51.628 INFO:tasks.workunit.client.1.smithi157.stdout:6/263: mkdir d1/d2f/d54/d5c 0 2024-04-22T23:34:51.628 INFO:tasks.workunit.client.1.smithi157.stdout:6/264: fsync d1/d2f/d3b/f2e 0 2024-04-22T23:34:51.633 INFO:tasks.workunit.client.1.smithi157.stdout:5/140: dread d1/f26 [0,4194304] 0 2024-04-22T23:34:51.633 INFO:tasks.workunit.client.1.smithi157.stdout:5/141: fdatasync d1/d5/dc/f1b 0 2024-04-22T23:34:51.633 INFO:tasks.workunit.client.1.smithi157.stdout:5/142: write d1/d5/f14 [2105613,18997] 0 2024-04-22T23:34:51.636 INFO:tasks.workunit.client.1.smithi157.stdout:5/143: link d1/d5/d7/c19 d1/d1f/c2d 0 2024-04-22T23:34:51.636 INFO:tasks.workunit.client.1.smithi157.stdout:5/144: read d1/d5/f15 [214408,111754] 0 2024-04-22T23:34:51.636 INFO:tasks.workunit.client.1.smithi157.stdout:5/145: write d1/d5/d7/f21 [905246,117088] 0 2024-04-22T23:34:51.636 INFO:tasks.workunit.client.1.smithi157.stdout:5/146: creat d1/d5/d7/f2e x:0 0 0 2024-04-22T23:34:51.638 INFO:tasks.workunit.client.1.smithi157.stdout:9/107: dread f3 [0,4194304] 0 2024-04-22T23:34:51.639 INFO:tasks.workunit.client.1.smithi157.stdout:4/142: dwrite d3/d4/f18 [0,4194304] 0 2024-04-22T23:34:51.639 INFO:tasks.workunit.client.1.smithi157.stdout:4/143: truncate d3/d11/d1d/f1e 221061 0 2024-04-22T23:34:51.639 INFO:tasks.workunit.client.1.smithi157.stdout:4/144: write d3/d4/f1c [410012,124248] 0 2024-04-22T23:34:51.640 INFO:tasks.workunit.client.1.smithi157.stdout:5/147: mkdir d1/d5/d7/d2f 0 2024-04-22T23:34:51.640 INFO:tasks.workunit.client.1.smithi157.stdout:5/148: write d1/f26 [4387907,37800] 0 2024-04-22T23:34:51.641 INFO:tasks.workunit.client.1.smithi157.stdout:9/108: symlink d4/df/l1c 0 2024-04-22T23:34:51.641 INFO:tasks.workunit.client.1.smithi157.stdout:9/109: stat d4/fc 0 2024-04-22T23:34:51.645 INFO:tasks.workunit.client.1.smithi157.stdout:9/110: dread d4/fc [0,4194304] 0 2024-04-22T23:34:51.647 INFO:tasks.workunit.client.1.smithi157.stdout:9/111: mkdir d4/d1d 0 2024-04-22T23:34:51.647 INFO:tasks.workunit.client.1.smithi157.stdout:9/112: chown d4/d15/f19 13806 1 2024-04-22T23:34:51.648 INFO:tasks.workunit.client.1.smithi157.stdout:6/265: dwrite d1/d2f/d3b/f2a [0,4194304] 0 2024-04-22T23:34:51.648 INFO:tasks.workunit.client.1.smithi157.stdout:6/266: chown d1/d17/c1f 775 1 2024-04-22T23:34:51.648 INFO:tasks.workunit.client.1.smithi157.stdout:6/267: stat d1/d2f/d3b/d49/c4b 0 2024-04-22T23:34:51.648 INFO:tasks.workunit.client.1.smithi157.stdout:5/149: fsync d1/d5/d7/f21 0 2024-04-22T23:34:51.654 INFO:tasks.workunit.client.1.smithi157.stdout:6/268: getdents d1/d2f/d3b/d49/d4c 0 2024-04-22T23:34:51.654 INFO:tasks.workunit.client.1.smithi157.stdout:6/269: fdatasync d1/d36/f57 0 2024-04-22T23:34:51.655 INFO:tasks.workunit.client.1.smithi157.stdout:5/150: symlink d1/d1f/d22/l30 0 2024-04-22T23:34:51.657 INFO:tasks.workunit.client.1.smithi157.stdout:5/151: link d1/d5/f15 d1/d5/d7/d10/f31 0 2024-04-22T23:34:51.661 INFO:tasks.workunit.client.1.smithi157.stdout:5/152: read d1/d5/dc/f1b [3157367,49850] 0 2024-04-22T23:34:51.664 INFO:tasks.workunit.client.1.smithi157.stdout:4/145: dwrite d3/d11/f22 [0,4194304] 0 2024-04-22T23:34:51.665 INFO:tasks.workunit.client.1.smithi157.stdout:9/113: dwrite d4/df/f12 [4194304,4194304] 0 2024-04-22T23:34:51.667 INFO:tasks.workunit.client.1.smithi157.stdout:6/270: truncate d1/d2f/d3b/f2a 1770314 0 2024-04-22T23:34:51.676 INFO:tasks.workunit.client.1.smithi157.stdout:9/114: write d4/df/f16 [1167434,85441] 0 2024-04-22T23:34:51.676 INFO:tasks.workunit.client.1.smithi157.stdout:9/115: chown d4/df/l10 37 1 2024-04-22T23:34:51.676 INFO:tasks.workunit.client.1.smithi157.stdout:9/116: write d4/fc [1466803,77716] 0 2024-04-22T23:34:51.678 INFO:tasks.workunit.client.1.smithi157.stdout:9/117: rename f3 to d4/d15/f1e 0 2024-04-22T23:34:51.679 INFO:tasks.workunit.client.1.smithi157.stdout:9/118: chown d4/df/l1c 182716 1 2024-04-22T23:34:51.679 INFO:tasks.workunit.client.1.smithi157.stdout:9/119: write d4/d15/f1e [4730052,94319] 0 2024-04-22T23:34:51.679 INFO:tasks.workunit.client.1.smithi157.stdout:9/120: write d4/df/f1b [650434,79160] 0 2024-04-22T23:34:51.679 INFO:tasks.workunit.client.1.smithi157.stdout:9/121: chown d4/d15/f1e 128154 1 2024-04-22T23:34:51.681 INFO:tasks.workunit.client.1.smithi157.stdout:6/271: dread d1/d17/f28 [0,4194304] 0 2024-04-22T23:34:51.835 INFO:tasks.workunit.client.1.smithi157.stdout:6/272: mkdir d1/d17/d5d 0 2024-04-22T23:34:51.835 INFO:tasks.workunit.client.1.smithi157.stdout:5/153: dwrite d1/d5/dc/f1b [4194304,4194304] 0 2024-04-22T23:34:51.835 INFO:tasks.workunit.client.1.smithi157.stdout:5/154: creat d1/d1f/d22/f32 x:0 0 0 2024-04-22T23:34:51.835 INFO:tasks.workunit.client.1.smithi157.stdout:4/146: dwrite d3/d11/f1f [0,4194304] 0 2024-04-22T23:34:51.835 INFO:tasks.workunit.client.1.smithi157.stdout:5/155: symlink d1/d27/d29/l33 0 2024-04-22T23:34:51.835 INFO:tasks.workunit.client.1.smithi157.stdout:5/156: creat d1/d5/d7/f34 x:0 0 0 2024-04-22T23:34:51.835 INFO:tasks.workunit.client.1.smithi157.stdout:4/147: symlink d3/d11/l27 0 2024-04-22T23:34:51.835 INFO:tasks.workunit.client.1.smithi157.stdout:4/148: write d3/d4/f18 [318758,26857] 0 2024-04-22T23:34:51.835 INFO:tasks.workunit.client.1.smithi157.stdout:4/149: chown d3/fc 2 1 2024-04-22T23:34:51.835 INFO:tasks.workunit.client.1.smithi157.stdout:4/150: mkdir d3/d4/d20/d28 0 2024-04-22T23:34:51.836 INFO:tasks.workunit.client.1.smithi157.stdout:9/122: dread d4/df/f1b [0,4194304] 0 2024-04-22T23:34:51.836 INFO:tasks.workunit.client.1.smithi157.stdout:9/123: stat d4/l14 0 2024-04-22T23:34:51.836 INFO:tasks.workunit.client.1.smithi157.stdout:9/124: readlink d4/ld 0 2024-04-22T23:34:51.836 INFO:tasks.workunit.client.1.smithi157.stdout:4/151: mknod d3/c29 0 2024-04-22T23:34:51.836 INFO:tasks.workunit.client.1.smithi157.stdout:4/152: dread - d3/d4/f17 zero size 2024-04-22T23:34:51.836 INFO:tasks.workunit.client.1.smithi157.stdout:4/153: dread - d3/d4/f17 zero size 2024-04-22T23:34:51.836 INFO:tasks.workunit.client.1.smithi157.stdout:0/87: sync 2024-04-22T23:34:51.836 INFO:tasks.workunit.client.1.smithi157.stdout:0/88: creat d17/d23/f27 x:0 0 0 2024-04-22T23:34:51.836 INFO:tasks.workunit.client.1.smithi157.stdout:9/125: getdents d4/d15 0 2024-04-22T23:34:51.836 INFO:tasks.workunit.client.1.smithi157.stdout:9/126: creat d4/df/f1f x:0 0 0 2024-04-22T23:34:51.836 INFO:tasks.workunit.client.1.smithi157.stdout:0/89: unlink d17/c18 0 2024-04-22T23:34:51.836 INFO:tasks.workunit.client.1.smithi157.stdout:0/90: creat d17/d23/f28 x:0 0 0 2024-04-22T23:34:51.836 INFO:tasks.workunit.client.1.smithi157.stdout:0/91: rename c14 to d17/d23/c29 0 2024-04-22T23:34:51.837 INFO:tasks.workunit.client.1.smithi157.stdout:0/92: stat d17/c1c 0 2024-04-22T23:34:51.837 INFO:tasks.workunit.client.1.smithi157.stdout:0/93: dread - d17/f1a zero size 2024-04-22T23:34:51.837 INFO:tasks.workunit.client.1.smithi157.stdout:0/94: write d17/f1a [84967,50624] 0 2024-04-22T23:34:51.837 INFO:tasks.workunit.client.1.smithi157.stdout:0/95: mknod d17/c2a 0 2024-04-22T23:34:51.837 INFO:tasks.workunit.client.1.smithi157.stdout:0/96: write f8 [785893,11397] 0 2024-04-22T23:34:51.837 INFO:tasks.workunit.client.1.smithi157.stdout:0/97: truncate d17/d23/f28 180780 0 2024-04-22T23:34:51.837 INFO:tasks.workunit.client.1.smithi157.stdout:0/98: mkdir d17/d2b 0 2024-04-22T23:34:51.837 INFO:tasks.workunit.client.1.smithi157.stdout:0/99: dread - d17/f1b zero size 2024-04-22T23:34:51.837 INFO:tasks.workunit.client.1.smithi157.stdout:5/157: dwrite d1/d5/d7/f2e [0,4194304] 0 2024-04-22T23:34:51.837 INFO:tasks.workunit.client.1.smithi157.stdout:5/158: creat d1/d1f/d22/f35 x:0 0 0 2024-04-22T23:34:51.837 INFO:tasks.workunit.client.1.smithi157.stdout:6/273: dwrite d1/d38/f50 [0,4194304] 0 2024-04-22T23:34:51.838 INFO:tasks.workunit.client.1.smithi157.stdout:9/127: dread d4/df/fb [0,4194304] 0 2024-04-22T23:34:51.838 INFO:tasks.workunit.client.1.smithi157.stdout:6/274: dread - d1/d2f/d3b/d39/d4e/f52 zero size 2024-04-22T23:34:51.838 INFO:tasks.workunit.client.1.smithi157.stdout:6/275: write d1/f34 [850242,71789] 0 2024-04-22T23:34:51.838 INFO:tasks.workunit.client.1.smithi157.stdout:5/159: unlink d1/d1f/d22/f32 0 2024-04-22T23:34:51.838 INFO:tasks.workunit.client.1.smithi157.stdout:4/154: dwrite d3/d4/d20/f21 [0,4194304] 0 2024-04-22T23:34:51.838 INFO:tasks.workunit.client.1.smithi157.stdout:9/128: rmdir d4 39 2024-04-22T23:34:51.838 INFO:tasks.workunit.client.1.smithi157.stdout:6/276: symlink d1/d36/l5e 0 2024-04-22T23:34:51.838 INFO:tasks.workunit.client.1.smithi157.stdout:6/277: getdents d1/d2f/d3b 0 2024-04-22T23:34:51.838 INFO:tasks.workunit.client.1.smithi157.stdout:6/278: creat d1/d2f/d3b/d39/d4e/f5f x:0 0 0 2024-04-22T23:34:51.838 INFO:tasks.workunit.client.1.smithi157.stdout:5/160: symlink d1/d5/l36 0 2024-04-22T23:34:51.838 INFO:tasks.workunit.client.1.smithi157.stdout:9/129: creat d4/f20 x:0 0 0 2024-04-22T23:34:51.838 INFO:tasks.workunit.client.1.smithi157.stdout:6/279: dread d1/d17/f43 [0,4194304] 0 2024-04-22T23:34:51.839 INFO:tasks.workunit.client.1.smithi157.stdout:4/155: write d3/d4/d20/f21 [1823847,83163] 0 2024-04-22T23:34:51.839 INFO:tasks.workunit.client.1.smithi157.stdout:5/161: symlink d1/d1f/l37 0 2024-04-22T23:34:51.839 INFO:tasks.workunit.client.1.smithi157.stdout:5/162: write d1/d5/d7/f34 [35679,43476] 0 2024-04-22T23:34:51.839 INFO:tasks.workunit.client.1.smithi157.stdout:5/163: truncate d1/d5/f9 920911 0 2024-04-22T23:34:51.839 INFO:tasks.workunit.client.1.smithi157.stdout:5/164: readlink d1/d1f/d22/l30 0 2024-04-22T23:34:51.839 INFO:tasks.workunit.client.1.smithi157.stdout:9/130: symlink d4/d15/l21 0 2024-04-22T23:34:51.839 INFO:tasks.workunit.client.1.smithi157.stdout:9/131: fsync d4/fc 0 2024-04-22T23:34:51.839 INFO:tasks.workunit.client.1.smithi157.stdout:6/280: creat d1/d2f/d54/f60 x:0 0 0 2024-04-22T23:34:51.839 INFO:tasks.workunit.client.1.smithi157.stdout:6/281: dread - d1/d2f/d3b/d39/d4e/f52 zero size 2024-04-22T23:34:51.839 INFO:tasks.workunit.client.1.smithi157.stdout:6/282: chown d1/d2f/d54/d5c 2839 1 2024-04-22T23:34:51.839 INFO:tasks.workunit.client.1.smithi157.stdout:0/100: dwrite f3 [0,4194304] 0 2024-04-22T23:34:51.839 INFO:tasks.workunit.client.1.smithi157.stdout:6/283: dread d1/f34 [0,4194304] 0 2024-04-22T23:34:51.839 INFO:tasks.workunit.client.1.smithi157.stdout:4/156: dread d3/d4/d20/f21 [0,4194304] 0 2024-04-22T23:34:51.840 INFO:tasks.workunit.client.1.smithi157.stdout:4/157: truncate d3/d4/f1c 966389 0 2024-04-22T23:34:51.840 INFO:tasks.workunit.client.1.smithi157.stdout:4/158: creat d3/d11/f2a x:0 0 0 2024-04-22T23:34:51.840 INFO:tasks.workunit.client.1.smithi157.stdout:4/159: dread - d3/d11/f2a zero size 2024-04-22T23:34:51.840 INFO:tasks.workunit.client.1.smithi157.stdout:4/160: truncate d3/d11/f2a 759861 0 2024-04-22T23:34:51.840 INFO:tasks.workunit.client.1.smithi157.stdout:6/284: creat d1/d2f/d3b/d49/d4c/f61 x:0 0 0 2024-04-22T23:34:51.840 INFO:tasks.workunit.client.1.smithi157.stdout:6/285: chown d1/c8 138121158 1 2024-04-22T23:34:51.840 INFO:tasks.workunit.client.1.smithi157.stdout:6/286: read - d1/d36/f4d zero size 2024-04-22T23:34:51.840 INFO:tasks.workunit.client.1.smithi157.stdout:6/287: fsync d1/d17/f28 0 2024-04-22T23:34:51.840 INFO:tasks.workunit.client.1.smithi157.stdout:9/132: read d4/df/f12 [3155473,82011] 0 2024-04-22T23:34:51.840 INFO:tasks.workunit.client.1.smithi157.stdout:9/133: chown d4/d15 27895 1 2024-04-22T23:34:51.840 INFO:tasks.workunit.client.1.smithi157.stdout:4/161: rename d3/d11/f19 to d3/d4/d20/d28/f2b 0 2024-04-22T23:34:51.841 INFO:tasks.workunit.client.1.smithi157.stdout:4/162: fdatasync d3/d4/d20/f21 0 2024-04-22T23:34:51.841 INFO:tasks.workunit.client.1.smithi157.stdout:6/288: rmdir d1/d38 39 2024-04-22T23:34:51.841 INFO:tasks.workunit.client.1.smithi157.stdout:9/134: creat d4/d1d/f22 x:0 0 0 2024-04-22T23:34:51.841 INFO:tasks.workunit.client.1.smithi157.stdout:9/135: mkdir d4/d15/d23 0 2024-04-22T23:34:51.841 INFO:tasks.workunit.client.1.smithi157.stdout:9/136: readlink d4/df/l10 0 2024-04-22T23:34:51.841 INFO:tasks.workunit.client.1.smithi157.stdout:9/137: write d4/df/f16 [2725367,10634] 0 2024-04-22T23:34:51.841 INFO:tasks.workunit.client.1.smithi157.stdout:9/138: write d4/d15/f19 [62018,60659] 0 2024-04-22T23:34:51.841 INFO:tasks.workunit.client.1.smithi157.stdout:9/139: write d4/d1d/f22 [86088,45015] 0 2024-04-22T23:34:51.841 INFO:tasks.workunit.client.1.smithi157.stdout:5/165: dwrite d1/d1f/d22/f35 [0,4194304] 0 2024-04-22T23:34:51.841 INFO:tasks.workunit.client.1.smithi157.stdout:4/163: write d3/d11/f22 [407474,9923] 0 2024-04-22T23:34:51.841 INFO:tasks.workunit.client.1.smithi157.stdout:5/166: link d1/d5/d7/f21 d1/d5/dc/f38 0 2024-04-22T23:34:51.841 INFO:tasks.workunit.client.1.smithi157.stdout:5/167: dread d1/d5/f9 [0,4194304] 0 2024-04-22T23:34:51.841 INFO:tasks.workunit.client.1.smithi157.stdout:5/168: link d1/d5/d7/cf d1/c39 0 2024-04-22T23:34:51.842 INFO:tasks.workunit.client.1.smithi157.stdout:5/169: dread d1/fe [0,4194304] 0 2024-04-22T23:34:51.842 INFO:tasks.workunit.client.1.smithi157.stdout:0/101: dwrite f3 [0,4194304] 0 2024-04-22T23:34:51.842 INFO:tasks.workunit.client.1.smithi157.stdout:5/170: link d1/d1f/l2c d1/d27/l3a 0 2024-04-22T23:34:51.842 INFO:tasks.workunit.client.1.smithi157.stdout:6/289: dwrite d1/f2c [0,4194304] 0 2024-04-22T23:34:51.842 INFO:tasks.workunit.client.1.smithi157.stdout:4/164: dwrite d3/d11/d1d/f24 [0,4194304] 0 2024-04-22T23:34:51.842 INFO:tasks.workunit.client.1.smithi157.stdout:5/171: getdents d1/d27 0 2024-04-22T23:34:51.842 INFO:tasks.workunit.client.1.smithi157.stdout:5/172: fsync d1/d5/d7/f21 0 2024-04-22T23:34:51.842 INFO:tasks.workunit.client.1.smithi157.stdout:6/290: rename d1/d17/l5a to d1/d2f/d3b/l62 0 2024-04-22T23:34:51.842 INFO:tasks.workunit.client.1.smithi157.stdout:4/165: symlink d3/d4/d20/l2c 0 2024-04-22T23:34:51.842 INFO:tasks.workunit.client.1.smithi157.stdout:6/291: unlink d1/d17/l1e 0 2024-04-22T23:34:51.842 INFO:tasks.workunit.client.1.smithi157.stdout:6/292: dread d1/d2f/d3b/f2e [0,4194304] 0 2024-04-22T23:34:51.843 INFO:tasks.workunit.client.1.smithi157.stdout:6/293: mknod d1/c63 0 2024-04-22T23:34:51.843 INFO:tasks.workunit.client.1.smithi157.stdout:0/102: dwrite d17/f1b [0,4194304] 0 2024-04-22T23:34:51.843 INFO:tasks.workunit.client.1.smithi157.stdout:9/140: dwrite d4/d15/f1e [0,4194304] 0 2024-04-22T23:34:51.843 INFO:tasks.workunit.client.1.smithi157.stdout:6/294: unlink d1/d2f/d54/f60 0 2024-04-22T23:34:51.843 INFO:tasks.workunit.client.1.smithi157.stdout:6/295: fsync d1/d36/f37 0 2024-04-22T23:34:51.843 INFO:tasks.workunit.client.1.smithi157.stdout:6/296: creat d1/d17/d5d/f64 x:0 0 0 2024-04-22T23:34:51.843 INFO:tasks.workunit.client.1.smithi157.stdout:6/297: symlink d1/d38/l65 0 2024-04-22T23:34:51.843 INFO:tasks.workunit.client.1.smithi157.stdout:5/173: truncate d1/d5/f9 124271 0 2024-04-22T23:34:51.843 INFO:tasks.workunit.client.1.smithi157.stdout:3/97: sync 2024-04-22T23:34:51.843 INFO:tasks.workunit.client.1.smithi157.stdout:1/129: sync 2024-04-22T23:34:51.843 INFO:tasks.workunit.client.1.smithi157.stdout:2/89: sync 2024-04-22T23:34:51.843 INFO:tasks.workunit.client.1.smithi157.stdout:7/81: sync 2024-04-22T23:34:51.843 INFO:tasks.workunit.client.1.smithi157.stdout:8/63: sync 2024-04-22T23:34:51.844 INFO:tasks.workunit.client.1.smithi157.stdout:3/98: mknod d0/d7/c20 0 2024-04-22T23:34:51.844 INFO:tasks.workunit.client.1.smithi157.stdout:3/99: write d0/f1c [2213261,11847] 0 2024-04-22T23:34:51.844 INFO:tasks.workunit.client.1.smithi157.stdout:3/100: write d0/d3/f1b [116040,22196] 0 2024-04-22T23:34:51.844 INFO:tasks.workunit.client.1.smithi157.stdout:8/64: getdents d2/d6/de 0 2024-04-22T23:34:51.844 INFO:tasks.workunit.client.1.smithi157.stdout:8/65: write d2/d4/f10 [968005,78528] 0 2024-04-22T23:34:51.844 INFO:tasks.workunit.client.1.smithi157.stdout:8/66: mkdir d2/d6/de/d11 0 2024-04-22T23:34:51.844 INFO:tasks.workunit.client.1.smithi157.stdout:8/67: write d2/f5 [1022434,50275] 0 2024-04-22T23:34:51.844 INFO:tasks.workunit.client.1.smithi157.stdout:2/90: dread f3 [0,4194304] 0 2024-04-22T23:34:51.844 INFO:tasks.workunit.client.1.smithi157.stdout:9/141: write d4/d15/f1e [3136084,77987] 0 2024-04-22T23:34:51.844 INFO:tasks.workunit.client.1.smithi157.stdout:4/166: dwrite d3/d11/d1d/f26 [0,4194304] 0 2024-04-22T23:34:51.844 INFO:tasks.workunit.client.1.smithi157.stdout:6/298: getdents d1/d2f/d54 0 2024-04-22T23:34:51.844 INFO:tasks.workunit.client.1.smithi157.stdout:2/91: unlink d7/d8/df/f16 0 2024-04-22T23:34:51.845 INFO:tasks.workunit.client.1.smithi157.stdout:2/92: stat d7/c10 0 2024-04-22T23:34:51.845 INFO:tasks.workunit.client.1.smithi157.stdout:9/142: creat d4/d15/d23/f24 x:0 0 0 2024-04-22T23:34:51.845 INFO:tasks.workunit.client.1.smithi157.stdout:0/103: dwrite d17/f22 [0,4194304] 0 2024-04-22T23:34:51.845 INFO:tasks.workunit.client.1.smithi157.stdout:1/130: dread d4/db/f1e [0,4194304] 0 2024-04-22T23:34:51.845 INFO:tasks.workunit.client.1.smithi157.stdout:0/104: fdatasync d17/f22 0 2024-04-22T23:34:51.845 INFO:tasks.workunit.client.1.smithi157.stdout:1/131: fsync d4/f26 0 2024-04-22T23:34:51.845 INFO:tasks.workunit.client.1.smithi157.stdout:1/132: chown d4/d17/d1d 381237 1 2024-04-22T23:34:51.845 INFO:tasks.workunit.client.1.smithi157.stdout:8/68: fdatasync d2/d6/de/fd 0 2024-04-22T23:34:51.845 INFO:tasks.workunit.client.1.smithi157.stdout:8/69: write d2/f5 [2185061,125267] 0 2024-04-22T23:34:51.845 INFO:tasks.workunit.client.1.smithi157.stdout:8/70: readlink d2/d4/l7 0 2024-04-22T23:34:51.845 INFO:tasks.workunit.client.1.smithi157.stdout:4/167: link d3/d11/f22 d3/d11/d1d/f2d 0 2024-04-22T23:34:51.846 INFO:tasks.workunit.client.1.smithi157.stdout:4/168: fsync d3/d4/f17 0 2024-04-22T23:34:51.846 INFO:tasks.workunit.client.1.smithi157.stdout:7/82: dwrite d0/f4 [0,4194304] 0 2024-04-22T23:34:51.846 INFO:tasks.workunit.client.1.smithi157.stdout:2/93: rename d7/c1a to d7/d8/dd/c1b 0 2024-04-22T23:34:51.846 INFO:tasks.workunit.client.1.smithi157.stdout:3/101: truncate d0/d3/f6 1784622 0 2024-04-22T23:34:51.846 INFO:tasks.workunit.client.1.smithi157.stdout:1/133: mknod d4/d17/c2a 0 2024-04-22T23:34:51.848 INFO:tasks.workunit.client.1.smithi157.stdout:0/105: dread d17/f22 [0,4194304] 0 2024-04-22T23:34:51.848 INFO:tasks.workunit.client.1.smithi157.stdout:8/71: creat d2/d6/de/f12 x:0 0 0 2024-04-22T23:34:51.848 INFO:tasks.workunit.client.1.smithi157.stdout:8/72: truncate d2/d6/de/f12 829410 0 2024-04-22T23:34:51.848 INFO:tasks.workunit.client.1.smithi157.stdout:8/73: stat d2/d4/l7 0 2024-04-22T23:34:51.848 INFO:tasks.workunit.client.1.smithi157.stdout:4/169: mkdir d3/d4/d20/d2e 0 2024-04-22T23:34:51.848 INFO:tasks.workunit.client.1.smithi157.stdout:4/170: dread d3/d4/f1c [0,4194304] 0 2024-04-22T23:34:51.848 INFO:tasks.workunit.client.1.smithi157.stdout:9/143: rename d4/l5 to d4/df/l25 0 2024-04-22T23:34:51.848 INFO:tasks.workunit.client.1.smithi157.stdout:9/144: creat d4/df/f26 x:0 0 0 2024-04-22T23:34:51.849 INFO:tasks.workunit.client.1.smithi157.stdout:2/94: symlink d7/d8/df/l1c 0 2024-04-22T23:34:51.849 INFO:tasks.workunit.client.1.smithi157.stdout:2/95: write d7/d8/dd/f11 [4549135,82212] 0 2024-04-22T23:34:51.849 INFO:tasks.workunit.client.1.smithi157.stdout:2/96: chown c6 6 1 2024-04-22T23:34:51.849 INFO:tasks.workunit.client.1.smithi157.stdout:1/134: truncate d4/f16 831938 0 2024-04-22T23:34:51.849 INFO:tasks.workunit.client.1.smithi157.stdout:0/106: creat d17/d2b/f2c x:0 0 0 2024-04-22T23:34:51.849 INFO:tasks.workunit.client.1.smithi157.stdout:4/171: creat d3/d4/f2f x:0 0 0 2024-04-22T23:34:51.849 INFO:tasks.workunit.client.1.smithi157.stdout:2/97: rename d7/d8/df/l1c to d7/d8/l1d 0 2024-04-22T23:34:51.849 INFO:tasks.workunit.client.1.smithi157.stdout:1/135: symlink d4/db/l2b 0 2024-04-22T23:34:51.849 INFO:tasks.workunit.client.1.smithi157.stdout:1/136: creat d4/d17/f2c x:0 0 0 2024-04-22T23:34:51.849 INFO:tasks.workunit.client.1.smithi157.stdout:1/137: creat d4/d17/f2d x:0 0 0 2024-04-22T23:34:51.849 INFO:tasks.workunit.client.1.smithi157.stdout:0/107: link d17/d23/f27 d17/f2d 0 2024-04-22T23:34:51.850 INFO:tasks.workunit.client.1.smithi157.stdout:6/299: dwrite d1/d2f/d3b/d49/d4c/d56/f59 [0,4194304] 0 2024-04-22T23:34:51.850 INFO:tasks.workunit.client.1.smithi157.stdout:7/83: dwrite d0/d9/db/f13 [0,4194304] 0 2024-04-22T23:34:51.850 INFO:tasks.workunit.client.1.smithi157.stdout:7/84: creat d0/d9/d10/f16 x:0 0 0 2024-04-22T23:34:51.854 INFO:tasks.workunit.client.1.smithi157.stdout:0/108: mknod d17/d2b/c2e 0 2024-04-22T23:34:51.854 INFO:tasks.workunit.client.1.smithi157.stdout:0/109: fdatasync d17/d23/f27 0 2024-04-22T23:34:51.854 INFO:tasks.workunit.client.1.smithi157.stdout:0/110: stat d17/f1b 0 2024-04-22T23:34:51.856 INFO:tasks.workunit.client.1.smithi157.stdout:0/111: dread d17/d23/f28 [0,4194304] 0 2024-04-22T23:34:51.857 INFO:tasks.workunit.client.1.smithi157.stdout:6/300: dread d1/d36/f3f [0,4194304] 0 2024-04-22T23:34:51.868 INFO:tasks.workunit.client.1.smithi157.stdout:7/85: creat d0/d9/f17 x:0 0 0 2024-04-22T23:34:51.868 INFO:tasks.workunit.client.1.smithi157.stdout:6/301: mkdir d1/d66 0 2024-04-22T23:34:51.868 INFO:tasks.workunit.client.1.smithi157.stdout:0/112: mknod d17/c2f 0 2024-04-22T23:34:51.872 INFO:tasks.workunit.client.1.smithi157.stdout:4/172: rename d3/d4/d20 to d3/d11/d30 0 2024-04-22T23:34:51.881 INFO:tasks.workunit.client.1.smithi157.stdout:0/113: truncate d17/f20 1570023 0 2024-04-22T23:34:51.881 INFO:tasks.workunit.client.1.smithi157.stdout:6/302: link d1/cc d1/d2f/d3b/d49/c67 0 2024-04-22T23:34:51.882 INFO:tasks.workunit.client.1.smithi157.stdout:6/303: fdatasync d1/d2f/d3b/d39/f46 0 2024-04-22T23:34:51.886 INFO:tasks.workunit.client.1.smithi157.stdout:9/145: fsync d4/df/f26 0 2024-04-22T23:34:51.890 INFO:tasks.workunit.client.1.smithi157.stdout:0/114: mknod d17/d2b/c30 0 2024-04-22T23:34:51.890 INFO:tasks.workunit.client.1.smithi157.stdout:0/115: creat d17/d23/f31 x:0 0 0 2024-04-22T23:34:51.891 INFO:tasks.workunit.client.1.smithi157.stdout:0/116: creat d17/d23/f32 x:0 0 0 2024-04-22T23:34:51.893 INFO:tasks.workunit.client.1.smithi157.stdout:9/146: mkdir d4/d15/d23/d27 0 2024-04-22T23:34:51.893 INFO:tasks.workunit.client.1.smithi157.stdout:9/147: dread - d4/df/f1f zero size 2024-04-22T23:34:51.893 INFO:tasks.workunit.client.1.smithi157.stdout:9/148: write d4/f8 [1032853,41464] 0 2024-04-22T23:34:51.896 INFO:tasks.workunit.client.1.smithi157.stdout:0/117: symlink d17/d23/l33 0 2024-04-22T23:34:51.900 INFO:tasks.workunit.client.1.smithi157.stdout:4/173: write d3/d4/f5 [883264,12054] 0 2024-04-22T23:34:51.902 INFO:tasks.workunit.client.1.smithi157.stdout:2/98: rmdir d7/d8/df 0 2024-04-22T23:34:51.906 INFO:tasks.workunit.client.1.smithi157.stdout:4/174: mkdir d3/d11/d30/d28/d31 0 2024-04-22T23:34:51.907 INFO:tasks.workunit.client.1.smithi157.stdout:2/99: symlink d7/l1e 0 2024-04-22T23:34:51.912 INFO:tasks.workunit.client.1.smithi157.stdout:4/175: mknod d3/d11/d30/d28/d31/c32 0 2024-04-22T23:34:51.923 INFO:tasks.workunit.client.1.smithi157.stdout:4/176: unlink d3/d11/l27 0 2024-04-22T23:34:51.945 INFO:tasks.workunit.client.1.smithi157.stdout:4/177: write d3/d4/f16 [639238,19219] 0 2024-04-22T23:34:51.945 INFO:tasks.workunit.client.1.smithi157.stdout:4/178: write d3/d4/f16 [1356070,49820] 0 2024-04-22T23:34:51.946 INFO:tasks.workunit.client.1.smithi157.stdout:4/179: read - d3/d4/f2f zero size 2024-04-22T23:34:51.946 INFO:tasks.workunit.client.1.smithi157.stdout:4/180: truncate d3/d4/f10 735901 0 2024-04-22T23:34:51.946 INFO:tasks.workunit.client.1.smithi157.stdout:4/181: chown d3/d11/d1d/f1e 1812817 1 2024-04-22T23:34:51.948 INFO:tasks.workunit.client.1.smithi157.stdout:3/102: write d0/f15 [1537869,3658] 0 2024-04-22T23:34:51.948 INFO:tasks.workunit.client.1.smithi157.stdout:3/103: readlink d0/d3/l12 0 2024-04-22T23:34:51.958 INFO:tasks.workunit.client.1.smithi157.stdout:7/86: read d0/f4 [4170669,77485] 0 2024-04-22T23:34:51.986 INFO:tasks.workunit.client.1.smithi157.stdout:7/87: dread d0/d9/d10/f12 [0,4194304] 0 2024-04-22T23:34:51.986 INFO:tasks.workunit.client.1.smithi157.stdout:7/88: creat d0/d9/d10/f18 x:0 0 0 2024-04-22T23:34:51.989 INFO:tasks.workunit.client.1.smithi157.stdout:7/89: rename d0/d9/d10/f16 to d0/d9/d10/f19 0 2024-04-22T23:34:51.991 INFO:tasks.workunit.client.1.smithi157.stdout:7/90: link d0/l5 d0/d9/l1a 0 2024-04-22T23:34:52.018 INFO:tasks.workunit.client.1.smithi157.stdout:8/74: dwrite d2/d6/de/fd [0,4194304] 0 2024-04-22T23:34:52.025 INFO:tasks.workunit.client.1.smithi157.stdout:8/75: mkdir d2/d6/de/d11/d13 0 2024-04-22T23:34:52.025 INFO:tasks.workunit.client.1.smithi157.stdout:8/76: rename d2 to d2/d6/de/d14 22 2024-04-22T23:34:52.025 INFO:tasks.workunit.client.1.smithi157.stdout:8/77: rename d2 to d2/d6/de/d11/d13/d15 22 2024-04-22T23:34:52.027 INFO:tasks.workunit.client.1.smithi157.stdout:8/78: rename d2/d4/ca to d2/d6/c16 0 2024-04-22T23:34:52.052 INFO:tasks.workunit.client.1.smithi157.stdout:8/79: dread d2/d6/de/fd [0,4194304] 0 2024-04-22T23:34:52.052 INFO:tasks.workunit.client.1.smithi157.stdout:8/80: chown d2/d6/de 21977467 1 2024-04-22T23:34:52.070 INFO:tasks.workunit.client.1.smithi157.stdout:8/81: dread d2/f5 [0,4194304] 0 2024-04-22T23:34:52.072 INFO:tasks.workunit.client.1.smithi157.stdout:8/82: mkdir d2/d6/de/d17 0 2024-04-22T23:34:52.073 INFO:tasks.workunit.client.1.smithi157.stdout:6/304: dwrite d1/d2f/f4a [0,4194304] 0 2024-04-22T23:34:52.073 INFO:tasks.workunit.client.1.smithi157.stdout:8/83: symlink d2/d6/de/d11/l18 0 2024-04-22T23:34:52.073 INFO:tasks.workunit.client.1.smithi157.stdout:8/84: write d2/f3 [785583,103668] 0 2024-04-22T23:34:52.107 INFO:tasks.workunit.client.1.smithi157.stdout:0/118: truncate d17/f20 1633838 0 2024-04-22T23:34:52.109 INFO:tasks.workunit.client.1.smithi157.stdout:0/119: mkdir d17/d23/d34 0 2024-04-22T23:34:52.111 INFO:tasks.workunit.client.1.smithi157.stdout:0/120: link d17/c21 d17/c35 0 2024-04-22T23:34:52.111 INFO:tasks.workunit.client.1.smithi157.stdout:0/121: dread - d17/f2d zero size 2024-04-22T23:34:52.111 INFO:tasks.workunit.client.1.smithi157.stdout:0/122: readlink l12 0 2024-04-22T23:34:52.116 INFO:tasks.workunit.client.1.smithi157.stdout:0/123: rmdir d17/d2b 39 2024-04-22T23:34:52.117 INFO:tasks.workunit.client.1.smithi157.stdout:0/124: rename d17/d2b/c30 to d17/c36 0 2024-04-22T23:34:52.120 INFO:tasks.workunit.client.1.smithi157.stdout:6/305: read d1/d17/f27 [1069746,74978] 0 2024-04-22T23:34:52.123 INFO:tasks.workunit.client.1.smithi157.stdout:0/125: mkdir d17/d23/d34/d37 0 2024-04-22T23:34:52.130 INFO:tasks.workunit.client.1.smithi157.stdout:9/149: dwrite d4/d15/f1e [0,4194304] 0 2024-04-22T23:34:52.130 INFO:tasks.workunit.client.1.smithi157.stdout:9/150: truncate d4/d15/f19 1098570 0 2024-04-22T23:34:52.132 INFO:tasks.workunit.client.1.smithi157.stdout:9/151: symlink d4/l28 0 2024-04-22T23:34:52.151 INFO:tasks.workunit.client.1.smithi157.stdout:0/126: dread f3 [0,4194304] 0 2024-04-22T23:34:52.151 INFO:tasks.workunit.client.1.smithi157.stdout:0/127: chown c13 7620208 1 2024-04-22T23:34:52.399 INFO:tasks.workunit.client.1.smithi157.stdout:2/100: dwrite d7/d8/dd/f11 [0,4194304] 0 2024-04-22T23:34:52.403 INFO:tasks.workunit.client.1.smithi157.stdout:2/101: mkdir d7/d1f 0 2024-04-22T23:34:52.406 INFO:tasks.workunit.client.1.smithi157.stdout:2/102: symlink d7/d12/l20 0 2024-04-22T23:34:52.413 INFO:tasks.workunit.client.1.smithi157.stdout:2/103: dread f3 [0,4194304] 0 2024-04-22T23:34:52.415 INFO:tasks.workunit.client.1.smithi157.stdout:2/104: creat d7/d8/dd/f21 x:0 0 0 2024-04-22T23:34:52.415 INFO:tasks.workunit.client.1.smithi157.stdout:2/105: chown d7/d8/dd/c1b 2155 1 2024-04-22T23:34:52.419 INFO:tasks.workunit.client.1.smithi157.stdout:2/106: rename d7/d12/c14 to d7/d8/c22 0 2024-04-22T23:34:52.421 INFO:tasks.workunit.client.1.smithi157.stdout:2/107: mkdir d7/d12/d23 0 2024-04-22T23:34:52.423 INFO:tasks.workunit.client.1.smithi157.stdout:2/108: rename d7/d8/lb to d7/d8/l24 0 2024-04-22T23:34:52.425 INFO:tasks.workunit.client.1.smithi157.stdout:2/109: truncate f3 646477 0 2024-04-22T23:34:52.426 INFO:tasks.workunit.client.1.smithi157.stdout:2/110: mkdir d7/d8/d25 0 2024-04-22T23:34:52.484 INFO:tasks.workunit.client.1.smithi157.stdout:4/182: dwrite d3/d11/d30/f21 [0,4194304] 0 2024-04-22T23:34:52.484 INFO:tasks.workunit.client.1.smithi157.stdout:4/183: fsync d3/d11/f1f 0 2024-04-22T23:34:52.484 INFO:tasks.workunit.client.1.smithi157.stdout:4/184: write d3/d4/f5 [1034736,115996] 0 2024-04-22T23:34:52.489 INFO:tasks.workunit.client.1.smithi157.stdout:0/128: dwrite d17/f20 [0,4194304] 0 2024-04-22T23:34:52.491 INFO:tasks.workunit.client.1.smithi157.stdout:0/129: mkdir d17/d38 0 2024-04-22T23:34:52.514 INFO:tasks.workunit.client.1.smithi157.stdout:6/306: dwrite d1/d2f/d3b/f32 [0,4194304] 0 2024-04-22T23:34:52.514 INFO:tasks.workunit.client.1.smithi157.stdout:6/307: chown d1/d2f/d54 882 1 2024-04-22T23:34:52.516 INFO:tasks.workunit.client.1.smithi157.stdout:6/308: truncate d1/d2f/d3b/f2a 1534910 0 2024-04-22T23:34:52.517 INFO:tasks.workunit.client.1.smithi157.stdout:6/309: getdents d1/d66 0 2024-04-22T23:34:52.517 INFO:tasks.workunit.client.1.smithi157.stdout:6/310: write d1/d17/d5d/f64 [120170,99821] 0 2024-04-22T23:34:52.517 INFO:tasks.workunit.client.1.smithi157.stdout:5/174: sync 2024-04-22T23:34:52.518 INFO:tasks.workunit.client.1.smithi157.stdout:1/138: sync 2024-04-22T23:34:52.520 INFO:tasks.workunit.client.1.smithi157.stdout:5/175: unlink d1/d5/f12 0 2024-04-22T23:34:52.520 INFO:tasks.workunit.client.1.smithi157.stdout:5/176: chown d1/d1f/d22/f35 26096 1 2024-04-22T23:34:52.522 INFO:tasks.workunit.client.1.smithi157.stdout:1/139: rename d4/f29 to d4/db/f2e 0 2024-04-22T23:34:52.522 INFO:tasks.workunit.client.1.smithi157.stdout:1/140: dread - d4/db/f2e zero size 2024-04-22T23:34:52.524 INFO:tasks.workunit.client.1.smithi157.stdout:0/130: dread fc [4194304,4194304] 0 2024-04-22T23:34:52.525 INFO:tasks.workunit.client.1.smithi157.stdout:5/177: mknod d1/d5/dc/c3b 0 2024-04-22T23:34:52.525 INFO:tasks.workunit.client.1.smithi157.stdout:5/178: write d1/d1f/d22/f35 [4624449,125710] 0 2024-04-22T23:34:52.525 INFO:tasks.workunit.client.1.smithi157.stdout:5/179: creat d1/d1f/d22/f3c x:0 0 0 2024-04-22T23:34:52.526 INFO:tasks.workunit.client.1.smithi157.stdout:6/311: rmdir d1/d66 0 2024-04-22T23:34:52.526 INFO:tasks.workunit.client.1.smithi157.stdout:6/312: write d1/d17/f24 [1673053,89209] 0 2024-04-22T23:34:52.527 INFO:tasks.workunit.client.1.smithi157.stdout:0/131: symlink d17/d2b/l39 0 2024-04-22T23:34:52.529 INFO:tasks.workunit.client.1.smithi157.stdout:5/180: truncate d1/d27/d29/f2a 1519747 0 2024-04-22T23:34:52.529 INFO:tasks.workunit.client.1.smithi157.stdout:6/313: rmdir d1/d38 39 2024-04-22T23:34:52.532 INFO:tasks.workunit.client.1.smithi157.stdout:0/132: link d17/l1d d17/d23/l3a 0 2024-04-22T23:34:52.532 INFO:tasks.workunit.client.1.smithi157.stdout:0/133: fsync f8 0 2024-04-22T23:34:52.533 INFO:tasks.workunit.client.1.smithi157.stdout:5/181: creat d1/d5/d7/d10/f3d x:0 0 0 2024-04-22T23:34:52.535 INFO:tasks.workunit.client.1.smithi157.stdout:6/314: symlink d1/d17/l68 0 2024-04-22T23:34:52.535 INFO:tasks.workunit.client.1.smithi157.stdout:0/134: mknod d17/c3b 0 2024-04-22T23:34:52.535 INFO:tasks.workunit.client.1.smithi157.stdout:0/135: fsync d17/d23/f28 0 2024-04-22T23:34:52.537 INFO:tasks.workunit.client.1.smithi157.stdout:5/182: mkdir d1/d5/d3e 0 2024-04-22T23:34:52.537 INFO:tasks.workunit.client.1.smithi157.stdout:5/183: creat d1/f3f x:0 0 0 2024-04-22T23:34:52.537 INFO:tasks.workunit.client.1.smithi157.stdout:0/136: mknod d17/d38/c3c 0 2024-04-22T23:34:52.540 INFO:tasks.workunit.client.1.smithi157.stdout:0/137: rename f8 to d17/d2b/f3d 0 2024-04-22T23:34:52.540 INFO:tasks.workunit.client.1.smithi157.stdout:0/138: write fe [791018,101346] 0 2024-04-22T23:34:52.540 INFO:tasks.workunit.client.1.smithi157.stdout:0/139: creat d17/d23/f3e x:0 0 0 2024-04-22T23:34:52.547 INFO:tasks.workunit.client.1.smithi157.stdout:5/184: dread d1/d5/dc/f1d [0,4194304] 0 2024-04-22T23:34:52.548 INFO:tasks.workunit.client.1.smithi157.stdout:5/185: write d1/d5/d7/f34 [519000,72630] 0 2024-04-22T23:34:52.548 INFO:tasks.workunit.client.1.smithi157.stdout:5/186: chown d1/d5/d7/cf 480 1 2024-04-22T23:34:52.613 INFO:tasks.workunit.client.1.smithi157.stdout:9/152: write d4/d15/f19 [1001358,118823] 0 2024-04-22T23:34:52.613 INFO:tasks.workunit.client.1.smithi157.stdout:9/153: read d4/d15/f19 [552331,27201] 0 2024-04-22T23:34:52.614 INFO:tasks.workunit.client.1.smithi157.stdout:9/154: creat d4/f29 x:0 0 0 2024-04-22T23:34:52.616 INFO:tasks.workunit.client.1.smithi157.stdout:9/155: symlink d4/l2a 0 2024-04-22T23:34:52.616 INFO:tasks.workunit.client.1.smithi157.stdout:9/156: creat d4/d1d/f2b x:0 0 0 2024-04-22T23:34:52.616 INFO:tasks.workunit.client.1.smithi157.stdout:9/157: write d4/df/f1b [1165021,64092] 0 2024-04-22T23:34:52.619 INFO:tasks.workunit.client.1.smithi157.stdout:9/158: rename d4/df/l18 to d4/d15/d23/l2c 0 2024-04-22T23:34:52.621 INFO:tasks.workunit.client.1.smithi157.stdout:6/315: write d1/d17/f31 [317706,85055] 0 2024-04-22T23:34:52.621 INFO:tasks.workunit.client.1.smithi157.stdout:9/159: mkdir d4/d2d 0 2024-04-22T23:34:52.621 INFO:tasks.workunit.client.1.smithi157.stdout:9/160: getdents d4/d15/d23/d27 0 2024-04-22T23:34:52.634 INFO:tasks.workunit.client.1.smithi157.stdout:4/185: dwrite d3/d11/d1d/f26 [4194304,4194304] 0 2024-04-22T23:34:52.637 INFO:tasks.workunit.client.1.smithi157.stdout:4/186: mkdir d3/d33 0 2024-04-22T23:34:52.640 INFO:tasks.workunit.client.1.smithi157.stdout:4/187: unlink d3/d11/d30/f21 0 2024-04-22T23:34:52.643 INFO:tasks.workunit.client.1.smithi157.stdout:4/188: mkdir d3/d11/d34 0 2024-04-22T23:34:52.645 INFO:tasks.workunit.client.1.smithi157.stdout:6/316: write d1/d17/f24 [755088,90639] 0 2024-04-22T23:34:52.647 INFO:tasks.workunit.client.1.smithi157.stdout:4/189: rmdir d3/d11/d30/d2e 0 2024-04-22T23:34:52.649 INFO:tasks.workunit.client.1.smithi157.stdout:4/190: mknod d3/d11/d30/c35 0 2024-04-22T23:34:52.654 INFO:tasks.workunit.client.1.smithi157.stdout:4/191: read d3/d11/d1d/f1e [59461,28836] 0 2024-04-22T23:34:52.654 INFO:tasks.workunit.client.1.smithi157.stdout:4/192: chown d3/d4/f10 120735075 1 2024-04-22T23:34:52.715 INFO:tasks.workunit.client.1.smithi157.stdout:4/193: dwrite d3/d4/f5 [0,4194304] 0 2024-04-22T23:34:52.715 INFO:tasks.workunit.client.1.smithi157.stdout:4/194: fsync d3/d11/f2a 0 2024-04-22T23:34:52.715 INFO:tasks.workunit.client.1.smithi157.stdout:4/195: write d3/fc [1145216,123398] 0 2024-04-22T23:34:52.728 INFO:tasks.workunit.client.1.smithi157.stdout:4/196: dread d3/d4/f5 [0,4194304] 0 2024-04-22T23:34:52.729 INFO:tasks.workunit.client.1.smithi157.stdout:4/197: mkdir d3/d11/d30/d36 0 2024-04-22T23:34:52.729 INFO:tasks.workunit.client.1.smithi157.stdout:4/198: truncate d3/d4/f17 830142 0 2024-04-22T23:34:52.767 INFO:tasks.workunit.client.1.smithi157.stdout:6/317: dread d1/d17/f24 [0,4194304] 0 2024-04-22T23:34:52.767 INFO:tasks.workunit.client.1.smithi157.stdout:6/318: fsync d1/d17/f28 0 2024-04-22T23:34:52.768 INFO:tasks.workunit.client.1.smithi157.stdout:6/319: symlink d1/d2f/d54/d5c/l69 0 2024-04-22T23:34:52.768 INFO:tasks.workunit.client.1.smithi157.stdout:6/320: chown d1/d17/d5d 118077 1 2024-04-22T23:34:52.771 INFO:tasks.workunit.client.1.smithi157.stdout:6/321: link d1/lb d1/d2f/d54/d5c/l6a 0 2024-04-22T23:34:52.771 INFO:tasks.workunit.client.1.smithi157.stdout:6/322: dread - d1/d36/f4d zero size 2024-04-22T23:34:52.781 INFO:tasks.workunit.client.1.smithi157.stdout:4/199: rmdir d3/d11/d30 39 2024-04-22T23:34:52.783 INFO:tasks.workunit.client.1.smithi157.stdout:4/200: rename d3/fc to d3/d11/d30/d28/f37 0 2024-04-22T23:34:52.785 INFO:tasks.workunit.client.1.smithi157.stdout:4/201: mknod d3/d11/d30/d28/d31/c38 0 2024-04-22T23:34:52.787 INFO:tasks.workunit.client.1.smithi157.stdout:4/202: mkdir d3/d11/d34/d39 0 2024-04-22T23:34:52.787 INFO:tasks.workunit.client.1.smithi157.stdout:4/203: write d3/d11/f2a [1618599,78107] 0 2024-04-22T23:34:52.787 INFO:tasks.workunit.client.1.smithi157.stdout:4/204: chown d3/d11/d30/d28/d31/c32 2 1 2024-04-22T23:34:52.810 INFO:tasks.workunit.client.1.smithi157.stdout:4/205: dwrite d3/d11/d1d/f1e [0,4194304] 0 2024-04-22T23:34:52.813 INFO:tasks.workunit.client.1.smithi157.stdout:4/206: symlink d3/d11/d30/d28/d31/l3a 0 2024-04-22T23:34:52.814 INFO:tasks.workunit.client.1.smithi157.stdout:5/187: dwrite d1/d5/f14 [4194304,4194304] 0 2024-04-22T23:34:52.814 INFO:tasks.workunit.client.1.smithi157.stdout:5/188: chown d1/d5/d7/cf 1135458 1 2024-04-22T23:34:52.815 INFO:tasks.workunit.client.1.smithi157.stdout:4/207: mknod d3/d11/d30/d36/c3b 0 2024-04-22T23:34:52.815 INFO:tasks.workunit.client.1.smithi157.stdout:4/208: chown d3/d4/ff 3978197 1 2024-04-22T23:34:52.817 INFO:tasks.workunit.client.1.smithi157.stdout:5/189: rename d1/d5/d7/d10/c11 to d1/c40 0 2024-04-22T23:34:52.819 INFO:tasks.workunit.client.1.smithi157.stdout:5/190: rename d1/c39 to d1/c41 0 2024-04-22T23:34:52.820 INFO:tasks.workunit.client.1.smithi157.stdout:5/191: truncate d1/d5/dc/f1d 283069 0 2024-04-22T23:34:52.822 INFO:tasks.workunit.client.1.smithi157.stdout:5/192: symlink d1/d1f/d22/l42 0 2024-04-22T23:34:52.824 INFO:tasks.workunit.client.1.smithi157.stdout:5/193: mkdir d1/d5/dc/d43 0 2024-04-22T23:34:52.824 INFO:tasks.workunit.client.1.smithi157.stdout:4/209: read d3/d11/d1d/f24 [1405934,44220] 0 2024-04-22T23:34:52.827 INFO:tasks.workunit.client.1.smithi157.stdout:4/210: rename d3/d4/f10 to d3/d11/d30/d28/f3c 0 2024-04-22T23:34:52.827 INFO:tasks.workunit.client.1.smithi157.stdout:4/211: readlink d3/d4/lb 0 2024-04-22T23:34:52.832 INFO:tasks.workunit.client.1.smithi157.stdout:4/212: fsync d3/d11/f2a 0 2024-04-22T23:34:52.833 INFO:tasks.workunit.client.1.smithi157.stdout:4/213: write d3/d4/ff [74483,49165] 0 2024-04-22T23:34:52.833 INFO:tasks.workunit.client.1.smithi157.stdout:4/214: stat d3/d11/d30/d28/d31/c38 0 2024-04-22T23:34:52.835 INFO:tasks.workunit.client.1.smithi157.stdout:4/215: rename d3/d4/f17 to d3/d33/f3d 0 2024-04-22T23:34:52.835 INFO:tasks.workunit.client.1.smithi157.stdout:4/216: creat d3/d4/f3e x:0 0 0 2024-04-22T23:34:52.837 INFO:tasks.workunit.client.1.smithi157.stdout:4/217: mknod d3/c3f 0 2024-04-22T23:34:52.837 INFO:tasks.workunit.client.1.smithi157.stdout:4/218: read d3/d4/f9 [57149,74413] 0 2024-04-22T23:34:52.838 INFO:tasks.workunit.client.1.smithi157.stdout:4/219: chown d3/d11/c12 32754 1 2024-04-22T23:34:52.838 INFO:tasks.workunit.client.1.smithi157.stdout:4/220: write d3/d11/d1d/f1e [4518672,71074] 0 2024-04-22T23:34:52.839 INFO:tasks.workunit.client.1.smithi157.stdout:4/221: mkdir d3/d33/d40 0 2024-04-22T23:34:52.849 INFO:tasks.workunit.client.1.smithi157.stdout:3/104: dread d0/d3/f1b [0,4194304] 0 2024-04-22T23:34:52.849 INFO:tasks.workunit.client.1.smithi157.stdout:3/105: chown d0/d3/fe 4 1 2024-04-22T23:34:52.849 INFO:tasks.workunit.client.1.smithi157.stdout:3/106: fdatasync d0/d3/f1f 0 2024-04-22T23:34:52.856 INFO:tasks.workunit.client.1.smithi157.stdout:7/91: dread d0/f7 [0,4194304] 0 2024-04-22T23:34:52.859 INFO:tasks.workunit.client.1.smithi157.stdout:3/107: read d0/f1 [2524421,57932] 0 2024-04-22T23:34:52.883 INFO:tasks.workunit.client.1.smithi157.stdout:7/92: dwrite d0/d9/d10/f19 [0,4194304] 0 2024-04-22T23:34:52.885 INFO:tasks.workunit.client.1.smithi157.stdout:7/93: mknod d0/d9/db/c1b 0 2024-04-22T23:34:52.889 INFO:tasks.workunit.client.1.smithi157.stdout:3/108: read d0/f14 [2401277,104564] 0 2024-04-22T23:34:52.895 INFO:tasks.workunit.client.1.smithi157.stdout:3/109: mkdir d0/d3/d13/d21 0 2024-04-22T23:34:52.895 INFO:tasks.workunit.client.1.smithi157.stdout:7/94: write d0/d9/d10/f19 [3804584,106009] 0 2024-04-22T23:34:52.939 INFO:tasks.workunit.client.1.smithi157.stdout:6/323: dread d1/d17/f21 [0,4194304] 0 2024-04-22T23:34:52.941 INFO:tasks.workunit.client.1.smithi157.stdout:6/324: creat d1/d2f/d54/f6b x:0 0 0 2024-04-22T23:34:52.941 INFO:tasks.workunit.client.1.smithi157.stdout:9/161: dread d4/d15/f19 [0,4194304] 0 2024-04-22T23:34:52.954 INFO:tasks.workunit.client.1.smithi157.stdout:6/325: write d1/d38/f50 [3586318,125773] 0 2024-04-22T23:34:52.956 INFO:tasks.workunit.client.1.smithi157.stdout:6/326: truncate d1/f22 1478581 0 2024-04-22T23:34:52.956 INFO:tasks.workunit.client.1.smithi157.stdout:6/327: write d1/f7 [1455520,27187] 0 2024-04-22T23:34:52.979 INFO:tasks.workunit.client.1.smithi157.stdout:9/162: dwrite d4/fc [0,4194304] 0 2024-04-22T23:34:52.981 INFO:tasks.workunit.client.1.smithi157.stdout:9/163: symlink d4/d15/d23/l2e 0 2024-04-22T23:34:52.982 INFO:tasks.workunit.client.1.smithi157.stdout:9/164: fsync d4/df/f1f 0 2024-04-22T23:34:53.001 INFO:tasks.workunit.client.1.smithi157.stdout:4/222: dwrite d3/d4/f9 [0,4194304] 0 2024-04-22T23:34:53.011 INFO:tasks.workunit.client.1.smithi157.stdout:9/165: rmdir d4/d15/d23 39 2024-04-22T23:34:53.011 INFO:tasks.workunit.client.1.smithi157.stdout:9/166: write d4/fc [4741832,105128] 0 2024-04-22T23:34:53.011 INFO:tasks.workunit.client.1.smithi157.stdout:9/167: write d4/df/f1f [760482,46342] 0 2024-04-22T23:34:53.013 INFO:tasks.workunit.client.1.smithi157.stdout:9/168: link d4/l14 d4/d2d/l2f 0 2024-04-22T23:34:53.015 INFO:tasks.workunit.client.1.smithi157.stdout:9/169: link d4/d15/d23/l2e d4/l30 0 2024-04-22T23:34:53.015 INFO:tasks.workunit.client.1.smithi157.stdout:9/170: fsync d4/d1d/f2b 0 2024-04-22T23:34:53.015 INFO:tasks.workunit.client.1.smithi157.stdout:9/171: write d4/f8 [1502031,120245] 0 2024-04-22T23:34:53.024 INFO:tasks.workunit.client.1.smithi157.stdout:4/223: dwrite d3/d4/f16 [0,4194304] 0 2024-04-22T23:34:53.027 INFO:tasks.workunit.client.1.smithi157.stdout:4/224: truncate d3/d4/f1c 307624 0 2024-04-22T23:34:53.029 INFO:tasks.workunit.client.1.smithi157.stdout:4/225: link d3/f1b d3/d11/d1d/f41 0 2024-04-22T23:34:53.029 INFO:tasks.workunit.client.1.smithi157.stdout:4/226: read d3/d11/d1d/f24 [3254336,129208] 0 2024-04-22T23:34:53.042 INFO:tasks.workunit.client.1.smithi157.stdout:4/227: write d3/d11/f1f [1522692,35997] 0 2024-04-22T23:34:53.042 INFO:tasks.workunit.client.1.smithi157.stdout:4/228: rename d3/d11/d30 to d3/d11/d30/d28/d31/d42 22 2024-04-22T23:34:53.058 INFO:tasks.workunit.client.1.smithi157.stdout:9/172: dwrite d4/f29 [0,4194304] 0 2024-04-22T23:34:53.067 INFO:tasks.workunit.client.1.smithi157.stdout:3/110: dwrite d0/f14 [0,4194304] 0 2024-04-22T23:34:53.070 INFO:tasks.workunit.client.1.smithi157.stdout:3/111: unlink d0/d3/l4 0 2024-04-22T23:34:53.072 INFO:tasks.workunit.client.1.smithi157.stdout:3/112: mknod d0/d3/c22 0 2024-04-22T23:34:53.072 INFO:tasks.workunit.client.1.smithi157.stdout:7/95: dwrite d0/d9/fc [0,4194304] 0 2024-04-22T23:34:53.073 INFO:tasks.workunit.client.1.smithi157.stdout:3/113: write d0/f15 [1653433,10797] 0 2024-04-22T23:34:53.074 INFO:tasks.workunit.client.1.smithi157.stdout:7/96: dread d0/d9/db/f15 [0,4194304] 0 2024-04-22T23:34:53.074 INFO:tasks.workunit.client.1.smithi157.stdout:7/97: write d0/d9/d10/f19 [3146610,127496] 0 2024-04-22T23:34:53.076 INFO:tasks.workunit.client.1.smithi157.stdout:3/114: truncate d0/f11 506976 0 2024-04-22T23:34:53.077 INFO:tasks.workunit.client.1.smithi157.stdout:7/98: rename d0/c1 to d0/d9/db/df/c1c 0 2024-04-22T23:34:53.084 INFO:tasks.workunit.client.1.smithi157.stdout:7/99: dread d0/f4 [0,4194304] 0 2024-04-22T23:34:53.084 INFO:tasks.workunit.client.1.smithi157.stdout:7/100: chown d0/f4 137745 1 2024-04-22T23:34:53.084 INFO:tasks.workunit.client.1.smithi157.stdout:7/101: creat d0/d9/f1d x:0 0 0 2024-04-22T23:34:53.084 INFO:tasks.workunit.client.1.smithi157.stdout:7/102: write d0/d9/f1d [198205,48773] 0 2024-04-22T23:34:53.086 INFO:tasks.workunit.client.1.smithi157.stdout:7/103: mknod d0/d9/de/c1e 0 2024-04-22T23:34:53.086 INFO:tasks.workunit.client.1.smithi157.stdout:7/104: chown d0/d9/d10/f12 984161 1 2024-04-22T23:34:53.086 INFO:tasks.workunit.client.1.smithi157.stdout:7/105: creat d0/d9/d10/f1f x:0 0 0 2024-04-22T23:34:53.090 INFO:tasks.workunit.client.1.smithi157.stdout:7/106: rmdir d0/d9/db/df 39 2024-04-22T23:34:53.090 INFO:tasks.workunit.client.1.smithi157.stdout:7/107: fsync d0/f4 0 2024-04-22T23:34:53.092 INFO:tasks.workunit.client.1.smithi157.stdout:7/108: stat d0/d9/l1a 0 2024-04-22T23:34:53.113 INFO:tasks.workunit.client.1.smithi157.stdout:7/109: unlink d0/d9/db/df/c1c 0 2024-04-22T23:34:53.117 INFO:tasks.workunit.client.1.smithi157.stdout:3/115: dwrite d0/f1c [0,4194304] 0 2024-04-22T23:34:53.117 INFO:tasks.workunit.client.1.smithi157.stdout:6/328: dwrite d1/d36/f37 [0,4194304] 0 2024-04-22T23:34:53.119 INFO:tasks.workunit.client.1.smithi157.stdout:6/329: chown d1/d38/c4f 0 1 2024-04-22T23:34:53.121 INFO:tasks.workunit.client.1.smithi157.stdout:6/330: mkdir d1/d2f/d3b/d6c 0 2024-04-22T23:34:53.121 INFO:tasks.workunit.client.1.smithi157.stdout:6/331: write d1/d17/d5d/f64 [787122,89221] 0 2024-04-22T23:34:53.121 INFO:tasks.workunit.client.1.smithi157.stdout:6/332: stat d1/c63 0 2024-04-22T23:34:53.121 INFO:tasks.workunit.client.1.smithi157.stdout:6/333: dread - d1/d36/f47 zero size 2024-04-22T23:34:53.122 INFO:tasks.workunit.client.1.smithi157.stdout:6/334: creat d1/d2f/d3b/d6c/f6d x:0 0 0 2024-04-22T23:34:53.124 INFO:tasks.workunit.client.1.smithi157.stdout:3/116: dread d0/f1 [0,4194304] 0 2024-04-22T23:34:53.129 INFO:tasks.workunit.client.1.smithi157.stdout:4/229: dread d3/d4/f18 [0,4194304] 0 2024-04-22T23:34:53.130 INFO:tasks.workunit.client.1.smithi157.stdout:4/230: creat d3/f43 x:0 0 0 2024-04-22T23:34:53.131 INFO:tasks.workunit.client.1.smithi157.stdout:9/173: dwrite d4/df/f12 [0,4194304] 0 2024-04-22T23:34:53.132 INFO:tasks.workunit.client.1.smithi157.stdout:9/174: chown d4/df/l10 5424 1 2024-04-22T23:34:53.132 INFO:tasks.workunit.client.1.smithi157.stdout:4/231: rename d3/c3f to d3/d4/c44 0 2024-04-22T23:34:53.135 INFO:tasks.workunit.client.1.smithi157.stdout:9/175: link d4/df/l10 d4/d15/d23/d27/l31 0 2024-04-22T23:34:53.136 INFO:tasks.workunit.client.1.smithi157.stdout:4/232: rename d3/d11/d30/d28/f37 to d3/d4/f45 0 2024-04-22T23:34:53.136 INFO:tasks.workunit.client.1.smithi157.stdout:3/117: dread d0/f1c [0,4194304] 0 2024-04-22T23:34:53.138 INFO:tasks.workunit.client.1.smithi157.stdout:3/118: rmdir d0/d7/d17 39 2024-04-22T23:34:53.140 INFO:tasks.workunit.client.1.smithi157.stdout:3/119: symlink d0/d3/l23 0 2024-04-22T23:34:53.140 INFO:tasks.workunit.client.1.smithi157.stdout:3/120: chown d0/d3/f1f 1 1 2024-04-22T23:34:53.140 INFO:tasks.workunit.client.1.smithi157.stdout:3/121: fdatasync d0/f11 0 2024-04-22T23:34:53.141 INFO:tasks.workunit.client.1.smithi157.stdout:3/122: creat d0/d7/f24 x:0 0 0 2024-04-22T23:34:53.141 INFO:tasks.workunit.client.1.smithi157.stdout:3/123: write d0/d7/f24 [529759,129171] 0 2024-04-22T23:34:53.143 INFO:tasks.workunit.client.1.smithi157.stdout:3/124: creat d0/d7/d17/f25 x:0 0 0 2024-04-22T23:34:53.143 INFO:tasks.workunit.client.1.smithi157.stdout:3/125: chown d0/d3/d13/d21 22 1 2024-04-22T23:34:53.143 INFO:tasks.workunit.client.1.smithi157.stdout:3/126: readlink d0/d7/d17/l1a 0 2024-04-22T23:34:53.143 INFO:tasks.workunit.client.1.smithi157.stdout:3/127: fsync d0/d3/f6 0 2024-04-22T23:34:53.146 INFO:tasks.workunit.client.1.smithi157.stdout:3/128: symlink d0/l26 0 2024-04-22T23:34:53.146 INFO:tasks.workunit.client.1.smithi157.stdout:9/176: write d4/df/f16 [3213503,53366] 0 2024-04-22T23:34:53.146 INFO:tasks.workunit.client.1.smithi157.stdout:3/129: dread - d0/d3/f1d zero size 2024-04-22T23:34:53.146 INFO:tasks.workunit.client.1.smithi157.stdout:9/177: readlink d4/df/l1c 0 2024-04-22T23:34:53.146 INFO:tasks.workunit.client.1.smithi157.stdout:3/130: dread - d0/d7/d17/f18 zero size 2024-04-22T23:34:53.146 INFO:tasks.workunit.client.1.smithi157.stdout:9/178: fsync d4/df/fb 0 2024-04-22T23:34:53.146 INFO:tasks.workunit.client.1.smithi157.stdout:3/131: readlink d0/d3/l23 0 2024-04-22T23:34:53.147 INFO:tasks.workunit.client.1.smithi157.stdout:9/179: stat d4/df/ce 0 2024-04-22T23:34:53.147 INFO:tasks.workunit.client.1.smithi157.stdout:4/233: dread d3/d4/f9 [0,4194304] 0 2024-04-22T23:34:53.154 INFO:tasks.workunit.client.1.smithi157.stdout:3/132: read d0/f15 [401068,34420] 0 2024-04-22T23:34:53.179 INFO:tasks.workunit.client.1.smithi157.stdout:8/85: sync 2024-04-22T23:34:53.184 INFO:tasks.workunit.client.1.smithi157.stdout:6/335: getdents d1/d2f/d3b/d6c 0 2024-04-22T23:34:53.184 INFO:tasks.workunit.client.1.smithi157.stdout:6/336: write d1/d17/f31 [385555,121011] 0 2024-04-22T23:34:53.190 INFO:tasks.workunit.client.1.smithi157.stdout:6/337: dread d1/d17/f28 [0,4194304] 0 2024-04-22T23:34:53.193 INFO:tasks.workunit.client.1.smithi157.stdout:4/234: fsync d3/d4/f45 0 2024-04-22T23:34:53.196 INFO:tasks.workunit.client.1.smithi157.stdout:6/338: read d1/d2f/d3b/f2a [752025,7787] 0 2024-04-22T23:34:53.197 INFO:tasks.workunit.client.1.smithi157.stdout:6/339: rmdir d1/d2f/d3b/d6c 39 2024-04-22T23:34:53.197 INFO:tasks.workunit.client.1.smithi157.stdout:6/340: write d1/d36/f3f [1625953,71598] 0 2024-04-22T23:34:53.199 INFO:tasks.workunit.client.1.smithi157.stdout:3/133: read d0/f14 [448506,11138] 0 2024-04-22T23:34:53.199 INFO:tasks.workunit.client.1.smithi157.stdout:3/134: creat d0/f27 x:0 0 0 2024-04-22T23:34:53.199 INFO:tasks.workunit.client.1.smithi157.stdout:3/135: readlink d0/l26 0 2024-04-22T23:34:53.200 INFO:tasks.workunit.client.1.smithi157.stdout:6/341: rename d1/d38/f50 to d1/d2f/d3b/d49/d4c/f6e 0 2024-04-22T23:34:53.202 INFO:tasks.workunit.client.1.smithi157.stdout:3/136: mknod d0/c28 0 2024-04-22T23:34:53.203 INFO:tasks.workunit.client.1.smithi157.stdout:3/137: rename d0/d7 to d0/d7/d29 22 2024-04-22T23:34:53.203 INFO:tasks.workunit.client.1.smithi157.stdout:3/138: stat d0/d3/d13 0 2024-04-22T23:34:53.203 INFO:tasks.workunit.client.1.smithi157.stdout:3/139: dread - d0/f27 zero size 2024-04-22T23:34:53.204 INFO:tasks.workunit.client.1.smithi157.stdout:6/342: unlink d1/d17/f33 0 2024-04-22T23:34:53.204 INFO:tasks.workunit.client.1.smithi157.stdout:4/235: dread d3/d11/d1d/f1e [0,4194304] 0 2024-04-22T23:34:53.204 INFO:tasks.workunit.client.1.smithi157.stdout:4/236: creat d3/d4/f46 x:0 0 0 2024-04-22T23:34:53.205 INFO:tasks.workunit.client.1.smithi157.stdout:8/86: write d2/d6/de/fd [1831490,106401] 0 2024-04-22T23:34:53.208 INFO:tasks.workunit.client.1.smithi157.stdout:8/87: mknod d2/d6/de/d17/c19 0 2024-04-22T23:34:53.217 INFO:tasks.workunit.client.1.smithi157.stdout:9/180: dwrite d4/df/f16 [0,4194304] 0 2024-04-22T23:34:53.222 INFO:tasks.workunit.client.1.smithi157.stdout:6/343: write d1/f2c [607015,91441] 0 2024-04-22T23:34:53.222 INFO:tasks.workunit.client.1.smithi157.stdout:6/344: write d1/d36/f47 [369149,53222] 0 2024-04-22T23:34:53.222 INFO:tasks.workunit.client.1.smithi157.stdout:6/345: write d1/d36/f3f [2198071,5800] 0 2024-04-22T23:34:53.222 INFO:tasks.workunit.client.1.smithi157.stdout:6/346: fsync d1/d2f/d3b/f2e 0 2024-04-22T23:34:53.222 INFO:tasks.workunit.client.1.smithi157.stdout:6/347: write d1/d2f/d3b/f2a [92200,41864] 0 2024-04-22T23:34:53.224 INFO:tasks.workunit.client.1.smithi157.stdout:6/348: getdents d1 0 2024-04-22T23:34:53.237 INFO:tasks.workunit.client.1.smithi157.stdout:4/237: dwrite d3/f8 [4194304,4194304] 0 2024-04-22T23:34:53.237 INFO:tasks.workunit.client.1.smithi157.stdout:4/238: write d3/d11/d1d/f41 [390220,44290] 0 2024-04-22T23:34:53.240 INFO:tasks.workunit.client.1.smithi157.stdout:4/239: read d3/d4/f1c [135128,66830] 0 2024-04-22T23:34:53.245 INFO:tasks.workunit.client.1.smithi157.stdout:3/140: dread d0/d7/f24 [0,4194304] 0 2024-04-22T23:34:53.245 INFO:tasks.workunit.client.1.smithi157.stdout:3/141: write d0/f15 [4033511,30953] 0 2024-04-22T23:34:53.245 INFO:tasks.workunit.client.1.smithi157.stdout:3/142: dread - d0/d7/d17/f18 zero size 2024-04-22T23:34:53.248 INFO:tasks.workunit.client.1.smithi157.stdout:3/143: unlink d0/d7/f24 0 2024-04-22T23:34:53.250 INFO:tasks.workunit.client.1.smithi157.stdout:9/181: dwrite d4/f8 [0,4194304] 0 2024-04-22T23:34:53.251 INFO:tasks.workunit.client.1.smithi157.stdout:8/88: rmdir d2/d6/de 39 2024-04-22T23:34:53.270 INFO:tasks.workunit.client.1.smithi157.stdout:9/182: write d4/df/f16 [3607841,101568] 0 2024-04-22T23:34:53.270 INFO:tasks.workunit.client.1.smithi157.stdout:4/240: dwrite d3/d4/ff [0,4194304] 0 2024-04-22T23:34:53.275 INFO:tasks.workunit.client.1.smithi157.stdout:4/241: unlink d3/d4/f2f 0 2024-04-22T23:34:53.281 INFO:tasks.workunit.client.1.smithi157.stdout:7/110: dwrite d0/f7 [0,4194304] 0 2024-04-22T23:34:53.282 INFO:tasks.workunit.client.1.smithi157.stdout:7/111: link d0/d9/db/f15 d0/d9/d10/f20 0 2024-04-22T23:34:53.282 INFO:tasks.workunit.client.1.smithi157.stdout:7/112: dread - d0/d9/d10/f18 zero size 2024-04-22T23:34:53.282 INFO:tasks.workunit.client.1.smithi157.stdout:7/113: stat d0/d9/d10/f12 0 2024-04-22T23:34:53.286 INFO:tasks.workunit.client.1.smithi157.stdout:7/114: rename d0/d9/d10/f19 to d0/f21 0 2024-04-22T23:34:53.286 INFO:tasks.workunit.client.1.smithi157.stdout:7/115: dread - d0/d9/d10/f18 zero size 2024-04-22T23:34:53.299 INFO:tasks.workunit.client.1.smithi157.stdout:4/242: dwrite d3/d11/f2a [0,4194304] 0 2024-04-22T23:34:53.303 INFO:tasks.workunit.client.1.smithi157.stdout:4/243: unlink d3/f8 0 2024-04-22T23:34:53.303 INFO:tasks.workunit.client.1.smithi157.stdout:4/244: truncate d3/f43 617524 0 2024-04-22T23:34:53.306 INFO:tasks.workunit.client.1.smithi157.stdout:7/116: dwrite d0/f21 [0,4194304] 0 2024-04-22T23:34:53.309 INFO:tasks.workunit.client.1.smithi157.stdout:7/117: rename d0/d9/fc to d0/f22 0 2024-04-22T23:34:53.321 INFO:tasks.workunit.client.1.smithi157.stdout:4/245: dwrite d3/d4/f3e [0,4194304] 0 2024-04-22T23:34:53.323 INFO:tasks.workunit.client.1.smithi157.stdout:4/246: rmdir d3/d11/d30/d28 39 2024-04-22T23:34:53.333 INFO:tasks.workunit.client.1.smithi157.stdout:4/247: write d3/d4/f16 [2147714,72807] 0 2024-04-22T23:34:53.333 INFO:tasks.workunit.client.1.smithi157.stdout:4/248: fsync d3/d4/f9 0 2024-04-22T23:34:53.334 INFO:tasks.workunit.client.1.smithi157.stdout:4/249: link d3/d4/l25 d3/d11/d30/l47 0 2024-04-22T23:34:53.336 INFO:tasks.workunit.client.1.smithi157.stdout:4/250: creat d3/d11/d34/f48 x:0 0 0 2024-04-22T23:34:53.336 INFO:tasks.workunit.client.1.smithi157.stdout:4/251: creat d3/d11/d1d/f49 x:0 0 0 2024-04-22T23:34:53.356 INFO:tasks.workunit.client.1.smithi157.stdout:4/252: dwrite d3/d4/f1c [0,4194304] 0 2024-04-22T23:34:53.356 INFO:tasks.workunit.client.1.smithi157.stdout:4/253: truncate d3/d4/f46 281568 0 2024-04-22T23:34:53.358 INFO:tasks.workunit.client.1.smithi157.stdout:4/254: mknod d3/d11/d34/c4a 0 2024-04-22T23:34:53.360 INFO:tasks.workunit.client.1.smithi157.stdout:4/255: rename d3/d4 to d3/d4b 0 2024-04-22T23:34:53.360 INFO:tasks.workunit.client.1.smithi157.stdout:4/256: chown d3/d11/d30/d28/d31/c32 1688749625 1 2024-04-22T23:34:53.361 INFO:tasks.workunit.client.1.smithi157.stdout:4/257: mkdir d3/d33/d4c 0 2024-04-22T23:34:53.363 INFO:tasks.workunit.client.1.smithi157.stdout:4/258: rmdir d3/d11/d34/d39 0 2024-04-22T23:34:53.377 INFO:tasks.workunit.client.1.smithi157.stdout:4/259: dwrite d3/d11/d1d/f26 [8388608,4194304] 0 2024-04-22T23:34:53.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:34:53 smithi157 ceph-mon[33877]: pgmap v253: 65 pgs: 65 active+clean; 287 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 15 MiB/s rd, 26 MiB/s wr, 815 op/s 2024-04-22T23:34:53.385 INFO:tasks.workunit.client.1.smithi157.stdout:4/260: dread d3/d4b/f9 [0,4194304] 0 2024-04-22T23:34:53.386 INFO:tasks.workunit.client.1.smithi157.stdout:4/261: rmdir d3/d11/d30/d28 39 2024-04-22T23:34:53.386 INFO:tasks.workunit.client.1.smithi157.stdout:4/262: chown d3/d4b/f46 150 1 2024-04-22T23:34:53.386 INFO:tasks.workunit.client.1.smithi157.stdout:4/263: chown d3/d4b/f5 92 1 2024-04-22T23:34:53.391 INFO:tasks.workunit.client.1.smithi157.stdout:4/264: write d3/d4b/f5 [3537551,121512] 0 2024-04-22T23:34:53.391 INFO:tasks.workunit.client.1.smithi157.stdout:4/265: chown d3/d11/f22 1 1 2024-04-22T23:34:53.392 INFO:tasks.workunit.client.1.smithi157.stdout:4/266: creat d3/d11/d30/f4d x:0 0 0 2024-04-22T23:34:53.392 INFO:tasks.workunit.client.1.smithi157.stdout:4/267: mknod d3/d11/d30/d28/c4e 0 2024-04-22T23:34:53.392 INFO:tasks.workunit.client.1.smithi157.stdout:4/268: write d3/d11/f1f [5435635,128488] 0 2024-04-22T23:34:53.394 INFO:tasks.workunit.client.1.smithi157.stdout:4/269: creat d3/d4b/f4f x:0 0 0 2024-04-22T23:34:53.395 INFO:tasks.workunit.client.1.smithi157.stdout:4/270: rename d3/l23 to d3/d33/l50 0 2024-04-22T23:34:53.397 INFO:tasks.workunit.client.1.smithi157.stdout:4/271: rmdir d3/d4b 39 2024-04-22T23:34:53.420 INFO:tasks.workunit.client.1.smithi157.stdout:4/272: dwrite d3/d11/f22 [0,4194304] 0 2024-04-22T23:34:53.420 INFO:tasks.workunit.client.1.smithi157.stdout:4/273: creat d3/d11/d34/f51 x:0 0 0 2024-04-22T23:34:53.421 INFO:tasks.workunit.client.1.smithi157.stdout:4/274: rename d3/d4b/f9 to d3/d11/d1d/f52 0 2024-04-22T23:34:53.440 INFO:tasks.workunit.client.1.smithi157.stdout:4/275: dwrite d3/d11/f1f [0,4194304] 0 2024-04-22T23:34:53.450 INFO:tasks.workunit.client.1.smithi157.stdout:4/276: write d3/d11/d1d/f2d [1556534,61516] 0 2024-04-22T23:34:53.450 INFO:tasks.workunit.client.1.smithi157.stdout:4/277: creat d3/d11/d1d/f53 x:0 0 0 2024-04-22T23:34:53.450 INFO:tasks.workunit.client.1.smithi157.stdout:4/278: read - d3/d11/d30/f4d zero size 2024-04-22T23:34:53.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:34:53 smithi107 ceph-mon[26355]: pgmap v253: 65 pgs: 65 active+clean; 287 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 15 MiB/s rd, 26 MiB/s wr, 815 op/s 2024-04-22T23:34:53.490 INFO:tasks.workunit.client.1.smithi157.stdout:2/111: sync 2024-04-22T23:34:53.490 INFO:tasks.workunit.client.1.smithi157.stdout:2/112: mkdir d7/d1f/d26 0 2024-04-22T23:34:53.492 INFO:tasks.workunit.client.1.smithi157.stdout:2/113: unlink f5 0 2024-04-22T23:34:53.493 INFO:tasks.workunit.client.1.smithi157.stdout:2/114: mkdir d7/d12/d23/d27 0 2024-04-22T23:34:53.493 INFO:tasks.workunit.client.1.smithi157.stdout:2/115: creat d7/d8/dd/f28 x:0 0 0 2024-04-22T23:34:53.494 INFO:tasks.workunit.client.1.smithi157.stdout:2/116: symlink d7/d12/d23/l29 0 2024-04-22T23:34:53.495 INFO:tasks.workunit.client.1.smithi157.stdout:2/117: mknod d7/d12/c2a 0 2024-04-22T23:34:53.496 INFO:tasks.workunit.client.1.smithi157.stdout:2/118: unlink d7/d8/l24 0 2024-04-22T23:34:53.498 INFO:tasks.workunit.client.1.smithi157.stdout:2/119: dread f3 [0,4194304] 0 2024-04-22T23:34:53.498 INFO:tasks.workunit.client.1.smithi157.stdout:2/120: chown d7/f13 18745674 1 2024-04-22T23:34:53.499 INFO:tasks.workunit.client.1.smithi157.stdout:2/121: dread f3 [0,4194304] 0 2024-04-22T23:34:53.499 INFO:tasks.workunit.client.1.smithi157.stdout:2/122: read d7/f13 [52924,86526] 0 2024-04-22T23:34:53.500 INFO:tasks.workunit.client.1.smithi157.stdout:2/123: unlink d7/d8/c15 0 2024-04-22T23:34:53.512 INFO:tasks.workunit.client.1.smithi157.stdout:2/124: dwrite d7/d8/dd/f11 [4194304,4194304] 0 2024-04-22T23:34:53.512 INFO:tasks.workunit.client.1.smithi157.stdout:2/125: chown d7/d8/d25 73706 1 2024-04-22T23:34:53.512 INFO:tasks.workunit.client.1.smithi157.stdout:2/126: fsync d7/f13 0 2024-04-22T23:34:53.512 INFO:tasks.workunit.client.1.smithi157.stdout:2/127: creat d7/d8/dd/f2b x:0 0 0 2024-04-22T23:34:53.512 INFO:tasks.workunit.client.1.smithi157.stdout:2/128: write d7/d8/f9 [1173959,12294] 0 2024-04-22T23:34:53.515 INFO:tasks.workunit.client.1.smithi157.stdout:2/129: creat d7/d12/d23/f2c x:0 0 0 2024-04-22T23:34:53.516 INFO:tasks.workunit.client.1.smithi157.stdout:2/130: mknod d7/d12/d23/d27/c2d 0 2024-04-22T23:34:53.519 INFO:tasks.workunit.client.1.smithi157.stdout:2/131: rename f3 to d7/f2e 0 2024-04-22T23:34:53.519 INFO:tasks.workunit.client.1.smithi157.stdout:2/132: truncate d7/d8/dd/f2b 754813 0 2024-04-22T23:34:53.519 INFO:tasks.workunit.client.1.smithi157.stdout:2/133: creat d7/d8/f2f x:0 0 0 2024-04-22T23:34:53.526 INFO:tasks.workunit.client.1.smithi157.stdout:2/134: rename d7/d12 to d7/d8/d30 0 2024-04-22T23:34:53.526 INFO:tasks.workunit.client.1.smithi157.stdout:2/135: fsync d7/d8/d30/d23/f2c 0 2024-04-22T23:34:53.528 INFO:tasks.workunit.client.1.smithi157.stdout:2/136: getdents d7/d8/dd 0 2024-04-22T23:34:53.529 INFO:tasks.workunit.client.1.smithi157.stdout:2/137: creat d7/d8/dd/f31 x:0 0 0 2024-04-22T23:34:53.530 INFO:tasks.workunit.client.1.smithi157.stdout:2/138: mknod d7/d8/d30/d23/c32 0 2024-04-22T23:34:53.531 INFO:tasks.workunit.client.1.smithi157.stdout:2/139: creat d7/d8/d25/f33 x:0 0 0 2024-04-22T23:34:53.532 INFO:tasks.workunit.client.1.smithi157.stdout:2/140: rename d7/d8/f9 to d7/d8/f34 0 2024-04-22T23:34:53.533 INFO:tasks.workunit.client.1.smithi157.stdout:2/141: rename d7/f2e to d7/d8/f35 0 2024-04-22T23:34:53.533 INFO:tasks.workunit.client.1.smithi157.stdout:2/142: read - d7/d8/dd/f21 zero size 2024-04-22T23:34:53.534 INFO:tasks.workunit.client.1.smithi157.stdout:2/143: fsync d7/d8/dd/f11 0 2024-04-22T23:34:53.534 INFO:tasks.workunit.client.1.smithi157.stdout:2/144: truncate d7/d8/f2f 326248 0 2024-04-22T23:34:53.535 INFO:tasks.workunit.client.1.smithi157.stdout:2/145: creat d7/d8/d30/f36 x:0 0 0 2024-04-22T23:34:53.536 INFO:tasks.workunit.client.1.smithi157.stdout:2/146: rename d7/d8/d30/d23/d27/c2d to d7/d8/d30/d23/d27/c37 0 2024-04-22T23:34:53.536 INFO:tasks.workunit.client.1.smithi157.stdout:2/147: dread - d7/d8/d25/f33 zero size 2024-04-22T23:34:53.537 INFO:tasks.workunit.client.1.smithi157.stdout:2/148: dread d7/d8/dd/f2b [0,4194304] 0 2024-04-22T23:34:53.537 INFO:tasks.workunit.client.1.smithi157.stdout:2/149: read d7/f13 [253256,89174] 0 2024-04-22T23:34:53.537 INFO:tasks.workunit.client.1.smithi157.stdout:2/150: write d7/d8/dd/f31 [440621,124667] 0 2024-04-22T23:34:53.538 INFO:tasks.workunit.client.1.smithi157.stdout:2/151: unlink d7/d8/d30/f36 0 2024-04-22T23:34:53.539 INFO:tasks.workunit.client.1.smithi157.stdout:2/152: dread d7/f13 [0,4194304] 0 2024-04-22T23:34:53.542 INFO:tasks.workunit.client.1.smithi157.stdout:2/153: rmdir d7/d8/dd 39 2024-04-22T23:34:53.542 INFO:tasks.workunit.client.1.smithi157.stdout:2/154: chown d7/d1f/d26 54740233 1 2024-04-22T23:34:53.544 INFO:tasks.workunit.client.1.smithi157.stdout:2/155: rename d7/d8/cc to d7/d8/d25/c38 0 2024-04-22T23:34:53.544 INFO:tasks.workunit.client.1.smithi157.stdout:2/156: write d7/d8/dd/f31 [1560906,126739] 0 2024-04-22T23:34:53.544 INFO:tasks.workunit.client.1.smithi157.stdout:2/157: truncate d7/d8/d25/f33 1038375 0 2024-04-22T23:34:53.544 INFO:tasks.workunit.client.1.smithi157.stdout:2/158: dread - d7/d8/d30/d23/f2c zero size 2024-04-22T23:34:53.545 INFO:tasks.workunit.client.1.smithi157.stdout:2/159: mknod d7/d8/d30/d23/c39 0 2024-04-22T23:34:53.545 INFO:tasks.workunit.client.1.smithi157.stdout:2/160: truncate d7/d8/f2f 709717 0 2024-04-22T23:34:53.547 INFO:tasks.workunit.client.1.smithi157.stdout:2/161: unlink d7/d8/dd/f28 0 2024-04-22T23:34:53.563 INFO:tasks.workunit.client.1.smithi157.stdout:2/162: dwrite d7/d8/dd/f31 [0,4194304] 0 2024-04-22T23:34:53.563 INFO:tasks.workunit.client.1.smithi157.stdout:2/163: chown d7/d8 327147011 1 2024-04-22T23:34:53.563 INFO:tasks.workunit.client.1.smithi157.stdout:2/164: chown d7/d8/d25/c38 259025125 1 2024-04-22T23:34:53.564 INFO:tasks.workunit.client.1.smithi157.stdout:2/165: getdents d7/d8/d30/d23/d27 0 2024-04-22T23:34:53.564 INFO:tasks.workunit.client.1.smithi157.stdout:2/166: write d7/d8/d25/f33 [1178938,55407] 0 2024-04-22T23:34:53.579 INFO:tasks.workunit.client.1.smithi157.stdout:2/167: dwrite d7/d8/f34 [0,4194304] 0 2024-04-22T23:34:53.582 INFO:tasks.workunit.client.1.smithi157.stdout:2/168: rename d7/d8/dd/f11 to d7/d8/d25/f3a 0 2024-04-22T23:34:53.583 INFO:tasks.workunit.client.1.smithi157.stdout:2/169: creat d7/f3b x:0 0 0 2024-04-22T23:34:53.584 INFO:tasks.workunit.client.1.smithi157.stdout:2/170: link d7/d8/d30/d23/c32 d7/d1f/c3c 0 2024-04-22T23:34:53.584 INFO:tasks.workunit.client.1.smithi157.stdout:2/171: symlink d7/d1f/l3d 0 2024-04-22T23:34:53.585 INFO:tasks.workunit.client.1.smithi157.stdout:2/172: mkdir d7/d1f/d3e 0 2024-04-22T23:34:53.586 INFO:tasks.workunit.client.1.smithi157.stdout:2/173: write d7/d8/f35 [221539,28354] 0 2024-04-22T23:34:53.586 INFO:tasks.workunit.client.1.smithi157.stdout:2/174: truncate d7/d8/dd/f21 451839 0 2024-04-22T23:34:53.587 INFO:tasks.workunit.client.1.smithi157.stdout:2/175: write d7/d8/d25/f33 [1060454,4790] 0 2024-04-22T23:34:53.588 INFO:tasks.workunit.client.1.smithi157.stdout:2/176: link d7/d8/dd/f21 d7/d8/f3f 0 2024-04-22T23:34:53.588 INFO:tasks.workunit.client.1.smithi157.stdout:2/177: rmdir d7/d8/d30/d23/d27 39 2024-04-22T23:34:53.590 INFO:tasks.workunit.client.1.smithi157.stdout:2/178: unlink d7/d1f/l3d 0 2024-04-22T23:34:53.590 INFO:tasks.workunit.client.1.smithi157.stdout:2/179: readlink d7/d8/l1d 0 2024-04-22T23:34:53.608 INFO:tasks.workunit.client.1.smithi157.stdout:2/180: dwrite d7/d8/dd/f2b [0,4194304] 0 2024-04-22T23:34:53.608 INFO:tasks.workunit.client.1.smithi157.stdout:2/181: write d7/f13 [558840,2349] 0 2024-04-22T23:34:53.609 INFO:tasks.workunit.client.1.smithi157.stdout:2/182: mknod d7/d8/d25/c40 0 2024-04-22T23:34:53.617 INFO:tasks.workunit.client.1.smithi157.stdout:2/183: dread d7/d8/f34 [0,4194304] 0 2024-04-22T23:34:53.617 INFO:tasks.workunit.client.1.smithi157.stdout:2/184: dread - d7/d8/d30/d23/f2c zero size 2024-04-22T23:34:53.624 INFO:tasks.workunit.client.1.smithi157.stdout:2/185: dread d7/d8/d25/f3a [4194304,4194304] 0 2024-04-22T23:34:53.715 INFO:tasks.workunit.client.1.smithi157.stdout:2/186: mkdir d7/d8/d25/d41 0 2024-04-22T23:34:53.715 INFO:tasks.workunit.client.1.smithi157.stdout:2/187: chown d7/d8/d30/d23/f2c 8169702 1 2024-04-22T23:34:53.717 INFO:tasks.workunit.client.1.smithi157.stdout:2/188: link d7/d8/d30/d23/l29 d7/d8/d30/d23/l42 0 2024-04-22T23:34:53.731 INFO:tasks.workunit.client.1.smithi157.stdout:1/141: sync 2024-04-22T23:34:53.734 INFO:tasks.workunit.client.1.smithi157.stdout:2/189: dwrite d7/d8/f3f [0,4194304] 0 2024-04-22T23:34:53.737 INFO:tasks.workunit.client.1.smithi157.stdout:2/190: rename d7/d8/d25/c38 to d7/d1f/c43 0 2024-04-22T23:34:53.738 INFO:tasks.workunit.client.1.smithi157.stdout:1/142: dread d4/db/f1e [0,4194304] 0 2024-04-22T23:34:53.738 INFO:tasks.workunit.client.1.smithi157.stdout:1/143: creat d4/db/f2f x:0 0 0 2024-04-22T23:34:53.738 INFO:tasks.workunit.client.1.smithi157.stdout:1/144: creat d4/d17/f30 x:0 0 0 2024-04-22T23:34:53.738 INFO:tasks.workunit.client.1.smithi157.stdout:1/145: dread - d4/db/f2f zero size 2024-04-22T23:34:53.740 INFO:tasks.workunit.client.1.smithi157.stdout:2/191: dread d7/d8/d25/f33 [0,4194304] 0 2024-04-22T23:34:53.741 INFO:tasks.workunit.client.1.smithi157.stdout:2/192: creat d7/d1f/d26/f44 x:0 0 0 2024-04-22T23:34:53.742 INFO:tasks.workunit.client.1.smithi157.stdout:2/193: mkdir d7/d8/d30/d23/d45 0 2024-04-22T23:34:53.743 INFO:tasks.workunit.client.1.smithi157.stdout:2/194: mknod d7/d1f/d26/c46 0 2024-04-22T23:34:53.748 INFO:tasks.workunit.client.1.smithi157.stdout:2/195: rename d7/d1f/c43 to d7/d8/d30/d23/d45/c47 0 2024-04-22T23:34:53.748 INFO:tasks.workunit.client.1.smithi157.stdout:2/196: write d7/d8/dd/f21 [4277250,82457] 0 2024-04-22T23:34:53.749 INFO:tasks.workunit.client.1.smithi157.stdout:2/197: creat d7/d8/d30/d23/f48 x:0 0 0 2024-04-22T23:34:53.749 INFO:tasks.workunit.client.1.smithi157.stdout:2/198: write d7/d8/d30/d23/f2c [310219,21396] 0 2024-04-22T23:34:53.749 INFO:tasks.workunit.client.1.smithi157.stdout:2/199: write d7/d8/d25/f33 [2053866,94719] 0 2024-04-22T23:34:53.749 INFO:tasks.workunit.client.1.smithi157.stdout:2/200: dread - d7/f3b zero size 2024-04-22T23:34:53.754 INFO:tasks.workunit.client.1.smithi157.stdout:1/146: dwrite d4/db/f13 [0,4194304] 0 2024-04-22T23:34:53.754 INFO:tasks.workunit.client.1.smithi157.stdout:1/147: stat d4/d17 0 2024-04-22T23:34:53.754 INFO:tasks.workunit.client.1.smithi157.stdout:1/148: stat d4/f18 0 2024-04-22T23:34:53.765 INFO:tasks.workunit.client.1.smithi157.stdout:2/201: dread d7/d8/dd/f21 [0,4194304] 0 2024-04-22T23:34:53.773 INFO:tasks.workunit.client.1.smithi157.stdout:1/149: dwrite d4/f18 [0,4194304] 0 2024-04-22T23:34:53.776 INFO:tasks.workunit.client.1.smithi157.stdout:1/150: mknod d4/d17/d1d/c31 0 2024-04-22T23:34:53.787 INFO:tasks.workunit.client.1.smithi157.stdout:2/202: dwrite d7/d8/f35 [0,4194304] 0 2024-04-22T23:34:53.790 INFO:tasks.workunit.client.1.smithi157.stdout:2/203: creat d7/d8/d30/d23/d27/f49 x:0 0 0 2024-04-22T23:34:53.790 INFO:tasks.workunit.client.1.smithi157.stdout:2/204: write d7/d8/d30/d23/f48 [537788,28745] 0 2024-04-22T23:34:53.791 INFO:tasks.workunit.client.1.smithi157.stdout:1/151: dread d4/db/f13 [0,4194304] 0 2024-04-22T23:34:53.791 INFO:tasks.workunit.client.1.smithi157.stdout:2/205: read d7/d8/f2f [15144,54577] 0 2024-04-22T23:34:53.792 INFO:tasks.workunit.client.1.smithi157.stdout:1/152: link d4/db/lf d4/l32 0 2024-04-22T23:34:53.793 INFO:tasks.workunit.client.1.smithi157.stdout:2/206: rename d7/d8/d30/d23/d45 to d7/d1f/d26/d4a 0 2024-04-22T23:34:53.794 INFO:tasks.workunit.client.1.smithi157.stdout:2/207: write d7/d8/f3f [5104922,124893] 0 2024-04-22T23:34:53.794 INFO:tasks.workunit.client.1.smithi157.stdout:2/208: chown d7/d8/d30/c2a 1 1 2024-04-22T23:34:53.794 INFO:tasks.workunit.client.1.smithi157.stdout:1/153: mkdir d4/d17/d1d/d33 0 2024-04-22T23:34:53.794 INFO:tasks.workunit.client.1.smithi157.stdout:2/209: stat d7/d1f/c3c 0 2024-04-22T23:34:53.794 INFO:tasks.workunit.client.1.smithi157.stdout:2/210: dread - d7/d1f/d26/f44 zero size 2024-04-22T23:34:53.794 INFO:tasks.workunit.client.1.smithi157.stdout:2/211: getdents d7/d1f/d3e 0 2024-04-22T23:34:53.795 INFO:tasks.workunit.client.1.smithi157.stdout:1/154: mkdir d4/d17/d34 0 2024-04-22T23:34:53.795 INFO:tasks.workunit.client.1.smithi157.stdout:1/155: write d4/f1b [35135,40989] 0 2024-04-22T23:34:53.822 INFO:tasks.workunit.client.1.smithi157.stdout:2/212: dwrite d7/f13 [0,4194304] 0 2024-04-22T23:34:53.823 INFO:tasks.workunit.client.1.smithi157.stdout:1/156: dwrite d4/d17/f2d [0,4194304] 0 2024-04-22T23:34:53.826 INFO:tasks.workunit.client.1.smithi157.stdout:2/213: dread d7/d8/d25/f3a [4194304,4194304] 0 2024-04-22T23:34:53.833 INFO:tasks.workunit.client.1.smithi157.stdout:1/157: dread d4/f18 [0,4194304] 0 2024-04-22T23:34:53.833 INFO:tasks.workunit.client.1.smithi157.stdout:1/158: chown d4/d17/d1d/c31 1 1 2024-04-22T23:34:53.833 INFO:tasks.workunit.client.1.smithi157.stdout:1/159: write d4/f26 [365442,78281] 0 2024-04-22T23:34:53.833 INFO:tasks.workunit.client.1.smithi157.stdout:1/160: creat d4/d17/f35 x:0 0 0 2024-04-22T23:34:53.835 INFO:tasks.workunit.client.1.smithi157.stdout:1/161: truncate d4/fc 1666870 0 2024-04-22T23:34:53.835 INFO:tasks.workunit.client.1.smithi157.stdout:1/162: chown d4/db/c1a 35464207 1 2024-04-22T23:34:53.836 INFO:tasks.workunit.client.1.smithi157.stdout:2/214: write d7/d8/dd/f2b [83453,32767] 0 2024-04-22T23:34:53.836 INFO:tasks.workunit.client.1.smithi157.stdout:1/163: link d4/f16 d4/db/f36 0 2024-04-22T23:34:53.837 INFO:tasks.workunit.client.1.smithi157.stdout:2/215: creat d7/d1f/d26/d4a/f4b x:0 0 0 2024-04-22T23:34:53.839 INFO:tasks.workunit.client.1.smithi157.stdout:2/216: fsync d7/d8/dd/f2b 0 2024-04-22T23:34:53.867 INFO:tasks.workunit.client.1.smithi157.stdout:2/217: dwrite d7/d8/dd/f2b [0,4194304] 0 2024-04-22T23:34:53.868 INFO:tasks.workunit.client.1.smithi157.stdout:1/164: dwrite d4/f5 [0,4194304] 0 2024-04-22T23:34:53.868 INFO:tasks.workunit.client.1.smithi157.stdout:1/165: write d4/f26 [1029699,43972] 0 2024-04-22T23:34:53.868 INFO:tasks.workunit.client.1.smithi157.stdout:1/166: chown d4/f26 24775589 1 2024-04-22T23:34:53.868 INFO:tasks.workunit.client.1.smithi157.stdout:1/167: truncate d4/f26 1379631 0 2024-04-22T23:34:53.868 INFO:tasks.workunit.client.1.smithi157.stdout:2/218: stat d7/d8/dd/c1b 0 2024-04-22T23:34:53.869 INFO:tasks.workunit.client.1.smithi157.stdout:1/168: symlink d4/d17/l37 0 2024-04-22T23:34:53.870 INFO:tasks.workunit.client.1.smithi157.stdout:1/169: creat d4/d17/d34/f38 x:0 0 0 2024-04-22T23:34:53.871 INFO:tasks.workunit.client.1.smithi157.stdout:1/170: mkdir d4/db/d39 0 2024-04-22T23:34:53.871 INFO:tasks.workunit.client.1.smithi157.stdout:1/171: write d4/d17/f2c [332988,47587] 0 2024-04-22T23:34:53.871 INFO:tasks.workunit.client.1.smithi157.stdout:1/172: readlink d4/d17/l21 0 2024-04-22T23:34:53.871 INFO:tasks.workunit.client.1.smithi157.stdout:1/173: readlink d4/l32 0 2024-04-22T23:34:53.873 INFO:tasks.workunit.client.1.smithi157.stdout:1/174: unlink d4/db/lf 0 2024-04-22T23:34:53.874 INFO:tasks.workunit.client.1.smithi157.stdout:1/175: unlink d4/db/f2f 0 2024-04-22T23:34:53.875 INFO:tasks.workunit.client.1.smithi157.stdout:1/176: creat d4/d17/f3a x:0 0 0 2024-04-22T23:34:53.876 INFO:tasks.workunit.client.1.smithi157.stdout:2/219: dread d7/d8/dd/f21 [0,4194304] 0 2024-04-22T23:34:53.876 INFO:tasks.workunit.client.1.smithi157.stdout:2/220: fdatasync d7/d8/d25/f3a 0 2024-04-22T23:34:53.878 INFO:tasks.workunit.client.1.smithi157.stdout:1/177: truncate d4/db/f13 3097465 0 2024-04-22T23:34:53.880 INFO:tasks.workunit.client.1.smithi157.stdout:2/221: creat d7/d1f/f4c x:0 0 0 2024-04-22T23:34:53.880 INFO:tasks.workunit.client.1.smithi157.stdout:1/178: dread d4/db/f36 [0,4194304] 0 2024-04-22T23:34:53.883 INFO:tasks.workunit.client.1.smithi157.stdout:1/179: mkdir d4/d17/d34/d3b 0 2024-04-22T23:34:53.887 INFO:tasks.workunit.client.1.smithi157.stdout:2/222: mkdir d7/d8/dd/d4d 0 2024-04-22T23:34:53.887 INFO:tasks.workunit.client.1.smithi157.stdout:2/223: chown d7/d8/d30/c2a 514904 1 2024-04-22T23:34:53.890 INFO:tasks.workunit.client.1.smithi157.stdout:2/224: link d7/d1f/c3c d7/d1f/d26/d4a/c4e 0 2024-04-22T23:34:53.891 INFO:tasks.workunit.client.1.smithi157.stdout:1/180: truncate d4/db/f24 31645 0 2024-04-22T23:34:53.894 INFO:tasks.workunit.client.1.smithi157.stdout:1/181: link d4/d17/f2d d4/db/f3c 0 2024-04-22T23:34:53.896 INFO:tasks.workunit.client.1.smithi157.stdout:1/182: mkdir d4/db/d3d 0 2024-04-22T23:34:53.897 INFO:tasks.workunit.client.1.smithi157.stdout:1/183: creat d4/f3e x:0 0 0 2024-04-22T23:34:53.900 INFO:tasks.workunit.client.1.smithi157.stdout:1/184: write d4/db/f13 [2499392,126128] 0 2024-04-22T23:34:53.901 INFO:tasks.workunit.client.1.smithi157.stdout:1/185: chown d4/d17/c2a 221654559 1 2024-04-22T23:34:53.901 INFO:tasks.workunit.client.1.smithi157.stdout:1/186: truncate d4/db/f3c 1368981 0 2024-04-22T23:34:53.901 INFO:tasks.workunit.client.1.smithi157.stdout:1/187: chown d4/d17/d34/d3b 4 1 2024-04-22T23:34:53.902 INFO:tasks.workunit.client.1.smithi157.stdout:1/188: fdatasync d4/db/f14 0 2024-04-22T23:34:53.902 INFO:tasks.workunit.client.1.smithi157.stdout:1/189: getdents d4/db/d39 0 2024-04-22T23:34:53.902 INFO:tasks.workunit.client.1.smithi157.stdout:1/190: write d4/d17/f2c [466007,5859] 0 2024-04-22T23:34:53.902 INFO:tasks.workunit.client.1.smithi157.stdout:1/191: rename d4/d17/f30 to d4/d17/f3f 0 2024-04-22T23:34:53.903 INFO:tasks.workunit.client.1.smithi157.stdout:1/192: symlink d4/db/l40 0 2024-04-22T23:34:53.905 INFO:tasks.workunit.client.1.smithi157.stdout:1/193: dread d4/f16 [0,4194304] 0 2024-04-22T23:34:53.905 INFO:tasks.workunit.client.1.smithi157.stdout:1/194: read - d4/d17/f28 zero size 2024-04-22T23:34:53.905 INFO:tasks.workunit.client.1.smithi157.stdout:1/195: fsync d4/db/f14 0 2024-04-22T23:34:53.906 INFO:tasks.workunit.client.1.smithi157.stdout:1/196: dread d4/db/f24 [0,4194304] 0 2024-04-22T23:34:53.907 INFO:tasks.workunit.client.1.smithi157.stdout:1/197: mkdir d4/d17/d1d/d33/d41 0 2024-04-22T23:34:53.907 INFO:tasks.workunit.client.1.smithi157.stdout:1/198: write d4/db/f13 [3249576,36200] 0 2024-04-22T23:34:53.907 INFO:tasks.workunit.client.1.smithi157.stdout:1/199: readlink d4/db/l19 0 2024-04-22T23:34:53.908 INFO:tasks.workunit.client.1.smithi157.stdout:1/200: dread d4/f16 [0,4194304] 0 2024-04-22T23:34:53.909 INFO:tasks.workunit.client.1.smithi157.stdout:1/201: creat d4/d17/d34/d3b/f42 x:0 0 0 2024-04-22T23:34:53.911 INFO:tasks.workunit.client.1.smithi157.stdout:1/202: getdents d4/d17/d34/d3b 0 2024-04-22T23:34:53.912 INFO:tasks.workunit.client.1.smithi157.stdout:1/203: mknod d4/d17/d34/d3b/c43 0 2024-04-22T23:34:53.912 INFO:tasks.workunit.client.1.smithi157.stdout:1/204: fsync d4/db/f1e 0 2024-04-22T23:34:53.924 INFO:tasks.workunit.client.1.smithi157.stdout:1/205: dwrite d4/d17/d34/f38 [0,4194304] 0 2024-04-22T23:34:53.925 INFO:tasks.workunit.client.1.smithi157.stdout:1/206: stat d4/d17/d1d/l27 0 2024-04-22T23:34:53.925 INFO:tasks.workunit.client.1.smithi157.stdout:1/207: getdents d4/db/d3d 0 2024-04-22T23:34:53.925 INFO:tasks.workunit.client.1.smithi157.stdout:1/208: fsync d4/db/f1e 0 2024-04-22T23:34:53.940 INFO:tasks.workunit.client.1.smithi157.stdout:1/209: dwrite d4/d17/f3a [0,4194304] 0 2024-04-22T23:34:53.941 INFO:tasks.workunit.client.1.smithi157.stdout:1/210: truncate d4/f23 4469350 0 2024-04-22T23:34:53.958 INFO:tasks.workunit.client.1.smithi157.stdout:1/211: dwrite d4/d17/f2d [0,4194304] 0 2024-04-22T23:34:53.958 INFO:tasks.workunit.client.1.smithi157.stdout:1/212: write d4/db/f2e [458297,45622] 0 2024-04-22T23:34:53.958 INFO:tasks.workunit.client.1.smithi157.stdout:1/213: readlink d4/d17/l21 0 2024-04-22T23:34:53.981 INFO:tasks.workunit.client.1.smithi157.stdout:1/214: dwrite d4/f25 [4194304,4194304] 0 2024-04-22T23:34:53.982 INFO:tasks.workunit.client.1.smithi157.stdout:1/215: dread d4/db/f24 [0,4194304] 0 2024-04-22T23:34:53.984 INFO:tasks.workunit.client.1.smithi157.stdout:1/216: symlink d4/db/d3d/l44 0 2024-04-22T23:34:53.984 INFO:tasks.workunit.client.1.smithi157.stdout:1/217: stat d4/d17/d1d/d33 0 2024-04-22T23:34:53.988 INFO:tasks.workunit.client.1.smithi157.stdout:1/218: rmdir d4/db/d39 0 2024-04-22T23:34:53.991 INFO:tasks.workunit.client.1.smithi157.stdout:1/219: mkdir d4/db/d45 0 2024-04-22T23:34:53.994 INFO:tasks.workunit.client.1.smithi157.stdout:1/220: rename d4/db/l19 to d4/d17/d34/d3b/l46 0 2024-04-22T23:34:53.994 INFO:tasks.workunit.client.1.smithi157.stdout:1/221: creat d4/db/f47 x:0 0 0 2024-04-22T23:34:53.995 INFO:tasks.workunit.client.1.smithi157.stdout:1/222: write d4/db/f2e [244724,45006] 0 2024-04-22T23:34:53.997 INFO:tasks.workunit.client.1.smithi157.stdout:1/223: link d4/d17/d1d/c31 d4/d17/c48 0 2024-04-22T23:34:53.998 INFO:tasks.workunit.client.1.smithi157.stdout:1/224: creat d4/d17/d34/f49 x:0 0 0 2024-04-22T23:34:54.000 INFO:tasks.workunit.client.1.smithi157.stdout:1/225: rename d4/d17/f3f to d4/db/f4a 0 2024-04-22T23:34:54.001 INFO:tasks.workunit.client.1.smithi157.stdout:1/226: rename d4/d17/f28 to d4/d17/d34/f4b 0 2024-04-22T23:34:54.002 INFO:tasks.workunit.client.1.smithi157.stdout:1/227: mknod d4/d17/d1d/d33/d41/c4c 0 2024-04-22T23:34:54.003 INFO:tasks.workunit.client.1.smithi157.stdout:1/228: chown d4/f18 13589844 1 2024-04-22T23:34:54.003 INFO:tasks.workunit.client.1.smithi157.stdout:1/229: creat d4/d17/f4d x:0 0 0 2024-04-22T23:34:54.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.098+0000 7f24f19f2700 1 -- 172.21.15.107:0/2591949260 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24ec0fcff0 msgr2=0x7f24ec0fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:34:54.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.098+0000 7f24f19f2700 1 --2- 172.21.15.107:0/2591949260 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24ec0fcff0 0x7f24ec0fd460 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f24e0004940 tx=0x7f24e0009e30 comp rx=0 tx=0).stop 2024-04-22T23:34:54.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.098+0000 7f24f19f2700 1 -- 172.21.15.107:0/2591949260 shutdown_connections 2024-04-22T23:34:54.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.098+0000 7f24f19f2700 1 --2- 172.21.15.107:0/2591949260 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24ec0fcff0 0x7f24ec0fd460 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:54.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.098+0000 7f24f19f2700 1 --2- 172.21.15.107:0/2591949260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24ec0fbd40 0x7f24ec0fc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:54.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.098+0000 7f24f19f2700 1 -- 172.21.15.107:0/2591949260 >> 172.21.15.107:0/2591949260 conn(0x7f24ec0f74c0 msgr2=0x7f24ec0f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:34:54.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.098+0000 7f24f19f2700 1 -- 172.21.15.107:0/2591949260 shutdown_connections 2024-04-22T23:34:54.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.098+0000 7f24f19f2700 1 -- 172.21.15.107:0/2591949260 wait complete. 2024-04-22T23:34:54.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.099+0000 7f24f19f2700 1 Processor -- start 2024-04-22T23:34:54.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.099+0000 7f24f19f2700 1 -- start start 2024-04-22T23:34:54.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.099+0000 7f24f19f2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24ec0fbd40 0x7f24ec10d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:34:54.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.099+0000 7f24f19f2700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24ec0fcff0 0x7f24ec10df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:34:54.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.099+0000 7f24f19f2700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24ec10e520 con 0x7f24ec0fbd40 2024-04-22T23:34:54.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.099+0000 7f24f19f2700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24ec10ed20 con 0x7f24ec0fcff0 2024-04-22T23:34:54.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.099+0000 7f24eaffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24ec0fbd40 0x7f24ec10d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:34:54.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.099+0000 7f24eaffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24ec0fbd40 0x7f24ec10d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51150/0 (socket says 172.21.15.107:51150) 2024-04-22T23:34:54.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.099+0000 7f24eaffd700 1 -- 172.21.15.107:0/1215718209 learned_addr learned my addr 172.21.15.107:0/1215718209 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:34:54.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.099+0000 7f24ea7fc700 1 --2- 172.21.15.107:0/1215718209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24ec0fcff0 0x7f24ec10df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:34:54.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.100+0000 7f24eaffd700 1 -- 172.21.15.107:0/1215718209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24ec0fcff0 msgr2=0x7f24ec10df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:34:54.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.100+0000 7f24eaffd700 1 --2- 172.21.15.107:0/1215718209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24ec0fcff0 0x7f24ec10df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:54.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.100+0000 7f24eaffd700 1 -- 172.21.15.107:0/1215718209 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24e0005040 con 0x7f24ec0fbd40 2024-04-22T23:34:54.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.100+0000 7f24eaffd700 1 --2- 172.21.15.107:0/1215718209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24ec0fbd40 0x7f24ec10d9e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f24dc002a80 tx=0x7f24dc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:34:54.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.100+0000 7f24d37fe700 1 -- 172.21.15.107:0/1215718209 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f24dc007cc0 con 0x7f24ec0fbd40 2024-04-22T23:34:54.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.100+0000 7f24d37fe700 1 -- 172.21.15.107:0/1215718209 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f24dc00f040 con 0x7f24ec0fbd40 2024-04-22T23:34:54.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.100+0000 7f24f19f2700 1 -- 172.21.15.107:0/1215718209 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24ec10f000 con 0x7f24ec0fbd40 2024-04-22T23:34:54.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.100+0000 7f24d37fe700 1 -- 172.21.15.107:0/1215718209 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f24dc00dac0 con 0x7f24ec0fbd40 2024-04-22T23:34:54.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.100+0000 7f24f19f2700 1 -- 172.21.15.107:0/1215718209 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24ec10f540 con 0x7f24ec0fbd40 2024-04-22T23:34:54.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.101+0000 7f24f19f2700 1 -- 172.21.15.107:0/1215718209 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24ec101b80 con 0x7f24ec0fbd40 2024-04-22T23:34:54.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.102+0000 7f24d37fe700 1 -- 172.21.15.107:0/1215718209 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f24dc00dc20 con 0x7f24ec0fbd40 2024-04-22T23:34:54.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.102+0000 7f24d37fe700 1 --2- 172.21.15.107:0/1215718209 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f24d40692b0 0x7f24d406b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:34:54.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.102+0000 7f24d37fe700 1 -- 172.21.15.107:0/1215718209 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f24dc087400 con 0x7f24ec0fbd40 2024-04-22T23:34:54.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.102+0000 7f24ea7fc700 1 --2- 172.21.15.107:0/1215718209 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f24d40692b0 0x7f24d406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:34:54.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.103+0000 7f24ea7fc700 1 --2- 172.21.15.107:0/1215718209 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f24d40692b0 0x7f24d406b760 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f24e000b3b0 tx=0x7f24e0009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:34:54.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.106+0000 7f24d37fe700 1 -- 172.21.15.107:0/1215718209 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f24dc057170 con 0x7f24ec0fbd40 2024-04-22T23:34:54.159 INFO:tasks.workunit.client.1.smithi157.stdout:0/140: sync 2024-04-22T23:34:54.159 INFO:tasks.workunit.client.1.smithi157.stdout:5/194: sync 2024-04-22T23:34:54.159 INFO:tasks.workunit.client.1.smithi157.stdout:3/144: sync 2024-04-22T23:34:54.159 INFO:tasks.workunit.client.1.smithi157.stdout:6/349: sync 2024-04-22T23:34:54.159 INFO:tasks.workunit.client.1.smithi157.stdout:5/195: stat d1/d5/d7/d10/l1e 0 2024-04-22T23:34:54.159 INFO:tasks.workunit.client.1.smithi157.stdout:6/350: fsync d1/d2f/d3b/f2a 0 2024-04-22T23:34:54.159 INFO:tasks.workunit.client.1.smithi157.stdout:5/196: fdatasync d1/d5/f15 0 2024-04-22T23:34:54.159 INFO:tasks.workunit.client.1.smithi157.stdout:8/89: sync 2024-04-22T23:34:54.160 INFO:tasks.workunit.client.1.smithi157.stdout:8/90: fsync d2/f3 0 2024-04-22T23:34:54.160 INFO:tasks.workunit.client.1.smithi157.stdout:5/197: write d1/f3f [169844,22564] 0 2024-04-22T23:34:54.160 INFO:tasks.workunit.client.1.smithi157.stdout:0/141: creat d17/d23/d34/d37/f3f x:0 0 0 2024-04-22T23:34:54.161 INFO:tasks.workunit.client.1.smithi157.stdout:3/145: symlink d0/d3/d13/d21/l2a 0 2024-04-22T23:34:54.161 INFO:tasks.workunit.client.1.smithi157.stdout:5/198: dread d1/d5/dc/f38 [0,4194304] 0 2024-04-22T23:34:54.162 INFO:tasks.workunit.client.1.smithi157.stdout:6/351: mknod d1/d2f/d3b/c6f 0 2024-04-22T23:34:54.162 INFO:tasks.workunit.client.1.smithi157.stdout:6/352: readlink d1/d36/l3c 0 2024-04-22T23:34:54.163 INFO:tasks.workunit.client.1.smithi157.stdout:8/91: rename d2/d6/fb to d2/d6/de/d11/f1a 0 2024-04-22T23:34:54.163 INFO:tasks.workunit.client.1.smithi157.stdout:8/92: fsync d2/f5 0 2024-04-22T23:34:54.163 INFO:tasks.workunit.client.1.smithi157.stdout:8/93: fsync d2/d6/de/fd 0 2024-04-22T23:34:54.165 INFO:tasks.workunit.client.1.smithi157.stdout:5/199: creat d1/d5/dc/d43/f44 x:0 0 0 2024-04-22T23:34:54.170 INFO:tasks.workunit.client.1.smithi157.stdout:6/353: dread d1/f18 [0,4194304] 0 2024-04-22T23:34:54.171 INFO:tasks.workunit.client.1.smithi157.stdout:8/94: rename d2/d6/de/d11/f1a to d2/d6/de/f1b 0 2024-04-22T23:34:54.176 INFO:tasks.workunit.client.1.smithi157.stdout:5/200: rename d1/d5/d3e to d1/d5/d7/d2f/d45 0 2024-04-22T23:34:54.177 INFO:tasks.workunit.client.1.smithi157.stdout:5/201: read - d1/d1f/d22/f3c zero size 2024-04-22T23:34:54.178 INFO:tasks.workunit.client.1.smithi157.stdout:6/354: link d1/d2f/d3b/d6c/f6d d1/d38/f70 0 2024-04-22T23:34:54.178 INFO:tasks.workunit.client.1.smithi157.stdout:6/355: creat d1/d2f/d3b/d49/d4c/d56/f71 x:0 0 0 2024-04-22T23:34:54.179 INFO:tasks.workunit.client.1.smithi157.stdout:5/202: read d1/d5/dc/f38 [823209,127104] 0 2024-04-22T23:34:54.179 INFO:tasks.workunit.client.1.smithi157.stdout:5/203: truncate d1/f3f 1200918 0 2024-04-22T23:34:54.181 INFO:tasks.workunit.client.1.smithi157.stdout:6/356: rename d1/d38/c4f to d1/d2f/d3b/d39/d4e/c72 0 2024-04-22T23:34:54.181 INFO:tasks.workunit.client.1.smithi157.stdout:6/357: write d1/f22 [1062125,14033] 0 2024-04-22T23:34:54.187 INFO:tasks.workunit.client.1.smithi157.stdout:0/142: dwrite d17/f19 [0,4194304] 0 2024-04-22T23:34:54.187 INFO:tasks.workunit.client.1.smithi157.stdout:0/143: fdatasync d17/f2d 0 2024-04-22T23:34:54.188 INFO:tasks.workunit.client.1.smithi157.stdout:3/146: dwrite d0/f15 [0,4194304] 0 2024-04-22T23:34:54.188 INFO:tasks.workunit.client.1.smithi157.stdout:8/95: getdents d2/d6/de/d11 0 2024-04-22T23:34:54.191 INFO:tasks.workunit.client.1.smithi157.stdout:3/147: creat d0/d3/d13/f2b x:0 0 0 2024-04-22T23:34:54.191 INFO:tasks.workunit.client.1.smithi157.stdout:3/148: chown d0/f14 32295 1 2024-04-22T23:34:54.192 INFO:tasks.workunit.client.1.smithi157.stdout:8/96: link d2/d4/f10 d2/d4/f1c 0 2024-04-22T23:34:54.192 INFO:tasks.workunit.client.1.smithi157.stdout:8/97: chown d2/d6 459708005 1 2024-04-22T23:34:54.192 INFO:tasks.workunit.client.1.smithi157.stdout:8/98: write d2/d6/de/f12 [1224814,15024] 0 2024-04-22T23:34:54.194 INFO:tasks.workunit.client.1.smithi157.stdout:6/358: dwrite d1/d38/f70 [0,4194304] 0 2024-04-22T23:34:54.195 INFO:tasks.workunit.client.1.smithi157.stdout:8/99: rename d2/lf to d2/d6/de/l1d 0 2024-04-22T23:34:54.196 INFO:tasks.workunit.client.1.smithi157.stdout:6/359: dread d1/d17/f31 [0,4194304] 0 2024-04-22T23:34:54.199 INFO:tasks.workunit.client.1.smithi157.stdout:8/100: creat d2/d6/de/d17/f1e x:0 0 0 2024-04-22T23:34:54.201 INFO:tasks.workunit.client.1.smithi157.stdout:8/101: symlink d2/l1f 0 2024-04-22T23:34:54.202 INFO:tasks.workunit.client.1.smithi157.stdout:8/102: creat d2/d6/de/d11/f20 x:0 0 0 2024-04-22T23:34:54.203 INFO:tasks.workunit.client.1.smithi157.stdout:8/103: write d2/d4/f10 [1926259,8089] 0 2024-04-22T23:34:54.204 INFO:tasks.workunit.client.1.smithi157.stdout:8/104: mkdir d2/d6/d21 0 2024-04-22T23:34:54.204 INFO:tasks.workunit.client.1.smithi157.stdout:8/105: write d2/f3 [1031954,83065] 0 2024-04-22T23:34:54.206 INFO:tasks.workunit.client.1.smithi157.stdout:8/106: fsync d2/d4/f10 0 2024-04-22T23:34:54.208 INFO:tasks.workunit.client.1.smithi157.stdout:8/107: mknod d2/d6/de/d11/d13/c22 0 2024-04-22T23:34:54.210 INFO:tasks.workunit.client.1.smithi157.stdout:8/108: rename d2/d6/de/d11/f20 to d2/d6/d21/f23 0 2024-04-22T23:34:54.210 INFO:tasks.workunit.client.1.smithi157.stdout:8/109: readlink d2/d4/l7 0 2024-04-22T23:34:54.211 INFO:tasks.workunit.client.1.smithi157.stdout:8/110: symlink d2/d6/de/d17/l24 0 2024-04-22T23:34:54.211 INFO:tasks.workunit.client.1.smithi157.stdout:8/111: write d2/d6/de/fd [2355835,75378] 0 2024-04-22T23:34:54.211 INFO:tasks.workunit.client.1.smithi157.stdout:8/112: write d2/d6/de/fd [4471866,119072] 0 2024-04-22T23:34:54.213 INFO:tasks.workunit.client.1.smithi157.stdout:8/113: fsync d2/d6/de/fd 0 2024-04-22T23:34:54.215 INFO:tasks.workunit.client.1.smithi157.stdout:8/114: creat d2/d6/d21/f25 x:0 0 0 2024-04-22T23:34:54.215 INFO:tasks.workunit.client.1.smithi157.stdout:8/115: fdatasync d2/d6/de/f1b 0 2024-04-22T23:34:54.216 INFO:tasks.workunit.client.1.smithi157.stdout:8/116: rename d2/d6/de/f12 to d2/d6/de/d11/d13/f26 0 2024-04-22T23:34:54.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.337+0000 7f24f19f2700 1 -- 172.21.15.107:0/1215718209 --> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f24ec10f760 con 0x7f24d40692b0 2024-04-22T23:34:54.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.338+0000 7f24d37fe700 1 -- 172.21.15.107:0/1215718209 <== mgr.14206 v2:172.21.15.107:6800/2516490111 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7f24ec10f760 con 0x7f24d40692b0 2024-04-22T23:34:54.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.340+0000 7f24f19f2700 1 -- 172.21.15.107:0/1215718209 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f24d40692b0 msgr2=0x7f24d406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:34:54.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.340+0000 7f24f19f2700 1 --2- 172.21.15.107:0/1215718209 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f24d40692b0 0x7f24d406b760 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f24e000b3b0 tx=0x7f24e0009e30 comp rx=0 tx=0).stop 2024-04-22T23:34:54.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.340+0000 7f24f19f2700 1 -- 172.21.15.107:0/1215718209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24ec0fbd40 msgr2=0x7f24ec10d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:34:54.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.340+0000 7f24f19f2700 1 --2- 172.21.15.107:0/1215718209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24ec0fbd40 0x7f24ec10d9e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f24dc002a80 tx=0x7f24dc0046b0 comp rx=0 tx=0).stop 2024-04-22T23:34:54.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.340+0000 7f24f19f2700 1 -- 172.21.15.107:0/1215718209 shutdown_connections 2024-04-22T23:34:54.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.340+0000 7f24f19f2700 1 --2- 172.21.15.107:0/1215718209 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f24d40692b0 0x7f24d406b760 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:54.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.340+0000 7f24f19f2700 1 --2- 172.21.15.107:0/1215718209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24ec0fbd40 0x7f24ec10d9e0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:54.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.340+0000 7f24f19f2700 1 --2- 172.21.15.107:0/1215718209 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24ec0fcff0 0x7f24ec10df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:54.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.340+0000 7f24f19f2700 1 -- 172.21.15.107:0/1215718209 >> 172.21.15.107:0/1215718209 conn(0x7f24ec0f74c0 msgr2=0x7f24ec065b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:34:54.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.341+0000 7f24f19f2700 1 -- 172.21.15.107:0/1215718209 shutdown_connections 2024-04-22T23:34:54.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.341+0000 7f24f19f2700 1 -- 172.21.15.107:0/1215718209 wait complete. 2024-04-22T23:34:54.341 INFO:tasks.workunit.client.1.smithi157.stdout:4/279: sync 2024-04-22T23:34:54.341 INFO:tasks.workunit.client.1.smithi157.stdout:7/118: sync 2024-04-22T23:34:54.341 INFO:tasks.workunit.client.1.smithi157.stdout:9/183: sync 2024-04-22T23:34:54.341 INFO:tasks.workunit.client.1.smithi157.stdout:1/230: sync 2024-04-22T23:34:54.343 INFO:tasks.workunit.client.1.smithi157.stdout:7/119: creat d0/d9/de/f23 x:0 0 0 2024-04-22T23:34:54.343 INFO:tasks.workunit.client.1.smithi157.stdout:7/120: stat d0/f8 0 2024-04-22T23:34:54.343 INFO:tasks.workunit.client.1.smithi157.stdout:7/121: readlink d0/d9/l1a 0 2024-04-22T23:34:54.343 INFO:tasks.workunit.client.1.smithi157.stdout:1/231: mkdir d4/db/d4e 0 2024-04-22T23:34:54.344 INFO:tasks.workunit.client.1.smithi157.stdout:4/280: symlink d3/d33/d40/l54 0 2024-04-22T23:34:54.346 INFO:tasks.workunit.client.1.smithi157.stdout:7/122: symlink d0/d9/l24 0 2024-04-22T23:34:54.346 INFO:tasks.workunit.client.1.smithi157.stdout:7/123: write d0/d9/f1d [1172446,67450] 0 2024-04-22T23:34:54.346 INFO:tasks.workunit.client.1.smithi157.stdout:7/124: stat d0/f22 0 2024-04-22T23:34:54.346 INFO:tasks.workunit.client.1.smithi157.stdout:7/125: stat d0/d9/de/f23 0 2024-04-22T23:34:54.346 INFO:tasks.workunit.client.1.smithi157.stdout:7/126: write d0/f4 [1543423,66356] 0 2024-04-22T23:34:54.346 INFO:tasks.workunit.client.1.smithi157.stdout:7/127: fsync d0/d9/d10/f18 0 2024-04-22T23:34:54.346 INFO:tasks.workunit.client.1.smithi157.stdout:7/128: truncate d0/f8 5075261 0 2024-04-22T23:34:54.346 INFO:tasks.workunit.client.1.smithi157.stdout:7/129: write d0/f8 [2503181,107008] 0 2024-04-22T23:34:54.346 INFO:tasks.workunit.client.1.smithi157.stdout:1/232: rmdir d4/d17/d1d/d33 39 2024-04-22T23:34:54.346 INFO:tasks.workunit.client.1.smithi157.stdout:1/233: dread - d4/d17/f4d zero size 2024-04-22T23:34:54.347 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-04-22T23:34:54.348 INFO:tasks.workunit.client.1.smithi157.stdout:7/130: mknod d0/c25 0 2024-04-22T23:34:54.348 INFO:tasks.workunit.client.1.smithi157.stdout:7/131: read - d0/d9/de/f23 zero size 2024-04-22T23:34:54.350 INFO:tasks.workunit.client.1.smithi157.stdout:1/234: mknod d4/d17/d1d/d33/d41/c4f 0 2024-04-22T23:34:54.350 INFO:tasks.workunit.client.1.smithi157.stdout:1/235: chown d4/d17/f35 159 1 2024-04-22T23:34:54.350 INFO:tasks.workunit.client.1.smithi157.stdout:2/225: sync 2024-04-22T23:34:54.351 INFO:tasks.workunit.client.1.smithi157.stdout:2/226: creat d7/d8/d25/f4f x:0 0 0 2024-04-22T23:34:54.351 INFO:tasks.workunit.client.1.smithi157.stdout:2/227: read - d7/d1f/d26/f44 zero size 2024-04-22T23:34:54.351 INFO:tasks.workunit.client.1.smithi157.stdout:2/228: creat d7/d8/d25/f50 x:0 0 0 2024-04-22T23:34:54.354 INFO:tasks.workunit.client.1.smithi157.stdout:7/132: getdents d0/d9/de 0 2024-04-22T23:34:54.356 INFO:tasks.workunit.client.1.smithi157.stdout:1/236: mkdir d4/d17/d34/d3b/d50 0 2024-04-22T23:34:54.356 INFO:tasks.workunit.client.1.smithi157.stdout:1/237: chown d4/d17/d1d/d33 103 1 2024-04-22T23:34:54.358 INFO:tasks.workunit.client.1.smithi157.stdout:7/133: dread d0/d9/d10/f12 [0,4194304] 0 2024-04-22T23:34:54.359 INFO:tasks.workunit.client.1.smithi157.stdout:4/281: dread d3/d4b/ff [0,4194304] 0 2024-04-22T23:34:54.359 INFO:tasks.workunit.client.1.smithi157.stdout:4/282: fsync d3/d11/d1d/f53 0 2024-04-22T23:34:54.359 INFO:tasks.workunit.client.1.smithi157.stdout:4/283: creat d3/f55 x:0 0 0 2024-04-22T23:34:54.359 INFO:tasks.workunit.client.1.smithi157.stdout:1/238: mknod d4/c51 0 2024-04-22T23:34:54.362 INFO:tasks.workunit.client.1.smithi157.stdout:4/284: unlink d3/d4b/c44 0 2024-04-22T23:34:54.364 INFO:tasks.workunit.client.1.smithi157.stdout:1/239: mkdir d4/d17/d34/d3b/d52 0 2024-04-22T23:34:54.364 INFO:tasks.workunit.client.1.smithi157.stdout:1/240: chown d4/db/f36 11 1 2024-04-22T23:34:54.364 INFO:tasks.workunit.client.1.smithi157.stdout:9/184: dwrite d4/df/f1b [0,4194304] 0 2024-04-22T23:34:54.370 INFO:tasks.workunit.client.1.smithi157.stdout:9/185: mknod d4/d15/d23/c32 0 2024-04-22T23:34:54.370 INFO:tasks.workunit.client.1.smithi157.stdout:9/186: truncate d4/df/f26 390282 0 2024-04-22T23:34:54.370 INFO:tasks.workunit.client.1.smithi157.stdout:4/285: dread d3/d4b/f16 [0,4194304] 0 2024-04-22T23:34:54.371 INFO:tasks.workunit.client.1.smithi157.stdout:9/187: creat d4/d15/d23/d27/f33 x:0 0 0 2024-04-22T23:34:54.371 INFO:tasks.workunit.client.1.smithi157.stdout:9/188: truncate d4/d15/f19 1583740 0 2024-04-22T23:34:54.372 INFO:tasks.workunit.client.1.smithi157.stdout:4/286: unlink d3/d11/d30/d28/d31/l3a 0 2024-04-22T23:34:54.373 INFO:tasks.workunit.client.1.smithi157.stdout:9/189: mknod d4/d1d/c34 0 2024-04-22T23:34:54.377 INFO:tasks.workunit.client.1.smithi157.stdout:9/190: rename d4/df/c13 to d4/d2d/c35 0 2024-04-22T23:34:54.383 INFO:tasks.workunit.client.1.smithi157.stdout:7/134: dwrite d0/f8 [0,4194304] 0 2024-04-22T23:34:54.387 INFO:tasks.workunit.client.1.smithi157.stdout:7/135: unlink d0/d9/db/f13 0 2024-04-22T23:34:54.387 INFO:tasks.workunit.client.1.smithi157.stdout:7/136: creat d0/d9/db/f26 x:0 0 0 2024-04-22T23:34:54.387 INFO:tasks.workunit.client.1.smithi157.stdout:4/287: getdents d3/d11/d30/d28/d31 0 2024-04-22T23:34:54.387 INFO:tasks.workunit.client.1.smithi157.stdout:4/288: write d3/d11/d1d/f52 [4722581,49886] 0 2024-04-22T23:34:54.388 INFO:tasks.workunit.client.1.smithi157.stdout:9/191: write d4/df/f1b [928731,85689] 0 2024-04-22T23:34:54.388 INFO:tasks.workunit.client.1.smithi157.stdout:9/192: truncate d4/d15/d23/f24 896401 0 2024-04-22T23:34:54.388 INFO:tasks.workunit.client.1.smithi157.stdout:4/289: dread d3/d11/d30/d28/f3c [0,4194304] 0 2024-04-22T23:34:54.388 INFO:tasks.workunit.client.1.smithi157.stdout:4/290: getdents d3/d11/d30/d28/d31 0 2024-04-22T23:34:54.390 INFO:tasks.workunit.client.1.smithi157.stdout:7/137: mknod d0/d9/d10/c27 0 2024-04-22T23:34:54.391 INFO:tasks.workunit.client.1.smithi157.stdout:4/291: truncate d3/d4b/f45 955324 0 2024-04-22T23:34:54.392 INFO:tasks.workunit.client.1.smithi157.stdout:9/193: getdents d4/d2d 0 2024-04-22T23:34:54.393 INFO:tasks.workunit.client.1.smithi157.stdout:7/138: mknod d0/d9/db/c28 0 2024-04-22T23:34:54.395 INFO:tasks.workunit.client.1.smithi157.stdout:4/292: rename d3/d11/d30/l47 to d3/d11/l56 0 2024-04-22T23:34:54.395 INFO:tasks.workunit.client.1.smithi157.stdout:4/293: creat d3/d11/d30/f57 x:0 0 0 2024-04-22T23:34:54.395 INFO:tasks.workunit.client.1.smithi157.stdout:4/294: creat d3/f58 x:0 0 0 2024-04-22T23:34:54.395 INFO:tasks.workunit.client.1.smithi157.stdout:4/295: creat d3/d11/d34/f59 x:0 0 0 2024-04-22T23:34:54.396 INFO:tasks.workunit.client.1.smithi157.stdout:9/194: unlink d4/d15/f1e 0 2024-04-22T23:34:54.397 INFO:tasks.workunit.client.1.smithi157.stdout:7/139: mkdir d0/d9/d29 0 2024-04-22T23:34:54.401 INFO:tasks.workunit.client.1.smithi157.stdout:4/296: symlink d3/d33/d40/l5a 0 2024-04-22T23:34:54.401 INFO:tasks.workunit.client.1.smithi157.stdout:4/297: fsync d3/d11/d34/f51 0 2024-04-22T23:34:54.403 INFO:tasks.workunit.client.1.smithi157.stdout:4/298: link d3/d33/d40/l5a d3/d4b/l5b 0 2024-04-22T23:34:54.429 INFO:tasks.workunit.client.1.smithi157.stdout:9/195: dwrite d4/df/f26 [0,4194304] 0 2024-04-22T23:34:54.429 INFO:tasks.workunit.client.1.smithi157.stdout:7/140: dwrite d0/d9/db/f26 [0,4194304] 0 2024-04-22T23:34:54.429 INFO:tasks.workunit.client.1.smithi157.stdout:7/141: creat d0/d9/f2a x:0 0 0 2024-04-22T23:34:54.429 INFO:tasks.workunit.client.1.smithi157.stdout:7/142: chown d0/d9/f2a 53052 1 2024-04-22T23:34:54.429 INFO:tasks.workunit.client.1.smithi157.stdout:7/143: chown d0/f22 2 1 2024-04-22T23:34:54.429 INFO:tasks.workunit.client.1.smithi157.stdout:4/299: dwrite d3/d4b/f4f [0,4194304] 0 2024-04-22T23:34:54.429 INFO:tasks.workunit.client.1.smithi157.stdout:4/300: fdatasync d3/d11/d30/f57 0 2024-04-22T23:34:54.435 INFO:tasks.workunit.client.1.smithi157.stdout:7/144: creat d0/d9/db/df/f2b x:0 0 0 2024-04-22T23:34:54.443 INFO:tasks.workunit.client.1.smithi157.stdout:9/196: write d4/fc [686781,67340] 0 2024-04-22T23:34:54.453 INFO:tasks.workunit.client.1.smithi157.stdout:4/301: dwrite d3/f1b [0,4194304] 0 2024-04-22T23:34:54.453 INFO:tasks.workunit.client.1.smithi157.stdout:4/302: write d3/d4b/f45 [1588589,9353] 0 2024-04-22T23:34:54.463 INFO:tasks.workunit.client.1.smithi157.stdout:7/145: dwrite d0/f4 [0,4194304] 0 2024-04-22T23:34:54.464 INFO:tasks.workunit.client.1.smithi157.stdout:7/146: fsync d0/d9/f2a 0 2024-04-22T23:34:54.464 INFO:tasks.workunit.client.1.smithi157.stdout:7/147: write d0/d9/d10/f12 [421592,16129] 0 2024-04-22T23:34:54.465 INFO:tasks.workunit.client.1.smithi157.stdout:7/148: symlink d0/d9/db/df/l2c 0 2024-04-22T23:34:54.465 INFO:tasks.workunit.client.1.smithi157.stdout:7/149: chown d0/d9/db/df 384378 1 2024-04-22T23:34:54.470 INFO:tasks.workunit.client.1.smithi157.stdout:9/197: dwrite d4/df/f16 [0,4194304] 0 2024-04-22T23:34:54.471 INFO:tasks.workunit.client.1.smithi157.stdout:4/303: dwrite d3/d33/f3d [0,4194304] 0 2024-04-22T23:34:54.471 INFO:tasks.workunit.client.1.smithi157.stdout:9/198: rename d4/d2d/c35 to d4/c36 0 2024-04-22T23:34:54.472 INFO:tasks.workunit.client.1.smithi157.stdout:9/199: readlink d4/d2d/l2f 0 2024-04-22T23:34:54.473 INFO:tasks.workunit.client.1.smithi157.stdout:4/304: mkdir d3/d5c 0 2024-04-22T23:34:54.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.483+0000 7f7cd3a4b700 1 -- 172.21.15.107:0/628413147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ccc0fff80 msgr2=0x7f7ccc102360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:34:54.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.483+0000 7f7cd3a4b700 1 --2- 172.21.15.107:0/628413147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ccc0fff80 0x7f7ccc102360 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc004940 tx=0x7f7cbc009e30 comp rx=0 tx=0).stop 2024-04-22T23:34:54.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.483+0000 7f7cd3a4b700 1 -- 172.21.15.107:0/628413147 shutdown_connections 2024-04-22T23:34:54.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.483+0000 7f7cd3a4b700 1 --2- 172.21.15.107:0/628413147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ccc0fff80 0x7f7ccc102360 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:54.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.483+0000 7f7cd3a4b700 1 --2- 172.21.15.107:0/628413147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ccc066b20 0x7f7ccc0ffa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:54.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.483+0000 7f7cd3a4b700 1 -- 172.21.15.107:0/628413147 >> 172.21.15.107:0/628413147 conn(0x7f7ccc0f7450 msgr2=0x7f7ccc0f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:34:54.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.484+0000 7f7cd3a4b700 1 -- 172.21.15.107:0/628413147 shutdown_connections 2024-04-22T23:34:54.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.484+0000 7f7cd3a4b700 1 -- 172.21.15.107:0/628413147 wait complete. 2024-04-22T23:34:54.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.484+0000 7f7cd3a4b700 1 Processor -- start 2024-04-22T23:34:54.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.484+0000 7f7cd3a4b700 1 -- start start 2024-04-22T23:34:54.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.485+0000 7f7cd3a4b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ccc066b20 0x7f7ccc10da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:34:54.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.485+0000 7f7cd3a4b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ccc0fff80 0x7f7ccc10df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:34:54.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.485+0000 7f7cd3a4b700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ccc10e540 con 0x7f7ccc066b20 2024-04-22T23:34:54.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.485+0000 7f7cd3a4b700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ccc10ed40 con 0x7f7ccc0fff80 2024-04-22T23:34:54.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.485+0000 7f7cd17e7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ccc066b20 0x7f7ccc10da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:34:54.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.485+0000 7f7cd17e7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ccc066b20 0x7f7ccc10da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51164/0 (socket says 172.21.15.107:51164) 2024-04-22T23:34:54.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.485+0000 7f7cd17e7700 1 -- 172.21.15.107:0/1883123795 learned_addr learned my addr 172.21.15.107:0/1883123795 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:34:54.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.485+0000 7f7cd0fe6700 1 --2- 172.21.15.107:0/1883123795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ccc0fff80 0x7f7ccc10df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:34:54.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.485+0000 7f7cd17e7700 1 -- 172.21.15.107:0/1883123795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ccc0fff80 msgr2=0x7f7ccc10df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:34:54.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.485+0000 7f7cd17e7700 1 --2- 172.21.15.107:0/1883123795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ccc0fff80 0x7f7ccc10df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:54.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.485+0000 7f7cd17e7700 1 -- 172.21.15.107:0/1883123795 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cbc005040 con 0x7f7ccc066b20 2024-04-22T23:34:54.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.485+0000 7f7cd17e7700 1 --2- 172.21.15.107:0/1883123795 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ccc066b20 0x7f7ccc10da00 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7cc800eb70 tx=0x7f7cc800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:34:54.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.486+0000 7f7cc1ffb700 1 -- 172.21.15.107:0/1883123795 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7cc8007cc0 con 0x7f7ccc066b20 2024-04-22T23:34:54.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.486+0000 7f7cd3a4b700 1 -- 172.21.15.107:0/1883123795 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ccc10efc0 con 0x7f7ccc066b20 2024-04-22T23:34:54.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.486+0000 7f7cc1ffb700 1 -- 172.21.15.107:0/1883123795 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7cc800f040 con 0x7f7ccc066b20 2024-04-22T23:34:54.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.486+0000 7f7cc1ffb700 1 -- 172.21.15.107:0/1883123795 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7cc800cb20 con 0x7f7ccc066b20 2024-04-22T23:34:54.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.486+0000 7f7cd3a4b700 1 -- 172.21.15.107:0/1883123795 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ccc10f500 con 0x7f7ccc066b20 2024-04-22T23:34:54.483 INFO:tasks.workunit.client.1.smithi157.stdout:9/200: mknod d4/d1d/c37 0 2024-04-22T23:34:54.483 INFO:tasks.workunit.client.1.smithi157.stdout:4/305: getdents d3/d33 0 2024-04-22T23:34:54.483 INFO:tasks.workunit.client.1.smithi157.stdout:4/306: mkdir d3/d4b/d5d 0 2024-04-22T23:34:54.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.487+0000 7f7cd3a4b700 1 -- 172.21.15.107:0/1883123795 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ccc05f5f0 con 0x7f7ccc066b20 2024-04-22T23:34:54.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.487+0000 7f7cc1ffb700 1 -- 172.21.15.107:0/1883123795 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f7cc800cc80 con 0x7f7ccc066b20 2024-04-22T23:34:54.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.488+0000 7f7cc1ffb700 1 --2- 172.21.15.107:0/1883123795 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f7cb80692a0 0x7f7cb806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:34:54.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.488+0000 7f7cc1ffb700 1 -- 172.21.15.107:0/1883123795 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f7cc8086b90 con 0x7f7ccc066b20 2024-04-22T23:34:54.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.488+0000 7f7cd0fe6700 1 --2- 172.21.15.107:0/1883123795 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f7cb80692a0 0x7f7cb806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:34:54.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.489+0000 7f7cd0fe6700 1 --2- 172.21.15.107:0/1883123795 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f7cb80692a0 0x7f7cb806b750 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc009b30 tx=0x7f7cbc009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:34:54.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.493+0000 7f7cc1ffb700 1 -- 172.21.15.107:0/1883123795 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f7cc804e250 con 0x7f7ccc066b20 2024-04-22T23:34:54.508 INFO:tasks.workunit.client.1.smithi157.stdout:9/201: dwrite d4/d15/d23/f24 [0,4194304] 0 2024-04-22T23:34:54.508 INFO:tasks.workunit.client.1.smithi157.stdout:9/202: chown d4/d2d/l2f 72556587 1 2024-04-22T23:34:54.508 INFO:tasks.workunit.client.1.smithi157.stdout:9/203: creat d4/d15/d23/d27/f38 x:0 0 0 2024-04-22T23:34:54.508 INFO:tasks.workunit.client.1.smithi157.stdout:9/204: fsync d4/d1d/f2b 0 2024-04-22T23:34:54.508 INFO:tasks.workunit.client.1.smithi157.stdout:4/307: dwrite d3/d11/d1d/f52 [0,4194304] 0 2024-04-22T23:34:54.508 INFO:tasks.workunit.client.1.smithi157.stdout:9/205: creat d4/d15/d23/d27/f39 x:0 0 0 2024-04-22T23:34:54.509 INFO:tasks.workunit.client.1.smithi157.stdout:9/206: write d4/fc [4280180,16762] 0 2024-04-22T23:34:54.509 INFO:tasks.workunit.client.1.smithi157.stdout:9/207: write d4/f20 [403445,708] 0 2024-04-22T23:34:54.512 INFO:tasks.workunit.client.1.smithi157.stdout:9/208: truncate d4/d15/f19 576769 0 2024-04-22T23:34:54.514 INFO:tasks.workunit.client.1.smithi157.stdout:9/209: unlink d4/d15/d23/f24 0 2024-04-22T23:34:54.534 INFO:tasks.workunit.client.1.smithi157.stdout:9/210: dwrite d4/fc [0,4194304] 0 2024-04-22T23:34:54.535 INFO:tasks.workunit.client.1.smithi157.stdout:9/211: mkdir d4/df/d3a 0 2024-04-22T23:34:54.538 INFO:tasks.workunit.client.1.smithi157.stdout:5/204: sync 2024-04-22T23:34:54.538 INFO:tasks.workunit.client.1.smithi157.stdout:5/205: creat d1/d5/f46 x:0 0 0 2024-04-22T23:34:54.539 INFO:tasks.workunit.client.1.smithi157.stdout:5/206: read d1/d5/dc/f38 [161400,20312] 0 2024-04-22T23:34:54.539 INFO:tasks.workunit.client.1.smithi157.stdout:5/207: getdents d1/d27 0 2024-04-22T23:34:54.539 INFO:tasks.workunit.client.1.smithi157.stdout:5/208: truncate d1/d1f/d22/f35 2380371 0 2024-04-22T23:34:54.540 INFO:tasks.workunit.client.1.smithi157.stdout:5/209: symlink d1/d5/d7/d2f/l47 0 2024-04-22T23:34:54.541 INFO:tasks.workunit.client.1.smithi157.stdout:5/210: write d1/d5/d7/d10/f31 [1414236,99577] 0 2024-04-22T23:34:54.541 INFO:tasks.workunit.client.1.smithi157.stdout:5/211: creat d1/d5/d7/d2f/d45/f48 x:0 0 0 2024-04-22T23:34:54.542 INFO:tasks.workunit.client.1.smithi157.stdout:9/212: dread d4/df/f12 [4194304,4194304] 0 2024-04-22T23:34:54.543 INFO:tasks.workunit.client.1.smithi157.stdout:9/213: chown d4/l28 3948625 1 2024-04-22T23:34:54.543 INFO:tasks.workunit.client.1.smithi157.stdout:9/214: chown d4/df/f12 481995 1 2024-04-22T23:34:54.544 INFO:tasks.workunit.client.1.smithi157.stdout:9/215: rename d4/df/f1b to d4/d15/f3b 0 2024-04-22T23:34:54.545 INFO:tasks.workunit.client.1.smithi157.stdout:9/216: readlink d4/df/l17 0 2024-04-22T23:34:54.614 INFO:tasks.workunit.client.1.smithi157.stdout:5/212: dwrite d1/f26 [0,4194304] 0 2024-04-22T23:34:54.617 INFO:tasks.workunit.client.1.smithi157.stdout:5/213: link d1/f26 d1/d5/d7/d10/f49 0 2024-04-22T23:34:54.618 INFO:tasks.workunit.client.1.smithi157.stdout:9/217: dwrite d4/d1d/f22 [0,4194304] 0 2024-04-22T23:34:54.618 INFO:tasks.workunit.client.1.smithi157.stdout:9/218: read - d4/d15/d23/d27/f33 zero size 2024-04-22T23:34:54.622 INFO:tasks.workunit.client.1.smithi157.stdout:5/214: rename d1/d1f/d22/l30 to d1/d5/d7/d2f/d45/l4a 0 2024-04-22T23:34:54.622 INFO:tasks.workunit.client.1.smithi157.stdout:5/215: stat d1/d1f/d22 0 2024-04-22T23:34:54.626 INFO:tasks.workunit.client.1.smithi157.stdout:0/144: sync 2024-04-22T23:34:54.629 INFO:tasks.workunit.client.1.smithi157.stdout:0/145: getdents d17 0 2024-04-22T23:34:54.630 INFO:tasks.workunit.client.1.smithi157.stdout:0/146: write d17/d23/f31 [161261,61507] 0 2024-04-22T23:34:54.630 INFO:tasks.workunit.client.1.smithi157.stdout:0/147: rmdir d17/d38 39 2024-04-22T23:34:54.630 INFO:tasks.workunit.client.1.smithi157.stdout:9/219: dwrite d4/d15/f19 [0,4194304] 0 2024-04-22T23:34:54.633 INFO:tasks.workunit.client.1.smithi157.stdout:9/220: symlink d4/d1d/l3c 0 2024-04-22T23:34:54.634 INFO:tasks.workunit.client.1.smithi157.stdout:9/221: mkdir d4/df/d3a/d3d 0 2024-04-22T23:34:54.635 INFO:tasks.workunit.client.1.smithi157.stdout:9/222: mkdir d4/d3e 0 2024-04-22T23:34:54.638 INFO:tasks.workunit.client.1.smithi157.stdout:5/216: dwrite d1/d5/d7/f34 [0,4194304] 0 2024-04-22T23:34:54.638 INFO:tasks.workunit.client.1.smithi157.stdout:5/217: chown d1/d5/d7/d10/l13 113492675 1 2024-04-22T23:34:54.639 INFO:tasks.workunit.client.1.smithi157.stdout:9/223: rename d4/df/d3a/d3d to d4/d15/d23/d3f 0 2024-04-22T23:34:54.646 INFO:tasks.workunit.client.1.smithi157.stdout:5/218: dread d1/d5/d7/f34 [0,4194304] 0 2024-04-22T23:34:54.652 INFO:tasks.workunit.client.1.smithi157.stdout:9/224: rmdir d4/df/d3a 0 2024-04-22T23:34:54.652 INFO:tasks.workunit.client.1.smithi157.stdout:9/225: write d4/f8 [4217923,96708] 0 2024-04-22T23:34:54.653 INFO:tasks.workunit.client.1.smithi157.stdout:9/226: mknod d4/c40 0 2024-04-22T23:34:54.657 INFO:tasks.workunit.client.1.smithi157.stdout:0/148: dwrite f11 [0,4194304] 0 2024-04-22T23:34:54.665 INFO:tasks.workunit.client.1.smithi157.stdout:5/219: dwrite d1/d5/f15 [0,4194304] 0 2024-04-22T23:34:54.668 INFO:tasks.workunit.client.1.smithi157.stdout:9/227: dread d4/df/f26 [0,4194304] 0 2024-04-22T23:34:54.669 INFO:tasks.workunit.client.1.smithi157.stdout:9/228: stat d4/df/f1f 0 2024-04-22T23:34:54.669 INFO:tasks.workunit.client.1.smithi157.stdout:9/229: symlink d4/d15/d23/d3f/l41 0 2024-04-22T23:34:54.670 INFO:tasks.workunit.client.1.smithi157.stdout:0/149: dread f11 [0,4194304] 0 2024-04-22T23:34:54.670 INFO:tasks.workunit.client.1.smithi157.stdout:0/150: readlink d17/d23/l33 0 2024-04-22T23:34:54.673 INFO:tasks.workunit.client.1.smithi157.stdout:9/230: creat d4/d15/d23/d3f/f42 x:0 0 0 2024-04-22T23:34:54.674 INFO:tasks.workunit.client.1.smithi157.stdout:0/151: creat d17/d23/d34/f40 x:0 0 0 2024-04-22T23:34:54.677 INFO:tasks.workunit.client.1.smithi157.stdout:9/231: rmdir d4/d3e 0 2024-04-22T23:34:54.677 INFO:tasks.workunit.client.1.smithi157.stdout:0/152: symlink d17/d2b/l41 0 2024-04-22T23:34:54.677 INFO:tasks.workunit.client.1.smithi157.stdout:0/153: write d17/f1a [1100370,58780] 0 2024-04-22T23:34:54.683 INFO:tasks.workunit.client.1.smithi157.stdout:5/220: dwrite d1/d5/dc/f1d [0,4194304] 0 2024-04-22T23:34:54.683 INFO:tasks.workunit.client.1.smithi157.stdout:5/221: readlink d1/d27/d29/l33 0 2024-04-22T23:34:54.685 INFO:tasks.workunit.client.1.smithi157.stdout:5/222: mkdir d1/d27/d29/d4b 0 2024-04-22T23:34:54.690 INFO:tasks.workunit.client.1.smithi157.stdout:5/223: symlink d1/d5/d7/d2f/d45/l4c 0 2024-04-22T23:34:54.690 INFO:tasks.workunit.client.1.smithi157.stdout:5/224: creat d1/d27/d29/d4b/f4d x:0 0 0 2024-04-22T23:34:54.691 INFO:tasks.workunit.client.1.smithi157.stdout:5/225: creat d1/d5/f4e x:0 0 0 2024-04-22T23:34:54.691 INFO:tasks.workunit.client.1.smithi157.stdout:9/232: write d4/df/f16 [939616,47705] 0 2024-04-22T23:34:54.691 INFO:tasks.workunit.client.1.smithi157.stdout:0/154: dwrite d17/d23/f27 [0,4194304] 0 2024-04-22T23:34:54.692 INFO:tasks.workunit.client.1.smithi157.stdout:5/226: mkdir d1/d5/dc/d43/d4f 0 2024-04-22T23:34:54.693 INFO:tasks.workunit.client.1.smithi157.stdout:9/233: rename d4/d15/d23/l2e to d4/d15/l43 0 2024-04-22T23:34:54.693 INFO:tasks.workunit.client.1.smithi157.stdout:9/234: chown d4/ld 360 1 2024-04-22T23:34:54.693 INFO:tasks.workunit.client.1.smithi157.stdout:9/235: creat d4/d1d/f44 x:0 0 0 2024-04-22T23:34:54.698 INFO:tasks.workunit.client.1.smithi157.stdout:0/155: write d17/f2d [362515,74889] 0 2024-04-22T23:34:54.702 INFO:tasks.workunit.client.1.smithi157.stdout:0/156: unlink d17/d38/c3c 0 2024-04-22T23:34:54.705 INFO:tasks.workunit.client.1.smithi157.stdout:0/157: getdents d17/d23/d34 0 2024-04-22T23:34:54.705 INFO:tasks.workunit.client.1.smithi157.stdout:0/158: creat d17/d23/d34/d37/f42 x:0 0 0 2024-04-22T23:34:54.707 INFO:tasks.workunit.client.1.smithi157.stdout:0/159: symlink d17/d23/d34/l43 0 2024-04-22T23:34:54.708 INFO:tasks.workunit.client.1.smithi157.stdout:5/227: getdents d1/d5 0 2024-04-22T23:34:54.709 INFO:tasks.workunit.client.1.smithi157.stdout:0/160: creat d17/d23/d34/d37/f44 x:0 0 0 2024-04-22T23:34:54.709 INFO:tasks.workunit.client.1.smithi157.stdout:0/161: fdatasync d17/d23/f3e 0 2024-04-22T23:34:54.715 INFO:tasks.workunit.client.1.smithi157.stdout:9/236: dwrite d4/d15/f3b [4194304,4194304] 0 2024-04-22T23:34:54.715 INFO:tasks.workunit.client.1.smithi157.stdout:9/237: chown d4/d15/l21 56 1 2024-04-22T23:34:54.715 INFO:tasks.workunit.client.1.smithi157.stdout:9/238: write d4/d15/d23/d27/f33 [106595,5906] 0 2024-04-22T23:34:54.715 INFO:tasks.workunit.client.1.smithi157.stdout:9/239: write d4/d15/d23/d3f/f42 [52624,34431] 0 2024-04-22T23:34:54.715 INFO:tasks.workunit.client.1.smithi157.stdout:0/162: rename d17/d23/l3a to d17/d23/d34/d37/l45 0 2024-04-22T23:34:54.717 INFO:tasks.workunit.client.1.smithi157.stdout:5/228: getdents d1/d27/d29/d4b 0 2024-04-22T23:34:54.727 INFO:tasks.workunit.client.1.smithi157.stdout:5/229: dread d1/d5/dc/f1b [4194304,4194304] 0 2024-04-22T23:34:54.729 INFO:tasks.workunit.client.1.smithi157.stdout:5/230: symlink d1/d5/dc/d43/d4f/l50 0 2024-04-22T23:34:54.729 INFO:tasks.workunit.client.1.smithi157.stdout:5/231: chown d1/d5/d7/d10/f3d 17 1 2024-04-22T23:34:54.730 INFO:tasks.workunit.client.1.smithi157.stdout:5/232: mkdir d1/d5/d7/d2f/d51 0 2024-04-22T23:34:54.735 INFO:tasks.workunit.client.1.smithi157.stdout:0/163: dwrite d17/f1a [0,4194304] 0 2024-04-22T23:34:54.737 INFO:tasks.workunit.client.1.smithi157.stdout:5/233: dread d1/d5/f14 [4194304,4194304] 0 2024-04-22T23:34:54.738 INFO:tasks.workunit.client.1.smithi157.stdout:9/240: dwrite d4/d1d/f44 [0,4194304] 0 2024-04-22T23:34:54.739 INFO:tasks.workunit.client.1.smithi157.stdout:0/164: dread fc [4194304,4194304] 0 2024-04-22T23:34:54.742 INFO:tasks.workunit.client.1.smithi157.stdout:5/234: rmdir d1/d27 39 2024-04-22T23:34:54.742 INFO:tasks.workunit.client.1.smithi157.stdout:5/235: dread - d1/d1f/d22/f3c zero size 2024-04-22T23:34:54.742 INFO:tasks.workunit.client.1.smithi157.stdout:5/236: chown d1/d1f/c2d 2058900111 1 2024-04-22T23:34:54.745 INFO:tasks.workunit.client.1.smithi157.stdout:9/241: creat d4/d15/d23/d3f/f45 x:0 0 0 2024-04-22T23:34:54.747 INFO:tasks.workunit.client.1.smithi157.stdout:5/237: symlink d1/d5/d7/d10/l52 0 2024-04-22T23:34:54.747 INFO:tasks.workunit.client.1.smithi157.stdout:0/165: dread d17/f1b [0,4194304] 0 2024-04-22T23:34:54.748 INFO:tasks.workunit.client.1.smithi157.stdout:9/242: rename d4/d15/l21 to d4/d15/l46 0 2024-04-22T23:34:54.748 INFO:tasks.workunit.client.1.smithi157.stdout:3/149: sync 2024-04-22T23:34:54.749 INFO:tasks.workunit.client.1.smithi157.stdout:5/238: creat d1/d5/d7/d2f/d45/f53 x:0 0 0 2024-04-22T23:34:54.749 INFO:tasks.workunit.client.1.smithi157.stdout:7/150: sync 2024-04-22T23:34:54.749 INFO:tasks.workunit.client.1.smithi157.stdout:6/360: sync 2024-04-22T23:34:54.749 INFO:tasks.workunit.client.1.smithi157.stdout:8/117: sync 2024-04-22T23:34:54.749 INFO:tasks.workunit.client.1.smithi157.stdout:4/308: sync 2024-04-22T23:34:54.749 INFO:tasks.workunit.client.1.smithi157.stdout:1/241: sync 2024-04-22T23:34:54.749 INFO:tasks.workunit.client.1.smithi157.stdout:2/229: sync 2024-04-22T23:34:54.749 INFO:tasks.workunit.client.1.smithi157.stdout:4/309: chown d3/d11/d30/d36 7 1 2024-04-22T23:34:54.749 INFO:tasks.workunit.client.1.smithi157.stdout:2/230: fsync d7/d8/f35 0 2024-04-22T23:34:54.750 INFO:tasks.workunit.client.1.smithi157.stdout:0/166: symlink d17/l46 0 2024-04-22T23:34:54.750 INFO:tasks.workunit.client.1.smithi157.stdout:9/243: mkdir d4/d15/d23/d27/d47 0 2024-04-22T23:34:54.752 INFO:tasks.workunit.client.1.smithi157.stdout:3/150: truncate d0/d3/f1e 1349145 0 2024-04-22T23:34:54.752 INFO:tasks.workunit.client.1.smithi157.stdout:3/151: truncate d0/f11 1238103 0 2024-04-22T23:34:54.752 INFO:tasks.workunit.client.1.smithi157.stdout:3/152: chown d0/f1c 10 1 2024-04-22T23:34:54.753 INFO:tasks.workunit.client.1.smithi157.stdout:8/118: dread d2/d4/f1c [0,4194304] 0 2024-04-22T23:34:54.753 INFO:tasks.workunit.client.1.smithi157.stdout:8/119: creat d2/d6/de/d11/f27 x:0 0 0 2024-04-22T23:34:54.753 INFO:tasks.workunit.client.1.smithi157.stdout:5/239: mkdir d1/d5/d7/d2f/d45/d54 0 2024-04-22T23:34:54.753 INFO:tasks.workunit.client.1.smithi157.stdout:1/242: chown d4/l32 55288334 1 2024-04-22T23:34:54.754 INFO:tasks.workunit.client.1.smithi157.stdout:6/361: truncate d1/d36/f37 1613567 0 2024-04-22T23:34:54.754 INFO:tasks.workunit.client.1.smithi157.stdout:6/362: stat d1/d2f/f4a 0 2024-04-22T23:34:54.755 INFO:tasks.workunit.client.1.smithi157.stdout:6/363: creat d1/d2f/d3b/d49/d4c/d56/f73 x:0 0 0 2024-04-22T23:34:54.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.759+0000 7f7cd3a4b700 1 -- 172.21.15.107:0/1883123795 --> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7ccc10f830 con 0x7f7cb80692a0 2024-04-22T23:34:54.755 INFO:tasks.workunit.client.1.smithi157.stdout:7/151: symlink d0/d9/d29/l2d 0 2024-04-22T23:34:54.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.761+0000 7f7cc1ffb700 1 -- 172.21.15.107:0/1883123795 <== mgr.14206 v2:172.21.15.107:6800/2516490111 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7f7ccc10f830 con 0x7f7cb80692a0 2024-04-22T23:34:54.756 INFO:tasks.workunit.client.1.smithi157.stdout:4/310: symlink d3/d11/d30/d28/l5e 0 2024-04-22T23:34:54.758 INFO:tasks.workunit.client.1.smithi157.stdout:2/231: symlink d7/d8/l51 0 2024-04-22T23:34:54.758 INFO:tasks.workunit.client.1.smithi157.stdout:2/232: creat d7/d1f/f52 x:0 0 0 2024-04-22T23:34:54.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.763+0000 7f7cd3a4b700 1 -- 172.21.15.107:0/1883123795 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f7cb80692a0 msgr2=0x7f7cb806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:34:54.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.763+0000 7f7cd3a4b700 1 --2- 172.21.15.107:0/1883123795 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f7cb80692a0 0x7f7cb806b750 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc009b30 tx=0x7f7cbc009e30 comp rx=0 tx=0).stop 2024-04-22T23:34:54.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.763+0000 7f7cd3a4b700 1 -- 172.21.15.107:0/1883123795 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ccc066b20 msgr2=0x7f7ccc10da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:34:54.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.763+0000 7f7cd3a4b700 1 --2- 172.21.15.107:0/1883123795 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ccc066b20 0x7f7ccc10da00 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7cc800eb70 tx=0x7f7cc800b040 comp rx=0 tx=0).stop 2024-04-22T23:34:54.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.763+0000 7f7cd3a4b700 1 -- 172.21.15.107:0/1883123795 shutdown_connections 2024-04-22T23:34:54.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.763+0000 7f7cd3a4b700 1 --2- 172.21.15.107:0/1883123795 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f7cb80692a0 0x7f7cb806b750 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:54.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.763+0000 7f7cd3a4b700 1 --2- 172.21.15.107:0/1883123795 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ccc066b20 0x7f7ccc10da00 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:54.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.763+0000 7f7cd3a4b700 1 --2- 172.21.15.107:0/1883123795 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ccc0fff80 0x7f7ccc10df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:54.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.763+0000 7f7cd3a4b700 1 -- 172.21.15.107:0/1883123795 >> 172.21.15.107:0/1883123795 conn(0x7f7ccc0f7450 msgr2=0x7f7ccc0f7870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:34:54.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.764+0000 7f7cd3a4b700 1 -- 172.21.15.107:0/1883123795 shutdown_connections 2024-04-22T23:34:54.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.764+0000 7f7cd3a4b700 1 -- 172.21.15.107:0/1883123795 wait complete. 2024-04-22T23:34:54.760 INFO:tasks.workunit.client.1.smithi157.stdout:0/167: creat d17/d2b/f47 x:0 0 0 2024-04-22T23:34:54.760 INFO:tasks.workunit.client.1.smithi157.stdout:0/168: fdatasync d17/d2b/f2c 0 2024-04-22T23:34:54.760 INFO:tasks.workunit.client.1.smithi157.stdout:9/244: unlink d4/d15/d23/d27/l31 0 2024-04-22T23:34:54.763 INFO:tasks.workunit.client.1.smithi157.stdout:3/153: symlink d0/d3/l2c 0 2024-04-22T23:34:54.766 INFO:tasks.workunit.client.1.smithi157.stdout:8/120: mknod d2/d6/d21/c28 0 2024-04-22T23:34:54.768 INFO:tasks.workunit.client.1.smithi157.stdout:5/240: creat d1/d27/d29/d4b/f55 x:0 0 0 2024-04-22T23:34:54.768 INFO:tasks.workunit.client.1.smithi157.stdout:5/241: creat d1/d5/d7/f56 x:0 0 0 2024-04-22T23:34:54.768 INFO:tasks.workunit.client.1.smithi157.stdout:5/242: getdents d1/d27 0 2024-04-22T23:34:54.769 INFO:tasks.workunit.client.1.smithi157.stdout:5/243: fsync d1/f3f 0 2024-04-22T23:34:54.769 INFO:tasks.workunit.client.1.smithi157.stdout:1/243: symlink d4/d17/d34/l53 0 2024-04-22T23:34:54.771 INFO:tasks.workunit.client.1.smithi157.stdout:4/311: mknod d3/d11/d30/c5f 0 2024-04-22T23:34:54.773 INFO:tasks.workunit.client.1.smithi157.stdout:8/121: symlink d2/d6/d21/l29 0 2024-04-22T23:34:54.776 INFO:tasks.workunit.client.1.smithi157.stdout:5/244: symlink d1/d1f/d22/l57 0 2024-04-22T23:34:54.776 INFO:tasks.workunit.client.1.smithi157.stdout:5/245: chown d1/d5/dc/f1b 919654 1 2024-04-22T23:34:54.776 INFO:tasks.workunit.client.1.smithi157.stdout:5/246: fdatasync d1/d1f/d22/f3c 0 2024-04-22T23:34:54.776 INFO:tasks.workunit.client.1.smithi157.stdout:6/364: rmdir d1/d36 39 2024-04-22T23:34:54.777 INFO:tasks.workunit.client.1.smithi157.stdout:2/233: dwrite d7/f3b [0,4194304] 0 2024-04-22T23:34:54.779 INFO:tasks.workunit.client.1.smithi157.stdout:4/312: link d3/d11/d30/d28/d31/c38 d3/d4b/d5d/c60 0 2024-04-22T23:34:54.780 INFO:tasks.workunit.client.1.smithi157.stdout:0/169: getdents d17/d38 0 2024-04-22T23:34:54.780 INFO:tasks.workunit.client.1.smithi157.stdout:0/170: stat l12 0 2024-04-22T23:34:54.780 INFO:tasks.workunit.client.1.smithi157.stdout:0/171: chown d17/d23/f3e 3 1 2024-04-22T23:34:54.780 INFO:tasks.workunit.client.1.smithi157.stdout:0/172: read - d17/d23/f32 zero size 2024-04-22T23:34:54.780 INFO:tasks.workunit.client.1.smithi157.stdout:7/152: dwrite d0/d9/db/f15 [0,4194304] 0 2024-04-22T23:34:54.780 INFO:tasks.workunit.client.1.smithi157.stdout:7/153: write d0/d9/d10/f18 [18916,56309] 0 2024-04-22T23:34:54.781 INFO:tasks.workunit.client.1.smithi157.stdout:8/122: truncate d2/f3 865617 0 2024-04-22T23:34:54.781 INFO:tasks.workunit.client.1.smithi157.stdout:1/244: write d4/fc [1083832,82520] 0 2024-04-22T23:34:54.781 INFO:tasks.workunit.client.1.smithi157.stdout:8/123: dread - d2/d6/d21/f23 zero size 2024-04-22T23:34:54.783 INFO:tasks.workunit.client.1.smithi157.stdout:5/247: creat d1/d27/d29/f58 x:0 0 0 2024-04-22T23:34:54.783 INFO:tasks.workunit.client.1.smithi157.stdout:5/248: chown d1/d5/dc/d43/f44 24108269 1 2024-04-22T23:34:54.786 INFO:tasks.workunit.client.1.smithi157.stdout:6/365: unlink d1/d17/c58 0 2024-04-22T23:34:54.788 INFO:tasks.workunit.client.1.smithi157.stdout:6/366: dread d1/d17/f31 [0,4194304] 0 2024-04-22T23:34:54.788 INFO:tasks.workunit.client.1.smithi157.stdout:6/367: chown d1/d2f/d3b/d49/d4c/d56/f73 2 1 2024-04-22T23:34:54.793 INFO:tasks.workunit.client.1.smithi157.stdout:3/154: dwrite d0/d3/f1d [0,4194304] 0 2024-04-22T23:34:54.795 INFO:tasks.workunit.client.1.smithi157.stdout:8/124: mknod d2/d4/c2a 0 2024-04-22T23:34:54.797 INFO:tasks.workunit.client.1.smithi157.stdout:8/125: read d2/d4/f10 [1653699,52460] 0 2024-04-22T23:34:54.797 INFO:tasks.workunit.client.1.smithi157.stdout:8/126: write d2/d6/de/d17/f1e [774400,14643] 0 2024-04-22T23:34:54.797 INFO:tasks.workunit.client.1.smithi157.stdout:1/245: mkdir d4/d17/d1d/d33/d41/d54 0 2024-04-22T23:34:54.799 INFO:tasks.workunit.client.1.smithi157.stdout:5/249: truncate d1/d5/d7/d10/f49 2996781 0 2024-04-22T23:34:54.799 INFO:tasks.workunit.client.1.smithi157.stdout:2/234: dwrite d7/d8/f2f [0,4194304] 0 2024-04-22T23:34:54.801 INFO:tasks.workunit.client.1.smithi157.stdout:6/368: mkdir d1/d2f/d3b/d39/d4e/d74 0 2024-04-22T23:34:54.801 INFO:tasks.workunit.client.1.smithi157.stdout:3/155: dread d0/f1c [0,4194304] 0 2024-04-22T23:34:54.802 INFO:tasks.workunit.client.1.smithi157.stdout:4/313: dwrite d3/d33/f3d [0,4194304] 0 2024-04-22T23:34:54.802 INFO:tasks.workunit.client.1.smithi157.stdout:4/314: chown d3/d11/d30/d36/c3b 163108645 1 2024-04-22T23:34:54.803 INFO:tasks.workunit.client.1.smithi157.stdout:7/154: dwrite d0/d9/db/df/f2b [0,4194304] 0 2024-04-22T23:34:54.803 INFO:tasks.workunit.client.1.smithi157.stdout:7/155: fsync d0/d9/d10/f18 0 2024-04-22T23:34:54.803 INFO:tasks.workunit.client.1.smithi157.stdout:0/173: rmdir d17/d23 39 2024-04-22T23:34:54.808 INFO:tasks.workunit.client.1.smithi157.stdout:1/246: rmdir d4/db/d4e 0 2024-04-22T23:34:54.811 INFO:tasks.workunit.client.1.smithi157.stdout:2/235: truncate d7/d8/d25/f33 685296 0 2024-04-22T23:34:54.811 INFO:tasks.workunit.client.1.smithi157.stdout:2/236: creat d7/d1f/d26/d4a/f53 x:0 0 0 2024-04-22T23:34:54.812 INFO:tasks.workunit.client.1.smithi157.stdout:6/369: mknod d1/c75 0 2024-04-22T23:34:54.812 INFO:tasks.workunit.client.1.smithi157.stdout:6/370: read - d1/d2f/d54/f6b zero size 2024-04-22T23:34:54.813 INFO:tasks.workunit.client.1.smithi157.stdout:3/156: symlink d0/d7/d17/l2d 0 2024-04-22T23:34:54.819 INFO:tasks.workunit.client.1.smithi157.stdout:7/156: link d0/ld d0/d9/d10/l2e 0 2024-04-22T23:34:54.819 INFO:tasks.workunit.client.1.smithi157.stdout:4/315: dwrite d3/d11/d1d/f53 [0,4194304] 0 2024-04-22T23:34:54.819 INFO:tasks.workunit.client.1.smithi157.stdout:7/157: read - d0/d9/de/f23 zero size 2024-04-22T23:34:54.820 INFO:tasks.workunit.client.1.smithi157.stdout:0/174: symlink d17/d38/l48 0 2024-04-22T23:34:54.820 INFO:tasks.workunit.client.1.smithi157.stdout:0/175: chown cf 221467705 1 2024-04-22T23:34:54.821 INFO:tasks.workunit.client.1.smithi157.stdout:8/127: truncate d2/f3 65109 0 2024-04-22T23:34:54.824 INFO:tasks.workunit.client.1.smithi157.stdout:1/247: unlink d4/db/l2b 0 2024-04-22T23:34:54.825 INFO:tasks.workunit.client.1.smithi157.stdout:1/248: write d4/db/fd [726854,59357] 0 2024-04-22T23:34:54.825 INFO:tasks.workunit.client.1.smithi157.stdout:1/249: chown d4/db/l40 0 1 2024-04-22T23:34:54.830 INFO:tasks.workunit.client.1.smithi157.stdout:3/157: symlink d0/d3/d13/l2e 0 2024-04-22T23:34:54.831 INFO:tasks.workunit.client.1.smithi157.stdout:0/176: dread d17/f19 [0,4194304] 0 2024-04-22T23:34:54.837 INFO:tasks.workunit.client.1.smithi157.stdout:7/158: symlink d0/l2f 0 2024-04-22T23:34:54.838 INFO:tasks.workunit.client.1.smithi157.stdout:0/177: dread d17/d23/f28 [0,4194304] 0 2024-04-22T23:34:54.842 INFO:tasks.workunit.client.1.smithi157.stdout:6/371: rmdir d1/d2f/d3b/d39 39 2024-04-22T23:34:54.847 INFO:tasks.workunit.client.1.smithi157.stdout:7/159: link d0/d9/f1d d0/d9/d10/f30 0 2024-04-22T23:34:54.848 INFO:tasks.workunit.client.1.smithi157.stdout:4/316: dwrite d3/d11/d1d/f41 [4194304,4194304] 0 2024-04-22T23:34:54.849 INFO:tasks.workunit.client.1.smithi157.stdout:0/178: rename d17/d23/l24 to d17/d2b/l49 0 2024-04-22T23:34:54.849 INFO:tasks.workunit.client.1.smithi157.stdout:0/179: chown d17/c36 5551 1 2024-04-22T23:34:54.851 INFO:tasks.workunit.client.1.smithi157.stdout:8/128: dwrite d2/d4/f1c [0,4194304] 0 2024-04-22T23:34:54.851 INFO:tasks.workunit.client.1.smithi157.stdout:7/160: dread d0/d9/d10/f30 [0,4194304] 0 2024-04-22T23:34:54.851 INFO:tasks.workunit.client.1.smithi157.stdout:3/158: dwrite d0/d3/f1e [0,4194304] 0 2024-04-22T23:34:54.852 INFO:tasks.workunit.client.1.smithi157.stdout:3/159: creat d0/f2f x:0 0 0 2024-04-22T23:34:54.852 INFO:tasks.workunit.client.1.smithi157.stdout:6/372: unlink d1/d2f/d3b/l62 0 2024-04-22T23:34:54.853 INFO:tasks.workunit.client.1.smithi157.stdout:2/237: rmdir d7/d1f/d26/d4a 39 2024-04-22T23:34:54.853 INFO:tasks.workunit.client.1.smithi157.stdout:2/238: write d7/d8/d25/f50 [166403,49096] 0 2024-04-22T23:34:54.856 INFO:tasks.workunit.client.1.smithi157.stdout:4/317: rename d3/d11/d34/f59 to d3/d33/d4c/f61 0 2024-04-22T23:34:54.856 INFO:tasks.workunit.client.1.smithi157.stdout:4/318: stat d3/d11/d30/d28/d31 0 2024-04-22T23:34:54.857 INFO:tasks.workunit.client.1.smithi157.stdout:0/180: symlink d17/d38/l4a 0 2024-04-22T23:34:54.859 INFO:tasks.workunit.client.1.smithi157.stdout:7/161: dread d0/d9/db/df/f2b [0,4194304] 0 2024-04-22T23:34:54.859 INFO:tasks.workunit.client.1.smithi157.stdout:7/162: fdatasync d0/d9/f2a 0 2024-04-22T23:34:54.861 INFO:tasks.workunit.client.1.smithi157.stdout:8/129: symlink d2/d6/l2b 0 2024-04-22T23:34:54.864 INFO:tasks.workunit.client.1.smithi157.stdout:6/373: mkdir d1/d17/d5d/d76 0 2024-04-22T23:34:54.864 INFO:tasks.workunit.client.1.smithi157.stdout:6/374: write d1/f5 [4974333,64929] 0 2024-04-22T23:34:54.865 INFO:tasks.workunit.client.1.smithi157.stdout:2/239: mkdir d7/d1f/d26/d4a/d54 0 2024-04-22T23:34:54.865 INFO:tasks.workunit.client.1.smithi157.stdout:2/240: fdatasync d7/d8/d30/d23/f48 0 2024-04-22T23:34:54.868 INFO:tasks.workunit.client.1.smithi157.stdout:3/160: rename d0/d3/f1b to d0/d3/d13/d21/f30 0 2024-04-22T23:34:54.870 INFO:tasks.workunit.client.1.smithi157.stdout:7/163: truncate d0/d9/d10/f20 2789764 0 2024-04-22T23:34:54.872 INFO:tasks.workunit.client.1.smithi157.stdout:8/130: mkdir d2/d4/d2c 0 2024-04-22T23:34:54.872 INFO:tasks.workunit.client.1.smithi157.stdout:8/131: creat d2/d6/d21/f2d x:0 0 0 2024-04-22T23:34:54.873 INFO:tasks.workunit.client.1.smithi157.stdout:4/319: dwrite d3/d4b/f45 [0,4194304] 0 2024-04-22T23:34:54.875 INFO:tasks.workunit.client.1.smithi157.stdout:6/375: symlink d1/d2f/d3b/d39/l77 0 2024-04-22T23:34:54.876 INFO:tasks.workunit.client.1.smithi157.stdout:7/164: creat d0/d9/d29/f31 x:0 0 0 2024-04-22T23:34:54.880 INFO:tasks.workunit.client.1.smithi157.stdout:0/181: dwrite d17/d23/f32 [0,4194304] 0 2024-04-22T23:34:54.883 INFO:tasks.workunit.client.1.smithi157.stdout:4/320: rename d3/d33 to d3/d11/d30/d36/d62 0 2024-04-22T23:34:54.883 INFO:tasks.workunit.client.1.smithi157.stdout:7/165: rmdir d0 39 2024-04-22T23:34:54.884 INFO:tasks.workunit.client.1.smithi157.stdout:7/166: read - d0/d9/d29/f31 zero size 2024-04-22T23:34:54.884 INFO:tasks.workunit.client.1.smithi157.stdout:7/167: getdents d0/d9/de 0 2024-04-22T23:34:54.885 INFO:tasks.workunit.client.1.smithi157.stdout:8/132: rmdir d2/d6/de/d11 39 2024-04-22T23:34:54.885 INFO:tasks.workunit.client.1.smithi157.stdout:8/133: creat d2/d6/d21/f2e x:0 0 0 2024-04-22T23:34:54.885 INFO:tasks.workunit.client.1.smithi157.stdout:8/134: write d2/d6/d21/f25 [903407,14871] 0 2024-04-22T23:34:54.886 INFO:tasks.workunit.client.1.smithi157.stdout:7/168: write d0/d9/f1d [559687,113569] 0 2024-04-22T23:34:54.888 INFO:tasks.workunit.client.1.smithi157.stdout:0/182: rmdir d17/d38 39 2024-04-22T23:34:54.889 INFO:tasks.workunit.client.1.smithi157.stdout:4/321: rename d3/d11/d30/d28/l5e to d3/d11/l63 0 2024-04-22T23:34:54.890 INFO:tasks.workunit.client.1.smithi157.stdout:4/322: write d3/f58 [331273,111227] 0 2024-04-22T23:34:54.890 INFO:tasks.workunit.client.1.smithi157.stdout:4/323: chown d3/f55 2744091 1 2024-04-22T23:34:54.891 INFO:tasks.workunit.client.1.smithi157.stdout:7/169: mknod d0/d9/db/df/c32 0 2024-04-22T23:34:54.894 INFO:tasks.workunit.client.1.smithi157.stdout:0/183: unlink l16 0 2024-04-22T23:34:54.896 INFO:tasks.workunit.client.1.smithi157.stdout:4/324: rename d3/d11/d30/d28/d31/c38 to d3/c64 0 2024-04-22T23:34:54.897 INFO:tasks.workunit.client.1.smithi157.stdout:4/325: chown d3/d4b/ca 1376005 1 2024-04-22T23:34:54.898 INFO:tasks.workunit.client.1.smithi157.stdout:0/184: link d17/d2b/l49 d17/d2b/l4b 0 2024-04-22T23:34:54.913 INFO:tasks.workunit.client.1.smithi157.stdout:3/161: dwrite d0/d3/f1e [0,4194304] 0 2024-04-22T23:34:54.915 INFO:tasks.workunit.client.1.smithi157.stdout:3/162: symlink d0/d3/l31 0 2024-04-22T23:34:54.917 INFO:tasks.workunit.client.1.smithi157.stdout:2/241: dwrite d7/d8/dd/f2b [0,4194304] 0 2024-04-22T23:34:54.917 INFO:tasks.workunit.client.1.smithi157.stdout:6/376: dwrite d1/d17/f27 [0,4194304] 0 2024-04-22T23:34:54.920 INFO:tasks.workunit.client.1.smithi157.stdout:7/170: dwrite d0/d9/d10/f12 [0,4194304] 0 2024-04-22T23:34:54.920 INFO:tasks.workunit.client.1.smithi157.stdout:6/377: mknod d1/d2f/d3b/d39/d4e/c78 0 2024-04-22T23:34:54.920 INFO:tasks.workunit.client.1.smithi157.stdout:6/378: chown d1/d2f/d3b/f2e 226318145 1 2024-04-22T23:34:54.920 INFO:tasks.workunit.client.1.smithi157.stdout:6/379: creat d1/d2f/d3b/d49/d4c/f79 x:0 0 0 2024-04-22T23:34:54.920 INFO:tasks.workunit.client.1.smithi157.stdout:0/185: dwrite d17/d23/d34/d37/f44 [0,4194304] 0 2024-04-22T23:34:54.922 INFO:tasks.workunit.client.1.smithi157.stdout:7/171: mkdir d0/d9/d29/d33 0 2024-04-22T23:34:54.924 INFO:tasks.workunit.client.1.smithi157.stdout:2/242: dread d7/f13 [0,4194304] 0 2024-04-22T23:34:54.927 INFO:tasks.workunit.client.1.smithi157.stdout:0/186: symlink d17/d23/d34/d37/l4c 0 2024-04-22T23:34:54.927 INFO:tasks.workunit.client.1.smithi157.stdout:0/187: fsync d17/d2b/f2c 0 2024-04-22T23:34:54.927 INFO:tasks.workunit.client.1.smithi157.stdout:0/188: write d17/d23/f28 [909788,18120] 0 2024-04-22T23:34:54.928 INFO:tasks.workunit.client.1.smithi157.stdout:3/163: dwrite d0/f11 [0,4194304] 0 2024-04-22T23:34:54.928 INFO:tasks.workunit.client.1.smithi157.stdout:3/164: creat d0/d7/d17/f32 x:0 0 0 2024-04-22T23:34:54.929 INFO:tasks.workunit.client.1.smithi157.stdout:7/172: creat d0/d9/d29/d33/f34 x:0 0 0 2024-04-22T23:34:54.929 INFO:tasks.workunit.client.1.smithi157.stdout:7/173: write d0/d9/d10/f18 [1025407,12291] 0 2024-04-22T23:34:54.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.935+0000 7fccfe65c700 1 -- 172.21.15.107:0/678735768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccf80fe050 msgr2=0x7fccf80fe4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:34:54.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.935+0000 7fccfe65c700 1 --2- 172.21.15.107:0/678735768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccf80fe050 0x7fccf80fe4c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fccec004940 tx=0x7fccec009e30 comp rx=0 tx=0).stop 2024-04-22T23:34:54.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.935+0000 7fccfe65c700 1 -- 172.21.15.107:0/678735768 shutdown_connections 2024-04-22T23:34:54.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.935+0000 7fccfe65c700 1 --2- 172.21.15.107:0/678735768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccf80fe050 0x7fccf80fe4c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:54.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.935+0000 7fccfe65c700 1 --2- 172.21.15.107:0/678735768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccf80fce50 0x7fccf80fd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:54.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.935+0000 7fccfe65c700 1 -- 172.21.15.107:0/678735768 >> 172.21.15.107:0/678735768 conn(0x7fccf80f8540 msgr2=0x7fccf80fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:34:54.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.935+0000 7fccfe65c700 1 -- 172.21.15.107:0/678735768 shutdown_connections 2024-04-22T23:34:54.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.935+0000 7fccfe65c700 1 -- 172.21.15.107:0/678735768 wait complete. 2024-04-22T23:34:54.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.935+0000 7fccfe65c700 1 Processor -- start 2024-04-22T23:34:54.932 INFO:tasks.workunit.client.1.smithi157.stdout:2/243: mkdir d7/d8/d25/d55 0 2024-04-22T23:34:54.932 INFO:tasks.workunit.client.1.smithi157.stdout:2/244: creat d7/d1f/f56 x:0 0 0 2024-04-22T23:34:54.932 INFO:tasks.workunit.client.1.smithi157.stdout:2/245: readlink d7/d8/d30/d23/l29 0 2024-04-22T23:34:54.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.936+0000 7fccfe65c700 1 -- start start 2024-04-22T23:34:54.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.936+0000 7fccfe65c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccf80fce50 0x7fccf810d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:34:54.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.936+0000 7fccfe65c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccf80fe050 0x7fccf810df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:34:54.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.936+0000 7fccfe65c700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf810e500 con 0x7fccf80fe050 2024-04-22T23:34:54.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.936+0000 7fccfe65c700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf810ed50 con 0x7fccf80fce50 2024-04-22T23:34:54.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.936+0000 7fccf77fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccf80fe050 0x7fccf810df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:34:54.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.936+0000 7fccf77fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccf80fe050 0x7fccf810df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51186/0 (socket says 172.21.15.107:51186) 2024-04-22T23:34:54.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.936+0000 7fccf77fe700 1 -- 172.21.15.107:0/1808318973 learned_addr learned my addr 172.21.15.107:0/1808318973 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:34:54.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.936+0000 7fccf7fff700 1 --2- 172.21.15.107:0/1808318973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccf80fce50 0x7fccf810d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:34:54.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.936+0000 7fccf77fe700 1 -- 172.21.15.107:0/1808318973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccf80fce50 msgr2=0x7fccf810d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:34:54.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.936+0000 7fccf77fe700 1 --2- 172.21.15.107:0/1808318973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccf80fce50 0x7fccf810d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:54.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.936+0000 7fccf77fe700 1 -- 172.21.15.107:0/1808318973 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccec005040 con 0x7fccf80fe050 2024-04-22T23:34:54.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.936+0000 7fccf7fff700 1 --2- 172.21.15.107:0/1808318973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccf80fce50 0x7fccf810d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T23:34:54.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.937+0000 7fccf77fe700 1 --2- 172.21.15.107:0/1808318973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccf80fe050 0x7fccf810df00 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fccec00b0d0 tx=0x7fccec009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:34:54.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.937+0000 7fccf4ff9700 1 -- 172.21.15.107:0/1808318973 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fccec00e030 con 0x7fccf80fe050 2024-04-22T23:34:54.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.937+0000 7fccf4ff9700 1 -- 172.21.15.107:0/1808318973 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fccec01c400 con 0x7fccf80fe050 2024-04-22T23:34:54.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.937+0000 7fccf4ff9700 1 -- 172.21.15.107:0/1808318973 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fccec00b650 con 0x7fccf80fe050 2024-04-22T23:34:54.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.937+0000 7fccfe65c700 1 -- 172.21.15.107:0/1808318973 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccf810efd0 con 0x7fccf80fe050 2024-04-22T23:34:54.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.937+0000 7fccfe65c700 1 -- 172.21.15.107:0/1808318973 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccf810f450 con 0x7fccf80fe050 2024-04-22T23:34:54.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.938+0000 7fccfe65c700 1 -- 172.21.15.107:0/1808318973 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccf805f5f0 con 0x7fccf80fe050 2024-04-22T23:34:54.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.939+0000 7fccf4ff9700 1 -- 172.21.15.107:0/1808318973 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fccec01c570 con 0x7fccf80fe050 2024-04-22T23:34:54.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.939+0000 7fccf4ff9700 1 --2- 172.21.15.107:0/1808318973 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fcce0069440 0x7fcce006b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:34:54.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.939+0000 7fccf4ff9700 1 -- 172.21.15.107:0/1808318973 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fccec088990 con 0x7fccf80fe050 2024-04-22T23:34:54.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.939+0000 7fccf7fff700 1 --2- 172.21.15.107:0/1808318973 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fcce0069440 0x7fcce006b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:34:54.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.940+0000 7fccf7fff700 1 --2- 172.21.15.107:0/1808318973 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fcce0069440 0x7fcce006b8f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcce8007fd0 tx=0x7fcce800b040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:34:54.937 INFO:tasks.workunit.client.1.smithi157.stdout:6/380: dread d1/d2f/d3b/d49/d4c/d56/f59 [0,4194304] 0 2024-04-22T23:34:54.937 INFO:tasks.workunit.client.1.smithi157.stdout:0/189: creat d17/d23/f4d x:0 0 0 2024-04-22T23:34:54.937 INFO:tasks.workunit.client.1.smithi157.stdout:0/190: stat d17/d38/l4a 0 2024-04-22T23:34:54.937 INFO:tasks.workunit.client.1.smithi157.stdout:0/191: write d17/d23/d34/d37/f42 [596844,6756] 0 2024-04-22T23:34:54.937 INFO:tasks.workunit.client.1.smithi157.stdout:0/192: chown f3 7338075 1 2024-04-22T23:34:54.939 INFO:tasks.workunit.client.1.smithi157.stdout:7/174: truncate d0/f4 4172687 0 2024-04-22T23:34:54.939 INFO:tasks.workunit.client.1.smithi157.stdout:0/193: dread d17/f1a [0,4194304] 0 2024-04-22T23:34:54.939 INFO:tasks.workunit.client.1.smithi157.stdout:0/194: write d17/f19 [5061668,76957] 0 2024-04-22T23:34:54.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:54.945+0000 7fccf4ff9700 1 -- 172.21.15.107:0/1808318973 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fccec0586e0 con 0x7fccf80fe050 2024-04-22T23:34:54.945 INFO:tasks.workunit.client.1.smithi157.stdout:3/165: rmdir d0/d3/d13 39 2024-04-22T23:34:54.945 INFO:tasks.workunit.client.1.smithi157.stdout:3/166: dread - d0/d7/d17/f18 zero size 2024-04-22T23:34:54.945 INFO:tasks.workunit.client.1.smithi157.stdout:3/167: write d0/d3/f6 [4289864,123709] 0 2024-04-22T23:34:54.945 INFO:tasks.workunit.client.1.smithi157.stdout:3/168: chown d0 103481553 1 2024-04-22T23:34:54.946 INFO:tasks.workunit.client.1.smithi157.stdout:7/175: mkdir d0/d9/db/d35 0 2024-04-22T23:34:54.958 INFO:tasks.workunit.client.1.smithi157.stdout:2/246: rmdir d7/d8 39 2024-04-22T23:34:54.960 INFO:tasks.workunit.client.1.smithi157.stdout:7/176: dwrite d0/f8 [0,4194304] 0 2024-04-22T23:34:54.962 INFO:tasks.workunit.client.1.smithi157.stdout:6/381: rename d1/d2f/d3b/d49/d4c to d1/d2f/d54/d7a 0 2024-04-22T23:34:54.962 INFO:tasks.workunit.client.1.smithi157.stdout:3/169: rename d0 to d0/d7/d17/d33 22 2024-04-22T23:34:54.963 INFO:tasks.workunit.client.1.smithi157.stdout:3/170: write d0/f14 [38878,79341] 0 2024-04-22T23:34:54.965 INFO:tasks.workunit.client.1.smithi157.stdout:6/382: dread d1/d36/f37 [0,4194304] 0 2024-04-22T23:34:54.969 INFO:tasks.workunit.client.1.smithi157.stdout:9/245: sync 2024-04-22T23:34:54.969 INFO:tasks.workunit.client.1.smithi157.stdout:9/246: dread - d4/d15/d23/d27/f38 zero size 2024-04-22T23:34:54.969 INFO:tasks.workunit.client.1.smithi157.stdout:9/247: chown d4/df 200335873 1 2024-04-22T23:34:54.969 INFO:tasks.workunit.client.1.smithi157.stdout:9/248: chown d4/d15/d23/d27/f33 72538755 1 2024-04-22T23:34:54.969 INFO:tasks.workunit.client.1.smithi157.stdout:9/249: stat d4/ld 0 2024-04-22T23:34:54.984 INFO:tasks.workunit.client.1.smithi157.stdout:2/247: dwrite d7/d8/d25/f4f [0,4194304] 0 2024-04-22T23:34:54.987 INFO:tasks.workunit.client.1.smithi157.stdout:2/248: mkdir d7/d8/d30/d23/d27/d57 0 2024-04-22T23:34:54.992 INFO:tasks.workunit.client.1.smithi157.stdout:2/249: dread d7/d8/f3f [4194304,4194304] 0 2024-04-22T23:34:55.007 INFO:tasks.workunit.client.1.smithi157.stdout:2/250: dread d7/d8/f35 [0,4194304] 0 2024-04-22T23:34:55.007 INFO:tasks.workunit.client.1.smithi157.stdout:2/251: chown d7 17 1 2024-04-22T23:34:55.007 INFO:tasks.workunit.client.1.smithi157.stdout:2/252: truncate d7/d1f/f52 454361 0 2024-04-22T23:34:55.007 INFO:tasks.workunit.client.1.smithi157.stdout:3/171: dwrite d0/f27 [0,4194304] 0 2024-04-22T23:34:55.009 INFO:tasks.workunit.client.1.smithi157.stdout:9/250: dwrite d4/fc [0,4194304] 0 2024-04-22T23:34:55.009 INFO:tasks.workunit.client.1.smithi157.stdout:6/383: dwrite d1/f2c [0,4194304] 0 2024-04-22T23:34:55.011 INFO:tasks.workunit.client.1.smithi157.stdout:9/251: rename d4/ld to d4/d15/d23/d3f/l48 0 2024-04-22T23:34:55.014 INFO:tasks.workunit.client.1.smithi157.stdout:6/384: unlink d1/d2f/d3b/d49/c4b 0 2024-04-22T23:34:55.014 INFO:tasks.workunit.client.1.smithi157.stdout:6/385: fsync d1/d36/f37 0 2024-04-22T23:34:55.014 INFO:tasks.workunit.client.1.smithi157.stdout:6/386: truncate d1/d17/f28 4663110 0 2024-04-22T23:34:55.015 INFO:tasks.workunit.client.1.smithi157.stdout:2/253: write d7/d8/d25/f3a [3334066,130693] 0 2024-04-22T23:34:55.015 INFO:tasks.workunit.client.1.smithi157.stdout:9/252: symlink d4/l49 0 2024-04-22T23:34:55.016 INFO:tasks.workunit.client.1.smithi157.stdout:2/254: truncate d7/d8/dd/f2b 852240 0 2024-04-22T23:34:55.018 INFO:tasks.workunit.client.1.smithi157.stdout:2/255: link d7/d8/dd/f21 d7/d8/d30/d23/d27/f58 0 2024-04-22T23:34:55.018 INFO:tasks.workunit.client.1.smithi157.stdout:2/256: dread - d7/d1f/d26/d4a/f53 zero size 2024-04-22T23:34:55.020 INFO:tasks.workunit.client.1.smithi157.stdout:2/257: mknod d7/d8/dd/c59 0 2024-04-22T23:34:55.021 INFO:tasks.workunit.client.1.smithi157.stdout:6/387: dread d1/d2f/d3b/f2a [0,4194304] 0 2024-04-22T23:34:55.021 INFO:tasks.workunit.client.1.smithi157.stdout:2/258: mknod d7/d1f/d26/c5a 0 2024-04-22T23:34:55.022 INFO:tasks.workunit.client.1.smithi157.stdout:6/388: symlink d1/l7b 0 2024-04-22T23:34:55.024 INFO:tasks.workunit.client.1.smithi157.stdout:3/172: dwrite d0/d3/f1f [0,4194304] 0 2024-04-22T23:34:55.025 INFO:tasks.workunit.client.1.smithi157.stdout:6/389: creat d1/d2f/d54/d7a/f7c x:0 0 0 2024-04-22T23:34:55.025 INFO:tasks.workunit.client.1.smithi157.stdout:6/390: write d1/d2f/d3b/f3e [233176,8541] 0 2024-04-22T23:34:55.026 INFO:tasks.workunit.client.1.smithi157.stdout:9/253: write d4/fc [3191817,50377] 0 2024-04-22T23:34:55.027 INFO:tasks.workunit.client.1.smithi157.stdout:2/259: mkdir d7/d8/d30/d5b 0 2024-04-22T23:34:55.035 INFO:tasks.workunit.client.1.smithi157.stdout:6/391: truncate d1/d38/f70 3493580 0 2024-04-22T23:34:55.036 INFO:tasks.workunit.client.1.smithi157.stdout:9/254: creat d4/d15/d23/d27/d47/f4a x:0 0 0 2024-04-22T23:34:55.036 INFO:tasks.workunit.client.1.smithi157.stdout:9/255: write d4/d15/d23/d27/f38 [742781,103759] 0 2024-04-22T23:34:55.037 INFO:tasks.workunit.client.1.smithi157.stdout:6/392: creat d1/d2f/d3b/d49/f7d x:0 0 0 2024-04-22T23:34:55.038 INFO:tasks.workunit.client.1.smithi157.stdout:6/393: readlink d1/l23 0 2024-04-22T23:34:55.038 INFO:tasks.workunit.client.1.smithi157.stdout:6/394: creat d1/d2f/d54/f7e x:0 0 0 2024-04-22T23:34:55.038 INFO:tasks.workunit.client.1.smithi157.stdout:9/256: creat d4/d15/f4b x:0 0 0 2024-04-22T23:34:55.038 INFO:tasks.workunit.client.1.smithi157.stdout:9/257: fsync d4/d15/f3b 0 2024-04-22T23:34:55.038 INFO:tasks.workunit.client.1.smithi157.stdout:9/258: readlink d4/d2d/l2f 0 2024-04-22T23:34:55.038 INFO:tasks.workunit.client.1.smithi157.stdout:9/259: creat d4/d15/f4c x:0 0 0 2024-04-22T23:34:55.038 INFO:tasks.workunit.client.1.smithi157.stdout:9/260: fdatasync d4/d15/d23/d27/d47/f4a 0 2024-04-22T23:34:55.039 INFO:tasks.workunit.client.1.smithi157.stdout:6/395: readlink d1/d38/l42 0 2024-04-22T23:34:55.041 INFO:tasks.workunit.client.1.smithi157.stdout:9/261: dread d4/d15/d23/d27/f33 [0,4194304] 0 2024-04-22T23:34:55.041 INFO:tasks.workunit.client.1.smithi157.stdout:3/173: dwrite d0/d3/f1f [0,4194304] 0 2024-04-22T23:34:55.044 INFO:tasks.workunit.client.1.smithi157.stdout:2/260: dwrite d7/d8/dd/f31 [0,4194304] 0 2024-04-22T23:34:55.045 INFO:tasks.workunit.client.1.smithi157.stdout:3/174: mkdir d0/d3/d34 0 2024-04-22T23:34:55.046 INFO:tasks.workunit.client.1.smithi157.stdout:9/262: rmdir d4/d1d 39 2024-04-22T23:34:55.047 INFO:tasks.workunit.client.1.smithi157.stdout:9/263: write d4/df/f1f [516176,53171] 0 2024-04-22T23:34:55.049 INFO:tasks.workunit.client.1.smithi157.stdout:2/261: link d7/c10 d7/c5c 0 2024-04-22T23:34:55.053 INFO:tasks.workunit.client.1.smithi157.stdout:3/175: rename d0/d3/d13/d21/f30 to d0/d3/f35 0 2024-04-22T23:34:55.054 INFO:tasks.workunit.client.1.smithi157.stdout:9/264: mknod d4/d15/d23/c4d 0 2024-04-22T23:34:55.055 INFO:tasks.workunit.client.1.smithi157.stdout:9/265: read d4/d15/d23/d3f/f42 [26150,51330] 0 2024-04-22T23:34:55.058 INFO:tasks.workunit.client.1.smithi157.stdout:2/262: write d7/d8/f2f [4185370,13358] 0 2024-04-22T23:34:55.060 INFO:tasks.workunit.client.1.smithi157.stdout:3/176: rename d0/d3/l23 to d0/d7/l36 0 2024-04-22T23:34:55.063 INFO:tasks.workunit.client.1.smithi157.stdout:9/266: dread d4/df/f16 [0,4194304] 0 2024-04-22T23:34:55.063 INFO:tasks.workunit.client.1.smithi157.stdout:9/267: creat d4/d15/f4e x:0 0 0 2024-04-22T23:34:55.068 INFO:tasks.workunit.client.1.smithi157.stdout:9/268: truncate d4/d1d/f44 3663945 0 2024-04-22T23:34:55.068 INFO:tasks.workunit.client.1.smithi157.stdout:9/269: fsync d4/d15/f4b 0 2024-04-22T23:34:55.084 INFO:tasks.workunit.client.1.smithi157.stdout:2/263: dwrite d7/d1f/f52 [0,4194304] 0 2024-04-22T23:34:55.084 INFO:tasks.workunit.client.1.smithi157.stdout:2/264: write d7/d8/d25/f33 [1645113,128534] 0 2024-04-22T23:34:55.084 INFO:tasks.workunit.client.1.smithi157.stdout:3/177: dwrite d0/f2f [0,4194304] 0 2024-04-22T23:34:55.084 INFO:tasks.workunit.client.1.smithi157.stdout:3/178: write d0/f1 [2730954,32415] 0 2024-04-22T23:34:55.086 INFO:tasks.workunit.client.1.smithi157.stdout:2/265: getdents d7/d1f/d26/d4a 0 2024-04-22T23:34:55.088 INFO:tasks.workunit.client.1.smithi157.stdout:9/270: dwrite d4/d1d/f44 [0,4194304] 0 2024-04-22T23:34:55.088 INFO:tasks.workunit.client.1.smithi157.stdout:9/271: fsync d4/d15/f3b 0 2024-04-22T23:34:55.089 INFO:tasks.workunit.client.1.smithi157.stdout:3/179: rename d0/d3/c22 to d0/c37 0 2024-04-22T23:34:55.093 INFO:tasks.workunit.client.1.smithi157.stdout:3/180: symlink d0/d7/l38 0 2024-04-22T23:34:55.093 INFO:tasks.workunit.client.1.smithi157.stdout:2/266: symlink d7/d8/d30/l5d 0 2024-04-22T23:34:55.098 INFO:tasks.workunit.client.1.smithi157.stdout:3/181: rename d0/d7/d17 to d0/d3/d34/d39 0 2024-04-22T23:34:55.111 INFO:tasks.workunit.client.1.smithi157.stdout:9/272: dwrite d4/df/f12 [4194304,4194304] 0 2024-04-22T23:34:55.111 INFO:tasks.workunit.client.1.smithi157.stdout:9/273: dread - d4/d15/d23/d27/f39 zero size 2024-04-22T23:34:55.112 INFO:tasks.workunit.client.1.smithi157.stdout:9/274: dread d4/d15/d23/d27/f38 [0,4194304] 0 2024-04-22T23:34:55.114 INFO:tasks.workunit.client.1.smithi157.stdout:2/267: dwrite d7/d1f/d26/f44 [0,4194304] 0 2024-04-22T23:34:55.115 INFO:tasks.workunit.client.1.smithi157.stdout:3/182: dwrite d0/d3/d34/d39/f25 [0,4194304] 0 2024-04-22T23:34:55.115 INFO:tasks.workunit.client.1.smithi157.stdout:3/183: write d0/f27 [4453502,126011] 0 2024-04-22T23:34:55.118 INFO:tasks.workunit.client.1.smithi157.stdout:5/250: sync 2024-04-22T23:34:55.119 INFO:tasks.workunit.client.1.smithi157.stdout:2/268: write d7/d8/f35 [1164670,117695] 0 2024-04-22T23:34:55.120 INFO:tasks.workunit.client.1.smithi157.stdout:5/251: mkdir d1/d5/d7/d2f/d45/d54/d59 0 2024-04-22T23:34:55.121 INFO:tasks.workunit.client.1.smithi157.stdout:5/252: chown d1/d1f/d22 1 1 2024-04-22T23:34:55.121 INFO:tasks.workunit.client.1.smithi157.stdout:5/253: creat d1/f5a x:0 0 0 2024-04-22T23:34:55.122 INFO:tasks.workunit.client.1.smithi157.stdout:2/269: creat d7/d1f/d3e/f5e x:0 0 0 2024-04-22T23:34:55.122 INFO:tasks.workunit.client.1.smithi157.stdout:2/270: dread - d7/d8/d30/d23/d27/f49 zero size 2024-04-22T23:34:55.124 INFO:tasks.workunit.client.1.smithi157.stdout:5/254: symlink d1/d5/dc/l5b 0 2024-04-22T23:34:55.124 INFO:tasks.workunit.client.1.smithi157.stdout:5/255: creat d1/d27/d29/d4b/f5c x:0 0 0 2024-04-22T23:34:55.124 INFO:tasks.workunit.client.1.smithi157.stdout:5/256: stat d1/d27/d29/f58 0 2024-04-22T23:34:55.124 INFO:tasks.workunit.client.1.smithi157.stdout:5/257: readlink d1/d5/d7/la 0 2024-04-22T23:34:55.125 INFO:tasks.workunit.client.1.smithi157.stdout:2/271: mkdir d7/d8/d30/d23/d27/d5f 0 2024-04-22T23:34:55.125 INFO:tasks.workunit.client.1.smithi157.stdout:2/272: write d7/d8/d25/f50 [384673,74909] 0 2024-04-22T23:34:55.126 INFO:tasks.workunit.client.1.smithi157.stdout:5/258: mkdir d1/d5/dc/d5d 0 2024-04-22T23:34:55.127 INFO:tasks.workunit.client.1.smithi157.stdout:5/259: mkdir d1/d5/dc/d43/d5e 0 2024-04-22T23:34:55.127 INFO:tasks.workunit.client.1.smithi157.stdout:5/260: creat d1/d27/d29/f5f x:0 0 0 2024-04-22T23:34:55.127 INFO:tasks.workunit.client.1.smithi157.stdout:5/261: chown d1/d5/d7/d10/l1e 2006 1 2024-04-22T23:34:55.127 INFO:tasks.workunit.client.1.smithi157.stdout:3/184: dread d0/d3/f1e [0,4194304] 0 2024-04-22T23:34:55.128 INFO:tasks.workunit.client.1.smithi157.stdout:5/262: mkdir d1/d5/dc/d43/d4f/d60 0 2024-04-22T23:34:55.129 INFO:tasks.workunit.client.1.smithi157.stdout:5/263: mknod d1/d5/d7/d2f/c61 0 2024-04-22T23:34:55.129 INFO:tasks.workunit.client.1.smithi157.stdout:5/264: creat d1/d5/dc/f62 x:0 0 0 2024-04-22T23:34:55.133 INFO:tasks.workunit.client.1.smithi157.stdout:9/275: dwrite d4/f20 [0,4194304] 0 2024-04-22T23:34:55.137 INFO:tasks.workunit.client.1.smithi157.stdout:5/265: unlink d1/d5/d7/f21 0 2024-04-22T23:34:55.137 INFO:tasks.workunit.client.1.smithi157.stdout:5/266: creat d1/d27/d29/d4b/f63 x:0 0 0 2024-04-22T23:34:55.140 INFO:tasks.workunit.client.1.smithi157.stdout:2/273: write d7/d1f/d26/f44 [3333979,31034] 0 2024-04-22T23:34:55.142 INFO:tasks.workunit.client.1.smithi157.stdout:2/274: write d7/d8/dd/f21 [5008913,25419] 0 2024-04-22T23:34:55.144 INFO:tasks.workunit.client.1.smithi157.stdout:3/185: dwrite d0/d3/d13/f2b [0,4194304] 0 2024-04-22T23:34:55.144 INFO:tasks.workunit.client.1.smithi157.stdout:2/275: dread d7/d8/d30/d23/f48 [0,4194304] 0 2024-04-22T23:34:55.146 INFO:tasks.workunit.client.1.smithi157.stdout:9/276: dread d4/d15/f19 [0,4194304] 0 2024-04-22T23:34:55.149 INFO:tasks.workunit.client.1.smithi157.stdout:2/276: creat d7/d1f/d26/d4a/f60 x:0 0 0 2024-04-22T23:34:55.151 INFO:tasks.workunit.client.1.smithi157.stdout:5/267: dwrite d1/f26 [0,4194304] 0 2024-04-22T23:34:55.154 INFO:tasks.workunit.client.1.smithi157.stdout:9/277: rename d4/d15/f3b to d4/d15/d23/d27/f4f 0 2024-04-22T23:34:55.155 INFO:tasks.workunit.client.1.smithi157.stdout:9/278: creat d4/f50 x:0 0 0 2024-04-22T23:34:55.155 INFO:tasks.workunit.client.1.smithi157.stdout:3/186: getdents d0/d7 0 2024-04-22T23:34:55.157 INFO:tasks.workunit.client.1.smithi157.stdout:2/277: symlink d7/d1f/d26/d4a/d54/l61 0 2024-04-22T23:34:55.157 INFO:tasks.workunit.client.1.smithi157.stdout:2/278: read - d7/d1f/d26/d4a/f60 zero size 2024-04-22T23:34:55.157 INFO:tasks.workunit.client.1.smithi157.stdout:2/279: fsync d7/d8/dd/f31 0 2024-04-22T23:34:55.158 INFO:tasks.workunit.client.1.smithi157.stdout:5/268: read d1/d1f/d22/f35 [877485,113659] 0 2024-04-22T23:34:55.158 INFO:tasks.workunit.client.1.smithi157.stdout:9/279: mknod d4/d1d/c51 0 2024-04-22T23:34:55.159 INFO:tasks.workunit.client.1.smithi157.stdout:5/269: mknod d1/d5/dc/d43/d4f/c64 0 2024-04-22T23:34:55.159 INFO:tasks.workunit.client.1.smithi157.stdout:5/270: chown d1/d5/d7/d10/l1e 2919179 1 2024-04-22T23:34:55.160 INFO:tasks.workunit.client.1.smithi157.stdout:3/187: dread d0/f14 [0,4194304] 0 2024-04-22T23:34:55.160 INFO:tasks.workunit.client.1.smithi157.stdout:3/188: chown d0/d3/f35 27 1 2024-04-22T23:34:55.161 INFO:tasks.workunit.client.1.smithi157.stdout:2/280: dread d7/d8/d25/f3a [0,4194304] 0 2024-04-22T23:34:55.161 INFO:tasks.workunit.client.1.smithi157.stdout:5/271: creat d1/d5/d7/d2f/d45/d54/f65 x:0 0 0 2024-04-22T23:34:55.161 INFO:tasks.workunit.client.1.smithi157.stdout:2/281: readlink d7/d8/d30/l20 0 2024-04-22T23:34:55.166 INFO:tasks.workunit.client.1.smithi157.stdout:9/280: mkdir d4/df/d52 0 2024-04-22T23:34:55.167 INFO:tasks.workunit.client.1.smithi157.stdout:5/272: symlink d1/d5/dc/l66 0 2024-04-22T23:34:55.167 INFO:tasks.workunit.client.1.smithi157.stdout:9/281: chown d4/d15/d23/d27/f39 29776803 1 2024-04-22T23:34:55.168 INFO:tasks.workunit.client.1.smithi157.stdout:3/189: dread d0/f11 [0,4194304] 0 2024-04-22T23:34:55.170 INFO:tasks.workunit.client.1.smithi157.stdout:3/190: rmdir d0/d7 39 2024-04-22T23:34:55.179 INFO:tasks.workunit.client.1.smithi157.stdout:5/273: dread d1/d5/d7/d10/f49 [0,4194304] 0 2024-04-22T23:34:55.179 INFO:tasks.workunit.client.1.smithi157.stdout:5/274: creat d1/d5/d7/f67 x:0 0 0 2024-04-22T23:34:55.179 INFO:tasks.workunit.client.1.smithi157.stdout:5/275: write d1/d5/d7/d2f/d45/f48 [472480,62539] 0 2024-04-22T23:34:55.180 INFO:tasks.workunit.client.1.smithi157.stdout:5/276: fsync d1/d5/d7/d10/f31 0 2024-04-22T23:34:55.180 INFO:tasks.workunit.client.1.smithi157.stdout:5/277: rmdir d1/d1f 39 2024-04-22T23:34:55.180 INFO:tasks.workunit.client.1.smithi157.stdout:9/282: dread d4/df/fb [0,4194304] 0 2024-04-22T23:34:55.182 INFO:tasks.workunit.client.1.smithi157.stdout:5/278: rename d1/d5/d7/d2f/d45/l4a to d1/d27/d29/l68 0 2024-04-22T23:34:55.183 INFO:tasks.workunit.client.1.smithi157.stdout:5/279: mknod d1/d5/d7/c69 0 2024-04-22T23:34:55.184 INFO:tasks.workunit.client.1.smithi157.stdout:5/280: read d1/d5/dc/f1b [2966485,102492] 0 2024-04-22T23:34:55.186 INFO:tasks.workunit.client.1.smithi157.stdout:5/281: creat d1/d5/d7/d2f/d45/d54/d59/f6a x:0 0 0 2024-04-22T23:34:55.186 INFO:tasks.workunit.client.1.smithi157.stdout:2/282: dwrite d7/d8/f35 [0,4194304] 0 2024-04-22T23:34:55.188 INFO:tasks.workunit.client.1.smithi157.stdout:9/283: dread d4/f8 [0,4194304] 0 2024-04-22T23:34:55.190 INFO:tasks.workunit.client.1.smithi157.stdout:2/283: symlink d7/d8/l62 0 2024-04-22T23:34:55.191 INFO:tasks.workunit.client.1.smithi157.stdout:3/191: dwrite d0/f14 [0,4194304] 0 2024-04-22T23:34:55.191 INFO:tasks.workunit.client.1.smithi157.stdout:9/284: unlink d4/d15/f19 0 2024-04-22T23:34:55.194 INFO:tasks.workunit.client.1.smithi157.stdout:5/282: write d1/d5/d7/d10/f31 [4049388,90829] 0 2024-04-22T23:34:55.195 INFO:tasks.workunit.client.1.smithi157.stdout:5/283: fdatasync d1/d5/f14 0 2024-04-22T23:34:55.195 INFO:tasks.workunit.client.1.smithi157.stdout:2/284: creat d7/d1f/d26/d4a/d54/f63 x:0 0 0 2024-04-22T23:34:55.197 INFO:tasks.workunit.client.1.smithi157.stdout:5/284: symlink d1/d27/d29/l6b 0 2024-04-22T23:34:55.199 INFO:tasks.workunit.client.1.smithi157.stdout:2/285: getdents d7/d1f/d26/d4a 0 2024-04-22T23:34:55.200 INFO:tasks.workunit.client.1.smithi157.stdout:2/286: creat d7/d1f/d26/f64 x:0 0 0 2024-04-22T23:34:55.200 INFO:tasks.workunit.client.1.smithi157.stdout:5/285: link d1/d5/dc/d43/d4f/l50 d1/d5/d7/d2f/d45/l6c 0 2024-04-22T23:34:55.202 INFO:tasks.workunit.client.1.smithi157.stdout:2/287: symlink d7/d1f/d26/d4a/d54/l65 0 2024-04-22T23:34:55.203 INFO:tasks.workunit.client.1.smithi157.stdout:5/286: mknod d1/d27/d29/c6d 0 2024-04-22T23:34:55.205 INFO:tasks.workunit.client.1.smithi157.stdout:5/287: mknod d1/d5/c6e 0 2024-04-22T23:34:55.205 INFO:tasks.workunit.client.1.smithi157.stdout:5/288: truncate d1/d27/d29/f58 557861 0 2024-04-22T23:34:55.205 INFO:tasks.workunit.client.1.smithi157.stdout:5/289: write d1/d27/d29/d4b/f63 [298584,91438] 0 2024-04-22T23:34:55.205 INFO:tasks.workunit.client.1.smithi157.stdout:5/290: fdatasync d1/d5/d7/d10/f3d 0 2024-04-22T23:34:55.206 INFO:tasks.workunit.client.1.smithi157.stdout:2/288: mknod d7/d8/d30/c66 0 2024-04-22T23:34:55.207 INFO:tasks.workunit.client.1.smithi157.stdout:9/285: dwrite d4/f50 [0,4194304] 0 2024-04-22T23:34:55.207 INFO:tasks.workunit.client.1.smithi157.stdout:2/289: write d7/d8/f3f [4199604,48770] 0 2024-04-22T23:34:55.207 INFO:tasks.workunit.client.1.smithi157.stdout:2/290: stat d7/d1f/d26/d4a/d54/f63 0 2024-04-22T23:34:55.207 INFO:tasks.workunit.client.1.smithi157.stdout:2/291: chown d7/d8/d30/d23 40718226 1 2024-04-22T23:34:55.207 INFO:tasks.workunit.client.1.smithi157.stdout:2/292: dread - d7/d1f/d26/d4a/f4b zero size 2024-04-22T23:34:55.207 INFO:tasks.workunit.client.1.smithi157.stdout:5/291: symlink d1/d5/d7/d2f/d45/d54/l6f 0 2024-04-22T23:34:55.210 INFO:tasks.workunit.client.1.smithi157.stdout:1/250: sync 2024-04-22T23:34:55.212 INFO:tasks.workunit.client.1.smithi157.stdout:2/293: unlink d7/d1f/d26/d4a/c47 0 2024-04-22T23:34:55.213 INFO:tasks.workunit.client.1.smithi157.stdout:5/292: rename d1/d5/d7/d10/f3d to d1/d5/d7/d2f/d45/d54/d59/f70 0 2024-04-22T23:34:55.215 INFO:tasks.workunit.client.1.smithi157.stdout:1/251: symlink d4/d17/d34/l55 0 2024-04-22T23:34:55.216 INFO:tasks.workunit.client.1.smithi157.stdout:1/252: creat d4/d17/d34/d3b/d50/f56 x:0 0 0 2024-04-22T23:34:55.219 INFO:tasks.workunit.client.1.smithi157.stdout:0/195: sync 2024-04-22T23:34:55.220 INFO:tasks.workunit.client.1.smithi157.stdout:7/177: sync 2024-04-22T23:34:55.220 INFO:tasks.workunit.client.1.smithi157.stdout:8/135: sync 2024-04-22T23:34:55.220 INFO:tasks.workunit.client.1.smithi157.stdout:6/396: sync 2024-04-22T23:34:55.220 INFO:tasks.workunit.client.1.smithi157.stdout:4/326: sync 2024-04-22T23:34:55.222 INFO:tasks.workunit.client.1.smithi157.stdout:5/293: dread d1/d5/f14 [4194304,4194304] 0 2024-04-22T23:34:55.222 INFO:tasks.workunit.client.1.smithi157.stdout:0/196: mknod d17/d23/c4e 0 2024-04-22T23:34:55.222 INFO:tasks.workunit.client.1.smithi157.stdout:0/197: truncate d17/d23/d34/d37/f3f 908418 0 2024-04-22T23:34:55.222 INFO:tasks.workunit.client.1.smithi157.stdout:0/198: write d17/f2d [3309755,109389] 0 2024-04-22T23:34:55.223 INFO:tasks.workunit.client.1.smithi157.stdout:7/178: mkdir d0/d9/d36 0 2024-04-22T23:34:55.223 INFO:tasks.workunit.client.1.smithi157.stdout:7/179: write d0/d9/d29/d33/f34 [376111,119606] 0 2024-04-22T23:34:55.223 INFO:tasks.workunit.client.1.smithi157.stdout:4/327: creat d3/d11/d30/d36/f65 x:0 0 0 2024-04-22T23:34:55.226 INFO:tasks.workunit.client.1.smithi157.stdout:4/328: mknod d3/d11/d34/c66 0 2024-04-22T23:34:55.226 INFO:tasks.workunit.client.1.smithi157.stdout:9/286: dwrite d4/d1d/f44 [0,4194304] 0 2024-04-22T23:34:55.226 INFO:tasks.workunit.client.1.smithi157.stdout:9/287: chown d4/df/f16 194149 1 2024-04-22T23:34:55.226 INFO:tasks.workunit.client.1.smithi157.stdout:4/329: write d3/d11/f22 [1307174,48629] 0 2024-04-22T23:34:55.227 INFO:tasks.workunit.client.1.smithi157.stdout:9/288: chown d4 418 1 2024-04-22T23:34:55.227 INFO:tasks.workunit.client.1.smithi157.stdout:4/330: dread - d3/d11/d30/d36/d62/d4c/f61 zero size 2024-04-22T23:34:55.227 INFO:tasks.workunit.client.1.smithi157.stdout:4/331: read d3/d11/d1d/f2d [3770295,95031] 0 2024-04-22T23:34:55.229 INFO:tasks.workunit.client.1.smithi157.stdout:4/332: rename d3/d11/d1d/f53 to d3/d11/d30/d36/d62/f67 0 2024-04-22T23:34:55.232 INFO:tasks.workunit.client.1.smithi157.stdout:2/294: dwrite d7/d1f/f56 [0,4194304] 0 2024-04-22T23:34:55.236 INFO:tasks.workunit.client.1.smithi157.stdout:9/289: read d4/df/f12 [3969290,85592] 0 2024-04-22T23:34:55.239 INFO:tasks.workunit.client.1.smithi157.stdout:4/333: dread d3/d4b/f16 [0,4194304] 0 2024-04-22T23:34:55.240 INFO:tasks.workunit.client.1.smithi157.stdout:4/334: fdatasync d3/d11/d1d/f49 0 2024-04-22T23:34:55.240 INFO:tasks.workunit.client.1.smithi157.stdout:4/335: chown d3/d4b/ff 666 1 2024-04-22T23:34:55.241 INFO:tasks.workunit.client.1.smithi157.stdout:4/336: symlink d3/d11/d30/d36/d62/d40/l68 0 2024-04-22T23:34:55.251 INFO:tasks.workunit.client.1.smithi157.stdout:6/397: dwrite d1/f34 [0,4194304] 0 2024-04-22T23:34:55.251 INFO:tasks.workunit.client.1.smithi157.stdout:6/398: chown d1/d2f/d54/d7a/d56/f71 62715 1 2024-04-22T23:34:55.251 INFO:tasks.workunit.client.1.smithi157.stdout:6/399: truncate d1/d2f/d3b/f3e 685727 0 2024-04-22T23:34:55.253 INFO:tasks.workunit.client.1.smithi157.stdout:1/253: dwrite d4/f16 [0,4194304] 0 2024-04-22T23:34:55.253 INFO:tasks.workunit.client.1.smithi157.stdout:2/295: dwrite d7/d1f/d26/d4a/f60 [0,4194304] 0 2024-04-22T23:34:55.255 INFO:tasks.workunit.client.1.smithi157.stdout:1/254: dread d4/db/fe [0,4194304] 0 2024-04-22T23:34:55.257 INFO:tasks.workunit.client.1.smithi157.stdout:0/199: dread d17/d23/f27 [0,4194304] 0 2024-04-22T23:34:55.257 INFO:tasks.workunit.client.1.smithi157.stdout:2/296: write d7/d8/d25/f3a [7230080,118193] 0 2024-04-22T23:34:55.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.262+0000 7fccfe65c700 1 -- 172.21.15.107:0/1808318973 --> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fccf810f830 con 0x7fcce0069440 2024-04-22T23:34:55.258 INFO:tasks.workunit.client.1.smithi157.stdout:8/136: dwrite d2/d6/d21/f2d [0,4194304] 0 2024-04-22T23:34:55.258 INFO:tasks.workunit.client.1.smithi157.stdout:8/137: chown d2/d6/d21/c28 0 1 2024-04-22T23:34:55.258 INFO:tasks.workunit.client.1.smithi157.stdout:8/138: stat d2/d6/de/d11 0 2024-04-22T23:34:55.258 INFO:tasks.workunit.client.1.smithi157.stdout:8/139: write d2/d6/d21/f23 [964225,91686] 0 2024-04-22T23:34:55.260 INFO:tasks.workunit.client.1.smithi157.stdout:9/290: dwrite d4/df/f1f [0,4194304] 0 2024-04-22T23:34:55.260 INFO:tasks.workunit.client.1.smithi157.stdout:9/291: chown d4/l49 4819416 1 2024-04-22T23:34:55.260 INFO:tasks.workunit.client.1.smithi157.stdout:9/292: read - d4/d15/d23/d27/f39 zero size 2024-04-22T23:34:55.260 INFO:tasks.workunit.client.1.smithi157.stdout:9/293: truncate d4/d15/d23/d27/d47/f4a 50183 0 2024-04-22T23:34:55.260 INFO:tasks.workunit.client.1.smithi157.stdout:9/294: chown d4/d1d/c51 38956004 1 2024-04-22T23:34:55.263 INFO:tasks.workunit.client.1.smithi157.stdout:6/400: dread d1/d17/f27 [0,4194304] 0 2024-04-22T23:34:55.264 INFO:tasks.workunit.client.1.smithi157.stdout:6/401: write d1/d2f/d3b/d39/d4e/f5f [390705,22554] 0 2024-04-22T23:34:55.264 INFO:tasks.workunit.client.1.smithi157.stdout:2/297: read d7/d8/d30/d23/f48 [29006,45139] 0 2024-04-22T23:34:55.268 INFO:tasks.workunit.client.1.smithi157.stdout:4/337: dwrite d3/d4b/f4f [0,4194304] 0 2024-04-22T23:34:55.268 INFO:tasks.workunit.client.1.smithi157.stdout:4/338: write d3/d11/d30/d28/f3c [875884,129376] 0 2024-04-22T23:34:55.271 INFO:tasks.workunit.client.1.smithi157.stdout:1/255: symlink d4/d17/d1d/d33/l57 0 2024-04-22T23:34:55.275 INFO:tasks.workunit.client.1.smithi157.stdout:9/295: unlink d4/d15/l46 0 2024-04-22T23:34:55.276 INFO:tasks.workunit.client.1.smithi157.stdout:9/296: chown d4/l49 233366 1 2024-04-22T23:34:55.277 INFO:tasks.workunit.client.1.smithi157.stdout:6/402: unlink d1/d38/f70 0 2024-04-22T23:34:55.279 INFO:tasks.workunit.client.1.smithi157.stdout:0/200: dwrite d17/d2b/f3d [0,4194304] 0 2024-04-22T23:34:55.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.285+0000 7fccf4ff9700 1 -- 172.21.15.107:0/1808318973 <== mgr.14206 v2:172.21.15.107:6800/2516490111 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3344 (secure 0 0 0) 0x7fccf810f830 con 0x7fcce0069440 2024-04-22T23:34:55.280 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-22T23:34:55.280 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (6m) 2m ago 7m 25.5M - 0.25.0 c8568f914cd2 4d90ceb89249 2024-04-22T23:34:55.280 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (8m) 2m ago 8m 6660k - 17.2.7-884-g70d0cb4b cafd8ddac8fa c10611c756e4 2024-04-22T23:34:55.280 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi157 smithi157 running (7m) 2m ago 7m 6669k - 17.2.7-884-g70d0cb4b cafd8ddac8fa 55d2d46294c9 2024-04-22T23:34:55.280 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (6m) 2m ago 7m 86.8M - 9.4.7 954c08fa6188 c2d1f8de1aa7 2024-04-22T23:34:55.280 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.dfvvkw smithi107 running (2m) 2m ago 2m 10.4M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 08d3aa8d717d 2024-04-22T23:34:55.280 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.ktuxjl smithi107 running (2m) 2m ago 2m 16.9M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 762f27fe0f30 2024-04-22T23:34:55.281 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi157.rcaris smithi157 running (2m) 2m ago 2m 16.8M - 17.2.7-884-g70d0cb4b cafd8ddac8fa d23366ac085d 2024-04-22T23:34:55.281 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi157.ufywse smithi157 running (2m) 2m ago 2m 11.0M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 8b00d61eb355 2024-04-22T23:34:55.281 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.uipxvm smithi107 *:9283 running (9m) 2m ago 9m 481M - 17.2.7-884-g70d0cb4b cafd8ddac8fa db96e8cbbdfb 2024-04-22T23:34:55.281 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi157.tuylis smithi157 *:8443,9283 running (7m) 2m ago 7m 431M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 0c12be10ffaf 2024-04-22T23:34:55.281 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (9m) 2m ago 9m 49.6M 2048M 17.2.7-884-g70d0cb4b cafd8ddac8fa 8314962fa1b9 2024-04-22T23:34:55.281 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi157 smithi157 running (6m) 2m ago 6m 36.6M 2048M 17.2.7-884-g70d0cb4b cafd8ddac8fa 0747a85008d9 2024-04-22T23:34:55.281 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (8m) 2m ago 8m 16.0M - 1.5.0 0da6a335fe13 8a7c595c114f 2024-04-22T23:34:55.281 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi157 smithi157 *:9100 running (7m) 2m ago 7m 16.1M - 1.5.0 0da6a335fe13 940164771fe3 2024-04-22T23:34:55.281 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (6m) 2m ago 6m 46.6M 4288M 17.2.7-884-g70d0cb4b cafd8ddac8fa 18a179b0a618 2024-04-22T23:34:55.281 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (5m) 2m ago 5m 45.9M 4288M 17.2.7-884-g70d0cb4b cafd8ddac8fa c7272eb84ddb 2024-04-22T23:34:55.281 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (5m) 2m ago 5m 46.2M 4288M 17.2.7-884-g70d0cb4b cafd8ddac8fa 9db1788096ad 2024-04-22T23:34:55.281 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi157 running (4m) 2m ago 4m 43.9M 5312M 17.2.7-884-g70d0cb4b cafd8ddac8fa c9fb2d1a13dd 2024-04-22T23:34:55.281 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi157 running (4m) 2m ago 4m 44.2M 5312M 17.2.7-884-g70d0cb4b cafd8ddac8fa 24a16e19951f 2024-04-22T23:34:55.281 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi157 running (3m) 2m ago 3m 43.1M 5312M 17.2.7-884-g70d0cb4b cafd8ddac8fa 4c5b7a7ef8bd 2024-04-22T23:34:55.281 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (6m) 2m ago 7m 51.2M - 2.43.0 a07b618ecd1d 8c5df40caabe 2024-04-22T23:34:55.282 INFO:tasks.workunit.client.1.smithi157.stdout:2/298: creat d7/d8/d30/d23/d27/f67 x:0 0 0 2024-04-22T23:34:55.282 INFO:tasks.workunit.client.1.smithi157.stdout:4/339: creat d3/d11/d30/d28/d31/f69 x:0 0 0 2024-04-22T23:34:55.282 INFO:tasks.workunit.client.1.smithi157.stdout:4/340: creat d3/d11/d30/d28/d31/f6a x:0 0 0 2024-04-22T23:34:55.282 INFO:tasks.workunit.client.1.smithi157.stdout:4/341: stat d3 0 2024-04-22T23:34:55.283 INFO:tasks.workunit.client.1.smithi157.stdout:4/342: fdatasync d3/d4b/f16 0 2024-04-22T23:34:55.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.288+0000 7fccfe65c700 1 -- 172.21.15.107:0/1808318973 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fcce0069440 msgr2=0x7fcce006b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:34:55.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.288+0000 7fccfe65c700 1 --2- 172.21.15.107:0/1808318973 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fcce0069440 0x7fcce006b8f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcce8007fd0 tx=0x7fcce800b040 comp rx=0 tx=0).stop 2024-04-22T23:34:55.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.288+0000 7fccfe65c700 1 -- 172.21.15.107:0/1808318973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccf80fe050 msgr2=0x7fccf810df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:34:55.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.288+0000 7fccfe65c700 1 --2- 172.21.15.107:0/1808318973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccf80fe050 0x7fccf810df00 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fccec00b0d0 tx=0x7fccec009e30 comp rx=0 tx=0).stop 2024-04-22T23:34:55.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.288+0000 7fccfe65c700 1 -- 172.21.15.107:0/1808318973 shutdown_connections 2024-04-22T23:34:55.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.288+0000 7fccfe65c700 1 --2- 172.21.15.107:0/1808318973 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fcce0069440 0x7fcce006b8f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:55.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.288+0000 7fccfe65c700 1 --2- 172.21.15.107:0/1808318973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccf80fce50 0x7fccf810d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:55.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.288+0000 7fccfe65c700 1 --2- 172.21.15.107:0/1808318973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccf80fe050 0x7fccf810df00 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:55.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.288+0000 7fccfe65c700 1 -- 172.21.15.107:0/1808318973 >> 172.21.15.107:0/1808318973 conn(0x7fccf80f8540 msgr2=0x7fccf80f9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:34:55.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.289+0000 7fccfe65c700 1 -- 172.21.15.107:0/1808318973 shutdown_connections 2024-04-22T23:34:55.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.289+0000 7fccfe65c700 1 -- 172.21.15.107:0/1808318973 wait complete. 2024-04-22T23:34:55.285 INFO:tasks.workunit.client.1.smithi157.stdout:9/297: unlink d4/d1d/c51 0 2024-04-22T23:34:55.285 INFO:tasks.workunit.client.1.smithi157.stdout:9/298: chown d4/d15/d23/d3f/f45 3305 1 2024-04-22T23:34:55.285 INFO:tasks.workunit.client.1.smithi157.stdout:6/403: symlink d1/d2f/d3b/d39/d4e/d74/l7f 0 2024-04-22T23:34:55.289 INFO:tasks.workunit.client.1.smithi157.stdout:0/201: truncate d17/f2d 1600115 0 2024-04-22T23:34:55.291 INFO:tasks.workunit.client.1.smithi157.stdout:2/299: creat d7/d8/d25/f68 x:0 0 0 2024-04-22T23:34:55.294 INFO:tasks.workunit.client.1.smithi157.stdout:8/140: dwrite d2/d4/f1c [0,4194304] 0 2024-04-22T23:34:55.295 INFO:tasks.workunit.client.1.smithi157.stdout:8/141: chown d2/d4 1813917 1 2024-04-22T23:34:55.295 INFO:tasks.workunit.client.1.smithi157.stdout:1/256: dwrite d4/db/fe [0,4194304] 0 2024-04-22T23:34:55.296 INFO:tasks.workunit.client.1.smithi157.stdout:6/404: rename d1/d2f/d3b/d39/d4e/l51 to d1/d2f/d3b/d6c/l80 0 2024-04-22T23:34:55.296 INFO:tasks.workunit.client.1.smithi157.stdout:6/405: stat d1/d2f/d3b 0 2024-04-22T23:34:55.296 INFO:tasks.workunit.client.1.smithi157.stdout:6/406: chown d1/d36/l3c 2 1 2024-04-22T23:34:55.296 INFO:tasks.workunit.client.1.smithi157.stdout:0/202: mkdir d17/d2b/d4f 0 2024-04-22T23:34:55.297 INFO:tasks.workunit.client.1.smithi157.stdout:0/203: chown d17/d23/d34/d37 188010046 1 2024-04-22T23:34:55.299 INFO:tasks.workunit.client.1.smithi157.stdout:0/204: dread d17/f1a [0,4194304] 0 2024-04-22T23:34:55.300 INFO:tasks.workunit.client.1.smithi157.stdout:9/299: dwrite d4/f20 [4194304,4194304] 0 2024-04-22T23:34:55.302 INFO:tasks.workunit.client.1.smithi157.stdout:0/205: dread d17/d23/f28 [0,4194304] 0 2024-04-22T23:34:55.302 INFO:tasks.workunit.client.1.smithi157.stdout:0/206: chown d17/c1c 3 1 2024-04-22T23:34:55.302 INFO:tasks.workunit.client.1.smithi157.stdout:0/207: stat d17/f1b 0 2024-04-22T23:34:55.303 INFO:tasks.workunit.client.1.smithi157.stdout:8/142: getdents d2/d6/de/d11 0 2024-04-22T23:34:55.303 INFO:tasks.workunit.client.1.smithi157.stdout:9/300: dread d4/df/f12 [0,4194304] 0 2024-04-22T23:34:55.303 INFO:tasks.workunit.client.1.smithi157.stdout:0/208: write d17/d23/d34/d37/f42 [140475,86924] 0 2024-04-22T23:34:55.303 INFO:tasks.workunit.client.1.smithi157.stdout:0/209: truncate d17/d23/f3e 67563 0 2024-04-22T23:34:55.305 INFO:tasks.workunit.client.1.smithi157.stdout:0/210: dread d17/d23/f28 [0,4194304] 0 2024-04-22T23:34:55.305 INFO:tasks.workunit.client.1.smithi157.stdout:1/257: mkdir d4/d17/d34/d3b/d52/d58 0 2024-04-22T23:34:55.305 INFO:tasks.workunit.client.1.smithi157.stdout:1/258: creat d4/d17/f59 x:0 0 0 2024-04-22T23:34:55.305 INFO:tasks.workunit.client.1.smithi157.stdout:1/259: creat d4/db/f5a x:0 0 0 2024-04-22T23:34:55.306 INFO:tasks.workunit.client.1.smithi157.stdout:6/407: symlink d1/d2f/d3b/d49/l81 0 2024-04-22T23:34:55.307 INFO:tasks.workunit.client.1.smithi157.stdout:2/300: mknod d7/d8/dd/c69 0 2024-04-22T23:34:55.311 INFO:tasks.workunit.client.1.smithi157.stdout:4/343: rmdir d3/d11/d30/d28/d31 39 2024-04-22T23:34:55.311 INFO:tasks.workunit.client.1.smithi157.stdout:4/344: write d3/d11/d1d/f49 [500042,100521] 0 2024-04-22T23:34:55.312 INFO:tasks.workunit.client.1.smithi157.stdout:8/143: truncate d2/d6/de/d17/f1e 292673 0 2024-04-22T23:34:55.314 INFO:tasks.workunit.client.1.smithi157.stdout:9/301: rename d4/d15/f4b to d4/d15/d23/d27/f53 0 2024-04-22T23:34:55.321 INFO:tasks.workunit.client.1.smithi157.stdout:4/345: symlink d3/d11/d30/d28/d31/l6b 0 2024-04-22T23:34:55.321 INFO:tasks.workunit.client.1.smithi157.stdout:8/144: link d2/d6/d21/f25 d2/d6/de/f2f 0 2024-04-22T23:34:55.332 INFO:tasks.workunit.client.1.smithi157.stdout:8/145: dread d2/d6/de/fd [0,4194304] 0 2024-04-22T23:34:55.332 INFO:tasks.workunit.client.1.smithi157.stdout:1/260: dwrite d4/d17/f35 [0,4194304] 0 2024-04-22T23:34:55.332 INFO:tasks.workunit.client.1.smithi157.stdout:1/261: stat d4/d17/f35 0 2024-04-22T23:34:55.332 INFO:tasks.workunit.client.1.smithi157.stdout:6/408: dwrite d1/d17/f21 [0,4194304] 0 2024-04-22T23:34:55.333 INFO:tasks.workunit.client.1.smithi157.stdout:2/301: dwrite d7/d8/d30/d23/f2c [0,4194304] 0 2024-04-22T23:34:55.337 INFO:tasks.workunit.client.1.smithi157.stdout:6/409: dread d1/d2f/d3b/d39/f46 [0,4194304] 0 2024-04-22T23:34:55.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:34:55 smithi157 ceph-mon[33877]: pgmap v254: 65 pgs: 65 active+clean; 314 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 16 MiB/s rd, 27 MiB/s wr, 816 op/s 2024-04-22T23:34:55.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:34:55 smithi157 ceph-mon[33877]: from='client.14636 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:34:55.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.401+0000 7f4064ce1700 1 -- 172.21.15.107:0/344376196 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40600fbd40 msgr2=0x7f40600fc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:34:55.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.401+0000 7f4064ce1700 1 --2- 172.21.15.107:0/344376196 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40600fbd40 0x7f40600fc150 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4050004940 tx=0x7f4050009e30 comp rx=0 tx=0).stop 2024-04-22T23:34:55.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.402+0000 7f4064ce1700 1 -- 172.21.15.107:0/344376196 shutdown_connections 2024-04-22T23:34:55.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.402+0000 7f4064ce1700 1 --2- 172.21.15.107:0/344376196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40600fcff0 0x7f40600fd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:55.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.402+0000 7f4064ce1700 1 --2- 172.21.15.107:0/344376196 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40600fbd40 0x7f40600fc150 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:55.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.402+0000 7f4064ce1700 1 -- 172.21.15.107:0/344376196 >> 172.21.15.107:0/344376196 conn(0x7f40600f74c0 msgr2=0x7f40600f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:34:55.397 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:34:55 smithi107 ceph-mon[26355]: pgmap v254: 65 pgs: 65 active+clean; 314 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 16 MiB/s rd, 27 MiB/s wr, 816 op/s 2024-04-22T23:34:55.397 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:34:55 smithi107 ceph-mon[26355]: from='client.14636 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:34:55.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.402+0000 7f4064ce1700 1 -- 172.21.15.107:0/344376196 shutdown_connections 2024-04-22T23:34:55.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.402+0000 7f4064ce1700 1 -- 172.21.15.107:0/344376196 wait complete. 2024-04-22T23:34:55.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.402+0000 7f4064ce1700 1 Processor -- start 2024-04-22T23:34:55.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.402+0000 7f4064ce1700 1 -- start start 2024-04-22T23:34:55.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.403+0000 7f4064ce1700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40600fbd40 0x7f406010d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:34:55.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.403+0000 7f4064ce1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40600fcff0 0x7f406010df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:34:55.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.403+0000 7f4064ce1700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406010e520 con 0x7f40600fcff0 2024-04-22T23:34:55.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.403+0000 7f4064ce1700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406010ed20 con 0x7f40600fbd40 2024-04-22T23:34:55.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.403+0000 7f405dd9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40600fcff0 0x7f406010df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:34:55.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.403+0000 7f405e59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40600fbd40 0x7f406010d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:34:55.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.403+0000 7f405dd9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40600fcff0 0x7f406010df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51204/0 (socket says 172.21.15.107:51204) 2024-04-22T23:34:55.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.403+0000 7f405e59c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40600fbd40 0x7f406010d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.107:36672/0 (socket says 172.21.15.107:36672) 2024-04-22T23:34:55.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.403+0000 7f405dd9b700 1 -- 172.21.15.107:0/2150163665 learned_addr learned my addr 172.21.15.107:0/2150163665 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:34:55.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.403+0000 7f405dd9b700 1 -- 172.21.15.107:0/2150163665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40600fbd40 msgr2=0x7f406010d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:34:55.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.403+0000 7f405dd9b700 1 --2- 172.21.15.107:0/2150163665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40600fbd40 0x7f406010d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:55.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.403+0000 7f405dd9b700 1 -- 172.21.15.107:0/2150163665 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4050005040 con 0x7f40600fcff0 2024-04-22T23:34:55.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.403+0000 7f405dd9b700 1 --2- 172.21.15.107:0/2150163665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40600fcff0 0x7f406010df20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4054002a80 tx=0x7f40540046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:34:55.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.403+0000 7f404effd700 1 -- 172.21.15.107:0/2150163665 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4054007cc0 con 0x7f40600fcff0 2024-04-22T23:34:55.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.403+0000 7f404effd700 1 -- 172.21.15.107:0/2150163665 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f405400f040 con 0x7f40600fcff0 2024-04-22T23:34:55.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.403+0000 7f4064ce1700 1 -- 172.21.15.107:0/2150163665 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f406010f000 con 0x7f40600fcff0 2024-04-22T23:34:55.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.404+0000 7f404effd700 1 -- 172.21.15.107:0/2150163665 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f405400dac0 con 0x7f40600fcff0 2024-04-22T23:34:55.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.404+0000 7f4064ce1700 1 -- 172.21.15.107:0/2150163665 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f406010f540 con 0x7f40600fcff0 2024-04-22T23:34:55.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.405+0000 7f4064ce1700 1 -- 172.21.15.107:0/2150163665 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f406005f5f0 con 0x7f40600fcff0 2024-04-22T23:34:55.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.405+0000 7f404effd700 1 -- 172.21.15.107:0/2150163665 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f405400dc20 con 0x7f40600fcff0 2024-04-22T23:34:55.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.405+0000 7f404effd700 1 --2- 172.21.15.107:0/2150163665 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f40480690a0 0x7f404806b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:34:55.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.406+0000 7f405e59c700 1 --2- 172.21.15.107:0/2150163665 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f40480690a0 0x7f404806b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:34:55.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.406+0000 7f404effd700 1 -- 172.21.15.107:0/2150163665 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f4054086c10 con 0x7f40600fcff0 2024-04-22T23:34:55.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.406+0000 7f405e59c700 1 --2- 172.21.15.107:0/2150163665 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f40480690a0 0x7f404806b550 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f405000b3b0 tx=0x7f4050009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:34:55.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.410+0000 7f404effd700 1 -- 172.21.15.107:0/2150163665 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4054056900 con 0x7f40600fcff0 2024-04-22T23:34:55.406 INFO:tasks.workunit.client.1.smithi157.stdout:9/302: mknod d4/df/d52/c54 0 2024-04-22T23:34:55.408 INFO:tasks.workunit.client.1.smithi157.stdout:0/211: link d17/c2a d17/d23/d34/c50 0 2024-04-22T23:34:55.408 INFO:tasks.workunit.client.1.smithi157.stdout:0/212: fsync d17/d2b/f3d 0 2024-04-22T23:34:55.408 INFO:tasks.workunit.client.1.smithi157.stdout:4/346: mkdir d3/d4b/d5d/d6c 0 2024-04-22T23:34:55.410 INFO:tasks.workunit.client.1.smithi157.stdout:8/146: mkdir d2/d4/d30 0 2024-04-22T23:34:55.413 INFO:tasks.workunit.client.1.smithi157.stdout:1/262: unlink d4/d17/l37 0 2024-04-22T23:34:55.416 INFO:tasks.workunit.client.1.smithi157.stdout:2/302: mknod d7/c6a 0 2024-04-22T23:34:55.417 INFO:tasks.workunit.client.1.smithi157.stdout:6/410: mknod d1/c82 0 2024-04-22T23:34:55.424 INFO:tasks.workunit.client.1.smithi157.stdout:9/303: mkdir d4/d15/d55 0 2024-04-22T23:34:55.424 INFO:tasks.workunit.client.1.smithi157.stdout:9/304: truncate d4/d15/f4e 334402 0 2024-04-22T23:34:55.426 INFO:tasks.workunit.client.1.smithi157.stdout:4/347: creat d3/d4b/d5d/d6c/f6d x:0 0 0 2024-04-22T23:34:55.428 INFO:tasks.workunit.client.1.smithi157.stdout:8/147: link d2/d6/de/d11/d13/c22 d2/d6/de/d11/d13/c31 0 2024-04-22T23:34:55.430 INFO:tasks.workunit.client.1.smithi157.stdout:2/303: mkdir d7/d8/d6b 0 2024-04-22T23:34:55.430 INFO:tasks.workunit.client.1.smithi157.stdout:2/304: stat d7/d1f/d26/d4a/d54/l65 0 2024-04-22T23:34:55.430 INFO:tasks.workunit.client.1.smithi157.stdout:2/305: write d7/d8/d30/d23/d27/f67 [166429,64973] 0 2024-04-22T23:34:55.431 INFO:tasks.workunit.client.1.smithi157.stdout:9/305: dread d4/df/f26 [0,4194304] 0 2024-04-22T23:34:55.432 INFO:tasks.workunit.client.1.smithi157.stdout:6/411: rename d1/d2f/d54/d7a/d56 to d1/d2f/d3b/d6c/d83 0 2024-04-22T23:34:55.432 INFO:tasks.workunit.client.1.smithi157.stdout:6/412: readlink d1/d2f/d3b/d6c/l80 0 2024-04-22T23:34:55.432 INFO:tasks.workunit.client.1.smithi157.stdout:6/413: read - d1/d2f/d3b/d49/f7d zero size 2024-04-22T23:34:55.433 INFO:tasks.workunit.client.1.smithi157.stdout:0/213: rmdir d17/d23 39 2024-04-22T23:34:55.435 INFO:tasks.workunit.client.1.smithi157.stdout:4/348: mkdir d3/d4b/d5d/d6e 0 2024-04-22T23:34:55.435 INFO:tasks.workunit.client.1.smithi157.stdout:4/349: truncate d3/d11/d1d/f24 5106044 0 2024-04-22T23:34:55.435 INFO:tasks.workunit.client.1.smithi157.stdout:4/350: write d3/d4b/f5 [4693800,50285] 0 2024-04-22T23:34:55.435 INFO:tasks.workunit.client.1.smithi157.stdout:4/351: write d3/d11/d30/d28/d31/f69 [532281,130627] 0 2024-04-22T23:34:55.436 INFO:tasks.workunit.client.1.smithi157.stdout:1/263: unlink d4/l20 0 2024-04-22T23:34:55.436 INFO:tasks.workunit.client.1.smithi157.stdout:4/352: dread d3/d11/d30/d28/f3c [0,4194304] 0 2024-04-22T23:34:55.437 INFO:tasks.workunit.client.1.smithi157.stdout:4/353: write d3/d11/d30/d28/d31/f6a [287697,120224] 0 2024-04-22T23:34:55.437 INFO:tasks.workunit.client.1.smithi157.stdout:2/306: creat d7/d8/d25/d55/f6c x:0 0 0 2024-04-22T23:34:55.438 INFO:tasks.workunit.client.1.smithi157.stdout:8/148: read d2/d4/f10 [2387066,129153] 0 2024-04-22T23:34:55.438 INFO:tasks.workunit.client.1.smithi157.stdout:8/149: chown d2/d6/de/fd 139 1 2024-04-22T23:34:55.438 INFO:tasks.workunit.client.1.smithi157.stdout:8/150: stat d2 0 2024-04-22T23:34:55.439 INFO:tasks.workunit.client.1.smithi157.stdout:6/414: mknod d1/d17/c84 0 2024-04-22T23:34:55.440 INFO:tasks.workunit.client.1.smithi157.stdout:0/214: creat d17/d23/f51 x:0 0 0 2024-04-22T23:34:55.441 INFO:tasks.workunit.client.1.smithi157.stdout:3/192: sync 2024-04-22T23:34:55.441 INFO:tasks.workunit.client.1.smithi157.stdout:7/180: sync 2024-04-22T23:34:55.441 INFO:tasks.workunit.client.1.smithi157.stdout:5/294: sync 2024-04-22T23:34:55.441 INFO:tasks.workunit.client.1.smithi157.stdout:7/181: read d0/d9/d29/d33/f34 [433278,31409] 0 2024-04-22T23:34:55.441 INFO:tasks.workunit.client.1.smithi157.stdout:7/182: truncate d0/d9/db/df/f2b 4861376 0 2024-04-22T23:34:55.442 INFO:tasks.workunit.client.1.smithi157.stdout:1/264: creat d4/d17/d1d/d33/d41/d54/f5b x:0 0 0 2024-04-22T23:34:55.442 INFO:tasks.workunit.client.1.smithi157.stdout:4/354: creat d3/d11/d30/d28/d31/f6f x:0 0 0 2024-04-22T23:34:55.443 INFO:tasks.workunit.client.1.smithi157.stdout:2/307: symlink d7/d8/dd/d4d/l6d 0 2024-04-22T23:34:55.443 INFO:tasks.workunit.client.1.smithi157.stdout:2/308: stat d7/d1f/d26/d4a/f53 0 2024-04-22T23:34:55.443 INFO:tasks.workunit.client.1.smithi157.stdout:2/309: fdatasync d7/d8/f35 0 2024-04-22T23:34:55.443 INFO:tasks.workunit.client.1.smithi157.stdout:8/151: creat d2/f32 x:0 0 0 2024-04-22T23:34:55.444 INFO:tasks.workunit.client.1.smithi157.stdout:9/306: link d4/df/ce d4/d15/d55/c56 0 2024-04-22T23:34:55.445 INFO:tasks.workunit.client.1.smithi157.stdout:6/415: rename d1/d36/c41 to d1/d2f/d3b/d49/c85 0 2024-04-22T23:34:55.445 INFO:tasks.workunit.client.1.smithi157.stdout:6/416: fsync d1/d2f/f4a 0 2024-04-22T23:34:55.445 INFO:tasks.workunit.client.1.smithi157.stdout:6/417: truncate d1/d2f/d54/f6b 744410 0 2024-04-22T23:34:55.446 INFO:tasks.workunit.client.1.smithi157.stdout:6/418: truncate d1/d36/f47 1248791 0 2024-04-22T23:34:55.447 INFO:tasks.workunit.client.1.smithi157.stdout:3/193: creat d0/d7/f3a x:0 0 0 2024-04-22T23:34:55.447 INFO:tasks.workunit.client.1.smithi157.stdout:7/183: mkdir d0/d9/d29/d33/d37 0 2024-04-22T23:34:55.447 INFO:tasks.workunit.client.1.smithi157.stdout:3/194: readlink d0/d3/d13/d21/l2a 0 2024-04-22T23:34:55.447 INFO:tasks.workunit.client.1.smithi157.stdout:3/195: chown d0/d3/fe 14867 1 2024-04-22T23:34:55.447 INFO:tasks.workunit.client.1.smithi157.stdout:3/196: creat d0/d3/f3b x:0 0 0 2024-04-22T23:34:55.447 INFO:tasks.workunit.client.1.smithi157.stdout:1/265: fsync d4/f1b 0 2024-04-22T23:34:55.448 INFO:tasks.workunit.client.1.smithi157.stdout:1/266: dread d4/db/f24 [0,4194304] 0 2024-04-22T23:34:55.448 INFO:tasks.workunit.client.1.smithi157.stdout:2/310: unlink d7/d8/d30/d23/f2c 0 2024-04-22T23:34:55.448 INFO:tasks.workunit.client.1.smithi157.stdout:2/311: write d7/d8/d25/f50 [1379668,8614] 0 2024-04-22T23:34:55.449 INFO:tasks.workunit.client.1.smithi157.stdout:2/312: stat d7/d8/d30/l5d 0 2024-04-22T23:34:55.449 INFO:tasks.workunit.client.1.smithi157.stdout:2/313: stat d7/f13 0 2024-04-22T23:34:55.450 INFO:tasks.workunit.client.1.smithi157.stdout:8/152: mkdir d2/d6/d21/d33 0 2024-04-22T23:34:55.450 INFO:tasks.workunit.client.1.smithi157.stdout:8/153: chown d2/d4/l7 594722946 1 2024-04-22T23:34:55.450 INFO:tasks.workunit.client.1.smithi157.stdout:8/154: chown d2/d6/de/l1d 3852 1 2024-04-22T23:34:55.454 INFO:tasks.workunit.client.1.smithi157.stdout:0/215: rename d17/d23/d34/d37 to d17/d2b/d52 0 2024-04-22T23:34:55.454 INFO:tasks.workunit.client.1.smithi157.stdout:0/216: write d17/d2b/f2c [149691,49893] 0 2024-04-22T23:34:55.457 INFO:tasks.workunit.client.1.smithi157.stdout:6/419: mkdir d1/d2f/d54/d86 0 2024-04-22T23:34:55.458 INFO:tasks.workunit.client.1.smithi157.stdout:1/267: mknod d4/d17/d1d/d33/d41/c5c 0 2024-04-22T23:34:55.458 INFO:tasks.workunit.client.1.smithi157.stdout:8/155: mkdir d2/d34 0 2024-04-22T23:34:55.460 INFO:tasks.workunit.client.1.smithi157.stdout:6/420: dread d1/d2f/d3b/f2a [0,4194304] 0 2024-04-22T23:34:55.461 INFO:tasks.workunit.client.1.smithi157.stdout:4/355: rename d3/d4b/ca to d3/d11/d30/d36/d62/d4c/c70 0 2024-04-22T23:34:55.461 INFO:tasks.workunit.client.1.smithi157.stdout:7/184: rename d0/d9/d29 to d0/d9/d29/d38 22 2024-04-22T23:34:55.461 INFO:tasks.workunit.client.1.smithi157.stdout:7/185: creat d0/d9/db/f39 x:0 0 0 2024-04-22T23:34:55.461 INFO:tasks.workunit.client.1.smithi157.stdout:4/356: creat d3/d11/d30/f71 x:0 0 0 2024-04-22T23:34:55.462 INFO:tasks.workunit.client.1.smithi157.stdout:4/357: chown d3/d11/d30/d28/d31/l6b 107044440 1 2024-04-22T23:34:55.463 INFO:tasks.workunit.client.1.smithi157.stdout:0/217: creat d17/d2b/d52/f53 x:0 0 0 2024-04-22T23:34:55.463 INFO:tasks.workunit.client.1.smithi157.stdout:0/218: fdatasync fe 0 2024-04-22T23:34:55.463 INFO:tasks.workunit.client.1.smithi157.stdout:6/421: dread d1/d17/d5d/f64 [0,4194304] 0 2024-04-22T23:34:55.463 INFO:tasks.workunit.client.1.smithi157.stdout:6/422: write d1/d36/f57 [1028895,114699] 0 2024-04-22T23:34:55.463 INFO:tasks.workunit.client.1.smithi157.stdout:6/423: creat d1/d2f/d54/f87 x:0 0 0 2024-04-22T23:34:55.464 INFO:tasks.workunit.client.1.smithi157.stdout:5/295: dwrite d1/d5/d7/f2e [0,4194304] 0 2024-04-22T23:34:55.471 INFO:tasks.workunit.client.1.smithi157.stdout:7/186: rename d0/d9/d29/d33/f34 to d0/d9/db/d35/f3a 0 2024-04-22T23:34:55.471 INFO:tasks.workunit.client.1.smithi157.stdout:7/187: readlink d0/d9/l24 0 2024-04-22T23:34:55.471 INFO:tasks.workunit.client.1.smithi157.stdout:7/188: creat d0/d9/d29/f3b x:0 0 0 2024-04-22T23:34:55.471 INFO:tasks.workunit.client.1.smithi157.stdout:7/189: creat d0/d9/f3c x:0 0 0 2024-04-22T23:34:55.472 INFO:tasks.workunit.client.1.smithi157.stdout:4/358: creat d3/d11/d30/d36/d62/d40/f72 x:0 0 0 2024-04-22T23:34:55.472 INFO:tasks.workunit.client.1.smithi157.stdout:9/307: dwrite d4/df/fb [0,4194304] 0 2024-04-22T23:34:55.472 INFO:tasks.workunit.client.1.smithi157.stdout:9/308: truncate d4/d1d/f44 5125707 0 2024-04-22T23:34:55.473 INFO:tasks.workunit.client.1.smithi157.stdout:3/197: dwrite d0/d3/d13/f2b [0,4194304] 0 2024-04-22T23:34:55.473 INFO:tasks.workunit.client.1.smithi157.stdout:0/219: symlink d17/d38/l54 0 2024-04-22T23:34:55.477 INFO:tasks.workunit.client.1.smithi157.stdout:6/424: link d1/d2f/d54/d7a/f61 d1/f88 0 2024-04-22T23:34:55.479 INFO:tasks.workunit.client.1.smithi157.stdout:5/296: symlink d1/d5/l71 0 2024-04-22T23:34:55.487 INFO:tasks.workunit.client.1.smithi157.stdout:7/190: dwrite d0/f8 [0,4194304] 0 2024-04-22T23:34:55.491 INFO:tasks.workunit.client.1.smithi157.stdout:4/359: mkdir d3/d11/d1d/d73 0 2024-04-22T23:34:55.494 INFO:tasks.workunit.client.1.smithi157.stdout:5/297: dwrite d1/d5/f14 [0,4194304] 0 2024-04-22T23:34:55.494 INFO:tasks.workunit.client.1.smithi157.stdout:5/298: creat d1/d5/d7/f72 x:0 0 0 2024-04-22T23:34:55.494 INFO:tasks.workunit.client.1.smithi157.stdout:5/299: dread - d1/d5/dc/d43/f44 zero size 2024-04-22T23:34:55.494 INFO:tasks.workunit.client.1.smithi157.stdout:0/220: mknod d17/d2b/c55 0 2024-04-22T23:34:55.496 INFO:tasks.workunit.client.1.smithi157.stdout:7/191: link d0/d9/l11 d0/d9/db/d35/l3d 0 2024-04-22T23:34:55.499 INFO:tasks.workunit.client.1.smithi157.stdout:4/360: link d3/d11/c12 d3/d11/d1d/c74 0 2024-04-22T23:34:55.499 INFO:tasks.workunit.client.1.smithi157.stdout:4/361: write d3/d4b/d5d/d6c/f6d [570475,81] 0 2024-04-22T23:34:55.501 INFO:tasks.workunit.client.1.smithi157.stdout:3/198: getdents d0 0 2024-04-22T23:34:55.502 INFO:tasks.workunit.client.1.smithi157.stdout:1/268: rmdir d4/d17 39 2024-04-22T23:34:55.505 INFO:tasks.workunit.client.1.smithi157.stdout:0/221: stat d17/c35 0 2024-04-22T23:34:55.505 INFO:tasks.workunit.client.1.smithi157.stdout:0/222: truncate d17/d2b/f47 636681 0 2024-04-22T23:34:55.509 INFO:tasks.workunit.client.1.smithi157.stdout:4/362: mkdir d3/d4b/d5d/d6c/d75 0 2024-04-22T23:34:55.511 INFO:tasks.workunit.client.1.smithi157.stdout:3/199: mknod d0/d7/c3c 0 2024-04-22T23:34:55.512 INFO:tasks.workunit.client.1.smithi157.stdout:4/363: truncate d3/d4b/f16 1770681 0 2024-04-22T23:34:55.515 INFO:tasks.workunit.client.1.smithi157.stdout:9/309: rename d4/d1d to d4/df/d57 0 2024-04-22T23:34:55.515 INFO:tasks.workunit.client.1.smithi157.stdout:9/310: chown d4/d15/d23/d3f/l41 55595143 1 2024-04-22T23:34:55.516 INFO:tasks.workunit.client.1.smithi157.stdout:3/200: mkdir d0/d3/d34/d3d 0 2024-04-22T23:34:55.516 INFO:tasks.workunit.client.1.smithi157.stdout:0/223: write d17/f22 [1852912,54281] 0 2024-04-22T23:34:55.518 INFO:tasks.workunit.client.1.smithi157.stdout:6/425: rename d1/f5 to d1/d2f/d3b/d6c/f89 0 2024-04-22T23:34:55.518 INFO:tasks.workunit.client.1.smithi157.stdout:6/426: readlink d1/d2f/d54/d5c/l6a 0 2024-04-22T23:34:55.520 INFO:tasks.workunit.client.1.smithi157.stdout:9/311: creat d4/d15/d55/f58 x:0 0 0 2024-04-22T23:34:55.522 INFO:tasks.workunit.client.1.smithi157.stdout:6/427: mknod d1/d2f/d3b/d39/d4e/c8a 0 2024-04-22T23:34:55.523 INFO:tasks.workunit.client.1.smithi157.stdout:4/364: write d3/d11/d30/d36/d62/f3d [887741,28994] 0 2024-04-22T23:34:55.527 INFO:tasks.workunit.client.1.smithi157.stdout:5/300: rename d1/d27/d29/c6d to d1/d1f/c73 0 2024-04-22T23:34:55.527 INFO:tasks.workunit.client.1.smithi157.stdout:5/301: creat d1/d5/d7/d2f/d45/d54/f74 x:0 0 0 2024-04-22T23:34:55.527 INFO:tasks.workunit.client.1.smithi157.stdout:9/312: mknod d4/d15/d23/d3f/c59 0 2024-04-22T23:34:55.527 INFO:tasks.workunit.client.1.smithi157.stdout:3/201: dread d0/f27 [0,4194304] 0 2024-04-22T23:34:55.527 INFO:tasks.workunit.client.1.smithi157.stdout:1/269: dwrite d4/db/f1e [0,4194304] 0 2024-04-22T23:34:55.528 INFO:tasks.workunit.client.1.smithi157.stdout:6/428: symlink d1/d2f/d3b/d6c/d83/l8b 0 2024-04-22T23:34:55.528 INFO:tasks.workunit.client.1.smithi157.stdout:4/365: rename d3/c29 to d3/d11/d30/d36/d62/d40/c76 0 2024-04-22T23:34:55.528 INFO:tasks.workunit.client.1.smithi157.stdout:4/366: write d3/d11/d1d/f41 [8774623,11844] 0 2024-04-22T23:34:55.528 INFO:tasks.workunit.client.1.smithi157.stdout:4/367: chown d3/d11/d1d/f24 27 1 2024-04-22T23:34:55.530 INFO:tasks.workunit.client.1.smithi157.stdout:5/302: link d1/d5/l71 d1/d5/dc/d43/d4f/d60/l75 0 2024-04-22T23:34:55.530 INFO:tasks.workunit.client.1.smithi157.stdout:9/313: mknod d4/d2d/c5a 0 2024-04-22T23:34:55.531 INFO:tasks.workunit.client.1.smithi157.stdout:1/270: write d4/f26 [2150774,118308] 0 2024-04-22T23:34:55.533 INFO:tasks.workunit.client.1.smithi157.stdout:6/429: mkdir d1/d2f/d3b/d39/d4e/d74/d8c 0 2024-04-22T23:34:55.533 INFO:tasks.workunit.client.1.smithi157.stdout:3/202: write d0/d3/f1d [2658800,12979] 0 2024-04-22T23:34:55.534 INFO:tasks.workunit.client.1.smithi157.stdout:4/368: link d3/d11/d30/d28/d31/c32 d3/d11/d30/d28/d31/c77 0 2024-04-22T23:34:55.535 INFO:tasks.workunit.client.1.smithi157.stdout:5/303: truncate d1/f3f 175799 0 2024-04-22T23:34:55.535 INFO:tasks.workunit.client.1.smithi157.stdout:5/304: write d1/d27/d29/d4b/f63 [923867,76458] 0 2024-04-22T23:34:55.537 INFO:tasks.workunit.client.1.smithi157.stdout:9/314: symlink d4/d15/d55/l5b 0 2024-04-22T23:34:55.538 INFO:tasks.workunit.client.1.smithi157.stdout:6/430: mknod d1/d17/d5d/d76/c8d 0 2024-04-22T23:34:55.540 INFO:tasks.workunit.client.1.smithi157.stdout:1/271: dread d4/f23 [0,4194304] 0 2024-04-22T23:34:55.542 INFO:tasks.workunit.client.1.smithi157.stdout:3/203: mknod d0/d3/d13/c3e 0 2024-04-22T23:34:55.544 INFO:tasks.workunit.client.1.smithi157.stdout:5/305: mknod d1/d27/d29/c76 0 2024-04-22T23:34:55.544 INFO:tasks.workunit.client.1.smithi157.stdout:4/369: dread d3/d11/d30/d36/d62/f67 [0,4194304] 0 2024-04-22T23:34:55.545 INFO:tasks.workunit.client.1.smithi157.stdout:3/204: read d0/f11 [2732497,51977] 0 2024-04-22T23:34:55.548 INFO:tasks.workunit.client.1.smithi157.stdout:6/431: getdents d1/d2f/d3b/d49 0 2024-04-22T23:34:55.552 INFO:tasks.workunit.client.1.smithi157.stdout:9/315: mknod d4/df/c5c 0 2024-04-22T23:34:55.553 INFO:tasks.workunit.client.1.smithi157.stdout:3/205: dread d0/d3/f6 [0,4194304] 0 2024-04-22T23:34:55.555 INFO:tasks.workunit.client.1.smithi157.stdout:9/316: dread d4/df/d57/f44 [4194304,4194304] 0 2024-04-22T23:34:55.555 INFO:tasks.workunit.client.1.smithi157.stdout:1/272: dread d4/f25 [4194304,4194304] 0 2024-04-22T23:34:55.562 INFO:tasks.workunit.client.1.smithi157.stdout:1/273: dread d4/f18 [0,4194304] 0 2024-04-22T23:34:55.562 INFO:tasks.workunit.client.1.smithi157.stdout:1/274: fdatasync d4/d17/d34/d3b/d50/f56 0 2024-04-22T23:34:55.562 INFO:tasks.workunit.client.1.smithi157.stdout:1/275: write d4/db/f14 [5097664,69984] 0 2024-04-22T23:34:55.564 INFO:tasks.workunit.client.1.smithi157.stdout:6/432: dwrite d1/d2f/d3b/d6c/d83/f71 [0,4194304] 0 2024-04-22T23:34:55.571 INFO:tasks.workunit.client.1.smithi157.stdout:5/306: mkdir d1/d5/dc/d5d/d77 0 2024-04-22T23:34:55.571 INFO:tasks.workunit.client.1.smithi157.stdout:5/307: chown d1/d5/d7/d2f/d45/f48 116 1 2024-04-22T23:34:55.571 INFO:tasks.workunit.client.1.smithi157.stdout:4/370: getdents d3/d4b/d5d/d6c 0 2024-04-22T23:34:55.573 INFO:tasks.workunit.client.1.smithi157.stdout:3/206: creat d0/d3/d13/d21/f3f x:0 0 0 2024-04-22T23:34:55.573 INFO:tasks.workunit.client.1.smithi157.stdout:3/207: dread - d0/d3/d13/d21/f3f zero size 2024-04-22T23:34:55.574 INFO:tasks.workunit.client.1.smithi157.stdout:9/317: getdents d4/d2d 0 2024-04-22T23:34:55.574 INFO:tasks.workunit.client.1.smithi157.stdout:1/276: rmdir d4/d17/d34/d3b/d52 39 2024-04-22T23:34:55.577 INFO:tasks.workunit.client.1.smithi157.stdout:6/433: getdents d1/d2f/d3b/d39/d4e/d74 0 2024-04-22T23:34:55.577 INFO:tasks.workunit.client.1.smithi157.stdout:6/434: readlink d1/d36/l3c 0 2024-04-22T23:34:55.577 INFO:tasks.workunit.client.1.smithi157.stdout:6/435: truncate d1/d2f/d3b/f3e 872226 0 2024-04-22T23:34:55.577 INFO:tasks.workunit.client.1.smithi157.stdout:5/308: mknod d1/d5/dc/d43/d5e/c78 0 2024-04-22T23:34:55.578 INFO:tasks.workunit.client.1.smithi157.stdout:5/309: chown d1/d1f 342253 1 2024-04-22T23:34:55.578 INFO:tasks.workunit.client.1.smithi157.stdout:4/371: link d3/d11/d30/c35 d3/d11/d30/d28/c78 0 2024-04-22T23:34:55.579 INFO:tasks.workunit.client.1.smithi157.stdout:3/208: creat d0/f40 x:0 0 0 2024-04-22T23:34:55.580 INFO:tasks.workunit.client.1.smithi157.stdout:3/209: readlink d0/d3/l12 0 2024-04-22T23:34:55.580 INFO:tasks.workunit.client.1.smithi157.stdout:3/210: chown d0/d3/d13/d21/l2a 930102 1 2024-04-22T23:34:55.580 INFO:tasks.workunit.client.1.smithi157.stdout:9/318: symlink d4/d15/d23/d27/d47/l5d 0 2024-04-22T23:34:55.581 INFO:tasks.workunit.client.1.smithi157.stdout:5/310: dread d1/d5/d7/d10/f49 [0,4194304] 0 2024-04-22T23:34:55.581 INFO:tasks.workunit.client.1.smithi157.stdout:1/277: symlink d4/d17/d34/d3b/d52/d58/l5d 0 2024-04-22T23:34:55.583 INFO:tasks.workunit.client.1.smithi157.stdout:6/436: symlink d1/d2f/d54/l8e 0 2024-04-22T23:34:55.583 INFO:tasks.workunit.client.1.smithi157.stdout:4/372: truncate d3/d4b/f5 2928447 0 2024-04-22T23:34:55.583 INFO:tasks.workunit.client.1.smithi157.stdout:4/373: read - d3/d11/d34/f48 zero size 2024-04-22T23:34:55.584 INFO:tasks.workunit.client.1.smithi157.stdout:3/211: truncate d0/d3/f6 4245658 0 2024-04-22T23:34:55.586 INFO:tasks.workunit.client.1.smithi157.stdout:5/311: truncate d1/d27/d29/f2a 1340724 0 2024-04-22T23:34:55.588 INFO:tasks.workunit.client.1.smithi157.stdout:4/374: rename d3/d4b/f1c to d3/d5c/f79 0 2024-04-22T23:34:55.588 INFO:tasks.workunit.client.1.smithi157.stdout:4/375: readlink d3/d11/l63 0 2024-04-22T23:34:55.589 INFO:tasks.workunit.client.1.smithi157.stdout:5/312: creat d1/d5/d7/d2f/f79 x:0 0 0 2024-04-22T23:34:55.589 INFO:tasks.workunit.client.1.smithi157.stdout:5/313: chown d1/d27/d29/d4b/f55 29486 1 2024-04-22T23:34:55.592 INFO:tasks.workunit.client.1.smithi157.stdout:4/376: mkdir d3/d4b/d7a 0 2024-04-22T23:34:55.592 INFO:tasks.workunit.client.1.smithi157.stdout:4/377: write d3/d11/d30/d36/d62/f3d [1607778,58573] 0 2024-04-22T23:34:55.592 INFO:tasks.workunit.client.1.smithi157.stdout:4/378: write d3/f58 [1431845,122543] 0 2024-04-22T23:34:55.595 INFO:tasks.workunit.client.1.smithi157.stdout:9/319: dwrite d4/d15/f4c [0,4194304] 0 2024-04-22T23:34:55.595 INFO:tasks.workunit.client.1.smithi157.stdout:5/314: mkdir d1/d5/dc/d43/d4f/d60/d7a 0 2024-04-22T23:34:55.599 INFO:tasks.workunit.client.1.smithi157.stdout:9/320: symlink d4/d15/d23/l5e 0 2024-04-22T23:34:55.601 INFO:tasks.workunit.client.1.smithi157.stdout:5/315: chown d1/d5/dc/f1d 99190 1 2024-04-22T23:34:55.601 INFO:tasks.workunit.client.1.smithi157.stdout:5/316: creat d1/d5/d7/d2f/d45/d54/f7b x:0 0 0 2024-04-22T23:34:55.601 INFO:tasks.workunit.client.1.smithi157.stdout:5/317: readlink d1/d27/d29/l33 0 2024-04-22T23:34:55.605 INFO:tasks.workunit.client.1.smithi157.stdout:5/318: link d1/d5/l71 d1/d5/d7/d2f/d45/l7c 0 2024-04-22T23:34:55.605 INFO:tasks.workunit.client.1.smithi157.stdout:5/319: chown d1/d5/d7/d2f/d45/l4c 28 1 2024-04-22T23:34:55.605 INFO:tasks.workunit.client.1.smithi157.stdout:5/320: chown d1/d5/f4e 0 1 2024-04-22T23:34:55.605 INFO:tasks.workunit.client.1.smithi157.stdout:5/321: chown d1/d1f/l2b 1507 1 2024-04-22T23:34:55.605 INFO:tasks.workunit.client.1.smithi157.stdout:5/322: chown d1/d1f/c2d 121723425 1 2024-04-22T23:34:55.605 INFO:tasks.workunit.client.1.smithi157.stdout:5/323: read - d1/d5/d7/d2f/d45/d54/d59/f6a zero size 2024-04-22T23:34:55.605 INFO:tasks.workunit.client.1.smithi157.stdout:5/324: chown d1/d27/d29/d4b/f5c 1002 1 2024-04-22T23:34:55.606 INFO:tasks.workunit.client.1.smithi157.stdout:5/325: mknod d1/d5/dc/d43/d4f/c7d 0 2024-04-22T23:34:55.606 INFO:tasks.workunit.client.1.smithi157.stdout:5/326: write d1/d5/d7/d2f/d45/d54/d59/f6a [464281,114286] 0 2024-04-22T23:34:55.606 INFO:tasks.workunit.client.1.smithi157.stdout:5/327: write d1/d27/d29/d4b/f63 [1649466,89913] 0 2024-04-22T23:34:55.607 INFO:tasks.workunit.client.1.smithi157.stdout:5/328: symlink d1/d1f/d22/l7e 0 2024-04-22T23:34:55.607 INFO:tasks.workunit.client.1.smithi157.stdout:5/329: write d1/d5/d7/f56 [908507,75305] 0 2024-04-22T23:34:55.608 INFO:tasks.workunit.client.1.smithi157.stdout:5/330: mkdir d1/d5/dc/d43/d4f/d7f 0 2024-04-22T23:34:55.609 INFO:tasks.workunit.client.1.smithi157.stdout:5/331: creat d1/d5/dc/d43/d5e/f80 x:0 0 0 2024-04-22T23:34:55.609 INFO:tasks.workunit.client.1.smithi157.stdout:5/332: fsync d1/d5/f46 0 2024-04-22T23:34:55.610 INFO:tasks.workunit.client.1.smithi157.stdout:3/212: dread d0/d3/f1e [0,4194304] 0 2024-04-22T23:34:55.610 INFO:tasks.workunit.client.1.smithi157.stdout:3/213: creat d0/f41 x:0 0 0 2024-04-22T23:34:55.610 INFO:tasks.workunit.client.1.smithi157.stdout:3/214: chown d0/d3/d34 480 1 2024-04-22T23:34:55.611 INFO:tasks.workunit.client.1.smithi157.stdout:9/321: read d4/df/d57/f44 [528221,8005] 0 2024-04-22T23:34:55.611 INFO:tasks.workunit.client.1.smithi157.stdout:9/322: chown d4/fc 27509 1 2024-04-22T23:34:55.611 INFO:tasks.workunit.client.1.smithi157.stdout:9/323: chown d4/d15/d55/f58 1 1 2024-04-22T23:34:55.611 INFO:tasks.workunit.client.1.smithi157.stdout:5/333: rename d1/d1f/l2b to d1/d27/d29/d4b/l81 0 2024-04-22T23:34:55.613 INFO:tasks.workunit.client.1.smithi157.stdout:9/324: mknod d4/d15/d23/d27/c5f 0 2024-04-22T23:34:55.615 INFO:tasks.workunit.client.1.smithi157.stdout:1/278: dwrite d4/db/fd [0,4194304] 0 2024-04-22T23:34:55.615 INFO:tasks.workunit.client.1.smithi157.stdout:5/334: dread d1/d1f/d22/f35 [0,4194304] 0 2024-04-22T23:34:55.615 INFO:tasks.workunit.client.1.smithi157.stdout:6/437: dwrite d1/f22 [0,4194304] 0 2024-04-22T23:34:55.616 INFO:tasks.workunit.client.1.smithi157.stdout:4/379: dwrite d3/d11/d30/d28/d31/f6a [0,4194304] 0 2024-04-22T23:34:55.617 INFO:tasks.workunit.client.1.smithi157.stdout:4/380: fdatasync d3/d11/d30/d36/d62/d4c/f61 0 2024-04-22T23:34:55.617 INFO:tasks.workunit.client.1.smithi157.stdout:4/381: stat d3/d4b/f3e 0 2024-04-22T23:34:55.618 INFO:tasks.workunit.client.1.smithi157.stdout:3/215: write d0/d3/d13/f2b [515114,25966] 0 2024-04-22T23:34:55.619 INFO:tasks.workunit.client.1.smithi157.stdout:1/279: creat d4/d17/d1d/d33/d41/f5e x:0 0 0 2024-04-22T23:34:55.622 INFO:tasks.workunit.client.1.smithi157.stdout:5/335: rename d1/d1f/d22/f35 to d1/d5/d7/d2f/d45/d54/f82 0 2024-04-22T23:34:55.624 INFO:tasks.workunit.client.1.smithi157.stdout:6/438: rename d1/d2f/d3b/c1c to d1/d2f/d3b/c8f 0 2024-04-22T23:34:55.628 INFO:tasks.workunit.client.1.smithi157.stdout:9/325: truncate d4/d15/f4c 139139 0 2024-04-22T23:34:55.628 INFO:tasks.workunit.client.1.smithi157.stdout:9/326: chown d4/d15/d23/l5e 1442331 1 2024-04-22T23:34:55.636 INFO:tasks.workunit.client.1.smithi157.stdout:6/439: link d1/c82 d1/d2f/d3b/d39/c90 0 2024-04-22T23:34:55.637 INFO:tasks.workunit.client.1.smithi157.stdout:6/440: chown d1/d2f/d3b/d49/c67 3 1 2024-04-22T23:34:55.637 INFO:tasks.workunit.client.1.smithi157.stdout:3/216: getdents d0/d3/d13/d21 0 2024-04-22T23:34:55.640 INFO:tasks.workunit.client.1.smithi157.stdout:1/280: mkdir d4/d5f 0 2024-04-22T23:34:55.640 INFO:tasks.workunit.client.1.smithi157.stdout:1/281: readlink d4/d17/d34/l53 0 2024-04-22T23:34:55.641 INFO:tasks.workunit.client.1.smithi157.stdout:9/327: getdents d4/d15 0 2024-04-22T23:34:55.641 INFO:tasks.workunit.client.1.smithi157.stdout:9/328: truncate d4/df/f12 8772466 0 2024-04-22T23:34:55.641 INFO:tasks.workunit.client.1.smithi157.stdout:9/329: write d4/fc [510489,78051] 0 2024-04-22T23:34:55.642 INFO:tasks.workunit.client.1.smithi157.stdout:6/441: dread d1/d2f/d3b/d6c/d83/f59 [0,4194304] 0 2024-04-22T23:34:55.643 INFO:tasks.workunit.client.1.smithi157.stdout:6/442: dread d1/d2f/d3b/f2e [0,4194304] 0 2024-04-22T23:34:55.647 INFO:tasks.workunit.client.1.smithi157.stdout:5/336: dwrite d1/d5/f9 [0,4194304] 0 2024-04-22T23:34:55.651 INFO:tasks.workunit.client.1.smithi157.stdout:9/330: read d4/f50 [1235292,8980] 0 2024-04-22T23:34:55.652 INFO:tasks.workunit.client.1.smithi157.stdout:5/337: mknod d1/d5/dc/c83 0 2024-04-22T23:34:55.653 INFO:tasks.workunit.client.1.smithi157.stdout:9/331: symlink d4/d2d/l60 0 2024-04-22T23:34:55.656 INFO:tasks.workunit.client.1.smithi157.stdout:1/282: getdents d4 0 2024-04-22T23:34:55.656 INFO:tasks.workunit.client.1.smithi157.stdout:3/217: dwrite d0/f1 [4194304,4194304] 0 2024-04-22T23:34:55.657 INFO:tasks.workunit.client.1.smithi157.stdout:9/332: mkdir d4/df/d61 0 2024-04-22T23:34:55.659 INFO:tasks.workunit.client.1.smithi157.stdout:3/218: rename d0/d3/d34/d39/f25 to d0/d3/d34/d3d/f42 0 2024-04-22T23:34:55.659 INFO:tasks.workunit.client.1.smithi157.stdout:3/219: readlink d0/d3/d13/l2e 0 2024-04-22T23:34:55.663 INFO:tasks.workunit.client.1.smithi157.stdout:1/283: dread d4/f16 [0,4194304] 0 2024-04-22T23:34:55.663 INFO:tasks.workunit.client.1.smithi157.stdout:6/443: dwrite d1/d17/f28 [0,4194304] 0 2024-04-22T23:34:55.667 INFO:tasks.workunit.client.1.smithi157.stdout:9/333: unlink d4/d15/d23/d3f/l41 0 2024-04-22T23:34:55.668 INFO:tasks.workunit.client.1.smithi157.stdout:3/220: mknod d0/d3/d34/c43 0 2024-04-22T23:34:55.668 INFO:tasks.workunit.client.1.smithi157.stdout:1/284: truncate d4/db/f13 150311 0 2024-04-22T23:34:55.669 INFO:tasks.workunit.client.1.smithi157.stdout:6/444: link d1/d2f/d3b/d39/d4e/c78 d1/d2f/d3b/d6c/d83/c91 0 2024-04-22T23:34:55.670 INFO:tasks.workunit.client.1.smithi157.stdout:5/338: dwrite d1/d5/d7/d2f/d45/d54/f7b [0,4194304] 0 2024-04-22T23:34:55.670 INFO:tasks.workunit.client.1.smithi157.stdout:1/285: rmdir d4/db/d3d 39 2024-04-22T23:34:55.671 INFO:tasks.workunit.client.1.smithi157.stdout:6/445: symlink d1/d36/l92 0 2024-04-22T23:34:55.671 INFO:tasks.workunit.client.1.smithi157.stdout:6/446: fsync d1/f3 0 2024-04-22T23:34:55.673 INFO:tasks.workunit.client.1.smithi157.stdout:5/339: link d1/d5/c6e d1/d5/d7/d2f/d51/c84 0 2024-04-22T23:34:55.674 INFO:tasks.workunit.client.1.smithi157.stdout:5/340: chown d1/d1f/c2d 72708 1 2024-04-22T23:34:55.674 INFO:tasks.workunit.client.1.smithi157.stdout:9/334: dread d4/df/d57/f44 [0,4194304] 0 2024-04-22T23:34:55.676 INFO:tasks.workunit.client.1.smithi157.stdout:6/447: rename d1/d17/d5d/d76 to d1/d2f/d3b/d6c/d83/d93 0 2024-04-22T23:34:55.677 INFO:tasks.workunit.client.1.smithi157.stdout:6/448: chown d1/f12 917446 1 2024-04-22T23:34:55.677 INFO:tasks.workunit.client.1.smithi157.stdout:6/449: truncate d1/d2f/d54/f6b 1520144 0 2024-04-22T23:34:55.678 INFO:tasks.workunit.client.1.smithi157.stdout:5/341: unlink d1/d5/d7/d2f/c61 0 2024-04-22T23:34:55.678 INFO:tasks.workunit.client.1.smithi157.stdout:9/335: stat d4/d15/d23/d3f/l48 0 2024-04-22T23:34:55.681 INFO:tasks.workunit.client.1.smithi157.stdout:1/286: symlink d4/d17/d34/d3b/l60 0 2024-04-22T23:34:55.681 INFO:tasks.workunit.client.1.smithi157.stdout:1/287: readlink d4/d17/d1d/l27 0 2024-04-22T23:34:55.682 INFO:tasks.workunit.client.1.smithi157.stdout:6/450: creat d1/d17/f94 x:0 0 0 2024-04-22T23:34:55.684 INFO:tasks.workunit.client.1.smithi157.stdout:5/342: rename d1/d27/d29/d4b/f55 to d1/d5/dc/d43/d4f/d60/f85 0 2024-04-22T23:34:55.684 INFO:tasks.workunit.client.1.smithi157.stdout:5/343: write d1/d27/d29/d4b/f5c [425236,84260] 0 2024-04-22T23:34:55.685 INFO:tasks.workunit.client.1.smithi157.stdout:9/336: creat d4/df/f62 x:0 0 0 2024-04-22T23:34:55.686 INFO:tasks.workunit.client.1.smithi157.stdout:3/221: dwrite d0/d3/d34/d39/f32 [0,4194304] 0 2024-04-22T23:34:55.686 INFO:tasks.workunit.client.1.smithi157.stdout:1/288: truncate d4/fc 7755 0 2024-04-22T23:34:55.687 INFO:tasks.workunit.client.1.smithi157.stdout:5/344: symlink d1/d5/d7/d2f/d45/l86 0 2024-04-22T23:34:55.688 INFO:tasks.workunit.client.1.smithi157.stdout:9/337: mknod d4/d15/d23/d27/d47/c63 0 2024-04-22T23:34:55.688 INFO:tasks.workunit.client.1.smithi157.stdout:9/338: creat d4/d15/d23/d27/f64 x:0 0 0 2024-04-22T23:34:55.690 INFO:tasks.workunit.client.1.smithi157.stdout:1/289: dread d4/f16 [0,4194304] 0 2024-04-22T23:34:55.693 INFO:tasks.workunit.client.1.smithi157.stdout:3/222: rename d0/d3/fe to d0/d3/d34/d3d/f44 0 2024-04-22T23:34:55.696 INFO:tasks.workunit.client.1.smithi157.stdout:5/345: link d1/d5/f46 d1/d5/dc/d43/d4f/d60/d7a/f87 0 2024-04-22T23:34:55.696 INFO:tasks.workunit.client.1.smithi157.stdout:5/346: read d1/d5/dc/f1b [1742691,17504] 0 2024-04-22T23:34:55.696 INFO:tasks.workunit.client.1.smithi157.stdout:5/347: creat d1/d5/dc/d43/f88 x:0 0 0 2024-04-22T23:34:55.696 INFO:tasks.workunit.client.1.smithi157.stdout:5/348: readlink d1/d1f/l37 0 2024-04-22T23:34:55.697 INFO:tasks.workunit.client.1.smithi157.stdout:1/290: truncate d4/d17/d34/f38 1719684 0 2024-04-22T23:34:55.705 INFO:tasks.workunit.client.1.smithi157.stdout:3/223: rename d0/l26 to d0/d3/d34/d3d/l45 0 2024-04-22T23:34:55.709 INFO:tasks.workunit.client.1.smithi157.stdout:3/224: rename d0/d3/l31 to d0/d7/l46 0 2024-04-22T23:34:55.709 INFO:tasks.workunit.client.1.smithi157.stdout:3/225: stat d0/d3/f35 0 2024-04-22T23:34:55.709 INFO:tasks.workunit.client.1.smithi157.stdout:3/226: chown d0/d3/d13/d21 303630 1 2024-04-22T23:34:55.711 INFO:tasks.workunit.client.1.smithi157.stdout:5/349: link d1/d5/d7/d10/f31 d1/d5/dc/f89 0 2024-04-22T23:34:55.712 INFO:tasks.workunit.client.1.smithi157.stdout:3/227: mkdir d0/d3/d34/d47 0 2024-04-22T23:34:55.712 INFO:tasks.workunit.client.1.smithi157.stdout:3/228: write d0/f11 [3779338,43671] 0 2024-04-22T23:34:55.712 INFO:tasks.workunit.client.1.smithi157.stdout:3/229: chown d0/f40 0 1 2024-04-22T23:34:55.713 INFO:tasks.workunit.client.1.smithi157.stdout:5/350: unlink d1/d5/dc/d43/d5e/f80 0 2024-04-22T23:34:55.715 INFO:tasks.workunit.client.1.smithi157.stdout:3/230: creat d0/d3/d34/f48 x:0 0 0 2024-04-22T23:34:55.716 INFO:tasks.workunit.client.1.smithi157.stdout:5/351: mknod d1/d1f/d22/c8a 0 2024-04-22T23:34:55.717 INFO:tasks.workunit.client.1.smithi157.stdout:5/352: symlink d1/d5/d7/d10/l8b 0 2024-04-22T23:34:55.729 INFO:tasks.workunit.client.1.smithi157.stdout:3/231: dwrite d0/d3/d34/d39/f32 [0,4194304] 0 2024-04-22T23:34:55.733 INFO:tasks.workunit.client.1.smithi157.stdout:3/232: dread d0/f15 [0,4194304] 0 2024-04-22T23:34:55.735 INFO:tasks.workunit.client.1.smithi157.stdout:3/233: rename d0/d3/f3b to d0/d3/d34/d39/f49 0 2024-04-22T23:34:55.748 INFO:tasks.workunit.client.1.smithi157.stdout:3/234: dwrite d0/d3/d34/d39/f32 [0,4194304] 0 2024-04-22T23:34:55.748 INFO:tasks.workunit.client.1.smithi157.stdout:3/235: truncate d0/f41 835411 0 2024-04-22T23:34:55.748 INFO:tasks.workunit.client.1.smithi157.stdout:3/236: fsync d0/f41 0 2024-04-22T23:34:55.748 INFO:tasks.workunit.client.1.smithi157.stdout:3/237: chown d0/f11 32893440 1 2024-04-22T23:34:55.755 INFO:tasks.workunit.client.1.smithi157.stdout:3/238: dread d0/d3/d34/d3d/f42 [0,4194304] 0 2024-04-22T23:34:55.756 INFO:tasks.workunit.client.1.smithi157.stdout:3/239: mkdir d0/d3/d34/d4a 0 2024-04-22T23:34:55.757 INFO:tasks.workunit.client.1.smithi157.stdout:3/240: unlink d0/f27 0 2024-04-22T23:34:55.758 INFO:tasks.workunit.client.1.smithi157.stdout:3/241: mknod d0/c4b 0 2024-04-22T23:34:55.759 INFO:tasks.workunit.client.1.smithi157.stdout:3/242: creat d0/d3/d34/d39/f4c x:0 0 0 2024-04-22T23:34:55.759 INFO:tasks.workunit.client.1.smithi157.stdout:3/243: stat d0/f14 0 2024-04-22T23:34:55.760 INFO:tasks.workunit.client.1.smithi157.stdout:3/244: mkdir d0/d3/d13/d21/d4d 0 2024-04-22T23:34:55.761 INFO:tasks.workunit.client.1.smithi157.stdout:3/245: creat d0/d3/d13/d21/f4e x:0 0 0 2024-04-22T23:34:55.761 INFO:tasks.workunit.client.1.smithi157.stdout:3/246: chown d0/d3/d34/d39 1163 1 2024-04-22T23:34:55.767 INFO:tasks.workunit.client.1.smithi157.stdout:3/247: write d0/f15 [1956843,44489] 0 2024-04-22T23:34:55.767 INFO:tasks.workunit.client.1.smithi157.stdout:3/248: chown d0/d3/d13 34 1 2024-04-22T23:34:55.767 INFO:tasks.workunit.client.1.smithi157.stdout:3/249: dread - d0/d3/d13/d21/f4e zero size 2024-04-22T23:34:55.767 INFO:tasks.workunit.client.1.smithi157.stdout:3/250: dread - d0/d3/d13/d21/f4e zero size 2024-04-22T23:34:55.773 INFO:tasks.workunit.client.1.smithi157.stdout:3/251: truncate d0/d3/f1e 3956937 0 2024-04-22T23:34:55.773 INFO:tasks.workunit.client.1.smithi157.stdout:3/252: readlink d0/d3/d34/d39/l2d 0 2024-04-22T23:34:55.773 INFO:tasks.workunit.client.1.smithi157.stdout:3/253: creat d0/d7/f4f x:0 0 0 2024-04-22T23:34:55.773 INFO:tasks.workunit.client.1.smithi157.stdout:3/254: write d0/d3/d34/d39/f4c [893062,88034] 0 2024-04-22T23:34:55.773 INFO:tasks.workunit.client.1.smithi157.stdout:3/255: write d0/d3/d13/d21/f3f [578272,114235] 0 2024-04-22T23:34:55.773 INFO:tasks.workunit.client.1.smithi157.stdout:3/256: chown d0/d3/d34/d3d/f42 221721525 1 2024-04-22T23:34:55.773 INFO:tasks.workunit.client.1.smithi157.stdout:3/257: chown d0/d3/d34/d39/l1a 178 1 2024-04-22T23:34:55.784 INFO:tasks.workunit.client.1.smithi157.stdout:3/258: dwrite d0/d3/d34/d39/f49 [0,4194304] 0 2024-04-22T23:34:55.784 INFO:tasks.workunit.client.1.smithi157.stdout:3/259: write d0/d3/d13/d21/f3f [1639822,45316] 0 2024-04-22T23:34:55.787 INFO:tasks.workunit.client.1.smithi157.stdout:3/260: rename d0/d7/l46 to d0/d3/d13/d21/d4d/l50 0 2024-04-22T23:34:55.787 INFO:tasks.workunit.client.1.smithi157.stdout:3/261: read d0/d3/d13/f2b [2131425,119294] 0 2024-04-22T23:34:55.789 INFO:tasks.workunit.client.1.smithi157.stdout:3/262: unlink d0/f40 0 2024-04-22T23:34:55.789 INFO:tasks.workunit.client.1.smithi157.stdout:3/263: creat d0/d3/f51 x:0 0 0 2024-04-22T23:34:55.803 INFO:tasks.workunit.client.1.smithi157.stdout:3/264: dwrite d0/d3/d34/d3d/f42 [0,4194304] 0 2024-04-22T23:34:55.803 INFO:tasks.workunit.client.1.smithi157.stdout:3/265: stat d0/c37 0 2024-04-22T23:34:55.803 INFO:tasks.workunit.client.1.smithi157.stdout:3/266: dread - d0/d3/d34/d39/f18 zero size 2024-04-22T23:34:55.805 INFO:tasks.workunit.client.1.smithi157.stdout:3/267: symlink d0/d3/d34/d39/l52 0 2024-04-22T23:34:55.805 INFO:tasks.workunit.client.1.smithi157.stdout:3/268: chown d0 16824326 1 2024-04-22T23:34:55.806 INFO:tasks.workunit.client.1.smithi157.stdout:3/269: creat d0/d3/d34/d39/f53 x:0 0 0 2024-04-22T23:34:55.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.819+0000 7f4064ce1700 1 -- 172.21.15.107:0/2150163665 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f406010f780 con 0x7f40600fcff0 2024-04-22T23:34:55.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.820+0000 7f404effd700 1 -- 172.21.15.107:0/2150163665 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7f4054056490 con 0x7f40600fcff0 2024-04-22T23:34:55.815 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-22T23:34:55.815 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-04-22T23:34:55.815 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 2 2024-04-22T23:34:55.815 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:34:55.815 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-04-22T23:34:55.815 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 2 2024-04-22T23:34:55.815 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:34:55.815 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-04-22T23:34:55.816 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 6 2024-04-22T23:34:55.816 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:34:55.816 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-04-22T23:34:55.816 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 4 2024-04-22T23:34:55.816 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:34:55.816 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-04-22T23:34:55.816 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 14 2024-04-22T23:34:55.816 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T23:34:55.816 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-22T23:34:55.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.822+0000 7f4064ce1700 1 -- 172.21.15.107:0/2150163665 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f40480690a0 msgr2=0x7f404806b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:34:55.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.822+0000 7f4064ce1700 1 --2- 172.21.15.107:0/2150163665 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f40480690a0 0x7f404806b550 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f405000b3b0 tx=0x7f4050009e30 comp rx=0 tx=0).stop 2024-04-22T23:34:55.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.822+0000 7f4064ce1700 1 -- 172.21.15.107:0/2150163665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40600fcff0 msgr2=0x7f406010df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:34:55.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.822+0000 7f4064ce1700 1 --2- 172.21.15.107:0/2150163665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40600fcff0 0x7f406010df20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4054002a80 tx=0x7f40540046b0 comp rx=0 tx=0).stop 2024-04-22T23:34:55.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.822+0000 7f4064ce1700 1 -- 172.21.15.107:0/2150163665 shutdown_connections 2024-04-22T23:34:55.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.822+0000 7f4064ce1700 1 --2- 172.21.15.107:0/2150163665 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f40480690a0 0x7f404806b550 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:55.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.822+0000 7f4064ce1700 1 --2- 172.21.15.107:0/2150163665 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40600fbd40 0x7f406010d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:55.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.822+0000 7f4064ce1700 1 --2- 172.21.15.107:0/2150163665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40600fcff0 0x7f406010df20 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:55.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.822+0000 7f4064ce1700 1 -- 172.21.15.107:0/2150163665 >> 172.21.15.107:0/2150163665 conn(0x7f40600f74c0 msgr2=0x7f40600f8b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:34:55.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.822+0000 7f4064ce1700 1 -- 172.21.15.107:0/2150163665 shutdown_connections 2024-04-22T23:34:55.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.822+0000 7f4064ce1700 1 -- 172.21.15.107:0/2150163665 wait complete. 2024-04-22T23:34:55.871 INFO:tasks.workunit.client.1.smithi157.stdout:4/382: sync 2024-04-22T23:34:55.871 INFO:tasks.workunit.client.1.smithi157.stdout:7/192: sync 2024-04-22T23:34:55.871 INFO:tasks.workunit.client.1.smithi157.stdout:2/314: sync 2024-04-22T23:34:55.871 INFO:tasks.workunit.client.1.smithi157.stdout:0/224: sync 2024-04-22T23:34:55.871 INFO:tasks.workunit.client.1.smithi157.stdout:8/156: sync 2024-04-22T23:34:55.872 INFO:tasks.workunit.client.1.smithi157.stdout:0/225: fdatasync d17/d2b/d52/f3f 0 2024-04-22T23:34:55.872 INFO:tasks.workunit.client.1.smithi157.stdout:8/157: truncate d2/f32 740243 0 2024-04-22T23:34:55.872 INFO:tasks.workunit.client.1.smithi157.stdout:8/158: read d2/d4/f1c [988013,91063] 0 2024-04-22T23:34:55.872 INFO:tasks.workunit.client.1.smithi157.stdout:8/159: write d2/d6/de/f1b [733311,49761] 0 2024-04-22T23:34:55.872 INFO:tasks.workunit.client.1.smithi157.stdout:8/160: chown d2/d6/d21/f23 1 1 2024-04-22T23:34:55.872 INFO:tasks.workunit.client.1.smithi157.stdout:8/161: read - d2/d6/de/d11/f27 zero size 2024-04-22T23:34:55.873 INFO:tasks.workunit.client.1.smithi157.stdout:7/193: rename d0/d9/f2a to d0/f3e 0 2024-04-22T23:34:55.873 INFO:tasks.workunit.client.1.smithi157.stdout:4/383: mkdir d3/d5c/d7b 0 2024-04-22T23:34:55.873 INFO:tasks.workunit.client.1.smithi157.stdout:0/226: rmdir d17 39 2024-04-22T23:34:55.874 INFO:tasks.workunit.client.1.smithi157.stdout:8/162: mknod d2/d4/d2c/c35 0 2024-04-22T23:34:55.877 INFO:tasks.workunit.client.1.smithi157.stdout:0/227: creat d17/d38/f56 x:0 0 0 2024-04-22T23:34:55.877 INFO:tasks.workunit.client.1.smithi157.stdout:0/228: write d17/f22 [1664706,68554] 0 2024-04-22T23:34:55.877 INFO:tasks.workunit.client.1.smithi157.stdout:8/163: symlink d2/d6/de/d17/l36 0 2024-04-22T23:34:55.878 INFO:tasks.workunit.client.1.smithi157.stdout:8/164: write d2/d6/d21/f2e [734836,121655] 0 2024-04-22T23:34:55.878 INFO:tasks.workunit.client.1.smithi157.stdout:4/384: getdents d3/d11/d34 0 2024-04-22T23:34:55.878 INFO:tasks.workunit.client.1.smithi157.stdout:2/315: write d7/d8/f34 [4001565,25365] 0 2024-04-22T23:34:55.878 INFO:tasks.workunit.client.1.smithi157.stdout:2/316: chown d7/d1f/d26/c46 11569 1 2024-04-22T23:34:55.879 INFO:tasks.workunit.client.1.smithi157.stdout:4/385: link d3/d4b/lb d3/d4b/l7c 0 2024-04-22T23:34:55.881 INFO:tasks.workunit.client.1.smithi157.stdout:4/386: creat d3/d4b/d5d/f7d x:0 0 0 2024-04-22T23:34:55.881 INFO:tasks.workunit.client.1.smithi157.stdout:4/387: dread - d3/d11/d34/f51 zero size 2024-04-22T23:34:55.881 INFO:tasks.workunit.client.1.smithi157.stdout:4/388: getdents d3/d4b/d5d/d6c 0 2024-04-22T23:34:55.881 INFO:tasks.workunit.client.1.smithi157.stdout:4/389: truncate d3/d11/d30/f57 151131 0 2024-04-22T23:34:55.881 INFO:tasks.workunit.client.1.smithi157.stdout:2/317: dread d7/d8/d25/f33 [0,4194304] 0 2024-04-22T23:34:55.882 INFO:tasks.workunit.client.1.smithi157.stdout:4/390: mknod d3/d11/d34/c7e 0 2024-04-22T23:34:55.882 INFO:tasks.workunit.client.1.smithi157.stdout:4/391: write d3/f43 [781504,80890] 0 2024-04-22T23:34:55.883 INFO:tasks.workunit.client.1.smithi157.stdout:4/392: write d3/d11/d34/f48 [733834,12974] 0 2024-04-22T23:34:55.883 INFO:tasks.workunit.client.1.smithi157.stdout:2/318: rmdir d7/d8/d30 39 2024-04-22T23:34:55.884 INFO:tasks.workunit.client.1.smithi157.stdout:2/319: creat d7/d8/dd/d4d/f6e x:0 0 0 2024-04-22T23:34:55.886 INFO:tasks.workunit.client.1.smithi157.stdout:2/320: unlink c6 0 2024-04-22T23:34:55.887 INFO:tasks.workunit.client.1.smithi157.stdout:2/321: fdatasync d7/d8/dd/f31 0 2024-04-22T23:34:55.887 INFO:tasks.workunit.client.1.smithi157.stdout:2/322: chown d7/d8/dd/d4d/l6d 0 1 2024-04-22T23:34:55.887 INFO:tasks.workunit.client.1.smithi157.stdout:0/229: dread d17/d2b/f3d [0,4194304] 0 2024-04-22T23:34:55.887 INFO:tasks.workunit.client.1.smithi157.stdout:0/230: write d17/d23/f27 [2135132,113651] 0 2024-04-22T23:34:55.890 INFO:tasks.workunit.client.1.smithi157.stdout:0/231: symlink d17/d2b/d52/l57 0 2024-04-22T23:34:55.890 INFO:tasks.workunit.client.1.smithi157.stdout:0/232: fsync fe 0 2024-04-22T23:34:55.890 INFO:tasks.workunit.client.1.smithi157.stdout:4/393: read d3/d4b/f3e [915413,12592] 0 2024-04-22T23:34:55.891 INFO:tasks.workunit.client.1.smithi157.stdout:0/233: write d17/f19 [4799916,72060] 0 2024-04-22T23:34:55.893 INFO:tasks.workunit.client.1.smithi157.stdout:4/394: mknod d3/d11/d34/c7f 0 2024-04-22T23:34:55.894 INFO:tasks.workunit.client.1.smithi157.stdout:0/234: mkdir d17/d58 0 2024-04-22T23:34:55.894 INFO:tasks.workunit.client.1.smithi157.stdout:0/235: stat f3 0 2024-04-22T23:34:55.894 INFO:tasks.workunit.client.1.smithi157.stdout:2/323: dread d7/d1f/d26/f44 [0,4194304] 0 2024-04-22T23:34:55.895 INFO:tasks.workunit.client.1.smithi157.stdout:4/395: mkdir d3/d11/d1d/d73/d80 0 2024-04-22T23:34:55.895 INFO:tasks.workunit.client.1.smithi157.stdout:0/236: symlink d17/d2b/d52/l59 0 2024-04-22T23:34:55.897 INFO:tasks.workunit.client.1.smithi157.stdout:2/324: unlink d7/d8/f35 0 2024-04-22T23:34:55.897 INFO:tasks.workunit.client.1.smithi157.stdout:2/325: write d7/d8/f2f [2957048,75284] 0 2024-04-22T23:34:55.900 INFO:tasks.workunit.client.1.smithi157.stdout:2/326: symlink d7/d8/d30/l6f 0 2024-04-22T23:34:55.901 INFO:tasks.workunit.client.1.smithi157.stdout:0/237: dread d17/d2b/f3d [0,4194304] 0 2024-04-22T23:34:55.901 INFO:tasks.workunit.client.1.smithi157.stdout:2/327: mkdir d7/d1f/d26/d70 0 2024-04-22T23:34:55.902 INFO:tasks.workunit.client.1.smithi157.stdout:0/238: mkdir d17/d2b/d4f/d5a 0 2024-04-22T23:34:55.904 INFO:tasks.workunit.client.1.smithi157.stdout:2/328: creat d7/d8/dd/f71 x:0 0 0 2024-04-22T23:34:55.906 INFO:tasks.workunit.client.1.smithi157.stdout:2/329: rename d7/d1f/d26/f64 to d7/d8/d30/d5b/f72 0 2024-04-22T23:34:55.908 INFO:tasks.workunit.client.1.smithi157.stdout:2/330: creat d7/d8/d30/d5b/f73 x:0 0 0 2024-04-22T23:34:55.908 INFO:tasks.workunit.client.1.smithi157.stdout:2/331: creat d7/d1f/d26/f74 x:0 0 0 2024-04-22T23:34:55.908 INFO:tasks.workunit.client.1.smithi157.stdout:2/332: truncate d7/d1f/f4c 240350 0 2024-04-22T23:34:55.911 INFO:tasks.workunit.client.1.smithi157.stdout:7/194: dwrite d0/f22 [4194304,4194304] 0 2024-04-22T23:34:55.912 INFO:tasks.workunit.client.1.smithi157.stdout:7/195: stat d0/d9/d29/l2d 0 2024-04-22T23:34:55.912 INFO:tasks.workunit.client.1.smithi157.stdout:7/196: chown d0/f3e 103001 1 2024-04-22T23:34:55.913 INFO:tasks.workunit.client.1.smithi157.stdout:7/197: rename d0/l2f to d0/d9/d10/l3f 0 2024-04-22T23:34:55.914 INFO:tasks.workunit.client.1.smithi157.stdout:7/198: symlink d0/d9/l40 0 2024-04-22T23:34:55.915 INFO:tasks.workunit.client.1.smithi157.stdout:7/199: mknod d0/d9/db/d35/c41 0 2024-04-22T23:34:55.916 INFO:tasks.workunit.client.1.smithi157.stdout:7/200: write d0/d9/d10/f30 [1739125,16082] 0 2024-04-22T23:34:55.916 INFO:tasks.workunit.client.1.smithi157.stdout:7/201: truncate d0/d9/de/f23 274454 0 2024-04-22T23:34:55.918 INFO:tasks.workunit.client.1.smithi157.stdout:8/165: dwrite d2/d4/f1c [0,4194304] 0 2024-04-22T23:34:55.918 INFO:tasks.workunit.client.1.smithi157.stdout:7/202: mkdir d0/d9/d42 0 2024-04-22T23:34:55.918 INFO:tasks.workunit.client.1.smithi157.stdout:8/166: creat d2/d6/f37 x:0 0 0 2024-04-22T23:34:55.919 INFO:tasks.workunit.client.1.smithi157.stdout:4/396: dwrite d3/f58 [0,4194304] 0 2024-04-22T23:34:55.919 INFO:tasks.workunit.client.1.smithi157.stdout:0/239: dwrite d17/d2b/f47 [0,4194304] 0 2024-04-22T23:34:55.919 INFO:tasks.workunit.client.1.smithi157.stdout:0/240: dread - d17/f25 zero size 2024-04-22T23:34:55.919 INFO:tasks.workunit.client.1.smithi157.stdout:0/241: chown d17/f22 19 1 2024-04-22T23:34:55.919 INFO:tasks.workunit.client.1.smithi157.stdout:0/242: write d17/d23/f51 [750286,22267] 0 2024-04-22T23:34:55.920 INFO:tasks.workunit.client.1.smithi157.stdout:0/243: dread - d17/d23/f4d zero size 2024-04-22T23:34:55.920 INFO:tasks.workunit.client.1.smithi157.stdout:7/203: mkdir d0/d9/d10/d43 0 2024-04-22T23:34:55.926 INFO:tasks.workunit.client.1.smithi157.stdout:0/244: getdents d17/d23 0 2024-04-22T23:34:55.926 INFO:tasks.workunit.client.1.smithi157.stdout:0/245: truncate d17/f1b 4953084 0 2024-04-22T23:34:55.927 INFO:tasks.workunit.client.1.smithi157.stdout:0/246: getdents d17/d58 0 2024-04-22T23:34:55.927 INFO:tasks.workunit.client.1.smithi157.stdout:2/333: dwrite d7/f13 [0,4194304] 0 2024-04-22T23:34:55.927 INFO:tasks.workunit.client.1.smithi157.stdout:7/204: getdents d0/d9/db 0 2024-04-22T23:34:55.928 INFO:tasks.workunit.client.1.smithi157.stdout:0/247: unlink d17/d23/d34/l43 0 2024-04-22T23:34:55.929 INFO:tasks.workunit.client.1.smithi157.stdout:2/334: link d7/d8/d30/d23/c32 d7/d8/d30/d23/d27/d57/c75 0 2024-04-22T23:34:55.929 INFO:tasks.workunit.client.1.smithi157.stdout:4/397: read d3/f1b [7268973,126281] 0 2024-04-22T23:34:55.930 INFO:tasks.workunit.client.1.smithi157.stdout:7/205: creat d0/d9/d10/d43/f44 x:0 0 0 2024-04-22T23:34:55.933 INFO:tasks.workunit.client.1.smithi157.stdout:2/335: creat d7/d8/d30/f76 x:0 0 0 2024-04-22T23:34:55.934 INFO:tasks.workunit.client.1.smithi157.stdout:2/336: read - d7/d1f/d26/d4a/d54/f63 zero size 2024-04-22T23:34:55.934 INFO:tasks.workunit.client.1.smithi157.stdout:7/206: creat d0/f45 x:0 0 0 2024-04-22T23:34:55.935 INFO:tasks.workunit.client.1.smithi157.stdout:2/337: creat d7/d1f/d26/d4a/f77 x:0 0 0 2024-04-22T23:34:55.936 INFO:tasks.workunit.client.1.smithi157.stdout:2/338: creat d7/d8/d30/d23/d27/d5f/f78 x:0 0 0 2024-04-22T23:34:55.937 INFO:tasks.workunit.client.1.smithi157.stdout:8/167: dwrite d2/f3 [0,4194304] 0 2024-04-22T23:34:55.937 INFO:tasks.workunit.client.1.smithi157.stdout:8/168: chown d2/d6/de/d17/l24 5 1 2024-04-22T23:34:55.939 INFO:tasks.workunit.client.1.smithi157.stdout:2/339: unlink d7/d1f/d26/d4a/f4b 0 2024-04-22T23:34:55.941 INFO:tasks.workunit.client.1.smithi157.stdout:2/340: creat d7/d8/d30/f79 x:0 0 0 2024-04-22T23:34:55.941 INFO:tasks.workunit.client.1.smithi157.stdout:0/248: dread f3 [0,4194304] 0 2024-04-22T23:34:55.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.947+0000 7fb5ce02e700 1 -- 172.21.15.107:0/2665273773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5c80fe050 msgr2=0x7fb5c80fe4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:34:55.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.947+0000 7fb5ce02e700 1 --2- 172.21.15.107:0/2665273773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5c80fe050 0x7fb5c80fe4c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc013bf0 tx=0x7fb5bc009e30 comp rx=0 tx=0).stop 2024-04-22T23:34:55.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.948+0000 7fb5ce02e700 1 -- 172.21.15.107:0/2665273773 shutdown_connections 2024-04-22T23:34:55.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.948+0000 7fb5ce02e700 1 --2- 172.21.15.107:0/2665273773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5c80fe050 0x7fb5c80fe4c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:55.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.948+0000 7fb5ce02e700 1 --2- 172.21.15.107:0/2665273773 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5c80fce50 0x7fb5c80fd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:55.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.948+0000 7fb5ce02e700 1 -- 172.21.15.107:0/2665273773 >> 172.21.15.107:0/2665273773 conn(0x7fb5c80f8540 msgr2=0x7fb5c80fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:34:55.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.948+0000 7fb5ce02e700 1 -- 172.21.15.107:0/2665273773 shutdown_connections 2024-04-22T23:34:55.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.948+0000 7fb5ce02e700 1 -- 172.21.15.107:0/2665273773 wait complete. 2024-04-22T23:34:55.945 INFO:tasks.workunit.client.1.smithi157.stdout:2/341: symlink d7/d8/d30/d5b/l7a 0 2024-04-22T23:34:55.945 INFO:tasks.workunit.client.1.smithi157.stdout:2/342: write d7/d8/d30/f76 [430452,21459] 0 2024-04-22T23:34:55.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.949+0000 7fb5ce02e700 1 Processor -- start 2024-04-22T23:34:55.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.950+0000 7fb5ce02e700 1 -- start start 2024-04-22T23:34:55.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.950+0000 7fb5ce02e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5c80fce50 0x7fb5c810b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:34:55.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.950+0000 7fb5ce02e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5c810bf10 0x7fb5c810cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:34:55.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.950+0000 7fb5ce02e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c810c460 con 0x7fb5c80fce50 2024-04-22T23:34:55.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.950+0000 7fb5ce02e700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c810c5a0 con 0x7fb5c810bf10 2024-04-22T23:34:55.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.951+0000 7fb5c77fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5c80fce50 0x7fb5c810b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:34:55.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.951+0000 7fb5c77fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5c80fce50 0x7fb5c810b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51228/0 (socket says 172.21.15.107:51228) 2024-04-22T23:34:55.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.951+0000 7fb5c77fe700 1 -- 172.21.15.107:0/4119674625 learned_addr learned my addr 172.21.15.107:0/4119674625 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:34:55.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.951+0000 7fb5c6ffd700 1 --2- 172.21.15.107:0/4119674625 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5c810bf10 0x7fb5c810cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:34:55.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.951+0000 7fb5c77fe700 1 -- 172.21.15.107:0/4119674625 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5c810bf10 msgr2=0x7fb5c810cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:34:55.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.951+0000 7fb5c77fe700 1 --2- 172.21.15.107:0/4119674625 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5c810bf10 0x7fb5c810cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:55.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.951+0000 7fb5c77fe700 1 -- 172.21.15.107:0/4119674625 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5bc005040 con 0x7fb5c80fce50 2024-04-22T23:34:55.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.951+0000 7fb5c77fe700 1 --2- 172.21.15.107:0/4119674625 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5c80fce50 0x7fb5c810b9d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb5b800b790 tx=0x7fb5b8009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:34:55.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.952+0000 7fb5affff700 1 -- 172.21.15.107:0/4119674625 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5b800f6f0 con 0x7fb5c80fce50 2024-04-22T23:34:55.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.952+0000 7fb5ce02e700 1 -- 172.21.15.107:0/4119674625 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5c810d440 con 0x7fb5c80fce50 2024-04-22T23:34:55.951 INFO:tasks.workunit.client.1.smithi157.stdout:2/343: fsync d7/d8/dd/f71 0 2024-04-22T23:34:55.951 INFO:tasks.workunit.client.1.smithi157.stdout:2/344: creat d7/d8/d30/d23/d27/f7b x:0 0 0 2024-04-22T23:34:55.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.952+0000 7fb5ce02e700 1 -- 172.21.15.107:0/4119674625 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5c810da00 con 0x7fb5c80fce50 2024-04-22T23:34:55.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.952+0000 7fb5affff700 1 -- 172.21.15.107:0/4119674625 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb5b800fd20 con 0x7fb5c80fce50 2024-04-22T23:34:55.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.952+0000 7fb5affff700 1 -- 172.21.15.107:0/4119674625 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5b80103a0 con 0x7fb5c80fce50 2024-04-22T23:34:55.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.954+0000 7fb5ce02e700 1 -- 172.21.15.107:0/4119674625 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5b40052e0 con 0x7fb5c80fce50 2024-04-22T23:34:55.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.956+0000 7fb5affff700 1 -- 172.21.15.107:0/4119674625 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fb5b8011400 con 0x7fb5c80fce50 2024-04-22T23:34:55.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.956+0000 7fb5affff700 1 --2- 172.21.15.107:0/4119674625 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fb5b0069370 0x7fb5b006b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:34:55.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.956+0000 7fb5affff700 1 -- 172.21.15.107:0/4119674625 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fb5b8086dc0 con 0x7fb5c80fce50 2024-04-22T23:34:55.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.957+0000 7fb5c6ffd700 1 --2- 172.21.15.107:0/4119674625 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fb5b0069370 0x7fb5b006b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:34:55.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.957+0000 7fb5c6ffd700 1 --2- 172.21.15.107:0/4119674625 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fb5b0069370 0x7fb5b006b820 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc000c00 tx=0x7fb5bc009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:34:55.955 INFO:tasks.workunit.client.1.smithi157.stdout:0/249: write d17/d2b/d52/f44 [2527946,116449] 0 2024-04-22T23:34:55.957 INFO:tasks.workunit.client.1.smithi157.stdout:0/250: truncate fc 6020583 0 2024-04-22T23:34:55.957 INFO:tasks.workunit.client.1.smithi157.stdout:0/251: chown d17/c1e 315077 1 2024-04-22T23:34:55.958 INFO:tasks.workunit.client.1.smithi157.stdout:2/345: getdents d7/d1f/d3e 0 2024-04-22T23:34:55.959 INFO:tasks.workunit.client.1.smithi157.stdout:4/398: dwrite d3/d11/f22 [4194304,4194304] 0 2024-04-22T23:34:55.959 INFO:tasks.workunit.client.1.smithi157.stdout:8/169: dwrite d2/d4/f1c [4194304,4194304] 0 2024-04-22T23:34:55.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:55.964+0000 7fb5affff700 1 -- 172.21.15.107:0/4119674625 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb5b8056940 con 0x7fb5c80fce50 2024-04-22T23:34:55.961 INFO:tasks.workunit.client.1.smithi157.stdout:7/207: dwrite d0/d9/d10/f1f [0,4194304] 0 2024-04-22T23:34:55.961 INFO:tasks.workunit.client.1.smithi157.stdout:0/252: mkdir d17/d2b/d5b 0 2024-04-22T23:34:55.962 INFO:tasks.workunit.client.1.smithi157.stdout:0/253: write f3 [4587980,71064] 0 2024-04-22T23:34:55.962 INFO:tasks.workunit.client.1.smithi157.stdout:0/254: write f11 [1634850,7784] 0 2024-04-22T23:34:55.966 INFO:tasks.workunit.client.1.smithi157.stdout:4/399: mkdir d3/d11/d30/d36/d62/d81 0 2024-04-22T23:34:55.969 INFO:tasks.workunit.client.1.smithi157.stdout:8/170: rename d2/d4/c9 to d2/d6/c38 0 2024-04-22T23:34:55.969 INFO:tasks.workunit.client.1.smithi157.stdout:8/171: stat d2/d4/l7 0 2024-04-22T23:34:55.969 INFO:tasks.workunit.client.1.smithi157.stdout:8/172: chown d2/d6/de/d17/l24 23017615 1 2024-04-22T23:34:55.971 INFO:tasks.workunit.client.1.smithi157.stdout:8/173: dread d2/d6/d21/f2e [0,4194304] 0 2024-04-22T23:34:55.972 INFO:tasks.workunit.client.1.smithi157.stdout:8/174: write d2/d6/de/f1b [68534,116246] 0 2024-04-22T23:34:55.975 INFO:tasks.workunit.client.1.smithi157.stdout:0/255: dwrite d17/d23/f3e [0,4194304] 0 2024-04-22T23:34:55.976 INFO:tasks.workunit.client.1.smithi157.stdout:0/256: creat d17/d2b/f5c x:0 0 0 2024-04-22T23:34:55.976 INFO:tasks.workunit.client.1.smithi157.stdout:0/257: truncate d17/f1b 5193443 0 2024-04-22T23:34:55.976 INFO:tasks.workunit.client.1.smithi157.stdout:0/258: chown d17/d2b/l4b 1795 1 2024-04-22T23:34:55.979 INFO:tasks.workunit.client.1.smithi157.stdout:7/208: link d0/d9/d10/f20 d0/f46 0 2024-04-22T23:34:55.984 INFO:tasks.workunit.client.1.smithi157.stdout:4/400: dwrite d3/d11/d30/d36/d62/f67 [0,4194304] 0 2024-04-22T23:34:56.001 INFO:tasks.workunit.client.1.smithi157.stdout:4/401: dwrite d3/d11/d1d/f52 [0,4194304] 0 2024-04-22T23:34:56.023 INFO:tasks.workunit.client.1.smithi157.stdout:4/402: dwrite d3/d11/d1d/f24 [0,4194304] 0 2024-04-22T23:34:56.023 INFO:tasks.workunit.client.1.smithi157.stdout:4/403: stat d3/f1b 0 2024-04-22T23:34:56.024 INFO:tasks.workunit.client.1.smithi157.stdout:4/404: chown d3/d5c 2936 1 2024-04-22T23:34:56.028 INFO:tasks.workunit.client.1.smithi157.stdout:2/346: rmdir d7/d8 39 2024-04-22T23:34:56.028 INFO:tasks.workunit.client.1.smithi157.stdout:2/347: chown d7/d1f/d26/f44 116716356 1 2024-04-22T23:34:56.028 INFO:tasks.workunit.client.1.smithi157.stdout:2/348: stat d7/d1f/d26/c46 0 2024-04-22T23:34:56.029 INFO:tasks.workunit.client.1.smithi157.stdout:2/349: chown d7/d8/d25/d55 36 1 2024-04-22T23:34:56.029 INFO:tasks.workunit.client.1.smithi157.stdout:2/350: read - d7/d1f/d26/d4a/f77 zero size 2024-04-22T23:34:56.032 INFO:tasks.workunit.client.1.smithi157.stdout:8/175: symlink d2/d6/d21/l39 0 2024-04-22T23:34:56.036 INFO:tasks.workunit.client.1.smithi157.stdout:0/259: symlink d17/d2b/d52/l5d 0 2024-04-22T23:34:56.038 INFO:tasks.workunit.client.1.smithi157.stdout:7/209: mknod d0/d9/db/c47 0 2024-04-22T23:34:56.046 INFO:tasks.workunit.client.1.smithi157.stdout:2/351: dwrite d7/d8/d25/f50 [0,4194304] 0 2024-04-22T23:34:56.046 INFO:tasks.workunit.client.1.smithi157.stdout:2/352: stat d7/d8/d30/d23/d27 0 2024-04-22T23:34:56.048 INFO:tasks.workunit.client.1.smithi157.stdout:2/353: dread d7/d8/d30/d23/d27/f67 [0,4194304] 0 2024-04-22T23:34:56.053 INFO:tasks.workunit.client.1.smithi157.stdout:0/260: dwrite d17/d2b/f5c [0,4194304] 0 2024-04-22T23:34:56.056 INFO:tasks.workunit.client.1.smithi157.stdout:2/354: dread d7/d1f/f56 [0,4194304] 0 2024-04-22T23:34:56.062 INFO:tasks.workunit.client.1.smithi157.stdout:4/405: creat d3/d11/d1d/d73/d80/f82 x:0 0 0 2024-04-22T23:34:56.074 INFO:tasks.workunit.client.1.smithi157.stdout:0/261: mknod d17/d2b/d5b/c5e 0 2024-04-22T23:34:56.077 INFO:tasks.workunit.client.1.smithi157.stdout:2/355: dwrite d7/d8/f2f [0,4194304] 0 2024-04-22T23:34:56.078 INFO:tasks.workunit.client.1.smithi157.stdout:4/406: dwrite d3/d11/d30/d28/d31/f69 [0,4194304] 0 2024-04-22T23:34:56.078 INFO:tasks.workunit.client.1.smithi157.stdout:4/407: write d3/d11/d1d/f41 [5336394,106347] 0 2024-04-22T23:34:56.078 INFO:tasks.workunit.client.1.smithi157.stdout:0/262: dread f3 [0,4194304] 0 2024-04-22T23:34:56.079 INFO:tasks.workunit.client.1.smithi157.stdout:0/263: write fc [6330755,23105] 0 2024-04-22T23:34:56.079 INFO:tasks.workunit.client.1.smithi157.stdout:2/356: symlink d7/d1f/d3e/l7c 0 2024-04-22T23:34:56.082 INFO:tasks.workunit.client.1.smithi157.stdout:0/264: stat d17/d2b/l4b 0 2024-04-22T23:34:56.084 INFO:tasks.workunit.client.1.smithi157.stdout:2/357: mknod d7/d8/dd/d4d/c7d 0 2024-04-22T23:34:56.084 INFO:tasks.workunit.client.1.smithi157.stdout:4/408: read d3/d11/d30/d28/d31/f69 [892705,26234] 0 2024-04-22T23:34:56.085 INFO:tasks.workunit.client.1.smithi157.stdout:0/265: creat d17/d58/f5f x:0 0 0 2024-04-22T23:34:56.088 INFO:tasks.workunit.client.1.smithi157.stdout:4/409: mknod d3/d11/d30/d36/c83 0 2024-04-22T23:34:56.095 INFO:tasks.workunit.client.1.smithi157.stdout:7/210: mknod d0/c48 0 2024-04-22T23:34:56.095 INFO:tasks.workunit.client.1.smithi157.stdout:0/266: dread d17/d23/f32 [0,4194304] 0 2024-04-22T23:34:56.095 INFO:tasks.workunit.client.1.smithi157.stdout:7/211: readlink d0/d9/l24 0 2024-04-22T23:34:56.095 INFO:tasks.workunit.client.1.smithi157.stdout:7/212: truncate d0/d9/f3c 410379 0 2024-04-22T23:34:56.096 INFO:tasks.workunit.client.1.smithi157.stdout:6/451: sync 2024-04-22T23:34:56.096 INFO:tasks.workunit.client.1.smithi157.stdout:7/213: dread d0/d9/f3c [0,4194304] 0 2024-04-22T23:34:56.096 INFO:tasks.workunit.client.1.smithi157.stdout:7/214: fsync d0/d9/d29/f31 0 2024-04-22T23:34:56.099 INFO:tasks.workunit.client.1.smithi157.stdout:6/452: dread d1/d36/f47 [0,4194304] 0 2024-04-22T23:34:56.100 INFO:tasks.workunit.client.1.smithi157.stdout:0/267: dread d17/d23/f32 [0,4194304] 0 2024-04-22T23:34:56.102 INFO:tasks.workunit.client.1.smithi157.stdout:2/358: dwrite d7/d1f/f52 [0,4194304] 0 2024-04-22T23:34:56.103 INFO:tasks.workunit.client.1.smithi157.stdout:6/453: rmdir d1/d17 39 2024-04-22T23:34:56.103 INFO:tasks.workunit.client.1.smithi157.stdout:6/454: fdatasync d1/d2f/f4a 0 2024-04-22T23:34:56.103 INFO:tasks.workunit.client.1.smithi157.stdout:6/455: fdatasync d1/d2f/d54/f87 0 2024-04-22T23:34:56.104 INFO:tasks.workunit.client.1.smithi157.stdout:0/268: dread d17/f2d [0,4194304] 0 2024-04-22T23:34:56.106 INFO:tasks.workunit.client.1.smithi157.stdout:6/456: rename d1/d36/f3f to d1/d2f/d54/f95 0 2024-04-22T23:34:56.108 INFO:tasks.workunit.client.1.smithi157.stdout:6/457: truncate d1/f3 1488939 0 2024-04-22T23:34:56.108 INFO:tasks.workunit.client.1.smithi157.stdout:6/458: truncate d1/d17/f94 843551 0 2024-04-22T23:34:56.111 INFO:tasks.workunit.client.1.smithi157.stdout:6/459: mknod d1/d2f/d54/d86/c96 0 2024-04-22T23:34:56.113 INFO:tasks.workunit.client.1.smithi157.stdout:6/460: rmdir d1/d2f/d3b/d49 39 2024-04-22T23:34:56.114 INFO:tasks.workunit.client.1.smithi157.stdout:2/359: read d7/d8/d25/f4f [148605,42627] 0 2024-04-22T23:34:56.123 INFO:tasks.workunit.client.1.smithi157.stdout:7/215: dwrite d0/d9/d10/f18 [0,4194304] 0 2024-04-22T23:34:56.123 INFO:tasks.workunit.client.1.smithi157.stdout:7/216: chown d0/d9/db/d35/f3a 129316 1 2024-04-22T23:34:56.123 INFO:tasks.workunit.client.1.smithi157.stdout:6/461: write d1/d17/f21 [1962585,62531] 0 2024-04-22T23:34:56.124 INFO:tasks.workunit.client.1.smithi157.stdout:2/360: write d7/d8/f3f [67707,50725] 0 2024-04-22T23:34:56.125 INFO:tasks.workunit.client.1.smithi157.stdout:4/410: dwrite d3/d11/d30/d36/d62/f3d [0,4194304] 0 2024-04-22T23:34:56.126 INFO:tasks.workunit.client.1.smithi157.stdout:2/361: creat d7/d8/d30/d23/d27/d5f/f7e x:0 0 0 2024-04-22T23:34:56.126 INFO:tasks.workunit.client.1.smithi157.stdout:4/411: getdents d3/d5c 0 2024-04-22T23:34:56.133 INFO:tasks.workunit.client.1.smithi157.stdout:7/217: write d0/f8 [577201,106810] 0 2024-04-22T23:34:56.133 INFO:tasks.workunit.client.1.smithi157.stdout:7/218: write d0/d9/d29/f31 [1047466,26097] 0 2024-04-22T23:34:56.133 INFO:tasks.workunit.client.1.smithi157.stdout:7/219: write d0/f3e [700058,26271] 0 2024-04-22T23:34:56.134 INFO:tasks.workunit.client.1.smithi157.stdout:7/220: mkdir d0/d9/d29/d49 0 2024-04-22T23:34:56.138 INFO:tasks.workunit.client.1.smithi157.stdout:6/462: dread d1/d2f/d3b/d6c/f6d [0,4194304] 0 2024-04-22T23:34:56.138 INFO:tasks.workunit.client.1.smithi157.stdout:7/221: mkdir d0/d9/d29/d49/d4a 0 2024-04-22T23:34:56.138 INFO:tasks.workunit.client.1.smithi157.stdout:7/222: fsync d0/f45 0 2024-04-22T23:34:56.138 INFO:tasks.workunit.client.1.smithi157.stdout:6/463: read d1/d17/f31 [843161,26743] 0 2024-04-22T23:34:56.138 INFO:tasks.workunit.client.1.smithi157.stdout:7/223: mknod d0/d9/d10/d43/c4b 0 2024-04-22T23:34:56.138 INFO:tasks.workunit.client.1.smithi157.stdout:6/464: mkdir d1/d38/d97 0 2024-04-22T23:34:56.139 INFO:tasks.workunit.client.1.smithi157.stdout:6/465: dread d1/d17/f31 [0,4194304] 0 2024-04-22T23:34:56.139 INFO:tasks.workunit.client.1.smithi157.stdout:6/466: fsync d1/d2f/d54/d7a/f79 0 2024-04-22T23:34:56.140 INFO:tasks.workunit.client.1.smithi157.stdout:6/467: rename d1/d2f/d54/d7a/f7c to d1/d2f/d3b/d6c/f98 0 2024-04-22T23:34:56.166 INFO:tasks.workunit.client.1.smithi157.stdout:0/269: dwrite d17/d23/f27 [0,4194304] 0 2024-04-22T23:34:56.168 INFO:tasks.workunit.client.1.smithi157.stdout:0/270: rename c13 to d17/d23/c60 0 2024-04-22T23:34:56.177 INFO:tasks.workunit.client.1.smithi157.stdout:4/412: dwrite d3/d4b/f16 [0,4194304] 0 2024-04-22T23:34:56.177 INFO:tasks.workunit.client.1.smithi157.stdout:4/413: stat d3/d11/d34/f48 0 2024-04-22T23:34:56.177 INFO:tasks.workunit.client.1.smithi157.stdout:0/271: write d17/d2b/f3d [3776663,65782] 0 2024-04-22T23:34:56.177 INFO:tasks.workunit.client.1.smithi157.stdout:0/272: creat d17/d58/f61 x:0 0 0 2024-04-22T23:34:56.177 INFO:tasks.workunit.client.1.smithi157.stdout:6/468: dread d1/d2f/d54/d7a/f6e [0,4194304] 0 2024-04-22T23:34:56.178 INFO:tasks.workunit.client.1.smithi157.stdout:7/224: truncate d0/f3e 592832 0 2024-04-22T23:34:56.178 INFO:tasks.workunit.client.1.smithi157.stdout:7/225: readlink d0/d9/db/d35/l3d 0 2024-04-22T23:34:56.184 INFO:tasks.workunit.client.1.smithi157.stdout:4/414: mknod d3/d4b/d5d/d6c/c84 0 2024-04-22T23:34:56.184 INFO:tasks.workunit.client.1.smithi157.stdout:4/415: readlink d3/d11/l63 0 2024-04-22T23:34:56.188 INFO:tasks.workunit.client.1.smithi157.stdout:7/226: creat d0/d9/d29/d33/d37/f4c x:0 0 0 2024-04-22T23:34:56.190 INFO:tasks.workunit.client.1.smithi157.stdout:4/416: symlink d3/d4b/l85 0 2024-04-22T23:34:56.191 INFO:tasks.workunit.client.1.smithi157.stdout:6/469: rmdir d1/d2f/d54/d7a 39 2024-04-22T23:34:56.191 INFO:tasks.workunit.client.1.smithi157.stdout:6/470: rename d1 to d1/d99 22 2024-04-22T23:34:56.194 INFO:tasks.workunit.client.1.smithi157.stdout:7/227: dread d0/d9/d10/f20 [0,4194304] 0 2024-04-22T23:34:56.197 INFO:tasks.workunit.client.1.smithi157.stdout:7/228: rename d0/d9/db/f26 to d0/d9/d29/f4d 0 2024-04-22T23:34:56.197 INFO:tasks.workunit.client.1.smithi157.stdout:7/229: creat d0/d9/d29/d33/f4e x:0 0 0 2024-04-22T23:34:56.197 INFO:tasks.workunit.client.1.smithi157.stdout:7/230: chown d0/d9/db/d35 1 1 2024-04-22T23:34:56.198 INFO:tasks.workunit.client.1.smithi157.stdout:7/231: write d0/d9/db/f39 [507329,73252] 0 2024-04-22T23:34:56.200 INFO:tasks.workunit.client.1.smithi157.stdout:7/232: unlink d0/ld 0 2024-04-22T23:34:56.202 INFO:tasks.workunit.client.1.smithi157.stdout:7/233: stat d0/d9/db/d35/l3d 0 2024-04-22T23:34:56.213 INFO:tasks.workunit.client.1.smithi157.stdout:7/234: write d0/d9/d10/f12 [3795702,70744] 0 2024-04-22T23:34:56.213 INFO:tasks.workunit.client.1.smithi157.stdout:7/235: dread - d0/f45 zero size 2024-04-22T23:34:56.215 INFO:tasks.workunit.client.1.smithi157.stdout:7/236: mknod d0/c4f 0 2024-04-22T23:34:56.215 INFO:tasks.workunit.client.1.smithi157.stdout:7/237: creat d0/d9/d29/d33/d37/f50 x:0 0 0 2024-04-22T23:34:56.215 INFO:tasks.workunit.client.1.smithi157.stdout:7/238: fdatasync d0/f8 0 2024-04-22T23:34:56.218 INFO:tasks.workunit.client.1.smithi157.stdout:7/239: rename d0/d9/d10/f20 to d0/d9/d42/f51 0 2024-04-22T23:34:56.220 INFO:tasks.workunit.client.1.smithi157.stdout:7/240: link d0/d9/d10/c27 d0/d9/db/c52 0 2024-04-22T23:34:56.226 INFO:tasks.workunit.client.1.smithi157.stdout:4/417: dwrite d3/d11/d30/d28/f3c [0,4194304] 0 2024-04-22T23:34:56.227 INFO:tasks.workunit.client.1.smithi157.stdout:6/471: dwrite d1/d2f/d3b/f3e [0,4194304] 0 2024-04-22T23:34:56.227 INFO:tasks.workunit.client.1.smithi157.stdout:6/472: write d1/d2f/d54/d7a/f79 [820433,11732] 0 2024-04-22T23:34:56.230 INFO:tasks.workunit.client.1.smithi157.stdout:4/418: mkdir d3/d4b/d7a/d86 0 2024-04-22T23:34:56.230 INFO:tasks.workunit.client.1.smithi157.stdout:4/419: chown d3/d11/d34/c7f 66369288 1 2024-04-22T23:34:56.236 INFO:tasks.workunit.client.1.smithi157.stdout:4/420: creat d3/d11/d30/d36/d62/d4c/f87 x:0 0 0 2024-04-22T23:34:56.237 INFO:tasks.workunit.client.1.smithi157.stdout:4/421: readlink d3/d11/l56 0 2024-04-22T23:34:56.240 INFO:tasks.workunit.client.1.smithi157.stdout:9/339: sync 2024-04-22T23:34:56.240 INFO:tasks.workunit.client.1.smithi157.stdout:1/291: sync 2024-04-22T23:34:56.250 INFO:tasks.workunit.client.1.smithi157.stdout:6/473: dwrite d1/d2f/d3b/f3e [0,4194304] 0 2024-04-22T23:34:56.251 INFO:tasks.workunit.client.1.smithi157.stdout:6/474: mknod d1/d2f/d3b/d6c/d83/c9a 0 2024-04-22T23:34:56.257 INFO:tasks.workunit.client.1.smithi157.stdout:6/475: dread d1/f7 [0,4194304] 0 2024-04-22T23:34:56.257 INFO:tasks.workunit.client.1.smithi157.stdout:1/292: dwrite d4/d17/d34/d3b/d50/f56 [0,4194304] 0 2024-04-22T23:34:56.257 INFO:tasks.workunit.client.1.smithi157.stdout:6/476: readlink d1/d2f/d54/l8e 0 2024-04-22T23:34:56.257 INFO:tasks.workunit.client.1.smithi157.stdout:6/477: fdatasync d1/d17/f21 0 2024-04-22T23:34:56.259 INFO:tasks.workunit.client.1.smithi157.stdout:1/293: mknod d4/d17/d1d/d33/d41/c61 0 2024-04-22T23:34:56.259 INFO:tasks.workunit.client.1.smithi157.stdout:9/340: dwrite d4/df/d57/f44 [0,4194304] 0 2024-04-22T23:34:56.260 INFO:tasks.workunit.client.1.smithi157.stdout:6/478: rename d1/d2f/d54/d7a to d1/d2f/d54/d9b 0 2024-04-22T23:34:56.260 INFO:tasks.workunit.client.1.smithi157.stdout:6/479: chown d1/d2f/d54/d9b 6828 1 2024-04-22T23:34:56.261 INFO:tasks.workunit.client.1.smithi157.stdout:9/341: mknod d4/df/d61/c65 0 2024-04-22T23:34:56.261 INFO:tasks.workunit.client.1.smithi157.stdout:9/342: truncate d4/d15/d23/d27/f39 325597 0 2024-04-22T23:34:56.261 INFO:tasks.workunit.client.1.smithi157.stdout:9/343: readlink d4/l11 0 2024-04-22T23:34:56.261 INFO:tasks.workunit.client.1.smithi157.stdout:9/344: fdatasync d4/df/f1f 0 2024-04-22T23:34:56.261 INFO:tasks.workunit.client.1.smithi157.stdout:9/345: truncate d4/d15/d55/f58 946392 0 2024-04-22T23:34:56.267 INFO:tasks.workunit.client.1.smithi157.stdout:6/480: write d1/f34 [968300,16521] 0 2024-04-22T23:34:56.268 INFO:tasks.workunit.client.1.smithi157.stdout:6/481: truncate d1/f22 3427060 0 2024-04-22T23:34:56.270 INFO:tasks.workunit.client.1.smithi157.stdout:1/294: dread d4/db/f3c [0,4194304] 0 2024-04-22T23:34:56.274 INFO:tasks.workunit.client.1.smithi157.stdout:1/295: link d4/d17/d1d/d33/l57 d4/d5f/l62 0 2024-04-22T23:34:56.287 INFO:tasks.workunit.client.1.smithi157.stdout:1/296: dwrite d4/f23 [4194304,4194304] 0 2024-04-22T23:34:56.288 INFO:tasks.workunit.client.1.smithi157.stdout:1/297: fsync d4/f26 0 2024-04-22T23:34:56.288 INFO:tasks.workunit.client.1.smithi157.stdout:1/298: creat d4/d17/d34/f63 x:0 0 0 2024-04-22T23:34:56.289 INFO:tasks.workunit.client.1.smithi157.stdout:1/299: creat d4/d17/f64 x:0 0 0 2024-04-22T23:34:56.290 INFO:tasks.workunit.client.1.smithi157.stdout:1/300: mkdir d4/d17/d34/d3b/d52/d65 0 2024-04-22T23:34:56.293 INFO:tasks.workunit.client.1.smithi157.stdout:1/301: rmdir d4/d17/d34 39 2024-04-22T23:34:56.298 INFO:tasks.workunit.client.1.smithi157.stdout:1/302: write d4/f25 [5380179,30009] 0 2024-04-22T23:34:56.299 INFO:tasks.workunit.client.1.smithi157.stdout:1/303: unlink d4/c51 0 2024-04-22T23:34:56.299 INFO:tasks.workunit.client.1.smithi157.stdout:1/304: readlink d4/db/d3d/l44 0 2024-04-22T23:34:56.300 INFO:tasks.workunit.client.1.smithi157.stdout:1/305: write d4/f26 [1202063,29309] 0 2024-04-22T23:34:56.300 INFO:tasks.workunit.client.1.smithi157.stdout:1/306: chown d4/d17/d1d/c22 0 1 2024-04-22T23:34:56.301 INFO:tasks.workunit.client.1.smithi157.stdout:1/307: creat d4/d17/d34/d3b/f66 x:0 0 0 2024-04-22T23:34:56.301 INFO:tasks.workunit.client.1.smithi157.stdout:1/308: dread - d4/db/f4a zero size 2024-04-22T23:34:56.302 INFO:tasks.workunit.client.1.smithi157.stdout:1/309: rename d4/d17/f3a to d4/d17/d1d/d33/d41/f67 0 2024-04-22T23:34:56.303 INFO:tasks.workunit.client.1.smithi157.stdout:1/310: mkdir d4/d17/d68 0 2024-04-22T23:34:56.304 INFO:tasks.workunit.client.1.smithi157.stdout:1/311: readlink d4/db/l40 0 2024-04-22T23:34:56.306 INFO:tasks.workunit.client.1.smithi157.stdout:1/312: getdents d4/db/d3d 0 2024-04-22T23:34:56.306 INFO:tasks.workunit.client.1.smithi157.stdout:1/313: getdents d4/d17/d68 0 2024-04-22T23:34:56.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.314+0000 7fb5ce02e700 1 -- 172.21.15.107:0/4119674625 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb5b4005da0 con 0x7fb5c80fce50 2024-04-22T23:34:56.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.314+0000 7fb5affff700 1 -- 172.21.15.107:0/4119674625 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1841 (secure 0 0 0) 0x7fb5b8056760 con 0x7fb5c80fce50 2024-04-22T23:34:56.309 INFO:teuthology.orchestra.run.smithi107.stdout:e18 2024-04-22T23:34:56.309 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T23:34:56.310 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:34:56.310 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-04-22T23:34:56.310 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:34:56.310 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-04-22T23:34:56.310 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-04-22T23:34:56.310 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 18 2024-04-22T23:34:56.310 INFO:teuthology.orchestra.run.smithi107.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-22T23:34:56.310 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-04-22T23:32:20.146130+0000 2024-04-22T23:34:56.310 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-04-22T23:34:31.577195+0000 2024-04-22T23:34:56.310 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-04-22T23:34:56.311 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-04-22T23:34:56.311 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-04-22T23:34:56.311 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-04-22T23:34:56.311 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-04-22T23:34:56.311 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-04-22T23:34:56.311 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-04-22T23:34:56.311 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 46 2024-04-22T23:34:56.311 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:34:56.311 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 2 2024-04-22T23:34:56.311 INFO:teuthology.orchestra.run.smithi107.stdout:in 0,1 2024-04-22T23:34:56.311 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=14468,1=24265} 2024-04-22T23:34:56.311 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-04-22T23:34:56.311 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-04-22T23:34:56.311 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-04-22T23:34:56.311 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-04-22T23:34:56.312 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-04-22T23:34:56.312 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data disabled 2024-04-22T23:34:56.312 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-04-22T23:34:56.312 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-04-22T23:34:56.312 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.ktuxjl{0:14468} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.107:6828/3335313253,v1:172.21.15.107:6829/3335313253] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:34:56.312 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi157.rcaris{1:24265} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.157:6824/2779251893,v1:172.21.15.157:6825/2779251893] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:34:56.312 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:34:56.312 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:34:56.312 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-04-22T23:34:56.312 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:34:56.312 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.dfvvkw{-1:14484} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/3548717247,v1:172.21.15.107:6827/3548717247] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:34:56.312 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi157.ufywse{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.157:6826/4072444199,v1:172.21.15.157:6827/4072444199] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:34:56.313 INFO:tasks.workunit.client.1.smithi157.stdout:1/314: dread d4/f5 [0,4194304] 0 2024-04-22T23:34:56.313 INFO:tasks.workunit.client.1.smithi157.stdout:1/315: creat d4/d17/d1d/d33/d41/f69 x:0 0 0 2024-04-22T23:34:56.314 INFO:tasks.workunit.client.1.smithi157.stdout:1/316: symlink d4/d17/d1d/d33/d41/l6a 0 2024-04-22T23:34:56.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.317+0000 7fb5ce02e700 1 -- 172.21.15.107:0/4119674625 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fb5b0069370 msgr2=0x7fb5b006b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:34:56.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.317+0000 7fb5ce02e700 1 --2- 172.21.15.107:0/4119674625 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fb5b0069370 0x7fb5b006b820 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc000c00 tx=0x7fb5bc009e30 comp rx=0 tx=0).stop 2024-04-22T23:34:56.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.317+0000 7fb5ce02e700 1 -- 172.21.15.107:0/4119674625 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5c80fce50 msgr2=0x7fb5c810b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:34:56.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.317+0000 7fb5ce02e700 1 --2- 172.21.15.107:0/4119674625 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5c80fce50 0x7fb5c810b9d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb5b800b790 tx=0x7fb5b8009220 comp rx=0 tx=0).stop 2024-04-22T23:34:56.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.317+0000 7fb5ce02e700 1 -- 172.21.15.107:0/4119674625 shutdown_connections 2024-04-22T23:34:56.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.317+0000 7fb5ce02e700 1 --2- 172.21.15.107:0/4119674625 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fb5b0069370 0x7fb5b006b820 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:56.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.317+0000 7fb5ce02e700 1 --2- 172.21.15.107:0/4119674625 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5c80fce50 0x7fb5c810b9d0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:56.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.317+0000 7fb5ce02e700 1 --2- 172.21.15.107:0/4119674625 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5c810bf10 0x7fb5c810cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:56.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.317+0000 7fb5ce02e700 1 -- 172.21.15.107:0/4119674625 >> 172.21.15.107:0/4119674625 conn(0x7fb5c80f8540 msgr2=0x7fb5c80f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:34:56.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.318+0000 7fb5ce02e700 1 -- 172.21.15.107:0/4119674625 shutdown_connections 2024-04-22T23:34:56.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.318+0000 7fb5ce02e700 1 -- 172.21.15.107:0/4119674625 wait complete. 2024-04-22T23:34:56.315 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 18 2024-04-22T23:34:56.316 INFO:tasks.workunit.client.1.smithi157.stdout:1/317: mkdir d4/d17/d6b 0 2024-04-22T23:34:56.320 INFO:tasks.workunit.client.1.smithi157.stdout:1/318: dread d4/d17/d34/d3b/d50/f56 [0,4194304] 0 2024-04-22T23:34:56.323 INFO:tasks.workunit.client.1.smithi157.stdout:1/319: rmdir d4/d17/d68 0 2024-04-22T23:34:56.329 INFO:tasks.workunit.client.1.smithi157.stdout:1/320: write d4/d17/f35 [1301086,51755] 0 2024-04-22T23:34:56.329 INFO:tasks.workunit.client.1.smithi157.stdout:1/321: chown d4/db/f2e 259384 1 2024-04-22T23:34:56.330 INFO:tasks.workunit.client.1.smithi157.stdout:1/322: symlink d4/d17/d1d/d33/d41/l6c 0 2024-04-22T23:34:56.331 INFO:tasks.workunit.client.1.smithi157.stdout:1/323: symlink d4/d17/d34/d3b/d52/d58/l6d 0 2024-04-22T23:34:56.332 INFO:tasks.workunit.client.1.smithi157.stdout:1/324: unlink d4/l10 0 2024-04-22T23:34:56.334 INFO:tasks.workunit.client.1.smithi157.stdout:8/176: sync 2024-04-22T23:34:56.334 INFO:tasks.workunit.client.1.smithi157.stdout:3/270: sync 2024-04-22T23:34:56.334 INFO:tasks.workunit.client.1.smithi157.stdout:3/271: dread - d0/d3/d34/d39/f18 zero size 2024-04-22T23:34:56.334 INFO:tasks.workunit.client.1.smithi157.stdout:5/353: sync 2024-04-22T23:34:56.334 INFO:tasks.workunit.client.1.smithi157.stdout:5/354: chown d1/d5/d7/d2f/d45 26643137 1 2024-04-22T23:34:56.335 INFO:tasks.workunit.client.1.smithi157.stdout:5/355: write d1/d27/d29/d4b/f5c [1172008,58463] 0 2024-04-22T23:34:56.336 INFO:tasks.workunit.client.1.smithi157.stdout:8/177: rename d2/d6/de/d17/c19 to d2/d6/de/d17/c3a 0 2024-04-22T23:34:56.338 INFO:tasks.workunit.client.1.smithi157.stdout:8/178: symlink d2/d4/d2c/l3b 0 2024-04-22T23:34:56.338 INFO:tasks.workunit.client.1.smithi157.stdout:8/179: chown d2/d6/d21/f2d 66 1 2024-04-22T23:34:56.339 INFO:tasks.workunit.client.1.smithi157.stdout:8/180: creat d2/d6/f3c x:0 0 0 2024-04-22T23:34:56.339 INFO:tasks.workunit.client.1.smithi157.stdout:8/181: truncate d2/d6/d21/f2e 1438980 0 2024-04-22T23:34:56.341 INFO:tasks.workunit.client.1.smithi157.stdout:8/182: mknod d2/d4/c3d 0 2024-04-22T23:34:56.344 INFO:tasks.workunit.client.1.smithi157.stdout:8/183: rename d2/d6/de/d11/l18 to d2/d4/d30/l3e 0 2024-04-22T23:34:56.350 INFO:tasks.workunit.client.1.smithi157.stdout:1/325: dwrite d4/d17/d34/f38 [0,4194304] 0 2024-04-22T23:34:56.350 INFO:tasks.workunit.client.1.smithi157.stdout:1/326: write d4/d17/d34/f49 [1037178,27418] 0 2024-04-22T23:34:56.351 INFO:tasks.workunit.client.1.smithi157.stdout:1/327: creat d4/f6e x:0 0 0 2024-04-22T23:34:56.356 INFO:tasks.workunit.client.1.smithi157.stdout:3/272: dwrite d0/d3/d34/d3d/f42 [0,4194304] 0 2024-04-22T23:34:56.356 INFO:tasks.workunit.client.1.smithi157.stdout:3/273: readlink d0/d3/d34/d39/l52 0 2024-04-22T23:34:56.359 INFO:tasks.workunit.client.1.smithi157.stdout:1/328: write d4/d17/f2d [321846,25519] 0 2024-04-22T23:34:56.359 INFO:tasks.workunit.client.1.smithi157.stdout:1/329: readlink d4/d5f/l62 0 2024-04-22T23:34:56.359 INFO:tasks.workunit.client.1.smithi157.stdout:1/330: dread - d4/d17/f59 zero size 2024-04-22T23:34:56.359 INFO:tasks.workunit.client.1.smithi157.stdout:1/331: write d4/db/f2e [153011,89023] 0 2024-04-22T23:34:56.359 INFO:tasks.workunit.client.1.smithi157.stdout:1/332: chown d4/db/f24 0 1 2024-04-22T23:34:56.359 INFO:tasks.workunit.client.1.smithi157.stdout:1/333: truncate d4/d17/f4d 315175 0 2024-04-22T23:34:56.361 INFO:tasks.workunit.client.1.smithi157.stdout:1/334: unlink d4/d17/d34/f4b 0 2024-04-22T23:34:56.361 INFO:tasks.workunit.client.1.smithi157.stdout:1/335: truncate d4/d17/d34/f38 4335212 0 2024-04-22T23:34:56.368 INFO:tasks.workunit.client.1.smithi157.stdout:5/356: dwrite d1/d27/d29/f2a [0,4194304] 0 2024-04-22T23:34:56.368 INFO:tasks.workunit.client.1.smithi157.stdout:5/357: readlink d1/d5/d7/d2f/d45/l4c 0 2024-04-22T23:34:56.369 INFO:tasks.workunit.client.1.smithi157.stdout:1/336: fdatasync d4/db/f2e 0 2024-04-22T23:34:56.369 INFO:tasks.workunit.client.1.smithi157.stdout:1/337: creat d4/db/f6f x:0 0 0 2024-04-22T23:34:56.370 INFO:tasks.workunit.client.1.smithi157.stdout:5/358: mknod d1/d27/d29/c8c 0 2024-04-22T23:34:56.370 INFO:tasks.workunit.client.1.smithi157.stdout:5/359: chown d1/d5 23 1 2024-04-22T23:34:56.370 INFO:tasks.workunit.client.1.smithi157.stdout:5/360: write d1/d5/d7/d2f/d45/d54/d59/f70 [746639,28951] 0 2024-04-22T23:34:56.371 INFO:tasks.workunit.client.1.smithi157.stdout:1/338: symlink d4/d17/d6b/l70 0 2024-04-22T23:34:56.371 INFO:tasks.workunit.client.1.smithi157.stdout:5/361: mknod d1/d5/d7/c8d 0 2024-04-22T23:34:56.371 INFO:tasks.workunit.client.1.smithi157.stdout:5/362: truncate d1/f5a 777275 0 2024-04-22T23:34:56.372 INFO:tasks.workunit.client.1.smithi157.stdout:1/339: dread d4/d17/f4d [0,4194304] 0 2024-04-22T23:34:56.372 INFO:tasks.workunit.client.1.smithi157.stdout:1/340: write d4/d17/d34/f49 [1574980,93511] 0 2024-04-22T23:34:56.372 INFO:tasks.workunit.client.1.smithi157.stdout:5/363: rmdir d1 39 2024-04-22T23:34:56.372 INFO:tasks.workunit.client.1.smithi157.stdout:5/364: chown d1/d5/d7/la 1374 1 2024-04-22T23:34:56.372 INFO:tasks.workunit.client.1.smithi157.stdout:8/184: dwrite d2/f32 [0,4194304] 0 2024-04-22T23:34:56.373 INFO:tasks.workunit.client.1.smithi157.stdout:8/185: readlink d2/d6/d21/l29 0 2024-04-22T23:34:56.373 INFO:tasks.workunit.client.1.smithi157.stdout:3/274: dwrite d0/d3/d13/d21/f3f [0,4194304] 0 2024-04-22T23:34:56.373 INFO:tasks.workunit.client.1.smithi157.stdout:1/341: mkdir d4/db/d71 0 2024-04-22T23:34:56.373 INFO:tasks.workunit.client.1.smithi157.stdout:1/342: write d4/fc [33011,67618] 0 2024-04-22T23:34:56.374 INFO:tasks.workunit.client.1.smithi157.stdout:5/365: rmdir d1/d5/d7/d2f/d51 39 2024-04-22T23:34:56.374 INFO:tasks.workunit.client.1.smithi157.stdout:5/366: readlink d1/d5/d7/d2f/d45/l6c 0 2024-04-22T23:34:56.376 INFO:tasks.workunit.client.1.smithi157.stdout:8/186: creat d2/d6/de/f3f x:0 0 0 2024-04-22T23:34:56.376 INFO:tasks.workunit.client.1.smithi157.stdout:8/187: truncate d2/d6/de/f1b 1689577 0 2024-04-22T23:34:56.376 INFO:tasks.workunit.client.1.smithi157.stdout:8/188: dread - d2/d6/de/f3f zero size 2024-04-22T23:34:56.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:34:56 smithi157 ceph-mon[33877]: from='client.14640 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:34:56.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:34:56 smithi157 ceph-mon[33877]: from='client.14644 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:34:56.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:34:56 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/2150163665' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:34:56.383 INFO:tasks.workunit.client.1.smithi157.stdout:3/275: rename d0/c37 to d0/d3/d34/d3d/c54 0 2024-04-22T23:34:56.383 INFO:tasks.workunit.client.1.smithi157.stdout:3/276: creat d0/f55 x:0 0 0 2024-04-22T23:34:56.383 INFO:tasks.workunit.client.1.smithi157.stdout:3/277: creat d0/d3/d34/d39/f56 x:0 0 0 2024-04-22T23:34:56.383 INFO:tasks.workunit.client.1.smithi157.stdout:3/278: fdatasync d0/d3/d34/d39/f32 0 2024-04-22T23:34:56.383 INFO:tasks.workunit.client.1.smithi157.stdout:3/279: readlink d0/d3/d34/d3d/l45 0 2024-04-22T23:34:56.383 INFO:tasks.workunit.client.1.smithi157.stdout:3/280: truncate d0/d7/f3a 470135 0 2024-04-22T23:34:56.386 INFO:tasks.workunit.client.1.smithi157.stdout:5/367: link d1/d27/l3a d1/d5/d7/d2f/l8e 0 2024-04-22T23:34:56.391 INFO:tasks.workunit.client.1.smithi157.stdout:3/281: mkdir d0/d3/d34/d3d/d57 0 2024-04-22T23:34:56.391 INFO:tasks.workunit.client.1.smithi157.stdout:3/282: creat d0/d3/d34/d39/f58 x:0 0 0 2024-04-22T23:34:56.391 INFO:tasks.workunit.client.1.smithi157.stdout:3/283: truncate d0/d7/f4f 665641 0 2024-04-22T23:34:56.391 INFO:tasks.workunit.client.1.smithi157.stdout:3/284: getdents d0/d3/d34/d3d/d57 0 2024-04-22T23:34:56.391 INFO:tasks.workunit.client.1.smithi157.stdout:3/285: dread - d0/f55 zero size 2024-04-22T23:34:56.391 INFO:tasks.workunit.client.1.smithi157.stdout:5/368: creat d1/d5/dc/d43/d4f/d60/d7a/f8f x:0 0 0 2024-04-22T23:34:56.394 INFO:tasks.workunit.client.1.smithi157.stdout:5/369: creat d1/f90 x:0 0 0 2024-04-22T23:34:56.395 INFO:tasks.workunit.client.1.smithi157.stdout:5/370: symlink d1/d1f/l91 0 2024-04-22T23:34:56.395 INFO:tasks.workunit.client.1.smithi157.stdout:5/371: fsync d1/d5/d7/d2f/f79 0 2024-04-22T23:34:56.397 INFO:tasks.workunit.client.1.smithi157.stdout:5/372: mknod d1/d5/d7/d10/c92 0 2024-04-22T23:34:56.397 INFO:tasks.workunit.client.1.smithi157.stdout:5/373: dread - d1/d5/f4e zero size 2024-04-22T23:34:56.397 INFO:tasks.workunit.client.1.smithi157.stdout:5/374: fsync d1/d5/f4e 0 2024-04-22T23:34:56.397 INFO:tasks.workunit.client.1.smithi157.stdout:5/375: chown d1/d1f/c73 13895706 1 2024-04-22T23:34:56.398 INFO:tasks.workunit.client.1.smithi157.stdout:1/343: dwrite d4/f18 [0,4194304] 0 2024-04-22T23:34:56.399 INFO:tasks.workunit.client.1.smithi157.stdout:1/344: link d4/d17/d1d/d33/d41/c4c d4/db/d3d/c72 0 2024-04-22T23:34:56.399 INFO:tasks.workunit.client.1.smithi157.stdout:1/345: truncate d4/db/f2e 1052288 0 2024-04-22T23:34:56.400 INFO:tasks.workunit.client.1.smithi157.stdout:1/346: mknod d4/d17/d34/d3b/d50/c73 0 2024-04-22T23:34:56.400 INFO:tasks.workunit.client.1.smithi157.stdout:1/347: read d4/db/f24 [18175,82517] 0 2024-04-22T23:34:56.401 INFO:tasks.workunit.client.1.smithi157.stdout:8/189: dwrite d2/d6/de/f3f [0,4194304] 0 2024-04-22T23:34:56.404 INFO:tasks.workunit.client.1.smithi157.stdout:8/190: mkdir d2/d6/de/d40 0 2024-04-22T23:34:56.404 INFO:tasks.workunit.client.1.smithi157.stdout:8/191: chown d2/d6/de/d17 1708 1 2024-04-22T23:34:56.405 INFO:tasks.workunit.client.1.smithi157.stdout:8/192: rename d2/d6/d21/f23 to d2/d34/f41 0 2024-04-22T23:34:56.406 INFO:tasks.workunit.client.1.smithi157.stdout:5/376: dread d1/d27/d29/d4b/f5c [0,4194304] 0 2024-04-22T23:34:56.409 INFO:tasks.workunit.client.1.smithi157.stdout:8/193: rename d2/d4/d2c to d2/d6/d21/d33/d42 0 2024-04-22T23:34:56.419 INFO:tasks.workunit.client.1.smithi157.stdout:3/286: dwrite d0/d3/d34/d39/f49 [0,4194304] 0 2024-04-22T23:34:56.421 INFO:tasks.workunit.client.1.smithi157.stdout:3/287: mkdir d0/d3/d13/d21/d4d/d59 0 2024-04-22T23:34:56.421 INFO:tasks.workunit.client.1.smithi157.stdout:3/288: stat d0/d3/d34/c43 0 2024-04-22T23:34:56.421 INFO:tasks.workunit.client.1.smithi157.stdout:3/289: getdents d0/d3/d34/d4a 0 2024-04-22T23:34:56.422 INFO:tasks.workunit.client.1.smithi157.stdout:3/290: rename d0/d3/d34/d3d/l45 to d0/d3/d34/d4a/l5a 0 2024-04-22T23:34:56.429 INFO:tasks.workunit.client.1.smithi157.stdout:1/348: dwrite d4/d17/f2d [0,4194304] 0 2024-04-22T23:34:56.429 INFO:tasks.workunit.client.1.smithi157.stdout:5/377: dwrite d1/d27/d29/f2a [0,4194304] 0 2024-04-22T23:34:56.430 INFO:tasks.workunit.client.1.smithi157.stdout:1/349: symlink d4/db/d71/l74 0 2024-04-22T23:34:56.431 INFO:tasks.workunit.client.1.smithi157.stdout:8/194: dwrite d2/d6/de/fd [0,4194304] 0 2024-04-22T23:34:56.431 INFO:tasks.workunit.client.1.smithi157.stdout:5/378: symlink d1/d5/d7/d2f/d45/d54/d59/l93 0 2024-04-22T23:34:56.431 INFO:tasks.workunit.client.1.smithi157.stdout:1/350: read d4/f1b [1475712,103867] 0 2024-04-22T23:34:56.431 INFO:tasks.workunit.client.1.smithi157.stdout:1/351: stat d4/db 0 2024-04-22T23:34:56.431 INFO:tasks.workunit.client.1.smithi157.stdout:1/352: chown d4/d17/d1d/d33/d41/l6a 3937 1 2024-04-22T23:34:56.433 INFO:tasks.workunit.client.1.smithi157.stdout:8/195: rmdir d2/d6/de/d40 0 2024-04-22T23:34:56.433 INFO:tasks.workunit.client.1.smithi157.stdout:8/196: fdatasync d2/d6/d21/f25 0 2024-04-22T23:34:56.433 INFO:tasks.workunit.client.1.smithi157.stdout:8/197: chown d2/d6/d21/l39 17 1 2024-04-22T23:34:56.433 INFO:tasks.workunit.client.1.smithi157.stdout:8/198: creat d2/f43 x:0 0 0 2024-04-22T23:34:56.440 INFO:tasks.workunit.client.1.smithi157.stdout:8/199: creat d2/d6/de/d11/f44 x:0 0 0 2024-04-22T23:34:56.451 INFO:tasks.workunit.client.1.smithi157.stdout:1/353: dwrite d4/db/f1e [0,4194304] 0 2024-04-22T23:34:56.451 INFO:tasks.workunit.client.1.smithi157.stdout:1/354: chown d4/d17/c2a 432334 1 2024-04-22T23:34:56.452 INFO:tasks.workunit.client.1.smithi157.stdout:1/355: unlink d4/d17/f2c 0 2024-04-22T23:34:56.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.458+0000 7fe6e2252700 1 -- 172.21.15.107:0/1316978073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6dc0fce60 msgr2=0x7fe6dc0fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:34:56.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.458+0000 7fe6e2252700 1 --2- 172.21.15.107:0/1316978073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6dc0fce60 0x7fe6dc0fd270 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc004940 tx=0x7fe6cc009e30 comp rx=0 tx=0).stop 2024-04-22T23:34:56.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.458+0000 7fe6e2252700 1 -- 172.21.15.107:0/1316978073 shutdown_connections 2024-04-22T23:34:56.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.458+0000 7fe6e2252700 1 --2- 172.21.15.107:0/1316978073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6dc0fe060 0x7fe6dc0fe4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:56.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.458+0000 7fe6e2252700 1 --2- 172.21.15.107:0/1316978073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6dc0fce60 0x7fe6dc0fd270 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:56.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.458+0000 7fe6e2252700 1 -- 172.21.15.107:0/1316978073 >> 172.21.15.107:0/1316978073 conn(0x7fe6dc0f8570 msgr2=0x7fe6dc0fa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:34:56.454 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:34:56 smithi107 ceph-mon[26355]: from='client.14640 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:34:56.454 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:34:56 smithi107 ceph-mon[26355]: from='client.14644 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:34:56.454 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:34:56 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/2150163665' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:34:56.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.459+0000 7fe6e2252700 1 -- 172.21.15.107:0/1316978073 shutdown_connections 2024-04-22T23:34:56.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.459+0000 7fe6e2252700 1 -- 172.21.15.107:0/1316978073 wait complete. 2024-04-22T23:34:56.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.459+0000 7fe6e2252700 1 Processor -- start 2024-04-22T23:34:56.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.459+0000 7fe6e2252700 1 -- start start 2024-04-22T23:34:56.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.460+0000 7fe6e2252700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6dc0fce60 0x7fe6dc10d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:34:56.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.460+0000 7fe6e2252700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6dc0fe060 0x7fe6dc10df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:34:56.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.460+0000 7fe6e2252700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6dc10e520 con 0x7fe6dc0fe060 2024-04-22T23:34:56.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.460+0000 7fe6e2252700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6dc10ed70 con 0x7fe6dc0fce60 2024-04-22T23:34:56.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.460+0000 7fe6daffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6dc0fe060 0x7fe6dc10df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:34:56.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.460+0000 7fe6db7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6dc0fce60 0x7fe6dc10d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:34:56.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.460+0000 7fe6daffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6dc0fe060 0x7fe6dc10df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51248/0 (socket says 172.21.15.107:51248) 2024-04-22T23:34:56.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.460+0000 7fe6db7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6dc0fce60 0x7fe6dc10d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.107:36724/0 (socket says 172.21.15.107:36724) 2024-04-22T23:34:56.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.460+0000 7fe6daffd700 1 -- 172.21.15.107:0/3424129359 learned_addr learned my addr 172.21.15.107:0/3424129359 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:34:56.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.460+0000 7fe6db7fe700 1 -- 172.21.15.107:0/3424129359 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6dc0fe060 msgr2=0x7fe6dc10df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:34:56.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.460+0000 7fe6db7fe700 1 --2- 172.21.15.107:0/3424129359 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6dc0fe060 0x7fe6dc10df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:56.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.460+0000 7fe6db7fe700 1 -- 172.21.15.107:0/3424129359 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6cc005040 con 0x7fe6dc0fce60 2024-04-22T23:34:56.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.461+0000 7fe6db7fe700 1 --2- 172.21.15.107:0/3424129359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6dc0fce60 0x7fe6dc10d9e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc004940 tx=0x7fe6cc009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:34:56.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.461+0000 7fe6c3fff700 1 -- 172.21.15.107:0/3424129359 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6cc00e040 con 0x7fe6dc0fce60 2024-04-22T23:34:56.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.461+0000 7fe6e2252700 1 -- 172.21.15.107:0/3424129359 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6dc10eff0 con 0x7fe6dc0fce60 2024-04-22T23:34:56.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.461+0000 7fe6e2252700 1 -- 172.21.15.107:0/3424129359 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6dc10f530 con 0x7fe6dc0fce60 2024-04-22T23:34:56.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.461+0000 7fe6c3fff700 1 -- 172.21.15.107:0/3424129359 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe6cc0091a0 con 0x7fe6dc0fce60 2024-04-22T23:34:56.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.461+0000 7fe6c3fff700 1 -- 172.21.15.107:0/3424129359 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6cc013b70 con 0x7fe6dc0fce60 2024-04-22T23:34:56.458 INFO:tasks.workunit.client.1.smithi157.stdout:1/356: truncate d4/db/f14 3407847 0 2024-04-22T23:34:56.458 INFO:tasks.workunit.client.1.smithi157.stdout:1/357: write d4/db/f2e [446267,543] 0 2024-04-22T23:34:56.459 INFO:tasks.workunit.client.1.smithi157.stdout:1/358: mkdir d4/d17/d34/d75 0 2024-04-22T23:34:56.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.462+0000 7fe6e2252700 1 -- 172.21.15.107:0/3424129359 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6dc05f5f0 con 0x7fe6dc0fce60 2024-04-22T23:34:56.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.463+0000 7fe6c3fff700 1 -- 172.21.15.107:0/3424129359 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fe6cc01c430 con 0x7fe6dc0fce60 2024-04-22T23:34:56.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.463+0000 7fe6c3fff700 1 --2- 172.21.15.107:0/3424129359 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fe6c40692a0 0x7fe6c406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:34:56.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.464+0000 7fe6c3fff700 1 -- 172.21.15.107:0/3424129359 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fe6cc087dd0 con 0x7fe6dc0fce60 2024-04-22T23:34:56.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.464+0000 7fe6daffd700 1 --2- 172.21.15.107:0/3424129359 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fe6c40692a0 0x7fe6c406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:34:56.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.464+0000 7fe6daffd700 1 --2- 172.21.15.107:0/3424129359 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fe6c40692a0 0x7fe6c406b750 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0004bc0 tx=0x7fe6d000b040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:34:56.464 INFO:tasks.workunit.client.1.smithi157.stdout:2/362: sync 2024-04-22T23:34:56.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.469+0000 7fe6c3fff700 1 -- 172.21.15.107:0/3424129359 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe6cc057ca0 con 0x7fe6dc0fce60 2024-04-22T23:34:56.465 INFO:tasks.workunit.client.1.smithi157.stdout:8/200: getdents d2/d6/de/d11 0 2024-04-22T23:34:56.465 INFO:tasks.workunit.client.1.smithi157.stdout:8/201: chown d2 27 1 2024-04-22T23:34:56.467 INFO:tasks.workunit.client.1.smithi157.stdout:8/202: truncate d2/f3 889137 0 2024-04-22T23:34:56.469 INFO:tasks.workunit.client.1.smithi157.stdout:8/203: unlink d2/f43 0 2024-04-22T23:34:56.471 INFO:tasks.workunit.client.1.smithi157.stdout:8/204: link d2/d6/d21/d33/d42/l3b d2/l45 0 2024-04-22T23:34:56.471 INFO:tasks.workunit.client.1.smithi157.stdout:8/205: creat d2/d6/f46 x:0 0 0 2024-04-22T23:34:56.471 INFO:tasks.workunit.client.1.smithi157.stdout:8/206: dread - d2/d6/de/d11/f44 zero size 2024-04-22T23:34:56.473 INFO:tasks.workunit.client.1.smithi157.stdout:8/207: mknod d2/d6/c47 0 2024-04-22T23:34:56.473 INFO:tasks.workunit.client.1.smithi157.stdout:8/208: creat d2/f48 x:0 0 0 2024-04-22T23:34:56.475 INFO:tasks.workunit.client.1.smithi157.stdout:1/359: dwrite d4/d17/f59 [0,4194304] 0 2024-04-22T23:34:56.477 INFO:tasks.workunit.client.1.smithi157.stdout:1/360: creat d4/d17/d34/d3b/f76 x:0 0 0 2024-04-22T23:34:56.482 INFO:tasks.workunit.client.1.smithi157.stdout:2/363: dwrite d7/f3b [0,4194304] 0 2024-04-22T23:34:56.509 INFO:tasks.workunit.client.1.smithi157.stdout:1/361: dwrite d4/db/f2e [0,4194304] 0 2024-04-22T23:34:56.509 INFO:tasks.workunit.client.1.smithi157.stdout:2/364: dwrite d7/f13 [4194304,4194304] 0 2024-04-22T23:34:56.511 INFO:tasks.workunit.client.1.smithi157.stdout:2/365: truncate d7/d8/d25/f50 3377885 0 2024-04-22T23:34:56.513 INFO:tasks.workunit.client.1.smithi157.stdout:2/366: creat d7/d8/d30/d23/f7f x:0 0 0 2024-04-22T23:34:56.515 INFO:tasks.workunit.client.1.smithi157.stdout:2/367: mkdir d7/d8/d30/d23/d27/d5f/d80 0 2024-04-22T23:34:56.517 INFO:tasks.workunit.client.1.smithi157.stdout:2/368: mknod d7/d8/d6b/c81 0 2024-04-22T23:34:56.517 INFO:tasks.workunit.client.1.smithi157.stdout:2/369: creat d7/d1f/f82 x:0 0 0 2024-04-22T23:34:56.519 INFO:tasks.workunit.client.1.smithi157.stdout:2/370: creat d7/d8/d30/f83 x:0 0 0 2024-04-22T23:34:56.520 INFO:tasks.workunit.client.1.smithi157.stdout:2/371: mknod d7/d8/dd/d4d/c84 0 2024-04-22T23:34:56.521 INFO:tasks.workunit.client.1.smithi157.stdout:2/372: truncate d7/d8/f34 1080908 0 2024-04-22T23:34:56.522 INFO:tasks.workunit.client.1.smithi157.stdout:2/373: write d7/d8/d25/f4f [4553210,2607] 0 2024-04-22T23:34:56.523 INFO:tasks.workunit.client.1.smithi157.stdout:2/374: mkdir d7/d85 0 2024-04-22T23:34:56.523 INFO:tasks.workunit.client.1.smithi157.stdout:2/375: chown d7/d8/d25/d55 318533 1 2024-04-22T23:34:56.529 INFO:tasks.workunit.client.1.smithi157.stdout:1/362: dwrite d4/d17/d1d/d33/d41/f5e [0,4194304] 0 2024-04-22T23:34:56.529 INFO:tasks.workunit.client.1.smithi157.stdout:1/363: stat d4/d17/d34/d3b/d52/d58 0 2024-04-22T23:34:56.537 INFO:tasks.workunit.client.1.smithi157.stdout:1/364: rmdir d4 39 2024-04-22T23:34:56.537 INFO:tasks.workunit.client.1.smithi157.stdout:1/365: read d4/f1b [343219,39402] 0 2024-04-22T23:34:56.537 INFO:tasks.workunit.client.1.smithi157.stdout:1/366: chown d4/d17/d34/d3b/d52/d58/l6d 2399240 1 2024-04-22T23:34:56.538 INFO:tasks.workunit.client.1.smithi157.stdout:1/367: mkdir d4/d77 0 2024-04-22T23:34:56.538 INFO:tasks.workunit.client.1.smithi157.stdout:1/368: chown d4/d17/d34/f63 8907 1 2024-04-22T23:34:56.538 INFO:tasks.workunit.client.1.smithi157.stdout:1/369: symlink d4/db/d71/l78 0 2024-04-22T23:34:56.540 INFO:tasks.workunit.client.1.smithi157.stdout:2/376: dwrite d7/d8/f2f [0,4194304] 0 2024-04-22T23:34:56.542 INFO:tasks.workunit.client.1.smithi157.stdout:1/370: write d4/f5 [698731,91902] 0 2024-04-22T23:34:56.542 INFO:tasks.workunit.client.1.smithi157.stdout:2/377: link d7/d1f/d26/c46 d7/d1f/d26/d4a/d54/c86 0 2024-04-22T23:34:56.544 INFO:tasks.workunit.client.1.smithi157.stdout:2/378: symlink d7/l87 0 2024-04-22T23:34:56.559 INFO:tasks.workunit.client.1.smithi157.stdout:1/371: dwrite d4/d17/f64 [0,4194304] 0 2024-04-22T23:34:56.559 INFO:tasks.workunit.client.1.smithi157.stdout:1/372: chown d4/db/fe 63 1 2024-04-22T23:34:56.559 INFO:tasks.workunit.client.1.smithi157.stdout:2/379: dwrite d7/d8/d30/d23/d27/f7b [0,4194304] 0 2024-04-22T23:34:56.559 INFO:tasks.workunit.client.1.smithi157.stdout:2/380: creat d7/d8/d30/d23/d27/d5f/f88 x:0 0 0 2024-04-22T23:34:56.569 INFO:tasks.workunit.client.1.smithi157.stdout:0/273: sync 2024-04-22T23:34:56.569 INFO:tasks.workunit.client.1.smithi157.stdout:7/241: sync 2024-04-22T23:34:56.569 INFO:tasks.workunit.client.1.smithi157.stdout:7/242: chown d0/d9/db 588532 1 2024-04-22T23:34:56.569 INFO:tasks.workunit.client.1.smithi157.stdout:7/243: write d0/d9/de/f23 [493264,46721] 0 2024-04-22T23:34:56.573 INFO:tasks.workunit.client.1.smithi157.stdout:7/244: link d0/l3 d0/d9/d29/d49/d4a/l53 0 2024-04-22T23:34:56.573 INFO:tasks.workunit.client.1.smithi157.stdout:7/245: chown d0/d9/db/c47 2360104 1 2024-04-22T23:34:56.578 INFO:tasks.workunit.client.1.smithi157.stdout:7/246: unlink d0/d9/d10/l2e 0 2024-04-22T23:34:56.593 INFO:tasks.workunit.client.1.smithi157.stdout:7/247: dwrite d0/f2 [0,4194304] 0 2024-04-22T23:34:56.593 INFO:tasks.workunit.client.1.smithi157.stdout:7/248: fdatasync d0/f46 0 2024-04-22T23:34:56.594 INFO:tasks.workunit.client.1.smithi157.stdout:1/373: dwrite d4/f1b [0,4194304] 0 2024-04-22T23:34:56.595 INFO:tasks.workunit.client.1.smithi157.stdout:2/381: dwrite d7/d1f/d26/f44 [0,4194304] 0 2024-04-22T23:34:56.595 INFO:tasks.workunit.client.1.smithi157.stdout:0/274: dwrite d17/d2b/f5c [0,4194304] 0 2024-04-22T23:34:56.595 INFO:tasks.workunit.client.1.smithi157.stdout:0/275: fdatasync d17/f22 0 2024-04-22T23:34:56.595 INFO:tasks.workunit.client.1.smithi157.stdout:0/276: write d17/d2b/d52/f42 [998496,92629] 0 2024-04-22T23:34:56.595 INFO:tasks.workunit.client.1.smithi157.stdout:0/277: write d17/d2b/d52/f3f [1777516,120379] 0 2024-04-22T23:34:56.596 INFO:tasks.workunit.client.1.smithi157.stdout:7/249: rename d0/d9/db/df/l2c to d0/d9/de/l54 0 2024-04-22T23:34:56.597 INFO:tasks.workunit.client.1.smithi157.stdout:1/374: truncate d4/f16 2205673 0 2024-04-22T23:34:56.599 INFO:tasks.workunit.client.1.smithi157.stdout:2/382: creat d7/d8/d6b/f89 x:0 0 0 2024-04-22T23:34:56.601 INFO:tasks.workunit.client.1.smithi157.stdout:7/250: mkdir d0/d9/d42/d55 0 2024-04-22T23:34:56.601 INFO:tasks.workunit.client.1.smithi157.stdout:7/251: truncate d0/d9/d29/f31 1283574 0 2024-04-22T23:34:56.602 INFO:tasks.workunit.client.1.smithi157.stdout:1/375: symlink d4/db/d45/l79 0 2024-04-22T23:34:56.606 INFO:tasks.workunit.client.1.smithi157.stdout:1/376: dread d4/f5 [0,4194304] 0 2024-04-22T23:34:56.606 INFO:tasks.workunit.client.1.smithi157.stdout:1/377: chown d4/d17/d1d/d33/d41/d54 442686639 1 2024-04-22T23:34:56.608 INFO:tasks.workunit.client.1.smithi157.stdout:2/383: mkdir d7/d8/d30/d5b/d8a 0 2024-04-22T23:34:56.609 INFO:tasks.workunit.client.1.smithi157.stdout:7/252: mkdir d0/d9/d42/d55/d56 0 2024-04-22T23:34:56.612 INFO:tasks.workunit.client.1.smithi157.stdout:1/378: link d4/db/d3d/l44 d4/d17/l7a 0 2024-04-22T23:34:56.614 INFO:tasks.workunit.client.1.smithi157.stdout:2/384: link d7/f3b d7/d8/d6b/f8b 0 2024-04-22T23:34:56.615 INFO:tasks.workunit.client.1.smithi157.stdout:7/253: read d0/d9/d10/f12 [3299251,104347] 0 2024-04-22T23:34:56.617 INFO:tasks.workunit.client.1.smithi157.stdout:1/379: creat d4/d17/d34/d3b/d50/f7b x:0 0 0 2024-04-22T23:34:56.619 INFO:tasks.workunit.client.1.smithi157.stdout:2/385: rename d7/d8/f34 to d7/d8/d6b/f8c 0 2024-04-22T23:34:56.619 INFO:tasks.workunit.client.1.smithi157.stdout:2/386: fdatasync d7/d8/d30/d5b/f72 0 2024-04-22T23:34:56.621 INFO:tasks.workunit.client.1.smithi157.stdout:1/380: mknod d4/db/d71/c7c 0 2024-04-22T23:34:56.621 INFO:tasks.workunit.client.1.smithi157.stdout:1/381: stat d4/db/f1e 0 2024-04-22T23:34:56.621 INFO:tasks.workunit.client.1.smithi157.stdout:1/382: dread d4/d17/f4d [0,4194304] 0 2024-04-22T23:34:56.623 INFO:tasks.workunit.client.1.smithi157.stdout:2/387: creat d7/d8/d30/d23/d27/f8d x:0 0 0 2024-04-22T23:34:56.628 INFO:tasks.workunit.client.1.smithi157.stdout:2/388: dread d7/d8/d30/d23/f48 [0,4194304] 0 2024-04-22T23:34:56.635 INFO:tasks.workunit.client.1.smithi157.stdout:7/254: dwrite d0/d9/db/df/f2b [4194304,4194304] 0 2024-04-22T23:34:56.636 INFO:tasks.workunit.client.1.smithi157.stdout:1/383: dwrite d4/f5 [0,4194304] 0 2024-04-22T23:34:56.637 INFO:tasks.workunit.client.1.smithi157.stdout:1/384: fdatasync d4/d17/d34/d3b/d50/f56 0 2024-04-22T23:34:56.637 INFO:tasks.workunit.client.1.smithi157.stdout:7/255: creat d0/d9/db/f57 x:0 0 0 2024-04-22T23:34:56.637 INFO:tasks.workunit.client.1.smithi157.stdout:7/256: chown d0/d9/d36 2032 1 2024-04-22T23:34:56.637 INFO:tasks.workunit.client.1.smithi157.stdout:7/257: readlink d0/d9/l1a 0 2024-04-22T23:34:56.639 INFO:tasks.workunit.client.1.smithi157.stdout:7/258: truncate d0/d9/f1d 620213 0 2024-04-22T23:34:56.640 INFO:tasks.workunit.client.1.smithi157.stdout:7/259: symlink d0/d9/d42/d55/d56/l58 0 2024-04-22T23:34:56.643 INFO:tasks.workunit.client.1.smithi157.stdout:1/385: dread d4/d17/d34/f38 [0,4194304] 0 2024-04-22T23:34:56.643 INFO:tasks.workunit.client.1.smithi157.stdout:2/389: rmdir d7/d8/d30/d23/d27 39 2024-04-22T23:34:56.646 INFO:tasks.workunit.client.1.smithi157.stdout:2/390: unlink d7/d1f/f4c 0 2024-04-22T23:34:56.648 INFO:tasks.workunit.client.1.smithi157.stdout:2/391: rename d7/d8/d30/d23/d27/d5f to d7/d8/d8e 0 2024-04-22T23:34:56.650 INFO:tasks.workunit.client.1.smithi157.stdout:2/392: rename d7/d1f/d26/d70 to d7/d1f/d26/d8f 0 2024-04-22T23:34:56.652 INFO:tasks.workunit.client.1.smithi157.stdout:1/386: write d4/d17/d1d/d33/d41/f67 [1913146,127954] 0 2024-04-22T23:34:56.653 INFO:tasks.workunit.client.1.smithi157.stdout:2/393: mkdir d7/d8/dd/d90 0 2024-04-22T23:34:56.653 INFO:tasks.workunit.client.1.smithi157.stdout:2/394: readlink d7/d1f/d3e/l7c 0 2024-04-22T23:34:56.653 INFO:tasks.workunit.client.1.smithi157.stdout:2/395: write d7/d8/d6b/f89 [259668,56298] 0 2024-04-22T23:34:56.654 INFO:tasks.workunit.client.1.smithi157.stdout:1/387: write d4/d17/d34/d3b/f42 [234712,4025] 0 2024-04-22T23:34:56.654 INFO:tasks.workunit.client.1.smithi157.stdout:2/396: mknod d7/d1f/d26/c91 0 2024-04-22T23:34:56.654 INFO:tasks.workunit.client.1.smithi157.stdout:2/397: write d7/d1f/d26/d4a/f77 [776551,125547] 0 2024-04-22T23:34:56.654 INFO:tasks.workunit.client.1.smithi157.stdout:2/398: chown d7/d8/d25/f3a 24492 1 2024-04-22T23:34:56.655 INFO:tasks.workunit.client.1.smithi157.stdout:1/388: creat d4/db/f7d x:0 0 0 2024-04-22T23:34:56.655 INFO:tasks.workunit.client.1.smithi157.stdout:1/389: chown d4/d17/d1d/d33/d41/d54/f5b 43152657 1 2024-04-22T23:34:56.655 INFO:tasks.workunit.client.1.smithi157.stdout:2/399: mkdir d7/d1f/d26/d4a/d54/d92 0 2024-04-22T23:34:56.657 INFO:tasks.workunit.client.1.smithi157.stdout:7/260: dwrite d0/d9/db/f15 [0,4194304] 0 2024-04-22T23:34:56.659 INFO:tasks.workunit.client.1.smithi157.stdout:2/400: fdatasync d7/d8/d6b/f89 0 2024-04-22T23:34:56.659 INFO:tasks.workunit.client.1.smithi157.stdout:2/401: chown d7/d8/d30/d23/d27/f7b 4841832 1 2024-04-22T23:34:56.659 INFO:tasks.workunit.client.1.smithi157.stdout:7/261: creat d0/d9/d29/d49/d4a/f59 x:0 0 0 2024-04-22T23:34:56.661 INFO:tasks.workunit.client.1.smithi157.stdout:7/262: link d0/d9/f3c d0/d9/db/f5a 0 2024-04-22T23:34:56.662 INFO:tasks.workunit.client.1.smithi157.stdout:1/390: dread d4/d17/d1d/d33/d41/f5e [0,4194304] 0 2024-04-22T23:34:56.662 INFO:tasks.workunit.client.1.smithi157.stdout:1/391: chown d4/d17/d34/d3b/d52/d58 113934840 1 2024-04-22T23:34:56.662 INFO:tasks.workunit.client.1.smithi157.stdout:7/263: dread d0/d9/db/f39 [0,4194304] 0 2024-04-22T23:34:56.664 INFO:tasks.workunit.client.1.smithi157.stdout:7/264: dread d0/d9/de/f23 [0,4194304] 0 2024-04-22T23:34:56.664 INFO:tasks.workunit.client.1.smithi157.stdout:7/265: readlink d0/l3 0 2024-04-22T23:34:56.664 INFO:tasks.workunit.client.1.smithi157.stdout:7/266: stat d0/d9/d42/d55 0 2024-04-22T23:34:56.665 INFO:tasks.workunit.client.1.smithi157.stdout:1/392: mkdir d4/d5f/d7e 0 2024-04-22T23:34:56.668 INFO:tasks.workunit.client.1.smithi157.stdout:2/402: rmdir d7/d1f/d26 39 2024-04-22T23:34:56.668 INFO:tasks.workunit.client.1.smithi157.stdout:2/403: chown d7/d8/d30/l20 1919218 1 2024-04-22T23:34:56.671 INFO:tasks.workunit.client.1.smithi157.stdout:1/393: rmdir d4/d17/d34/d3b/d52 39 2024-04-22T23:34:56.671 INFO:tasks.workunit.client.1.smithi157.stdout:1/394: chown d4/d17/d34/f49 41291 1 2024-04-22T23:34:56.673 INFO:tasks.workunit.client.1.smithi157.stdout:2/404: unlink d7/d8/dd/f31 0 2024-04-22T23:34:56.673 INFO:tasks.workunit.client.1.smithi157.stdout:2/405: readlink d7/d8/l62 0 2024-04-22T23:34:56.674 INFO:tasks.workunit.client.1.smithi157.stdout:7/267: dread d0/f4 [0,4194304] 0 2024-04-22T23:34:56.674 INFO:tasks.workunit.client.1.smithi157.stdout:7/268: fdatasync d0/f2 0 2024-04-22T23:34:56.674 INFO:tasks.workunit.client.1.smithi157.stdout:7/269: write d0/d9/db/f5a [1183653,82827] 0 2024-04-22T23:34:56.674 INFO:tasks.workunit.client.1.smithi157.stdout:7/270: fsync d0/f3e 0 2024-04-22T23:34:56.675 INFO:tasks.workunit.client.1.smithi157.stdout:2/406: link d7/d8/d6b/c81 d7/d1f/c93 0 2024-04-22T23:34:56.675 INFO:tasks.workunit.client.1.smithi157.stdout:7/271: symlink d0/d9/d29/d33/l5b 0 2024-04-22T23:34:56.675 INFO:tasks.workunit.client.1.smithi157.stdout:7/272: chown d0/d9/l11 608599496 1 2024-04-22T23:34:56.676 INFO:tasks.workunit.client.1.smithi157.stdout:2/407: mkdir d7/d8/d8e/d94 0 2024-04-22T23:34:56.679 INFO:tasks.workunit.client.1.smithi157.stdout:2/408: rmdir d7/d8/dd/d90 0 2024-04-22T23:34:56.680 INFO:tasks.workunit.client.1.smithi157.stdout:2/409: fsync d7/d1f/d26/f74 0 2024-04-22T23:34:56.680 INFO:tasks.workunit.client.1.smithi157.stdout:2/410: dread - d7/d1f/d3e/f5e zero size 2024-04-22T23:34:56.683 INFO:tasks.workunit.client.1.smithi157.stdout:7/273: write d0/f2 [2861126,68188] 0 2024-04-22T23:34:56.691 INFO:tasks.workunit.client.1.smithi157.stdout:1/395: dwrite d4/f18 [0,4194304] 0 2024-04-22T23:34:56.692 INFO:tasks.workunit.client.1.smithi157.stdout:1/396: creat d4/d17/d34/d3b/d52/f7f x:0 0 0 2024-04-22T23:34:56.693 INFO:tasks.workunit.client.1.smithi157.stdout:7/274: dread d0/f7 [0,4194304] 0 2024-04-22T23:34:56.693 INFO:tasks.workunit.client.1.smithi157.stdout:1/397: symlink d4/d17/d6b/l80 0 2024-04-22T23:34:56.695 INFO:tasks.workunit.client.1.smithi157.stdout:7/275: symlink d0/d9/db/d35/l5c 0 2024-04-22T23:34:56.696 INFO:tasks.workunit.client.1.smithi157.stdout:1/398: link d4/db/f13 d4/db/d45/f81 0 2024-04-22T23:34:56.697 INFO:tasks.workunit.client.1.smithi157.stdout:2/411: dwrite d7/d8/d30/d23/f7f [0,4194304] 0 2024-04-22T23:34:56.698 INFO:tasks.workunit.client.1.smithi157.stdout:2/412: dread - d7/d8/d8e/f7e zero size 2024-04-22T23:34:56.698 INFO:tasks.workunit.client.1.smithi157.stdout:2/413: stat d7 0 2024-04-22T23:34:56.698 INFO:tasks.workunit.client.1.smithi157.stdout:2/414: write d7/d8/d30/f76 [1495573,22059] 0 2024-04-22T23:34:56.699 INFO:tasks.workunit.client.1.smithi157.stdout:7/276: unlink d0/d9/db/c28 0 2024-04-22T23:34:56.700 INFO:tasks.workunit.client.1.smithi157.stdout:7/277: write d0/d9/db/f5a [2304104,93729] 0 2024-04-22T23:34:56.701 INFO:tasks.workunit.client.1.smithi157.stdout:1/399: link d4/d17/f2d d4/db/d3d/f82 0 2024-04-22T23:34:56.706 INFO:tasks.workunit.client.1.smithi157.stdout:7/278: creat d0/d9/d10/f5d x:0 0 0 2024-04-22T23:34:56.707 INFO:tasks.workunit.client.1.smithi157.stdout:2/415: write d7/d8/f2f [1739290,50532] 0 2024-04-22T23:34:56.707 INFO:tasks.workunit.client.1.smithi157.stdout:2/416: fsync d7/d8/d30/d5b/f73 0 2024-04-22T23:34:56.707 INFO:tasks.workunit.client.1.smithi157.stdout:7/279: mkdir d0/d9/d29/d5e 0 2024-04-22T23:34:56.708 INFO:tasks.workunit.client.1.smithi157.stdout:1/400: write d4/f26 [3129372,66573] 0 2024-04-22T23:34:56.709 INFO:tasks.workunit.client.1.smithi157.stdout:2/417: rename d7/d1f/d26/d4a/d54 to d7/d8/d95 0 2024-04-22T23:34:56.711 INFO:tasks.workunit.client.1.smithi157.stdout:1/401: truncate d4/db/d3d/f82 2533425 0 2024-04-22T23:34:56.714 INFO:tasks.workunit.client.1.smithi157.stdout:7/280: link d0/d9/d10/d43/f44 d0/d9/de/f5f 0 2024-04-22T23:34:56.715 INFO:tasks.workunit.client.1.smithi157.stdout:1/402: symlink d4/d17/d34/l83 0 2024-04-22T23:34:56.717 INFO:tasks.workunit.client.1.smithi157.stdout:7/281: mkdir d0/d9/db/df/d60 0 2024-04-22T23:34:56.731 INFO:tasks.workunit.client.1.smithi157.stdout:2/418: dwrite d7/d1f/d26/d4a/f60 [4194304,4194304] 0 2024-04-22T23:34:56.733 INFO:tasks.workunit.client.1.smithi157.stdout:9/346: sync 2024-04-22T23:34:56.733 INFO:tasks.workunit.client.1.smithi157.stdout:4/422: sync 2024-04-22T23:34:56.734 INFO:tasks.workunit.client.1.smithi157.stdout:2/419: rename d7/d1f/f56 to d7/d1f/d3e/f96 0 2024-04-22T23:34:56.735 INFO:tasks.workunit.client.1.smithi157.stdout:2/420: dread - d7/d8/d30/f83 zero size 2024-04-22T23:34:56.735 INFO:tasks.workunit.client.1.smithi157.stdout:2/421: truncate d7/d8/d30/f83 267708 0 2024-04-22T23:34:56.736 INFO:tasks.workunit.client.1.smithi157.stdout:4/423: unlink d3/d11/d30/d28/c4e 0 2024-04-22T23:34:56.736 INFO:tasks.workunit.client.1.smithi157.stdout:4/424: write d3/d11/d1d/d73/d80/f82 [377215,94711] 0 2024-04-22T23:34:56.738 INFO:tasks.workunit.client.1.smithi157.stdout:9/347: rename d4/fc to d4/df/d61/f66 0 2024-04-22T23:34:56.739 INFO:tasks.workunit.client.1.smithi157.stdout:4/425: mknod d3/d5c/d7b/c88 0 2024-04-22T23:34:56.741 INFO:tasks.workunit.client.1.smithi157.stdout:1/403: dwrite d4/db/f14 [0,4194304] 0 2024-04-22T23:34:56.742 INFO:tasks.workunit.client.1.smithi157.stdout:7/282: dwrite d0/d9/d29/d49/d4a/f59 [0,4194304] 0 2024-04-22T23:34:56.742 INFO:tasks.workunit.client.1.smithi157.stdout:2/422: rename d7/d8/d30/d23/d27/f8d to d7/d8/d6b/f97 0 2024-04-22T23:34:56.744 INFO:tasks.workunit.client.1.smithi157.stdout:9/348: mkdir d4/d15/d23/d67 0 2024-04-22T23:34:56.745 INFO:tasks.workunit.client.1.smithi157.stdout:1/404: mkdir d4/d17/d34/d84 0 2024-04-22T23:34:56.752 INFO:tasks.workunit.client.1.smithi157.stdout:9/349: write d4/df/f26 [497785,111752] 0 2024-04-22T23:34:56.752 INFO:tasks.workunit.client.1.smithi157.stdout:4/426: rename d3/d11/d30/d36/d62/d4c/f61 to d3/d11/d1d/d73/d80/f89 0 2024-04-22T23:34:56.752 INFO:tasks.workunit.client.1.smithi157.stdout:4/427: write d3/d11/d30/d28/d31/f69 [2279430,29724] 0 2024-04-22T23:34:56.755 INFO:tasks.workunit.client.1.smithi157.stdout:2/423: symlink d7/d8/d8e/l98 0 2024-04-22T23:34:56.755 INFO:tasks.workunit.client.1.smithi157.stdout:2/424: truncate d7/d8/d6b/f97 884479 0 2024-04-22T23:34:56.756 INFO:tasks.workunit.client.1.smithi157.stdout:1/405: creat d4/d17/d1d/d33/d41/f85 x:0 0 0 2024-04-22T23:34:56.756 INFO:tasks.workunit.client.1.smithi157.stdout:1/406: readlink d4/d5f/l62 0 2024-04-22T23:34:56.756 INFO:tasks.workunit.client.1.smithi157.stdout:1/407: fdatasync d4/db/fe 0 2024-04-22T23:34:56.758 INFO:tasks.workunit.client.1.smithi157.stdout:9/350: getdents d4/d2d 0 2024-04-22T23:34:56.758 INFO:tasks.workunit.client.1.smithi157.stdout:9/351: getdents d4/d15/d23/d67 0 2024-04-22T23:34:56.758 INFO:tasks.workunit.client.1.smithi157.stdout:9/352: write d4/d15/f4e [637228,82811] 0 2024-04-22T23:34:56.761 INFO:tasks.workunit.client.1.smithi157.stdout:7/283: rename d0/d9/d10/f12 to d0/d9/db/f61 0 2024-04-22T23:34:56.763 INFO:tasks.workunit.client.1.smithi157.stdout:4/428: link d3/d11/d30/d36/d62/d4c/c70 d3/d11/d30/d36/d62/d81/c8a 0 2024-04-22T23:34:56.766 INFO:tasks.workunit.client.1.smithi157.stdout:2/425: rmdir d7/d85 0 2024-04-22T23:34:56.766 INFO:tasks.workunit.client.1.smithi157.stdout:2/426: read - d7/d8/d30/d5b/f73 zero size 2024-04-22T23:34:56.768 INFO:tasks.workunit.client.1.smithi157.stdout:1/408: creat d4/d17/d6b/f86 x:0 0 0 2024-04-22T23:34:56.768 INFO:tasks.workunit.client.1.smithi157.stdout:1/409: write d4/f3e [587648,24107] 0 2024-04-22T23:34:56.768 INFO:tasks.workunit.client.1.smithi157.stdout:1/410: stat d4/f16 0 2024-04-22T23:34:56.769 INFO:tasks.workunit.client.1.smithi157.stdout:7/284: mknod d0/d9/d29/d33/d37/c62 0 2024-04-22T23:34:56.770 INFO:tasks.workunit.client.1.smithi157.stdout:7/285: chown d0/d9/d10/d43/c4b 55229 1 2024-04-22T23:34:56.771 INFO:tasks.workunit.client.1.smithi157.stdout:7/286: read d0/d9/d29/f31 [401050,74652] 0 2024-04-22T23:34:56.771 INFO:tasks.workunit.client.1.smithi157.stdout:9/353: dwrite d4/d15/d23/d27/f33 [0,4194304] 0 2024-04-22T23:34:56.772 INFO:tasks.workunit.client.1.smithi157.stdout:9/354: truncate d4/d15/d23/d27/f53 768545 0 2024-04-22T23:34:56.772 INFO:tasks.workunit.client.1.smithi157.stdout:4/429: link d3/d4b/f5 d3/d4b/d5d/d6c/f8b 0 2024-04-22T23:34:56.776 INFO:tasks.workunit.client.1.smithi157.stdout:1/411: creat d4/d17/d1d/d33/d41/d54/f87 x:0 0 0 2024-04-22T23:34:56.777 INFO:tasks.workunit.client.1.smithi157.stdout:2/427: rename d7/f3b to d7/d8/dd/d4d/f99 0 2024-04-22T23:34:56.777 INFO:tasks.workunit.client.1.smithi157.stdout:2/428: stat d7/d8/d25/d41 0 2024-04-22T23:34:56.781 INFO:tasks.workunit.client.1.smithi157.stdout:7/287: dread d0/d9/d29/d49/d4a/f59 [0,4194304] 0 2024-04-22T23:34:56.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.788+0000 7fe6e2252700 1 -- 172.21.15.107:0/3424129359 --> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe6dc10f960 con 0x7fe6c40692a0 2024-04-22T23:34:56.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.790+0000 7fe6c3fff700 1 -- 172.21.15.107:0/3424129359 <== mgr.14206 v2:172.21.15.107:6800/2516490111 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7fe6dc10f960 con 0x7fe6c40692a0 2024-04-22T23:34:56.785 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-22T23:34:56.785 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c", 2024-04-22T23:34:56.785 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-04-22T23:34:56.785 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-22T23:34:56.785 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [], 2024-04-22T23:34:56.786 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "0/21 daemons upgraded", 2024-04-22T23:34:56.786 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c image on host smithi157", 2024-04-22T23:34:56.786 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-04-22T23:34:56.786 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-22T23:34:56.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.793+0000 7fe6e2252700 1 -- 172.21.15.107:0/3424129359 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fe6c40692a0 msgr2=0x7fe6c406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:34:56.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.793+0000 7fe6e2252700 1 --2- 172.21.15.107:0/3424129359 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fe6c40692a0 0x7fe6c406b750 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0004bc0 tx=0x7fe6d000b040 comp rx=0 tx=0).stop 2024-04-22T23:34:56.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.793+0000 7fe6e2252700 1 -- 172.21.15.107:0/3424129359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6dc0fce60 msgr2=0x7fe6dc10d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:34:56.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.793+0000 7fe6e2252700 1 --2- 172.21.15.107:0/3424129359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6dc0fce60 0x7fe6dc10d9e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc004940 tx=0x7fe6cc009e30 comp rx=0 tx=0).stop 2024-04-22T23:34:56.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.793+0000 7fe6e2252700 1 -- 172.21.15.107:0/3424129359 shutdown_connections 2024-04-22T23:34:56.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.793+0000 7fe6e2252700 1 --2- 172.21.15.107:0/3424129359 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fe6c40692a0 0x7fe6c406b750 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:56.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.793+0000 7fe6e2252700 1 --2- 172.21.15.107:0/3424129359 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6dc0fce60 0x7fe6dc10d9e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:56.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.793+0000 7fe6e2252700 1 --2- 172.21.15.107:0/3424129359 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6dc0fe060 0x7fe6dc10df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:56.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.793+0000 7fe6e2252700 1 -- 172.21.15.107:0/3424129359 >> 172.21.15.107:0/3424129359 conn(0x7fe6dc0f8570 msgr2=0x7fe6dc0f9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:34:56.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.794+0000 7fe6e2252700 1 -- 172.21.15.107:0/3424129359 shutdown_connections 2024-04-22T23:34:56.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.794+0000 7fe6e2252700 1 -- 172.21.15.107:0/3424129359 wait complete. 2024-04-22T23:34:56.792 INFO:tasks.workunit.client.1.smithi157.stdout:4/430: rename d3/d4b/ff to d3/d11/d1d/d73/d80/f8c 0 2024-04-22T23:34:56.792 INFO:tasks.workunit.client.1.smithi157.stdout:4/431: chown d3/d4b/d5d/c60 15018074 1 2024-04-22T23:34:56.792 INFO:tasks.workunit.client.1.smithi157.stdout:4/432: truncate d3/d11/d30/d28/f2b 1195973 0 2024-04-22T23:34:56.793 INFO:tasks.workunit.client.1.smithi157.stdout:2/429: link d7/d1f/f82 d7/d8/d95/f9a 0 2024-04-22T23:34:56.794 INFO:tasks.workunit.client.1.smithi157.stdout:1/412: dwrite d4/db/f13 [0,4194304] 0 2024-04-22T23:34:56.794 INFO:tasks.workunit.client.1.smithi157.stdout:7/288: mknod d0/d9/c63 0 2024-04-22T23:34:56.796 INFO:tasks.workunit.client.1.smithi157.stdout:4/433: mknod d3/d11/d30/c8d 0 2024-04-22T23:34:56.796 INFO:tasks.workunit.client.1.smithi157.stdout:4/434: write d3/d11/d1d/d73/d80/f89 [224583,29192] 0 2024-04-22T23:34:56.798 INFO:tasks.workunit.client.1.smithi157.stdout:2/430: creat d7/d8/dd/d4d/f9b x:0 0 0 2024-04-22T23:34:56.798 INFO:tasks.workunit.client.1.smithi157.stdout:2/431: chown d7/d8/d95/f63 1836483399 1 2024-04-22T23:34:56.798 INFO:tasks.workunit.client.1.smithi157.stdout:2/432: chown d7/d8/d8e/f7e 29051 1 2024-04-22T23:34:56.805 INFO:tasks.workunit.client.1.smithi157.stdout:1/413: write d4/d17/f59 [559948,309] 0 2024-04-22T23:34:56.814 INFO:tasks.workunit.client.1.smithi157.stdout:1/414: symlink d4/d17/d1d/l88 0 2024-04-22T23:34:56.815 INFO:tasks.workunit.client.1.smithi157.stdout:6/482: sync 2024-04-22T23:34:56.817 INFO:tasks.workunit.client.1.smithi157.stdout:7/289: rmdir d0/d9 39 2024-04-22T23:34:56.818 INFO:tasks.workunit.client.1.smithi157.stdout:1/415: dread d4/db/f36 [0,4194304] 0 2024-04-22T23:34:56.818 INFO:tasks.workunit.client.1.smithi157.stdout:1/416: dread - d4/db/f4a zero size 2024-04-22T23:34:56.819 INFO:tasks.workunit.client.1.smithi157.stdout:7/290: mkdir d0/d9/d29/d49/d64 0 2024-04-22T23:34:56.819 INFO:tasks.workunit.client.1.smithi157.stdout:7/291: chown d0/d9/db 23153 1 2024-04-22T23:34:56.819 INFO:tasks.workunit.client.1.smithi157.stdout:7/292: stat d0/d9/d36 0 2024-04-22T23:34:56.819 INFO:tasks.workunit.client.1.smithi157.stdout:1/417: write d4/d17/d34/d3b/d52/f7f [32810,61945] 0 2024-04-22T23:34:56.819 INFO:tasks.workunit.client.1.smithi157.stdout:7/293: rmdir d0/d9/d10/d43 39 2024-04-22T23:34:56.819 INFO:tasks.workunit.client.1.smithi157.stdout:1/418: creat d4/d17/d34/d75/f89 x:0 0 0 2024-04-22T23:34:56.820 INFO:tasks.workunit.client.1.smithi157.stdout:7/294: creat d0/d9/d29/d5e/f65 x:0 0 0 2024-04-22T23:34:56.822 INFO:tasks.workunit.client.1.smithi157.stdout:7/295: rename d0/d9/d29/f31 to d0/d9/f66 0 2024-04-22T23:34:56.822 INFO:tasks.workunit.client.1.smithi157.stdout:7/296: rmdir d0/d9/d42 39 2024-04-22T23:34:56.822 INFO:tasks.workunit.client.1.smithi157.stdout:1/419: dread d4/d17/d34/d3b/d50/f56 [0,4194304] 0 2024-04-22T23:34:56.823 INFO:tasks.workunit.client.1.smithi157.stdout:1/420: chown d4/d17/d1d/d33/d41/f85 158 1 2024-04-22T23:34:56.823 INFO:tasks.workunit.client.1.smithi157.stdout:7/297: creat d0/d9/d29/d49/d4a/f67 x:0 0 0 2024-04-22T23:34:56.824 INFO:tasks.workunit.client.1.smithi157.stdout:1/421: mkdir d4/d17/d34/d75/d8a 0 2024-04-22T23:34:56.825 INFO:tasks.workunit.client.1.smithi157.stdout:7/298: mkdir d0/d9/db/df/d68 0 2024-04-22T23:34:56.825 INFO:tasks.workunit.client.1.smithi157.stdout:7/299: write d0/d9/d29/f3b [389224,129109] 0 2024-04-22T23:34:56.825 INFO:tasks.workunit.client.1.smithi157.stdout:7/300: truncate d0/d9/d10/f5d 608257 0 2024-04-22T23:34:56.825 INFO:tasks.workunit.client.1.smithi157.stdout:1/422: symlink d4/db/d3d/l8b 0 2024-04-22T23:34:56.827 INFO:tasks.workunit.client.1.smithi157.stdout:7/301: mknod d0/d9/d29/c69 0 2024-04-22T23:34:56.827 INFO:tasks.workunit.client.1.smithi157.stdout:7/302: creat d0/d9/d29/d5e/f6a x:0 0 0 2024-04-22T23:34:56.827 INFO:tasks.workunit.client.1.smithi157.stdout:2/433: dwrite d7/d8/d30/d23/f48 [0,4194304] 0 2024-04-22T23:34:56.827 INFO:tasks.workunit.client.1.smithi157.stdout:4/435: dwrite d3/d4b/f16 [0,4194304] 0 2024-04-22T23:34:56.827 INFO:tasks.workunit.client.1.smithi157.stdout:6/483: write d1/f22 [2097877,15500] 0 2024-04-22T23:34:56.827 INFO:tasks.workunit.client.1.smithi157.stdout:4/436: dread - d3/d11/d34/f51 zero size 2024-04-22T23:34:56.827 INFO:tasks.workunit.client.1.smithi157.stdout:6/484: truncate d1/d2f/f40 1982386 0 2024-04-22T23:34:56.827 INFO:tasks.workunit.client.1.smithi157.stdout:4/437: chown d3/d11/d34/c66 358435 1 2024-04-22T23:34:56.828 INFO:tasks.workunit.client.1.smithi157.stdout:1/423: mkdir d4/d17/d34/d3b/d52/d65/d8c 0 2024-04-22T23:34:56.829 INFO:tasks.workunit.client.1.smithi157.stdout:7/303: link d0/c25 d0/d9/d29/d49/c6b 0 2024-04-22T23:34:56.830 INFO:tasks.workunit.client.1.smithi157.stdout:7/304: write d0/d9/f66 [1056291,87219] 0 2024-04-22T23:34:56.830 INFO:tasks.workunit.client.1.smithi157.stdout:2/434: rmdir d7/d8/d25 39 2024-04-22T23:34:56.830 INFO:tasks.workunit.client.1.smithi157.stdout:2/435: stat d7/d1f/d26/d8f 0 2024-04-22T23:34:56.838 INFO:tasks.workunit.client.1.smithi157.stdout:4/438: mknod d3/d4b/d5d/d6c/c8e 0 2024-04-22T23:34:56.838 INFO:tasks.workunit.client.1.smithi157.stdout:4/439: write d3/d11/d30/d36/f65 [352870,19332] 0 2024-04-22T23:34:56.838 INFO:tasks.workunit.client.1.smithi157.stdout:4/440: truncate d3/d11/d30/d36/d62/d40/f72 205843 0 2024-04-22T23:34:56.840 INFO:tasks.workunit.client.1.smithi157.stdout:6/485: link d1/d2f/d54/d9b/f61 d1/d2f/d3b/d6c/d83/f9c 0 2024-04-22T23:34:56.841 INFO:tasks.workunit.client.1.smithi157.stdout:4/441: dread d3/d11/d30/d36/f65 [0,4194304] 0 2024-04-22T23:34:56.841 INFO:tasks.workunit.client.1.smithi157.stdout:4/442: stat d3/d11/d1d/f24 0 2024-04-22T23:34:56.843 INFO:tasks.workunit.client.1.smithi157.stdout:1/424: link d4/db/d3d/f82 d4/d17/d6b/f8d 0 2024-04-22T23:34:56.843 INFO:tasks.workunit.client.1.smithi157.stdout:7/305: symlink d0/d9/d29/d5e/l6c 0 2024-04-22T23:34:56.845 INFO:tasks.workunit.client.1.smithi157.stdout:6/486: mknod d1/d38/d97/c9d 0 2024-04-22T23:34:56.845 INFO:tasks.workunit.client.1.smithi157.stdout:6/487: chown d1/f18 274166076 1 2024-04-22T23:34:56.846 INFO:tasks.workunit.client.1.smithi157.stdout:4/443: unlink d3/d11/d30/d36/d62/d4c/f87 0 2024-04-22T23:34:56.847 INFO:tasks.workunit.client.1.smithi157.stdout:7/306: mknod d0/d9/d29/d33/d37/c6d 0 2024-04-22T23:34:56.847 INFO:tasks.workunit.client.1.smithi157.stdout:7/307: readlink d0/d9/db/d35/l3d 0 2024-04-22T23:34:56.847 INFO:tasks.workunit.client.1.smithi157.stdout:5/379: sync 2024-04-22T23:34:56.847 INFO:tasks.workunit.client.1.smithi157.stdout:3/291: sync 2024-04-22T23:34:56.848 INFO:tasks.workunit.client.1.smithi157.stdout:8/209: sync 2024-04-22T23:34:56.848 INFO:tasks.workunit.client.1.smithi157.stdout:0/278: sync 2024-04-22T23:34:56.848 INFO:tasks.workunit.client.1.smithi157.stdout:3/292: readlink d0/d3/d34/d39/l52 0 2024-04-22T23:34:56.848 INFO:tasks.workunit.client.1.smithi157.stdout:0/279: chown d17/d2b/d52/f44 9805 1 2024-04-22T23:34:56.849 INFO:tasks.workunit.client.1.smithi157.stdout:2/436: link d7/d8/d6b/f8b d7/d8/d30/f9c 0 2024-04-22T23:34:56.851 INFO:tasks.workunit.client.1.smithi157.stdout:6/488: mknod d1/d36/c9e 0 2024-04-22T23:34:56.853 INFO:tasks.workunit.client.1.smithi157.stdout:4/444: rmdir d3/d11/d34 39 2024-04-22T23:34:56.854 INFO:tasks.workunit.client.1.smithi157.stdout:7/308: rmdir d0/d9/db 39 2024-04-22T23:34:56.854 INFO:tasks.workunit.client.1.smithi157.stdout:7/309: write d0/d9/db/f57 [409587,8562] 0 2024-04-22T23:34:56.856 INFO:tasks.workunit.client.1.smithi157.stdout:8/210: rename d2/f3 to d2/d6/de/d17/f49 0 2024-04-22T23:34:56.856 INFO:tasks.workunit.client.1.smithi157.stdout:0/280: mknod d17/d2b/d52/c62 0 2024-04-22T23:34:56.858 INFO:tasks.workunit.client.1.smithi157.stdout:4/445: rename d3/d11/c15 to d3/d11/c8f 0 2024-04-22T23:34:56.858 INFO:tasks.workunit.client.1.smithi157.stdout:7/310: mknod d0/d9/db/df/c6e 0 2024-04-22T23:34:56.859 INFO:tasks.workunit.client.1.smithi157.stdout:5/380: getdents d1/d5/dc/d43/d4f/d60/d7a 0 2024-04-22T23:34:56.859 INFO:tasks.workunit.client.1.smithi157.stdout:5/381: dread - d1/d5/dc/d43/d4f/d60/d7a/f87 zero size 2024-04-22T23:34:56.859 INFO:tasks.workunit.client.1.smithi157.stdout:5/382: stat d1/d5/d7/d2f/d45/d54/f82 0 2024-04-22T23:34:56.861 INFO:tasks.workunit.client.1.smithi157.stdout:8/211: creat d2/d6/de/d11/f4a x:0 0 0 2024-04-22T23:34:56.861 INFO:tasks.workunit.client.1.smithi157.stdout:8/212: creat d2/d6/de/d11/f4b x:0 0 0 2024-04-22T23:34:56.862 INFO:tasks.workunit.client.1.smithi157.stdout:3/293: rename d0/d3/d13/d21/d4d/d59 to d0/d3/d34/d3d/d5b 0 2024-04-22T23:34:56.862 INFO:tasks.workunit.client.1.smithi157.stdout:3/294: stat d0/d3/d13/l2e 0 2024-04-22T23:34:56.863 INFO:tasks.workunit.client.1.smithi157.stdout:8/213: write d2/d6/de/d17/f49 [820793,51056] 0 2024-04-22T23:34:56.865 INFO:tasks.workunit.client.1.smithi157.stdout:0/281: creat d17/d2b/d4f/d5a/f63 x:0 0 0 2024-04-22T23:34:56.866 INFO:tasks.workunit.client.1.smithi157.stdout:7/311: symlink d0/d9/d29/d49/d4a/l6f 0 2024-04-22T23:34:56.868 INFO:tasks.workunit.client.1.smithi157.stdout:1/425: dwrite d4/d17/d6b/f86 [0,4194304] 0 2024-04-22T23:34:56.869 INFO:tasks.workunit.client.1.smithi157.stdout:5/383: rename d1/d27/d29/c76 to d1/d27/d29/d4b/c94 0 2024-04-22T23:34:56.870 INFO:tasks.workunit.client.1.smithi157.stdout:8/214: symlink d2/l4c 0 2024-04-22T23:34:56.871 INFO:tasks.workunit.client.1.smithi157.stdout:4/446: read d3/d11/d1d/f26 [2837467,67241] 0 2024-04-22T23:34:56.871 INFO:tasks.workunit.client.1.smithi157.stdout:4/447: write d3/d11/d30/d28/f3c [4609769,3027] 0 2024-04-22T23:34:56.873 INFO:tasks.workunit.client.1.smithi157.stdout:5/384: mknod d1/d5/dc/d43/c95 0 2024-04-22T23:34:56.875 INFO:tasks.workunit.client.1.smithi157.stdout:0/282: write fe [393739,41983] 0 2024-04-22T23:34:56.875 INFO:tasks.workunit.client.1.smithi157.stdout:1/426: read d4/db/f36 [191996,21630] 0 2024-04-22T23:34:56.877 INFO:tasks.workunit.client.1.smithi157.stdout:8/215: symlink d2/d34/l4d 0 2024-04-22T23:34:56.878 INFO:tasks.workunit.client.1.smithi157.stdout:4/448: link d3/d11/d34/c7f d3/d4b/d7a/d86/c90 0 2024-04-22T23:34:56.879 INFO:tasks.workunit.client.1.smithi157.stdout:4/449: chown d3/d11/c8f 6230609 1 2024-04-22T23:34:56.879 INFO:tasks.workunit.client.1.smithi157.stdout:5/385: symlink d1/d5/d7/d2f/d45/d54/d59/l96 0 2024-04-22T23:34:56.881 INFO:tasks.workunit.client.1.smithi157.stdout:0/283: mknod d17/d2b/d4f/c64 0 2024-04-22T23:34:56.881 INFO:tasks.workunit.client.1.smithi157.stdout:1/427: creat d4/f8e x:0 0 0 2024-04-22T23:34:56.884 INFO:tasks.workunit.client.1.smithi157.stdout:5/386: unlink d1/d5/d7/d2f/d45/l86 0 2024-04-22T23:34:56.886 INFO:tasks.workunit.client.1.smithi157.stdout:4/450: getdents d3/d11/d30/d28/d31 0 2024-04-22T23:34:56.886 INFO:tasks.workunit.client.1.smithi157.stdout:4/451: write d3/d11/d30/f57 [72008,59730] 0 2024-04-22T23:34:56.886 INFO:tasks.workunit.client.1.smithi157.stdout:4/452: fsync d3/d11/d1d/f26 0 2024-04-22T23:34:56.887 INFO:tasks.workunit.client.1.smithi157.stdout:5/387: creat d1/d5/d7/d2f/d51/f97 x:0 0 0 2024-04-22T23:34:56.889 INFO:tasks.workunit.client.1.smithi157.stdout:0/284: rename d17/d23/d34/f40 to d17/d38/f65 0 2024-04-22T23:34:56.892 INFO:tasks.workunit.client.1.smithi157.stdout:4/453: mkdir d3/d11/d30/d28/d31/d91 0 2024-04-22T23:34:56.892 INFO:tasks.workunit.client.1.smithi157.stdout:8/216: write d2/f32 [2767993,52651] 0 2024-04-22T23:34:56.892 INFO:tasks.workunit.client.1.smithi157.stdout:8/217: stat d2/d6/d21/d33 0 2024-04-22T23:34:56.893 INFO:tasks.workunit.client.1.smithi157.stdout:8/218: creat d2/d6/de/d17/f4e x:0 0 0 2024-04-22T23:34:56.893 INFO:tasks.workunit.client.1.smithi157.stdout:5/388: write d1/d5/d7/d10/f49 [1394587,20507] 0 2024-04-22T23:34:56.893 INFO:tasks.workunit.client.1.smithi157.stdout:0/285: getdents d17/d2b 0 2024-04-22T23:34:56.893 INFO:tasks.workunit.client.1.smithi157.stdout:0/286: write d17/d23/f31 [563986,88859] 0 2024-04-22T23:34:56.894 INFO:tasks.workunit.client.1.smithi157.stdout:7/312: dwrite d0/d9/d29/f3b [0,4194304] 0 2024-04-22T23:34:56.894 INFO:tasks.workunit.client.1.smithi157.stdout:7/313: readlink d0/d9/d42/d55/d56/l58 0 2024-04-22T23:34:56.894 INFO:tasks.workunit.client.1.smithi157.stdout:7/314: write d0/f2 [9250449,49324] 0 2024-04-22T23:34:56.896 INFO:tasks.workunit.client.1.smithi157.stdout:4/454: link d3/d4b/d5d/d6c/c84 d3/d11/d30/d36/d62/d4c/c92 0 2024-04-22T23:34:56.896 INFO:tasks.workunit.client.1.smithi157.stdout:3/295: dwrite d0/d7/f4f [0,4194304] 0 2024-04-22T23:34:56.896 INFO:tasks.workunit.client.1.smithi157.stdout:3/296: write d0/d3/d13/d21/f4e [221153,76667] 0 2024-04-22T23:34:56.901 INFO:tasks.workunit.client.1.smithi157.stdout:8/219: rename d2/d6/d21/f25 to d2/d6/de/d17/f4f 0 2024-04-22T23:34:56.901 INFO:tasks.workunit.client.1.smithi157.stdout:8/220: creat d2/d4/f50 x:0 0 0 2024-04-22T23:34:56.912 INFO:tasks.workunit.client.1.smithi157.stdout:0/287: fsync d17/d38/f65 0 2024-04-22T23:34:56.916 INFO:tasks.workunit.client.1.smithi157.stdout:4/455: symlink d3/d11/d30/d36/d62/d81/l93 0 2024-04-22T23:34:56.925 INFO:tasks.workunit.client.1.smithi157.stdout:3/297: rename d0/d3/d13/d21/d4d to d0/d3/d34/d3d/d57/d5c 0 2024-04-22T23:34:56.926 INFO:tasks.workunit.client.1.smithi157.stdout:8/221: mknod d2/d6/d21/d33/d42/c51 0 2024-04-22T23:34:56.926 INFO:tasks.workunit.client.1.smithi157.stdout:8/222: creat d2/d4/f52 x:0 0 0 2024-04-22T23:34:56.929 INFO:tasks.workunit.client.1.smithi157.stdout:5/389: link d1/d5/dc/l16 d1/d5/d7/d2f/d51/l98 0 2024-04-22T23:34:56.929 INFO:tasks.workunit.client.1.smithi157.stdout:0/288: dwrite d17/d38/f56 [0,4194304] 0 2024-04-22T23:34:56.930 INFO:tasks.workunit.client.1.smithi157.stdout:4/456: creat d3/d4b/d5d/d6c/f94 x:0 0 0 2024-04-22T23:34:56.931 INFO:tasks.workunit.client.1.smithi157.stdout:3/298: unlink d0/d7/f3a 0 2024-04-22T23:34:56.932 INFO:tasks.workunit.client.1.smithi157.stdout:3/299: stat d0/d3/f6 0 2024-04-22T23:34:56.933 INFO:tasks.workunit.client.1.smithi157.stdout:5/390: dread d1/d5/d7/d10/f31 [0,4194304] 0 2024-04-22T23:34:56.933 INFO:tasks.workunit.client.1.smithi157.stdout:5/391: fsync d1/d5/d7/d2f/d45/d54/f74 0 2024-04-22T23:34:56.933 INFO:tasks.workunit.client.1.smithi157.stdout:5/392: fsync d1/d5/d7/d2f/d45/d54/f7b 0 2024-04-22T23:34:56.934 INFO:tasks.workunit.client.1.smithi157.stdout:0/289: creat d17/d2b/f66 x:0 0 0 2024-04-22T23:34:56.935 INFO:tasks.workunit.client.1.smithi157.stdout:4/457: creat d3/d5c/f95 x:0 0 0 2024-04-22T23:34:56.936 INFO:tasks.workunit.client.1.smithi157.stdout:4/458: write d3/d11/d1d/f1e [4315701,54035] 0 2024-04-22T23:34:56.937 INFO:tasks.workunit.client.1.smithi157.stdout:8/223: write d2/f5 [3885925,67385] 0 2024-04-22T23:34:56.937 INFO:tasks.workunit.client.1.smithi157.stdout:3/300: unlink d0/f15 0 2024-04-22T23:34:56.937 INFO:tasks.workunit.client.1.smithi157.stdout:3/301: truncate d0/d3/d13/d21/f4e 954097 0 2024-04-22T23:34:56.942 INFO:tasks.workunit.client.1.smithi157.stdout:8/224: mknod d2/d6/de/d17/c53 0 2024-04-22T23:34:56.942 INFO:tasks.workunit.client.1.smithi157.stdout:8/225: chown d2/f48 333426 1 2024-04-22T23:34:56.950 INFO:tasks.workunit.client.1.smithi157.stdout:4/459: rename d3/d11/d30/d36/d62 to d3/d96 0 2024-04-22T23:34:56.950 INFO:tasks.workunit.client.1.smithi157.stdout:4/460: readlink d3/d4b/l7c 0 2024-04-22T23:34:56.950 INFO:tasks.workunit.client.1.smithi157.stdout:4/461: readlink d3/d96/d81/l93 0 2024-04-22T23:34:56.950 INFO:tasks.workunit.client.1.smithi157.stdout:4/462: fdatasync d3/d5c/f95 0 2024-04-22T23:34:56.953 INFO:tasks.workunit.client.1.smithi157.stdout:0/290: dwrite d17/d2b/f5c [0,4194304] 0 2024-04-22T23:34:56.954 INFO:tasks.workunit.client.1.smithi157.stdout:0/291: chown d17/d58/f61 22974849 1 2024-04-22T23:34:56.954 INFO:tasks.workunit.client.1.smithi157.stdout:0/292: chown d17/d2b/d52/f53 249898843 1 2024-04-22T23:34:56.954 INFO:tasks.workunit.client.1.smithi157.stdout:4/463: truncate d3/d11/d1d/d73/d80/f8c 2716892 0 2024-04-22T23:34:56.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.960+0000 7f739416e700 1 -- 172.21.15.107:0/2742061971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f738c0fc140 msgr2=0x7f738c0fc550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:34:56.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.960+0000 7f739416e700 1 --2- 172.21.15.107:0/2742061971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f738c0fc140 0x7f738c0fc550 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7384009ce0 tx=0x7f7384004d10 comp rx=0 tx=0).stop 2024-04-22T23:34:56.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.960+0000 7f739416e700 1 -- 172.21.15.107:0/2742061971 shutdown_connections 2024-04-22T23:34:56.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.960+0000 7f739416e700 1 --2- 172.21.15.107:0/2742061971 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f738c0fca90 0x7f738c0faea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:56.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.960+0000 7f739416e700 1 --2- 172.21.15.107:0/2742061971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f738c0fc140 0x7f738c0fc550 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:56.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.960+0000 7f739416e700 1 -- 172.21.15.107:0/2742061971 >> 172.21.15.107:0/2742061971 conn(0x7f738c0f6bd0 msgr2=0x7f738c0f8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:34:56.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.960+0000 7f739416e700 1 -- 172.21.15.107:0/2742061971 shutdown_connections 2024-04-22T23:34:56.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.960+0000 7f739416e700 1 -- 172.21.15.107:0/2742061971 wait complete. 2024-04-22T23:34:56.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.961+0000 7f739416e700 1 Processor -- start 2024-04-22T23:34:56.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.961+0000 7f739416e700 1 -- start start 2024-04-22T23:34:56.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.961+0000 7f739416e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f738c0fc140 0x7f738c1141f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:34:56.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.961+0000 7f739416e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f738c0fca90 0x7f738c114730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:34:56.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.961+0000 7f739416e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738c114d30 con 0x7f738c0fc140 2024-04-22T23:34:56.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.961+0000 7f739416e700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738c115580 con 0x7f738c0fca90 2024-04-22T23:34:56.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.962+0000 7f7391f0a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f738c0fc140 0x7f738c1141f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:34:56.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.962+0000 7f7391f0a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f738c0fc140 0x7f738c1141f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51262/0 (socket says 172.21.15.107:51262) 2024-04-22T23:34:56.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.962+0000 7f7391f0a700 1 -- 172.21.15.107:0/1593697064 learned_addr learned my addr 172.21.15.107:0/1593697064 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:34:56.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.962+0000 7f7391f0a700 1 -- 172.21.15.107:0/1593697064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f738c0fca90 msgr2=0x7f738c114730 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-22T23:34:56.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.962+0000 7f7391f0a700 1 --2- 172.21.15.107:0/1593697064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f738c0fca90 0x7f738c114730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:56.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.962+0000 7f7391f0a700 1 -- 172.21.15.107:0/1593697064 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7384009940 con 0x7f738c0fc140 2024-04-22T23:34:56.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.963+0000 7f7391f0a700 1 --2- 172.21.15.107:0/1593697064 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f738c0fc140 0x7f738c1141f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7384009380 tx=0x7f7384004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:34:56.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.963+0000 7f73827fc700 1 -- 172.21.15.107:0/1593697064 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f738400e040 con 0x7f738c0fc140 2024-04-22T23:34:56.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.963+0000 7f739416e700 1 -- 172.21.15.107:0/1593697064 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f738c115800 con 0x7f738c0fc140 2024-04-22T23:34:56.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.963+0000 7f739416e700 1 -- 172.21.15.107:0/1593697064 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f738c115d40 con 0x7f738c0fc140 2024-04-22T23:34:56.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.964+0000 7f73827fc700 1 -- 172.21.15.107:0/1593697064 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f73840128e0 con 0x7f738c0fc140 2024-04-22T23:34:56.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.964+0000 7f73827fc700 1 -- 172.21.15.107:0/1593697064 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f738401c860 con 0x7f738c0fc140 2024-04-22T23:34:56.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.966+0000 7f739416e700 1 -- 172.21.15.107:0/1593697064 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73700052e0 con 0x7f738c0fc140 2024-04-22T23:34:56.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.966+0000 7f73827fc700 1 -- 172.21.15.107:0/1593697064 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f7384012410 con 0x7f738c0fc140 2024-04-22T23:34:56.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.967+0000 7f73827fc700 1 --2- 172.21.15.107:0/1593697064 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f7378069300 0x7f737806b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:34:56.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.967+0000 7f73827fc700 1 -- 172.21.15.107:0/1593697064 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f7384088d60 con 0x7f738c0fc140 2024-04-22T23:34:56.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.967+0000 7f7391709700 1 --2- 172.21.15.107:0/1593697064 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f7378069300 0x7f737806b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:34:56.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.968+0000 7f7391709700 1 --2- 172.21.15.107:0/1593697064 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f7378069300 0x7f737806b7b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f737c004ad0 tx=0x7f737c007d40 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:34:56.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:56.973+0000 7f73827fc700 1 -- 172.21.15.107:0/1593697064 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f7384058ad0 con 0x7f738c0fc140 2024-04-22T23:34:56.972 INFO:tasks.workunit.client.1.smithi157.stdout:8/226: dwrite d2/d6/f37 [0,4194304] 0 2024-04-22T23:34:56.972 INFO:tasks.workunit.client.1.smithi157.stdout:8/227: dread - d2/d4/f52 zero size 2024-04-22T23:34:56.976 INFO:tasks.workunit.client.1.smithi157.stdout:8/228: rename d2/d6/d21/f2d to d2/d6/d21/f54 0 2024-04-22T23:34:56.976 INFO:tasks.workunit.client.1.smithi157.stdout:8/229: truncate d2/f32 4299685 0 2024-04-22T23:34:56.976 INFO:tasks.workunit.client.1.smithi157.stdout:8/230: creat d2/d6/de/f55 x:0 0 0 2024-04-22T23:34:56.976 INFO:tasks.workunit.client.1.smithi157.stdout:8/231: creat d2/d6/de/f56 x:0 0 0 2024-04-22T23:34:56.976 INFO:tasks.workunit.client.1.smithi157.stdout:3/302: dwrite d0/f1c [0,4194304] 0 2024-04-22T23:34:56.977 INFO:tasks.workunit.client.1.smithi157.stdout:4/464: dwrite d3/d11/d34/f48 [0,4194304] 0 2024-04-22T23:34:56.978 INFO:tasks.workunit.client.1.smithi157.stdout:0/293: dwrite d17/d58/f5f [0,4194304] 0 2024-04-22T23:34:56.978 INFO:tasks.workunit.client.1.smithi157.stdout:0/294: chown d17/f19 0 1 2024-04-22T23:34:56.979 INFO:tasks.workunit.client.1.smithi157.stdout:0/295: truncate d17/f25 8620 0 2024-04-22T23:34:56.979 INFO:tasks.workunit.client.1.smithi157.stdout:4/465: write d3/d11/f1f [4874163,104762] 0 2024-04-22T23:34:56.979 INFO:tasks.workunit.client.1.smithi157.stdout:8/232: dread d2/d6/de/fd [0,4194304] 0 2024-04-22T23:34:56.982 INFO:tasks.workunit.client.1.smithi157.stdout:4/466: dread d3/d4b/d5d/d6c/f6d [0,4194304] 0 2024-04-22T23:34:56.984 INFO:tasks.workunit.client.1.smithi157.stdout:8/233: rename d2/d6/de/d11/d13/c31 to d2/d6/de/c57 0 2024-04-22T23:34:56.984 INFO:tasks.workunit.client.1.smithi157.stdout:8/234: truncate d2/d4/f52 358231 0 2024-04-22T23:34:56.986 INFO:tasks.workunit.client.1.smithi157.stdout:0/296: mkdir d17/d23/d34/d67 0 2024-04-22T23:34:56.990 INFO:tasks.workunit.client.1.smithi157.stdout:8/235: unlink d2/d6/c47 0 2024-04-22T23:34:56.990 INFO:tasks.workunit.client.1.smithi157.stdout:8/236: chown d2/d6/d21/d33/d42/c35 743 1 2024-04-22T23:34:56.990 INFO:tasks.workunit.client.1.smithi157.stdout:8/237: write d2/d6/d21/f2e [2278276,14097] 0 2024-04-22T23:34:56.991 INFO:tasks.workunit.client.1.smithi157.stdout:8/238: write d2/d6/d21/f2e [1228704,22684] 0 2024-04-22T23:34:56.991 INFO:tasks.workunit.client.1.smithi157.stdout:8/239: truncate d2/d6/f46 661953 0 2024-04-22T23:34:56.992 INFO:tasks.workunit.client.1.smithi157.stdout:0/297: dread d17/f22 [0,4194304] 0 2024-04-22T23:34:56.993 INFO:tasks.workunit.client.1.smithi157.stdout:3/303: dwrite d0/d3/f6 [4194304,4194304] 0 2024-04-22T23:34:56.996 INFO:tasks.workunit.client.1.smithi157.stdout:4/467: write d3/f58 [3580792,2744] 0 2024-04-22T23:34:56.996 INFO:tasks.workunit.client.1.smithi157.stdout:4/468: truncate d3/d4b/f45 4452511 0 2024-04-22T23:34:56.997 INFO:tasks.workunit.client.1.smithi157.stdout:8/240: mkdir d2/d6/d21/d33/d42/d58 0 2024-04-22T23:34:56.997 INFO:tasks.workunit.client.1.smithi157.stdout:8/241: readlink d2/l4c 0 2024-04-22T23:34:56.998 INFO:tasks.workunit.client.1.smithi157.stdout:0/298: truncate f3 4644227 0 2024-04-22T23:34:57.003 INFO:tasks.workunit.client.1.smithi157.stdout:3/304: truncate d0/d3/d13/d21/f3f 3565844 0 2024-04-22T23:34:57.005 INFO:tasks.workunit.client.1.smithi157.stdout:4/469: dread d3/d11/f22 [4194304,4194304] 0 2024-04-22T23:34:57.006 INFO:tasks.workunit.client.1.smithi157.stdout:0/299: write d17/f2d [3139211,28220] 0 2024-04-22T23:34:57.006 INFO:tasks.workunit.client.1.smithi157.stdout:0/300: stat d17/d58/f61 0 2024-04-22T23:34:57.007 INFO:tasks.workunit.client.1.smithi157.stdout:8/242: creat d2/d6/d21/d33/d42/f59 x:0 0 0 2024-04-22T23:34:57.008 INFO:tasks.workunit.client.1.smithi157.stdout:8/243: write d2/d6/d21/f2e [551340,108291] 0 2024-04-22T23:34:57.008 INFO:tasks.workunit.client.1.smithi157.stdout:8/244: creat d2/d6/de/d11/f5a x:0 0 0 2024-04-22T23:34:57.008 INFO:tasks.workunit.client.1.smithi157.stdout:8/245: fdatasync d2/d6/de/f2f 0 2024-04-22T23:34:57.008 INFO:tasks.workunit.client.1.smithi157.stdout:8/246: creat d2/d4/f5b x:0 0 0 2024-04-22T23:34:57.008 INFO:tasks.workunit.client.1.smithi157.stdout:8/247: creat d2/d6/d21/d33/d42/f5c x:0 0 0 2024-04-22T23:34:57.009 INFO:tasks.workunit.client.1.smithi157.stdout:3/305: link d0/d3/f1e d0/d3/d34/f5d 0 2024-04-22T23:34:57.013 INFO:tasks.workunit.client.1.smithi157.stdout:4/470: rename d3/d4b/d5d/f7d to d3/d4b/d5d/f97 0 2024-04-22T23:34:57.021 INFO:tasks.workunit.client.1.smithi157.stdout:3/306: creat d0/d3/d34/d3d/f5e x:0 0 0 2024-04-22T23:34:57.021 INFO:tasks.workunit.client.1.smithi157.stdout:3/307: chown d0/d3/d13/l2e 232203 1 2024-04-22T23:34:57.021 INFO:tasks.workunit.client.1.smithi157.stdout:3/308: truncate d0/d3/d34/d39/f56 935714 0 2024-04-22T23:34:57.023 INFO:tasks.workunit.client.1.smithi157.stdout:4/471: rename d3/d96/d40/c76 to d3/d4b/d5d/d6c/d75/c98 0 2024-04-22T23:34:57.024 INFO:tasks.workunit.client.1.smithi157.stdout:0/301: symlink d17/d23/d34/l68 0 2024-04-22T23:34:57.025 INFO:tasks.workunit.client.1.smithi157.stdout:0/302: fsync d17/d2b/f47 0 2024-04-22T23:34:57.026 INFO:tasks.workunit.client.1.smithi157.stdout:8/248: getdents d2/d6/d21 0 2024-04-22T23:34:57.030 INFO:tasks.workunit.client.1.smithi157.stdout:4/472: rename d3/d4b/f46 to d3/d96/d81/f99 0 2024-04-22T23:34:57.035 INFO:tasks.workunit.client.1.smithi157.stdout:9/355: sync 2024-04-22T23:34:57.039 INFO:tasks.workunit.client.1.smithi157.stdout:0/303: dwrite d17/d2b/f66 [0,4194304] 0 2024-04-22T23:34:57.039 INFO:tasks.workunit.client.1.smithi157.stdout:4/473: write d3/f1b [1358984,61986] 0 2024-04-22T23:34:57.039 INFO:tasks.workunit.client.1.smithi157.stdout:4/474: stat d3/d11/d34/c66 0 2024-04-22T23:34:57.039 INFO:tasks.workunit.client.1.smithi157.stdout:9/356: rmdir d4/d15/d23 39 2024-04-22T23:34:57.043 INFO:tasks.workunit.client.1.smithi157.stdout:0/304: truncate d17/d58/f5f 1105585 0 2024-04-22T23:34:57.045 INFO:tasks.workunit.client.1.smithi157.stdout:4/475: unlink d3/d11/d1d/f1e 0 2024-04-22T23:34:57.045 INFO:tasks.workunit.client.1.smithi157.stdout:4/476: chown d3/d96/l50 7842 1 2024-04-22T23:34:57.046 INFO:tasks.workunit.client.1.smithi157.stdout:4/477: creat d3/d96/f9a x:0 0 0 2024-04-22T23:34:57.047 INFO:tasks.workunit.client.1.smithi157.stdout:9/357: write d4/df/f1f [2494508,102206] 0 2024-04-22T23:34:57.047 INFO:tasks.workunit.client.1.smithi157.stdout:9/358: chown d4/d15/d23/l5e 439 1 2024-04-22T23:34:57.048 INFO:tasks.workunit.client.1.smithi157.stdout:9/359: chown d4/d15/d23/d3f/c59 3494204 1 2024-04-22T23:34:57.048 INFO:tasks.workunit.client.1.smithi157.stdout:4/478: creat d3/d11/d30/d28/d31/d91/f9b x:0 0 0 2024-04-22T23:34:57.050 INFO:tasks.workunit.client.1.smithi157.stdout:4/479: creat d3/d4b/d7a/f9c x:0 0 0 2024-04-22T23:34:57.050 INFO:tasks.workunit.client.1.smithi157.stdout:0/305: write d17/d2b/f5c [1483982,29581] 0 2024-04-22T23:34:57.051 INFO:tasks.workunit.client.1.smithi157.stdout:0/306: getdents d17/d2b/d4f 0 2024-04-22T23:34:57.053 INFO:tasks.workunit.client.1.smithi157.stdout:4/480: dread d3/d11/d30/d36/f65 [0,4194304] 0 2024-04-22T23:34:57.053 INFO:tasks.workunit.client.1.smithi157.stdout:4/481: chown d3/d4b/d5d/d6c/c8e 87528219 1 2024-04-22T23:34:57.054 INFO:tasks.workunit.client.1.smithi157.stdout:9/360: dread d4/df/d61/f66 [0,4194304] 0 2024-04-22T23:34:57.055 INFO:tasks.workunit.client.1.smithi157.stdout:0/307: dread d17/d23/f28 [0,4194304] 0 2024-04-22T23:34:57.055 INFO:tasks.workunit.client.1.smithi157.stdout:4/482: write d3/d4b/d5d/d6c/f6d [315301,102105] 0 2024-04-22T23:34:57.055 INFO:tasks.workunit.client.1.smithi157.stdout:4/483: fdatasync d3/d96/d81/f99 0 2024-04-22T23:34:57.056 INFO:tasks.workunit.client.1.smithi157.stdout:0/308: mkdir d17/d58/d69 0 2024-04-22T23:34:57.057 INFO:tasks.workunit.client.1.smithi157.stdout:4/484: mkdir d3/d11/d34/d9d 0 2024-04-22T23:34:57.057 INFO:tasks.workunit.client.1.smithi157.stdout:0/309: mkdir d17/d38/d6a 0 2024-04-22T23:34:57.058 INFO:tasks.workunit.client.1.smithi157.stdout:4/485: symlink d3/d11/d1d/d73/l9e 0 2024-04-22T23:34:57.060 INFO:tasks.workunit.client.1.smithi157.stdout:4/486: mkdir d3/d11/d30/d28/d31/d91/d9f 0 2024-04-22T23:34:57.063 INFO:tasks.workunit.client.1.smithi157.stdout:4/487: getdents d3/d11 0 2024-04-22T23:34:57.064 INFO:tasks.workunit.client.1.smithi157.stdout:4/488: mknod d3/d11/d30/d36/ca0 0 2024-04-22T23:34:57.066 INFO:tasks.workunit.client.1.smithi157.stdout:4/489: mknod d3/d4b/ca1 0 2024-04-22T23:34:57.068 INFO:tasks.workunit.client.1.smithi157.stdout:4/490: rename d3/d4b/d5d/d6c/d75 to d3/d11/d30/d28/d31/da2 0 2024-04-22T23:34:57.075 INFO:tasks.workunit.client.1.smithi157.stdout:4/491: dread d3/d11/d1d/f41 [4194304,4194304] 0 2024-04-22T23:34:57.075 INFO:tasks.workunit.client.1.smithi157.stdout:4/492: truncate d3/d11/d30/f71 749168 0 2024-04-22T23:34:57.075 INFO:tasks.workunit.client.1.smithi157.stdout:0/310: dwrite d17/f19 [0,4194304] 0 2024-04-22T23:34:57.075 INFO:tasks.workunit.client.1.smithi157.stdout:0/311: readlink l12 0 2024-04-22T23:34:57.080 INFO:tasks.workunit.client.1.smithi157.stdout:0/312: dread d17/f22 [0,4194304] 0 2024-04-22T23:34:57.081 INFO:tasks.workunit.client.1.smithi157.stdout:4/493: rename d3/d11/d30/d28 to d3/d11/d1d/d73/da3 0 2024-04-22T23:34:57.085 INFO:tasks.workunit.client.1.smithi157.stdout:0/313: dread d17/d23/f32 [0,4194304] 0 2024-04-22T23:34:57.087 INFO:tasks.workunit.client.1.smithi157.stdout:0/314: rename d17/d2b/l49 to d17/d38/l6b 0 2024-04-22T23:34:57.089 INFO:tasks.workunit.client.1.smithi157.stdout:0/315: truncate d17/d23/f31 545749 0 2024-04-22T23:34:57.089 INFO:tasks.workunit.client.1.smithi157.stdout:0/316: dread - d17/d2b/d52/f53 zero size 2024-04-22T23:34:57.089 INFO:tasks.workunit.client.1.smithi157.stdout:0/317: write d17/d23/f4d [889533,36973] 0 2024-04-22T23:34:57.090 INFO:tasks.workunit.client.1.smithi157.stdout:0/318: mknod d17/d2b/d52/c6c 0 2024-04-22T23:34:57.090 INFO:tasks.workunit.client.1.smithi157.stdout:0/319: truncate d17/d2b/d52/f53 1029989 0 2024-04-22T23:34:57.098 INFO:tasks.workunit.client.1.smithi157.stdout:4/494: dwrite d3/d11/d30/f4d [0,4194304] 0 2024-04-22T23:34:57.098 INFO:tasks.workunit.client.1.smithi157.stdout:4/495: write d3/d4b/d7a/f9c [348708,126490] 0 2024-04-22T23:34:57.098 INFO:tasks.workunit.client.1.smithi157.stdout:4/496: chown d3/d11/d1d/d73/da3/f3c 60 1 2024-04-22T23:34:57.099 INFO:tasks.workunit.client.1.smithi157.stdout:4/497: dread d3/d11/d30/d36/f65 [0,4194304] 0 2024-04-22T23:34:57.099 INFO:tasks.workunit.client.1.smithi157.stdout:4/498: write d3/d11/d1d/f41 [7566512,112027] 0 2024-04-22T23:34:57.101 INFO:tasks.workunit.client.1.smithi157.stdout:4/499: mkdir d3/d11/d34/da4 0 2024-04-22T23:34:57.111 INFO:tasks.workunit.client.1.smithi157.stdout:4/500: read d3/d11/f1f [3055548,40879] 0 2024-04-22T23:34:57.111 INFO:tasks.workunit.client.1.smithi157.stdout:0/320: dwrite d17/f22 [0,4194304] 0 2024-04-22T23:34:57.111 INFO:tasks.workunit.client.1.smithi157.stdout:0/321: creat d17/d38/f6d x:0 0 0 2024-04-22T23:34:57.111 INFO:tasks.workunit.client.1.smithi157.stdout:0/322: chown d17/d2b/f66 53394 1 2024-04-22T23:34:57.112 INFO:tasks.workunit.client.1.smithi157.stdout:4/501: creat d3/d11/d1d/d73/da3/d31/d91/fa5 x:0 0 0 2024-04-22T23:34:57.112 INFO:tasks.workunit.client.1.smithi157.stdout:4/502: stat d3/d96/f67 0 2024-04-22T23:34:57.113 INFO:tasks.workunit.client.1.smithi157.stdout:0/323: symlink d17/d2b/d5b/l6e 0 2024-04-22T23:34:57.114 INFO:tasks.workunit.client.1.smithi157.stdout:4/503: mkdir d3/d11/d34/da6 0 2024-04-22T23:34:57.114 INFO:tasks.workunit.client.1.smithi157.stdout:0/324: mknod d17/d38/d6a/c6f 0 2024-04-22T23:34:57.114 INFO:tasks.workunit.client.1.smithi157.stdout:0/325: truncate d17/d38/f65 941095 0 2024-04-22T23:34:57.115 INFO:tasks.workunit.client.1.smithi157.stdout:4/504: mkdir d3/d11/d1d/d73/da7 0 2024-04-22T23:34:57.129 INFO:tasks.workunit.client.1.smithi157.stdout:0/326: write d17/f20 [1278462,108815] 0 2024-04-22T23:34:57.130 INFO:tasks.workunit.client.1.smithi157.stdout:4/505: read d3/d11/d1d/f26 [7392757,101384] 0 2024-04-22T23:34:57.130 INFO:tasks.workunit.client.1.smithi157.stdout:0/327: read d17/d23/f31 [390049,61417] 0 2024-04-22T23:34:57.135 INFO:tasks.workunit.client.1.smithi157.stdout:4/506: truncate d3/d11/d1d/f41 452078 0 2024-04-22T23:34:57.136 INFO:tasks.workunit.client.1.smithi157.stdout:4/507: stat d3/d4b/l5b 0 2024-04-22T23:34:57.137 INFO:tasks.workunit.client.1.smithi157.stdout:4/508: creat d3/d5c/d7b/fa8 x:0 0 0 2024-04-22T23:34:57.137 INFO:tasks.workunit.client.1.smithi157.stdout:4/509: write d3/d4b/d5d/d6c/f6d [1215179,33066] 0 2024-04-22T23:34:57.137 INFO:tasks.workunit.client.1.smithi157.stdout:4/510: chown d3/d11/d1d/d73/da3/f2b 21234787 1 2024-04-22T23:34:57.138 INFO:tasks.workunit.client.1.smithi157.stdout:1/428: sync 2024-04-22T23:34:57.138 INFO:tasks.workunit.client.1.smithi157.stdout:2/437: sync 2024-04-22T23:34:57.138 INFO:tasks.workunit.client.1.smithi157.stdout:6/489: sync 2024-04-22T23:34:57.138 INFO:tasks.workunit.client.1.smithi157.stdout:1/429: chown d4/d17/d34/f49 0 1 2024-04-22T23:34:57.138 INFO:tasks.workunit.client.1.smithi157.stdout:4/511: dread d3/d11/d30/d36/f65 [0,4194304] 0 2024-04-22T23:34:57.141 INFO:tasks.workunit.client.1.smithi157.stdout:6/490: creat d1/d2f/d3b/d6c/d83/f9f x:0 0 0 2024-04-22T23:34:57.142 INFO:tasks.workunit.client.1.smithi157.stdout:0/328: truncate f11 86624 0 2024-04-22T23:34:57.143 INFO:tasks.workunit.client.1.smithi157.stdout:4/512: link d3/d11/d1d/d73/da3/d31/d91/f9b d3/d11/d1d/d73/da3/fa9 0 2024-04-22T23:34:57.145 INFO:tasks.workunit.client.1.smithi157.stdout:2/438: write d7/d1f/d3e/f96 [424815,110654] 0 2024-04-22T23:34:57.145 INFO:tasks.workunit.client.1.smithi157.stdout:2/439: chown d7/d8/d30/d23/d27 45 1 2024-04-22T23:34:57.145 INFO:tasks.workunit.client.1.smithi157.stdout:2/440: chown d7/d8/d30/l5d 13 1 2024-04-22T23:34:57.146 INFO:tasks.workunit.client.1.smithi157.stdout:0/329: getdents d17/d2b 0 2024-04-22T23:34:57.146 INFO:tasks.workunit.client.1.smithi157.stdout:0/330: write d17/f20 [318288,7679] 0 2024-04-22T23:34:57.148 INFO:tasks.workunit.client.1.smithi157.stdout:2/441: link d7/d8/d95/l65 d7/d8/d30/d5b/d8a/l9d 0 2024-04-22T23:34:57.150 INFO:tasks.workunit.client.1.smithi157.stdout:2/442: truncate d7/d8/d6b/f8c 797393 0 2024-04-22T23:34:57.151 INFO:tasks.workunit.client.1.smithi157.stdout:2/443: mknod d7/d8/d30/d5b/d8a/c9e 0 2024-04-22T23:34:57.153 INFO:tasks.workunit.client.0.smithi107.stderr:++ readlink -f fsstress 2024-04-22T23:34:57.154 INFO:tasks.workunit.client.0.smithi107.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-04-22T23:34:57.154 INFO:tasks.workunit.client.0.smithi107.stderr:+ popd 2024-04-22T23:34:57.155 INFO:tasks.workunit.client.1.smithi157.stdout:2/444: truncate d7/d8/d30/d23/f48 2941470 0 2024-04-22T23:34:57.155 INFO:tasks.workunit.client.1.smithi157.stdout:6/491: read d1/d17/f28 [1505341,3013] 0 2024-04-22T23:34:57.155 INFO:tasks.workunit.client.1.smithi157.stdout:6/492: readlink d1/l1b 0 2024-04-22T23:34:57.157 INFO:tasks.workunit.client.1.smithi157.stdout:2/445: mkdir d7/d9f 0 2024-04-22T23:34:57.157 INFO:tasks.workunit.client.1.smithi157.stdout:2/446: write d7/d8/dd/d4d/f6e [48999,29572] 0 2024-04-22T23:34:57.158 INFO:tasks.workunit.client.1.smithi157.stdout:6/493: link d1/d17/f43 d1/d36/fa0 0 2024-04-22T23:34:57.159 INFO:tasks.workunit.client.0.smithi107.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-22T23:34:57.160 INFO:tasks.workunit.client.0.smithi107.stderr:+ popd 2024-04-22T23:34:57.160 INFO:tasks.workunit.client.0.smithi107.stdout:~/cephtest/mnt.0/client.0/tmp 2024-04-22T23:34:57.160 INFO:tasks.workunit.client.0.smithi107.stderr:++ mktemp -d -p . 2024-04-22T23:34:57.161 INFO:tasks.workunit.client.1.smithi157.stdout:6/494: dread d1/d17/f31 [0,4194304] 0 2024-04-22T23:34:57.161 INFO:tasks.workunit.client.1.smithi157.stdout:6/495: chown d1/d2f/d3b/d6c/f89 5 1 2024-04-22T23:34:57.161 INFO:tasks.workunit.client.1.smithi157.stdout:4/513: dwrite d3/d96/d81/f99 [0,4194304] 0 2024-04-22T23:34:57.162 INFO:tasks.workunit.client.1.smithi157.stdout:2/447: creat d7/d8/fa0 x:0 0 0 2024-04-22T23:34:57.162 INFO:tasks.workunit.client.1.smithi157.stdout:2/448: fdatasync d7/d8/d25/f68 0 2024-04-22T23:34:57.162 INFO:tasks.workunit.client.0.smithi107.stderr:+ T=./tmp.M51zPrONoe 2024-04-22T23:34:57.162 INFO:tasks.workunit.client.0.smithi107.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.M51zPrONoe -l 1 -n 1000 -p 10 -v 2024-04-22T23:34:57.163 INFO:tasks.workunit.client.1.smithi157.stdout:6/496: unlink d1/d36/f47 0 2024-04-22T23:34:57.163 INFO:tasks.workunit.client.0.smithi107.stdout:seed = 1713660624 2024-04-22T23:34:57.167 INFO:tasks.workunit.client.1.smithi157.stdout:4/514: symlink d3/d11/d1d/d73/da3/d31/laa 0 2024-04-22T23:34:57.168 INFO:tasks.workunit.client.0.smithi107.stdout:1/0: chown . 40 1 2024-04-22T23:34:57.169 INFO:tasks.workunit.client.0.smithi107.stdout:0/0: mknod c0 0 2024-04-22T23:34:57.169 INFO:tasks.workunit.client.0.smithi107.stdout:0/1: dwrite - no filename 2024-04-22T23:34:57.169 INFO:tasks.workunit.client.0.smithi107.stdout:0/2: dwrite - no filename 2024-04-22T23:34:57.170 INFO:tasks.workunit.client.0.smithi107.stdout:0/3: dwrite - no filename 2024-04-22T23:34:57.170 INFO:tasks.workunit.client.1.smithi157.stdout:6/497: mknod d1/d2f/d54/d5c/ca1 0 2024-04-22T23:34:57.171 INFO:tasks.workunit.client.1.smithi157.stdout:6/498: chown d1/d2f/d3b/d39/d4e/d74 3 1 2024-04-22T23:34:57.171 INFO:tasks.workunit.client.1.smithi157.stdout:6/499: write d1/d2f/d54/d9b/f61 [393706,6940] 0 2024-04-22T23:34:57.171 INFO:tasks.workunit.client.1.smithi157.stdout:6/500: stat d1/d2f 0 2024-04-22T23:34:57.171 INFO:tasks.workunit.client.1.smithi157.stdout:2/449: read d7/d8/d30/f9c [1490696,95925] 0 2024-04-22T23:34:57.171 INFO:tasks.workunit.client.1.smithi157.stdout:2/450: readlink d7/d8/d95/l61 0 2024-04-22T23:34:57.171 INFO:tasks.workunit.client.1.smithi157.stdout:2/451: write d7/d8/dd/d4d/f99 [2012507,36908] 0 2024-04-22T23:34:57.171 INFO:tasks.workunit.client.1.smithi157.stdout:2/452: truncate d7/d1f/d26/d4a/f53 170326 0 2024-04-22T23:34:57.172 INFO:tasks.workunit.client.0.smithi107.stdout:1/1: mknod c0 0 2024-04-22T23:34:57.172 INFO:tasks.workunit.client.0.smithi107.stdout:1/2: readlink - no filename 2024-04-22T23:34:57.172 INFO:tasks.workunit.client.1.smithi157.stdout:4/515: dread d3/d11/d1d/d73/d80/f8c [0,4194304] 0 2024-04-22T23:34:57.173 INFO:tasks.workunit.client.1.smithi157.stdout:2/453: link d7/d8/d30/c66 d7/ca1 0 2024-04-22T23:34:57.173 INFO:tasks.workunit.client.0.smithi107.stdout:0/4: mkdir d1 0 2024-04-22T23:34:57.174 INFO:tasks.workunit.client.0.smithi107.stdout:0/5: dwrite - no filename 2024-04-22T23:34:57.174 INFO:tasks.workunit.client.0.smithi107.stdout:0/6: dwrite - no filename 2024-04-22T23:34:57.174 INFO:tasks.workunit.client.0.smithi107.stdout:0/7: dread - no filename 2024-04-22T23:34:57.174 INFO:tasks.workunit.client.0.smithi107.stdout:0/8: stat d1 0 2024-04-22T23:34:57.174 INFO:tasks.workunit.client.0.smithi107.stdout:0/9: dwrite - no filename 2024-04-22T23:34:57.174 INFO:tasks.workunit.client.0.smithi107.stdout:0/10: chown d1 2 1 2024-04-22T23:34:57.174 INFO:tasks.workunit.client.0.smithi107.stdout:0/11: write - no filename 2024-04-22T23:34:57.175 INFO:tasks.workunit.client.0.smithi107.stdout:3/0: chown . 29598 1 2024-04-22T23:34:57.175 INFO:tasks.workunit.client.0.smithi107.stdout:3/1: write - no filename 2024-04-22T23:34:57.175 INFO:tasks.workunit.client.0.smithi107.stdout:3/2: chown . 15660810 1 2024-04-22T23:34:57.175 INFO:tasks.workunit.client.0.smithi107.stdout:3/3: unlink - no file 2024-04-22T23:34:57.175 INFO:tasks.workunit.client.0.smithi107.stdout:3/4: getdents . 0 2024-04-22T23:34:57.175 INFO:tasks.workunit.client.0.smithi107.stdout:3/5: write - no filename 2024-04-22T23:34:57.175 INFO:tasks.workunit.client.1.smithi157.stdout:4/516: getdents d3/d5c 0 2024-04-22T23:34:57.176 INFO:tasks.workunit.client.0.smithi107.stdout:2/0: mknod c0 0 2024-04-22T23:34:57.177 INFO:tasks.workunit.client.0.smithi107.stdout:1/3: creat f1 x:0 0 0 2024-04-22T23:34:57.177 INFO:tasks.workunit.client.0.smithi107.stdout:1/4: write f1 [618147,100412] 0 2024-04-22T23:34:57.177 INFO:tasks.workunit.client.1.smithi157.stdout:4/517: link d3/d5c/f79 d3/d96/d40/fab 0 2024-04-22T23:34:57.179 INFO:tasks.workunit.client.0.smithi107.stdout:0/12: creat d1/f2 x:0 0 0 2024-04-22T23:34:57.179 INFO:tasks.workunit.client.0.smithi107.stdout:1/5: fdatasync f1 0 2024-04-22T23:34:57.180 INFO:tasks.workunit.client.1.smithi157.stdout:4/518: rename d3/d4b/d5d/d6e to d3/d11/d1d/d73/d80/dac 0 2024-04-22T23:34:57.180 INFO:tasks.workunit.client.1.smithi157.stdout:2/454: dread d7/d8/dd/d4d/f99 [0,4194304] 0 2024-04-22T23:34:57.180 INFO:tasks.workunit.client.1.smithi157.stdout:2/455: chown d7/d8/d30/l6f 849 1 2024-04-22T23:34:57.180 INFO:tasks.workunit.client.0.smithi107.stdout:4/0: write - no filename 2024-04-22T23:34:57.180 INFO:tasks.workunit.client.0.smithi107.stdout:4/1: getdents . 0 2024-04-22T23:34:57.180 INFO:tasks.workunit.client.0.smithi107.stdout:4/2: dwrite - no filename 2024-04-22T23:34:57.180 INFO:tasks.workunit.client.0.smithi107.stdout:4/3: dwrite - no filename 2024-04-22T23:34:57.181 INFO:tasks.workunit.client.0.smithi107.stdout:4/4: unlink - no file 2024-04-22T23:34:57.181 INFO:tasks.workunit.client.0.smithi107.stdout:4/5: write - no filename 2024-04-22T23:34:57.181 INFO:tasks.workunit.client.0.smithi107.stdout:4/6: chown . 167398240 1 2024-04-22T23:34:57.181 INFO:tasks.workunit.client.0.smithi107.stdout:4/7: fdatasync - no filename 2024-04-22T23:34:57.181 INFO:tasks.workunit.client.0.smithi107.stdout:4/8: rmdir - no directory 2024-04-22T23:34:57.181 INFO:tasks.workunit.client.0.smithi107.stdout:4/9: chown . 2537 1 2024-04-22T23:34:57.181 INFO:tasks.workunit.client.0.smithi107.stdout:4/10: rename - no filename 2024-04-22T23:34:57.181 INFO:tasks.workunit.client.0.smithi107.stdout:4/11: rename - no filename 2024-04-22T23:34:57.181 INFO:tasks.workunit.client.0.smithi107.stdout:3/6: creat f0 x:0 0 0 2024-04-22T23:34:57.182 INFO:tasks.workunit.client.0.smithi107.stdout:3/7: chown f0 0 1 2024-04-22T23:34:57.182 INFO:tasks.workunit.client.0.smithi107.stdout:3/8: creat f1 x:0 0 0 2024-04-22T23:34:57.182 INFO:tasks.workunit.client.0.smithi107.stdout:2/1: mkdir d1 0 2024-04-22T23:34:57.182 INFO:tasks.workunit.client.0.smithi107.stdout:2/2: dread - no filename 2024-04-22T23:34:57.182 INFO:tasks.workunit.client.0.smithi107.stdout:2/3: stat d1 0 2024-04-22T23:34:57.184 INFO:tasks.workunit.client.0.smithi107.stdout:0/13: mknod d1/c3 0 2024-04-22T23:34:57.184 INFO:tasks.workunit.client.1.smithi157.stdout:2/456: dread d7/d8/dd/f2b [0,4194304] 0 2024-04-22T23:34:57.184 INFO:tasks.workunit.client.1.smithi157.stdout:2/457: write d7/d8/dd/d4d/f9b [378086,92600] 0 2024-04-22T23:34:57.186 INFO:tasks.workunit.client.1.smithi157.stdout:6/501: write d1/d2f/d3b/f32 [2842870,87673] 0 2024-04-22T23:34:57.186 INFO:tasks.workunit.client.0.smithi107.stdout:1/6: rename f1 to f2 0 2024-04-22T23:34:57.187 INFO:tasks.workunit.client.0.smithi107.stdout:5/0: rename - no filename 2024-04-22T23:34:57.187 INFO:tasks.workunit.client.0.smithi107.stdout:5/1: dread - no filename 2024-04-22T23:34:57.188 INFO:tasks.workunit.client.1.smithi157.stdout:6/502: creat d1/d17/fa2 x:0 0 0 2024-04-22T23:34:57.188 INFO:tasks.workunit.client.1.smithi157.stdout:6/503: stat d1/d17/l68 0 2024-04-22T23:34:57.188 INFO:tasks.workunit.client.1.smithi157.stdout:6/504: fdatasync d1/d17/fa2 0 2024-04-22T23:34:57.188 INFO:tasks.workunit.client.1.smithi157.stdout:6/505: stat d1/f2c 0 2024-04-22T23:34:57.188 INFO:tasks.workunit.client.0.smithi107.stdout:4/12: creat f0 x:0 0 0 2024-04-22T23:34:57.189 INFO:tasks.workunit.client.0.smithi107.stdout:3/9: mknod c2 0 2024-04-22T23:34:57.190 INFO:tasks.workunit.client.0.smithi107.stdout:1/7: dread f2 [0,4194304] 0 2024-04-22T23:34:57.190 INFO:tasks.workunit.client.0.smithi107.stdout:1/8: creat f3 x:0 0 0 2024-04-22T23:34:57.191 INFO:tasks.workunit.client.0.smithi107.stdout:2/4: symlink d1/l2 0 2024-04-22T23:34:57.192 INFO:tasks.workunit.client.1.smithi157.stdout:6/506: creat d1/d36/fa3 x:0 0 0 2024-04-22T23:34:57.192 INFO:tasks.workunit.client.0.smithi107.stdout:0/14: rename c0 to d1/c4 0 2024-04-22T23:34:57.193 INFO:tasks.workunit.client.0.smithi107.stdout:2/5: rename d1 to d1/d3 22 2024-04-22T23:34:57.193 INFO:tasks.workunit.client.0.smithi107.stdout:2/6: fdatasync - no filename 2024-04-22T23:34:57.193 INFO:tasks.workunit.client.0.smithi107.stdout:2/7: write - no filename 2024-04-22T23:34:57.193 INFO:tasks.workunit.client.0.smithi107.stdout:2/8: write - no filename 2024-04-22T23:34:57.193 INFO:tasks.workunit.client.0.smithi107.stdout:6/0: write - no filename 2024-04-22T23:34:57.193 INFO:tasks.workunit.client.0.smithi107.stdout:6/1: truncate - no filename 2024-04-22T23:34:57.193 INFO:tasks.workunit.client.0.smithi107.stdout:6/2: rename - no filename 2024-04-22T23:34:57.193 INFO:tasks.workunit.client.0.smithi107.stdout:6/3: dwrite - no filename 2024-04-22T23:34:57.193 INFO:tasks.workunit.client.0.smithi107.stdout:6/4: write - no filename 2024-04-22T23:34:57.194 INFO:tasks.workunit.client.0.smithi107.stdout:6/5: dread - no filename 2024-04-22T23:34:57.194 INFO:tasks.workunit.client.0.smithi107.stdout:6/6: write - no filename 2024-04-22T23:34:57.194 INFO:tasks.workunit.client.0.smithi107.stdout:5/2: creat f0 x:0 0 0 2024-04-22T23:34:57.194 INFO:tasks.workunit.client.0.smithi107.stdout:5/3: rmdir - no directory 2024-04-22T23:34:57.195 INFO:tasks.workunit.client.0.smithi107.stdout:5/4: read - f0 zero size 2024-04-22T23:34:57.195 INFO:tasks.workunit.client.0.smithi107.stdout:5/5: truncate f0 23612 0 2024-04-22T23:34:57.195 INFO:tasks.workunit.client.0.smithi107.stdout:4/13: mkdir d1 0 2024-04-22T23:34:57.195 INFO:tasks.workunit.client.1.smithi157.stdout:6/507: rename d1/d2f/d3b/d6c/d83/l8b to d1/la4 0 2024-04-22T23:34:57.195 INFO:tasks.workunit.client.1.smithi157.stdout:6/508: dread - d1/d2f/d3b/d49/f7d zero size 2024-04-22T23:34:57.196 INFO:tasks.workunit.client.1.smithi157.stdout:6/509: creat d1/d2f/d54/fa5 x:0 0 0 2024-04-22T23:34:57.198 INFO:tasks.workunit.client.0.smithi107.stdout:1/9: link c0 c4 0 2024-04-22T23:34:57.198 INFO:tasks.workunit.client.0.smithi107.stdout:1/10: write f3 [501278,86519] 0 2024-04-22T23:34:57.198 INFO:tasks.workunit.client.0.smithi107.stdout:1/11: rmdir - no directory 2024-04-22T23:34:57.200 INFO:tasks.workunit.client.0.smithi107.stdout:2/9: creat d1/f4 x:0 0 0 2024-04-22T23:34:57.201 INFO:tasks.workunit.client.0.smithi107.stdout:2/10: creat d1/f5 x:0 0 0 2024-04-22T23:34:57.201 INFO:tasks.workunit.client.0.smithi107.stdout:2/11: creat d1/f6 x:0 0 0 2024-04-22T23:34:57.201 INFO:tasks.workunit.client.0.smithi107.stdout:7/0: read - no filename 2024-04-22T23:34:57.202 INFO:tasks.workunit.client.0.smithi107.stdout:7/1: dread - no filename 2024-04-22T23:34:57.202 INFO:tasks.workunit.client.0.smithi107.stdout:7/2: dwrite - no filename 2024-04-22T23:34:57.202 INFO:tasks.workunit.client.0.smithi107.stdout:1/12: dread f2 [0,4194304] 0 2024-04-22T23:34:57.202 INFO:tasks.workunit.client.0.smithi107.stdout:1/13: write f2 [734127,109950] 0 2024-04-22T23:34:57.202 INFO:tasks.workunit.client.0.smithi107.stdout:1/14: chown f3 1208055 1 2024-04-22T23:34:57.203 INFO:tasks.workunit.client.0.smithi107.stdout:6/7: creat f0 x:0 0 0 2024-04-22T23:34:57.204 INFO:tasks.workunit.client.0.smithi107.stdout:5/6: rename f0 to f1 0 2024-04-22T23:34:57.206 INFO:tasks.workunit.client.0.smithi107.stdout:4/14: mknod d1/c2 0 2024-04-22T23:34:57.206 INFO:tasks.workunit.client.0.smithi107.stdout:4/15: chown f0 1 1 2024-04-22T23:34:57.208 INFO:tasks.workunit.client.1.smithi157.stdout:4/519: dwrite d3/d11/d1d/f41 [0,4194304] 0 2024-04-22T23:34:57.208 INFO:tasks.workunit.client.1.smithi157.stdout:2/458: dwrite d7/d8/fa0 [0,4194304] 0 2024-04-22T23:34:57.209 INFO:tasks.workunit.client.0.smithi107.stdout:2/12: mknod d1/c7 0 2024-04-22T23:34:57.210 INFO:tasks.workunit.client.1.smithi157.stdout:2/459: mknod d7/d8/d8e/ca2 0 2024-04-22T23:34:57.212 INFO:tasks.workunit.client.0.smithi107.stdout:7/3: symlink l0 0 2024-04-22T23:34:57.212 INFO:tasks.workunit.client.1.smithi157.stdout:4/520: dread d3/d4b/f18 [0,4194304] 0 2024-04-22T23:34:57.212 INFO:tasks.workunit.client.1.smithi157.stdout:2/460: mknod d7/d8/d30/d23/d27/d57/ca3 0 2024-04-22T23:34:57.212 INFO:tasks.workunit.client.1.smithi157.stdout:2/461: readlink d7/d8/l1d 0 2024-04-22T23:34:57.213 INFO:tasks.workunit.client.1.smithi157.stdout:2/462: chown d7/d8/d6b/f89 4881046 1 2024-04-22T23:34:57.213 INFO:tasks.workunit.client.1.smithi157.stdout:2/463: creat d7/d8/fa4 x:0 0 0 2024-04-22T23:34:57.213 INFO:tasks.workunit.client.1.smithi157.stdout:4/521: write d3/d4b/f18 [1842458,94275] 0 2024-04-22T23:34:57.213 INFO:tasks.workunit.client.0.smithi107.stdout:1/15: mknod c5 0 2024-04-22T23:34:57.213 INFO:tasks.workunit.client.1.smithi157.stdout:2/464: truncate d7/d8/d25/f4f 1891360 0 2024-04-22T23:34:57.214 INFO:tasks.workunit.client.1.smithi157.stdout:2/465: dread - d7/d1f/d26/f74 zero size 2024-04-22T23:34:57.214 INFO:tasks.workunit.client.0.smithi107.stdout:5/7: mknod c2 0 2024-04-22T23:34:57.215 INFO:tasks.workunit.client.1.smithi157.stdout:4/522: link d3/d11/d34/f51 d3/d11/d1d/d73/da7/fad 0 2024-04-22T23:34:57.215 INFO:tasks.workunit.client.1.smithi157.stdout:4/523: fsync d3/f58 0 2024-04-22T23:34:57.217 INFO:tasks.workunit.client.1.smithi157.stdout:2/466: creat d7/d8/d30/d5b/fa5 x:0 0 0 2024-04-22T23:34:57.217 INFO:tasks.workunit.client.0.smithi107.stdout:4/16: rename f0 to d1/f3 0 2024-04-22T23:34:57.217 INFO:tasks.workunit.client.0.smithi107.stdout:4/17: read - d1/f3 zero size 2024-04-22T23:34:57.220 INFO:tasks.workunit.client.0.smithi107.stdout:2/13: unlink d1/c7 0 2024-04-22T23:34:57.221 INFO:tasks.workunit.client.0.smithi107.stdout:2/14: write d1/f4 [54166,129081] 0 2024-04-22T23:34:57.221 INFO:tasks.workunit.client.0.smithi107.stdout:2/15: creat d1/f8 x:0 0 0 2024-04-22T23:34:57.221 INFO:tasks.workunit.client.0.smithi107.stdout:2/16: chown d1/f8 0 1 2024-04-22T23:34:57.221 INFO:tasks.workunit.client.0.smithi107.stdout:2/17: dread - d1/f8 zero size 2024-04-22T23:34:57.221 INFO:tasks.workunit.client.1.smithi157.stdout:2/467: truncate d7/d8/d25/f3a 1331325 0 2024-04-22T23:34:57.222 INFO:tasks.workunit.client.1.smithi157.stdout:6/510: dwrite d1/d2f/d3b/d6c/f89 [0,4194304] 0 2024-04-22T23:34:57.222 INFO:tasks.workunit.client.1.smithi157.stdout:6/511: write d1/f22 [1901594,48983] 0 2024-04-22T23:34:57.222 INFO:tasks.workunit.client.1.smithi157.stdout:6/512: creat d1/d2f/d3b/d6c/d83/fa6 x:0 0 0 2024-04-22T23:34:57.222 INFO:tasks.workunit.client.1.smithi157.stdout:6/513: write d1/d2f/d3b/d49/f7d [235762,10984] 0 2024-04-22T23:34:57.222 INFO:tasks.workunit.client.1.smithi157.stdout:6/514: write d1/d2f/d3b/d6c/d83/f73 [495977,50795] 0 2024-04-22T23:34:57.223 INFO:tasks.workunit.client.0.smithi107.stdout:9/0: creat f0 x:0 0 0 2024-04-22T23:34:57.224 INFO:tasks.workunit.client.0.smithi107.stdout:9/1: dread - f0 zero size 2024-04-22T23:34:57.224 INFO:tasks.workunit.client.0.smithi107.stdout:9/2: dread - f0 zero size 2024-04-22T23:34:57.224 INFO:tasks.workunit.client.0.smithi107.stdout:9/3: truncate f0 823823 0 2024-04-22T23:34:57.224 INFO:tasks.workunit.client.0.smithi107.stdout:7/4: rename l0 to l1 0 2024-04-22T23:34:57.224 INFO:tasks.workunit.client.0.smithi107.stdout:7/5: fsync - no filename 2024-04-22T23:34:57.224 INFO:tasks.workunit.client.0.smithi107.stdout:7/6: chown l1 319 1 2024-04-22T23:34:57.224 INFO:tasks.workunit.client.0.smithi107.stdout:7/7: write - no filename 2024-04-22T23:34:57.225 INFO:tasks.workunit.client.0.smithi107.stdout:7/8: rmdir - no directory 2024-04-22T23:34:57.225 INFO:tasks.workunit.client.0.smithi107.stdout:7/9: truncate - no filename 2024-04-22T23:34:57.225 INFO:tasks.workunit.client.0.smithi107.stdout:7/10: write - no filename 2024-04-22T23:34:57.225 INFO:tasks.workunit.client.0.smithi107.stdout:7/11: truncate - no filename 2024-04-22T23:34:57.225 INFO:tasks.workunit.client.1.smithi157.stdout:2/468: creat d7/d8/d30/fa6 x:0 0 0 2024-04-22T23:34:57.225 INFO:tasks.workunit.client.1.smithi157.stdout:2/469: fsync d7/d8/d30/d23/d27/f58 0 2024-04-22T23:34:57.227 INFO:tasks.workunit.client.0.smithi107.stdout:2/18: symlink d1/l9 0 2024-04-22T23:34:57.230 INFO:tasks.workunit.client.0.smithi107.stdout:8/0: mkdir d0 0 2024-04-22T23:34:57.230 INFO:tasks.workunit.client.1.smithi157.stdout:2/470: creat d7/d8/d30/d23/d27/d57/fa7 x:0 0 0 2024-04-22T23:34:57.231 INFO:tasks.workunit.client.0.smithi107.stdout:7/12: creat f2 x:0 0 0 2024-04-22T23:34:57.231 INFO:tasks.workunit.client.0.smithi107.stdout:7/13: chown f2 172361 1 2024-04-22T23:34:57.231 INFO:tasks.workunit.client.0.smithi107.stdout:7/14: write f2 [1012684,63224] 0 2024-04-22T23:34:57.233 INFO:tasks.workunit.client.1.smithi157.stdout:2/471: creat d7/d8/d95/fa8 x:0 0 0 2024-04-22T23:34:57.233 INFO:tasks.workunit.client.1.smithi157.stdout:2/472: truncate d7/d8/d30/f79 875708 0 2024-04-22T23:34:57.233 INFO:tasks.workunit.client.1.smithi157.stdout:2/473: dread - d7/d8/d8e/f88 zero size 2024-04-22T23:34:57.233 INFO:tasks.workunit.client.1.smithi157.stdout:2/474: getdents d7/d8/d25/d41 0 2024-04-22T23:34:57.234 INFO:tasks.workunit.client.1.smithi157.stdout:6/515: dread d1/d2f/d54/d9b/f61 [0,4194304] 0 2024-04-22T23:34:57.234 INFO:tasks.workunit.client.1.smithi157.stdout:2/475: symlink d7/d8/d30/d5b/d8a/la9 0 2024-04-22T23:34:57.239 INFO:tasks.workunit.client.1.smithi157.stdout:6/516: dread d1/f12 [0,4194304] 0 2024-04-22T23:34:57.239 INFO:tasks.workunit.client.1.smithi157.stdout:6/517: write d1/f7 [399638,117462] 0 2024-04-22T23:34:57.240 INFO:tasks.workunit.client.0.smithi107.stdout:5/8: dwrite f1 [0,4194304] 0 2024-04-22T23:34:57.241 INFO:tasks.workunit.client.0.smithi107.stdout:6/8: dwrite f0 [0,4194304] 0 2024-04-22T23:34:57.243 INFO:tasks.workunit.client.1.smithi157.stdout:4/524: dwrite d3/d4b/f5 [0,4194304] 0 2024-04-22T23:34:57.254 INFO:tasks.workunit.client.1.smithi157.stdout:4/525: dread d3/d11/d34/f48 [0,4194304] 0 2024-04-22T23:34:57.254 INFO:tasks.workunit.client.1.smithi157.stdout:4/526: chown d3/d96/d81/c8a 0 1 2024-04-22T23:34:57.254 INFO:tasks.workunit.client.1.smithi157.stdout:4/527: chown d3/d11/d1d/d73/da3/d31/da2 117170 1 2024-04-22T23:34:57.255 INFO:tasks.workunit.client.1.smithi157.stdout:2/476: dwrite d7/d8/d30/d23/d27/f58 [0,4194304] 0 2024-04-22T23:34:57.261 INFO:tasks.workunit.client.1.smithi157.stdout:6/518: dwrite d1/d2f/d3b/d6c/d83/f73 [0,4194304] 0 2024-04-22T23:34:57.263 INFO:tasks.workunit.client.1.smithi157.stdout:6/519: write d1/d36/f37 [803701,92921] 0 2024-04-22T23:34:57.265 INFO:tasks.workunit.client.1.smithi157.stdout:2/477: truncate d7/d8/d30/f9c 3013845 0 2024-04-22T23:34:57.265 INFO:tasks.workunit.client.1.smithi157.stdout:5/393: sync 2024-04-22T23:34:57.265 INFO:tasks.workunit.client.1.smithi157.stdout:7/315: sync 2024-04-22T23:34:57.266 INFO:tasks.workunit.client.1.smithi157.stdout:2/478: dread d7/d8/d30/d23/d27/f67 [0,4194304] 0 2024-04-22T23:34:57.266 INFO:tasks.workunit.client.1.smithi157.stdout:2/479: truncate d7/d8/d25/f68 831030 0 2024-04-22T23:34:57.266 INFO:tasks.workunit.client.1.smithi157.stdout:2/480: fsync d7/d8/d25/f4f 0 2024-04-22T23:34:57.266 INFO:tasks.workunit.client.1.smithi157.stdout:7/316: mknod d0/d9/d29/d49/d64/c70 0 2024-04-22T23:34:57.268 INFO:tasks.workunit.client.1.smithi157.stdout:5/394: mkdir d1/d5/dc/d43/d4f/d7f/d99 0 2024-04-22T23:34:57.268 INFO:tasks.workunit.client.1.smithi157.stdout:5/395: write d1/d5/d7/d10/f49 [682397,60570] 0 2024-04-22T23:34:57.268 INFO:tasks.workunit.client.1.smithi157.stdout:7/317: mknod d0/d9/db/df/d68/c71 0 2024-04-22T23:34:57.272 INFO:tasks.workunit.client.1.smithi157.stdout:4/528: dwrite d3/f1b [4194304,4194304] 0 2024-04-22T23:34:57.272 INFO:tasks.workunit.client.1.smithi157.stdout:7/318: rename d0/d9/d10/d43 to d0/d9/d72 0 2024-04-22T23:34:57.273 INFO:tasks.workunit.client.1.smithi157.stdout:4/529: symlink d3/d96/lae 0 2024-04-22T23:34:57.280 INFO:tasks.workunit.client.1.smithi157.stdout:5/396: dread d1/d5/f9 [0,4194304] 0 2024-04-22T23:34:57.280 INFO:tasks.workunit.client.1.smithi157.stdout:4/530: dread d3/d4b/d7a/f9c [0,4194304] 0 2024-04-22T23:34:57.282 INFO:tasks.workunit.client.1.smithi157.stdout:4/531: getdents d3/d11/d30 0 2024-04-22T23:34:57.284 INFO:tasks.workunit.client.1.smithi157.stdout:4/532: rename d3/d4b/d7a/d86 to d3/d11/d1d/daf 0 2024-04-22T23:34:57.284 INFO:tasks.workunit.client.1.smithi157.stdout:6/520: dwrite d1/d2f/d3b/d6c/d83/f71 [0,4194304] 0 2024-04-22T23:34:57.286 INFO:tasks.workunit.client.1.smithi157.stdout:4/533: mknod d3/d11/d1d/daf/cb0 0 2024-04-22T23:34:57.286 INFO:tasks.workunit.client.1.smithi157.stdout:6/521: mkdir d1/d2f/d3b/d6c/da7 0 2024-04-22T23:34:57.287 INFO:tasks.workunit.client.1.smithi157.stdout:6/522: truncate d1/d2f/d3b/d6c/d83/f9f 590056 0 2024-04-22T23:34:57.288 INFO:tasks.workunit.client.1.smithi157.stdout:7/319: dwrite d0/f22 [8388608,4194304] 0 2024-04-22T23:34:57.291 INFO:tasks.workunit.client.1.smithi157.stdout:5/397: dread d1/d27/d29/f2a [0,4194304] 0 2024-04-22T23:34:57.291 INFO:tasks.workunit.client.1.smithi157.stdout:5/398: fdatasync d1/d27/d29/d4b/f5c 0 2024-04-22T23:34:57.293 INFO:tasks.workunit.client.1.smithi157.stdout:5/399: link d1/d1f/d22/l7e d1/d5/d7/d2f/d45/d54/l9a 0 2024-04-22T23:34:57.293 INFO:tasks.workunit.client.1.smithi157.stdout:5/400: getdents d1/d5/d7 0 2024-04-22T23:34:57.294 INFO:tasks.workunit.client.1.smithi157.stdout:5/401: symlink d1/d5/dc/d43/l9b 0 2024-04-22T23:34:57.295 INFO:tasks.workunit.client.1.smithi157.stdout:4/534: dread d3/d11/d1d/f26 [8388608,4194304] 0 2024-04-22T23:34:57.295 INFO:tasks.workunit.client.1.smithi157.stdout:5/402: write d1/d5/d7/d2f/d45/f48 [73569,15474] 0 2024-04-22T23:34:57.296 INFO:tasks.workunit.client.1.smithi157.stdout:4/535: mkdir d3/d11/d34/da4/db1 0 2024-04-22T23:34:57.297 INFO:tasks.workunit.client.1.smithi157.stdout:5/403: mkdir d1/d5/d7/d2f/d9c 0 2024-04-22T23:34:57.298 INFO:tasks.workunit.client.1.smithi157.stdout:4/536: symlink d3/d96/d40/lb2 0 2024-04-22T23:34:57.298 INFO:tasks.workunit.client.1.smithi157.stdout:4/537: write d3/d11/d1d/d73/da3/d31/f6f [948849,63749] 0 2024-04-22T23:34:57.299 INFO:tasks.workunit.client.1.smithi157.stdout:4/538: mkdir d3/d96/d81/db3 0 2024-04-22T23:34:57.301 INFO:tasks.workunit.client.1.smithi157.stdout:6/523: write d1/d2f/f4a [3524571,1604] 0 2024-04-22T23:34:57.303 INFO:tasks.workunit.client.1.smithi157.stdout:7/320: read d0/d9/d42/f51 [1614842,63232] 0 2024-04-22T23:34:57.305 INFO:tasks.workunit.client.1.smithi157.stdout:4/539: dread d3/d11/d1d/d73/da3/d31/f6f [0,4194304] 0 2024-04-22T23:34:57.308 INFO:tasks.workunit.client.1.smithi157.stdout:7/321: mknod d0/d9/d42/d55/c73 0 2024-04-22T23:34:57.310 INFO:tasks.workunit.client.1.smithi157.stdout:7/322: link d0/d9/f66 d0/d9/db/f74 0 2024-04-22T23:34:57.320 INFO:tasks.workunit.client.1.smithi157.stdout:6/524: dwrite d1/d17/d5d/f64 [0,4194304] 0 2024-04-22T23:34:57.320 INFO:tasks.workunit.client.1.smithi157.stdout:5/404: dwrite d1/d5/d7/d2f/d45/d54/f82 [0,4194304] 0 2024-04-22T23:34:57.322 INFO:tasks.workunit.client.1.smithi157.stdout:6/525: unlink d1/d2f/d3b/d6c/f89 0 2024-04-22T23:34:57.324 INFO:tasks.workunit.client.1.smithi157.stdout:5/405: rename d1/d1f/l2c to d1/d5/dc/d43/d4f/d60/l9d 0 2024-04-22T23:34:57.325 INFO:tasks.workunit.client.1.smithi157.stdout:6/526: truncate d1/d2f/d54/d9b/f61 63442 0 2024-04-22T23:34:57.325 INFO:tasks.workunit.client.1.smithi157.stdout:6/527: stat d1/d2f/d3b/c6f 0 2024-04-22T23:34:57.325 INFO:tasks.workunit.client.1.smithi157.stdout:6/528: chown d1/d2f/d3b/d39/c90 140405 1 2024-04-22T23:34:57.327 INFO:tasks.workunit.client.1.smithi157.stdout:6/529: rmdir d1/d2f/d3b/d39/d4e/d74 39 2024-04-22T23:34:57.327 INFO:tasks.workunit.client.1.smithi157.stdout:5/406: dread d1/d27/d29/d4b/f5c [0,4194304] 0 2024-04-22T23:34:57.327 INFO:tasks.workunit.client.1.smithi157.stdout:5/407: dread - d1/d5/d7/f72 zero size 2024-04-22T23:34:57.327 INFO:tasks.workunit.client.1.smithi157.stdout:5/408: chown d1/d27/d29/f58 1221354 1 2024-04-22T23:34:57.327 INFO:tasks.workunit.client.1.smithi157.stdout:5/409: write d1/d5/dc/f89 [62256,72802] 0 2024-04-22T23:34:57.327 INFO:tasks.workunit.client.1.smithi157.stdout:5/410: write d1/d5/d7/d2f/d45/d54/f74 [850805,38210] 0 2024-04-22T23:34:57.327 INFO:tasks.workunit.client.1.smithi157.stdout:5/411: dread d1/d27/d29/f58 [0,4194304] 0 2024-04-22T23:34:57.328 INFO:tasks.workunit.client.1.smithi157.stdout:5/412: symlink d1/d5/l9e 0 2024-04-22T23:34:57.329 INFO:tasks.workunit.client.1.smithi157.stdout:6/530: dread d1/f3 [0,4194304] 0 2024-04-22T23:34:57.331 INFO:tasks.workunit.client.1.smithi157.stdout:5/413: link d1/d5/d7/d2f/d51/c84 d1/d27/d29/d4b/c9f 0 2024-04-22T23:34:57.332 INFO:tasks.workunit.client.1.smithi157.stdout:6/531: mkdir d1/d2f/d3b/d39/d4e/da8 0 2024-04-22T23:34:57.333 INFO:tasks.workunit.client.1.smithi157.stdout:5/414: rmdir d1/d5/dc/d43/d4f 39 2024-04-22T23:34:57.333 INFO:tasks.workunit.client.1.smithi157.stdout:5/415: chown d1/d5/dc/l16 15663 1 2024-04-22T23:34:57.335 INFO:tasks.workunit.client.1.smithi157.stdout:6/532: link d1/d2f/f40 d1/d2f/d3b/d39/d4e/d74/d8c/fa9 0 2024-04-22T23:34:57.356 INFO:tasks.workunit.client.1.smithi157.stdout:5/416: dwrite d1/d5/d7/d2f/f79 [0,4194304] 0 2024-04-22T23:34:57.356 INFO:tasks.workunit.client.1.smithi157.stdout:5/417: read - d1/d5/d7/d2f/d45/f53 zero size 2024-04-22T23:34:57.359 INFO:tasks.workunit.client.1.smithi157.stdout:5/418: symlink d1/d5/d7/d2f/d45/d54/la0 0 2024-04-22T23:34:57.359 INFO:tasks.workunit.client.1.smithi157.stdout:5/419: write d1/d5/d7/f56 [924983,87136] 0 2024-04-22T23:34:57.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:57.367+0000 7f739416e700 1 -- 172.21.15.107:0/1593697064 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f7370005590 con 0x7f738c0fc140 2024-04-22T23:34:57.362 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:34:57 smithi107 ceph-mon[26355]: pgmap v255: 65 pgs: 65 active+clean; 472 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 34 MiB/s rd, 48 MiB/s wr, 968 op/s 2024-04-22T23:34:57.362 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:34:57 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/4119674625' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:34:57.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:57.367+0000 7f73827fc700 1 -- 172.21.15.107:0/1593697064 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f7384004e10 con 0x7f738c0fc140 2024-04-22T23:34:57.363 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_OK 2024-04-22T23:34:57.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:57.370+0000 7f739416e700 1 -- 172.21.15.107:0/1593697064 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f7378069300 msgr2=0x7f737806b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:34:57.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:57.370+0000 7f739416e700 1 --2- 172.21.15.107:0/1593697064 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f7378069300 0x7f737806b7b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f737c004ad0 tx=0x7f737c007d40 comp rx=0 tx=0).stop 2024-04-22T23:34:57.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:57.370+0000 7f739416e700 1 -- 172.21.15.107:0/1593697064 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f738c0fc140 msgr2=0x7f738c1141f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:34:57.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:57.370+0000 7f739416e700 1 --2- 172.21.15.107:0/1593697064 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f738c0fc140 0x7f738c1141f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7384009380 tx=0x7f7384004d10 comp rx=0 tx=0).stop 2024-04-22T23:34:57.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:57.370+0000 7f739416e700 1 -- 172.21.15.107:0/1593697064 shutdown_connections 2024-04-22T23:34:57.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:57.370+0000 7f739416e700 1 --2- 172.21.15.107:0/1593697064 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f7378069300 0x7f737806b7b0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:57.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:57.370+0000 7f739416e700 1 --2- 172.21.15.107:0/1593697064 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f738c0fc140 0x7f738c1141f0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:57.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:57.370+0000 7f739416e700 1 --2- 172.21.15.107:0/1593697064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f738c0fca90 0x7f738c114730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:34:57.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:57.370+0000 7f739416e700 1 -- 172.21.15.107:0/1593697064 >> 172.21.15.107:0/1593697064 conn(0x7f738c0f6bd0 msgr2=0x7f738c0f8340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:34:57.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:57.370+0000 7f739416e700 1 -- 172.21.15.107:0/1593697064 shutdown_connections 2024-04-22T23:34:57.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:34:57.370+0000 7f739416e700 1 -- 172.21.15.107:0/1593697064 wait complete. 2024-04-22T23:34:57.372 INFO:tasks.workunit.client.1.smithi157.stdout:6/533: dwrite d1/d2f/d3b/d39/d4e/d74/d8c/fa9 [0,4194304] 0 2024-04-22T23:34:57.372 INFO:tasks.workunit.client.1.smithi157.stdout:6/534: fdatasync d1/d2f/d3b/f2a 0 2024-04-22T23:34:57.373 INFO:tasks.workunit.client.1.smithi157.stdout:5/420: dwrite d1/d5/d7/d2f/d45/d54/d59/f70 [0,4194304] 0 2024-04-22T23:34:57.373 INFO:tasks.workunit.client.1.smithi157.stdout:5/421: write d1/d5/dc/d43/d4f/d60/d7a/f8f [219263,25979] 0 2024-04-22T23:34:57.374 INFO:tasks.workunit.client.1.smithi157.stdout:6/535: rmdir d1/d2f/d3b/d6c/d83/d93 39 2024-04-22T23:34:57.378 INFO:tasks.workunit.client.1.smithi157.stdout:5/422: unlink d1/d5/dc/l16 0 2024-04-22T23:34:57.378 INFO:tasks.workunit.client.1.smithi157.stdout:5/423: getdents d1/d5/d7/d2f/d9c 0 2024-04-22T23:34:57.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:34:57 smithi157 ceph-mon[33877]: pgmap v255: 65 pgs: 65 active+clean; 472 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 34 MiB/s rd, 48 MiB/s wr, 968 op/s 2024-04-22T23:34:57.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:34:57 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/4119674625' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:34:57.383 INFO:tasks.workunit.client.1.smithi157.stdout:5/424: rename d1/f3f to d1/fa1 0 2024-04-22T23:34:57.384 INFO:tasks.workunit.client.1.smithi157.stdout:5/425: stat d1/cd 0 2024-04-22T23:34:57.390 INFO:tasks.workunit.client.1.smithi157.stdout:5/426: mknod d1/d1f/d22/ca2 0 2024-04-22T23:34:57.391 INFO:tasks.workunit.client.1.smithi157.stdout:6/536: dread d1/f18 [0,4194304] 0 2024-04-22T23:34:57.393 INFO:tasks.workunit.client.1.smithi157.stdout:6/537: dread d1/d2f/d54/d9b/f79 [0,4194304] 0 2024-04-22T23:34:57.393 INFO:tasks.workunit.client.1.smithi157.stdout:6/538: stat d1/d2f/d3b/d6c/d83/c5b 0 2024-04-22T23:34:57.412 INFO:tasks.workunit.client.1.smithi157.stdout:5/427: dwrite d1/f26 [4194304,4194304] 0 2024-04-22T23:34:57.414 INFO:tasks.workunit.client.1.smithi157.stdout:5/428: creat d1/d5/dc/d43/d4f/d60/d7a/fa3 x:0 0 0 2024-04-22T23:34:57.415 INFO:tasks.workunit.client.1.smithi157.stdout:6/539: dwrite d1/d36/fa3 [0,4194304] 0 2024-04-22T23:34:57.417 INFO:tasks.workunit.client.1.smithi157.stdout:6/540: dread d1/d17/f24 [0,4194304] 0 2024-04-22T23:34:57.420 INFO:tasks.workunit.client.1.smithi157.stdout:6/541: rename d1/f22 to d1/d38/faa 0 2024-04-22T23:34:57.420 INFO:tasks.workunit.client.1.smithi157.stdout:6/542: chown d1/d38/d97 680069 1 2024-04-22T23:34:57.421 INFO:tasks.workunit.client.1.smithi157.stdout:6/543: getdents d1/d2f/d3b/d39 0 2024-04-22T23:34:57.421 INFO:tasks.workunit.client.1.smithi157.stdout:6/544: stat d1/f2c 0 2024-04-22T23:34:57.421 INFO:tasks.workunit.client.1.smithi157.stdout:6/545: symlink d1/d2f/d3b/d39/lab 0 2024-04-22T23:34:57.422 INFO:tasks.workunit.client.1.smithi157.stdout:6/546: write d1/d2f/f4a [2227983,14141] 0 2024-04-22T23:34:57.431 INFO:tasks.workunit.client.1.smithi157.stdout:6/547: write d1/d2f/d3b/f3e [2289924,109641] 0 2024-04-22T23:34:57.432 INFO:tasks.workunit.client.1.smithi157.stdout:6/548: mkdir d1/d2f/d3b/d39/d4e/da8/dac 0 2024-04-22T23:34:57.432 INFO:tasks.workunit.client.1.smithi157.stdout:6/549: rename d1/d2f/d3b to d1/d2f/d3b/d6c/d83/dad 22 2024-04-22T23:34:57.432 INFO:tasks.workunit.client.1.smithi157.stdout:6/550: truncate d1/d2f/d54/fa5 223129 0 2024-04-22T23:34:57.443 INFO:tasks.workunit.client.1.smithi157.stdout:6/551: write d1/d2f/d3b/d6c/d83/f59 [4157694,62585] 0 2024-04-22T23:34:57.444 INFO:tasks.workunit.client.1.smithi157.stdout:6/552: getdents d1/d2f/d3b/d6c/d83/d93 0 2024-04-22T23:34:57.445 INFO:tasks.workunit.client.1.smithi157.stdout:6/553: getdents d1/d2f/d3b 0 2024-04-22T23:34:57.445 INFO:tasks.workunit.client.1.smithi157.stdout:6/554: write d1/d2f/d3b/f2e [130537,62794] 0 2024-04-22T23:34:57.445 INFO:tasks.workunit.client.1.smithi157.stdout:3/309: sync 2024-04-22T23:34:57.446 INFO:tasks.workunit.client.1.smithi157.stdout:6/555: truncate d1/d38/faa 1787840 0 2024-04-22T23:34:57.446 INFO:tasks.workunit.client.1.smithi157.stdout:6/556: chown d1/f34 366762585 1 2024-04-22T23:34:57.447 INFO:tasks.workunit.client.1.smithi157.stdout:3/310: symlink d0/d3/d34/d39/l5f 0 2024-04-22T23:34:57.447 INFO:tasks.workunit.client.1.smithi157.stdout:6/557: rmdir d1/d2f/d3b/d39/d4e/d74 39 2024-04-22T23:34:57.449 INFO:tasks.workunit.client.1.smithi157.stdout:6/558: dread d1/d17/f94 [0,4194304] 0 2024-04-22T23:34:57.449 INFO:tasks.workunit.client.1.smithi157.stdout:6/559: chown d1/l1b 37258 1 2024-04-22T23:34:57.454 INFO:tasks.workunit.client.1.smithi157.stdout:6/560: dread d1/d36/f37 [0,4194304] 0 2024-04-22T23:34:57.454 INFO:tasks.workunit.client.1.smithi157.stdout:6/561: write d1/d2f/d54/f95 [7122076,76794] 0 2024-04-22T23:34:57.454 INFO:tasks.workunit.client.1.smithi157.stdout:6/562: write d1/d17/f28 [2937863,97709] 0 2024-04-22T23:34:57.457 INFO:tasks.workunit.client.1.smithi157.stdout:6/563: rename d1/d2f/d54/d86/c96 to d1/d2f/d3b/d6c/cae 0 2024-04-22T23:34:57.459 INFO:tasks.workunit.client.1.smithi157.stdout:6/564: mkdir d1/d38/daf 0 2024-04-22T23:34:57.463 INFO:tasks.workunit.client.0.smithi107.stdout:3/10: fsync f0 0 2024-04-22T23:34:57.463 INFO:tasks.workunit.client.0.smithi107.stdout:3/11: read - f1 zero size 2024-04-22T23:34:57.463 INFO:tasks.workunit.client.0.smithi107.stdout:3/12: chown f0 24289010 1 2024-04-22T23:34:57.463 INFO:tasks.workunit.client.0.smithi107.stdout:3/13: dread - f1 zero size 2024-04-22T23:34:57.463 INFO:tasks.workunit.client.1.smithi157.stdout:6/565: mkdir d1/d2f/d3b/d39/d4e/da8/dac/db0 0 2024-04-22T23:34:57.463 INFO:tasks.workunit.client.1.smithi157.stdout:6/566: chown d1/d36/c44 29506 1 2024-04-22T23:34:57.464 INFO:tasks.workunit.client.0.smithi107.stdout:3/14: mknod c3 0 2024-04-22T23:34:57.465 INFO:tasks.workunit.client.1.smithi157.stdout:3/311: dwrite d0/f11 [0,4194304] 0 2024-04-22T23:34:57.468 INFO:tasks.workunit.client.0.smithi107.stdout:0/15: rmdir d1 39 2024-04-22T23:34:57.468 INFO:tasks.workunit.client.0.smithi107.stdout:0/16: write d1/f2 [351847,39485] 0 2024-04-22T23:34:57.470 INFO:tasks.workunit.client.0.smithi107.stdout:0/17: mknod d1/c5 0 2024-04-22T23:34:57.470 INFO:tasks.workunit.client.0.smithi107.stdout:0/18: chown d1/c5 22902072 1 2024-04-22T23:34:57.472 INFO:tasks.workunit.client.0.smithi107.stdout:0/19: rename d1/c5 to d1/c6 0 2024-04-22T23:34:57.472 INFO:tasks.workunit.client.0.smithi107.stdout:0/20: write d1/f2 [459106,105072] 0 2024-04-22T23:34:57.474 INFO:tasks.workunit.client.0.smithi107.stdout:0/21: mknod d1/c7 0 2024-04-22T23:34:57.477 INFO:tasks.workunit.client.0.smithi107.stdout:0/22: mknod d1/c8 0 2024-04-22T23:34:57.479 INFO:tasks.workunit.client.1.smithi157.stdout:6/567: write d1/f2c [3865066,88640] 0 2024-04-22T23:34:57.481 INFO:tasks.workunit.client.1.smithi157.stdout:3/312: dwrite d0/d3/d34/d3d/f5e [0,4194304] 0 2024-04-22T23:34:57.481 INFO:tasks.workunit.client.1.smithi157.stdout:3/313: write d0/d3/d34/d39/f53 [360753,117182] 0 2024-04-22T23:34:57.481 INFO:tasks.workunit.client.1.smithi157.stdout:3/314: readlink d0/d3/d34/d39/l52 0 2024-04-22T23:34:57.482 INFO:tasks.workunit.client.0.smithi107.stdout:5/9: rename f1 to f3 0 2024-04-22T23:34:57.482 INFO:tasks.workunit.client.0.smithi107.stdout:5/10: chown f3 431 1 2024-04-22T23:34:57.483 INFO:tasks.workunit.client.0.smithi107.stdout:6/9: getdents . 0 2024-04-22T23:34:57.483 INFO:tasks.workunit.client.1.smithi157.stdout:3/315: mkdir d0/d3/d34/d3d/d57/d60 0 2024-04-22T23:34:57.486 INFO:tasks.workunit.client.1.smithi157.stdout:3/316: rename d0/d3/d13/d21/l2a to d0/d3/l61 0 2024-04-22T23:34:57.487 INFO:tasks.workunit.client.1.smithi157.stdout:3/317: mkdir d0/d3/d34/d62 0 2024-04-22T23:34:57.487 INFO:tasks.workunit.client.1.smithi157.stdout:3/318: creat d0/d3/d34/d3d/f63 x:0 0 0 2024-04-22T23:34:57.487 INFO:tasks.workunit.client.1.smithi157.stdout:3/319: chown d0/d3/f1e 226444 1 2024-04-22T23:34:57.487 INFO:tasks.workunit.client.1.smithi157.stdout:3/320: write d0/d3/d34/d39/f58 [162527,24570] 0 2024-04-22T23:34:57.492 INFO:tasks.workunit.client.0.smithi107.stdout:7/15: link l1 l3 0 2024-04-22T23:34:57.493 INFO:tasks.workunit.client.0.smithi107.stdout:5/11: write f3 [656343,15547] 0 2024-04-22T23:34:57.493 INFO:tasks.workunit.client.0.smithi107.stdout:5/12: creat f4 x:0 0 0 2024-04-22T23:34:57.494 INFO:tasks.workunit.client.0.smithi107.stdout:3/15: dwrite f0 [0,4194304] 0 2024-04-22T23:34:57.494 INFO:tasks.workunit.client.0.smithi107.stdout:3/16: dread - f1 zero size 2024-04-22T23:34:57.495 INFO:tasks.workunit.client.1.smithi157.stdout:3/321: write d0/d3/f1e [7682416,60678] 0 2024-04-22T23:34:57.495 INFO:tasks.workunit.client.1.smithi157.stdout:3/322: stat d0/d3/d34/d39/f18 0 2024-04-22T23:34:57.495 INFO:tasks.workunit.client.0.smithi107.stdout:1/16: truncate f2 702388 0 2024-04-22T23:34:57.496 INFO:tasks.workunit.client.0.smithi107.stdout:1/17: chown f2 1453916125 1 2024-04-22T23:34:57.496 INFO:tasks.workunit.client.1.smithi157.stdout:3/323: creat d0/d3/d34/d3d/d57/f64 x:0 0 0 2024-04-22T23:34:57.496 INFO:tasks.workunit.client.1.smithi157.stdout:3/324: write d0/d3/f51 [59519,6435] 0 2024-04-22T23:34:57.497 INFO:tasks.workunit.client.1.smithi157.stdout:3/325: creat d0/d3/d34/d3d/d57/f65 x:0 0 0 2024-04-22T23:34:57.497 INFO:tasks.workunit.client.0.smithi107.stdout:5/13: mknod c5 0 2024-04-22T23:34:57.500 INFO:tasks.workunit.client.0.smithi107.stdout:3/17: symlink l4 0 2024-04-22T23:34:57.500 INFO:tasks.workunit.client.1.smithi157.stdout:3/326: rename d0/f1 to d0/d3/d34/d39/f66 0 2024-04-22T23:34:57.501 INFO:tasks.workunit.client.0.smithi107.stdout:1/18: symlink l6 0 2024-04-22T23:34:57.501 INFO:tasks.workunit.client.0.smithi107.stdout:5/14: mknod c6 0 2024-04-22T23:34:57.502 INFO:tasks.workunit.client.1.smithi157.stdout:3/327: unlink d0/d3/d13/l2e 0 2024-04-22T23:34:57.502 INFO:tasks.workunit.client.1.smithi157.stdout:3/328: chown d0/d7/c3c 43454649 1 2024-04-22T23:34:57.503 INFO:tasks.workunit.client.1.smithi157.stdout:3/329: mkdir d0/d3/d34/d67 0 2024-04-22T23:34:57.503 INFO:tasks.workunit.client.1.smithi157.stdout:3/330: chown d0/f11 1806 1 2024-04-22T23:34:57.504 INFO:tasks.workunit.client.1.smithi157.stdout:3/331: symlink d0/l68 0 2024-04-22T23:34:57.510 INFO:tasks.workunit.client.0.smithi107.stdout:7/16: dwrite f2 [0,4194304] 0 2024-04-22T23:34:57.511 INFO:tasks.workunit.client.0.smithi107.stdout:6/10: dwrite f0 [0,4194304] 0 2024-04-22T23:34:57.511 INFO:tasks.workunit.client.0.smithi107.stdout:3/18: dread f0 [0,4194304] 0 2024-04-22T23:34:57.511 INFO:tasks.workunit.client.0.smithi107.stdout:3/19: stat l4 0 2024-04-22T23:34:57.512 INFO:tasks.workunit.client.0.smithi107.stdout:7/17: rename l3 to l4 0 2024-04-22T23:34:57.514 INFO:tasks.workunit.client.0.smithi107.stdout:3/20: mknod c5 0 2024-04-22T23:34:57.514 INFO:tasks.workunit.client.0.smithi107.stdout:3/21: dread - f1 zero size 2024-04-22T23:34:57.521 INFO:tasks.workunit.client.0.smithi107.stdout:6/11: dread f0 [0,4194304] 0 2024-04-22T23:34:57.533 INFO:tasks.workunit.client.1.smithi157.stdout:3/332: dwrite d0/d3/d34/d3d/f44 [8388608,4194304] 0 2024-04-22T23:34:57.533 INFO:tasks.workunit.client.0.smithi107.stdout:5/15: dwrite f3 [0,4194304] 0 2024-04-22T23:34:57.543 INFO:tasks.workunit.client.0.smithi107.stdout:3/22: dwrite f1 [0,4194304] 0 2024-04-22T23:34:57.548 INFO:tasks.workunit.client.0.smithi107.stdout:7/18: unlink l4 0 2024-04-22T23:34:57.548 INFO:tasks.workunit.client.0.smithi107.stdout:1/19: dread f2 [0,4194304] 0 2024-04-22T23:34:57.556 INFO:tasks.workunit.client.0.smithi107.stdout:5/16: dwrite f3 [0,4194304] 0 2024-04-22T23:34:57.556 INFO:tasks.workunit.client.0.smithi107.stdout:5/17: creat f7 x:0 0 0 2024-04-22T23:34:57.556 INFO:tasks.workunit.client.0.smithi107.stdout:5/18: creat f8 x:0 0 0 2024-04-22T23:34:57.556 INFO:tasks.workunit.client.0.smithi107.stdout:5/19: dread - f8 zero size 2024-04-22T23:34:57.559 INFO:tasks.workunit.client.1.smithi157.stdout:3/333: dwrite d0/d3/d34/d39/f32 [0,4194304] 0 2024-04-22T23:34:57.560 INFO:tasks.workunit.client.0.smithi107.stdout:7/19: write f2 [1871009,22067] 0 2024-04-22T23:34:57.562 INFO:tasks.workunit.client.0.smithi107.stdout:5/20: mkdir d9 0 2024-04-22T23:34:57.562 INFO:tasks.workunit.client.0.smithi107.stdout:5/21: write f4 [995444,111079] 0 2024-04-22T23:34:57.562 INFO:tasks.workunit.client.0.smithi107.stdout:5/22: write f7 [234320,105370] 0 2024-04-22T23:34:57.563 INFO:tasks.workunit.client.1.smithi157.stdout:3/334: rename d0/f14 to d0/d3/d34/d3d/f69 0 2024-04-22T23:34:57.563 INFO:tasks.workunit.client.1.smithi157.stdout:3/335: chown d0/d3/d34/d39/l2d 1 1 2024-04-22T23:34:57.563 INFO:tasks.workunit.client.1.smithi157.stdout:3/336: truncate d0/d3/f51 443605 0 2024-04-22T23:34:57.564 INFO:tasks.workunit.client.0.smithi107.stdout:3/23: write f1 [2464504,86595] 0 2024-04-22T23:34:57.564 INFO:tasks.workunit.client.0.smithi107.stdout:6/12: dwrite f0 [0,4194304] 0 2024-04-22T23:34:57.564 INFO:tasks.workunit.client.0.smithi107.stdout:6/13: readlink - no filename 2024-04-22T23:34:57.564 INFO:tasks.workunit.client.0.smithi107.stdout:3/24: fdatasync f0 0 2024-04-22T23:34:57.566 INFO:tasks.workunit.client.1.smithi157.stdout:3/337: link d0/d3/d34/c43 d0/d3/d34/d62/c6a 0 2024-04-22T23:34:57.569 INFO:tasks.workunit.client.0.smithi107.stdout:7/20: dread f2 [0,4194304] 0 2024-04-22T23:34:57.571 INFO:tasks.workunit.client.0.smithi107.stdout:7/21: mkdir d5 0 2024-04-22T23:34:57.573 INFO:tasks.workunit.client.0.smithi107.stdout:1/20: dwrite f2 [0,4194304] 0 2024-04-22T23:34:57.573 INFO:tasks.workunit.client.0.smithi107.stdout:5/23: write f3 [2714903,97837] 0 2024-04-22T23:34:57.576 INFO:tasks.workunit.client.1.smithi157.stdout:3/338: dread d0/d3/d13/d21/f3f [0,4194304] 0 2024-04-22T23:34:57.584 INFO:tasks.workunit.client.0.smithi107.stdout:3/25: dread f1 [0,4194304] 0 2024-04-22T23:34:57.584 INFO:tasks.workunit.client.0.smithi107.stdout:3/26: write f0 [4576547,979] 0 2024-04-22T23:34:57.591 INFO:tasks.workunit.client.0.smithi107.stdout:6/14: dwrite f0 [0,4194304] 0 2024-04-22T23:34:57.592 INFO:tasks.workunit.client.0.smithi107.stdout:3/27: write f0 [162018,94560] 0 2024-04-22T23:34:57.592 INFO:tasks.workunit.client.0.smithi107.stdout:3/28: read f0 [3534886,105683] 0 2024-04-22T23:34:57.592 INFO:tasks.workunit.client.0.smithi107.stdout:1/21: read f2 [381871,100713] 0 2024-04-22T23:34:57.593 INFO:tasks.workunit.client.1.smithi157.stdout:3/339: dwrite d0/d3/d13/d21/f3f [0,4194304] 0 2024-04-22T23:34:57.593 INFO:tasks.workunit.client.0.smithi107.stdout:6/15: mknod c1 0 2024-04-22T23:34:57.593 INFO:tasks.workunit.client.0.smithi107.stdout:6/16: fsync f0 0 2024-04-22T23:34:57.594 INFO:tasks.workunit.client.0.smithi107.stdout:3/29: unlink c5 0 2024-04-22T23:34:57.594 INFO:tasks.workunit.client.1.smithi157.stdout:3/340: truncate d0/d3/d13/d21/f4e 49442 0 2024-04-22T23:34:57.594 INFO:tasks.workunit.client.1.smithi157.stdout:3/341: dread - d0/d3/d34/d3d/f63 zero size 2024-04-22T23:34:57.595 INFO:tasks.workunit.client.0.smithi107.stdout:1/22: mkdir d7 0 2024-04-22T23:34:57.595 INFO:tasks.workunit.client.1.smithi157.stdout:3/342: mkdir d0/d3/d34/d3d/d57/d60/d6b 0 2024-04-22T23:34:57.595 INFO:tasks.workunit.client.1.smithi157.stdout:3/343: write d0/d3/d34/d39/f49 [4288798,77431] 0 2024-04-22T23:34:57.596 INFO:tasks.workunit.client.0.smithi107.stdout:5/24: dwrite f3 [0,4194304] 0 2024-04-22T23:34:57.596 INFO:tasks.workunit.client.0.smithi107.stdout:5/25: rename d9 to d9/da 22 2024-04-22T23:34:57.596 INFO:tasks.workunit.client.0.smithi107.stdout:1/23: symlink d7/l8 0 2024-04-22T23:34:57.597 INFO:tasks.workunit.client.0.smithi107.stdout:5/26: mkdir d9/db 0 2024-04-22T23:34:57.597 INFO:tasks.workunit.client.0.smithi107.stdout:5/27: write f8 [849509,39796] 0 2024-04-22T23:34:57.597 INFO:tasks.workunit.client.1.smithi157.stdout:3/344: mkdir d0/d7/d6c 0 2024-04-22T23:34:57.597 INFO:tasks.workunit.client.1.smithi157.stdout:3/345: fsync d0/d3/d34/f5d 0 2024-04-22T23:34:57.598 INFO:tasks.workunit.client.0.smithi107.stdout:5/28: mknod d9/db/cc 0 2024-04-22T23:34:57.598 INFO:tasks.workunit.client.0.smithi107.stdout:5/29: write f4 [2040168,43176] 0 2024-04-22T23:34:57.599 INFO:tasks.workunit.client.1.smithi157.stdout:3/346: creat d0/d3/d34/d47/f6d x:0 0 0 2024-04-22T23:34:57.601 INFO:tasks.workunit.client.1.smithi157.stdout:3/347: link d0/d3/lf d0/d3/d34/d67/l6e 0 2024-04-22T23:34:57.602 INFO:tasks.workunit.client.0.smithi107.stdout:3/30: getdents . 0 2024-04-22T23:34:57.602 INFO:tasks.workunit.client.0.smithi107.stdout:3/31: read f1 [1589717,91251] 0 2024-04-22T23:34:57.602 INFO:tasks.workunit.client.0.smithi107.stdout:3/32: write f1 [5117234,111665] 0 2024-04-22T23:34:57.602 INFO:tasks.workunit.client.0.smithi107.stdout:3/33: write f1 [1255063,127549] 0 2024-04-22T23:34:57.607 INFO:tasks.workunit.client.0.smithi107.stdout:3/34: rename f1 to f6 0 2024-04-22T23:34:57.610 INFO:tasks.workunit.client.1.smithi157.stdout:1/430: sync 2024-04-22T23:34:57.610 INFO:tasks.workunit.client.1.smithi157.stdout:1/431: write d4/db/f47 [527906,127096] 0 2024-04-22T23:34:57.610 INFO:tasks.workunit.client.1.smithi157.stdout:8/249: sync 2024-04-22T23:34:57.610 INFO:tasks.workunit.client.1.smithi157.stdout:8/250: readlink d2/d6/de/d17/l36 0 2024-04-22T23:34:57.610 INFO:tasks.workunit.client.0.smithi107.stdout:3/35: mknod c7 0 2024-04-22T23:34:57.610 INFO:tasks.workunit.client.1.smithi157.stdout:0/331: sync 2024-04-22T23:34:57.611 INFO:tasks.workunit.client.1.smithi157.stdout:9/361: sync 2024-04-22T23:34:57.614 INFO:tasks.workunit.client.1.smithi157.stdout:1/432: rename d4/db/c15 to d4/d17/d1d/d33/d41/d54/c8f 0 2024-04-22T23:34:57.616 INFO:tasks.workunit.client.1.smithi157.stdout:1/433: truncate d4/d17/f2d 2251106 0 2024-04-22T23:34:57.619 INFO:tasks.workunit.client.1.smithi157.stdout:1/434: write d4/d17/d34/f49 [284393,17513] 0 2024-04-22T23:34:57.621 INFO:tasks.workunit.client.0.smithi107.stdout:5/30: dread f4 [0,4194304] 0 2024-04-22T23:34:57.621 INFO:tasks.workunit.client.1.smithi157.stdout:1/435: mknod d4/d5f/d7e/c90 0 2024-04-22T23:34:57.623 INFO:tasks.workunit.client.1.smithi157.stdout:0/332: dread d17/d23/f3e [0,4194304] 0 2024-04-22T23:34:57.623 INFO:tasks.workunit.client.1.smithi157.stdout:0/333: chown d17/d38/d6a/c6f 5 1 2024-04-22T23:34:57.623 INFO:tasks.workunit.client.1.smithi157.stdout:0/334: write d17/d2b/f5c [752316,97470] 0 2024-04-22T23:34:57.623 INFO:tasks.workunit.client.1.smithi157.stdout:0/335: creat d17/d58/f70 x:0 0 0 2024-04-22T23:34:57.623 INFO:tasks.workunit.client.1.smithi157.stdout:0/336: chown d17/d58/d69 825 1 2024-04-22T23:34:57.623 INFO:tasks.workunit.client.1.smithi157.stdout:0/337: chown d17/d23/d34/d67 0 1 2024-04-22T23:34:57.624 INFO:tasks.workunit.client.0.smithi107.stdout:3/36: rename f6 to f8 0 2024-04-22T23:34:57.624 INFO:tasks.workunit.client.0.smithi107.stdout:3/37: chown f8 12 1 2024-04-22T23:34:57.625 INFO:tasks.workunit.client.1.smithi157.stdout:0/338: mkdir d17/d2b/d71 0 2024-04-22T23:34:57.628 INFO:tasks.workunit.client.1.smithi157.stdout:0/339: mknod d17/d23/d34/c72 0 2024-04-22T23:34:57.630 INFO:tasks.workunit.client.0.smithi107.stdout:6/17: dwrite f0 [0,4194304] 0 2024-04-22T23:34:57.631 INFO:tasks.workunit.client.1.smithi157.stdout:0/340: link d17/f19 d17/d38/f73 0 2024-04-22T23:34:57.631 INFO:tasks.workunit.client.1.smithi157.stdout:0/341: stat d17/l26 0 2024-04-22T23:34:57.631 INFO:tasks.workunit.client.1.smithi157.stdout:0/342: fsync d17/d23/f3e 0 2024-04-22T23:34:57.631 INFO:tasks.workunit.client.1.smithi157.stdout:0/343: readlink d17/l1d 0 2024-04-22T23:34:57.632 INFO:tasks.workunit.client.1.smithi157.stdout:0/344: creat d17/d58/f74 x:0 0 0 2024-04-22T23:34:57.643 INFO:tasks.workunit.client.0.smithi107.stdout:6/18: write f0 [3142241,12763] 0 2024-04-22T23:34:57.643 INFO:tasks.workunit.client.0.smithi107.stdout:6/19: write f0 [1317290,14433] 0 2024-04-22T23:34:57.646 INFO:tasks.workunit.client.1.smithi157.stdout:3/348: dwrite d0/f1c [0,4194304] 0 2024-04-22T23:34:57.651 INFO:tasks.workunit.client.0.smithi107.stdout:1/24: dwrite f2 [0,4194304] 0 2024-04-22T23:34:57.657 INFO:tasks.workunit.client.0.smithi107.stdout:5/31: dwrite f7 [0,4194304] 0 2024-04-22T23:34:57.661 INFO:tasks.workunit.client.0.smithi107.stdout:5/32: creat d9/fd x:0 0 0 2024-04-22T23:34:57.662 INFO:tasks.workunit.client.1.smithi157.stdout:0/345: dwrite d17/d23/f4d [0,4194304] 0 2024-04-22T23:34:57.662 INFO:tasks.workunit.client.1.smithi157.stdout:0/346: chown d17/d38/l6b 17199875 1 2024-04-22T23:34:57.663 INFO:tasks.workunit.client.0.smithi107.stdout:6/20: fdatasync f0 0 2024-04-22T23:34:57.664 INFO:tasks.workunit.client.0.smithi107.stdout:5/33: read f4 [425897,58994] 0 2024-04-22T23:34:57.665 INFO:tasks.workunit.client.1.smithi157.stdout:0/347: symlink d17/d2b/d4f/l75 0 2024-04-22T23:34:57.665 INFO:tasks.workunit.client.1.smithi157.stdout:0/348: truncate d17/d2b/d52/f3f 1993644 0 2024-04-22T23:34:57.666 INFO:tasks.workunit.client.1.smithi157.stdout:0/349: symlink d17/d2b/d71/l76 0 2024-04-22T23:34:57.667 INFO:tasks.workunit.client.1.smithi157.stdout:9/362: dwrite d4/d15/d55/f58 [0,4194304] 0 2024-04-22T23:34:57.669 INFO:tasks.workunit.client.1.smithi157.stdout:8/251: dwrite d2/d6/de/d11/f4a [0,4194304] 0 2024-04-22T23:34:57.669 INFO:tasks.workunit.client.1.smithi157.stdout:8/252: creat d2/f5d x:0 0 0 2024-04-22T23:34:57.669 INFO:tasks.workunit.client.1.smithi157.stdout:8/253: read - d2/d6/de/d11/f27 zero size 2024-04-22T23:34:57.669 INFO:tasks.workunit.client.1.smithi157.stdout:8/254: write d2/d4/f52 [1228287,29650] 0 2024-04-22T23:34:57.669 INFO:tasks.workunit.client.1.smithi157.stdout:8/255: write d2/d4/f52 [1366516,126886] 0 2024-04-22T23:34:57.672 INFO:tasks.workunit.client.1.smithi157.stdout:0/350: write d17/f1a [2737167,1987] 0 2024-04-22T23:34:57.672 INFO:tasks.workunit.client.1.smithi157.stdout:3/349: dwrite d0/d3/d34/d39/f18 [0,4194304] 0 2024-04-22T23:34:57.673 INFO:tasks.workunit.client.0.smithi107.stdout:5/34: rmdir d9 39 2024-04-22T23:34:57.674 INFO:tasks.workunit.client.0.smithi107.stdout:5/35: truncate d9/fd 408492 0 2024-04-22T23:34:57.674 INFO:tasks.workunit.client.0.smithi107.stdout:3/38: dread f8 [0,4194304] 0 2024-04-22T23:34:57.676 INFO:tasks.workunit.client.1.smithi157.stdout:8/256: dread d2/d6/d21/f2e [0,4194304] 0 2024-04-22T23:34:57.676 INFO:tasks.workunit.client.1.smithi157.stdout:8/257: dread - d2/d4/f5b zero size 2024-04-22T23:34:57.676 INFO:tasks.workunit.client.1.smithi157.stdout:8/258: fdatasync d2/d6/de/d11/f44 0 2024-04-22T23:34:57.683 INFO:tasks.workunit.client.1.smithi157.stdout:1/436: dwrite d4/db/f3c [0,4194304] 0 2024-04-22T23:34:57.683 INFO:tasks.workunit.client.1.smithi157.stdout:3/350: creat d0/d7/d6c/f6f x:0 0 0 2024-04-22T23:34:57.683 INFO:tasks.workunit.client.1.smithi157.stdout:3/351: fsync d0/d3/d34/d39/f53 0 2024-04-22T23:34:57.685 INFO:tasks.workunit.client.1.smithi157.stdout:8/259: mknod d2/d4/c5e 0 2024-04-22T23:34:57.685 INFO:tasks.workunit.client.1.smithi157.stdout:8/260: dread - d2/f48 zero size 2024-04-22T23:34:57.685 INFO:tasks.workunit.client.1.smithi157.stdout:8/261: write d2/d6/de/f2f [1564724,126694] 0 2024-04-22T23:34:57.685 INFO:tasks.workunit.client.1.smithi157.stdout:8/262: stat d2/d4/f5b 0 2024-04-22T23:34:57.685 INFO:tasks.workunit.client.1.smithi157.stdout:0/351: rmdir d17/d58 39 2024-04-22T23:34:57.687 INFO:tasks.workunit.client.1.smithi157.stdout:1/437: creat d4/db/d3d/f91 x:0 0 0 2024-04-22T23:34:57.688 INFO:tasks.workunit.client.1.smithi157.stdout:3/352: getdents d0/d3/d34/d3d/d57/d5c 0 2024-04-22T23:34:57.688 INFO:tasks.workunit.client.1.smithi157.stdout:3/353: truncate d0/d3/d34/f48 540609 0 2024-04-22T23:34:57.688 INFO:tasks.workunit.client.1.smithi157.stdout:3/354: creat d0/d3/d34/d47/f70 x:0 0 0 2024-04-22T23:34:57.688 INFO:tasks.workunit.client.0.smithi107.stdout:5/36: dread f7 [0,4194304] 0 2024-04-22T23:34:57.689 INFO:tasks.workunit.client.0.smithi107.stdout:5/37: write d9/fd [93469,40969] 0 2024-04-22T23:34:57.689 INFO:tasks.workunit.client.0.smithi107.stdout:5/38: chown c2 490312 1 2024-04-22T23:34:57.689 INFO:tasks.workunit.client.0.smithi107.stdout:5/39: truncate f4 2377738 0 2024-04-22T23:34:57.690 INFO:tasks.workunit.client.0.smithi107.stdout:5/40: mkdir d9/db/de 0 2024-04-22T23:34:57.691 INFO:tasks.workunit.client.1.smithi157.stdout:0/352: creat d17/d58/f77 x:0 0 0 2024-04-22T23:34:57.692 INFO:tasks.workunit.client.1.smithi157.stdout:1/438: mknod d4/c92 0 2024-04-22T23:34:57.692 INFO:tasks.workunit.client.1.smithi157.stdout:1/439: fsync d4/d17/d1d/d33/d41/d54/f5b 0 2024-04-22T23:34:57.694 INFO:tasks.workunit.client.1.smithi157.stdout:0/353: dread d17/f2d [0,4194304] 0 2024-04-22T23:34:57.694 INFO:tasks.workunit.client.1.smithi157.stdout:0/354: chown d17/d2b/d52/l57 14865 1 2024-04-22T23:34:57.695 INFO:tasks.workunit.client.1.smithi157.stdout:3/355: rmdir d0/d3/d34/d3d/d57/d5c 39 2024-04-22T23:34:57.696 INFO:tasks.workunit.client.1.smithi157.stdout:9/363: dwrite d4/f29 [0,4194304] 0 2024-04-22T23:34:57.696 INFO:tasks.workunit.client.0.smithi107.stdout:3/39: dwrite f8 [4194304,4194304] 0 2024-04-22T23:34:57.699 INFO:tasks.workunit.client.1.smithi157.stdout:1/440: creat d4/d77/f93 x:0 0 0 2024-04-22T23:34:57.699 INFO:tasks.workunit.client.1.smithi157.stdout:1/441: fdatasync d4/d17/d1d/d33/d41/f5e 0 2024-04-22T23:34:57.699 INFO:tasks.workunit.client.1.smithi157.stdout:1/442: fsync d4/f26 0 2024-04-22T23:34:57.699 INFO:tasks.workunit.client.1.smithi157.stdout:1/443: write d4/d17/d34/d3b/d50/f7b [353170,50465] 0 2024-04-22T23:34:57.699 INFO:tasks.workunit.client.1.smithi157.stdout:0/355: mkdir d17/d23/d34/d67/d78 0 2024-04-22T23:34:57.700 INFO:tasks.workunit.client.1.smithi157.stdout:3/356: unlink d0/d7/l36 0 2024-04-22T23:34:57.702 INFO:tasks.workunit.client.1.smithi157.stdout:7/323: sync 2024-04-22T23:34:57.702 INFO:tasks.workunit.client.1.smithi157.stdout:4/540: sync 2024-04-22T23:34:57.702 INFO:tasks.workunit.client.1.smithi157.stdout:4/541: chown d3/d11/d1d/d73/l9e 38959 1 2024-04-22T23:34:57.702 INFO:tasks.workunit.client.1.smithi157.stdout:9/364: link d4/d15/d23/d27/d47/f4a d4/df/d61/f68 0 2024-04-22T23:34:57.703 INFO:tasks.workunit.client.1.smithi157.stdout:8/263: rename d2/d6/de to d2/d6/d21/d5f 0 2024-04-22T23:34:57.705 INFO:tasks.workunit.client.1.smithi157.stdout:0/356: link l12 d17/d2b/l79 0 2024-04-22T23:34:57.706 INFO:tasks.workunit.client.1.smithi157.stdout:2/481: sync 2024-04-22T23:34:57.706 INFO:tasks.workunit.client.1.smithi157.stdout:5/429: sync 2024-04-22T23:34:57.707 INFO:tasks.workunit.client.1.smithi157.stdout:7/324: unlink d0/d9/d72/c4b 0 2024-04-22T23:34:57.707 INFO:tasks.workunit.client.1.smithi157.stdout:7/325: dread - d0/d9/d29/d33/d37/f4c zero size 2024-04-22T23:34:57.707 INFO:tasks.workunit.client.1.smithi157.stdout:7/326: stat d0/d9/db/c1b 0 2024-04-22T23:34:57.708 INFO:tasks.workunit.client.1.smithi157.stdout:7/327: chown d0/d9/l24 16092 1 2024-04-22T23:34:57.708 INFO:tasks.workunit.client.1.smithi157.stdout:6/568: sync 2024-04-22T23:34:57.708 INFO:tasks.workunit.client.1.smithi157.stdout:6/569: chown d1/d36/fa0 34 1 2024-04-22T23:34:57.708 INFO:tasks.workunit.client.1.smithi157.stdout:0/357: dread d17/d58/f5f [0,4194304] 0 2024-04-22T23:34:57.709 INFO:tasks.workunit.client.1.smithi157.stdout:0/358: fdatasync d17/d2b/d52/f53 0 2024-04-22T23:34:57.709 INFO:tasks.workunit.client.1.smithi157.stdout:0/359: stat d17/d38/d6a 0 2024-04-22T23:34:57.709 INFO:tasks.workunit.client.1.smithi157.stdout:4/542: symlink d3/d11/d1d/lb4 0 2024-04-22T23:34:57.709 INFO:tasks.workunit.client.1.smithi157.stdout:0/360: write d17/d2b/f2c [82291,48935] 0 2024-04-22T23:34:57.709 INFO:tasks.workunit.client.0.smithi107.stdout:5/41: dwrite f8 [0,4194304] 0 2024-04-22T23:34:57.709 INFO:tasks.workunit.client.0.smithi107.stdout:5/42: chown f8 11302 1 2024-04-22T23:34:57.710 INFO:tasks.workunit.client.1.smithi157.stdout:6/570: dread d1/d2f/d3b/d39/d4e/f5f [0,4194304] 0 2024-04-22T23:34:57.710 INFO:tasks.workunit.client.1.smithi157.stdout:6/571: dread - d1/d36/f4d zero size 2024-04-22T23:34:57.710 INFO:tasks.workunit.client.1.smithi157.stdout:9/365: creat d4/d15/d23/f69 x:0 0 0 2024-04-22T23:34:57.712 INFO:tasks.workunit.client.1.smithi157.stdout:8/264: creat d2/d34/f60 x:0 0 0 2024-04-22T23:34:57.715 INFO:tasks.workunit.client.0.smithi107.stdout:5/43: dread d9/fd [0,4194304] 0 2024-04-22T23:34:57.715 INFO:tasks.workunit.client.1.smithi157.stdout:2/482: link d7/d8/l62 d7/d8/d95/laa 0 2024-04-22T23:34:57.716 INFO:tasks.workunit.client.1.smithi157.stdout:5/430: mkdir d1/d5/d7/d2f/d45/da4 0 2024-04-22T23:34:57.718 INFO:tasks.workunit.client.1.smithi157.stdout:7/328: unlink d0/d9/db/d35/l3d 0 2024-04-22T23:34:57.718 INFO:tasks.workunit.client.1.smithi157.stdout:3/357: dread d0/d3/f1e [8388608,4194304] 0 2024-04-22T23:34:57.720 INFO:tasks.workunit.client.0.smithi107.stdout:3/40: dwrite f8 [0,4194304] 0 2024-04-22T23:34:57.722 INFO:tasks.workunit.client.1.smithi157.stdout:0/361: link d17/d2b/f5c d17/d23/f7a 0 2024-04-22T23:34:57.723 INFO:tasks.workunit.client.0.smithi107.stdout:3/41: mknod c9 0 2024-04-22T23:34:57.726 INFO:tasks.workunit.client.0.smithi107.stdout:3/42: rename c2 to ca 0 2024-04-22T23:34:57.728 INFO:tasks.workunit.client.1.smithi157.stdout:3/358: write d0/d3/d34/d3d/f42 [2624263,94207] 0 2024-04-22T23:34:57.728 INFO:tasks.workunit.client.0.smithi107.stdout:3/43: fsync f0 0 2024-04-22T23:34:57.728 INFO:tasks.workunit.client.1.smithi157.stdout:3/359: write d0/d3/d34/d39/f53 [1027577,131024] 0 2024-04-22T23:34:57.729 INFO:tasks.workunit.client.1.smithi157.stdout:7/329: dread d0/f46 [0,4194304] 0 2024-04-22T23:34:57.729 INFO:tasks.workunit.client.1.smithi157.stdout:1/444: getdents d4/d77 0 2024-04-22T23:34:57.729 INFO:tasks.workunit.client.1.smithi157.stdout:2/483: creat d7/d1f/d26/d8f/fab x:0 0 0 2024-04-22T23:34:57.729 INFO:tasks.workunit.client.1.smithi157.stdout:1/445: read d4/d17/d34/f49 [460129,72290] 0 2024-04-22T23:34:57.729 INFO:tasks.workunit.client.1.smithi157.stdout:5/431: symlink d1/d5/d7/la5 0 2024-04-22T23:34:57.730 INFO:tasks.workunit.client.1.smithi157.stdout:2/484: write d7/d8/d30/f79 [491793,111091] 0 2024-04-22T23:34:57.730 INFO:tasks.workunit.client.1.smithi157.stdout:2/485: fsync d7/d8/d30/d23/d27/d57/fa7 0 2024-04-22T23:34:57.733 INFO:tasks.workunit.client.1.smithi157.stdout:8/265: rmdir d2/d6 39 2024-04-22T23:34:57.733 INFO:tasks.workunit.client.1.smithi157.stdout:8/266: chown d2/d6/d21/d5f/d11/f27 503 1 2024-04-22T23:34:57.733 INFO:tasks.workunit.client.1.smithi157.stdout:6/572: dread d1/d2f/d3b/f3e [0,4194304] 0 2024-04-22T23:34:57.736 INFO:tasks.workunit.client.1.smithi157.stdout:1/446: mkdir d4/d17/d34/d84/d94 0 2024-04-22T23:34:57.736 INFO:tasks.workunit.client.1.smithi157.stdout:4/543: rename d3/d4b/d5d/d6c to d3/d11/d1d/d73/d80/dac/db5 0 2024-04-22T23:34:57.736 INFO:tasks.workunit.client.1.smithi157.stdout:1/447: fsync d4/db/f47 0 2024-04-22T23:34:57.738 INFO:tasks.workunit.client.1.smithi157.stdout:5/432: symlink d1/d5/d7/d2f/d45/d54/la6 0 2024-04-22T23:34:57.740 INFO:tasks.workunit.client.1.smithi157.stdout:8/267: creat d2/d6/d21/d33/d42/f61 x:0 0 0 2024-04-22T23:34:57.741 INFO:tasks.workunit.client.1.smithi157.stdout:8/268: write d2/d6/d21/d5f/d11/f5a [39117,117683] 0 2024-04-22T23:34:57.743 INFO:tasks.workunit.client.1.smithi157.stdout:8/269: unlink d2/d4/c3d 0 2024-04-22T23:34:57.743 INFO:tasks.workunit.client.1.smithi157.stdout:2/486: dread d7/d8/d30/d23/f7f [0,4194304] 0 2024-04-22T23:34:57.743 INFO:tasks.workunit.client.1.smithi157.stdout:2/487: dread - d7/d8/d95/f63 zero size 2024-04-22T23:34:57.745 INFO:tasks.workunit.client.1.smithi157.stdout:6/573: write d1/d2f/f40 [2802113,61318] 0 2024-04-22T23:34:57.747 INFO:tasks.workunit.client.1.smithi157.stdout:5/433: write d1/d5/d7/f34 [3786234,120414] 0 2024-04-22T23:34:57.747 INFO:tasks.workunit.client.1.smithi157.stdout:5/434: truncate d1/d5/dc/d43/d4f/d60/d7a/f87 148586 0 2024-04-22T23:34:57.747 INFO:tasks.workunit.client.1.smithi157.stdout:2/488: symlink d7/d8/d30/d5b/d8a/lac 0 2024-04-22T23:34:57.747 INFO:tasks.workunit.client.0.smithi107.stdout:3/44: dread f8 [4194304,4194304] 0 2024-04-22T23:34:57.748 INFO:tasks.workunit.client.1.smithi157.stdout:6/574: mknod d1/d2f/d54/d86/cb1 0 2024-04-22T23:34:57.749 INFO:tasks.workunit.client.1.smithi157.stdout:2/489: mknod d7/d8/d30/d23/d27/cad 0 2024-04-22T23:34:57.750 INFO:tasks.workunit.client.1.smithi157.stdout:2/490: chown d7/d8/d25/f68 447 1 2024-04-22T23:34:57.751 INFO:tasks.workunit.client.1.smithi157.stdout:6/575: link d1/d2f/d3b/d39/d4e/d74/l7f d1/d38/daf/lb2 0 2024-04-22T23:34:57.751 INFO:tasks.workunit.client.1.smithi157.stdout:6/576: write d1/d2f/d54/fa5 [536478,24244] 0 2024-04-22T23:34:57.752 INFO:tasks.workunit.client.1.smithi157.stdout:1/448: read d4/db/f3c [996991,71516] 0 2024-04-22T23:34:57.753 INFO:tasks.workunit.client.1.smithi157.stdout:5/435: rename d1/d5/dc/l1a to d1/la7 0 2024-04-22T23:34:57.753 INFO:tasks.workunit.client.1.smithi157.stdout:5/436: write d1/d5/dc/d43/d4f/d60/d7a/f87 [1048067,97779] 0 2024-04-22T23:34:57.753 INFO:tasks.workunit.client.1.smithi157.stdout:2/491: rmdir d7/d8/d30/d5b/d8a 39 2024-04-22T23:34:57.753 INFO:tasks.workunit.client.1.smithi157.stdout:8/270: write d2/d4/f1c [6856722,129119] 0 2024-04-22T23:34:57.755 INFO:tasks.workunit.client.1.smithi157.stdout:6/577: creat d1/d2f/d3b/d49/fb3 x:0 0 0 2024-04-22T23:34:57.755 INFO:tasks.workunit.client.1.smithi157.stdout:6/578: write d1/d2f/d54/f95 [561822,45479] 0 2024-04-22T23:34:57.755 INFO:tasks.workunit.client.0.smithi107.stdout:3/45: read f8 [4605739,53465] 0 2024-04-22T23:34:57.756 INFO:tasks.workunit.client.1.smithi157.stdout:5/437: mknod d1/d5/dc/d43/d4f/d60/ca8 0 2024-04-22T23:34:57.757 INFO:tasks.workunit.client.1.smithi157.stdout:8/271: creat d2/d4/d30/f62 x:0 0 0 2024-04-22T23:34:57.759 INFO:tasks.workunit.client.1.smithi157.stdout:6/579: unlink d1/d36/f4d 0 2024-04-22T23:34:57.759 INFO:tasks.workunit.client.0.smithi107.stdout:5/44: dwrite f4 [0,4194304] 0 2024-04-22T23:34:57.759 INFO:tasks.workunit.client.0.smithi107.stdout:5/45: chown f7 2 1 2024-04-22T23:34:57.760 INFO:tasks.workunit.client.1.smithi157.stdout:5/438: mknod d1/d5/d7/d10/ca9 0 2024-04-22T23:34:57.761 INFO:tasks.workunit.client.1.smithi157.stdout:0/362: dwrite d17/f22 [0,4194304] 0 2024-04-22T23:34:57.762 INFO:tasks.workunit.client.0.smithi107.stdout:5/46: mknod d9/db/cf 0 2024-04-22T23:34:57.763 INFO:tasks.workunit.client.1.smithi157.stdout:4/544: read d3/d4b/f5 [2215936,44317] 0 2024-04-22T23:34:57.763 INFO:tasks.workunit.client.1.smithi157.stdout:3/360: dwrite d0/d3/d34/d3d/f5e [0,4194304] 0 2024-04-22T23:34:57.764 INFO:tasks.workunit.client.1.smithi157.stdout:4/545: fdatasync d3/d11/d1d/f2d 0 2024-04-22T23:34:57.764 INFO:tasks.workunit.client.1.smithi157.stdout:3/361: truncate d0/f41 1744315 0 2024-04-22T23:34:57.765 INFO:tasks.workunit.client.0.smithi107.stdout:5/47: dread d9/fd [0,4194304] 0 2024-04-22T23:34:57.767 INFO:tasks.workunit.client.1.smithi157.stdout:7/330: dwrite d0/d9/d10/f30 [0,4194304] 0 2024-04-22T23:34:57.769 INFO:tasks.workunit.client.1.smithi157.stdout:5/439: rename d1/d1f/d22/f3c to d1/d5/d7/d2f/d9c/faa 0 2024-04-22T23:34:57.769 INFO:tasks.workunit.client.1.smithi157.stdout:2/492: rmdir d7/d8/d25/d41 0 2024-04-22T23:34:57.769 INFO:tasks.workunit.client.1.smithi157.stdout:8/272: symlink d2/d6/d21/d5f/l63 0 2024-04-22T23:34:57.769 INFO:tasks.workunit.client.1.smithi157.stdout:2/493: write d7/d1f/d26/d4a/f53 [278765,29653] 0 2024-04-22T23:34:57.769 INFO:tasks.workunit.client.1.smithi157.stdout:2/494: readlink d7/d8/d30/d5b/d8a/lac 0 2024-04-22T23:34:57.771 INFO:tasks.workunit.client.1.smithi157.stdout:4/546: mknod d3/d4b/cb6 0 2024-04-22T23:34:57.773 INFO:tasks.workunit.client.1.smithi157.stdout:5/440: mkdir d1/d5/dc/d43/d4f/d7f/dab 0 2024-04-22T23:34:57.775 INFO:tasks.workunit.client.1.smithi157.stdout:1/449: dread d4/db/f13 [0,4194304] 0 2024-04-22T23:34:57.775 INFO:tasks.workunit.client.1.smithi157.stdout:2/495: link d7/c6a d7/d1f/d26/d4a/cae 0 2024-04-22T23:34:57.776 INFO:tasks.workunit.client.0.smithi107.stdout:3/46: truncate f8 5136973 0 2024-04-22T23:34:57.777 INFO:tasks.workunit.client.1.smithi157.stdout:2/496: symlink d7/d9f/laf 0 2024-04-22T23:34:57.777 INFO:tasks.workunit.client.1.smithi157.stdout:2/497: readlink d7/d8/d95/l61 0 2024-04-22T23:34:57.778 INFO:tasks.workunit.client.1.smithi157.stdout:2/498: chown d7/d1f/d26/f44 1972819518 1 2024-04-22T23:34:57.778 INFO:tasks.workunit.client.1.smithi157.stdout:2/499: dread - d7/d8/d95/fa8 zero size 2024-04-22T23:34:57.778 INFO:tasks.workunit.client.0.smithi107.stdout:3/47: truncate f0 2760221 0 2024-04-22T23:34:57.779 INFO:tasks.workunit.client.0.smithi107.stdout:3/48: readlink l4 0 2024-04-22T23:34:57.781 INFO:tasks.workunit.client.1.smithi157.stdout:2/500: creat d7/d1f/d26/fb0 x:0 0 0 2024-04-22T23:34:57.781 INFO:tasks.workunit.client.1.smithi157.stdout:2/501: chown d7/c6a 126 1 2024-04-22T23:34:57.799 INFO:tasks.workunit.client.0.smithi107.stdout:5/48: dwrite f3 [0,4194304] 0 2024-04-22T23:34:57.799 INFO:tasks.workunit.client.0.smithi107.stdout:5/49: readlink - no filename 2024-04-22T23:34:57.799 INFO:tasks.workunit.client.1.smithi157.stdout:6/580: dwrite d1/d2f/d3b/d6c/d83/fa6 [0,4194304] 0 2024-04-22T23:34:57.801 INFO:tasks.workunit.client.1.smithi157.stdout:6/581: creat d1/d38/d97/fb4 x:0 0 0 2024-04-22T23:34:57.801 INFO:tasks.workunit.client.0.smithi107.stdout:5/50: creat d9/db/f10 x:0 0 0 2024-04-22T23:34:57.802 INFO:tasks.workunit.client.1.smithi157.stdout:2/502: write d7/d8/d30/d23/f48 [180524,100195] 0 2024-04-22T23:34:57.802 INFO:tasks.workunit.client.1.smithi157.stdout:6/582: creat d1/d2f/d3b/d39/d4e/d74/fb5 x:0 0 0 2024-04-22T23:34:57.803 INFO:tasks.workunit.client.1.smithi157.stdout:2/503: creat d7/fb1 x:0 0 0 2024-04-22T23:34:57.804 INFO:tasks.workunit.client.1.smithi157.stdout:6/583: symlink d1/d2f/d3b/d39/d4e/lb6 0 2024-04-22T23:34:57.804 INFO:tasks.workunit.client.1.smithi157.stdout:2/504: mknod d7/d8/d8e/d94/cb2 0 2024-04-22T23:34:57.805 INFO:tasks.workunit.client.1.smithi157.stdout:0/363: dwrite d17/d23/f28 [0,4194304] 0 2024-04-22T23:34:57.807 INFO:tasks.workunit.client.1.smithi157.stdout:0/364: unlink d17/c35 0 2024-04-22T23:34:57.807 INFO:tasks.workunit.client.1.smithi157.stdout:0/365: fdatasync d17/d38/f65 0 2024-04-22T23:34:57.808 INFO:tasks.workunit.client.1.smithi157.stdout:0/366: creat d17/d23/d34/f7b x:0 0 0 2024-04-22T23:34:57.810 INFO:tasks.workunit.client.1.smithi157.stdout:5/441: fdatasync d1/d5/dc/d43/d4f/d60/d7a/f87 0 2024-04-22T23:34:57.810 INFO:tasks.workunit.client.1.smithi157.stdout:5/442: chown d1/d5 78 1 2024-04-22T23:34:57.810 INFO:tasks.workunit.client.1.smithi157.stdout:5/443: write d1/f90 [479990,119410] 0 2024-04-22T23:34:57.811 INFO:tasks.workunit.client.1.smithi157.stdout:7/331: dwrite d0/d9/d29/d33/d37/f4c [0,4194304] 0 2024-04-22T23:34:57.811 INFO:tasks.workunit.client.1.smithi157.stdout:7/332: chown d0/d9/db/f61 11606959 1 2024-04-22T23:34:57.812 INFO:tasks.workunit.client.1.smithi157.stdout:4/547: dwrite d3/d96/f67 [0,4194304] 0 2024-04-22T23:34:57.813 INFO:tasks.workunit.client.1.smithi157.stdout:3/362: dwrite d0/d3/d34/d39/f66 [0,4194304] 0 2024-04-22T23:34:57.815 INFO:tasks.workunit.client.1.smithi157.stdout:2/505: write d7/f13 [3119175,91932] 0 2024-04-22T23:34:57.815 INFO:tasks.workunit.client.1.smithi157.stdout:2/506: write d7/d8/d25/f4f [1284260,41619] 0 2024-04-22T23:34:57.815 INFO:tasks.workunit.client.1.smithi157.stdout:1/450: dwrite d4/d17/d34/d3b/d52/f7f [0,4194304] 0 2024-04-22T23:34:57.815 INFO:tasks.workunit.client.1.smithi157.stdout:1/451: write d4/d17/d1d/d33/d41/d54/f5b [246053,43405] 0 2024-04-22T23:34:57.819 INFO:tasks.workunit.client.1.smithi157.stdout:5/444: creat d1/d5/d7/d2f/d45/da4/fac x:0 0 0 2024-04-22T23:34:57.823 INFO:tasks.workunit.client.1.smithi157.stdout:0/367: read d17/f20 [1035843,109235] 0 2024-04-22T23:34:57.826 INFO:tasks.workunit.client.1.smithi157.stdout:3/363: unlink d0/d3/d34/d39/f53 0 2024-04-22T23:34:57.827 INFO:tasks.workunit.client.1.smithi157.stdout:3/364: chown d0/d3/d34/d39/f49 3188 1 2024-04-22T23:34:57.827 INFO:tasks.workunit.client.1.smithi157.stdout:1/452: dread d4/d17/d34/d3b/d50/f56 [0,4194304] 0 2024-04-22T23:34:57.827 INFO:tasks.workunit.client.1.smithi157.stdout:1/453: chown d4/db/d45/l79 209 1 2024-04-22T23:34:57.834 INFO:tasks.workunit.client.0.smithi107.stdout:5/51: dwrite d9/db/f10 [0,4194304] 0 2024-04-22T23:34:57.835 INFO:tasks.workunit.client.1.smithi157.stdout:6/584: dwrite d1/d2f/d54/f87 [0,4194304] 0 2024-04-22T23:34:57.835 INFO:tasks.workunit.client.1.smithi157.stdout:2/507: dwrite d7/d1f/d26/d4a/f53 [0,4194304] 0 2024-04-22T23:34:57.835 INFO:tasks.workunit.client.1.smithi157.stdout:7/333: dwrite d0/f7 [0,4194304] 0 2024-04-22T23:34:57.837 INFO:tasks.workunit.client.1.smithi157.stdout:7/334: write d0/d9/f3c [879667,92416] 0 2024-04-22T23:34:57.838 INFO:tasks.workunit.client.1.smithi157.stdout:6/585: dread d1/d36/f37 [0,4194304] 0 2024-04-22T23:34:57.838 INFO:tasks.workunit.client.1.smithi157.stdout:6/586: fdatasync d1/d17/f28 0 2024-04-22T23:34:57.838 INFO:tasks.workunit.client.0.smithi107.stdout:5/52: rename d9/db/cf to d9/db/c11 0 2024-04-22T23:34:57.840 INFO:tasks.workunit.client.0.smithi107.stdout:3/49: truncate f8 2625138 0 2024-04-22T23:34:57.840 INFO:tasks.workunit.client.1.smithi157.stdout:5/445: creat d1/d5/dc/d5d/d77/fad x:0 0 0 2024-04-22T23:34:57.840 INFO:tasks.workunit.client.1.smithi157.stdout:5/446: creat d1/d27/d29/d4b/fae x:0 0 0 2024-04-22T23:34:57.841 INFO:tasks.workunit.client.1.smithi157.stdout:0/368: mkdir d17/d2b/d4f/d5a/d7c 0 2024-04-22T23:34:57.841 INFO:tasks.workunit.client.1.smithi157.stdout:0/369: chown d17/c2a 2550997 1 2024-04-22T23:34:57.842 INFO:tasks.workunit.client.1.smithi157.stdout:3/365: mknod d0/d3/d34/d3d/d57/d60/c71 0 2024-04-22T23:34:57.844 INFO:tasks.workunit.client.1.smithi157.stdout:3/366: dread d0/d3/f35 [0,4194304] 0 2024-04-22T23:34:57.844 INFO:tasks.workunit.client.1.smithi157.stdout:1/454: unlink d4/db/d3d/l44 0 2024-04-22T23:34:57.845 INFO:tasks.workunit.client.0.smithi107.stdout:3/50: write f8 [3664997,39438] 0 2024-04-22T23:34:57.846 INFO:tasks.workunit.client.1.smithi157.stdout:8/273: truncate d2/d4/f10 7307360 0 2024-04-22T23:34:57.847 INFO:tasks.workunit.client.0.smithi107.stdout:3/51: mkdir db 0 2024-04-22T23:34:57.849 INFO:tasks.workunit.client.1.smithi157.stdout:2/508: creat d7/d8/d30/fb3 x:0 0 0 2024-04-22T23:34:57.850 INFO:tasks.workunit.client.1.smithi157.stdout:7/335: dwrite d0/f3e [0,4194304] 0 2024-04-22T23:34:57.852 INFO:tasks.workunit.client.1.smithi157.stdout:6/587: creat d1/d2f/d54/d9b/fb7 x:0 0 0 2024-04-22T23:34:57.853 INFO:tasks.workunit.client.1.smithi157.stdout:6/588: chown d1/d36 909964 1 2024-04-22T23:34:57.854 INFO:tasks.workunit.client.1.smithi157.stdout:5/447: getdents d1/d5/d7 0 2024-04-22T23:34:57.855 INFO:tasks.workunit.client.1.smithi157.stdout:4/548: rename d3/d11/d30/d36 to d3/d4b/d7a/db7 0 2024-04-22T23:34:57.855 INFO:tasks.workunit.client.1.smithi157.stdout:4/549: chown d3/d11/d1d/daf/cb0 119508 1 2024-04-22T23:34:57.855 INFO:tasks.workunit.client.1.smithi157.stdout:4/550: write d3/d11/d34/f51 [873751,47936] 0 2024-04-22T23:34:57.856 INFO:tasks.workunit.client.1.smithi157.stdout:4/551: read d3/d11/d34/f51 [379785,43759] 0 2024-04-22T23:34:57.856 INFO:tasks.workunit.client.1.smithi157.stdout:3/367: mkdir d0/d3/d34/d3d/d5b/d72 0 2024-04-22T23:34:57.857 INFO:tasks.workunit.client.1.smithi157.stdout:1/455: mknod d4/d5f/d7e/c95 0 2024-04-22T23:34:57.858 INFO:tasks.workunit.client.1.smithi157.stdout:2/509: mkdir d7/d1f/d3e/db4 0 2024-04-22T23:34:57.861 INFO:tasks.workunit.client.1.smithi157.stdout:7/336: mknod d0/d9/d42/d55/d56/c75 0 2024-04-22T23:34:57.861 INFO:tasks.workunit.client.1.smithi157.stdout:7/337: fsync d0/f4 0 2024-04-22T23:34:57.862 INFO:tasks.workunit.client.1.smithi157.stdout:2/510: dread d7/d8/d30/d23/f48 [0,4194304] 0 2024-04-22T23:34:57.862 INFO:tasks.workunit.client.1.smithi157.stdout:2/511: readlink d7/d8/d30/d5b/d8a/lac 0 2024-04-22T23:34:57.863 INFO:tasks.workunit.client.1.smithi157.stdout:5/448: link d1/d5/dc/f1b d1/d1f/faf 0 2024-04-22T23:34:57.863 INFO:tasks.workunit.client.1.smithi157.stdout:5/449: stat d1/d27/d29/d4b/f4d 0 2024-04-22T23:34:57.865 INFO:tasks.workunit.client.1.smithi157.stdout:0/370: rename d17/d2b/l41 to d17/l7d 0 2024-04-22T23:34:57.868 INFO:tasks.workunit.client.1.smithi157.stdout:7/338: truncate d0/d9/db/f61 3311965 0 2024-04-22T23:34:57.870 INFO:tasks.workunit.client.1.smithi157.stdout:4/552: rename d3/d11/c8f to d3/d96/d40/cb8 0 2024-04-22T23:34:57.870 INFO:tasks.workunit.client.1.smithi157.stdout:4/553: write d3/d96/d81/f99 [4496864,96757] 0 2024-04-22T23:34:57.870 INFO:tasks.workunit.client.1.smithi157.stdout:4/554: write d3/d4b/d7a/db7/f65 [982221,19061] 0 2024-04-22T23:34:57.870 INFO:tasks.workunit.client.1.smithi157.stdout:0/371: getdents d17/d2b/d52 0 2024-04-22T23:34:57.871 INFO:tasks.workunit.client.1.smithi157.stdout:3/368: rmdir d0/d7/d6c 39 2024-04-22T23:34:57.874 INFO:tasks.workunit.client.1.smithi157.stdout:1/456: dwrite d4/f8e [0,4194304] 0 2024-04-22T23:34:57.878 INFO:tasks.workunit.client.1.smithi157.stdout:2/512: dwrite d7/d8/d6b/f8b [0,4194304] 0 2024-04-22T23:34:57.879 INFO:tasks.workunit.client.1.smithi157.stdout:1/457: symlink d4/d17/d34/d3b/d52/d58/l96 0 2024-04-22T23:34:57.879 INFO:tasks.workunit.client.1.smithi157.stdout:1/458: creat d4/d17/d6b/f97 x:0 0 0 2024-04-22T23:34:57.883 INFO:tasks.workunit.client.1.smithi157.stdout:2/513: getdents d7/d1f 0 2024-04-22T23:34:57.883 INFO:tasks.workunit.client.1.smithi157.stdout:2/514: write d7/d8/d8e/f78 [181486,128364] 0 2024-04-22T23:34:57.888 INFO:tasks.workunit.client.1.smithi157.stdout:7/339: dwrite d0/d9/f1d [0,4194304] 0 2024-04-22T23:34:57.890 INFO:tasks.workunit.client.1.smithi157.stdout:0/372: dwrite d17/d58/f5f [0,4194304] 0 2024-04-22T23:34:57.891 INFO:tasks.workunit.client.1.smithi157.stdout:2/515: dread d7/d8/dd/f2b [0,4194304] 0 2024-04-22T23:34:57.891 INFO:tasks.workunit.client.1.smithi157.stdout:2/516: truncate d7/d8/d30/fa6 52015 0 2024-04-22T23:34:57.891 INFO:tasks.workunit.client.1.smithi157.stdout:2/517: fsync d7/d8/fa0 0 2024-04-22T23:34:57.891 INFO:tasks.workunit.client.1.smithi157.stdout:2/518: truncate d7/d8/d30/d5b/f72 841435 0 2024-04-22T23:34:57.891 INFO:tasks.workunit.client.1.smithi157.stdout:2/519: read - d7/d1f/d26/f74 zero size 2024-04-22T23:34:57.892 INFO:tasks.workunit.client.1.smithi157.stdout:2/520: dread d7/d8/d30/f76 [0,4194304] 0 2024-04-22T23:34:57.893 INFO:tasks.workunit.client.1.smithi157.stdout:2/521: creat d7/d8/d30/fb5 x:0 0 0 2024-04-22T23:34:57.894 INFO:tasks.workunit.client.1.smithi157.stdout:1/459: dwrite d4/d17/d34/d3b/d50/f56 [4194304,4194304] 0 2024-04-22T23:34:57.894 INFO:tasks.workunit.client.1.smithi157.stdout:0/373: dread d17/d23/f7a [0,4194304] 0 2024-04-22T23:34:57.894 INFO:tasks.workunit.client.1.smithi157.stdout:1/460: chown d4/f1b 78008375 1 2024-04-22T23:34:57.894 INFO:tasks.workunit.client.1.smithi157.stdout:2/522: symlink d7/d8/dd/d4d/lb6 0 2024-04-22T23:34:57.897 INFO:tasks.workunit.client.1.smithi157.stdout:0/374: mkdir d17/d2b/d52/d7e 0 2024-04-22T23:34:57.899 INFO:tasks.workunit.client.1.smithi157.stdout:2/523: creat d7/d8/d8e/d94/fb7 x:0 0 0 2024-04-22T23:34:57.900 INFO:tasks.workunit.client.1.smithi157.stdout:0/375: dread d17/f2d [0,4194304] 0 2024-04-22T23:34:57.900 INFO:tasks.workunit.client.1.smithi157.stdout:0/376: fsync d17/f22 0 2024-04-22T23:34:57.900 INFO:tasks.workunit.client.1.smithi157.stdout:1/461: rename d4/db/d45/f81 to d4/d17/d1d/d33/d41/d54/f98 0 2024-04-22T23:34:57.901 INFO:tasks.workunit.client.1.smithi157.stdout:2/524: truncate d7/d8/d25/f4f 584170 0 2024-04-22T23:34:57.903 INFO:tasks.workunit.client.1.smithi157.stdout:1/462: rmdir d4/db/d45 39 2024-04-22T23:34:57.905 INFO:tasks.workunit.client.1.smithi157.stdout:4/555: unlink d3/d96/d40/cb8 0 2024-04-22T23:34:57.908 INFO:tasks.workunit.client.1.smithi157.stdout:4/556: truncate d3/d5c/f79 1317680 0 2024-04-22T23:34:57.910 INFO:tasks.workunit.client.1.smithi157.stdout:2/525: mkdir d7/d8/d30/db8 0 2024-04-22T23:34:57.910 INFO:tasks.workunit.client.1.smithi157.stdout:2/526: fsync d7/d8/d6b/f97 0 2024-04-22T23:34:57.910 INFO:tasks.workunit.client.1.smithi157.stdout:2/527: write d7/d1f/d26/d4a/f77 [1920032,20458] 0 2024-04-22T23:34:57.910 INFO:tasks.workunit.client.1.smithi157.stdout:2/528: truncate d7/d1f/d26/d4a/f77 2328037 0 2024-04-22T23:34:57.911 INFO:tasks.workunit.client.1.smithi157.stdout:0/377: rename d17/d58 to d17/d23/d34/d67/d7f 0 2024-04-22T23:34:57.913 INFO:tasks.workunit.client.1.smithi157.stdout:1/463: creat d4/d17/d34/d3b/d52/d65/d8c/f99 x:0 0 0 2024-04-22T23:34:57.914 INFO:tasks.workunit.client.1.smithi157.stdout:1/464: write d4/d17/d34/d75/f89 [680421,98527] 0 2024-04-22T23:34:57.914 INFO:tasks.workunit.client.1.smithi157.stdout:4/557: symlink d3/d4b/d5d/lb9 0 2024-04-22T23:34:57.915 INFO:tasks.workunit.client.1.smithi157.stdout:4/558: write d3/d4b/f18 [874813,100573] 0 2024-04-22T23:34:57.916 INFO:tasks.workunit.client.1.smithi157.stdout:0/378: link d17/d23/d34/d67/d7f/f77 d17/d38/f80 0 2024-04-22T23:34:57.918 INFO:tasks.workunit.client.1.smithi157.stdout:1/465: creat d4/d17/d1d/d33/f9a x:0 0 0 2024-04-22T23:34:57.918 INFO:tasks.workunit.client.1.smithi157.stdout:1/466: chown d4/d17/d34/f38 6 1 2024-04-22T23:34:57.920 INFO:tasks.workunit.client.1.smithi157.stdout:4/559: creat d3/d11/d34/da4/db1/fba x:0 0 0 2024-04-22T23:34:57.922 INFO:tasks.workunit.client.1.smithi157.stdout:0/379: rename d17/d23/f3e to d17/d2b/d4f/d5a/f81 0 2024-04-22T23:34:57.923 INFO:tasks.workunit.client.1.smithi157.stdout:2/529: dwrite d7/d8/d25/f3a [0,4194304] 0 2024-04-22T23:34:57.923 INFO:tasks.workunit.client.1.smithi157.stdout:2/530: write d7/d8/d30/fb3 [913233,77743] 0 2024-04-22T23:34:57.923 INFO:tasks.workunit.client.1.smithi157.stdout:2/531: truncate d7/d8/d8e/f7e 296333 0 2024-04-22T23:34:57.924 INFO:tasks.workunit.client.1.smithi157.stdout:1/467: symlink d4/d17/d34/d3b/d52/d65/d8c/l9b 0 2024-04-22T23:34:57.926 INFO:tasks.workunit.client.1.smithi157.stdout:1/468: truncate d4/db/f1e 936705 0 2024-04-22T23:34:57.929 INFO:tasks.workunit.client.1.smithi157.stdout:0/380: link d17/d2b/d4f/l75 d17/d23/d34/l82 0 2024-04-22T23:34:57.932 INFO:tasks.workunit.client.1.smithi157.stdout:0/381: truncate d17/d23/f7a 1525835 0 2024-04-22T23:34:57.933 INFO:tasks.workunit.client.1.smithi157.stdout:2/532: dread d7/d1f/d26/f44 [0,4194304] 0 2024-04-22T23:34:57.934 INFO:tasks.workunit.client.1.smithi157.stdout:1/469: rename d4/d17/l7a to d4/d17/d1d/d33/l9c 0 2024-04-22T23:34:57.936 INFO:tasks.workunit.client.1.smithi157.stdout:2/533: symlink d7/d1f/lb9 0 2024-04-22T23:34:57.936 INFO:tasks.workunit.client.1.smithi157.stdout:2/534: rename d7/d8/d25 to d7/d8/d25/d55/dba 22 2024-04-22T23:34:57.936 INFO:tasks.workunit.client.1.smithi157.stdout:2/535: chown d7/d1f/d26 23556403 1 2024-04-22T23:34:57.936 INFO:tasks.workunit.client.1.smithi157.stdout:2/536: chown d7/l87 8392 1 2024-04-22T23:34:57.936 INFO:tasks.workunit.client.1.smithi157.stdout:2/537: stat d7/d8/d30 0 2024-04-22T23:34:57.937 INFO:tasks.workunit.client.1.smithi157.stdout:2/538: symlink d7/d8/lbb 0 2024-04-22T23:34:57.937 INFO:tasks.workunit.client.1.smithi157.stdout:2/539: chown d7/d8/d25 0 1 2024-04-22T23:34:57.939 INFO:tasks.workunit.client.1.smithi157.stdout:2/540: symlink d7/d1f/d26/lbc 0 2024-04-22T23:34:57.939 INFO:tasks.workunit.client.1.smithi157.stdout:2/541: chown d7 9481 1 2024-04-22T23:34:57.940 INFO:tasks.workunit.client.1.smithi157.stdout:4/560: dwrite d3/d11/d1d/f2d [4194304,4194304] 0 2024-04-22T23:34:57.940 INFO:tasks.workunit.client.1.smithi157.stdout:4/561: chown d3/d11/d34 3556 1 2024-04-22T23:34:57.943 INFO:tasks.workunit.client.1.smithi157.stdout:4/562: symlink d3/d11/d1d/d73/d80/dac/lbb 0 2024-04-22T23:34:57.944 INFO:tasks.workunit.client.1.smithi157.stdout:1/470: dread d4/d17/f35 [0,4194304] 0 2024-04-22T23:34:57.944 INFO:tasks.workunit.client.1.smithi157.stdout:1/471: creat d4/d17/d1d/d33/d41/f9d x:0 0 0 2024-04-22T23:34:57.951 INFO:tasks.workunit.client.1.smithi157.stdout:4/563: unlink d3/d4b/f18 0 2024-04-22T23:34:57.953 INFO:tasks.workunit.client.1.smithi157.stdout:1/472: mknod d4/d5f/d7e/c9e 0 2024-04-22T23:34:57.955 INFO:tasks.workunit.client.1.smithi157.stdout:4/564: creat d3/d11/d34/da4/fbc x:0 0 0 2024-04-22T23:34:57.955 INFO:tasks.workunit.client.1.smithi157.stdout:4/565: chown d3/d4b/d7a/db7/c3b 342251530 1 2024-04-22T23:34:57.955 INFO:tasks.workunit.client.1.smithi157.stdout:4/566: fdatasync d3/d4b/d7a/db7/f65 0 2024-04-22T23:34:57.956 INFO:tasks.workunit.client.1.smithi157.stdout:1/473: link d4/d17/d34/d3b/d52/d58/l5d d4/d17/d34/l9f 0 2024-04-22T23:34:57.963 INFO:tasks.workunit.client.1.smithi157.stdout:4/567: dread d3/d11/d1d/d73/da3/f3c [0,4194304] 0 2024-04-22T23:34:57.965 INFO:tasks.workunit.client.1.smithi157.stdout:4/568: creat d3/d11/d1d/d73/fbd x:0 0 0 2024-04-22T23:34:57.969 INFO:tasks.workunit.client.1.smithi157.stdout:1/474: dwrite d4/d17/d34/d3b/d52/f7f [0,4194304] 0 2024-04-22T23:34:57.970 INFO:tasks.workunit.client.1.smithi157.stdout:1/475: read d4/d17/d1d/d33/d41/d54/f5b [119457,26719] 0 2024-04-22T23:34:57.971 INFO:tasks.workunit.client.1.smithi157.stdout:4/569: dread d3/d11/d1d/d73/d80/dac/db5/f8b [0,4194304] 0 2024-04-22T23:34:57.972 INFO:tasks.workunit.client.1.smithi157.stdout:4/570: creat d3/d11/d1d/d73/d80/dac/fbe x:0 0 0 2024-04-22T23:34:57.972 INFO:tasks.workunit.client.1.smithi157.stdout:4/571: fdatasync d3/d11/d1d/f24 0 2024-04-22T23:34:57.974 INFO:tasks.workunit.client.1.smithi157.stdout:4/572: link d3/d96/d4c/c92 d3/d11/d34/da6/cbf 0 2024-04-22T23:34:57.974 INFO:tasks.workunit.client.1.smithi157.stdout:4/573: chown d3/d4b/cb6 8 1 2024-04-22T23:34:57.975 INFO:tasks.workunit.client.1.smithi157.stdout:4/574: creat d3/d4b/d7a/db7/fc0 x:0 0 0 2024-04-22T23:34:57.975 INFO:tasks.workunit.client.1.smithi157.stdout:4/575: chown d3/d11/d34/f48 412482 1 2024-04-22T23:34:57.981 INFO:tasks.workunit.client.1.smithi157.stdout:4/576: write d3/f1b [6595482,86704] 0 2024-04-22T23:34:57.981 INFO:tasks.workunit.client.1.smithi157.stdout:4/577: chown d3/f43 7841 1 2024-04-22T23:34:57.988 INFO:tasks.workunit.client.1.smithi157.stdout:9/366: sync 2024-04-22T23:34:57.991 INFO:tasks.workunit.client.1.smithi157.stdout:1/476: dwrite d4/d17/d1d/d33/d41/f5e [0,4194304] 0 2024-04-22T23:34:58.003 INFO:tasks.workunit.client.1.smithi157.stdout:1/477: dread d4/d17/d6b/f8d [0,4194304] 0 2024-04-22T23:34:58.005 INFO:tasks.workunit.client.1.smithi157.stdout:1/478: link d4/d17/c1c d4/d17/d34/ca0 0 2024-04-22T23:34:58.005 INFO:tasks.workunit.client.1.smithi157.stdout:1/479: chown d4/d17/d34/f63 1229926 1 2024-04-22T23:34:58.005 INFO:tasks.workunit.client.1.smithi157.stdout:1/480: chown d4/db/f47 0 1 2024-04-22T23:34:58.005 INFO:tasks.workunit.client.1.smithi157.stdout:1/481: fdatasync d4/f3e 0 2024-04-22T23:34:58.006 INFO:tasks.workunit.client.1.smithi157.stdout:1/482: write d4/f23 [9192019,124771] 0 2024-04-22T23:34:58.006 INFO:tasks.workunit.client.1.smithi157.stdout:1/483: creat d4/db/d3d/fa1 x:0 0 0 2024-04-22T23:34:58.006 INFO:tasks.workunit.client.1.smithi157.stdout:1/484: creat d4/d17/d1d/d33/fa2 x:0 0 0 2024-04-22T23:34:58.006 INFO:tasks.workunit.client.1.smithi157.stdout:1/485: chown d4/d17/d1d/d33/d41/d54/f98 165868509 1 2024-04-22T23:34:58.006 INFO:tasks.workunit.client.1.smithi157.stdout:9/367: dwrite d4/df/f62 [0,4194304] 0 2024-04-22T23:34:58.008 INFO:tasks.workunit.client.1.smithi157.stdout:1/486: rename d4/d17/d1d/d33/l9c to d4/d17/la3 0 2024-04-22T23:34:58.008 INFO:tasks.workunit.client.1.smithi157.stdout:1/487: fdatasync d4/d17/d1d/d33/d41/f9d 0 2024-04-22T23:34:58.009 INFO:tasks.workunit.client.1.smithi157.stdout:1/488: symlink d4/db/d45/la4 0 2024-04-22T23:34:58.010 INFO:tasks.workunit.client.1.smithi157.stdout:1/489: link d4/f23 d4/d17/d34/d84/d94/fa5 0 2024-04-22T23:34:58.011 INFO:tasks.workunit.client.1.smithi157.stdout:9/368: write d4/df/d57/f44 [634500,65077] 0 2024-04-22T23:34:58.011 INFO:tasks.workunit.client.1.smithi157.stdout:9/369: fdatasync d4/d15/d23/d27/f53 0 2024-04-22T23:34:58.012 INFO:tasks.workunit.client.1.smithi157.stdout:1/490: mkdir d4/d17/da6 0 2024-04-22T23:34:58.012 INFO:tasks.workunit.client.1.smithi157.stdout:9/370: creat d4/df/f6a x:0 0 0 2024-04-22T23:34:58.012 INFO:tasks.workunit.client.1.smithi157.stdout:9/371: getdents d4/d15 0 2024-04-22T23:34:58.013 INFO:tasks.workunit.client.1.smithi157.stdout:1/491: link d4/d5f/l62 d4/d17/d1d/d33/d41/la7 0 2024-04-22T23:34:58.014 INFO:tasks.workunit.client.1.smithi157.stdout:9/372: rmdir d4/df/d61 39 2024-04-22T23:34:58.014 INFO:tasks.workunit.client.1.smithi157.stdout:9/373: write d4/d15/d23/d27/d47/f4a [1038352,97936] 0 2024-04-22T23:34:58.015 INFO:tasks.workunit.client.1.smithi157.stdout:1/492: truncate d4/f3e 444867 0 2024-04-22T23:34:58.016 INFO:tasks.workunit.client.1.smithi157.stdout:9/374: getdents d4/d15/d23 0 2024-04-22T23:34:58.016 INFO:tasks.workunit.client.1.smithi157.stdout:9/375: write d4/d15/f4e [1386973,71713] 0 2024-04-22T23:34:58.016 INFO:tasks.workunit.client.1.smithi157.stdout:1/493: getdents d4/d17/d34/d75 0 2024-04-22T23:34:58.019 INFO:tasks.workunit.client.1.smithi157.stdout:9/376: unlink d4/df/d61/f68 0 2024-04-22T23:34:58.021 INFO:tasks.workunit.client.1.smithi157.stdout:9/377: mkdir d4/df/d57/d6b 0 2024-04-22T23:34:58.025 INFO:tasks.workunit.client.1.smithi157.stdout:1/494: dread d4/d17/f2d [0,4194304] 0 2024-04-22T23:34:58.025 INFO:tasks.workunit.client.1.smithi157.stdout:9/378: dread d4/d15/d23/d3f/f42 [0,4194304] 0 2024-04-22T23:34:58.026 INFO:tasks.workunit.client.1.smithi157.stdout:1/495: mknod d4/d17/d34/d75/ca8 0 2024-04-22T23:34:58.027 INFO:tasks.workunit.client.1.smithi157.stdout:1/496: creat d4/d17/da6/fa9 x:0 0 0 2024-04-22T23:34:58.028 INFO:tasks.workunit.client.1.smithi157.stdout:1/497: getdents d4/d17 0 2024-04-22T23:34:58.028 INFO:tasks.workunit.client.1.smithi157.stdout:1/498: write d4/f25 [7026808,62449] 0 2024-04-22T23:34:58.028 INFO:tasks.workunit.client.1.smithi157.stdout:1/499: read - d4/d17/d1d/d33/d41/f85 zero size 2024-04-22T23:34:58.029 INFO:tasks.workunit.client.1.smithi157.stdout:1/500: write d4/db/f47 [467341,113408] 0 2024-04-22T23:34:58.030 INFO:tasks.workunit.client.1.smithi157.stdout:1/501: creat d4/db/d45/faa x:0 0 0 2024-04-22T23:34:58.031 INFO:tasks.workunit.client.1.smithi157.stdout:1/502: rmdir d4/d17/d1d/d33/d41/d54 39 2024-04-22T23:34:58.032 INFO:tasks.workunit.client.1.smithi157.stdout:9/379: dread d4/f50 [0,4194304] 0 2024-04-22T23:34:58.032 INFO:tasks.workunit.client.1.smithi157.stdout:9/380: dread - d4/d15/d23/d27/f64 zero size 2024-04-22T23:34:58.033 INFO:tasks.workunit.client.1.smithi157.stdout:1/503: creat d4/d5f/d7e/fab x:0 0 0 2024-04-22T23:34:58.033 INFO:tasks.workunit.client.1.smithi157.stdout:1/504: stat d4/d17/d1d/l27 0 2024-04-22T23:34:58.033 INFO:tasks.workunit.client.1.smithi157.stdout:9/381: dread d4/d15/f4c [0,4194304] 0 2024-04-22T23:34:58.033 INFO:tasks.workunit.client.1.smithi157.stdout:1/505: symlink d4/d17/d34/d3b/d52/lac 0 2024-04-22T23:34:58.034 INFO:tasks.workunit.client.1.smithi157.stdout:9/382: truncate d4/f20 3133297 0 2024-04-22T23:34:58.037 INFO:tasks.workunit.client.1.smithi157.stdout:1/506: write d4/d17/d34/d3b/d50/f56 [645617,83646] 0 2024-04-22T23:34:58.037 INFO:tasks.workunit.client.1.smithi157.stdout:1/507: write d4/d5f/d7e/fab [865099,54070] 0 2024-04-22T23:34:58.039 INFO:tasks.workunit.client.1.smithi157.stdout:1/508: fdatasync d4/d17/d34/d84/d94/fa5 0 2024-04-22T23:34:58.057 INFO:tasks.workunit.client.1.smithi157.stdout:9/383: dwrite d4/df/f16 [0,4194304] 0 2024-04-22T23:34:58.059 INFO:tasks.workunit.client.1.smithi157.stdout:1/509: dwrite d4/d17/da6/fa9 [0,4194304] 0 2024-04-22T23:34:58.064 INFO:tasks.workunit.client.1.smithi157.stdout:9/384: dread d4/df/f12 [0,4194304] 0 2024-04-22T23:34:58.085 INFO:tasks.workunit.client.1.smithi157.stdout:8/274: sync 2024-04-22T23:34:58.086 INFO:tasks.workunit.client.1.smithi157.stdout:8/275: symlink d2/d4/d30/l64 0 2024-04-22T23:34:58.087 INFO:tasks.workunit.client.1.smithi157.stdout:8/276: chown d2/d4/f52 29817 1 2024-04-22T23:34:58.087 INFO:tasks.workunit.client.1.smithi157.stdout:8/277: chown d2/d6/d21/d5f/d17/f49 2 1 2024-04-22T23:34:58.087 INFO:tasks.workunit.client.1.smithi157.stdout:8/278: symlink d2/d4/d30/l65 0 2024-04-22T23:34:58.087 INFO:tasks.workunit.client.1.smithi157.stdout:8/279: read - d2/f5d zero size 2024-04-22T23:34:58.088 INFO:tasks.workunit.client.1.smithi157.stdout:8/280: mkdir d2/d6/d66 0 2024-04-22T23:34:58.088 INFO:tasks.workunit.client.1.smithi157.stdout:8/281: chown d2/d6/d66 125461 1 2024-04-22T23:34:58.090 INFO:tasks.workunit.client.1.smithi157.stdout:8/282: dread d2/d34/f41 [0,4194304] 0 2024-04-22T23:34:58.090 INFO:tasks.workunit.client.1.smithi157.stdout:8/283: fsync d2/d6/f3c 0 2024-04-22T23:34:58.091 INFO:tasks.workunit.client.1.smithi157.stdout:8/284: mkdir d2/d6/d66/d67 0 2024-04-22T23:34:58.099 INFO:tasks.workunit.client.1.smithi157.stdout:8/285: write d2/d4/f10 [3559109,24185] 0 2024-04-22T23:34:58.099 INFO:tasks.workunit.client.1.smithi157.stdout:8/286: readlink d2/d6/d21/d5f/d17/l24 0 2024-04-22T23:34:58.101 INFO:tasks.workunit.client.1.smithi157.stdout:8/287: symlink d2/d6/d21/d5f/d11/l68 0 2024-04-22T23:34:58.119 INFO:tasks.workunit.client.1.smithi157.stdout:8/288: dwrite d2/d6/d21/d5f/d11/f4b [0,4194304] 0 2024-04-22T23:34:58.119 INFO:tasks.workunit.client.1.smithi157.stdout:8/289: stat d2/d6/d21/d33/d42/f61 0 2024-04-22T23:34:58.119 INFO:tasks.workunit.client.1.smithi157.stdout:8/290: chown d2/d6/d21/d33/d42/f59 717 1 2024-04-22T23:34:58.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:34:58 smithi157 ceph-mon[33877]: from='client.24411 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:34:58.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:34:58 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/1593697064' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:34:58.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:34:58 smithi107 ceph-mon[26355]: from='client.24411 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:34:58.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:34:58 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/1593697064' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:34:59.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:34:59 smithi157 ceph-mon[33877]: pgmap v256: 65 pgs: 65 active+clean; 791 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 68 MiB/s rd, 93 MiB/s wr, 1.15k op/s 2024-04-22T23:34:59.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:34:59 smithi107 ceph-mon[26355]: pgmap v256: 65 pgs: 65 active+clean; 791 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 68 MiB/s rd, 93 MiB/s wr, 1.15k op/s 2024-04-22T23:34:59.818 INFO:tasks.workunit.client.0.smithi107.stdout:9/4: sync 2024-04-22T23:34:59.818 INFO:tasks.workunit.client.0.smithi107.stdout:5/53: sync 2024-04-22T23:34:59.818 INFO:tasks.workunit.client.0.smithi107.stdout:8/1: sync 2024-04-22T23:34:59.818 INFO:tasks.workunit.client.0.smithi107.stdout:2/19: sync 2024-04-22T23:34:59.818 INFO:tasks.workunit.client.0.smithi107.stdout:2/20: chown d1/f4 95760461 1 2024-04-22T23:34:59.818 INFO:tasks.workunit.client.0.smithi107.stdout:0/23: sync 2024-04-22T23:34:59.818 INFO:tasks.workunit.client.0.smithi107.stdout:7/22: sync 2024-04-22T23:34:59.818 INFO:tasks.workunit.client.0.smithi107.stdout:3/52: sync 2024-04-22T23:34:59.818 INFO:tasks.workunit.client.0.smithi107.stdout:4/18: sync 2024-04-22T23:34:59.818 INFO:tasks.workunit.client.0.smithi107.stdout:6/21: sync 2024-04-22T23:34:59.819 INFO:tasks.workunit.client.0.smithi107.stdout:1/25: sync 2024-04-22T23:34:59.819 INFO:tasks.workunit.client.0.smithi107.stdout:1/26: write f3 [1604028,129404] 0 2024-04-22T23:34:59.819 INFO:tasks.workunit.client.0.smithi107.stdout:9/5: rename f0 to f1 0 2024-04-22T23:34:59.819 INFO:tasks.workunit.client.0.smithi107.stdout:9/6: chown f1 134809 1 2024-04-22T23:34:59.819 INFO:tasks.workunit.client.0.smithi107.stdout:9/7: creat f2 x:0 0 0 2024-04-22T23:34:59.820 INFO:tasks.workunit.client.0.smithi107.stdout:5/54: link f7 d9/f12 0 2024-04-22T23:34:59.821 INFO:tasks.workunit.client.0.smithi107.stdout:2/21: mkdir d1/da 0 2024-04-22T23:34:59.821 INFO:tasks.workunit.client.0.smithi107.stdout:2/22: truncate d1/f6 184358 0 2024-04-22T23:34:59.821 INFO:tasks.workunit.client.0.smithi107.stdout:2/23: getdents d1/da 0 2024-04-22T23:34:59.821 INFO:tasks.workunit.client.0.smithi107.stdout:2/24: chown c0 4 1 2024-04-22T23:34:59.821 INFO:tasks.workunit.client.0.smithi107.stdout:2/25: creat d1/fb x:0 0 0 2024-04-22T23:34:59.821 INFO:tasks.workunit.client.0.smithi107.stdout:2/26: truncate d1/fb 760069 0 2024-04-22T23:34:59.821 INFO:tasks.workunit.client.0.smithi107.stdout:8/2: creat d0/f1 x:0 0 0 2024-04-22T23:34:59.821 INFO:tasks.workunit.client.0.smithi107.stdout:8/3: dread - d0/f1 zero size 2024-04-22T23:34:59.822 INFO:tasks.workunit.client.0.smithi107.stdout:8/4: dread - d0/f1 zero size 2024-04-22T23:34:59.822 INFO:tasks.workunit.client.0.smithi107.stdout:7/23: rmdir d5 0 2024-04-22T23:34:59.825 INFO:tasks.workunit.client.0.smithi107.stdout:7/24: write f2 [4262716,75925] 0 2024-04-22T23:34:59.825 INFO:tasks.workunit.client.0.smithi107.stdout:7/25: rmdir - no directory 2024-04-22T23:34:59.826 INFO:tasks.workunit.client.0.smithi107.stdout:0/24: mkdir d1/d9 0 2024-04-22T23:34:59.826 INFO:tasks.workunit.client.0.smithi107.stdout:0/25: write d1/f2 [613688,105299] 0 2024-04-22T23:34:59.826 INFO:tasks.workunit.client.0.smithi107.stdout:3/53: mknod db/cc 0 2024-04-22T23:34:59.827 INFO:tasks.workunit.client.0.smithi107.stdout:4/19: symlink d1/l4 0 2024-04-22T23:34:59.827 INFO:tasks.workunit.client.0.smithi107.stdout:1/27: creat d7/f9 x:0 0 0 2024-04-22T23:34:59.828 INFO:tasks.workunit.client.0.smithi107.stdout:5/55: link f8 d9/db/f13 0 2024-04-22T23:34:59.830 INFO:tasks.workunit.client.0.smithi107.stdout:2/27: symlink d1/da/lc 0 2024-04-22T23:34:59.830 INFO:tasks.workunit.client.0.smithi107.stdout:2/28: stat d1/f6 0 2024-04-22T23:34:59.832 INFO:tasks.workunit.client.0.smithi107.stdout:8/5: rename d0/f1 to d0/f2 0 2024-04-22T23:34:59.832 INFO:tasks.workunit.client.0.smithi107.stdout:8/6: stat d0 0 2024-04-22T23:34:59.833 INFO:tasks.workunit.client.0.smithi107.stdout:7/26: symlink l6 0 2024-04-22T23:34:59.834 INFO:tasks.workunit.client.0.smithi107.stdout:0/26: mknod d1/ca 0 2024-04-22T23:34:59.835 INFO:tasks.workunit.client.0.smithi107.stdout:3/54: mkdir db/dd 0 2024-04-22T23:34:59.836 INFO:tasks.workunit.client.0.smithi107.stdout:4/20: creat d1/f5 x:0 0 0 2024-04-22T23:34:59.836 INFO:tasks.workunit.client.0.smithi107.stdout:4/21: creat d1/f6 x:0 0 0 2024-04-22T23:34:59.837 INFO:tasks.workunit.client.0.smithi107.stdout:5/56: mknod d9/c14 0 2024-04-22T23:34:59.838 INFO:tasks.workunit.client.0.smithi107.stdout:0/27: symlink d1/d9/lb 0 2024-04-22T23:34:59.841 INFO:tasks.workunit.client.0.smithi107.stdout:4/22: mkdir d1/d7 0 2024-04-22T23:34:59.842 INFO:tasks.workunit.client.0.smithi107.stdout:5/57: symlink d9/l15 0 2024-04-22T23:34:59.843 INFO:tasks.workunit.client.0.smithi107.stdout:4/23: mkdir d1/d8 0 2024-04-22T23:34:59.844 INFO:tasks.workunit.client.0.smithi107.stdout:6/22: dwrite f0 [0,4194304] 0 2024-04-22T23:34:59.844 INFO:tasks.workunit.client.0.smithi107.stdout:3/55: dread f0 [0,4194304] 0 2024-04-22T23:34:59.845 INFO:tasks.workunit.client.0.smithi107.stdout:4/24: mkdir d1/d8/d9 0 2024-04-22T23:34:59.845 INFO:tasks.workunit.client.0.smithi107.stdout:0/28: write d1/f2 [268030,35574] 0 2024-04-22T23:34:59.846 INFO:tasks.workunit.client.0.smithi107.stdout:6/23: symlink l2 0 2024-04-22T23:34:59.847 INFO:tasks.workunit.client.0.smithi107.stdout:3/56: mknod db/dd/ce 0 2024-04-22T23:34:59.850 INFO:tasks.workunit.client.0.smithi107.stdout:4/25: creat d1/d7/fa x:0 0 0 2024-04-22T23:34:59.851 INFO:tasks.workunit.client.0.smithi107.stdout:5/58: write f3 [1346872,63059] 0 2024-04-22T23:34:59.851 INFO:tasks.workunit.client.0.smithi107.stdout:0/29: creat d1/d9/fc x:0 0 0 2024-04-22T23:34:59.852 INFO:tasks.workunit.client.0.smithi107.stdout:3/57: chown c3 0 1 2024-04-22T23:34:59.852 INFO:tasks.workunit.client.0.smithi107.stdout:3/58: stat db/cc 0 2024-04-22T23:34:59.854 INFO:tasks.workunit.client.0.smithi107.stdout:4/26: symlink d1/d8/d9/lb 0 2024-04-22T23:34:59.854 INFO:tasks.workunit.client.0.smithi107.stdout:4/27: creat d1/d7/fc x:0 0 0 2024-04-22T23:34:59.854 INFO:tasks.workunit.client.0.smithi107.stdout:4/28: creat d1/d7/fd x:0 0 0 2024-04-22T23:34:59.858 INFO:tasks.workunit.client.0.smithi107.stdout:1/28: dwrite f3 [0,4194304] 0 2024-04-22T23:34:59.858 INFO:tasks.workunit.client.0.smithi107.stdout:1/29: stat f2 0 2024-04-22T23:34:59.858 INFO:tasks.workunit.client.0.smithi107.stdout:1/30: dread - d7/f9 zero size 2024-04-22T23:34:59.858 INFO:tasks.workunit.client.0.smithi107.stdout:1/31: truncate d7/f9 299490 0 2024-04-22T23:34:59.858 INFO:tasks.workunit.client.0.smithi107.stdout:5/59: link c6 d9/db/de/c16 0 2024-04-22T23:34:59.860 INFO:tasks.workunit.client.0.smithi107.stdout:3/59: creat db/ff x:0 0 0 2024-04-22T23:34:59.861 INFO:tasks.workunit.client.0.smithi107.stdout:0/30: dread d1/f2 [0,4194304] 0 2024-04-22T23:34:59.861 INFO:tasks.workunit.client.0.smithi107.stdout:2/29: dwrite d1/fb [0,4194304] 0 2024-04-22T23:34:59.864 INFO:tasks.workunit.client.0.smithi107.stdout:4/29: rename d1/f6 to d1/d8/fe 0 2024-04-22T23:34:59.865 INFO:tasks.workunit.client.0.smithi107.stdout:9/8: fsync f2 0 2024-04-22T23:34:59.868 INFO:tasks.workunit.client.0.smithi107.stdout:1/32: mknod d7/ca 0 2024-04-22T23:34:59.870 INFO:tasks.workunit.client.0.smithi107.stdout:3/60: symlink db/dd/l10 0 2024-04-22T23:34:59.871 INFO:tasks.workunit.client.0.smithi107.stdout:8/7: fsync d0/f2 0 2024-04-22T23:34:59.871 INFO:tasks.workunit.client.0.smithi107.stdout:8/8: fdatasync d0/f2 0 2024-04-22T23:34:59.874 INFO:tasks.workunit.client.0.smithi107.stdout:7/27: getdents . 0 2024-04-22T23:34:59.874 INFO:tasks.workunit.client.0.smithi107.stdout:7/28: chown l1 117 1 2024-04-22T23:34:59.876 INFO:tasks.workunit.client.0.smithi107.stdout:6/24: dwrite f0 [0,4194304] 0 2024-04-22T23:34:59.877 INFO:tasks.workunit.client.0.smithi107.stdout:5/60: dwrite f8 [4194304,4194304] 0 2024-04-22T23:34:59.878 INFO:tasks.workunit.client.0.smithi107.stdout:0/31: rename d1/c8 to d1/d9/cd 0 2024-04-22T23:34:59.880 INFO:tasks.workunit.client.0.smithi107.stdout:4/30: unlink d1/d8/d9/lb 0 2024-04-22T23:34:59.880 INFO:tasks.workunit.client.0.smithi107.stdout:0/32: dread d1/f2 [0,4194304] 0 2024-04-22T23:34:59.882 INFO:tasks.workunit.client.0.smithi107.stdout:8/9: rename d0/f2 to d0/f3 0 2024-04-22T23:34:59.882 INFO:tasks.workunit.client.0.smithi107.stdout:8/10: stat d0/f3 0 2024-04-22T23:34:59.882 INFO:tasks.workunit.client.0.smithi107.stdout:7/29: mkdir d7 0 2024-04-22T23:34:59.884 INFO:tasks.workunit.client.0.smithi107.stdout:9/9: dwrite f2 [0,4194304] 0 2024-04-22T23:34:59.884 INFO:tasks.workunit.client.0.smithi107.stdout:9/10: fsync f1 0 2024-04-22T23:34:59.885 INFO:tasks.workunit.client.0.smithi107.stdout:6/25: creat f3 x:0 0 0 2024-04-22T23:34:59.885 INFO:tasks.workunit.client.0.smithi107.stdout:6/26: fdatasync f3 0 2024-04-22T23:34:59.890 INFO:tasks.workunit.client.0.smithi107.stdout:5/61: symlink d9/l17 0 2024-04-22T23:34:59.891 INFO:tasks.workunit.client.0.smithi107.stdout:3/61: rename c3 to db/c11 0 2024-04-22T23:34:59.896 INFO:tasks.workunit.client.0.smithi107.stdout:1/33: getdents d7 0 2024-04-22T23:34:59.899 INFO:tasks.workunit.client.0.smithi107.stdout:5/62: mknod d9/db/de/c18 0 2024-04-22T23:34:59.900 INFO:tasks.workunit.client.0.smithi107.stdout:3/62: mkdir db/dd/d12 0 2024-04-22T23:34:59.901 INFO:tasks.workunit.client.0.smithi107.stdout:5/63: dread d9/fd [0,4194304] 0 2024-04-22T23:34:59.901 INFO:tasks.workunit.client.0.smithi107.stdout:4/31: rmdir d1/d7 39 2024-04-22T23:34:59.902 INFO:tasks.workunit.client.0.smithi107.stdout:8/11: rmdir d0 39 2024-04-22T23:34:59.903 INFO:tasks.workunit.client.0.smithi107.stdout:1/34: mknod d7/cb 0 2024-04-22T23:34:59.904 INFO:tasks.workunit.client.0.smithi107.stdout:9/11: getdents . 0 2024-04-22T23:34:59.905 INFO:tasks.workunit.client.0.smithi107.stdout:6/27: truncate f0 3122896 0 2024-04-22T23:34:59.906 INFO:tasks.workunit.client.0.smithi107.stdout:6/28: creat f4 x:0 0 0 2024-04-22T23:34:59.907 INFO:tasks.workunit.client.0.smithi107.stdout:3/63: unlink c9 0 2024-04-22T23:34:59.907 INFO:tasks.workunit.client.0.smithi107.stdout:3/64: write f0 [829594,49421] 0 2024-04-22T23:34:59.909 INFO:tasks.workunit.client.0.smithi107.stdout:4/32: symlink d1/d7/lf 0 2024-04-22T23:34:59.913 INFO:tasks.workunit.client.0.smithi107.stdout:6/29: rename l2 to l5 0 2024-04-22T23:34:59.913 INFO:tasks.workunit.client.0.smithi107.stdout:6/30: dread - f4 zero size 2024-04-22T23:34:59.916 INFO:tasks.workunit.client.0.smithi107.stdout:3/65: rename ca to db/c13 0 2024-04-22T23:34:59.927 INFO:tasks.workunit.client.0.smithi107.stdout:9/12: write f2 [3037172,97461] 0 2024-04-22T23:34:59.927 INFO:tasks.workunit.client.0.smithi107.stdout:9/13: write f2 [287145,73968] 0 2024-04-22T23:34:59.927 INFO:tasks.workunit.client.0.smithi107.stdout:7/30: dwrite f2 [0,4194304] 0 2024-04-22T23:34:59.929 INFO:tasks.workunit.client.0.smithi107.stdout:3/66: fdatasync f0 0 2024-04-22T23:34:59.929 INFO:tasks.workunit.client.0.smithi107.stdout:3/67: readlink l4 0 2024-04-22T23:34:59.929 INFO:tasks.workunit.client.0.smithi107.stdout:3/68: write db/ff [883900,105837] 0 2024-04-22T23:34:59.929 INFO:tasks.workunit.client.0.smithi107.stdout:9/14: dread f1 [0,4194304] 0 2024-04-22T23:34:59.929 INFO:tasks.workunit.client.0.smithi107.stdout:9/15: truncate f1 1822803 0 2024-04-22T23:34:59.929 INFO:tasks.workunit.client.0.smithi107.stdout:5/64: dwrite d9/f12 [0,4194304] 0 2024-04-22T23:34:59.930 INFO:tasks.workunit.client.0.smithi107.stdout:9/16: dread f1 [0,4194304] 0 2024-04-22T23:34:59.933 INFO:tasks.workunit.client.0.smithi107.stdout:3/69: symlink db/dd/d12/l14 0 2024-04-22T23:34:59.933 INFO:tasks.workunit.client.0.smithi107.stdout:3/70: chown db/c11 87094 1 2024-04-22T23:34:59.934 INFO:tasks.workunit.client.0.smithi107.stdout:5/65: symlink d9/db/de/l19 0 2024-04-22T23:34:59.935 INFO:tasks.workunit.client.0.smithi107.stdout:9/17: creat f3 x:0 0 0 2024-04-22T23:34:59.935 INFO:tasks.workunit.client.0.smithi107.stdout:9/18: creat f4 x:0 0 0 2024-04-22T23:34:59.938 INFO:tasks.workunit.client.0.smithi107.stdout:5/66: symlink d9/db/de/l1a 0 2024-04-22T23:34:59.938 INFO:tasks.workunit.client.0.smithi107.stdout:5/67: write f4 [4550029,73225] 0 2024-04-22T23:34:59.940 INFO:tasks.workunit.client.0.smithi107.stdout:5/68: mknod d9/c1b 0 2024-04-22T23:34:59.941 INFO:tasks.workunit.client.0.smithi107.stdout:4/33: dwrite d1/f3 [0,4194304] 0 2024-04-22T23:34:59.941 INFO:tasks.workunit.client.0.smithi107.stdout:4/34: write d1/d8/fe [840490,128337] 0 2024-04-22T23:34:59.941 INFO:tasks.workunit.client.0.smithi107.stdout:4/35: creat d1/f10 x:0 0 0 2024-04-22T23:34:59.942 INFO:tasks.workunit.client.0.smithi107.stdout:5/69: symlink d9/db/de/l1c 0 2024-04-22T23:34:59.942 INFO:tasks.workunit.client.0.smithi107.stdout:8/12: dwrite d0/f3 [0,4194304] 0 2024-04-22T23:34:59.942 INFO:tasks.workunit.client.0.smithi107.stdout:7/31: dread f2 [0,4194304] 0 2024-04-22T23:34:59.946 INFO:tasks.workunit.client.0.smithi107.stdout:1/35: dwrite d7/f9 [0,4194304] 0 2024-04-22T23:34:59.947 INFO:tasks.workunit.client.0.smithi107.stdout:4/36: rename d1/d7/fa to d1/d7/f11 0 2024-04-22T23:34:59.950 INFO:tasks.workunit.client.0.smithi107.stdout:5/70: dread f3 [0,4194304] 0 2024-04-22T23:34:59.950 INFO:tasks.workunit.client.0.smithi107.stdout:7/32: symlink d7/l8 0 2024-04-22T23:34:59.955 INFO:tasks.workunit.client.0.smithi107.stdout:3/71: rename db/dd/d12 to db/dd/d15 0 2024-04-22T23:34:59.956 INFO:tasks.workunit.client.0.smithi107.stdout:4/37: symlink d1/d7/l12 0 2024-04-22T23:34:59.956 INFO:tasks.workunit.client.0.smithi107.stdout:9/19: dwrite f1 [0,4194304] 0 2024-04-22T23:34:59.957 INFO:tasks.workunit.client.0.smithi107.stdout:6/31: dwrite f0 [0,4194304] 0 2024-04-22T23:34:59.959 INFO:tasks.workunit.client.0.smithi107.stdout:5/71: link f8 d9/db/f1d 0 2024-04-22T23:34:59.960 INFO:tasks.workunit.client.0.smithi107.stdout:3/72: truncate f0 1119592 0 2024-04-22T23:34:59.965 INFO:tasks.workunit.client.0.smithi107.stdout:5/72: dread d9/fd [0,4194304] 0 2024-04-22T23:34:59.965 INFO:tasks.workunit.client.0.smithi107.stdout:5/73: chown d9/db/cc 125506 1 2024-04-22T23:34:59.965 INFO:tasks.workunit.client.0.smithi107.stdout:5/74: readlink d9/db/de/l19 0 2024-04-22T23:34:59.965 INFO:tasks.workunit.client.0.smithi107.stdout:9/20: unlink f2 0 2024-04-22T23:34:59.966 INFO:tasks.workunit.client.0.smithi107.stdout:9/21: fsync f1 0 2024-04-22T23:34:59.966 INFO:tasks.workunit.client.0.smithi107.stdout:9/22: write f4 [866645,117869] 0 2024-04-22T23:34:59.966 INFO:tasks.workunit.client.0.smithi107.stdout:9/23: rmdir - no directory 2024-04-22T23:34:59.966 INFO:tasks.workunit.client.0.smithi107.stdout:9/24: rmdir - no directory 2024-04-22T23:34:59.970 INFO:tasks.workunit.client.0.smithi107.stdout:3/73: mknod db/dd/c16 0 2024-04-22T23:34:59.972 INFO:tasks.workunit.client.0.smithi107.stdout:1/36: dwrite f2 [0,4194304] 0 2024-04-22T23:34:59.974 INFO:tasks.workunit.client.0.smithi107.stdout:7/33: dwrite f2 [0,4194304] 0 2024-04-22T23:34:59.974 INFO:tasks.workunit.client.0.smithi107.stdout:5/75: write d9/db/f10 [3525268,9632] 0 2024-04-22T23:34:59.975 INFO:tasks.workunit.client.0.smithi107.stdout:3/74: mknod db/c17 0 2024-04-22T23:34:59.975 INFO:tasks.workunit.client.0.smithi107.stdout:3/75: creat db/f18 x:0 0 0 2024-04-22T23:34:59.978 INFO:tasks.workunit.client.0.smithi107.stdout:7/34: symlink d7/l9 0 2024-04-22T23:34:59.982 INFO:tasks.workunit.client.0.smithi107.stdout:6/32: dwrite f3 [0,4194304] 0 2024-04-22T23:34:59.982 INFO:tasks.workunit.client.0.smithi107.stdout:6/33: write f4 [330710,71312] 0 2024-04-22T23:34:59.983 INFO:tasks.workunit.client.0.smithi107.stdout:3/76: mknod db/dd/c19 0 2024-04-22T23:34:59.983 INFO:tasks.workunit.client.0.smithi107.stdout:3/77: readlink l4 0 2024-04-22T23:34:59.987 INFO:tasks.workunit.client.0.smithi107.stdout:5/76: read f8 [4436527,37474] 0 2024-04-22T23:34:59.987 INFO:tasks.workunit.client.0.smithi107.stdout:3/78: mkdir db/dd/d15/d1a 0 2024-04-22T23:34:59.989 INFO:tasks.workunit.client.0.smithi107.stdout:9/25: dwrite f4 [0,4194304] 0 2024-04-22T23:34:59.993 INFO:tasks.workunit.client.0.smithi107.stdout:9/26: mkdir d5 0 2024-04-22T23:34:59.993 INFO:tasks.workunit.client.0.smithi107.stdout:9/27: dread - f3 zero size 2024-04-22T23:34:59.996 INFO:tasks.workunit.client.0.smithi107.stdout:9/28: symlink d5/l6 0 2024-04-22T23:34:59.996 INFO:tasks.workunit.client.0.smithi107.stdout:9/29: chown d5 793390656 1 2024-04-22T23:34:59.997 INFO:tasks.workunit.client.0.smithi107.stdout:6/34: write f0 [2561767,82502] 0 2024-04-22T23:34:59.997 INFO:tasks.workunit.client.0.smithi107.stdout:6/35: write f0 [5167082,46543] 0 2024-04-22T23:34:59.997 INFO:tasks.workunit.client.0.smithi107.stdout:6/36: chown l5 246485980 1 2024-04-22T23:35:00.000 INFO:tasks.workunit.client.0.smithi107.stdout:3/79: dread f8 [0,4194304] 0 2024-04-22T23:35:00.000 INFO:tasks.workunit.client.0.smithi107.stdout:3/80: chown db/dd/c19 39943 1 2024-04-22T23:35:00.003 INFO:tasks.workunit.client.0.smithi107.stdout:3/81: rename db/dd/c19 to db/c1b 0 2024-04-22T23:35:00.003 INFO:tasks.workunit.client.0.smithi107.stdout:3/82: chown db/f18 3 1 2024-04-22T23:35:00.003 INFO:tasks.workunit.client.0.smithi107.stdout:3/83: creat db/f1c x:0 0 0 2024-04-22T23:35:00.003 INFO:tasks.workunit.client.0.smithi107.stdout:3/84: readlink db/dd/d15/l14 0 2024-04-22T23:35:00.003 INFO:tasks.workunit.client.0.smithi107.stdout:3/85: write db/ff [1090475,96660] 0 2024-04-22T23:35:00.004 INFO:tasks.workunit.client.0.smithi107.stdout:2/30: sync 2024-04-22T23:35:00.005 INFO:tasks.workunit.client.0.smithi107.stdout:3/86: mknod db/dd/d15/c1d 0 2024-04-22T23:35:00.006 INFO:tasks.workunit.client.0.smithi107.stdout:1/37: dwrite d7/f9 [0,4194304] 0 2024-04-22T23:35:00.006 INFO:tasks.workunit.client.0.smithi107.stdout:2/31: symlink d1/ld 0 2024-04-22T23:35:00.007 INFO:tasks.workunit.client.0.smithi107.stdout:3/87: symlink db/dd/d15/d1a/l1e 0 2024-04-22T23:35:00.010 INFO:tasks.workunit.client.0.smithi107.stdout:3/88: symlink db/dd/d15/l1f 0 2024-04-22T23:35:00.010 INFO:tasks.workunit.client.0.smithi107.stdout:9/30: dread f4 [0,4194304] 0 2024-04-22T23:35:00.010 INFO:tasks.workunit.client.0.smithi107.stdout:1/38: mknod d7/cc 0 2024-04-22T23:35:00.011 INFO:tasks.workunit.client.0.smithi107.stdout:3/89: creat db/dd/f20 x:0 0 0 2024-04-22T23:35:00.013 INFO:tasks.workunit.client.0.smithi107.stdout:3/90: mkdir db/d21 0 2024-04-22T23:35:00.014 INFO:tasks.workunit.client.0.smithi107.stdout:5/77: dwrite d9/db/f10 [0,4194304] 0 2024-04-22T23:35:00.015 INFO:tasks.workunit.client.0.smithi107.stdout:1/39: rename c4 to d7/cd 0 2024-04-22T23:35:00.015 INFO:tasks.workunit.client.0.smithi107.stdout:9/31: write f4 [100545,14415] 0 2024-04-22T23:35:00.015 INFO:tasks.workunit.client.0.smithi107.stdout:9/32: read - f3 zero size 2024-04-22T23:35:00.017 INFO:tasks.workunit.client.0.smithi107.stdout:1/40: unlink c5 0 2024-04-22T23:35:00.018 INFO:tasks.workunit.client.0.smithi107.stdout:9/33: mkdir d5/d7 0 2024-04-22T23:35:00.019 INFO:tasks.workunit.client.0.smithi107.stdout:5/78: dread f4 [4194304,4194304] 0 2024-04-22T23:35:00.019 INFO:tasks.workunit.client.0.smithi107.stdout:5/79: readlink d9/db/de/l19 0 2024-04-22T23:35:00.019 INFO:tasks.workunit.client.0.smithi107.stdout:5/80: write d9/db/f1d [4349965,18801] 0 2024-04-22T23:35:00.019 INFO:tasks.workunit.client.0.smithi107.stdout:9/34: link f3 d5/d7/f8 0 2024-04-22T23:35:00.020 INFO:tasks.workunit.client.0.smithi107.stdout:9/35: unlink f3 0 2024-04-22T23:35:00.030 INFO:tasks.workunit.client.0.smithi107.stdout:6/37: dwrite f3 [0,4194304] 0 2024-04-22T23:35:00.031 INFO:tasks.workunit.client.0.smithi107.stdout:6/38: creat f6 x:0 0 0 2024-04-22T23:35:00.031 INFO:tasks.workunit.client.0.smithi107.stdout:2/32: dwrite d1/f4 [0,4194304] 0 2024-04-22T23:35:00.031 INFO:tasks.workunit.client.0.smithi107.stdout:2/33: write d1/f8 [51996,53068] 0 2024-04-22T23:35:00.031 INFO:tasks.workunit.client.0.smithi107.stdout:6/39: mknod c7 0 2024-04-22T23:35:00.032 INFO:tasks.workunit.client.0.smithi107.stdout:2/34: symlink d1/da/le 0 2024-04-22T23:35:00.033 INFO:tasks.workunit.client.0.smithi107.stdout:6/40: symlink l8 0 2024-04-22T23:35:00.035 INFO:tasks.workunit.client.0.smithi107.stdout:2/35: rename d1/ld to d1/lf 0 2024-04-22T23:35:00.036 INFO:tasks.workunit.client.0.smithi107.stdout:6/41: symlink l9 0 2024-04-22T23:35:00.036 INFO:tasks.workunit.client.0.smithi107.stdout:6/42: stat f6 0 2024-04-22T23:35:00.037 INFO:tasks.workunit.client.0.smithi107.stdout:2/36: creat d1/da/f10 x:0 0 0 2024-04-22T23:35:00.038 INFO:tasks.workunit.client.0.smithi107.stdout:6/43: mknod ca 0 2024-04-22T23:35:00.039 INFO:tasks.workunit.client.0.smithi107.stdout:9/36: dread f4 [0,4194304] 0 2024-04-22T23:35:00.039 INFO:tasks.workunit.client.0.smithi107.stdout:9/37: stat d5/d7 0 2024-04-22T23:35:00.039 INFO:tasks.workunit.client.0.smithi107.stdout:2/37: link d1/fb d1/f11 0 2024-04-22T23:35:00.041 INFO:tasks.workunit.client.0.smithi107.stdout:9/38: creat d5/f9 x:0 0 0 2024-04-22T23:35:00.042 INFO:tasks.workunit.client.0.smithi107.stdout:1/41: dwrite f3 [0,4194304] 0 2024-04-22T23:35:00.042 INFO:tasks.workunit.client.0.smithi107.stdout:2/38: mkdir d1/d12 0 2024-04-22T23:35:00.042 INFO:tasks.workunit.client.0.smithi107.stdout:9/39: truncate d5/d7/f8 964122 0 2024-04-22T23:35:00.043 INFO:tasks.workunit.client.0.smithi107.stdout:1/42: unlink d7/cc 0 2024-04-22T23:35:00.044 INFO:tasks.workunit.client.0.smithi107.stdout:2/39: mkdir d1/da/d13 0 2024-04-22T23:35:00.045 INFO:tasks.workunit.client.0.smithi107.stdout:9/40: symlink d5/la 0 2024-04-22T23:35:00.045 INFO:tasks.workunit.client.0.smithi107.stdout:9/41: write d5/f9 [136528,108502] 0 2024-04-22T23:35:00.046 INFO:tasks.workunit.client.0.smithi107.stdout:5/81: dwrite f3 [0,4194304] 0 2024-04-22T23:35:00.046 INFO:tasks.workunit.client.0.smithi107.stdout:6/44: truncate f4 74642 0 2024-04-22T23:35:00.046 INFO:tasks.workunit.client.0.smithi107.stdout:1/43: unlink f3 0 2024-04-22T23:35:00.050 INFO:tasks.workunit.client.0.smithi107.stdout:9/42: creat d5/d7/fb x:0 0 0 2024-04-22T23:35:00.050 INFO:tasks.workunit.client.0.smithi107.stdout:9/43: creat d5/fc x:0 0 0 2024-04-22T23:35:00.050 INFO:tasks.workunit.client.0.smithi107.stdout:9/44: creat d5/d7/fd x:0 0 0 2024-04-22T23:35:00.050 INFO:tasks.workunit.client.0.smithi107.stdout:9/45: write d5/f9 [650005,101626] 0 2024-04-22T23:35:00.051 INFO:tasks.workunit.client.0.smithi107.stdout:3/91: dwrite db/ff [0,4194304] 0 2024-04-22T23:35:00.055 INFO:tasks.workunit.client.0.smithi107.stdout:2/40: write d1/f4 [100013,62754] 0 2024-04-22T23:35:00.055 INFO:tasks.workunit.client.0.smithi107.stdout:2/41: chown c0 90 1 2024-04-22T23:35:00.058 INFO:tasks.workunit.client.0.smithi107.stdout:5/82: dread d9/db/f10 [0,4194304] 0 2024-04-22T23:35:00.059 INFO:tasks.workunit.client.0.smithi107.stdout:6/45: mkdir db 0 2024-04-22T23:35:00.060 INFO:tasks.workunit.client.0.smithi107.stdout:1/44: dread d7/f9 [0,4194304] 0 2024-04-22T23:35:00.060 INFO:tasks.workunit.client.0.smithi107.stdout:1/45: fdatasync f2 0 2024-04-22T23:35:00.063 INFO:tasks.workunit.client.0.smithi107.stdout:9/46: mknod d5/ce 0 2024-04-22T23:35:00.066 INFO:tasks.workunit.client.0.smithi107.stdout:3/92: mkdir db/dd/d22 0 2024-04-22T23:35:00.066 INFO:tasks.workunit.client.0.smithi107.stdout:3/93: chown db/dd/ce 18 1 2024-04-22T23:35:00.066 INFO:tasks.workunit.client.0.smithi107.stdout:3/94: creat db/dd/f23 x:0 0 0 2024-04-22T23:35:00.066 INFO:tasks.workunit.client.0.smithi107.stdout:3/95: dread - db/dd/f23 zero size 2024-04-22T23:35:00.067 INFO:tasks.workunit.client.0.smithi107.stdout:2/42: creat d1/d12/f14 x:0 0 0 2024-04-22T23:35:00.069 INFO:tasks.workunit.client.0.smithi107.stdout:5/83: rename c2 to d9/db/de/c1e 0 2024-04-22T23:35:00.071 INFO:tasks.workunit.client.0.smithi107.stdout:6/46: mknod db/cc 0 2024-04-22T23:35:00.073 INFO:tasks.workunit.client.0.smithi107.stdout:2/43: dread d1/f8 [0,4194304] 0 2024-04-22T23:35:00.077 INFO:tasks.workunit.client.0.smithi107.stdout:3/96: mkdir db/dd/d15/d24 0 2024-04-22T23:35:00.080 INFO:tasks.workunit.client.0.smithi107.stdout:6/47: rmdir db 39 2024-04-22T23:35:00.080 INFO:tasks.workunit.client.0.smithi107.stdout:6/48: stat db/cc 0 2024-04-22T23:35:00.080 INFO:tasks.workunit.client.0.smithi107.stdout:1/46: getdents d7 0 2024-04-22T23:35:00.080 INFO:tasks.workunit.client.0.smithi107.stdout:1/47: chown l6 1066846775 1 2024-04-22T23:35:00.082 INFO:tasks.workunit.client.0.smithi107.stdout:2/44: link d1/l9 d1/d12/l15 0 2024-04-22T23:35:00.082 INFO:tasks.workunit.client.0.smithi107.stdout:2/45: truncate d1/da/f10 665670 0 2024-04-22T23:35:00.082 INFO:tasks.workunit.client.0.smithi107.stdout:2/46: write d1/f8 [456431,125418] 0 2024-04-22T23:35:00.082 INFO:tasks.workunit.client.0.smithi107.stdout:2/47: read - d1/d12/f14 zero size 2024-04-22T23:35:00.082 INFO:tasks.workunit.client.0.smithi107.stdout:3/97: mkdir db/dd/d22/d25 0 2024-04-22T23:35:00.085 INFO:tasks.workunit.client.0.smithi107.stdout:3/98: symlink db/dd/d15/d24/l26 0 2024-04-22T23:35:00.085 INFO:tasks.workunit.client.0.smithi107.stdout:1/48: mknod d7/ce 0 2024-04-22T23:35:00.088 INFO:tasks.workunit.client.0.smithi107.stdout:0/33: sync 2024-04-22T23:35:00.088 INFO:tasks.workunit.client.0.smithi107.stdout:6/49: write f3 [206256,44298] 0 2024-04-22T23:35:00.095 INFO:tasks.workunit.client.0.smithi107.stdout:6/50: rmdir db 39 2024-04-22T23:35:00.096 INFO:tasks.workunit.client.0.smithi107.stdout:4/38: sync 2024-04-22T23:35:00.096 INFO:tasks.workunit.client.0.smithi107.stdout:8/13: sync 2024-04-22T23:35:00.096 INFO:tasks.workunit.client.0.smithi107.stdout:0/34: getdents d1/d9 0 2024-04-22T23:35:00.097 INFO:tasks.workunit.client.0.smithi107.stdout:0/35: chown d1/d9/fc 230110 1 2024-04-22T23:35:00.099 INFO:tasks.workunit.client.0.smithi107.stdout:4/39: rename d1/f10 to d1/f13 0 2024-04-22T23:35:00.099 INFO:tasks.workunit.client.0.smithi107.stdout:8/14: creat d0/f4 x:0 0 0 2024-04-22T23:35:00.100 INFO:tasks.workunit.client.0.smithi107.stdout:0/36: symlink d1/d9/le 0 2024-04-22T23:35:00.101 INFO:tasks.workunit.client.0.smithi107.stdout:4/40: symlink d1/l14 0 2024-04-22T23:35:00.101 INFO:tasks.workunit.client.0.smithi107.stdout:4/41: dread - d1/d7/fc zero size 2024-04-22T23:35:00.101 INFO:tasks.workunit.client.0.smithi107.stdout:4/42: truncate d1/f5 821514 0 2024-04-22T23:35:00.102 INFO:tasks.workunit.client.0.smithi107.stdout:1/49: getdents d7 0 2024-04-22T23:35:00.103 INFO:tasks.workunit.client.0.smithi107.stdout:0/37: rmdir d1 39 2024-04-22T23:35:00.111 INFO:tasks.workunit.client.0.smithi107.stdout:3/99: rename db/dd to db/d21/d27 0 2024-04-22T23:35:00.112 INFO:tasks.workunit.client.0.smithi107.stdout:1/50: creat d7/ff x:0 0 0 2024-04-22T23:35:00.114 INFO:tasks.workunit.client.0.smithi107.stdout:8/15: getdents d0 0 2024-04-22T23:35:00.114 INFO:tasks.workunit.client.0.smithi107.stdout:8/16: write d0/f4 [254319,81030] 0 2024-04-22T23:35:00.118 INFO:tasks.workunit.client.0.smithi107.stdout:3/100: mknod db/d21/d27/d22/d25/c28 0 2024-04-22T23:35:00.118 INFO:tasks.workunit.client.0.smithi107.stdout:3/101: write db/d21/d27/f23 [990101,30917] 0 2024-04-22T23:35:00.119 INFO:tasks.workunit.client.0.smithi107.stdout:3/102: chown db/d21/d27/d22/d25/c28 71359324 1 2024-04-22T23:35:00.119 INFO:tasks.workunit.client.0.smithi107.stdout:3/103: fsync db/d21/d27/f20 0 2024-04-22T23:35:00.119 INFO:tasks.workunit.client.0.smithi107.stdout:3/104: dread - db/f1c zero size 2024-04-22T23:35:00.119 INFO:tasks.workunit.client.0.smithi107.stdout:1/51: readlink d7/l8 0 2024-04-22T23:35:00.119 INFO:tasks.workunit.client.0.smithi107.stdout:1/52: chown d7/cd 17 1 2024-04-22T23:35:00.121 INFO:tasks.workunit.client.0.smithi107.stdout:1/53: link d7/f9 d7/f10 0 2024-04-22T23:35:00.122 INFO:tasks.workunit.client.0.smithi107.stdout:3/105: dread f8 [0,4194304] 0 2024-04-22T23:35:00.123 INFO:tasks.workunit.client.0.smithi107.stdout:1/54: rename d7/ce to d7/c11 0 2024-04-22T23:35:00.124 INFO:tasks.workunit.client.0.smithi107.stdout:3/106: getdents db/d21/d27/d15 0 2024-04-22T23:35:00.125 INFO:tasks.workunit.client.0.smithi107.stdout:0/38: dwrite d1/f2 [0,4194304] 0 2024-04-22T23:35:00.125 INFO:tasks.workunit.client.0.smithi107.stdout:3/107: mkdir db/d21/d27/d15/d29 0 2024-04-22T23:35:00.127 INFO:tasks.workunit.client.0.smithi107.stdout:0/39: truncate d1/d9/fc 360747 0 2024-04-22T23:35:00.129 INFO:tasks.workunit.client.0.smithi107.stdout:1/55: rename d7/ca to d7/c12 0 2024-04-22T23:35:00.130 INFO:tasks.workunit.client.0.smithi107.stdout:0/40: rename d1/c7 to d1/cf 0 2024-04-22T23:35:00.131 INFO:tasks.workunit.client.0.smithi107.stdout:6/51: read f0 [4163611,54475] 0 2024-04-22T23:35:00.133 INFO:tasks.workunit.client.0.smithi107.stdout:3/108: rename db/d21/d27/d15/d1a to db/d21/d27/d15/d29/d2a 0 2024-04-22T23:35:00.133 INFO:tasks.workunit.client.0.smithi107.stdout:2/48: dwrite d1/f8 [0,4194304] 0 2024-04-22T23:35:00.133 INFO:tasks.workunit.client.0.smithi107.stdout:6/52: dread f4 [0,4194304] 0 2024-04-22T23:35:00.133 INFO:tasks.workunit.client.0.smithi107.stdout:1/56: stat d7/cb 0 2024-04-22T23:35:00.136 INFO:tasks.workunit.client.0.smithi107.stdout:4/43: dwrite d1/f5 [0,4194304] 0 2024-04-22T23:35:00.137 INFO:tasks.workunit.client.0.smithi107.stdout:3/109: symlink db/d21/d27/d22/d25/l2b 0 2024-04-22T23:35:00.139 INFO:tasks.workunit.client.0.smithi107.stdout:2/49: link d1/da/le d1/da/l16 0 2024-04-22T23:35:00.140 INFO:tasks.workunit.client.0.smithi107.stdout:6/53: rename f3 to db/fd 0 2024-04-22T23:35:00.141 INFO:tasks.workunit.client.0.smithi107.stdout:1/57: creat d7/f13 x:0 0 0 2024-04-22T23:35:00.143 INFO:tasks.workunit.client.0.smithi107.stdout:2/50: read d1/da/f10 [59022,78142] 0 2024-04-22T23:35:00.143 INFO:tasks.workunit.client.0.smithi107.stdout:2/51: fdatasync d1/d12/f14 0 2024-04-22T23:35:00.143 INFO:tasks.workunit.client.0.smithi107.stdout:0/41: link d1/ca d1/d9/c10 0 2024-04-22T23:35:00.143 INFO:tasks.workunit.client.0.smithi107.stdout:9/47: dwrite d5/d7/f8 [0,4194304] 0 2024-04-22T23:35:00.151 INFO:tasks.workunit.client.0.smithi107.stdout:1/58: rename d7 to d7/d14 22 2024-04-22T23:35:00.152 INFO:tasks.workunit.client.0.smithi107.stdout:2/52: dread d1/f8 [0,4194304] 0 2024-04-22T23:35:00.154 INFO:tasks.workunit.client.0.smithi107.stdout:4/44: dread d1/f5 [0,4194304] 0 2024-04-22T23:35:00.154 INFO:tasks.workunit.client.0.smithi107.stdout:4/45: fdatasync d1/d8/fe 0 2024-04-22T23:35:00.154 INFO:tasks.workunit.client.0.smithi107.stdout:4/46: fdatasync d1/f13 0 2024-04-22T23:35:00.156 INFO:tasks.workunit.client.0.smithi107.stdout:1/59: creat d7/f15 x:0 0 0 2024-04-22T23:35:00.159 INFO:tasks.workunit.client.0.smithi107.stdout:9/48: rename d5/d7/f8 to d5/d7/ff 0 2024-04-22T23:35:00.162 INFO:tasks.workunit.client.0.smithi107.stdout:7/35: sync 2024-04-22T23:35:00.164 INFO:tasks.workunit.client.0.smithi107.stdout:7/36: creat d7/fa x:0 0 0 2024-04-22T23:35:00.166 INFO:tasks.workunit.client.0.smithi107.stdout:7/37: mknod d7/cb 0 2024-04-22T23:35:00.166 INFO:tasks.workunit.client.0.smithi107.stdout:4/47: write d1/f3 [1616652,47364] 0 2024-04-22T23:35:00.166 INFO:tasks.workunit.client.0.smithi107.stdout:4/48: read d1/f3 [2560100,714] 0 2024-04-22T23:35:00.166 INFO:tasks.workunit.client.0.smithi107.stdout:4/49: write d1/d7/fc [782886,113805] 0 2024-04-22T23:35:00.166 INFO:tasks.workunit.client.0.smithi107.stdout:4/50: write d1/f3 [4547055,53894] 0 2024-04-22T23:35:00.168 INFO:tasks.workunit.client.0.smithi107.stdout:4/51: symlink d1/l15 0 2024-04-22T23:35:00.169 INFO:tasks.workunit.client.0.smithi107.stdout:4/52: rename d1/d7/l12 to d1/d8/l16 0 2024-04-22T23:35:00.170 INFO:tasks.workunit.client.0.smithi107.stdout:4/53: rename d1/d7/lf to d1/d7/l17 0 2024-04-22T23:35:00.170 INFO:tasks.workunit.client.0.smithi107.stdout:9/49: write f1 [3312205,15564] 0 2024-04-22T23:35:00.171 INFO:tasks.workunit.client.0.smithi107.stdout:4/54: symlink d1/d7/l18 0 2024-04-22T23:35:00.172 INFO:tasks.workunit.client.0.smithi107.stdout:9/50: rename d5/f9 to d5/f10 0 2024-04-22T23:35:00.172 INFO:tasks.workunit.client.0.smithi107.stdout:9/51: creat d5/f11 x:0 0 0 2024-04-22T23:35:00.172 INFO:tasks.workunit.client.0.smithi107.stdout:9/52: write d5/f11 [897805,79542] 0 2024-04-22T23:35:00.174 INFO:tasks.workunit.client.0.smithi107.stdout:9/53: mknod d5/d7/c12 0 2024-04-22T23:35:00.174 INFO:tasks.workunit.client.0.smithi107.stdout:9/54: creat d5/f13 x:0 0 0 2024-04-22T23:35:00.175 INFO:tasks.workunit.client.0.smithi107.stdout:9/55: mkdir d5/d14 0 2024-04-22T23:35:00.179 INFO:tasks.workunit.client.0.smithi107.stdout:2/53: dwrite d1/f8 [0,4194304] 0 2024-04-22T23:35:00.181 INFO:tasks.workunit.client.0.smithi107.stdout:3/110: dwrite db/d21/d27/f20 [0,4194304] 0 2024-04-22T23:35:00.181 INFO:tasks.workunit.client.0.smithi107.stdout:6/54: dwrite db/fd [0,4194304] 0 2024-04-22T23:35:00.183 INFO:tasks.workunit.client.0.smithi107.stdout:3/111: symlink db/d21/d27/l2c 0 2024-04-22T23:35:00.185 INFO:tasks.workunit.client.0.smithi107.stdout:6/55: symlink db/le 0 2024-04-22T23:35:00.186 INFO:tasks.workunit.client.0.smithi107.stdout:3/112: symlink db/d21/d27/d15/d29/l2d 0 2024-04-22T23:35:00.188 INFO:tasks.workunit.client.0.smithi107.stdout:7/38: getdents d7 0 2024-04-22T23:35:00.190 INFO:tasks.workunit.client.0.smithi107.stdout:6/56: creat db/ff x:0 0 0 2024-04-22T23:35:00.190 INFO:tasks.workunit.client.0.smithi107.stdout:6/57: chown db/ff 1831622601 1 2024-04-22T23:35:00.190 INFO:tasks.workunit.client.0.smithi107.stdout:6/58: readlink l9 0 2024-04-22T23:35:00.192 INFO:tasks.workunit.client.0.smithi107.stdout:7/39: rename d7/cb to d7/cc 0 2024-04-22T23:35:00.192 INFO:tasks.workunit.client.0.smithi107.stdout:7/40: write f2 [4286210,91284] 0 2024-04-22T23:35:00.194 INFO:tasks.workunit.client.0.smithi107.stdout:9/56: dwrite d5/d7/ff [0,4194304] 0 2024-04-22T23:35:00.194 INFO:tasks.workunit.client.0.smithi107.stdout:9/57: stat d5/f11 0 2024-04-22T23:35:00.194 INFO:tasks.workunit.client.0.smithi107.stdout:7/41: chown l1 5216253 1 2024-04-22T23:35:00.207 INFO:tasks.workunit.client.0.smithi107.stdout:7/42: write f2 [3572891,59328] 0 2024-04-22T23:35:00.210 INFO:tasks.workunit.client.0.smithi107.stdout:7/43: rename d7/fa to d7/fd 0 2024-04-22T23:35:00.221 INFO:tasks.workunit.client.0.smithi107.stdout:3/113: dwrite db/ff [0,4194304] 0 2024-04-22T23:35:00.221 INFO:tasks.workunit.client.0.smithi107.stdout:3/114: chown db/d21/d27/d22 170210 1 2024-04-22T23:35:00.221 INFO:tasks.workunit.client.0.smithi107.stdout:2/54: dwrite d1/f11 [0,4194304] 0 2024-04-22T23:35:00.221 INFO:tasks.workunit.client.0.smithi107.stdout:2/55: chown d1/da/f10 37872 1 2024-04-22T23:35:00.222 INFO:tasks.workunit.client.0.smithi107.stdout:3/115: rmdir db/d21/d27/d15 39 2024-04-22T23:35:00.223 INFO:tasks.workunit.client.0.smithi107.stdout:6/59: dwrite f6 [0,4194304] 0 2024-04-22T23:35:00.223 INFO:tasks.workunit.client.0.smithi107.stdout:4/55: dwrite d1/d7/fc [0,4194304] 0 2024-04-22T23:35:00.223 INFO:tasks.workunit.client.0.smithi107.stdout:3/116: mknod db/d21/d27/d22/d25/c2e 0 2024-04-22T23:35:00.223 INFO:tasks.workunit.client.0.smithi107.stdout:3/117: truncate db/f18 629168 0 2024-04-22T23:35:00.224 INFO:tasks.workunit.client.0.smithi107.stdout:3/118: stat db/d21/d27/d15/d29/d2a 0 2024-04-22T23:35:00.224 INFO:tasks.workunit.client.0.smithi107.stdout:9/58: dwrite f4 [0,4194304] 0 2024-04-22T23:35:00.225 INFO:tasks.workunit.client.0.smithi107.stdout:6/60: unlink f4 0 2024-04-22T23:35:00.226 INFO:tasks.workunit.client.0.smithi107.stdout:2/56: dread d1/f4 [0,4194304] 0 2024-04-22T23:35:00.226 INFO:tasks.workunit.client.0.smithi107.stdout:2/57: write d1/f6 [335466,109656] 0 2024-04-22T23:35:00.227 INFO:tasks.workunit.client.0.smithi107.stdout:4/56: rename d1/d8/fe to d1/d8/d9/f19 0 2024-04-22T23:35:00.229 INFO:tasks.workunit.client.0.smithi107.stdout:3/119: symlink db/d21/d27/d22/d25/l2f 0 2024-04-22T23:35:00.229 INFO:tasks.workunit.client.0.smithi107.stdout:3/120: fsync db/f18 0 2024-04-22T23:35:00.234 INFO:tasks.workunit.client.0.smithi107.stdout:9/59: creat d5/d14/f15 x:0 0 0 2024-04-22T23:35:00.234 INFO:tasks.workunit.client.0.smithi107.stdout:9/60: write d5/f10 [942106,8013] 0 2024-04-22T23:35:00.235 INFO:tasks.workunit.client.0.smithi107.stdout:7/44: fsync d7/fd 0 2024-04-22T23:35:00.235 INFO:tasks.workunit.client.0.smithi107.stdout:6/61: read db/fd [1901948,51758] 0 2024-04-22T23:35:00.241 INFO:tasks.workunit.client.0.smithi107.stdout:4/57: dwrite d1/f13 [0,4194304] 0 2024-04-22T23:35:00.241 INFO:tasks.workunit.client.0.smithi107.stdout:2/58: rename d1/da/le to d1/l17 0 2024-04-22T23:35:00.241 INFO:tasks.workunit.client.0.smithi107.stdout:6/62: dread db/fd [0,4194304] 0 2024-04-22T23:35:00.241 INFO:tasks.workunit.client.0.smithi107.stdout:6/63: truncate db/ff 423437 0 2024-04-22T23:35:00.242 INFO:tasks.workunit.client.0.smithi107.stdout:6/64: dread db/ff [0,4194304] 0 2024-04-22T23:35:00.242 INFO:tasks.workunit.client.0.smithi107.stdout:7/45: dread f2 [0,4194304] 0 2024-04-22T23:35:00.243 INFO:tasks.workunit.client.0.smithi107.stdout:9/61: creat d5/d7/f16 x:0 0 0 2024-04-22T23:35:00.243 INFO:tasks.workunit.client.0.smithi107.stdout:9/62: readlink d5/la 0 2024-04-22T23:35:00.245 INFO:tasks.workunit.client.0.smithi107.stdout:4/58: mknod d1/d7/c1a 0 2024-04-22T23:35:00.246 INFO:tasks.workunit.client.0.smithi107.stdout:7/46: dread f2 [0,4194304] 0 2024-04-22T23:35:00.246 INFO:tasks.workunit.client.0.smithi107.stdout:2/59: mknod d1/d12/c18 0 2024-04-22T23:35:00.246 INFO:tasks.workunit.client.0.smithi107.stdout:2/60: creat d1/d12/f19 x:0 0 0 2024-04-22T23:35:00.246 INFO:tasks.workunit.client.0.smithi107.stdout:2/61: chown d1/f8 1226354 1 2024-04-22T23:35:00.252 INFO:tasks.workunit.client.0.smithi107.stdout:9/63: truncate f1 2333746 0 2024-04-22T23:35:00.253 INFO:tasks.workunit.client.0.smithi107.stdout:7/47: rename l1 to d7/le 0 2024-04-22T23:35:00.253 INFO:tasks.workunit.client.0.smithi107.stdout:7/48: write f2 [4682867,119268] 0 2024-04-22T23:35:00.254 INFO:tasks.workunit.client.0.smithi107.stdout:7/49: write f2 [3291401,9569] 0 2024-04-22T23:35:00.255 INFO:tasks.workunit.client.0.smithi107.stdout:9/64: truncate d5/f11 625001 0 2024-04-22T23:35:00.259 INFO:tasks.workunit.client.0.smithi107.stdout:2/62: rename d1/d12/f19 to d1/f1a 0 2024-04-22T23:35:00.262 INFO:tasks.workunit.client.0.smithi107.stdout:9/65: mkdir d5/d7/d17 0 2024-04-22T23:35:00.262 INFO:tasks.workunit.client.0.smithi107.stdout:9/66: fdatasync d5/d7/ff 0 2024-04-22T23:35:00.265 INFO:tasks.workunit.client.0.smithi107.stdout:9/67: creat d5/d7/d17/f18 x:0 0 0 2024-04-22T23:35:00.265 INFO:tasks.workunit.client.0.smithi107.stdout:9/68: readlink d5/l6 0 2024-04-22T23:35:00.265 INFO:tasks.workunit.client.0.smithi107.stdout:9/69: chown d5/d7/d17/f18 114 1 2024-04-22T23:35:00.265 INFO:tasks.workunit.client.0.smithi107.stdout:9/70: rename d5/d7 to d5/d7/d17/d19 22 2024-04-22T23:35:00.265 INFO:tasks.workunit.client.0.smithi107.stdout:9/71: dread - d5/fc zero size 2024-04-22T23:35:00.265 INFO:tasks.workunit.client.0.smithi107.stdout:9/72: write d5/d7/f16 [534223,102826] 0 2024-04-22T23:35:00.266 INFO:tasks.workunit.client.0.smithi107.stdout:5/84: sync 2024-04-22T23:35:00.266 INFO:tasks.workunit.client.0.smithi107.stdout:8/17: sync 2024-04-22T23:35:00.266 INFO:tasks.workunit.client.0.smithi107.stdout:8/18: readlink - no filename 2024-04-22T23:35:00.266 INFO:tasks.workunit.client.0.smithi107.stdout:7/50: dread f2 [0,4194304] 0 2024-04-22T23:35:00.266 INFO:tasks.workunit.client.0.smithi107.stdout:9/73: fsync d5/d7/fb 0 2024-04-22T23:35:00.268 INFO:tasks.workunit.client.0.smithi107.stdout:8/19: creat d0/f5 x:0 0 0 2024-04-22T23:35:00.268 INFO:tasks.workunit.client.0.smithi107.stdout:2/63: write d1/f8 [829221,56517] 0 2024-04-22T23:35:00.269 INFO:tasks.workunit.client.0.smithi107.stdout:8/20: mkdir d0/d6 0 2024-04-22T23:35:00.269 INFO:tasks.workunit.client.0.smithi107.stdout:8/21: write d0/f5 [400788,34574] 0 2024-04-22T23:35:00.271 INFO:tasks.workunit.client.0.smithi107.stdout:8/22: mkdir d0/d7 0 2024-04-22T23:35:00.271 INFO:tasks.workunit.client.0.smithi107.stdout:8/23: write d0/f4 [1337114,126495] 0 2024-04-22T23:35:00.272 INFO:tasks.workunit.client.0.smithi107.stdout:8/24: mkdir d0/d7/d8 0 2024-04-22T23:35:00.275 INFO:tasks.workunit.client.0.smithi107.stdout:6/65: dwrite f6 [4194304,4194304] 0 2024-04-22T23:35:00.276 INFO:tasks.workunit.client.0.smithi107.stdout:6/66: truncate db/ff 792710 0 2024-04-22T23:35:00.276 INFO:tasks.workunit.client.0.smithi107.stdout:4/59: dwrite d1/d8/d9/f19 [0,4194304] 0 2024-04-22T23:35:00.278 INFO:tasks.workunit.client.0.smithi107.stdout:9/74: read f4 [1574612,123570] 0 2024-04-22T23:35:00.279 INFO:tasks.workunit.client.0.smithi107.stdout:5/85: dread d9/f12 [0,4194304] 0 2024-04-22T23:35:00.279 INFO:tasks.workunit.client.0.smithi107.stdout:5/86: chown d9/db/de/c1e 292818 1 2024-04-22T23:35:00.280 INFO:tasks.workunit.client.0.smithi107.stdout:6/67: rename db/cc to db/c10 0 2024-04-22T23:35:00.282 INFO:tasks.workunit.client.0.smithi107.stdout:8/25: dread d0/f5 [0,4194304] 0 2024-04-22T23:35:00.282 INFO:tasks.workunit.client.0.smithi107.stdout:9/75: mknod d5/c1a 0 2024-04-22T23:35:00.286 INFO:tasks.workunit.client.0.smithi107.stdout:4/60: write d1/f5 [3418089,41384] 0 2024-04-22T23:35:00.286 INFO:tasks.workunit.client.0.smithi107.stdout:4/61: chown d1/d7/l18 8864428 1 2024-04-22T23:35:00.290 INFO:tasks.workunit.client.0.smithi107.stdout:6/68: dread f0 [0,4194304] 0 2024-04-22T23:35:00.292 INFO:tasks.workunit.client.0.smithi107.stdout:5/87: read d9/f12 [3979269,70775] 0 2024-04-22T23:35:00.293 INFO:tasks.workunit.client.0.smithi107.stdout:2/64: dwrite d1/f8 [0,4194304] 0 2024-04-22T23:35:00.296 INFO:tasks.workunit.client.0.smithi107.stdout:2/65: symlink d1/d12/l1b 0 2024-04-22T23:35:00.299 INFO:tasks.workunit.client.0.smithi107.stdout:2/66: dread d1/f4 [0,4194304] 0 2024-04-22T23:35:00.301 INFO:tasks.workunit.client.0.smithi107.stdout:6/69: dread db/fd [0,4194304] 0 2024-04-22T23:35:00.301 INFO:tasks.workunit.client.0.smithi107.stdout:7/51: dwrite d7/fd [0,4194304] 0 2024-04-22T23:35:00.301 INFO:tasks.workunit.client.0.smithi107.stdout:7/52: fdatasync d7/fd 0 2024-04-22T23:35:00.301 INFO:tasks.workunit.client.0.smithi107.stdout:7/53: read f2 [3027549,98838] 0 2024-04-22T23:35:00.301 INFO:tasks.workunit.client.0.smithi107.stdout:7/54: stat d7 0 2024-04-22T23:35:00.313 INFO:tasks.workunit.client.0.smithi107.stdout:4/62: write d1/d7/fc [192917,58310] 0 2024-04-22T23:35:00.319 INFO:tasks.workunit.client.0.smithi107.stdout:2/67: write d1/f11 [2335145,55418] 0 2024-04-22T23:35:00.319 INFO:tasks.workunit.client.0.smithi107.stdout:8/26: dwrite d0/f5 [0,4194304] 0 2024-04-22T23:35:00.319 INFO:tasks.workunit.client.0.smithi107.stdout:8/27: chown d0 913687 1 2024-04-22T23:35:00.323 INFO:tasks.workunit.client.0.smithi107.stdout:2/68: unlink d1/da/lc 0 2024-04-22T23:35:00.324 INFO:tasks.workunit.client.0.smithi107.stdout:2/69: mknod d1/d12/c1c 0 2024-04-22T23:35:00.325 INFO:tasks.workunit.client.0.smithi107.stdout:2/70: dread d1/da/f10 [0,4194304] 0 2024-04-22T23:35:00.331 INFO:tasks.workunit.client.0.smithi107.stdout:9/76: dwrite d5/d7/d17/f18 [0,4194304] 0 2024-04-22T23:35:00.334 INFO:tasks.workunit.client.0.smithi107.stdout:9/77: mkdir d5/d7/d17/d1b 0 2024-04-22T23:35:00.335 INFO:tasks.workunit.client.0.smithi107.stdout:7/55: dwrite f2 [0,4194304] 0 2024-04-22T23:35:00.335 INFO:tasks.workunit.client.0.smithi107.stdout:5/88: dwrite f7 [0,4194304] 0 2024-04-22T23:35:00.335 INFO:tasks.workunit.client.0.smithi107.stdout:2/71: fsync d1/fb 0 2024-04-22T23:35:00.335 INFO:tasks.workunit.client.0.smithi107.stdout:2/72: write d1/f1a [992192,59888] 0 2024-04-22T23:35:00.337 INFO:tasks.workunit.client.0.smithi107.stdout:6/70: dwrite db/fd [0,4194304] 0 2024-04-22T23:35:00.337 INFO:tasks.workunit.client.0.smithi107.stdout:4/63: dwrite d1/d7/fd [0,4194304] 0 2024-04-22T23:35:00.337 INFO:tasks.workunit.client.0.smithi107.stdout:9/78: dread d5/f10 [0,4194304] 0 2024-04-22T23:35:00.337 INFO:tasks.workunit.client.0.smithi107.stdout:6/71: creat db/f11 x:0 0 0 2024-04-22T23:35:00.337 INFO:tasks.workunit.client.0.smithi107.stdout:6/72: creat db/f12 x:0 0 0 2024-04-22T23:35:00.343 INFO:tasks.workunit.client.0.smithi107.stdout:9/79: dread f4 [0,4194304] 0 2024-04-22T23:35:00.344 INFO:tasks.workunit.client.0.smithi107.stdout:6/73: dread f0 [0,4194304] 0 2024-04-22T23:35:00.346 INFO:tasks.workunit.client.0.smithi107.stdout:2/73: getdents d1/da 0 2024-04-22T23:35:00.346 INFO:tasks.workunit.client.0.smithi107.stdout:2/74: write d1/d12/f14 [830985,18947] 0 2024-04-22T23:35:00.347 INFO:tasks.workunit.client.0.smithi107.stdout:9/80: symlink d5/d7/d17/d1b/l1c 0 2024-04-22T23:35:00.347 INFO:tasks.workunit.client.0.smithi107.stdout:9/81: write d5/d7/fb [518520,117519] 0 2024-04-22T23:35:00.349 INFO:tasks.workunit.client.0.smithi107.stdout:2/75: rename d1/d12/c18 to d1/c1d 0 2024-04-22T23:35:00.349 INFO:tasks.workunit.client.0.smithi107.stdout:2/76: readlink d1/l17 0 2024-04-22T23:35:00.350 INFO:tasks.workunit.client.0.smithi107.stdout:2/77: stat d1/d12 0 2024-04-22T23:35:00.350 INFO:tasks.workunit.client.0.smithi107.stdout:2/78: readlink d1/lf 0 2024-04-22T23:35:00.350 INFO:tasks.workunit.client.0.smithi107.stdout:9/82: creat d5/d7/f1d x:0 0 0 2024-04-22T23:35:00.351 INFO:tasks.workunit.client.0.smithi107.stdout:2/79: mknod d1/da/c1e 0 2024-04-22T23:35:00.356 INFO:tasks.workunit.client.0.smithi107.stdout:6/74: read f6 [6664787,48326] 0 2024-04-22T23:35:00.363 INFO:tasks.workunit.client.0.smithi107.stdout:1/60: sync 2024-04-22T23:35:00.364 INFO:tasks.workunit.client.0.smithi107.stdout:3/121: sync 2024-04-22T23:35:00.364 INFO:tasks.workunit.client.0.smithi107.stdout:0/42: sync 2024-04-22T23:35:00.365 INFO:tasks.workunit.client.0.smithi107.stdout:3/122: getdents db/d21/d27/d15/d29/d2a 0 2024-04-22T23:35:00.365 INFO:tasks.workunit.client.0.smithi107.stdout:3/123: write f0 [864368,118415] 0 2024-04-22T23:35:00.366 INFO:tasks.workunit.client.0.smithi107.stdout:0/43: mkdir d1/d11 0 2024-04-22T23:35:00.368 INFO:tasks.workunit.client.0.smithi107.stdout:1/61: symlink d7/l16 0 2024-04-22T23:35:00.368 INFO:tasks.workunit.client.0.smithi107.stdout:1/62: readlink l6 0 2024-04-22T23:35:00.369 INFO:tasks.workunit.client.0.smithi107.stdout:7/56: dwrite f2 [0,4194304] 0 2024-04-22T23:35:00.369 INFO:tasks.workunit.client.0.smithi107.stdout:5/89: dwrite d9/f12 [4194304,4194304] 0 2024-04-22T23:35:00.369 INFO:tasks.workunit.client.0.smithi107.stdout:2/80: fdatasync d1/d12/f14 0 2024-04-22T23:35:00.370 INFO:tasks.workunit.client.0.smithi107.stdout:4/64: dwrite d1/f5 [0,4194304] 0 2024-04-22T23:35:00.373 INFO:tasks.workunit.client.0.smithi107.stdout:5/90: link f4 d9/db/de/f1f 0 2024-04-22T23:35:00.375 INFO:tasks.workunit.client.0.smithi107.stdout:2/81: creat d1/da/f1f x:0 0 0 2024-04-22T23:35:00.376 INFO:tasks.workunit.client.0.smithi107.stdout:9/83: dwrite d5/f13 [0,4194304] 0 2024-04-22T23:35:00.377 INFO:tasks.workunit.client.0.smithi107.stdout:9/84: creat d5/f1e x:0 0 0 2024-04-22T23:35:00.377 INFO:tasks.workunit.client.0.smithi107.stdout:1/63: write d7/f9 [456176,30396] 0 2024-04-22T23:35:00.377 INFO:tasks.workunit.client.0.smithi107.stdout:4/65: mkdir d1/d1b 0 2024-04-22T23:35:00.377 INFO:tasks.workunit.client.0.smithi107.stdout:5/91: mknod d9/c20 0 2024-04-22T23:35:00.377 INFO:tasks.workunit.client.0.smithi107.stdout:5/92: fdatasync d9/db/f1d 0 2024-04-22T23:35:00.378 INFO:tasks.workunit.client.0.smithi107.stdout:0/44: write d1/f2 [1408951,29811] 0 2024-04-22T23:35:00.380 INFO:tasks.workunit.client.0.smithi107.stdout:1/64: creat d7/f17 x:0 0 0 2024-04-22T23:35:00.381 INFO:tasks.workunit.client.0.smithi107.stdout:5/93: symlink d9/l21 0 2024-04-22T23:35:00.381 INFO:tasks.workunit.client.0.smithi107.stdout:5/94: creat d9/db/f22 x:0 0 0 2024-04-22T23:35:00.382 INFO:tasks.workunit.client.0.smithi107.stdout:2/82: fsync d1/f1a 0 2024-04-22T23:35:00.382 INFO:tasks.workunit.client.0.smithi107.stdout:0/45: fdatasync d1/f2 0 2024-04-22T23:35:00.382 INFO:tasks.workunit.client.0.smithi107.stdout:6/75: dwrite f0 [0,4194304] 0 2024-04-22T23:35:00.383 INFO:tasks.workunit.client.0.smithi107.stdout:9/85: unlink d5/f1e 0 2024-04-22T23:35:00.385 INFO:tasks.workunit.client.0.smithi107.stdout:4/66: write d1/f13 [438134,118279] 0 2024-04-22T23:35:00.391 INFO:tasks.workunit.client.0.smithi107.stdout:9/86: mkdir d5/d7/d17/d1b/d1f 0 2024-04-22T23:35:00.391 INFO:tasks.workunit.client.0.smithi107.stdout:9/87: creat d5/d14/f20 x:0 0 0 2024-04-22T23:35:00.392 INFO:tasks.workunit.client.0.smithi107.stdout:9/88: chown d5/d14/f20 4064004 1 2024-04-22T23:35:00.392 INFO:tasks.workunit.client.0.smithi107.stdout:9/89: chown d5/fc 24121908 1 2024-04-22T23:35:00.395 INFO:tasks.workunit.client.0.smithi107.stdout:4/67: mknod d1/d7/c1c 0 2024-04-22T23:35:00.396 INFO:tasks.workunit.client.0.smithi107.stdout:2/83: rmdir d1/d12 39 2024-04-22T23:35:00.396 INFO:tasks.workunit.client.0.smithi107.stdout:2/84: write d1/f11 [2210320,48460] 0 2024-04-22T23:35:00.397 INFO:tasks.workunit.client.0.smithi107.stdout:0/46: symlink d1/d9/l12 0 2024-04-22T23:35:00.397 INFO:tasks.workunit.client.0.smithi107.stdout:7/57: truncate f2 1585072 0 2024-04-22T23:35:00.401 INFO:tasks.workunit.client.0.smithi107.stdout:9/90: symlink d5/d7/d17/d1b/d1f/l21 0 2024-04-22T23:35:00.402 INFO:tasks.workunit.client.0.smithi107.stdout:9/91: stat d5/ce 0 2024-04-22T23:35:00.402 INFO:tasks.workunit.client.0.smithi107.stdout:6/76: dwrite f6 [0,4194304] 0 2024-04-22T23:35:00.402 INFO:tasks.workunit.client.0.smithi107.stdout:3/124: dwrite f0 [0,4194304] 0 2024-04-22T23:35:00.402 INFO:tasks.workunit.client.0.smithi107.stdout:4/68: unlink d1/f3 0 2024-04-22T23:35:00.405 INFO:tasks.workunit.client.0.smithi107.stdout:2/85: mknod d1/da/c20 0 2024-04-22T23:35:00.406 INFO:tasks.workunit.client.0.smithi107.stdout:0/47: creat d1/f13 x:0 0 0 2024-04-22T23:35:00.409 INFO:tasks.workunit.client.0.smithi107.stdout:9/92: symlink d5/l22 0 2024-04-22T23:35:00.412 INFO:tasks.workunit.client.0.smithi107.stdout:3/125: creat db/d21/d27/d15/f30 x:0 0 0 2024-04-22T23:35:00.413 INFO:tasks.workunit.client.0.smithi107.stdout:4/69: mknod d1/d1b/c1d 0 2024-04-22T23:35:00.414 INFO:tasks.workunit.client.0.smithi107.stdout:2/86: mknod d1/da/d13/c21 0 2024-04-22T23:35:00.416 INFO:tasks.workunit.client.0.smithi107.stdout:5/95: truncate f7 5894513 0 2024-04-22T23:35:00.416 INFO:tasks.workunit.client.0.smithi107.stdout:6/77: dwrite db/fd [0,4194304] 0 2024-04-22T23:35:00.418 INFO:tasks.workunit.client.0.smithi107.stdout:9/93: unlink d5/d7/d17/f18 0 2024-04-22T23:35:00.419 INFO:tasks.workunit.client.0.smithi107.stdout:4/70: mknod d1/d1b/c1e 0 2024-04-22T23:35:00.420 INFO:tasks.workunit.client.0.smithi107.stdout:4/71: fdatasync d1/d7/fc 0 2024-04-22T23:35:00.421 INFO:tasks.workunit.client.0.smithi107.stdout:0/48: mkdir d1/d9/d14 0 2024-04-22T23:35:00.421 INFO:tasks.workunit.client.0.smithi107.stdout:0/49: getdents d1/d11 0 2024-04-22T23:35:00.424 INFO:tasks.workunit.client.0.smithi107.stdout:6/78: unlink l8 0 2024-04-22T23:35:00.424 INFO:tasks.workunit.client.0.smithi107.stdout:6/79: write db/f11 [39300,120776] 0 2024-04-22T23:35:00.424 INFO:tasks.workunit.client.0.smithi107.stdout:6/80: stat f6 0 2024-04-22T23:35:00.424 INFO:tasks.workunit.client.0.smithi107.stdout:9/94: symlink d5/d7/d17/d1b/l23 0 2024-04-22T23:35:00.426 INFO:tasks.workunit.client.0.smithi107.stdout:4/72: creat d1/d8/f1f x:0 0 0 2024-04-22T23:35:00.426 INFO:tasks.workunit.client.0.smithi107.stdout:4/73: readlink d1/l14 0 2024-04-22T23:35:00.426 INFO:tasks.workunit.client.0.smithi107.stdout:4/74: read - d1/d8/f1f zero size 2024-04-22T23:35:00.427 INFO:tasks.workunit.client.0.smithi107.stdout:7/58: rename f2 to d7/ff 0 2024-04-22T23:35:00.427 INFO:tasks.workunit.client.0.smithi107.stdout:2/87: rename d1/da to d1/da/d22 22 2024-04-22T23:35:00.427 INFO:tasks.workunit.client.0.smithi107.stdout:2/88: chown d1/da/d13 589 1 2024-04-22T23:35:00.430 INFO:tasks.workunit.client.0.smithi107.stdout:4/75: mkdir d1/d8/d20 0 2024-04-22T23:35:00.431 INFO:tasks.workunit.client.0.smithi107.stdout:7/59: rmdir d7 39 2024-04-22T23:35:00.433 INFO:tasks.workunit.client.0.smithi107.stdout:5/96: rename d9/db/f22 to d9/db/de/f23 0 2024-04-22T23:35:00.433 INFO:tasks.workunit.client.0.smithi107.stdout:6/81: dread f6 [0,4194304] 0 2024-04-22T23:35:00.434 INFO:tasks.workunit.client.0.smithi107.stdout:2/89: creat d1/da/d13/f23 x:0 0 0 2024-04-22T23:35:00.438 INFO:tasks.workunit.client.0.smithi107.stdout:3/126: fsync db/d21/d27/d15/f30 0 2024-04-22T23:35:00.438 INFO:tasks.workunit.client.0.smithi107.stdout:3/127: write db/f18 [1106103,33118] 0 2024-04-22T23:35:00.438 INFO:tasks.workunit.client.0.smithi107.stdout:6/82: link db/fd db/f13 0 2024-04-22T23:35:00.438 INFO:tasks.workunit.client.0.smithi107.stdout:6/83: creat db/f14 x:0 0 0 2024-04-22T23:35:00.440 INFO:tasks.workunit.client.0.smithi107.stdout:2/90: link d1/l9 d1/da/l24 0 2024-04-22T23:35:00.440 INFO:tasks.workunit.client.0.smithi107.stdout:2/91: chown d1/da/c20 14 1 2024-04-22T23:35:00.440 INFO:tasks.workunit.client.0.smithi107.stdout:2/92: truncate d1/d12/f14 1786387 0 2024-04-22T23:35:00.440 INFO:tasks.workunit.client.0.smithi107.stdout:2/93: write d1/da/f10 [1194478,42234] 0 2024-04-22T23:35:00.442 INFO:tasks.workunit.client.0.smithi107.stdout:0/50: rmdir d1/d11 0 2024-04-22T23:35:00.446 INFO:tasks.workunit.client.0.smithi107.stdout:9/95: dwrite d5/d7/ff [0,4194304] 0 2024-04-22T23:35:00.446 INFO:tasks.workunit.client.0.smithi107.stdout:9/96: chown d5/d7 333 1 2024-04-22T23:35:00.447 INFO:tasks.workunit.client.0.smithi107.stdout:5/97: read d9/db/de/f1f [3928274,93356] 0 2024-04-22T23:35:00.447 INFO:tasks.workunit.client.0.smithi107.stdout:7/60: unlink d7/l9 0 2024-04-22T23:35:00.449 INFO:tasks.workunit.client.0.smithi107.stdout:9/97: dread d5/f11 [0,4194304] 0 2024-04-22T23:35:00.449 INFO:tasks.workunit.client.0.smithi107.stdout:6/84: link db/f11 db/f15 0 2024-04-22T23:35:00.450 INFO:tasks.workunit.client.0.smithi107.stdout:0/51: stat d1/c4 0 2024-04-22T23:35:00.457 INFO:tasks.workunit.client.0.smithi107.stdout:7/61: dread d7/ff [0,4194304] 0 2024-04-22T23:35:00.460 INFO:tasks.workunit.client.0.smithi107.stdout:6/85: mknod db/c16 0 2024-04-22T23:35:00.462 INFO:tasks.workunit.client.0.smithi107.stdout:8/28: sync 2024-04-22T23:35:00.462 INFO:tasks.workunit.client.0.smithi107.stdout:0/52: unlink d1/d9/le 0 2024-04-22T23:35:00.462 INFO:tasks.workunit.client.0.smithi107.stdout:0/53: creat d1/f15 x:0 0 0 2024-04-22T23:35:00.469 INFO:tasks.workunit.client.0.smithi107.stdout:7/62: mknod d7/c10 0 2024-04-22T23:35:00.469 INFO:tasks.workunit.client.0.smithi107.stdout:7/63: chown d7/le 748132 1 2024-04-22T23:35:00.469 INFO:tasks.workunit.client.0.smithi107.stdout:7/64: chown d7/ff 44161 1 2024-04-22T23:35:00.470 INFO:tasks.workunit.client.0.smithi107.stdout:0/54: mknod d1/d9/c16 0 2024-04-22T23:35:00.470 INFO:tasks.workunit.client.0.smithi107.stdout:0/55: chown d1/d9/d14 3 1 2024-04-22T23:35:00.470 INFO:tasks.workunit.client.0.smithi107.stdout:0/56: fdatasync d1/f15 0 2024-04-22T23:35:00.470 INFO:tasks.workunit.client.0.smithi107.stdout:6/86: write db/fd [3224798,67749] 0 2024-04-22T23:35:00.472 INFO:tasks.workunit.client.0.smithi107.stdout:7/65: dread d7/ff [0,4194304] 0 2024-04-22T23:35:00.472 INFO:tasks.workunit.client.0.smithi107.stdout:6/87: symlink db/l17 0 2024-04-22T23:35:00.472 INFO:tasks.workunit.client.0.smithi107.stdout:6/88: readlink db/l17 0 2024-04-22T23:35:00.472 INFO:tasks.workunit.client.0.smithi107.stdout:6/89: stat db/le 0 2024-04-22T23:35:00.472 INFO:tasks.workunit.client.0.smithi107.stdout:7/66: stat d7/cc 0 2024-04-22T23:35:00.473 INFO:tasks.workunit.client.0.smithi107.stdout:6/90: mknod db/c18 0 2024-04-22T23:35:00.476 INFO:tasks.workunit.client.0.smithi107.stdout:4/76: dwrite d1/d7/fc [0,4194304] 0 2024-04-22T23:35:00.476 INFO:tasks.workunit.client.0.smithi107.stdout:0/57: dread d1/f2 [0,4194304] 0 2024-04-22T23:35:00.476 INFO:tasks.workunit.client.0.smithi107.stdout:0/58: truncate d1/d9/fc 1249476 0 2024-04-22T23:35:00.477 INFO:tasks.workunit.client.0.smithi107.stdout:4/77: mkdir d1/d21 0 2024-04-22T23:35:00.477 INFO:tasks.workunit.client.0.smithi107.stdout:4/78: rename d1/d21 to d1/d21/d22 22 2024-04-22T23:35:00.478 INFO:tasks.workunit.client.0.smithi107.stdout:0/59: mkdir d1/d17 0 2024-04-22T23:35:00.479 INFO:tasks.workunit.client.0.smithi107.stdout:4/79: mkdir d1/d8/d23 0 2024-04-22T23:35:00.480 INFO:tasks.workunit.client.0.smithi107.stdout:4/80: mkdir d1/d1b/d24 0 2024-04-22T23:35:00.482 INFO:tasks.workunit.client.0.smithi107.stdout:7/67: dread d7/fd [0,4194304] 0 2024-04-22T23:35:00.483 INFO:tasks.workunit.client.0.smithi107.stdout:4/81: unlink d1/d8/d9/f19 0 2024-04-22T23:35:00.485 INFO:tasks.workunit.client.0.smithi107.stdout:4/82: rmdir d1/d1b/d24 0 2024-04-22T23:35:00.485 INFO:tasks.workunit.client.0.smithi107.stdout:7/68: dread d7/ff [0,4194304] 0 2024-04-22T23:35:00.485 INFO:tasks.workunit.client.0.smithi107.stdout:7/69: fsync d7/ff 0 2024-04-22T23:35:00.485 INFO:tasks.workunit.client.0.smithi107.stdout:6/91: write f0 [1350786,96678] 0 2024-04-22T23:35:00.485 INFO:tasks.workunit.client.0.smithi107.stdout:6/92: write f6 [9233052,95391] 0 2024-04-22T23:35:00.486 INFO:tasks.workunit.client.0.smithi107.stdout:2/94: dwrite d1/f8 [0,4194304] 0 2024-04-22T23:35:00.486 INFO:tasks.workunit.client.0.smithi107.stdout:4/83: rmdir d1/d8/d20 0 2024-04-22T23:35:00.487 INFO:tasks.workunit.client.0.smithi107.stdout:3/128: dwrite db/ff [4194304,4194304] 0 2024-04-22T23:35:00.487 INFO:tasks.workunit.client.0.smithi107.stdout:3/129: readlink db/d21/d27/l2c 0 2024-04-22T23:35:00.487 INFO:tasks.workunit.client.0.smithi107.stdout:7/70: dread d7/ff [0,4194304] 0 2024-04-22T23:35:00.487 INFO:tasks.workunit.client.0.smithi107.stdout:6/93: rename db/l17 to db/l19 0 2024-04-22T23:35:00.490 INFO:tasks.workunit.client.0.smithi107.stdout:7/71: readlink l6 0 2024-04-22T23:35:00.492 INFO:tasks.workunit.client.0.smithi107.stdout:2/95: rmdir d1/da/d13 39 2024-04-22T23:35:00.493 INFO:tasks.workunit.client.0.smithi107.stdout:2/96: read d1/f6 [79829,63901] 0 2024-04-22T23:35:00.493 INFO:tasks.workunit.client.0.smithi107.stdout:2/97: write d1/da/d13/f23 [533784,40288] 0 2024-04-22T23:35:00.494 INFO:tasks.workunit.client.0.smithi107.stdout:9/98: dwrite f4 [0,4194304] 0 2024-04-22T23:35:00.495 INFO:tasks.workunit.client.0.smithi107.stdout:8/29: dwrite d0/f4 [0,4194304] 0 2024-04-22T23:35:00.496 INFO:tasks.workunit.client.0.smithi107.stdout:2/98: rename d1/l2 to d1/da/l25 0 2024-04-22T23:35:00.496 INFO:tasks.workunit.client.0.smithi107.stdout:2/99: readlink d1/da/l25 0 2024-04-22T23:35:00.496 INFO:tasks.workunit.client.0.smithi107.stdout:0/60: dwrite d1/f13 [0,4194304] 0 2024-04-22T23:35:00.499 INFO:tasks.workunit.client.0.smithi107.stdout:5/98: dwrite d9/db/de/f1f [4194304,4194304] 0 2024-04-22T23:35:00.499 INFO:tasks.workunit.client.0.smithi107.stdout:1/65: sync 2024-04-22T23:35:00.499 INFO:tasks.workunit.client.0.smithi107.stdout:1/66: stat d7/cd 0 2024-04-22T23:35:00.502 INFO:tasks.workunit.client.0.smithi107.stdout:2/100: stat d1/da/l25 0 2024-04-22T23:35:00.502 INFO:tasks.workunit.client.0.smithi107.stdout:4/84: write d1/f5 [1971155,67541] 0 2024-04-22T23:35:00.503 INFO:tasks.workunit.client.0.smithi107.stdout:3/130: write db/d21/d27/f20 [2926890,30192] 0 2024-04-22T23:35:00.507 INFO:tasks.workunit.client.0.smithi107.stdout:9/99: getdents d5/d7 0 2024-04-22T23:35:00.507 INFO:tasks.workunit.client.0.smithi107.stdout:9/100: dread - d5/d7/fd zero size 2024-04-22T23:35:00.507 INFO:tasks.workunit.client.0.smithi107.stdout:9/101: write f1 [2050313,75226] 0 2024-04-22T23:35:00.507 INFO:tasks.workunit.client.0.smithi107.stdout:8/30: write d0/f3 [2146125,104176] 0 2024-04-22T23:35:00.508 INFO:tasks.workunit.client.0.smithi107.stdout:9/102: write d5/d7/fb [386184,8678] 0 2024-04-22T23:35:00.509 INFO:tasks.workunit.client.0.smithi107.stdout:2/101: truncate d1/da/f10 714907 0 2024-04-22T23:35:00.510 INFO:tasks.workunit.client.0.smithi107.stdout:7/72: dwrite d7/ff [0,4194304] 0 2024-04-22T23:35:00.510 INFO:tasks.workunit.client.0.smithi107.stdout:7/73: stat d7/cc 0 2024-04-22T23:35:00.511 INFO:tasks.workunit.client.0.smithi107.stdout:3/131: symlink db/d21/d27/d22/l31 0 2024-04-22T23:35:00.511 INFO:tasks.workunit.client.0.smithi107.stdout:5/99: dread f7 [0,4194304] 0 2024-04-22T23:35:00.511 INFO:tasks.workunit.client.0.smithi107.stdout:5/100: stat f4 0 2024-04-22T23:35:00.512 INFO:tasks.workunit.client.0.smithi107.stdout:0/61: dread d1/f13 [0,4194304] 0 2024-04-22T23:35:00.512 INFO:tasks.workunit.client.0.smithi107.stdout:1/67: mknod d7/c18 0 2024-04-22T23:35:00.512 INFO:tasks.workunit.client.0.smithi107.stdout:1/68: write d7/f13 [898675,17558] 0 2024-04-22T23:35:00.513 INFO:tasks.workunit.client.0.smithi107.stdout:0/62: dread d1/d9/fc [0,4194304] 0 2024-04-22T23:35:00.514 INFO:tasks.workunit.client.0.smithi107.stdout:9/103: mkdir d5/d24 0 2024-04-22T23:35:00.516 INFO:tasks.workunit.client.0.smithi107.stdout:2/102: unlink d1/da/c1e 0 2024-04-22T23:35:00.522 INFO:tasks.workunit.client.0.smithi107.stdout:6/94: getdents db 0 2024-04-22T23:35:00.523 INFO:tasks.workunit.client.0.smithi107.stdout:9/104: dread d5/f13 [0,4194304] 0 2024-04-22T23:35:00.527 INFO:tasks.workunit.client.0.smithi107.stdout:5/101: creat d9/f24 x:0 0 0 2024-04-22T23:35:00.527 INFO:tasks.workunit.client.0.smithi107.stdout:5/102: fdatasync d9/fd 0 2024-04-22T23:35:00.527 INFO:tasks.workunit.client.0.smithi107.stdout:7/74: rmdir d7 39 2024-04-22T23:35:00.529 INFO:tasks.workunit.client.0.smithi107.stdout:3/132: symlink db/d21/d27/d15/d24/l32 0 2024-04-22T23:35:00.530 INFO:tasks.workunit.client.0.smithi107.stdout:6/95: rmdir db 39 2024-04-22T23:35:00.531 INFO:tasks.workunit.client.0.smithi107.stdout:6/96: stat db/f14 0 2024-04-22T23:35:00.531 INFO:tasks.workunit.client.0.smithi107.stdout:4/85: dwrite d1/d7/fd [0,4194304] 0 2024-04-22T23:35:00.531 INFO:tasks.workunit.client.0.smithi107.stdout:4/86: creat d1/d8/f25 x:0 0 0 2024-04-22T23:35:00.531 INFO:tasks.workunit.client.0.smithi107.stdout:4/87: write d1/f13 [2264644,43557] 0 2024-04-22T23:35:00.531 INFO:tasks.workunit.client.0.smithi107.stdout:4/88: read - d1/d8/f1f zero size 2024-04-22T23:35:00.531 INFO:tasks.workunit.client.0.smithi107.stdout:4/89: fsync d1/d7/fd 0 2024-04-22T23:35:00.531 INFO:tasks.workunit.client.0.smithi107.stdout:4/90: chown d1/d8/f1f 39724603 1 2024-04-22T23:35:00.532 INFO:tasks.workunit.client.0.smithi107.stdout:0/63: rename d1/d9/l12 to d1/d9/l18 0 2024-04-22T23:35:00.533 INFO:tasks.workunit.client.0.smithi107.stdout:7/75: creat d7/f11 x:0 0 0 2024-04-22T23:35:00.534 INFO:tasks.workunit.client.0.smithi107.stdout:2/103: dwrite d1/d12/f14 [0,4194304] 0 2024-04-22T23:35:00.534 INFO:tasks.workunit.client.0.smithi107.stdout:2/104: readlink d1/da/l16 0 2024-04-22T23:35:00.534 INFO:tasks.workunit.client.0.smithi107.stdout:2/105: truncate d1/da/f1f 233683 0 2024-04-22T23:35:00.534 INFO:tasks.workunit.client.0.smithi107.stdout:6/97: rename l9 to db/l1a 0 2024-04-22T23:35:00.536 INFO:tasks.workunit.client.0.smithi107.stdout:4/91: symlink d1/l26 0 2024-04-22T23:35:00.536 INFO:tasks.workunit.client.0.smithi107.stdout:4/92: write d1/d7/f11 [693575,96797] 0 2024-04-22T23:35:00.536 INFO:tasks.workunit.client.0.smithi107.stdout:4/93: dread - d1/d8/f25 zero size 2024-04-22T23:35:00.536 INFO:tasks.workunit.client.0.smithi107.stdout:0/64: symlink d1/d9/d14/l19 0 2024-04-22T23:35:00.536 INFO:tasks.workunit.client.0.smithi107.stdout:8/31: dwrite d0/f5 [0,4194304] 0 2024-04-22T23:35:00.537 INFO:tasks.workunit.client.0.smithi107.stdout:5/103: read f3 [3888993,31782] 0 2024-04-22T23:35:00.537 INFO:tasks.workunit.client.0.smithi107.stdout:5/104: creat d9/f25 x:0 0 0 2024-04-22T23:35:00.537 INFO:tasks.workunit.client.0.smithi107.stdout:5/105: write f3 [2150759,113654] 0 2024-04-22T23:35:00.537 INFO:tasks.workunit.client.0.smithi107.stdout:5/106: chown d9 207008 1 2024-04-22T23:35:00.540 INFO:tasks.workunit.client.0.smithi107.stdout:1/69: rmdir d7 39 2024-04-22T23:35:00.541 INFO:tasks.workunit.client.0.smithi107.stdout:2/106: link d1/f11 d1/d12/f26 0 2024-04-22T23:35:00.543 INFO:tasks.workunit.client.0.smithi107.stdout:2/107: dread d1/da/f1f [0,4194304] 0 2024-04-22T23:35:00.543 INFO:tasks.workunit.client.0.smithi107.stdout:2/108: readlink d1/d12/l1b 0 2024-04-22T23:35:00.544 INFO:tasks.workunit.client.0.smithi107.stdout:4/94: dread d1/d7/f11 [0,4194304] 0 2024-04-22T23:35:00.547 INFO:tasks.workunit.client.0.smithi107.stdout:8/32: mkdir d0/d9 0 2024-04-22T23:35:00.550 INFO:tasks.workunit.client.0.smithi107.stdout:6/98: getdents db 0 2024-04-22T23:35:00.550 INFO:tasks.workunit.client.0.smithi107.stdout:6/99: write db/f12 [305986,30136] 0 2024-04-22T23:35:00.552 INFO:tasks.workunit.client.0.smithi107.stdout:5/107: mkdir d9/d26 0 2024-04-22T23:35:00.553 INFO:tasks.workunit.client.0.smithi107.stdout:0/65: dwrite d1/f2 [0,4194304] 0 2024-04-22T23:35:00.553 INFO:tasks.workunit.client.0.smithi107.stdout:6/100: dread f0 [4194304,4194304] 0 2024-04-22T23:35:00.553 INFO:tasks.workunit.client.0.smithi107.stdout:6/101: fsync db/ff 0 2024-04-22T23:35:00.561 INFO:tasks.workunit.client.0.smithi107.stdout:7/76: mknod d7/c12 0 2024-04-22T23:35:00.563 INFO:tasks.workunit.client.0.smithi107.stdout:2/109: creat d1/d12/f27 x:0 0 0 2024-04-22T23:35:00.565 INFO:tasks.workunit.client.0.smithi107.stdout:8/33: creat d0/d7/fa x:0 0 0 2024-04-22T23:35:00.566 INFO:tasks.workunit.client.0.smithi107.stdout:8/34: truncate d0/f3 4811061 0 2024-04-22T23:35:00.568 INFO:tasks.workunit.client.0.smithi107.stdout:0/66: unlink d1/f2 0 2024-04-22T23:35:00.568 INFO:tasks.workunit.client.0.smithi107.stdout:0/67: fdatasync d1/f15 0 2024-04-22T23:35:00.569 INFO:tasks.workunit.client.0.smithi107.stdout:6/102: creat db/f1b x:0 0 0 2024-04-22T23:35:00.569 INFO:tasks.workunit.client.0.smithi107.stdout:6/103: fsync db/fd 0 2024-04-22T23:35:00.569 INFO:tasks.workunit.client.0.smithi107.stdout:6/104: read - db/f1b zero size 2024-04-22T23:35:00.570 INFO:tasks.workunit.client.0.smithi107.stdout:7/77: creat d7/f13 x:0 0 0 2024-04-22T23:35:00.572 INFO:tasks.workunit.client.0.smithi107.stdout:1/70: mkdir d7/d19 0 2024-04-22T23:35:00.574 INFO:tasks.workunit.client.0.smithi107.stdout:2/110: symlink d1/da/d13/l28 0 2024-04-22T23:35:00.574 INFO:tasks.workunit.client.0.smithi107.stdout:2/111: creat d1/da/f29 x:0 0 0 2024-04-22T23:35:00.574 INFO:tasks.workunit.client.0.smithi107.stdout:2/112: creat d1/da/f2a x:0 0 0 2024-04-22T23:35:00.583 INFO:tasks.workunit.client.0.smithi107.stdout:8/35: dwrite d0/f4 [4194304,4194304] 0 2024-04-22T23:35:00.583 INFO:tasks.workunit.client.0.smithi107.stdout:8/36: creat d0/fb x:0 0 0 2024-04-22T23:35:00.583 INFO:tasks.workunit.client.0.smithi107.stdout:8/37: chown d0/fb 11094585 1 2024-04-22T23:35:00.585 INFO:tasks.workunit.client.0.smithi107.stdout:2/113: mkdir d1/d2b 0 2024-04-22T23:35:00.587 INFO:tasks.workunit.client.0.smithi107.stdout:4/95: rename d1/d8/d9 to d1/d8/d23/d27 0 2024-04-22T23:35:00.588 INFO:tasks.workunit.client.0.smithi107.stdout:0/68: link d1/c4 d1/d9/c1a 0 2024-04-22T23:35:00.595 INFO:tasks.workunit.client.0.smithi107.stdout:6/105: dwrite db/f14 [0,4194304] 0 2024-04-22T23:35:00.595 INFO:tasks.workunit.client.0.smithi107.stdout:6/106: creat db/f1c x:0 0 0 2024-04-22T23:35:00.595 INFO:tasks.workunit.client.0.smithi107.stdout:6/107: write db/f15 [1168321,17341] 0 2024-04-22T23:35:00.595 INFO:tasks.workunit.client.0.smithi107.stdout:6/108: chown db/f1b 127370659 1 2024-04-22T23:35:00.595 INFO:tasks.workunit.client.0.smithi107.stdout:6/109: readlink db/l1a 0 2024-04-22T23:35:00.595 INFO:tasks.workunit.client.0.smithi107.stdout:6/110: creat db/f1d x:0 0 0 2024-04-22T23:35:00.595 INFO:tasks.workunit.client.0.smithi107.stdout:6/111: creat db/f1e x:0 0 0 2024-04-22T23:35:00.595 INFO:tasks.workunit.client.0.smithi107.stdout:6/112: chown db/f1d 88051 1 2024-04-22T23:35:00.598 INFO:tasks.workunit.client.0.smithi107.stdout:5/108: rename d9/f25 to d9/db/f27 0 2024-04-22T23:35:00.599 INFO:tasks.workunit.client.0.smithi107.stdout:8/38: dread d0/f4 [4194304,4194304] 0 2024-04-22T23:35:00.599 INFO:tasks.workunit.client.0.smithi107.stdout:1/71: dwrite d7/f10 [0,4194304] 0 2024-04-22T23:35:00.599 INFO:tasks.workunit.client.0.smithi107.stdout:1/72: readlink l6 0 2024-04-22T23:35:00.599 INFO:tasks.workunit.client.0.smithi107.stdout:0/69: getdents d1/d9/d14 0 2024-04-22T23:35:00.601 INFO:tasks.workunit.client.0.smithi107.stdout:7/78: getdents d7 0 2024-04-22T23:35:00.608 INFO:tasks.workunit.client.0.smithi107.stdout:2/114: dwrite d1/f11 [0,4194304] 0 2024-04-22T23:35:00.609 INFO:tasks.workunit.client.0.smithi107.stdout:9/105: sync 2024-04-22T23:35:00.609 INFO:tasks.workunit.client.0.smithi107.stdout:9/106: chown d5/f10 829 1 2024-04-22T23:35:00.610 INFO:tasks.workunit.client.0.smithi107.stdout:8/39: link d0/fb d0/fc 0 2024-04-22T23:35:00.612 INFO:tasks.workunit.client.0.smithi107.stdout:1/73: unlink d7/f9 0 2024-04-22T23:35:00.612 INFO:tasks.workunit.client.0.smithi107.stdout:0/70: chown d1/d9/cd 168 1 2024-04-22T23:35:00.613 INFO:tasks.workunit.client.0.smithi107.stdout:4/96: dwrite d1/d7/f11 [0,4194304] 0 2024-04-22T23:35:00.614 INFO:tasks.workunit.client.0.smithi107.stdout:7/79: symlink d7/l14 0 2024-04-22T23:35:00.614 INFO:tasks.workunit.client.0.smithi107.stdout:7/80: readlink d7/l14 0 2024-04-22T23:35:00.615 INFO:tasks.workunit.client.0.smithi107.stdout:0/71: dread d1/d9/fc [0,4194304] 0 2024-04-22T23:35:00.615 INFO:tasks.workunit.client.0.smithi107.stdout:0/72: fsync d1/f15 0 2024-04-22T23:35:00.615 INFO:tasks.workunit.client.0.smithi107.stdout:0/73: write d1/d9/fc [1850474,40222] 0 2024-04-22T23:35:00.617 INFO:tasks.workunit.client.0.smithi107.stdout:6/113: rename db/f1b to db/f1f 0 2024-04-22T23:35:00.617 INFO:tasks.workunit.client.0.smithi107.stdout:6/114: write db/f1e [623758,45435] 0 2024-04-22T23:35:00.618 INFO:tasks.workunit.client.0.smithi107.stdout:2/115: rename d1/f4 to d1/da/f2c 0 2024-04-22T23:35:00.618 INFO:tasks.workunit.client.0.smithi107.stdout:2/116: read - d1/da/f29 zero size 2024-04-22T23:35:00.618 INFO:tasks.workunit.client.0.smithi107.stdout:6/115: write db/f12 [74082,72965] 0 2024-04-22T23:35:00.618 INFO:tasks.workunit.client.0.smithi107.stdout:2/117: creat d1/f2d x:0 0 0 2024-04-22T23:35:00.621 INFO:tasks.workunit.client.0.smithi107.stdout:7/81: truncate d7/ff 4242699 0 2024-04-22T23:35:00.622 INFO:tasks.workunit.client.0.smithi107.stdout:0/74: truncate d1/f13 131872 0 2024-04-22T23:35:00.624 INFO:tasks.workunit.client.0.smithi107.stdout:1/74: rename d7/ff to d7/d19/f1a 0 2024-04-22T23:35:00.631 INFO:tasks.workunit.client.0.smithi107.stdout:2/118: mknod d1/da/c2e 0 2024-04-22T23:35:00.631 INFO:tasks.workunit.client.0.smithi107.stdout:2/119: write d1/da/f29 [1044526,13650] 0 2024-04-22T23:35:00.631 INFO:tasks.workunit.client.0.smithi107.stdout:7/82: truncate d7/f11 802004 0 2024-04-22T23:35:00.633 INFO:tasks.workunit.client.0.smithi107.stdout:3/133: sync 2024-04-22T23:35:00.633 INFO:tasks.workunit.client.0.smithi107.stdout:3/134: truncate db/f1c 383631 0 2024-04-22T23:35:00.634 INFO:tasks.workunit.client.0.smithi107.stdout:3/135: fdatasync db/f1c 0 2024-04-22T23:35:00.634 INFO:tasks.workunit.client.0.smithi107.stdout:6/116: dread db/f1e [0,4194304] 0 2024-04-22T23:35:00.636 INFO:tasks.workunit.client.0.smithi107.stdout:0/75: rename d1/d9/c16 to d1/d17/c1b 0 2024-04-22T23:35:00.636 INFO:tasks.workunit.client.0.smithi107.stdout:2/120: mkdir d1/d2f 0 2024-04-22T23:35:00.642 INFO:tasks.workunit.client.0.smithi107.stdout:8/40: dwrite d0/d7/fa [0,4194304] 0 2024-04-22T23:35:00.642 INFO:tasks.workunit.client.0.smithi107.stdout:1/75: dread d7/f13 [0,4194304] 0 2024-04-22T23:35:00.643 INFO:tasks.workunit.client.0.smithi107.stdout:3/136: creat db/d21/f33 x:0 0 0 2024-04-22T23:35:00.643 INFO:tasks.workunit.client.0.smithi107.stdout:3/137: chown db 226 1 2024-04-22T23:35:00.643 INFO:tasks.workunit.client.0.smithi107.stdout:3/138: write db/d21/f33 [891035,16431] 0 2024-04-22T23:35:00.643 INFO:tasks.workunit.client.0.smithi107.stdout:3/139: chown c7 28 1 2024-04-22T23:35:00.643 INFO:tasks.workunit.client.0.smithi107.stdout:9/107: dwrite d5/f13 [0,4194304] 0 2024-04-22T23:35:00.643 INFO:tasks.workunit.client.0.smithi107.stdout:9/108: truncate d5/d7/fd 374045 0 2024-04-22T23:35:00.643 INFO:tasks.workunit.client.0.smithi107.stdout:4/97: dwrite d1/d7/f11 [0,4194304] 0 2024-04-22T23:35:00.643 INFO:tasks.workunit.client.0.smithi107.stdout:4/98: dread - d1/d8/f1f zero size 2024-04-22T23:35:00.643 INFO:tasks.workunit.client.0.smithi107.stdout:4/99: write d1/d8/f25 [919731,114429] 0 2024-04-22T23:35:00.643 INFO:tasks.workunit.client.0.smithi107.stdout:4/100: creat d1/f28 x:0 0 0 2024-04-22T23:35:00.644 INFO:tasks.workunit.client.0.smithi107.stdout:3/140: dread db/f18 [0,4194304] 0 2024-04-22T23:35:00.650 INFO:tasks.workunit.client.0.smithi107.stdout:6/117: dwrite db/f1f [0,4194304] 0 2024-04-22T23:35:00.650 INFO:tasks.workunit.client.0.smithi107.stdout:8/41: rmdir d0/d9 0 2024-04-22T23:35:00.656 INFO:tasks.workunit.client.0.smithi107.stdout:9/109: symlink d5/l25 0 2024-04-22T23:35:00.657 INFO:tasks.workunit.client.0.smithi107.stdout:0/76: mknod d1/d9/c1c 0 2024-04-22T23:35:00.659 INFO:tasks.workunit.client.0.smithi107.stdout:1/76: chown d7/f10 188196 1 2024-04-22T23:35:00.662 INFO:tasks.workunit.client.0.smithi107.stdout:4/101: mkdir d1/d21/d29 0 2024-04-22T23:35:00.665 INFO:tasks.workunit.client.0.smithi107.stdout:2/121: rmdir d1 39 2024-04-22T23:35:00.666 INFO:tasks.workunit.client.0.smithi107.stdout:3/141: dwrite db/d21/d27/f23 [0,4194304] 0 2024-04-22T23:35:00.667 INFO:tasks.workunit.client.0.smithi107.stdout:6/118: truncate db/f14 2045508 0 2024-04-22T23:35:00.667 INFO:tasks.workunit.client.0.smithi107.stdout:6/119: write db/f13 [927869,18262] 0 2024-04-22T23:35:00.668 INFO:tasks.workunit.client.0.smithi107.stdout:6/120: chown db/f14 1241007033 1 2024-04-22T23:35:00.668 INFO:tasks.workunit.client.0.smithi107.stdout:0/77: mknod d1/d9/d14/c1d 0 2024-04-22T23:35:00.669 INFO:tasks.workunit.client.0.smithi107.stdout:0/78: truncate d1/f13 943851 0 2024-04-22T23:35:00.669 INFO:tasks.workunit.client.0.smithi107.stdout:0/79: write d1/f15 [677084,127620] 0 2024-04-22T23:35:00.670 INFO:tasks.workunit.client.0.smithi107.stdout:2/122: creat d1/f30 x:0 0 0 2024-04-22T23:35:00.671 INFO:tasks.workunit.client.0.smithi107.stdout:3/142: readlink db/d21/d27/d15/l14 0 2024-04-22T23:35:00.671 INFO:tasks.workunit.client.0.smithi107.stdout:7/83: dwrite d7/f13 [0,4194304] 0 2024-04-22T23:35:00.672 INFO:tasks.workunit.client.0.smithi107.stdout:8/42: dwrite d0/f5 [4194304,4194304] 0 2024-04-22T23:35:00.672 INFO:tasks.workunit.client.0.smithi107.stdout:9/110: dwrite d5/d7/f16 [0,4194304] 0 2024-04-22T23:35:00.672 INFO:tasks.workunit.client.0.smithi107.stdout:6/121: unlink db/f13 0 2024-04-22T23:35:00.673 INFO:tasks.workunit.client.0.smithi107.stdout:4/102: write d1/d8/f25 [877717,87359] 0 2024-04-22T23:35:00.674 INFO:tasks.workunit.client.0.smithi107.stdout:0/80: rmdir d1/d17 39 2024-04-22T23:35:00.674 INFO:tasks.workunit.client.0.smithi107.stdout:0/81: truncate d1/d9/fc 2237906 0 2024-04-22T23:35:00.674 INFO:tasks.workunit.client.0.smithi107.stdout:0/82: truncate d1/f13 1650489 0 2024-04-22T23:35:00.675 INFO:tasks.workunit.client.0.smithi107.stdout:2/123: creat d1/d12/f31 x:0 0 0 2024-04-22T23:35:00.678 INFO:tasks.workunit.client.0.smithi107.stdout:7/84: readlink d7/l8 0 2024-04-22T23:35:00.683 INFO:tasks.workunit.client.0.smithi107.stdout:8/43: mknod d0/d7/d8/cd 0 2024-04-22T23:35:00.683 INFO:tasks.workunit.client.0.smithi107.stdout:8/44: stat d0/fc 0 2024-04-22T23:35:00.687 INFO:tasks.workunit.client.0.smithi107.stdout:9/111: symlink d5/d7/l26 0 2024-04-22T23:35:00.693 INFO:tasks.workunit.client.0.smithi107.stdout:0/83: dwrite d1/f13 [0,4194304] 0 2024-04-22T23:35:00.693 INFO:tasks.workunit.client.0.smithi107.stdout:3/143: dwrite db/d21/f33 [0,4194304] 0 2024-04-22T23:35:00.693 INFO:tasks.workunit.client.0.smithi107.stdout:8/45: write d0/f5 [3633517,38539] 0 2024-04-22T23:35:00.694 INFO:tasks.workunit.client.0.smithi107.stdout:8/46: chown d0/d7/d8 91429 1 2024-04-22T23:35:00.697 INFO:tasks.workunit.client.0.smithi107.stdout:0/84: dread d1/f15 [0,4194304] 0 2024-04-22T23:35:00.699 INFO:tasks.workunit.client.0.smithi107.stdout:6/122: mkdir db/d20 0 2024-04-22T23:35:00.700 INFO:tasks.workunit.client.0.smithi107.stdout:4/103: mknod d1/d21/d29/c2a 0 2024-04-22T23:35:00.700 INFO:tasks.workunit.client.0.smithi107.stdout:2/124: creat d1/da/d13/f32 x:0 0 0 2024-04-22T23:35:00.702 INFO:tasks.workunit.client.0.smithi107.stdout:9/112: rmdir d5/d7/d17/d1b/d1f 39 2024-04-22T23:35:00.704 INFO:tasks.workunit.client.0.smithi107.stdout:0/85: write d1/f13 [4105550,59137] 0 2024-04-22T23:35:00.705 INFO:tasks.workunit.client.0.smithi107.stdout:0/86: chown d1/f15 38 1 2024-04-22T23:35:00.708 INFO:tasks.workunit.client.0.smithi107.stdout:3/144: rename f8 to db/d21/d27/d22/d25/f34 0 2024-04-22T23:35:00.710 INFO:tasks.workunit.client.0.smithi107.stdout:4/104: link d1/d7/l17 d1/d21/d29/l2b 0 2024-04-22T23:35:00.712 INFO:tasks.workunit.client.0.smithi107.stdout:2/125: creat d1/d2f/f33 x:0 0 0 2024-04-22T23:35:00.712 INFO:tasks.workunit.client.0.smithi107.stdout:2/126: creat d1/d12/f34 x:0 0 0 2024-04-22T23:35:00.713 INFO:tasks.workunit.client.0.smithi107.stdout:8/47: dwrite d0/f5 [4194304,4194304] 0 2024-04-22T23:35:00.713 INFO:tasks.workunit.client.0.smithi107.stdout:8/48: creat d0/d7/fe x:0 0 0 2024-04-22T23:35:00.713 INFO:tasks.workunit.client.0.smithi107.stdout:8/49: chown d0/fb 152341368 1 2024-04-22T23:35:00.713 INFO:tasks.workunit.client.0.smithi107.stdout:0/87: dread d1/f15 [0,4194304] 0 2024-04-22T23:35:00.714 INFO:tasks.workunit.client.0.smithi107.stdout:7/85: rename d7/l8 to d7/l15 0 2024-04-22T23:35:00.715 INFO:tasks.workunit.client.0.smithi107.stdout:9/113: mkdir d5/d7/d27 0 2024-04-22T23:35:00.716 INFO:tasks.workunit.client.0.smithi107.stdout:4/105: symlink d1/d8/l2c 0 2024-04-22T23:35:00.716 INFO:tasks.workunit.client.0.smithi107.stdout:4/106: write d1/f28 [141942,90593] 0 2024-04-22T23:35:00.718 INFO:tasks.workunit.client.0.smithi107.stdout:6/123: dwrite db/f14 [0,4194304] 0 2024-04-22T23:35:00.718 INFO:tasks.workunit.client.0.smithi107.stdout:6/124: truncate db/f12 942480 0 2024-04-22T23:35:00.719 INFO:tasks.workunit.client.0.smithi107.stdout:6/125: write f0 [5684786,32740] 0 2024-04-22T23:35:00.719 INFO:tasks.workunit.client.0.smithi107.stdout:2/127: mkdir d1/d12/d35 0 2024-04-22T23:35:00.719 INFO:tasks.workunit.client.0.smithi107.stdout:3/145: dread db/ff [0,4194304] 0 2024-04-22T23:35:00.721 INFO:tasks.workunit.client.0.smithi107.stdout:8/50: truncate d0/f4 4788685 0 2024-04-22T23:35:00.721 INFO:tasks.workunit.client.0.smithi107.stdout:8/51: stat d0/d7/fe 0 2024-04-22T23:35:00.722 INFO:tasks.workunit.client.0.smithi107.stdout:0/88: truncate d1/f15 769427 0 2024-04-22T23:35:00.723 INFO:tasks.workunit.client.0.smithi107.stdout:9/114: dread f4 [0,4194304] 0 2024-04-22T23:35:00.724 INFO:tasks.workunit.client.0.smithi107.stdout:5/109: sync 2024-04-22T23:35:00.724 INFO:tasks.workunit.client.0.smithi107.stdout:5/110: chown d9/db/de/l1c 0 1 2024-04-22T23:35:00.726 INFO:tasks.workunit.client.0.smithi107.stdout:9/115: dread d5/d7/fd [0,4194304] 0 2024-04-22T23:35:00.726 INFO:tasks.workunit.client.0.smithi107.stdout:9/116: fdatasync d5/f13 0 2024-04-22T23:35:00.727 INFO:tasks.workunit.client.0.smithi107.stdout:0/89: dread d1/f13 [0,4194304] 0 2024-04-22T23:35:00.730 INFO:tasks.workunit.client.0.smithi107.stdout:0/90: dread d1/d9/fc [0,4194304] 0 2024-04-22T23:35:00.733 INFO:tasks.workunit.client.0.smithi107.stdout:2/128: mknod d1/d2b/c36 0 2024-04-22T23:35:00.735 INFO:tasks.workunit.client.0.smithi107.stdout:8/52: mkdir d0/d7/df 0 2024-04-22T23:35:00.737 INFO:tasks.workunit.client.0.smithi107.stdout:7/86: link d7/l14 d7/l16 0 2024-04-22T23:35:00.740 INFO:tasks.workunit.client.0.smithi107.stdout:9/117: unlink d5/l22 0 2024-04-22T23:35:00.742 INFO:tasks.workunit.client.0.smithi107.stdout:2/129: link d1/da/c2e d1/da/c37 0 2024-04-22T23:35:00.744 INFO:tasks.workunit.client.0.smithi107.stdout:2/130: symlink d1/d12/l38 0 2024-04-22T23:35:00.744 INFO:tasks.workunit.client.0.smithi107.stdout:2/131: chown d1/d2f 13011973 1 2024-04-22T23:35:00.744 INFO:tasks.workunit.client.0.smithi107.stdout:0/91: dread d1/f15 [0,4194304] 0 2024-04-22T23:35:00.745 INFO:tasks.workunit.client.0.smithi107.stdout:8/53: rename d0/d7/fe to d0/d6/f10 0 2024-04-22T23:35:00.746 INFO:tasks.workunit.client.0.smithi107.stdout:7/87: mknod d7/c17 0 2024-04-22T23:35:00.752 INFO:tasks.workunit.client.0.smithi107.stdout:3/146: dwrite db/d21/d27/d22/d25/f34 [0,4194304] 0 2024-04-22T23:35:00.753 INFO:tasks.workunit.client.0.smithi107.stdout:5/111: dwrite d9/db/f13 [4194304,4194304] 0 2024-04-22T23:35:00.753 INFO:tasks.workunit.client.0.smithi107.stdout:2/132: write d1/fb [501305,130116] 0 2024-04-22T23:35:00.754 INFO:tasks.workunit.client.0.smithi107.stdout:6/126: dwrite db/f11 [0,4194304] 0 2024-04-22T23:35:00.754 INFO:tasks.workunit.client.0.smithi107.stdout:6/127: chown c1 831106 1 2024-04-22T23:35:00.754 INFO:tasks.workunit.client.0.smithi107.stdout:6/128: dread - db/f1c zero size 2024-04-22T23:35:00.754 INFO:tasks.workunit.client.0.smithi107.stdout:6/129: creat db/f21 x:0 0 0 2024-04-22T23:35:00.754 INFO:tasks.workunit.client.0.smithi107.stdout:1/77: sync 2024-04-22T23:35:00.755 INFO:tasks.workunit.client.0.smithi107.stdout:9/118: dwrite d5/d7/ff [0,4194304] 0 2024-04-22T23:35:00.755 INFO:tasks.workunit.client.0.smithi107.stdout:3/147: rename db/d21/d27/d15/l14 to db/d21/d27/l35 0 2024-04-22T23:35:00.760 INFO:tasks.workunit.client.0.smithi107.stdout:2/133: rmdir d1/d2b 39 2024-04-22T23:35:00.760 INFO:tasks.workunit.client.0.smithi107.stdout:2/134: stat d1/d12 0 2024-04-22T23:35:00.760 INFO:tasks.workunit.client.0.smithi107.stdout:2/135: write d1/f30 [915405,129289] 0 2024-04-22T23:35:00.763 INFO:tasks.workunit.client.0.smithi107.stdout:1/78: write d7/d19/f1a [511432,13972] 0 2024-04-22T23:35:00.765 INFO:tasks.workunit.client.0.smithi107.stdout:6/130: rename c7 to db/d20/c22 0 2024-04-22T23:35:00.767 INFO:tasks.workunit.client.0.smithi107.stdout:5/112: write f4 [5829282,100368] 0 2024-04-22T23:35:00.768 INFO:tasks.workunit.client.0.smithi107.stdout:2/136: truncate d1/da/f10 684882 0 2024-04-22T23:35:00.768 INFO:tasks.workunit.client.0.smithi107.stdout:6/131: dread db/ff [0,4194304] 0 2024-04-22T23:35:00.768 INFO:tasks.workunit.client.0.smithi107.stdout:1/79: creat d7/f1b x:0 0 0 2024-04-22T23:35:00.768 INFO:tasks.workunit.client.0.smithi107.stdout:6/132: dread - db/f1d zero size 2024-04-22T23:35:00.769 INFO:tasks.workunit.client.0.smithi107.stdout:6/133: dread db/ff [0,4194304] 0 2024-04-22T23:35:00.770 INFO:tasks.workunit.client.0.smithi107.stdout:5/113: unlink d9/db/de/l19 0 2024-04-22T23:35:00.774 INFO:tasks.workunit.client.0.smithi107.stdout:5/114: mknod d9/db/c28 0 2024-04-22T23:35:00.774 INFO:tasks.workunit.client.0.smithi107.stdout:5/115: creat d9/f29 x:0 0 0 2024-04-22T23:35:00.777 INFO:tasks.workunit.client.0.smithi107.stdout:1/80: unlink d7/f13 0 2024-04-22T23:35:00.778 INFO:tasks.workunit.client.0.smithi107.stdout:1/81: dread - d7/f15 zero size 2024-04-22T23:35:00.778 INFO:tasks.workunit.client.0.smithi107.stdout:1/82: mkdir d7/d19/d1c 0 2024-04-22T23:35:00.780 INFO:tasks.workunit.client.0.smithi107.stdout:7/88: dwrite d7/ff [4194304,4194304] 0 2024-04-22T23:35:00.782 INFO:tasks.workunit.client.0.smithi107.stdout:7/89: creat d7/f18 x:0 0 0 2024-04-22T23:35:00.786 INFO:tasks.workunit.client.0.smithi107.stdout:9/119: dwrite f4 [0,4194304] 0 2024-04-22T23:35:00.786 INFO:tasks.workunit.client.0.smithi107.stdout:8/54: dwrite d0/fb [0,4194304] 0 2024-04-22T23:35:00.786 INFO:tasks.workunit.client.0.smithi107.stdout:8/55: creat d0/d7/f11 x:0 0 0 2024-04-22T23:35:00.788 INFO:tasks.workunit.client.0.smithi107.stdout:1/83: write f2 [1759124,25957] 0 2024-04-22T23:35:00.790 INFO:tasks.workunit.client.0.smithi107.stdout:8/56: symlink d0/l12 0 2024-04-22T23:35:00.792 INFO:tasks.workunit.client.0.smithi107.stdout:7/90: read d7/ff [6681260,47601] 0 2024-04-22T23:35:00.793 INFO:tasks.workunit.client.0.smithi107.stdout:3/148: dwrite db/d21/f33 [4194304,4194304] 0 2024-04-22T23:35:00.793 INFO:tasks.workunit.client.0.smithi107.stdout:8/57: mknod d0/d7/df/c13 0 2024-04-22T23:35:00.793 INFO:tasks.workunit.client.0.smithi107.stdout:5/116: dwrite d9/f29 [0,4194304] 0 2024-04-22T23:35:00.796 INFO:tasks.workunit.client.0.smithi107.stdout:9/120: write f4 [237517,28265] 0 2024-04-22T23:35:00.796 INFO:tasks.workunit.client.0.smithi107.stdout:9/121: read - d5/d14/f15 zero size 2024-04-22T23:35:00.797 INFO:tasks.workunit.client.0.smithi107.stdout:6/134: write db/f15 [4734278,38151] 0 2024-04-22T23:35:00.797 INFO:tasks.workunit.client.0.smithi107.stdout:6/135: read - db/f1d zero size 2024-04-22T23:35:00.797 INFO:tasks.workunit.client.0.smithi107.stdout:6/136: fsync f0 0 2024-04-22T23:35:00.798 INFO:tasks.workunit.client.0.smithi107.stdout:7/91: dread d7/f11 [0,4194304] 0 2024-04-22T23:35:00.805 INFO:tasks.workunit.client.0.smithi107.stdout:5/117: write d9/db/f13 [5247026,102057] 0 2024-04-22T23:35:00.806 INFO:tasks.workunit.client.0.smithi107.stdout:2/137: truncate d1/d12/f26 87187 0 2024-04-22T23:35:00.807 INFO:tasks.workunit.client.0.smithi107.stdout:2/138: fsync d1/da/f10 0 2024-04-22T23:35:00.807 INFO:tasks.workunit.client.0.smithi107.stdout:2/139: chown d1/da/d13/f32 651715 1 2024-04-22T23:35:00.807 INFO:tasks.workunit.client.0.smithi107.stdout:2/140: dread - d1/f5 zero size 2024-04-22T23:35:00.811 INFO:tasks.workunit.client.0.smithi107.stdout:9/122: mkdir d5/d14/d28 0 2024-04-22T23:35:00.812 INFO:tasks.workunit.client.0.smithi107.stdout:6/137: getdents db/d20 0 2024-04-22T23:35:00.815 INFO:tasks.workunit.client.0.smithi107.stdout:8/58: truncate d0/fb 2851096 0 2024-04-22T23:35:00.817 INFO:tasks.workunit.client.0.smithi107.stdout:5/118: rename d9/db/de/c1e to d9/c2a 0 2024-04-22T23:35:00.817 INFO:tasks.workunit.client.0.smithi107.stdout:6/138: dread f6 [8388608,4194304] 0 2024-04-22T23:35:00.818 INFO:tasks.workunit.client.0.smithi107.stdout:2/141: creat d1/d2b/f39 x:0 0 0 2024-04-22T23:35:00.818 INFO:tasks.workunit.client.0.smithi107.stdout:2/142: readlink d1/d12/l1b 0 2024-04-22T23:35:00.818 INFO:tasks.workunit.client.0.smithi107.stdout:2/143: chown d1/da/c20 4 1 2024-04-22T23:35:00.818 INFO:tasks.workunit.client.0.smithi107.stdout:2/144: readlink d1/da/d13/l28 0 2024-04-22T23:35:00.819 INFO:tasks.workunit.client.0.smithi107.stdout:7/92: symlink d7/l19 0 2024-04-22T23:35:00.821 INFO:tasks.workunit.client.0.smithi107.stdout:1/84: dwrite d7/f10 [0,4194304] 0 2024-04-22T23:35:00.821 INFO:tasks.workunit.client.0.smithi107.stdout:3/149: dwrite f0 [0,4194304] 0 2024-04-22T23:35:00.821 INFO:tasks.workunit.client.0.smithi107.stdout:5/119: symlink d9/db/de/l2b 0 2024-04-22T23:35:00.826 INFO:tasks.workunit.client.0.smithi107.stdout:2/145: unlink d1/f8 0 2024-04-22T23:35:00.829 INFO:tasks.workunit.client.0.smithi107.stdout:2/146: creat d1/da/f3a x:0 0 0 2024-04-22T23:35:00.833 INFO:tasks.workunit.client.0.smithi107.stdout:9/123: dwrite d5/d7/ff [0,4194304] 0 2024-04-22T23:35:00.834 INFO:tasks.workunit.client.0.smithi107.stdout:2/147: link d1/d12/f14 d1/d12/d35/f3b 0 2024-04-22T23:35:00.834 INFO:tasks.workunit.client.0.smithi107.stdout:2/148: truncate d1/d12/f34 989441 0 2024-04-22T23:35:00.834 INFO:tasks.workunit.client.0.smithi107.stdout:2/149: creat d1/f3c x:0 0 0 2024-04-22T23:35:00.836 INFO:tasks.workunit.client.0.smithi107.stdout:1/85: link d7/cd d7/c1d 0 2024-04-22T23:35:00.836 INFO:tasks.workunit.client.0.smithi107.stdout:1/86: dread - d7/f1b zero size 2024-04-22T23:35:00.836 INFO:tasks.workunit.client.0.smithi107.stdout:8/59: dwrite d0/d6/f10 [0,4194304] 0 2024-04-22T23:35:00.837 INFO:tasks.workunit.client.0.smithi107.stdout:7/93: dread d7/f13 [0,4194304] 0 2024-04-22T23:35:00.837 INFO:tasks.workunit.client.0.smithi107.stdout:3/150: dread db/d21/d27/d22/d25/f34 [0,4194304] 0 2024-04-22T23:35:00.837 INFO:tasks.workunit.client.0.smithi107.stdout:3/151: dread - db/d21/d27/d15/f30 zero size 2024-04-22T23:35:00.840 INFO:tasks.workunit.client.0.smithi107.stdout:1/87: dread d7/d19/f1a [0,4194304] 0 2024-04-22T23:35:00.841 INFO:tasks.workunit.client.0.smithi107.stdout:1/88: chown d7/c1d 51723133 1 2024-04-22T23:35:00.841 INFO:tasks.workunit.client.0.smithi107.stdout:1/89: truncate d7/f15 982190 0 2024-04-22T23:35:00.841 INFO:tasks.workunit.client.0.smithi107.stdout:1/90: stat d7 0 2024-04-22T23:35:00.841 INFO:tasks.workunit.client.0.smithi107.stdout:9/124: mkdir d5/d29 0 2024-04-22T23:35:00.843 INFO:tasks.workunit.client.0.smithi107.stdout:7/94: write d7/f13 [1690613,49778] 0 2024-04-22T23:35:00.845 INFO:tasks.workunit.client.0.smithi107.stdout:8/60: mknod d0/d7/d8/c14 0 2024-04-22T23:35:00.846 INFO:tasks.workunit.client.0.smithi107.stdout:3/152: mkdir db/d21/d27/d36 0 2024-04-22T23:35:00.847 INFO:tasks.workunit.client.0.smithi107.stdout:1/91: mknod d7/d19/d1c/c1e 0 2024-04-22T23:35:00.848 INFO:tasks.workunit.client.0.smithi107.stdout:2/150: truncate d1/f11 676556 0 2024-04-22T23:35:00.850 INFO:tasks.workunit.client.0.smithi107.stdout:5/120: dwrite d9/fd [0,4194304] 0 2024-04-22T23:35:00.850 INFO:tasks.workunit.client.0.smithi107.stdout:5/121: chown d9/db/f10 796232 1 2024-04-22T23:35:00.850 INFO:tasks.workunit.client.0.smithi107.stdout:7/95: creat d7/f1a x:0 0 0 2024-04-22T23:35:00.853 INFO:tasks.workunit.client.0.smithi107.stdout:8/61: link d0/d7/df/c13 d0/d7/c15 0 2024-04-22T23:35:00.854 INFO:tasks.workunit.client.0.smithi107.stdout:9/125: write d5/f13 [325983,63118] 0 2024-04-22T23:35:00.855 INFO:tasks.workunit.client.0.smithi107.stdout:1/92: symlink d7/d19/l1f 0 2024-04-22T23:35:00.855 INFO:tasks.workunit.client.0.smithi107.stdout:1/93: stat d7/d19/l1f 0 2024-04-22T23:35:00.855 INFO:tasks.workunit.client.0.smithi107.stdout:1/94: chown d7/d19/d1c 0 1 2024-04-22T23:35:00.856 INFO:tasks.workunit.client.0.smithi107.stdout:2/151: mkdir d1/d2b/d3d 0 2024-04-22T23:35:00.856 INFO:tasks.workunit.client.0.smithi107.stdout:3/153: write f0 [4093485,61744] 0 2024-04-22T23:35:00.856 INFO:tasks.workunit.client.0.smithi107.stdout:3/154: getdents db/d21/d27/d36 0 2024-04-22T23:35:00.859 INFO:tasks.workunit.client.0.smithi107.stdout:4/107: sync 2024-04-22T23:35:00.861 INFO:tasks.workunit.client.0.smithi107.stdout:2/152: read d1/da/f1f [1516,33582] 0 2024-04-22T23:35:00.861 INFO:tasks.workunit.client.0.smithi107.stdout:2/153: creat d1/da/f3e x:0 0 0 2024-04-22T23:35:00.861 INFO:tasks.workunit.client.0.smithi107.stdout:2/154: write d1/da/f1f [885641,93780] 0 2024-04-22T23:35:00.861 INFO:tasks.workunit.client.0.smithi107.stdout:2/155: chown d1/f1a 7 1 2024-04-22T23:35:00.866 INFO:tasks.workunit.client.0.smithi107.stdout:9/126: symlink d5/d14/l2a 0 2024-04-22T23:35:00.868 INFO:tasks.workunit.client.0.smithi107.stdout:3/155: creat db/d21/d27/d22/d25/f37 x:0 0 0 2024-04-22T23:35:00.869 INFO:tasks.workunit.client.0.smithi107.stdout:1/95: dwrite d7/f1b [0,4194304] 0 2024-04-22T23:35:00.869 INFO:tasks.workunit.client.0.smithi107.stdout:1/96: write d7/f10 [4742861,22740] 0 2024-04-22T23:35:00.869 INFO:tasks.workunit.client.0.smithi107.stdout:4/108: symlink d1/d21/d29/l2d 0 2024-04-22T23:35:00.871 INFO:tasks.workunit.client.0.smithi107.stdout:2/156: creat d1/d12/d35/f3f x:0 0 0 2024-04-22T23:35:00.871 INFO:tasks.workunit.client.0.smithi107.stdout:2/157: chown d1/lf 42 1 2024-04-22T23:35:00.872 INFO:tasks.workunit.client.0.smithi107.stdout:2/158: fsync d1/d2f/f33 0 2024-04-22T23:35:00.872 INFO:tasks.workunit.client.0.smithi107.stdout:2/159: write d1/f3c [1009101,79033] 0 2024-04-22T23:35:00.872 INFO:tasks.workunit.client.0.smithi107.stdout:2/160: dread - d1/d2f/f33 zero size 2024-04-22T23:35:00.872 INFO:tasks.workunit.client.0.smithi107.stdout:2/161: write d1/da/f2a [679619,53106] 0 2024-04-22T23:35:00.872 INFO:tasks.workunit.client.0.smithi107.stdout:2/162: fsync d1/da/f29 0 2024-04-22T23:35:00.873 INFO:tasks.workunit.client.0.smithi107.stdout:7/96: mkdir d7/d1b 0 2024-04-22T23:35:00.873 INFO:tasks.workunit.client.0.smithi107.stdout:7/97: write d7/f13 [4966010,67471] 0 2024-04-22T23:35:00.874 INFO:tasks.workunit.client.0.smithi107.stdout:8/62: rmdir d0/d7 39 2024-04-22T23:35:00.874 INFO:tasks.workunit.client.0.smithi107.stdout:8/63: chown d0/d7/c15 69812 1 2024-04-22T23:35:00.876 INFO:tasks.workunit.client.0.smithi107.stdout:9/127: link d5/d7/d17/d1b/d1f/l21 d5/d7/d17/d1b/d1f/l2b 0 2024-04-22T23:35:00.877 INFO:tasks.workunit.client.0.smithi107.stdout:3/156: mknod db/d21/d27/d15/d24/c38 0 2024-04-22T23:35:00.879 INFO:tasks.workunit.client.0.smithi107.stdout:0/92: sync 2024-04-22T23:35:00.881 INFO:tasks.workunit.client.0.smithi107.stdout:4/109: mkdir d1/d21/d29/d2e 0 2024-04-22T23:35:00.881 INFO:tasks.workunit.client.0.smithi107.stdout:4/110: truncate d1/f5 4964641 0 2024-04-22T23:35:00.881 INFO:tasks.workunit.client.0.smithi107.stdout:4/111: fsync d1/f5 0 2024-04-22T23:35:00.882 INFO:tasks.workunit.client.0.smithi107.stdout:1/97: dwrite d7/f1b [0,4194304] 0 2024-04-22T23:35:00.882 INFO:tasks.workunit.client.0.smithi107.stdout:2/163: link d1/f6 d1/da/d13/f40 0 2024-04-22T23:35:00.884 INFO:tasks.workunit.client.0.smithi107.stdout:8/64: mknod d0/d7/df/c16 0 2024-04-22T23:35:00.886 INFO:tasks.workunit.client.0.smithi107.stdout:9/128: creat d5/d29/f2c x:0 0 0 2024-04-22T23:35:00.886 INFO:tasks.workunit.client.0.smithi107.stdout:9/129: chown d5/fc 1857603 1 2024-04-22T23:35:00.888 INFO:tasks.workunit.client.0.smithi107.stdout:3/157: link db/f1c db/d21/d27/f39 0 2024-04-22T23:35:00.888 INFO:tasks.workunit.client.0.smithi107.stdout:3/158: chown db/d21/d27/f23 3190 1 2024-04-22T23:35:00.889 INFO:tasks.workunit.client.0.smithi107.stdout:9/130: write d5/d7/fd [322275,88368] 0 2024-04-22T23:35:00.889 INFO:tasks.workunit.client.0.smithi107.stdout:4/112: dread d1/d7/f11 [0,4194304] 0 2024-04-22T23:35:00.889 INFO:tasks.workunit.client.0.smithi107.stdout:0/93: rename d1/d17/c1b to d1/d17/c1e 0 2024-04-22T23:35:00.892 INFO:tasks.workunit.client.0.smithi107.stdout:1/98: mkdir d7/d19/d20 0 2024-04-22T23:35:00.893 INFO:tasks.workunit.client.0.smithi107.stdout:2/164: symlink d1/d12/d35/l41 0 2024-04-22T23:35:00.894 INFO:tasks.workunit.client.0.smithi107.stdout:7/98: getdents d7 0 2024-04-22T23:35:00.894 INFO:tasks.workunit.client.0.smithi107.stdout:7/99: write d7/f18 [572658,4401] 0 2024-04-22T23:35:00.894 INFO:tasks.workunit.client.0.smithi107.stdout:8/65: rmdir d0/d6 39 2024-04-22T23:35:00.895 INFO:tasks.workunit.client.0.smithi107.stdout:3/159: symlink db/d21/d27/l3a 0 2024-04-22T23:35:00.896 INFO:tasks.workunit.client.0.smithi107.stdout:4/113: dread d1/d7/fd [0,4194304] 0 2024-04-22T23:35:00.896 INFO:tasks.workunit.client.0.smithi107.stdout:9/131: symlink d5/l2d 0 2024-04-22T23:35:00.897 INFO:tasks.workunit.client.0.smithi107.stdout:0/94: mknod d1/c1f 0 2024-04-22T23:35:00.898 INFO:tasks.workunit.client.0.smithi107.stdout:1/99: read - d7/f17 zero size 2024-04-22T23:35:00.898 INFO:tasks.workunit.client.0.smithi107.stdout:1/100: fsync d7/d19/f1a 0 2024-04-22T23:35:00.903 INFO:tasks.workunit.client.0.smithi107.stdout:2/165: symlink d1/d2f/l42 0 2024-04-22T23:35:00.903 INFO:tasks.workunit.client.0.smithi107.stdout:2/166: write d1/f3c [1157636,109511] 0 2024-04-22T23:35:00.903 INFO:tasks.workunit.client.0.smithi107.stdout:9/132: dread d5/d7/ff [0,4194304] 0 2024-04-22T23:35:00.904 INFO:tasks.workunit.client.0.smithi107.stdout:7/100: link d7/c10 d7/d1b/c1c 0 2024-04-22T23:35:00.904 INFO:tasks.workunit.client.0.smithi107.stdout:7/101: stat l6 0 2024-04-22T23:35:00.905 INFO:tasks.workunit.client.0.smithi107.stdout:8/66: mkdir d0/d7/df/d17 0 2024-04-22T23:35:00.905 INFO:tasks.workunit.client.0.smithi107.stdout:7/102: fdatasync d7/f18 0 2024-04-22T23:35:00.906 INFO:tasks.workunit.client.0.smithi107.stdout:3/160: truncate db/d21/d27/f20 1141549 0 2024-04-22T23:35:00.907 INFO:tasks.workunit.client.0.smithi107.stdout:4/114: link d1/f28 d1/d8/d23/d27/f2f 0 2024-04-22T23:35:00.909 INFO:tasks.workunit.client.0.smithi107.stdout:0/95: unlink d1/d9/l18 0 2024-04-22T23:35:00.909 INFO:tasks.workunit.client.0.smithi107.stdout:1/101: readlink d7/l16 0 2024-04-22T23:35:00.910 INFO:tasks.workunit.client.0.smithi107.stdout:2/167: mknod d1/da/c43 0 2024-04-22T23:35:00.910 INFO:tasks.workunit.client.0.smithi107.stdout:0/96: write d1/d9/fc [26974,97414] 0 2024-04-22T23:35:00.911 INFO:tasks.workunit.client.0.smithi107.stdout:0/97: write d1/f15 [1746701,6250] 0 2024-04-22T23:35:00.911 INFO:tasks.workunit.client.0.smithi107.stdout:9/133: mkdir d5/d14/d2e 0 2024-04-22T23:35:00.911 INFO:tasks.workunit.client.0.smithi107.stdout:9/134: creat d5/d14/f2f x:0 0 0 2024-04-22T23:35:00.912 INFO:tasks.workunit.client.0.smithi107.stdout:2/168: dread d1/da/d13/f40 [0,4194304] 0 2024-04-22T23:35:00.912 INFO:tasks.workunit.client.0.smithi107.stdout:8/67: mkdir d0/d7/df/d18 0 2024-04-22T23:35:00.912 INFO:tasks.workunit.client.0.smithi107.stdout:8/68: truncate d0/d7/f11 479847 0 2024-04-22T23:35:00.913 INFO:tasks.workunit.client.0.smithi107.stdout:7/103: mknod d7/c1d 0 2024-04-22T23:35:00.913 INFO:tasks.workunit.client.0.smithi107.stdout:7/104: write d7/f18 [616077,104140] 0 2024-04-22T23:35:00.914 INFO:tasks.workunit.client.0.smithi107.stdout:3/161: mknod db/d21/d27/d15/d29/c3b 0 2024-04-22T23:35:00.914 INFO:tasks.workunit.client.0.smithi107.stdout:6/139: sync 2024-04-22T23:35:00.916 INFO:tasks.workunit.client.0.smithi107.stdout:9/135: dread f4 [0,4194304] 0 2024-04-22T23:35:00.916 INFO:tasks.workunit.client.0.smithi107.stdout:4/115: creat d1/d8/d23/d27/f30 x:0 0 0 2024-04-22T23:35:00.919 INFO:tasks.workunit.client.0.smithi107.stdout:1/102: mknod d7/d19/d20/c21 0 2024-04-22T23:35:00.923 INFO:tasks.workunit.client.0.smithi107.stdout:0/98: unlink d1/f13 0 2024-04-22T23:35:00.928 INFO:tasks.workunit.client.0.smithi107.stdout:3/162: link db/c17 db/d21/c3c 0 2024-04-22T23:35:00.928 INFO:tasks.workunit.client.0.smithi107.stdout:3/163: getdents db/d21/d27/d15/d29/d2a 0 2024-04-22T23:35:00.928 INFO:tasks.workunit.client.0.smithi107.stdout:6/140: stat l5 0 2024-04-22T23:35:00.929 INFO:tasks.workunit.client.0.smithi107.stdout:9/136: creat d5/d7/d17/d1b/f30 x:0 0 0 2024-04-22T23:35:00.930 INFO:tasks.workunit.client.0.smithi107.stdout:4/116: symlink d1/d7/l31 0 2024-04-22T23:35:00.930 INFO:tasks.workunit.client.0.smithi107.stdout:4/117: read d1/d8/f25 [229333,92963] 0 2024-04-22T23:35:00.930 INFO:tasks.workunit.client.0.smithi107.stdout:4/118: chown d1/d7/c1c 3 1 2024-04-22T23:35:00.931 INFO:tasks.workunit.client.0.smithi107.stdout:1/103: rename d7/d19/f1a to d7/d19/f22 0 2024-04-22T23:35:00.932 INFO:tasks.workunit.client.0.smithi107.stdout:2/169: dwrite d1/f5 [0,4194304] 0 2024-04-22T23:35:00.932 INFO:tasks.workunit.client.0.smithi107.stdout:2/170: dread - d1/d12/f27 zero size 2024-04-22T23:35:00.932 INFO:tasks.workunit.client.0.smithi107.stdout:2/171: write d1/da/f1f [1376947,19673] 0 2024-04-22T23:35:00.932 INFO:tasks.workunit.client.0.smithi107.stdout:2/172: chown d1/l17 101162798 1 2024-04-22T23:35:00.933 INFO:tasks.workunit.client.0.smithi107.stdout:0/99: mkdir d1/d17/d20 0 2024-04-22T23:35:00.933 INFO:tasks.workunit.client.0.smithi107.stdout:0/100: write d1/f15 [2800591,89674] 0 2024-04-22T23:35:00.933 INFO:tasks.workunit.client.0.smithi107.stdout:8/69: dwrite d0/d7/f11 [0,4194304] 0 2024-04-22T23:35:00.933 INFO:tasks.workunit.client.0.smithi107.stdout:8/70: creat d0/f19 x:0 0 0 2024-04-22T23:35:00.934 INFO:tasks.workunit.client.0.smithi107.stdout:3/164: symlink db/d21/d27/d22/l3d 0 2024-04-22T23:35:00.936 INFO:tasks.workunit.client.0.smithi107.stdout:6/141: unlink db/ff 0 2024-04-22T23:35:00.936 INFO:tasks.workunit.client.0.smithi107.stdout:3/165: dread db/f18 [0,4194304] 0 2024-04-22T23:35:00.939 INFO:tasks.workunit.client.0.smithi107.stdout:4/119: truncate d1/d7/fc 3858911 0 2024-04-22T23:35:00.946 INFO:tasks.workunit.client.0.smithi107.stdout:2/173: dwrite d1/f6 [0,4194304] 0 2024-04-22T23:35:00.949 INFO:tasks.workunit.client.0.smithi107.stdout:0/101: creat d1/d17/f21 x:0 0 0 2024-04-22T23:35:00.950 INFO:tasks.workunit.client.0.smithi107.stdout:5/122: sync 2024-04-22T23:35:00.950 INFO:tasks.workunit.client.0.smithi107.stdout:5/123: truncate f7 6703000 0 2024-04-22T23:35:00.950 INFO:tasks.workunit.client.0.smithi107.stdout:6/142: truncate db/f11 1566285 0 2024-04-22T23:35:00.952 INFO:tasks.workunit.client.0.smithi107.stdout:3/166: mknod db/c3e 0 2024-04-22T23:35:00.954 INFO:tasks.workunit.client.0.smithi107.stdout:9/137: rmdir d5/d7/d17/d1b 39 2024-04-22T23:35:00.955 INFO:tasks.workunit.client.0.smithi107.stdout:1/104: mknod d7/c23 0 2024-04-22T23:35:00.961 INFO:tasks.workunit.client.0.smithi107.stdout:2/174: symlink d1/da/d13/l44 0 2024-04-22T23:35:00.963 INFO:tasks.workunit.client.0.smithi107.stdout:4/120: getdents d1/d8 0 2024-04-22T23:35:00.966 INFO:tasks.workunit.client.0.smithi107.stdout:3/167: write db/f18 [28819,63287] 0 2024-04-22T23:35:00.966 INFO:tasks.workunit.client.0.smithi107.stdout:1/105: creat d7/d19/d20/f24 x:0 0 0 2024-04-22T23:35:00.966 INFO:tasks.workunit.client.0.smithi107.stdout:1/106: chown d7/f1b 83299773 1 2024-04-22T23:35:00.968 INFO:tasks.workunit.client.0.smithi107.stdout:4/121: dread d1/d8/f25 [0,4194304] 0 2024-04-22T23:35:00.968 INFO:tasks.workunit.client.0.smithi107.stdout:9/138: dread d5/f10 [0,4194304] 0 2024-04-22T23:35:00.968 INFO:tasks.workunit.client.0.smithi107.stdout:9/139: creat d5/f31 x:0 0 0 2024-04-22T23:35:00.969 INFO:tasks.workunit.client.0.smithi107.stdout:1/107: read d7/f15 [361166,47738] 0 2024-04-22T23:35:00.971 INFO:tasks.workunit.client.0.smithi107.stdout:3/168: symlink db/l3f 0 2024-04-22T23:35:00.971 INFO:tasks.workunit.client.0.smithi107.stdout:3/169: fdatasync db/d21/d27/f23 0 2024-04-22T23:35:00.971 INFO:tasks.workunit.client.0.smithi107.stdout:3/170: write db/f18 [1909550,34884] 0 2024-04-22T23:35:00.971 INFO:tasks.workunit.client.0.smithi107.stdout:3/171: dread - db/d21/d27/d15/f30 zero size 2024-04-22T23:35:00.971 INFO:tasks.workunit.client.0.smithi107.stdout:3/172: chown l4 44737 1 2024-04-22T23:35:00.971 INFO:tasks.workunit.client.0.smithi107.stdout:3/173: stat f0 0 2024-04-22T23:35:00.972 INFO:tasks.workunit.client.0.smithi107.stdout:4/122: rename d1/d7/fd to d1/d7/f32 0 2024-04-22T23:35:00.974 INFO:tasks.workunit.client.0.smithi107.stdout:3/174: mkdir db/d21/d27/d15/d29/d2a/d40 0 2024-04-22T23:35:00.976 INFO:tasks.workunit.client.0.smithi107.stdout:4/123: creat d1/d7/f33 x:0 0 0 2024-04-22T23:35:00.977 INFO:tasks.workunit.client.0.smithi107.stdout:0/102: fsync d1/d17/f21 0 2024-04-22T23:35:00.977 INFO:tasks.workunit.client.0.smithi107.stdout:0/103: dread - d1/d17/f21 zero size 2024-04-22T23:35:00.979 INFO:tasks.workunit.client.0.smithi107.stdout:4/124: creat d1/d21/d29/f34 x:0 0 0 2024-04-22T23:35:00.982 INFO:tasks.workunit.client.0.smithi107.stdout:0/104: link d1/d9/d14/l19 d1/l22 0 2024-04-22T23:35:00.985 INFO:tasks.workunit.client.0.smithi107.stdout:0/105: mknod d1/d9/c23 0 2024-04-22T23:35:00.985 INFO:tasks.workunit.client.0.smithi107.stdout:0/106: chown d1/d9/c10 0 1 2024-04-22T23:35:00.987 INFO:tasks.workunit.client.0.smithi107.stdout:0/107: symlink d1/d17/d20/l24 0 2024-04-22T23:35:00.987 INFO:tasks.workunit.client.0.smithi107.stdout:0/108: rename d1/d9/d14 to d1/d9/d14/d25 22 2024-04-22T23:35:00.987 INFO:tasks.workunit.client.0.smithi107.stdout:5/124: dwrite f4 [0,4194304] 0 2024-04-22T23:35:00.989 INFO:tasks.workunit.client.0.smithi107.stdout:5/125: link d9/f29 d9/d26/f2c 0 2024-04-22T23:35:00.989 INFO:tasks.workunit.client.0.smithi107.stdout:5/126: fdatasync d9/db/f10 0 2024-04-22T23:35:00.990 INFO:tasks.workunit.client.0.smithi107.stdout:0/109: dread d1/d9/fc [0,4194304] 0 2024-04-22T23:35:00.990 INFO:tasks.workunit.client.0.smithi107.stdout:0/110: chown d1/d9/cd 210695077 1 2024-04-22T23:35:00.990 INFO:tasks.workunit.client.0.smithi107.stdout:0/111: read d1/d9/fc [713987,88196] 0 2024-04-22T23:35:00.990 INFO:tasks.workunit.client.0.smithi107.stdout:0/112: write d1/d9/fc [2720720,5] 0 2024-04-22T23:35:00.991 INFO:tasks.workunit.client.0.smithi107.stdout:5/127: mknod d9/c2d 0 2024-04-22T23:35:00.994 INFO:tasks.workunit.client.0.smithi107.stdout:0/113: link d1/d9/c23 d1/d17/c26 0 2024-04-22T23:35:00.994 INFO:tasks.workunit.client.0.smithi107.stdout:2/175: dwrite d1/da/f2a [0,4194304] 0 2024-04-22T23:35:00.994 INFO:tasks.workunit.client.0.smithi107.stdout:6/143: dwrite f6 [4194304,4194304] 0 2024-04-22T23:35:00.998 INFO:tasks.workunit.client.0.smithi107.stdout:5/128: rename c5 to d9/db/de/c2e 0 2024-04-22T23:35:00.998 INFO:tasks.workunit.client.0.smithi107.stdout:1/108: dwrite f2 [0,4194304] 0 2024-04-22T23:35:00.998 INFO:tasks.workunit.client.0.smithi107.stdout:1/109: write d7/f15 [1325348,117910] 0 2024-04-22T23:35:00.998 INFO:tasks.workunit.client.0.smithi107.stdout:4/125: dwrite d1/d8/f1f [0,4194304] 0 2024-04-22T23:35:01.005 INFO:tasks.workunit.client.0.smithi107.stdout:9/140: getdents d5 0 2024-04-22T23:35:01.005 INFO:tasks.workunit.client.0.smithi107.stdout:9/141: read f4 [2674978,12075] 0 2024-04-22T23:35:01.005 INFO:tasks.workunit.client.0.smithi107.stdout:9/142: write d5/d7/fb [566904,96068] 0 2024-04-22T23:35:01.006 INFO:tasks.workunit.client.0.smithi107.stdout:6/144: write f6 [7854143,44748] 0 2024-04-22T23:35:01.012 INFO:tasks.workunit.client.0.smithi107.stdout:4/126: dread d1/d8/f1f [0,4194304] 0 2024-04-22T23:35:01.013 INFO:tasks.workunit.client.0.smithi107.stdout:1/110: mknod d7/d19/c25 0 2024-04-22T23:35:01.016 INFO:tasks.workunit.client.0.smithi107.stdout:2/176: dwrite d1/d2f/f33 [0,4194304] 0 2024-04-22T23:35:01.017 INFO:tasks.workunit.client.1.smithi157.stdout:5/450: sync 2024-04-22T23:35:01.017 INFO:tasks.workunit.client.1.smithi157.stdout:8/291: sync 2024-04-22T23:35:01.017 INFO:tasks.workunit.client.1.smithi157.stdout:9/385: sync 2024-04-22T23:35:01.017 INFO:tasks.workunit.client.1.smithi157.stdout:0/382: sync 2024-04-22T23:35:01.017 INFO:tasks.workunit.client.1.smithi157.stdout:8/292: chown d2/d6/d21/d5f/l1d 102066495 1 2024-04-22T23:35:01.017 INFO:tasks.workunit.client.1.smithi157.stdout:7/340: sync 2024-04-22T23:35:01.017 INFO:tasks.workunit.client.1.smithi157.stdout:2/542: sync 2024-04-22T23:35:01.018 INFO:tasks.workunit.client.1.smithi157.stdout:3/369: sync 2024-04-22T23:35:01.018 INFO:tasks.workunit.client.1.smithi157.stdout:6/589: sync 2024-04-22T23:35:01.018 INFO:tasks.workunit.client.1.smithi157.stdout:4/578: sync 2024-04-22T23:35:01.019 INFO:tasks.workunit.client.1.smithi157.stdout:1/510: sync 2024-04-22T23:35:01.019 INFO:tasks.workunit.client.1.smithi157.stdout:4/579: write d3/d96/d81/f99 [5430897,112250] 0 2024-04-22T23:35:01.019 INFO:tasks.workunit.client.0.smithi107.stdout:5/129: dwrite d9/db/f13 [4194304,4194304] 0 2024-04-22T23:35:01.019 INFO:tasks.workunit.client.0.smithi107.stdout:5/130: fsync d9/f24 0 2024-04-22T23:35:01.019 INFO:tasks.workunit.client.0.smithi107.stdout:5/131: fsync d9/db/de/f1f 0 2024-04-22T23:35:01.019 INFO:tasks.workunit.client.0.smithi107.stdout:5/132: stat d9/d26/f2c 0 2024-04-22T23:35:01.019 INFO:tasks.workunit.client.0.smithi107.stdout:5/133: readlink d9/l17 0 2024-04-22T23:35:01.019 INFO:tasks.workunit.client.0.smithi107.stdout:2/177: write d1/da/f1f [285046,103464] 0 2024-04-22T23:35:01.019 INFO:tasks.workunit.client.0.smithi107.stdout:2/178: creat d1/d12/d35/f45 x:0 0 0 2024-04-22T23:35:01.019 INFO:tasks.workunit.client.0.smithi107.stdout:2/179: creat d1/d12/f46 x:0 0 0 2024-04-22T23:35:01.021 INFO:tasks.workunit.client.0.smithi107.stdout:4/127: rename d1/d21/d29/c2a to d1/d21/d29/c35 0 2024-04-22T23:35:01.022 INFO:tasks.workunit.client.0.smithi107.stdout:1/111: rmdir d7/d19 39 2024-04-22T23:35:01.026 INFO:tasks.workunit.client.0.smithi107.stdout:4/128: dread d1/d7/f32 [0,4194304] 0 2024-04-22T23:35:01.029 INFO:tasks.workunit.client.0.smithi107.stdout:5/134: write d9/f12 [3569887,114520] 0 2024-04-22T23:35:01.029 INFO:tasks.workunit.client.0.smithi107.stdout:5/135: write f3 [1445315,110507] 0 2024-04-22T23:35:01.031 INFO:tasks.workunit.client.1.smithi157.stdout:3/370: dread d0/d3/d34/d3d/f44 [0,4194304] 0 2024-04-22T23:35:01.032 INFO:tasks.workunit.client.0.smithi107.stdout:2/180: mknod d1/da/c47 0 2024-04-22T23:35:01.034 INFO:tasks.workunit.client.0.smithi107.stdout:0/114: rename d1/d17 to d1/d27 0 2024-04-22T23:35:01.034 INFO:tasks.workunit.client.0.smithi107.stdout:0/115: fsync d1/d9/fc 0 2024-04-22T23:35:01.034 INFO:tasks.workunit.client.0.smithi107.stdout:0/116: chown d1/d27 12345662 1 2024-04-22T23:35:01.035 INFO:tasks.workunit.client.0.smithi107.stdout:0/117: chown d1 0 1 2024-04-22T23:35:01.035 INFO:tasks.workunit.client.1.smithi157.stdout:8/293: write d2/d6/f37 [361829,34053] 0 2024-04-22T23:35:01.035 INFO:tasks.workunit.client.0.smithi107.stdout:1/112: creat d7/d19/f26 x:0 0 0 2024-04-22T23:35:01.036 INFO:tasks.workunit.client.0.smithi107.stdout:4/129: creat d1/d21/f36 x:0 0 0 2024-04-22T23:35:01.038 INFO:tasks.workunit.client.0.smithi107.stdout:5/136: truncate d9/f29 2081281 0 2024-04-22T23:35:01.039 INFO:tasks.workunit.client.1.smithi157.stdout:6/590: dwrite d1/d2f/d3b/d6c/d83/fa6 [0,4194304] 0 2024-04-22T23:35:01.041 INFO:tasks.workunit.client.0.smithi107.stdout:0/118: mknod d1/d27/c28 0 2024-04-22T23:35:01.041 INFO:tasks.workunit.client.0.smithi107.stdout:0/119: dread - d1/d27/f21 zero size 2024-04-22T23:35:01.041 INFO:tasks.workunit.client.0.smithi107.stdout:0/120: write d1/d27/f21 [556006,85] 0 2024-04-22T23:35:01.042 INFO:tasks.workunit.client.0.smithi107.stdout:1/113: creat d7/d19/d1c/f27 x:0 0 0 2024-04-22T23:35:01.043 INFO:tasks.workunit.client.0.smithi107.stdout:4/130: rmdir d1/d1b 39 2024-04-22T23:35:01.044 INFO:tasks.workunit.client.0.smithi107.stdout:5/137: truncate d9/db/de/f1f 5772046 0 2024-04-22T23:35:01.044 INFO:tasks.workunit.client.0.smithi107.stdout:5/138: truncate d9/db/de/f23 470033 0 2024-04-22T23:35:01.044 INFO:tasks.workunit.client.0.smithi107.stdout:5/139: chown d9/db/f27 870904 1 2024-04-22T23:35:01.045 INFO:tasks.workunit.client.0.smithi107.stdout:7/105: sync 2024-04-22T23:35:01.051 INFO:tasks.workunit.client.0.smithi107.stdout:0/121: getdents d1/d9/d14 0 2024-04-22T23:35:01.052 INFO:tasks.workunit.client.0.smithi107.stdout:2/181: rename d1/da to d1/d12/d35/d48 0 2024-04-22T23:35:01.052 INFO:tasks.workunit.client.0.smithi107.stdout:2/182: write d1/d12/d35/d48/f2c [5006011,55935] 0 2024-04-22T23:35:01.052 INFO:tasks.workunit.client.0.smithi107.stdout:2/183: chown d1/d2b/d3d 1 1 2024-04-22T23:35:01.053 INFO:tasks.workunit.client.0.smithi107.stdout:1/114: read d7/f1b [2854375,94073] 0 2024-04-22T23:35:01.053 INFO:tasks.workunit.client.0.smithi107.stdout:1/115: stat f2 0 2024-04-22T23:35:01.054 INFO:tasks.workunit.client.0.smithi107.stdout:5/140: rmdir d9/db 39 2024-04-22T23:35:01.054 INFO:tasks.workunit.client.0.smithi107.stdout:5/141: fdatasync d9/db/de/f1f 0 2024-04-22T23:35:01.054 INFO:tasks.workunit.client.0.smithi107.stdout:5/142: fsync f8 0 2024-04-22T23:35:01.054 INFO:tasks.workunit.client.0.smithi107.stdout:5/143: stat d9/l17 0 2024-04-22T23:35:01.054 INFO:tasks.workunit.client.0.smithi107.stdout:5/144: fsync d9/db/f10 0 2024-04-22T23:35:01.054 INFO:tasks.workunit.client.0.smithi107.stdout:5/145: fsync d9/db/f10 0 2024-04-22T23:35:01.060 INFO:tasks.workunit.client.0.smithi107.stdout:4/131: dwrite d1/d21/d29/f34 [0,4194304] 0 2024-04-22T23:35:01.060 INFO:tasks.workunit.client.0.smithi107.stdout:2/184: mkdir d1/d2b/d49 0 2024-04-22T23:35:01.060 INFO:tasks.workunit.client.0.smithi107.stdout:2/185: stat d1/d12/d35/d48/f2a 0 2024-04-22T23:35:01.060 INFO:tasks.workunit.client.0.smithi107.stdout:2/186: write d1/d12/d35/d48/f2c [5119962,5579] 0 2024-04-22T23:35:01.061 INFO:tasks.workunit.client.0.smithi107.stdout:2/187: chown d1/d12/d35/d48/d13/l44 14240 1 2024-04-22T23:35:01.062 INFO:tasks.workunit.client.0.smithi107.stdout:5/146: symlink d9/db/l2f 0 2024-04-22T23:35:01.064 INFO:tasks.workunit.client.0.smithi107.stdout:4/132: mknod d1/c37 0 2024-04-22T23:35:01.064 INFO:tasks.workunit.client.0.smithi107.stdout:4/133: readlink d1/d7/l31 0 2024-04-22T23:35:01.066 INFO:tasks.workunit.client.0.smithi107.stdout:4/134: rename d1/d8/l16 to d1/d8/d23/d27/l38 0 2024-04-22T23:35:01.066 INFO:tasks.workunit.client.0.smithi107.stdout:4/135: creat d1/d21/d29/f39 x:0 0 0 2024-04-22T23:35:01.066 INFO:tasks.workunit.client.0.smithi107.stdout:4/136: dread - d1/d8/d23/d27/f30 zero size 2024-04-22T23:35:01.067 INFO:tasks.workunit.client.0.smithi107.stdout:0/122: dwrite d1/d27/f21 [0,4194304] 0 2024-04-22T23:35:01.068 INFO:tasks.workunit.client.0.smithi107.stdout:1/116: dread d7/f10 [0,4194304] 0 2024-04-22T23:35:01.070 INFO:tasks.workunit.client.1.smithi157.stdout:9/386: symlink d4/d15/d23/d67/l6c 0 2024-04-22T23:35:01.070 INFO:tasks.workunit.client.0.smithi107.stdout:2/188: dread d1/d2f/f33 [0,4194304] 0 2024-04-22T23:35:01.071 INFO:tasks.workunit.client.1.smithi157.stdout:7/341: creat d0/d9/db/d35/f76 x:0 0 0 2024-04-22T23:35:01.072 INFO:tasks.workunit.client.0.smithi107.stdout:4/137: symlink d1/d8/l3a 0 2024-04-22T23:35:01.072 INFO:tasks.workunit.client.0.smithi107.stdout:4/138: fsync d1/f28 0 2024-04-22T23:35:01.072 INFO:tasks.workunit.client.0.smithi107.stdout:4/139: fsync d1/d8/f25 0 2024-04-22T23:35:01.074 INFO:tasks.workunit.client.1.smithi157.stdout:2/543: truncate d7/d8/d30/f9c 3459718 0 2024-04-22T23:35:01.076 INFO:tasks.workunit.client.1.smithi157.stdout:1/511: mkdir d4/d17/d34/d3b/d50/dad 0 2024-04-22T23:35:01.077 INFO:tasks.workunit.client.0.smithi107.stdout:0/123: creat d1/d9/f29 x:0 0 0 2024-04-22T23:35:01.080 INFO:tasks.workunit.client.0.smithi107.stdout:2/189: symlink d1/d12/l4a 0 2024-04-22T23:35:01.080 INFO:tasks.workunit.client.0.smithi107.stdout:4/140: getdents d1/d8/d23 0 2024-04-22T23:35:01.081 INFO:tasks.workunit.client.0.smithi107.stdout:0/124: mkdir d1/d2a 0 2024-04-22T23:35:01.083 INFO:tasks.workunit.client.0.smithi107.stdout:1/117: rename d7/c18 to d7/d19/c28 0 2024-04-22T23:35:01.083 INFO:tasks.workunit.client.0.smithi107.stdout:1/118: creat d7/d19/d1c/f29 x:0 0 0 2024-04-22T23:35:01.083 INFO:tasks.workunit.client.0.smithi107.stdout:1/119: write d7/d19/d1c/f27 [571806,112403] 0 2024-04-22T23:35:01.083 INFO:tasks.workunit.client.0.smithi107.stdout:1/120: chown l6 12 1 2024-04-22T23:35:01.083 INFO:tasks.workunit.client.0.smithi107.stdout:1/121: write d7/d19/d1c/f29 [1001112,97620] 0 2024-04-22T23:35:01.083 INFO:tasks.workunit.client.0.smithi107.stdout:5/147: dwrite d9/fd [0,4194304] 0 2024-04-22T23:35:01.084 INFO:tasks.workunit.client.0.smithi107.stdout:5/148: chown f3 130749 1 2024-04-22T23:35:01.084 INFO:tasks.workunit.client.0.smithi107.stdout:5/149: chown d9/db/de/l1a 34 1 2024-04-22T23:35:01.084 INFO:tasks.workunit.client.0.smithi107.stdout:2/190: mknod d1/c4b 0 2024-04-22T23:35:01.085 INFO:tasks.workunit.client.0.smithi107.stdout:0/125: mknod d1/d27/c2b 0 2024-04-22T23:35:01.085 INFO:tasks.workunit.client.0.smithi107.stdout:2/191: dread d1/d12/f34 [0,4194304] 0 2024-04-22T23:35:01.085 INFO:tasks.workunit.client.0.smithi107.stdout:2/192: creat d1/f4c x:0 0 0 2024-04-22T23:35:01.085 INFO:tasks.workunit.client.0.smithi107.stdout:2/193: truncate d1/d12/f26 1490956 0 2024-04-22T23:35:01.085 INFO:tasks.workunit.client.0.smithi107.stdout:2/194: dread - d1/d2b/f39 zero size 2024-04-22T23:35:01.085 INFO:tasks.workunit.client.0.smithi107.stdout:2/195: stat d1/d12/d35/d48/d13/c21 0 2024-04-22T23:35:01.085 INFO:tasks.workunit.client.0.smithi107.stdout:2/196: stat d1/d2b 0 2024-04-22T23:35:01.085 INFO:tasks.workunit.client.0.smithi107.stdout:2/197: readlink d1/d2f/l42 0 2024-04-22T23:35:01.086 INFO:tasks.workunit.client.0.smithi107.stdout:1/122: creat d7/f2a x:0 0 0 2024-04-22T23:35:01.087 INFO:tasks.workunit.client.1.smithi157.stdout:9/387: symlink d4/d15/d55/l6d 0 2024-04-22T23:35:01.087 INFO:tasks.workunit.client.1.smithi157.stdout:9/388: creat d4/d15/d55/f6e x:0 0 0 2024-04-22T23:35:01.087 INFO:tasks.workunit.client.1.smithi157.stdout:2/544: mknod d7/d8/d8e/d94/cbd 0 2024-04-22T23:35:01.087 INFO:tasks.workunit.client.1.smithi157.stdout:2/545: truncate d7/d8/d8e/f88 48348 0 2024-04-22T23:35:01.088 INFO:tasks.workunit.client.0.smithi107.stdout:5/150: link d9/f29 d9/d26/f30 0 2024-04-22T23:35:01.088 INFO:tasks.workunit.client.0.smithi107.stdout:5/151: creat d9/f31 x:0 0 0 2024-04-22T23:35:01.089 INFO:tasks.workunit.client.0.smithi107.stdout:0/126: mkdir d1/d27/d20/d2c 0 2024-04-22T23:35:01.089 INFO:tasks.workunit.client.0.smithi107.stdout:8/71: sync 2024-04-22T23:35:01.090 INFO:tasks.workunit.client.1.smithi157.stdout:7/342: dwrite d0/d9/db/d35/f76 [0,4194304] 0 2024-04-22T23:35:01.091 INFO:tasks.workunit.client.0.smithi107.stdout:2/198: symlink d1/l4d 0 2024-04-22T23:35:01.092 INFO:tasks.workunit.client.1.smithi157.stdout:9/389: symlink d4/l6f 0 2024-04-22T23:35:01.094 INFO:tasks.workunit.client.0.smithi107.stdout:0/127: symlink d1/l2d 0 2024-04-22T23:35:01.095 INFO:tasks.workunit.client.0.smithi107.stdout:1/123: link d7/l8 d7/d19/l2b 0 2024-04-22T23:35:01.095 INFO:tasks.workunit.client.0.smithi107.stdout:1/124: write d7/f15 [1352326,64244] 0 2024-04-22T23:35:01.096 INFO:tasks.workunit.client.0.smithi107.stdout:1/125: chown d7/l16 176822 1 2024-04-22T23:35:01.096 INFO:tasks.workunit.client.0.smithi107.stdout:0/128: mkdir d1/d27/d20/d2c/d2e 0 2024-04-22T23:35:01.097 INFO:tasks.workunit.client.0.smithi107.stdout:1/126: rename d7/d19/f22 to d7/d19/f2c 0 2024-04-22T23:35:01.099 INFO:tasks.workunit.client.0.smithi107.stdout:0/129: symlink d1/d27/d20/d2c/l2f 0 2024-04-22T23:35:01.099 INFO:tasks.workunit.client.0.smithi107.stdout:8/72: write d0/d7/f11 [3889104,84743] 0 2024-04-22T23:35:01.100 INFO:tasks.workunit.client.0.smithi107.stdout:8/73: mknod d0/d7/df/c1a 0 2024-04-22T23:35:01.100 INFO:tasks.workunit.client.0.smithi107.stdout:8/74: chown d0/f19 109 1 2024-04-22T23:35:01.101 INFO:tasks.workunit.client.0.smithi107.stdout:0/130: symlink d1/d27/d20/d2c/l30 0 2024-04-22T23:35:01.102 INFO:tasks.workunit.client.0.smithi107.stdout:2/199: dread d1/d12/f14 [0,4194304] 0 2024-04-22T23:35:01.102 INFO:tasks.workunit.client.0.smithi107.stdout:2/200: fdatasync d1/d12/d35/d48/d13/f40 0 2024-04-22T23:35:01.102 INFO:tasks.workunit.client.0.smithi107.stdout:8/75: symlink d0/d6/l1b 0 2024-04-22T23:35:01.103 INFO:tasks.workunit.client.0.smithi107.stdout:0/131: symlink d1/d27/d20/d2c/d2e/l31 0 2024-04-22T23:35:01.104 INFO:tasks.workunit.client.0.smithi107.stdout:5/152: read f8 [6775638,30867] 0 2024-04-22T23:35:01.106 INFO:tasks.workunit.client.1.smithi157.stdout:2/546: dwrite d7/d8/d95/fa8 [0,4194304] 0 2024-04-22T23:35:01.106 INFO:tasks.workunit.client.0.smithi107.stdout:2/201: link d1/f1a d1/d2b/f4e 0 2024-04-22T23:35:01.107 INFO:tasks.workunit.client.0.smithi107.stdout:0/132: truncate d1/f15 34204 0 2024-04-22T23:35:01.107 INFO:tasks.workunit.client.0.smithi107.stdout:0/133: getdents d1/d9/d14 0 2024-04-22T23:35:01.107 INFO:tasks.workunit.client.0.smithi107.stdout:0/134: chown d1/d9/d14/l19 473 1 2024-04-22T23:35:01.108 INFO:tasks.workunit.client.1.smithi157.stdout:1/512: rename d4/d17/d34/d3b/d52/d65/d8c to d4/d17/d34/d3b/d52/d65/dae 0 2024-04-22T23:35:01.110 INFO:tasks.workunit.client.0.smithi107.stdout:2/202: mkdir d1/d12/d35/d48/d13/d4f 0 2024-04-22T23:35:01.112 INFO:tasks.workunit.client.0.smithi107.stdout:8/76: dread d0/f3 [0,4194304] 0 2024-04-22T23:35:01.113 INFO:tasks.workunit.client.0.smithi107.stdout:8/77: creat d0/d7/df/d17/f1c x:0 0 0 2024-04-22T23:35:01.115 INFO:tasks.workunit.client.0.smithi107.stdout:8/78: creat d0/d7/d8/f1d x:0 0 0 2024-04-22T23:35:01.140 INFO:tasks.workunit.client.1.smithi157.stdout:3/371: mknod d0/d3/d13/d21/c73 0 2024-04-22T23:35:01.145 INFO:tasks.workunit.client.1.smithi157.stdout:4/580: mkdir d3/d11/dc1 0 2024-04-22T23:35:01.147 INFO:tasks.workunit.client.1.smithi157.stdout:3/372: write d0/d3/d34/d3d/f69 [636326,293] 0 2024-04-22T23:35:01.147 INFO:tasks.workunit.client.1.smithi157.stdout:9/390: link d4/d15/d23/d3f/l48 d4/d15/l70 0 2024-04-22T23:35:01.148 INFO:tasks.workunit.client.1.smithi157.stdout:3/373: unlink d0/d3/d34/d39/l52 0 2024-04-22T23:35:01.149 INFO:tasks.workunit.client.0.smithi107.stdout:0/135: dwrite d1/d9/fc [0,4194304] 0 2024-04-22T23:35:01.149 INFO:tasks.workunit.client.0.smithi107.stdout:1/127: dwrite d7/d19/f2c [0,4194304] 0 2024-04-22T23:35:01.150 INFO:tasks.workunit.client.0.smithi107.stdout:1/128: chown d7/cd 1 1 2024-04-22T23:35:01.150 INFO:tasks.workunit.client.1.smithi157.stdout:9/391: creat d4/df/d61/f71 x:0 0 0 2024-04-22T23:35:01.152 INFO:tasks.workunit.client.1.smithi157.stdout:9/392: truncate d4/df/f26 1900373 0 2024-04-22T23:35:01.154 INFO:tasks.workunit.client.1.smithi157.stdout:9/393: mkdir d4/df/d72 0 2024-04-22T23:35:01.156 INFO:tasks.workunit.client.1.smithi157.stdout:1/513: link d4/db/f5a d4/d17/d34/faf 0 2024-04-22T23:35:01.159 INFO:tasks.workunit.client.1.smithi157.stdout:1/514: symlink d4/d17/lb0 0 2024-04-22T23:35:01.160 INFO:tasks.workunit.client.1.smithi157.stdout:3/374: dread d0/d3/d34/f5d [8388608,4194304] 0 2024-04-22T23:35:01.162 INFO:tasks.workunit.client.1.smithi157.stdout:1/515: mknod d4/d17/d34/d75/cb1 0 2024-04-22T23:35:01.162 INFO:tasks.workunit.client.0.smithi107.stdout:0/136: dread d1/d9/fc [0,4194304] 0 2024-04-22T23:35:01.164 INFO:tasks.workunit.client.0.smithi107.stdout:5/153: dwrite d9/f29 [0,4194304] 0 2024-04-22T23:35:01.165 INFO:tasks.workunit.client.0.smithi107.stdout:0/137: mknod d1/d9/c32 0 2024-04-22T23:35:01.166 INFO:tasks.workunit.client.1.smithi157.stdout:1/516: getdents d4/d17/d34/d3b 0 2024-04-22T23:35:01.166 INFO:tasks.workunit.client.1.smithi157.stdout:1/517: read d4/d17/d34/d3b/d50/f56 [414641,62904] 0 2024-04-22T23:35:01.168 INFO:tasks.workunit.client.1.smithi157.stdout:2/547: dwrite d7/d8/d30/d23/f48 [0,4194304] 0 2024-04-22T23:35:01.168 INFO:tasks.workunit.client.0.smithi107.stdout:2/203: dwrite d1/f2d [0,4194304] 0 2024-04-22T23:35:01.170 INFO:tasks.workunit.client.1.smithi157.stdout:5/451: rename d1/d5/dc/d43/d4f/d60 to d1/db0 0 2024-04-22T23:35:01.170 INFO:tasks.workunit.client.1.smithi157.stdout:5/452: write d1/db0/f85 [821804,7328] 0 2024-04-22T23:35:01.171 INFO:tasks.workunit.client.0.smithi107.stdout:0/138: symlink d1/d27/d20/l33 0 2024-04-22T23:35:01.173 INFO:tasks.workunit.client.0.smithi107.stdout:0/139: mknod d1/d9/d14/c34 0 2024-04-22T23:35:01.174 INFO:tasks.workunit.client.1.smithi157.stdout:9/394: write d4/df/f62 [3973060,3446] 0 2024-04-22T23:35:01.175 INFO:tasks.workunit.client.1.smithi157.stdout:1/518: mkdir d4/d17/d1d/db2 0 2024-04-22T23:35:01.175 INFO:tasks.workunit.client.1.smithi157.stdout:1/519: write d4/d17/d1d/d33/d41/f67 [3801158,59795] 0 2024-04-22T23:35:01.175 INFO:tasks.workunit.client.1.smithi157.stdout:1/520: fsync d4/db/fe 0 2024-04-22T23:35:01.176 INFO:tasks.workunit.client.0.smithi107.stdout:0/140: mknod d1/d9/c35 0 2024-04-22T23:35:01.176 INFO:tasks.workunit.client.1.smithi157.stdout:8/294: creat d2/f69 x:0 0 0 2024-04-22T23:35:01.181 INFO:tasks.workunit.client.1.smithi157.stdout:0/383: rename d17/d23/d34/c50 to d17/d2b/d4f/d5a/d7c/c83 0 2024-04-22T23:35:01.183 INFO:tasks.workunit.client.1.smithi157.stdout:5/453: link d1/d5/d7/d10/c1c d1/d27/d29/cb1 0 2024-04-22T23:35:01.184 INFO:tasks.workunit.client.1.smithi157.stdout:5/454: creat d1/d5/dc/d5d/d77/fb2 x:0 0 0 2024-04-22T23:35:01.184 INFO:tasks.workunit.client.1.smithi157.stdout:5/455: dread - d1/d5/dc/d43/f88 zero size 2024-04-22T23:35:01.184 INFO:tasks.workunit.client.0.smithi107.stdout:5/154: dread f3 [0,4194304] 0 2024-04-22T23:35:01.185 INFO:tasks.workunit.client.1.smithi157.stdout:9/395: creat d4/d15/f73 x:0 0 0 2024-04-22T23:35:01.187 INFO:tasks.workunit.client.1.smithi157.stdout:1/521: mkdir d4/d17/d1d/d33/d41/d54/db3 0 2024-04-22T23:35:01.187 INFO:tasks.workunit.client.0.smithi107.stdout:5/155: creat d9/d26/f32 x:0 0 0 2024-04-22T23:35:01.190 INFO:tasks.workunit.client.1.smithi157.stdout:8/295: mkdir d2/d34/d6a 0 2024-04-22T23:35:01.190 INFO:tasks.workunit.client.1.smithi157.stdout:8/296: fsync d2/d6/d21/d5f/fd 0 2024-04-22T23:35:01.190 INFO:tasks.workunit.client.1.smithi157.stdout:8/297: truncate d2/d6/d21/d5f/d17/f1e 789257 0 2024-04-22T23:35:01.190 INFO:tasks.workunit.client.1.smithi157.stdout:0/384: dread d17/d2b/d52/f42 [0,4194304] 0 2024-04-22T23:35:01.192 INFO:tasks.workunit.client.1.smithi157.stdout:6/591: link d1/d36/c44 d1/d2f/d54/cb8 0 2024-04-22T23:35:01.192 INFO:tasks.workunit.client.1.smithi157.stdout:7/343: rename d0/d9/c63 to d0/d9/d29/d49/c77 0 2024-04-22T23:35:01.192 INFO:tasks.workunit.client.1.smithi157.stdout:5/456: symlink d1/d5/dc/d43/d5e/lb3 0 2024-04-22T23:35:01.193 INFO:tasks.workunit.client.0.smithi107.stdout:9/143: sync 2024-04-22T23:35:01.194 INFO:tasks.workunit.client.1.smithi157.stdout:6/592: dread d1/d17/f94 [0,4194304] 0 2024-04-22T23:35:01.194 INFO:tasks.workunit.client.1.smithi157.stdout:9/396: creat d4/d15/d23/d27/d47/f74 x:0 0 0 2024-04-22T23:35:01.195 INFO:tasks.workunit.client.1.smithi157.stdout:1/522: mkdir d4/d77/db4 0 2024-04-22T23:35:01.196 INFO:tasks.workunit.client.1.smithi157.stdout:7/344: dread d0/d9/db/f57 [0,4194304] 0 2024-04-22T23:35:01.197 INFO:tasks.workunit.client.1.smithi157.stdout:7/345: write d0/f2 [495023,104008] 0 2024-04-22T23:35:01.197 INFO:tasks.workunit.client.1.smithi157.stdout:7/346: fdatasync d0/d9/db/f61 0 2024-04-22T23:35:01.197 INFO:tasks.workunit.client.1.smithi157.stdout:7/347: write d0/d9/d29/d33/d37/f50 [782416,70660] 0 2024-04-22T23:35:01.197 INFO:tasks.workunit.client.1.smithi157.stdout:0/385: unlink d17/d23/c29 0 2024-04-22T23:35:01.198 INFO:tasks.workunit.client.1.smithi157.stdout:4/581: rename d3/d11/d1d/d73/da3/d31/f6a to d3/d5c/d7b/fc2 0 2024-04-22T23:35:01.198 INFO:tasks.workunit.client.1.smithi157.stdout:4/582: stat d3/f1b 0 2024-04-22T23:35:01.199 INFO:tasks.workunit.client.1.smithi157.stdout:0/386: dread d17/d23/f7a [0,4194304] 0 2024-04-22T23:35:01.199 INFO:tasks.workunit.client.1.smithi157.stdout:0/387: chown d17/c2a 277546982 1 2024-04-22T23:35:01.199 INFO:tasks.workunit.client.1.smithi157.stdout:9/397: mkdir d4/d15/d23/d27/d75 0 2024-04-22T23:35:01.199 INFO:tasks.workunit.client.1.smithi157.stdout:9/398: creat d4/d15/d55/f76 x:0 0 0 2024-04-22T23:35:01.200 INFO:tasks.workunit.client.1.smithi157.stdout:1/523: truncate d4/d17/f64 1157837 0 2024-04-22T23:35:01.200 INFO:tasks.workunit.client.1.smithi157.stdout:7/348: rmdir d0/d9/d10 39 2024-04-22T23:35:01.202 INFO:tasks.workunit.client.0.smithi107.stdout:1/129: dwrite d7/d19/d1c/f27 [0,4194304] 0 2024-04-22T23:35:01.203 INFO:tasks.workunit.client.0.smithi107.stdout:1/130: chown d7/d19/d1c 2395 1 2024-04-22T23:35:01.203 INFO:tasks.workunit.client.0.smithi107.stdout:1/131: readlink d7/l16 0 2024-04-22T23:35:01.204 INFO:tasks.workunit.client.1.smithi157.stdout:3/375: dwrite d0/d3/d34/d39/f18 [0,4194304] 0 2024-04-22T23:35:01.204 INFO:tasks.workunit.client.1.smithi157.stdout:4/583: mkdir d3/d96/d40/dc3 0 2024-04-22T23:35:01.204 INFO:tasks.workunit.client.1.smithi157.stdout:2/548: dwrite d7/d1f/d26/d8f/fab [0,4194304] 0 2024-04-22T23:35:01.205 INFO:tasks.workunit.client.0.smithi107.stdout:2/204: dwrite d1/f2d [0,4194304] 0 2024-04-22T23:35:01.207 INFO:tasks.workunit.client.0.smithi107.stdout:0/141: dwrite d1/f15 [0,4194304] 0 2024-04-22T23:35:01.207 INFO:tasks.workunit.client.0.smithi107.stdout:5/156: dread d9/d26/f30 [0,4194304] 0 2024-04-22T23:35:01.207 INFO:tasks.workunit.client.0.smithi107.stdout:5/157: read d9/db/de/f1f [4392997,44991] 0 2024-04-22T23:35:01.209 INFO:tasks.workunit.client.0.smithi107.stdout:2/205: creat d1/d2b/d49/f50 x:0 0 0 2024-04-22T23:35:01.210 INFO:tasks.workunit.client.0.smithi107.stdout:3/175: sync 2024-04-22T23:35:01.211 INFO:tasks.workunit.client.1.smithi157.stdout:5/457: read d1/d1f/faf [8364813,50573] 0 2024-04-22T23:35:01.211 INFO:tasks.workunit.client.1.smithi157.stdout:5/458: chown d1/d5/d7/d2f/d45/f53 11814 1 2024-04-22T23:35:01.211 INFO:tasks.workunit.client.1.smithi157.stdout:9/399: mknod d4/d15/d23/d27/d75/c77 0 2024-04-22T23:35:01.212 INFO:tasks.workunit.client.1.smithi157.stdout:4/584: dread d3/d11/d1d/d73/d80/dac/db5/f6d [0,4194304] 0 2024-04-22T23:35:01.212 INFO:tasks.workunit.client.1.smithi157.stdout:6/593: dread d1/d36/fa3 [0,4194304] 0 2024-04-22T23:35:01.212 INFO:tasks.workunit.client.1.smithi157.stdout:6/594: readlink d1/d36/l92 0 2024-04-22T23:35:01.212 INFO:tasks.workunit.client.1.smithi157.stdout:6/595: write d1/d2f/d3b/f2e [237748,42698] 0 2024-04-22T23:35:01.213 INFO:tasks.workunit.client.0.smithi107.stdout:1/132: write d7/d19/d1c/f27 [1478376,11835] 0 2024-04-22T23:35:01.213 INFO:tasks.workunit.client.0.smithi107.stdout:1/133: fsync d7/d19/d20/f24 0 2024-04-22T23:35:01.213 INFO:tasks.workunit.client.0.smithi107.stdout:0/142: unlink d1/d9/d14/c34 0 2024-04-22T23:35:01.215 INFO:tasks.workunit.client.0.smithi107.stdout:5/158: symlink d9/l33 0 2024-04-22T23:35:01.215 INFO:tasks.workunit.client.1.smithi157.stdout:2/549: stat d7/ca1 0 2024-04-22T23:35:01.216 INFO:tasks.workunit.client.1.smithi157.stdout:2/550: dread - d7/d8/d95/f9a zero size 2024-04-22T23:35:01.216 INFO:tasks.workunit.client.0.smithi107.stdout:3/176: stat db/d21/d27/d15/l1f 0 2024-04-22T23:35:01.216 INFO:tasks.workunit.client.0.smithi107.stdout:3/177: chown db/d21/d27/d15/d24/l32 98054979 1 2024-04-22T23:35:01.216 INFO:tasks.workunit.client.0.smithi107.stdout:3/178: fdatasync db/f18 0 2024-04-22T23:35:01.216 INFO:tasks.workunit.client.0.smithi107.stdout:3/179: stat db/d21/d27/d22/d25/f37 0 2024-04-22T23:35:01.217 INFO:tasks.workunit.client.1.smithi157.stdout:8/298: dwrite d2/f5 [0,4194304] 0 2024-04-22T23:35:01.217 INFO:tasks.workunit.client.1.smithi157.stdout:7/349: dread d0/f7 [0,4194304] 0 2024-04-22T23:35:01.217 INFO:tasks.workunit.client.1.smithi157.stdout:7/350: fdatasync d0/d9/d29/d33/d37/f4c 0 2024-04-22T23:35:01.217 INFO:tasks.workunit.client.1.smithi157.stdout:9/400: creat d4/d2d/f78 x:0 0 0 2024-04-22T23:35:01.217 INFO:tasks.workunit.client.1.smithi157.stdout:5/459: dread d1/d5/d7/d2f/d45/d54/d59/f6a [0,4194304] 0 2024-04-22T23:35:01.219 INFO:tasks.workunit.client.1.smithi157.stdout:1/524: dread d4/f23 [0,4194304] 0 2024-04-22T23:35:01.219 INFO:tasks.workunit.client.1.smithi157.stdout:1/525: read - d4/d17/d1d/d33/d41/f69 zero size 2024-04-22T23:35:01.219 INFO:tasks.workunit.client.1.smithi157.stdout:1/526: chown d4/db/d3d/fa1 12 1 2024-04-22T23:35:01.219 INFO:tasks.workunit.client.0.smithi107.stdout:0/143: creat d1/d27/d20/f36 x:0 0 0 2024-04-22T23:35:01.219 INFO:tasks.workunit.client.1.smithi157.stdout:2/551: creat d7/d8/d30/d23/fbe x:0 0 0 2024-04-22T23:35:01.220 INFO:tasks.workunit.client.1.smithi157.stdout:2/552: chown d7/d8/d30/d23/f7f 444475384 1 2024-04-22T23:35:01.220 INFO:tasks.workunit.client.1.smithi157.stdout:2/553: write d7/d8/d30/f83 [1143290,115460] 0 2024-04-22T23:35:01.220 INFO:tasks.workunit.client.0.smithi107.stdout:5/159: unlink d9/c14 0 2024-04-22T23:35:01.221 INFO:tasks.workunit.client.0.smithi107.stdout:9/144: dwrite d5/d7/d17/d1b/f30 [0,4194304] 0 2024-04-22T23:35:01.223 INFO:tasks.workunit.client.1.smithi157.stdout:6/596: write d1/d36/fa3 [1972617,118764] 0 2024-04-22T23:35:01.223 INFO:tasks.workunit.client.0.smithi107.stdout:3/180: creat db/d21/d27/f41 x:0 0 0 2024-04-22T23:35:01.223 INFO:tasks.workunit.client.0.smithi107.stdout:3/181: dread - db/d21/d27/d15/f30 zero size 2024-04-22T23:35:01.223 INFO:tasks.workunit.client.0.smithi107.stdout:3/182: stat db/d21/d27/d36 0 2024-04-22T23:35:01.224 INFO:tasks.workunit.client.1.smithi157.stdout:7/351: creat d0/d9/d42/d55/f78 x:0 0 0 2024-04-22T23:35:01.226 INFO:tasks.workunit.client.0.smithi107.stdout:1/134: rmdir d7/d19/d20 39 2024-04-22T23:35:01.226 INFO:tasks.workunit.client.0.smithi107.stdout:1/135: readlink d7/d19/l2b 0 2024-04-22T23:35:01.226 INFO:tasks.workunit.client.1.smithi157.stdout:9/401: creat d4/f79 x:0 0 0 2024-04-22T23:35:01.226 INFO:tasks.workunit.client.1.smithi157.stdout:9/402: write d4/df/d61/f66 [2985732,45440] 0 2024-04-22T23:35:01.228 INFO:tasks.workunit.client.0.smithi107.stdout:9/145: symlink d5/d7/l32 0 2024-04-22T23:35:01.230 INFO:tasks.workunit.client.1.smithi157.stdout:3/376: dwrite d0/f41 [0,4194304] 0 2024-04-22T23:35:01.230 INFO:tasks.workunit.client.1.smithi157.stdout:3/377: chown d0/l68 55203237 1 2024-04-22T23:35:01.230 INFO:tasks.workunit.client.1.smithi157.stdout:3/378: truncate d0/d3/d34/d39/f32 5008951 0 2024-04-22T23:35:01.231 INFO:tasks.workunit.client.0.smithi107.stdout:3/183: rename db/d21/d27/d22/d25/f37 to db/d21/d27/d15/d29/f42 0 2024-04-22T23:35:01.231 INFO:tasks.workunit.client.1.smithi157.stdout:5/460: rename d1/f26 to d1/fb4 0 2024-04-22T23:35:01.233 INFO:tasks.workunit.client.0.smithi107.stdout:9/146: dread f1 [0,4194304] 0 2024-04-22T23:35:01.233 INFO:tasks.workunit.client.0.smithi107.stdout:3/184: read db/f1c [209542,127086] 0 2024-04-22T23:35:01.234 INFO:tasks.workunit.client.0.smithi107.stdout:0/144: write d1/f15 [977069,35120] 0 2024-04-22T23:35:01.235 INFO:tasks.workunit.client.0.smithi107.stdout:9/147: symlink d5/d29/l33 0 2024-04-22T23:35:01.236 INFO:tasks.workunit.client.1.smithi157.stdout:6/597: symlink d1/d2f/d54/lb9 0 2024-04-22T23:35:01.237 INFO:tasks.workunit.client.0.smithi107.stdout:3/185: creat db/d21/d27/d15/d29/d2a/f43 x:0 0 0 2024-04-22T23:35:01.237 INFO:tasks.workunit.client.0.smithi107.stdout:3/186: dread - db/d21/d27/f41 zero size 2024-04-22T23:35:01.237 INFO:tasks.workunit.client.0.smithi107.stdout:3/187: readlink db/d21/d27/l35 0 2024-04-22T23:35:01.238 INFO:tasks.workunit.client.1.smithi157.stdout:7/352: symlink d0/d9/d29/l79 0 2024-04-22T23:35:01.238 INFO:tasks.workunit.client.0.smithi107.stdout:3/188: symlink db/d21/l44 0 2024-04-22T23:35:01.239 INFO:tasks.workunit.client.1.smithi157.stdout:5/461: dread d1/fb4 [0,4194304] 0 2024-04-22T23:35:01.240 INFO:tasks.workunit.client.1.smithi157.stdout:0/388: getdents d17/d23/d34 0 2024-04-22T23:35:01.240 INFO:tasks.workunit.client.0.smithi107.stdout:3/189: mkdir db/d21/d27/d15/d29/d2a/d45 0 2024-04-22T23:35:01.241 INFO:tasks.workunit.client.1.smithi157.stdout:7/353: write d0/d9/db/f39 [160509,71010] 0 2024-04-22T23:35:01.241 INFO:tasks.workunit.client.1.smithi157.stdout:7/354: chown d0/f4 28292 1 2024-04-22T23:35:01.241 INFO:tasks.workunit.client.1.smithi157.stdout:7/355: write d0/d9/db/f5a [3399822,73192] 0 2024-04-22T23:35:01.246 INFO:tasks.workunit.client.0.smithi107.stdout:0/145: fsync d1/f15 0 2024-04-22T23:35:01.246 INFO:tasks.workunit.client.0.smithi107.stdout:0/146: write d1/d27/d20/f36 [622967,112801] 0 2024-04-22T23:35:01.249 INFO:tasks.workunit.client.0.smithi107.stdout:0/147: mknod d1/d2a/c37 0 2024-04-22T23:35:01.249 INFO:tasks.workunit.client.0.smithi107.stdout:9/148: write d5/d7/d17/d1b/f30 [1935860,49350] 0 2024-04-22T23:35:01.249 INFO:tasks.workunit.client.0.smithi107.stdout:9/149: truncate d5/d14/f15 677679 0 2024-04-22T23:35:01.249 INFO:tasks.workunit.client.0.smithi107.stdout:9/150: write d5/d7/fd [615072,84498] 0 2024-04-22T23:35:01.249 INFO:tasks.workunit.client.1.smithi157.stdout:2/554: dwrite d7/d8/d95/fa8 [0,4194304] 0 2024-04-22T23:35:01.250 INFO:tasks.workunit.client.1.smithi157.stdout:2/555: readlink d7/d8/l1d 0 2024-04-22T23:35:01.250 INFO:tasks.workunit.client.1.smithi157.stdout:8/299: dwrite d2/d6/d21/d5f/d11/f27 [0,4194304] 0 2024-04-22T23:35:01.253 INFO:tasks.workunit.client.0.smithi107.stdout:0/148: creat d1/d27/f38 x:0 0 0 2024-04-22T23:35:01.254 INFO:tasks.workunit.client.0.smithi107.stdout:0/149: chown d1/d9/c1a 2 1 2024-04-22T23:35:01.255 INFO:tasks.workunit.client.1.smithi157.stdout:9/403: dwrite d4/df/fb [4194304,4194304] 0 2024-04-22T23:35:01.255 INFO:tasks.workunit.client.1.smithi157.stdout:7/356: dread d0/d9/f1d [0,4194304] 0 2024-04-22T23:35:01.255 INFO:tasks.workunit.client.1.smithi157.stdout:7/357: write d0/d9/de/f5f [732,35517] 0 2024-04-22T23:35:01.255 INFO:tasks.workunit.client.0.smithi107.stdout:9/151: rename d5/d7/d17/d1b/d1f/l2b to d5/d24/l34 0 2024-04-22T23:35:01.255 INFO:tasks.workunit.client.0.smithi107.stdout:9/152: write d5/d7/fd [1376623,8555] 0 2024-04-22T23:35:01.255 INFO:tasks.workunit.client.0.smithi107.stdout:9/153: stat d5/d14/d28 0 2024-04-22T23:35:01.255 INFO:tasks.workunit.client.0.smithi107.stdout:9/154: readlink d5/d7/l26 0 2024-04-22T23:35:01.256 INFO:tasks.workunit.client.0.smithi107.stdout:0/150: read d1/d9/fc [2595283,92894] 0 2024-04-22T23:35:01.256 INFO:tasks.workunit.client.0.smithi107.stdout:3/190: getdents db/d21/d27/d15/d29 0 2024-04-22T23:35:01.256 INFO:tasks.workunit.client.0.smithi107.stdout:3/191: symlink db/d21/d27/d36/l46 0 2024-04-22T23:35:01.256 INFO:tasks.workunit.client.0.smithi107.stdout:3/192: readlink db/d21/d27/d22/d25/l2f 0 2024-04-22T23:35:01.257 INFO:tasks.workunit.client.0.smithi107.stdout:3/193: truncate db/d21/d27/f39 46253 0 2024-04-22T23:35:01.259 INFO:tasks.workunit.client.0.smithi107.stdout:3/194: symlink db/d21/d27/d22/d25/l47 0 2024-04-22T23:35:01.259 INFO:tasks.workunit.client.0.smithi107.stdout:3/195: fsync db/f18 0 2024-04-22T23:35:01.261 INFO:tasks.workunit.client.0.smithi107.stdout:3/196: link db/d21/d27/l2c db/d21/d27/d22/l48 0 2024-04-22T23:35:01.262 INFO:tasks.workunit.client.0.smithi107.stdout:3/197: truncate db/d21/d27/f23 1315124 0 2024-04-22T23:35:01.271 INFO:tasks.workunit.client.0.smithi107.stdout:3/198: getdents db/d21/d27/d22 0 2024-04-22T23:35:01.271 INFO:tasks.workunit.client.0.smithi107.stdout:3/199: truncate db/d21/d27/d22/d25/f34 4212309 0 2024-04-22T23:35:01.271 INFO:tasks.workunit.client.0.smithi107.stdout:3/200: stat l4 0 2024-04-22T23:35:01.272 INFO:tasks.workunit.client.0.smithi107.stdout:9/155: dwrite d5/d7/f1d [0,4194304] 0 2024-04-22T23:35:01.272 INFO:tasks.workunit.client.0.smithi107.stdout:9/156: write d5/f31 [898344,93307] 0 2024-04-22T23:35:01.272 INFO:tasks.workunit.client.0.smithi107.stdout:0/151: dwrite d1/d27/f38 [0,4194304] 0 2024-04-22T23:35:01.273 INFO:tasks.workunit.client.0.smithi107.stdout:3/201: creat db/d21/d27/d15/d29/d2a/d45/f49 x:0 0 0 2024-04-22T23:35:01.280 INFO:tasks.workunit.client.0.smithi107.stdout:3/202: link db/c13 db/c4a 0 2024-04-22T23:35:01.280 INFO:tasks.workunit.client.0.smithi107.stdout:3/203: truncate db/d21/d27/f23 2290817 0 2024-04-22T23:35:01.283 INFO:tasks.workunit.client.0.smithi107.stdout:3/204: unlink db/d21/d27/d15/f30 0 2024-04-22T23:35:01.292 INFO:tasks.workunit.client.0.smithi107.stdout:9/157: dwrite d5/f10 [0,4194304] 0 2024-04-22T23:35:01.294 INFO:tasks.workunit.client.0.smithi107.stdout:3/205: dread db/d21/f33 [4194304,4194304] 0 2024-04-22T23:35:01.295 INFO:tasks.workunit.client.0.smithi107.stdout:9/158: symlink d5/d29/l35 0 2024-04-22T23:35:01.295 INFO:tasks.workunit.client.0.smithi107.stdout:9/159: fsync d5/fc 0 2024-04-22T23:35:01.296 INFO:tasks.workunit.client.0.smithi107.stdout:9/160: mkdir d5/d7/d17/d1b/d1f/d36 0 2024-04-22T23:35:01.296 INFO:tasks.workunit.client.0.smithi107.stdout:9/161: chown d5/f11 853245007 1 2024-04-22T23:35:01.296 INFO:tasks.workunit.client.0.smithi107.stdout:0/152: dwrite d1/d9/f29 [0,4194304] 0 2024-04-22T23:35:01.297 INFO:tasks.workunit.client.0.smithi107.stdout:9/162: symlink d5/d7/l37 0 2024-04-22T23:35:01.297 INFO:tasks.workunit.client.0.smithi107.stdout:9/163: fdatasync d5/f10 0 2024-04-22T23:35:01.297 INFO:tasks.workunit.client.0.smithi107.stdout:9/164: write d5/f31 [1445837,103279] 0 2024-04-22T23:35:01.297 INFO:tasks.workunit.client.0.smithi107.stdout:9/165: write d5/f13 [2481413,59642] 0 2024-04-22T23:35:01.302 INFO:tasks.workunit.client.0.smithi107.stdout:0/153: getdents d1/d9 0 2024-04-22T23:35:01.303 INFO:tasks.workunit.client.0.smithi107.stdout:9/166: getdents d5/d7/d17 0 2024-04-22T23:35:01.305 INFO:tasks.workunit.client.0.smithi107.stdout:0/154: creat d1/d27/d20/d2c/d2e/f39 x:0 0 0 2024-04-22T23:35:01.305 INFO:tasks.workunit.client.0.smithi107.stdout:0/155: truncate d1/d27/d20/d2c/d2e/f39 1023708 0 2024-04-22T23:35:01.305 INFO:tasks.workunit.client.0.smithi107.stdout:0/156: write d1/d9/fc [3470983,104093] 0 2024-04-22T23:35:01.306 INFO:tasks.workunit.client.0.smithi107.stdout:9/167: creat d5/d7/d17/d1b/d1f/d36/f38 x:0 0 0 2024-04-22T23:35:01.309 INFO:tasks.workunit.client.0.smithi107.stdout:0/157: rename d1/d9/c1a to d1/d27/d20/d2c/c3a 0 2024-04-22T23:35:01.309 INFO:tasks.workunit.client.0.smithi107.stdout:0/158: write d1/f15 [3370588,101930] 0 2024-04-22T23:35:01.311 INFO:tasks.workunit.client.0.smithi107.stdout:0/159: mknod d1/d9/d14/c3b 0 2024-04-22T23:35:01.311 INFO:tasks.workunit.client.0.smithi107.stdout:3/206: dwrite db/ff [0,4194304] 0 2024-04-22T23:35:01.312 INFO:tasks.workunit.client.0.smithi107.stdout:0/160: symlink d1/d27/d20/l3c 0 2024-04-22T23:35:01.312 INFO:tasks.workunit.client.0.smithi107.stdout:3/207: dread db/d21/d27/f39 [0,4194304] 0 2024-04-22T23:35:01.313 INFO:tasks.workunit.client.1.smithi157.stdout:3/379: rmdir d0/d3 39 2024-04-22T23:35:01.313 INFO:tasks.workunit.client.1.smithi157.stdout:3/380: chown d0/f41 464 1 2024-04-22T23:35:01.313 INFO:tasks.workunit.client.1.smithi157.stdout:3/381: read - d0/d7/d6c/f6f zero size 2024-04-22T23:35:01.319 INFO:tasks.workunit.client.1.smithi157.stdout:6/598: unlink d1/d36/fa0 0 2024-04-22T23:35:01.319 INFO:tasks.workunit.client.1.smithi157.stdout:6/599: write d1/d2f/f4a [2666355,124285] 0 2024-04-22T23:35:01.320 INFO:tasks.workunit.client.1.smithi157.stdout:5/462: fdatasync d1/d5/dc/f38 0 2024-04-22T23:35:01.320 INFO:tasks.workunit.client.1.smithi157.stdout:5/463: write d1/d5/d7/d10/f31 [1858121,81629] 0 2024-04-22T23:35:01.320 INFO:tasks.workunit.client.0.smithi107.stdout:3/208: rename db/c4a to db/d21/d27/d22/c4b 0 2024-04-22T23:35:01.320 INFO:tasks.workunit.client.0.smithi107.stdout:3/209: creat db/d21/d27/d15/d29/d2a/f4c x:0 0 0 2024-04-22T23:35:01.320 INFO:tasks.workunit.client.0.smithi107.stdout:3/210: write db/d21/d27/f39 [666515,17673] 0 2024-04-22T23:35:01.320 INFO:tasks.workunit.client.0.smithi107.stdout:3/211: write db/f18 [63962,4401] 0 2024-04-22T23:35:01.321 INFO:tasks.workunit.client.1.smithi157.stdout:0/389: creat d17/d2b/f84 x:0 0 0 2024-04-22T23:35:01.321 INFO:tasks.workunit.client.0.smithi107.stdout:9/168: dwrite d5/d14/f15 [0,4194304] 0 2024-04-22T23:35:01.322 INFO:tasks.workunit.client.0.smithi107.stdout:9/169: getdents d5/d24 0 2024-04-22T23:35:01.323 INFO:tasks.workunit.client.0.smithi107.stdout:9/170: mknod d5/d7/d17/d1b/c39 0 2024-04-22T23:35:01.324 INFO:tasks.workunit.client.0.smithi107.stdout:9/171: rmdir d5/d7/d17 39 2024-04-22T23:35:01.324 INFO:tasks.workunit.client.1.smithi157.stdout:2/556: rename d7/d8/fa0 to d7/d8/fbf 0 2024-04-22T23:35:01.325 INFO:tasks.workunit.client.0.smithi107.stdout:9/172: fsync d5/d7/d17/d1b/d1f/d36/f38 0 2024-04-22T23:35:01.329 INFO:tasks.workunit.client.1.smithi157.stdout:8/300: mkdir d2/d6/d6b 0 2024-04-22T23:35:01.329 INFO:tasks.workunit.client.0.smithi107.stdout:6/145: sync 2024-04-22T23:35:01.329 INFO:tasks.workunit.client.0.smithi107.stdout:6/146: stat db/f11 0 2024-04-22T23:35:01.330 INFO:tasks.workunit.client.0.smithi107.stdout:3/212: getdents db 0 2024-04-22T23:35:01.331 INFO:tasks.workunit.client.1.smithi157.stdout:5/464: write d1/d5/d7/d2f/f79 [1620672,129248] 0 2024-04-22T23:35:01.331 INFO:tasks.workunit.client.1.smithi157.stdout:5/465: write d1/d5/dc/f38 [1147999,110196] 0 2024-04-22T23:35:01.331 INFO:tasks.workunit.client.1.smithi157.stdout:7/358: creat d0/d9/d42/f7a x:0 0 0 2024-04-22T23:35:01.332 INFO:tasks.workunit.client.1.smithi157.stdout:7/359: chown d0 1042026 1 2024-04-22T23:35:01.332 INFO:tasks.workunit.client.1.smithi157.stdout:7/360: readlink d0/l3 0 2024-04-22T23:35:01.333 INFO:tasks.workunit.client.0.smithi107.stdout:3/213: rename db/d21/d27/d22/d25/c2e to db/d21/d27/d15/d24/c4d 0 2024-04-22T23:35:01.333 INFO:tasks.workunit.client.0.smithi107.stdout:6/147: dread db/f11 [0,4194304] 0 2024-04-22T23:35:01.333 INFO:tasks.workunit.client.0.smithi107.stdout:6/148: write db/f11 [1584100,9590] 0 2024-04-22T23:35:01.333 INFO:tasks.workunit.client.1.smithi157.stdout:3/382: symlink d0/d3/d34/d3d/d57/l74 0 2024-04-22T23:35:01.335 INFO:tasks.workunit.client.0.smithi107.stdout:9/173: dread d5/d7/f16 [0,4194304] 0 2024-04-22T23:35:01.335 INFO:tasks.workunit.client.0.smithi107.stdout:9/174: write d5/fc [404977,1073] 0 2024-04-22T23:35:01.335 INFO:tasks.workunit.client.0.smithi107.stdout:6/149: mknod db/d20/c23 0 2024-04-22T23:35:01.335 INFO:tasks.workunit.client.0.smithi107.stdout:6/150: fsync db/f12 0 2024-04-22T23:35:01.335 INFO:tasks.workunit.client.1.smithi157.stdout:6/600: creat d1/d38/fba x:0 0 0 2024-04-22T23:35:01.337 INFO:tasks.workunit.client.0.smithi107.stdout:9/175: rename d5/d7/d17/d1b/d1f/d36/f38 to d5/d29/f3a 0 2024-04-22T23:35:01.337 INFO:tasks.workunit.client.0.smithi107.stdout:9/176: truncate d5/d29/f2c 240154 0 2024-04-22T23:35:01.338 INFO:tasks.workunit.client.1.smithi157.stdout:0/390: creat d17/d23/d34/d67/d7f/f85 x:0 0 0 2024-04-22T23:35:01.339 INFO:tasks.workunit.client.1.smithi157.stdout:2/557: mknod d7/cc0 0 2024-04-22T23:35:01.341 INFO:tasks.workunit.client.1.smithi157.stdout:8/301: link d2/f48 d2/d6/d21/d5f/d11/d13/f6c 0 2024-04-22T23:35:01.343 INFO:tasks.workunit.client.1.smithi157.stdout:9/404: getdents d4 0 2024-04-22T23:35:01.346 INFO:tasks.workunit.client.0.smithi107.stdout:9/177: dread d5/f13 [0,4194304] 0 2024-04-22T23:35:01.347 INFO:tasks.workunit.client.0.smithi107.stdout:9/178: symlink d5/d14/l3b 0 2024-04-22T23:35:01.348 INFO:tasks.workunit.client.0.smithi107.stdout:9/179: chown d5/d24/l34 922031 1 2024-04-22T23:35:01.348 INFO:tasks.workunit.client.1.smithi157.stdout:5/466: rename d1/d5/d7/d2f/d45/d54/d59/f6a to d1/d5/fb5 0 2024-04-22T23:35:01.348 INFO:tasks.workunit.client.1.smithi157.stdout:5/467: stat d1/db0/f85 0 2024-04-22T23:35:01.348 INFO:tasks.workunit.client.1.smithi157.stdout:6/601: dread d1/d2f/d54/f87 [0,4194304] 0 2024-04-22T23:35:01.348 INFO:tasks.workunit.client.1.smithi157.stdout:7/361: symlink d0/d9/db/df/d60/l7b 0 2024-04-22T23:35:01.349 INFO:tasks.workunit.client.1.smithi157.stdout:7/362: write d0/f45 [858677,116723] 0 2024-04-22T23:35:01.350 INFO:tasks.workunit.client.1.smithi157.stdout:3/383: unlink d0/c4b 0 2024-04-22T23:35:01.350 INFO:tasks.workunit.client.1.smithi157.stdout:3/384: write d0/d3/d34/d3d/f63 [85706,4588] 0 2024-04-22T23:35:01.350 INFO:tasks.workunit.client.1.smithi157.stdout:8/302: dread d2/d6/d21/d5f/fd [0,4194304] 0 2024-04-22T23:35:01.350 INFO:tasks.workunit.client.1.smithi157.stdout:3/385: truncate d0/d3/d34/d3d/f63 412252 0 2024-04-22T23:35:01.351 INFO:tasks.workunit.client.1.smithi157.stdout:0/391: link d17/d23/f27 d17/d2b/f86 0 2024-04-22T23:35:01.351 INFO:tasks.workunit.client.0.smithi107.stdout:6/151: dwrite db/f14 [4194304,4194304] 0 2024-04-22T23:35:01.353 INFO:tasks.workunit.client.1.smithi157.stdout:2/558: creat d7/d8/d30/fc1 x:0 0 0 2024-04-22T23:35:01.355 INFO:tasks.workunit.client.1.smithi157.stdout:5/468: mkdir d1/d5/dc/d43/d5e/db6 0 2024-04-22T23:35:01.356 INFO:tasks.workunit.client.1.smithi157.stdout:8/303: dread d2/d6/d21/d5f/d17/f4f [0,4194304] 0 2024-04-22T23:35:01.356 INFO:tasks.workunit.client.1.smithi157.stdout:5/469: chown d1/fa1 2136971 1 2024-04-22T23:35:01.356 INFO:tasks.workunit.client.1.smithi157.stdout:8/304: fsync d2/d6/d21/d5f/d11/f5a 0 2024-04-22T23:35:01.356 INFO:tasks.workunit.client.1.smithi157.stdout:5/470: write d1/f5a [1508915,87796] 0 2024-04-22T23:35:01.357 INFO:tasks.workunit.client.1.smithi157.stdout:7/363: unlink d0/d9/db/df/c6e 0 2024-04-22T23:35:01.357 INFO:tasks.workunit.client.1.smithi157.stdout:7/364: write d0/d9/d72/f44 [537581,79980] 0 2024-04-22T23:35:01.357 INFO:tasks.workunit.client.1.smithi157.stdout:7/365: fsync d0/f2 0 2024-04-22T23:35:01.357 INFO:tasks.workunit.client.0.smithi107.stdout:6/152: mknod db/c24 0 2024-04-22T23:35:01.358 INFO:tasks.workunit.client.1.smithi157.stdout:3/386: link d0/d7/c20 d0/d3/d34/d3d/d57/d60/c75 0 2024-04-22T23:35:01.358 INFO:tasks.workunit.client.1.smithi157.stdout:9/405: dread d4/d15/d23/d27/d47/f4a [0,4194304] 0 2024-04-22T23:35:01.358 INFO:tasks.workunit.client.1.smithi157.stdout:3/387: chown d0 644 1 2024-04-22T23:35:01.358 INFO:tasks.workunit.client.1.smithi157.stdout:9/406: creat d4/d15/f7a x:0 0 0 2024-04-22T23:35:01.360 INFO:tasks.workunit.client.0.smithi107.stdout:6/153: mknod db/d20/c25 0 2024-04-22T23:35:01.360 INFO:tasks.workunit.client.0.smithi107.stdout:6/154: creat db/f26 x:0 0 0 2024-04-22T23:35:01.360 INFO:tasks.workunit.client.0.smithi107.stdout:6/155: chown db/d20/c23 105 1 2024-04-22T23:35:01.361 INFO:tasks.workunit.client.1.smithi157.stdout:0/392: link d17/d2b/d71/l76 d17/d2b/d71/l87 0 2024-04-22T23:35:01.361 INFO:tasks.workunit.client.1.smithi157.stdout:0/393: creat d17/d23/d34/d67/d7f/f88 x:0 0 0 2024-04-22T23:35:01.361 INFO:tasks.workunit.client.1.smithi157.stdout:0/394: dread - d17/d23/d34/d67/d7f/f77 zero size 2024-04-22T23:35:01.361 INFO:tasks.workunit.client.1.smithi157.stdout:0/395: stat d17/d2b/f86 0 2024-04-22T23:35:01.363 INFO:tasks.workunit.client.0.smithi107.stdout:6/156: write db/f12 [689379,106358] 0 2024-04-22T23:35:01.364 INFO:tasks.workunit.client.1.smithi157.stdout:2/559: mknod d7/d1f/d26/d8f/cc2 0 2024-04-22T23:35:01.364 INFO:tasks.workunit.client.1.smithi157.stdout:8/305: creat d2/d34/d6a/f6d x:0 0 0 2024-04-22T23:35:01.365 INFO:tasks.workunit.client.0.smithi107.stdout:6/157: symlink db/d20/l27 0 2024-04-22T23:35:01.365 INFO:tasks.workunit.client.0.smithi107.stdout:6/158: write db/f1c [400992,88874] 0 2024-04-22T23:35:01.365 INFO:tasks.workunit.client.1.smithi157.stdout:5/471: mknod d1/d5/d7/d2f/d45/d54/d59/cb7 0 2024-04-22T23:35:01.365 INFO:tasks.workunit.client.1.smithi157.stdout:5/472: chown d1/d5/d7/d10/f31 140 1 2024-04-22T23:35:01.365 INFO:tasks.workunit.client.1.smithi157.stdout:5/473: write d1/d27/d29/d4b/f4d [794243,115825] 0 2024-04-22T23:35:01.365 INFO:tasks.workunit.client.1.smithi157.stdout:5/474: fdatasync d1/d5/f4e 0 2024-04-22T23:35:01.369 INFO:tasks.workunit.client.0.smithi107.stdout:6/159: dread f0 [4194304,4194304] 0 2024-04-22T23:35:01.370 INFO:tasks.workunit.client.0.smithi107.stdout:9/180: dwrite d5/d7/ff [4194304,4194304] 0 2024-04-22T23:35:01.371 INFO:tasks.workunit.client.1.smithi157.stdout:6/602: dwrite d1/d17/f94 [0,4194304] 0 2024-04-22T23:35:01.371 INFO:tasks.workunit.client.1.smithi157.stdout:9/407: rename d4/d2d/l60 to d4/df/d57/l7b 0 2024-04-22T23:35:01.371 INFO:tasks.workunit.client.1.smithi157.stdout:9/408: chown d4/f29 141 1 2024-04-22T23:35:01.372 INFO:tasks.workunit.client.1.smithi157.stdout:9/409: write d4/f50 [2859714,53719] 0 2024-04-22T23:35:01.372 INFO:tasks.workunit.client.1.smithi157.stdout:9/410: stat d4/l28 0 2024-04-22T23:35:01.372 INFO:tasks.workunit.client.1.smithi157.stdout:9/411: fsync d4/d15/d55/f6e 0 2024-04-22T23:35:01.372 INFO:tasks.workunit.client.1.smithi157.stdout:9/412: readlink d4/d15/d23/l5e 0 2024-04-22T23:35:01.373 INFO:tasks.workunit.client.1.smithi157.stdout:2/560: write d7/d8/dd/d4d/f99 [3231919,28176] 0 2024-04-22T23:35:01.374 INFO:tasks.workunit.client.0.smithi107.stdout:9/181: truncate f4 845917 0 2024-04-22T23:35:01.374 INFO:tasks.workunit.client.1.smithi157.stdout:8/306: unlink d2/d6/d21/d5f/c57 0 2024-04-22T23:35:01.375 INFO:tasks.workunit.client.1.smithi157.stdout:8/307: fsync d2/d6/f37 0 2024-04-22T23:35:01.375 INFO:tasks.workunit.client.1.smithi157.stdout:5/475: symlink d1/d5/lb8 0 2024-04-22T23:35:01.378 INFO:tasks.workunit.client.1.smithi157.stdout:8/308: write d2/d6/d21/d5f/fd [3327863,43910] 0 2024-04-22T23:35:01.378 INFO:tasks.workunit.client.1.smithi157.stdout:0/396: dwrite d17/d23/d34/f7b [0,4194304] 0 2024-04-22T23:35:01.382 INFO:tasks.workunit.client.1.smithi157.stdout:2/561: write d7/d8/d25/f50 [264266,46463] 0 2024-04-22T23:35:01.384 INFO:tasks.workunit.client.0.smithi107.stdout:9/182: creat d5/d24/f3c x:0 0 0 2024-04-22T23:35:01.384 INFO:tasks.workunit.client.0.smithi107.stdout:6/160: getdents db 0 2024-04-22T23:35:01.386 INFO:tasks.workunit.client.0.smithi107.stdout:6/161: symlink db/d20/l28 0 2024-04-22T23:35:01.386 INFO:tasks.workunit.client.0.smithi107.stdout:6/162: chown db/f15 1120 1 2024-04-22T23:35:01.386 INFO:tasks.workunit.client.0.smithi107.stdout:6/163: chown db/d20 1 1 2024-04-22T23:35:01.386 INFO:tasks.workunit.client.0.smithi107.stdout:6/164: chown db/l19 3634 1 2024-04-22T23:35:01.386 INFO:tasks.workunit.client.0.smithi107.stdout:6/165: dread - db/f21 zero size 2024-04-22T23:35:01.388 INFO:tasks.workunit.client.0.smithi107.stdout:9/183: dread d5/f31 [0,4194304] 0 2024-04-22T23:35:01.388 INFO:tasks.workunit.client.0.smithi107.stdout:6/166: rename db/f14 to db/f29 0 2024-04-22T23:35:01.389 INFO:tasks.workunit.client.0.smithi107.stdout:6/167: write db/f12 [1463903,129615] 0 2024-04-22T23:35:01.390 INFO:tasks.workunit.client.0.smithi107.stdout:9/184: rename d5/c1a to d5/d7/d17/d1b/d1f/d36/c3d 0 2024-04-22T23:35:01.391 INFO:tasks.workunit.client.1.smithi157.stdout:6/603: symlink d1/d2f/d3b/d6c/da7/lbb 0 2024-04-22T23:35:01.392 INFO:tasks.workunit.client.0.smithi107.stdout:9/185: mknod d5/d7/d17/d1b/d1f/d36/c3e 0 2024-04-22T23:35:01.393 INFO:tasks.workunit.client.0.smithi107.stdout:9/186: creat d5/d14/d2e/f3f x:0 0 0 2024-04-22T23:35:01.394 INFO:tasks.workunit.client.1.smithi157.stdout:9/413: link d4/c40 d4/d15/d23/d3f/c7c 0 2024-04-22T23:35:01.395 INFO:tasks.workunit.client.1.smithi157.stdout:9/414: stat d4/df/l17 0 2024-04-22T23:35:01.396 INFO:tasks.workunit.client.1.smithi157.stdout:3/388: rmdir d0 39 2024-04-22T23:35:01.396 INFO:tasks.workunit.client.1.smithi157.stdout:3/389: truncate d0/d3/d34/d3d/d57/f64 420454 0 2024-04-22T23:35:01.396 INFO:tasks.workunit.client.1.smithi157.stdout:3/390: write d0/d3/f1d [4059491,78642] 0 2024-04-22T23:35:01.398 INFO:tasks.workunit.client.0.smithi107.stdout:9/187: fsync d5/d24/f3c 0 2024-04-22T23:35:01.398 INFO:tasks.workunit.client.0.smithi107.stdout:9/188: readlink d5/la 0 2024-04-22T23:35:01.398 INFO:tasks.workunit.client.0.smithi107.stdout:9/189: readlink d5/l6 0 2024-04-22T23:35:01.399 INFO:tasks.workunit.client.1.smithi157.stdout:5/476: rename d1/d27/d29/l6b to d1/d5/dc/d5d/lb9 0 2024-04-22T23:35:01.400 INFO:tasks.workunit.client.0.smithi107.stdout:9/190: link d5/d24/l34 d5/d14/d2e/l40 0 2024-04-22T23:35:01.400 INFO:tasks.workunit.client.1.smithi157.stdout:8/309: dwrite d2/d6/f46 [0,4194304] 0 2024-04-22T23:35:01.407 INFO:tasks.workunit.client.1.smithi157.stdout:6/604: creat d1/d2f/d3b/d39/d4e/da8/dac/fbc x:0 0 0 2024-04-22T23:35:01.408 INFO:tasks.workunit.client.0.smithi107.stdout:9/191: write d5/d7/ff [2924492,58978] 0 2024-04-22T23:35:01.408 INFO:tasks.workunit.client.0.smithi107.stdout:9/192: stat d5/d14/f20 0 2024-04-22T23:35:01.409 INFO:tasks.workunit.client.0.smithi107.stdout:9/193: write d5/d29/f3a [965451,108192] 0 2024-04-22T23:35:01.409 INFO:tasks.workunit.client.1.smithi157.stdout:9/415: creat d4/df/d52/f7d x:0 0 0 2024-04-22T23:35:01.410 INFO:tasks.workunit.client.0.smithi107.stdout:9/194: mknod d5/d29/c41 0 2024-04-22T23:35:01.410 INFO:tasks.workunit.client.0.smithi107.stdout:9/195: write d5/f13 [1833340,9158] 0 2024-04-22T23:35:01.410 INFO:tasks.workunit.client.0.smithi107.stdout:9/196: symlink d5/d7/d27/l42 0 2024-04-22T23:35:01.410 INFO:tasks.workunit.client.1.smithi157.stdout:3/391: truncate d0/d3/d34/f5d 1238768 0 2024-04-22T23:35:01.411 INFO:tasks.workunit.client.1.smithi157.stdout:2/562: rename d7/d8/d8e/ca2 to d7/d1f/cc3 0 2024-04-22T23:35:01.412 INFO:tasks.workunit.client.1.smithi157.stdout:5/477: symlink d1/d5/d7/d10/lba 0 2024-04-22T23:35:01.415 INFO:tasks.workunit.client.0.smithi107.stdout:6/168: dwrite f6 [4194304,4194304] 0 2024-04-22T23:35:01.416 INFO:tasks.workunit.client.1.smithi157.stdout:8/310: link d2/d4/d30/l3e d2/d6/d21/d5f/d17/l6e 0 2024-04-22T23:35:01.416 INFO:tasks.workunit.client.1.smithi157.stdout:8/311: fdatasync d2/d34/d6a/f6d 0 2024-04-22T23:35:01.416 INFO:tasks.workunit.client.1.smithi157.stdout:8/312: chown d2/d6/d21/d5f/d11/f27 394 1 2024-04-22T23:35:01.417 INFO:tasks.workunit.client.1.smithi157.stdout:9/416: getdents d4/d15/d23/d27 0 2024-04-22T23:35:01.418 INFO:tasks.workunit.client.1.smithi157.stdout:3/392: creat d0/d3/d34/d39/f76 x:0 0 0 2024-04-22T23:35:01.418 INFO:tasks.workunit.client.1.smithi157.stdout:2/563: write d7/d8/d30/d23/d27/f7b [115198,72570] 0 2024-04-22T23:35:01.419 INFO:tasks.workunit.client.1.smithi157.stdout:8/313: creat d2/d6/d21/d5f/f6f x:0 0 0 2024-04-22T23:35:01.421 INFO:tasks.workunit.client.0.smithi107.stdout:9/197: write d5/d14/f15 [3808381,34857] 0 2024-04-22T23:35:01.426 INFO:tasks.workunit.client.0.smithi107.stdout:9/198: mkdir d5/d14/d2e/d43 0 2024-04-22T23:35:01.426 INFO:tasks.workunit.client.0.smithi107.stdout:9/199: chown d5/d7/f16 2508484 1 2024-04-22T23:35:01.426 INFO:tasks.workunit.client.0.smithi107.stdout:9/200: chown d5/d7/d27/l42 0 1 2024-04-22T23:35:01.427 INFO:tasks.workunit.client.1.smithi157.stdout:6/605: getdents d1/d2f/d3b/d39/d4e/da8/dac 0 2024-04-22T23:35:01.428 INFO:tasks.workunit.client.0.smithi107.stdout:9/201: unlink d5/d14/d2e/l40 0 2024-04-22T23:35:01.431 INFO:tasks.workunit.client.0.smithi107.stdout:9/202: mknod d5/d14/d2e/d43/c44 0 2024-04-22T23:35:01.434 INFO:tasks.workunit.client.0.smithi107.stdout:9/203: link d5/d7/d17/d1b/d1f/d36/c3e d5/d7/d27/c45 0 2024-04-22T23:35:01.434 INFO:tasks.workunit.client.0.smithi107.stdout:6/169: dread db/fd [0,4194304] 0 2024-04-22T23:35:01.436 INFO:tasks.workunit.client.0.smithi107.stdout:9/204: mkdir d5/d7/d27/d46 0 2024-04-22T23:35:01.439 INFO:tasks.workunit.client.0.smithi107.stdout:9/205: creat d5/f47 x:0 0 0 2024-04-22T23:35:01.439 INFO:tasks.workunit.client.0.smithi107.stdout:9/206: getdents d5/d7/d27/d46 0 2024-04-22T23:35:01.439 INFO:tasks.workunit.client.1.smithi157.stdout:6/606: dread d1/d2f/f40 [0,4194304] 0 2024-04-22T23:35:01.441 INFO:tasks.workunit.client.1.smithi157.stdout:6/607: truncate d1/f7 1594205 0 2024-04-22T23:35:01.443 INFO:tasks.workunit.client.1.smithi157.stdout:6/608: rename d1/d2f/d3b/c20 to d1/d2f/d54/d9b/cbd 0 2024-04-22T23:35:01.444 INFO:tasks.workunit.client.1.smithi157.stdout:6/609: mkdir d1/d2f/d3b/d39/d4e/da8/dac/dbe 0 2024-04-22T23:35:01.444 INFO:tasks.workunit.client.0.smithi107.stdout:6/170: write f6 [7508050,1424] 0 2024-04-22T23:35:01.444 INFO:tasks.workunit.client.0.smithi107.stdout:6/171: fsync db/fd 0 2024-04-22T23:35:01.447 INFO:tasks.workunit.client.0.smithi107.stdout:6/172: mkdir db/d20/d2a 0 2024-04-22T23:35:01.447 INFO:tasks.workunit.client.0.smithi107.stdout:6/173: chown db/f29 12946135 1 2024-04-22T23:35:01.458 INFO:tasks.workunit.client.0.smithi107.stdout:6/174: write db/f1f [3138685,11167] 0 2024-04-22T23:35:01.458 INFO:tasks.workunit.client.0.smithi107.stdout:6/175: dread - db/f1d zero size 2024-04-22T23:35:01.466 INFO:tasks.workunit.client.1.smithi157.stdout:2/564: dwrite d7/fb1 [0,4194304] 0 2024-04-22T23:35:01.466 INFO:tasks.workunit.client.1.smithi157.stdout:9/417: dwrite d4/f20 [0,4194304] 0 2024-04-22T23:35:01.468 INFO:tasks.workunit.client.1.smithi157.stdout:3/393: dwrite d0/d3/d34/d3d/f44 [0,4194304] 0 2024-04-22T23:35:01.468 INFO:tasks.workunit.client.1.smithi157.stdout:6/610: dwrite d1/d2f/d3b/d49/fb3 [0,4194304] 0 2024-04-22T23:35:01.469 INFO:tasks.workunit.client.1.smithi157.stdout:2/565: rename d7/d1f/d26/d4a/f77 to d7/d1f/d3e/db4/fc4 0 2024-04-22T23:35:01.469 INFO:tasks.workunit.client.1.smithi157.stdout:2/566: chown d7/c6a 449442 1 2024-04-22T23:35:01.472 INFO:tasks.workunit.client.1.smithi157.stdout:2/567: creat d7/d8/d30/fc5 x:0 0 0 2024-04-22T23:35:01.478 INFO:tasks.workunit.client.1.smithi157.stdout:2/568: dread d7/d8/d30/f83 [0,4194304] 0 2024-04-22T23:35:01.480 INFO:tasks.workunit.client.1.smithi157.stdout:9/418: dwrite d4/d2d/f78 [0,4194304] 0 2024-04-22T23:35:01.481 INFO:tasks.workunit.client.1.smithi157.stdout:9/419: dread - d4/d15/d23/d27/d47/f74 zero size 2024-04-22T23:35:01.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:35:01 smithi107 ceph-mon[26355]: pgmap v257: 65 pgs: 65 active+clean; 984 MiB data, 4.4 GiB used, 532 GiB / 536 GiB avail; 101 MiB/s rd, 123 MiB/s wr, 754 op/s 2024-04-22T23:35:01.485 INFO:tasks.workunit.client.1.smithi157.stdout:9/420: mkdir d4/d7e 0 2024-04-22T23:35:01.485 INFO:tasks.workunit.client.1.smithi157.stdout:9/421: write d4/d15/f4c [1067877,122702] 0 2024-04-22T23:35:01.485 INFO:tasks.workunit.client.1.smithi157.stdout:9/422: write d4/d15/d55/f76 [172277,47050] 0 2024-04-22T23:35:01.487 INFO:tasks.workunit.client.0.smithi107.stdout:4/141: sync 2024-04-22T23:35:01.488 INFO:tasks.workunit.client.0.smithi107.stdout:2/206: sync 2024-04-22T23:35:01.488 INFO:tasks.workunit.client.0.smithi107.stdout:8/79: sync 2024-04-22T23:35:01.488 INFO:tasks.workunit.client.0.smithi107.stdout:7/106: sync 2024-04-22T23:35:01.489 INFO:tasks.workunit.client.0.smithi107.stdout:7/107: readlink d7/l15 0 2024-04-22T23:35:01.489 INFO:tasks.workunit.client.0.smithi107.stdout:4/142: rmdir d1/d8/d23 39 2024-04-22T23:35:01.490 INFO:tasks.workunit.client.0.smithi107.stdout:8/80: unlink d0/d7/c15 0 2024-04-22T23:35:01.492 INFO:tasks.workunit.client.0.smithi107.stdout:7/108: mkdir d7/d1b/d1e 0 2024-04-22T23:35:01.492 INFO:tasks.workunit.client.0.smithi107.stdout:7/109: truncate d7/f11 1669195 0 2024-04-22T23:35:01.492 INFO:tasks.workunit.client.0.smithi107.stdout:7/110: dread - d7/f1a zero size 2024-04-22T23:35:01.492 INFO:tasks.workunit.client.0.smithi107.stdout:2/207: dread d1/d12/d35/d48/f29 [0,4194304] 0 2024-04-22T23:35:01.493 INFO:tasks.workunit.client.0.smithi107.stdout:2/208: creat d1/d2b/d49/f51 x:0 0 0 2024-04-22T23:35:01.494 INFO:tasks.workunit.client.1.smithi157.stdout:2/569: dread d7/d1f/d26/d8f/fab [0,4194304] 0 2024-04-22T23:35:01.494 INFO:tasks.workunit.client.0.smithi107.stdout:4/143: unlink d1/d7/l17 0 2024-04-22T23:35:01.494 INFO:tasks.workunit.client.0.smithi107.stdout:8/81: getdents d0/d6 0 2024-04-22T23:35:01.494 INFO:tasks.workunit.client.0.smithi107.stdout:8/82: chown d0/d7/df/c1a 1812419 1 2024-04-22T23:35:01.494 INFO:tasks.workunit.client.0.smithi107.stdout:8/83: creat d0/d7/d8/f1e x:0 0 0 2024-04-22T23:35:01.495 INFO:tasks.workunit.client.1.smithi157.stdout:2/570: truncate d7/d8/d25/f33 63178 0 2024-04-22T23:35:01.496 INFO:tasks.workunit.client.1.smithi157.stdout:2/571: read d7/d8/dd/f2b [520241,73224] 0 2024-04-22T23:35:01.496 INFO:tasks.workunit.client.1.smithi157.stdout:2/572: stat d7/d8/fbf 0 2024-04-22T23:35:01.496 INFO:tasks.workunit.client.0.smithi107.stdout:7/111: rmdir d7/d1b/d1e 0 2024-04-22T23:35:01.500 INFO:tasks.workunit.client.0.smithi107.stdout:4/144: dread d1/f13 [0,4194304] 0 2024-04-22T23:35:01.500 INFO:tasks.workunit.client.0.smithi107.stdout:4/145: read d1/f13 [3373119,18071] 0 2024-04-22T23:35:01.501 INFO:tasks.workunit.client.0.smithi107.stdout:2/209: unlink d1/d12/d35/d48/l16 0 2024-04-22T23:35:01.501 INFO:tasks.workunit.client.1.smithi157.stdout:3/394: dwrite d0/d3/d34/d39/f32 [0,4194304] 0 2024-04-22T23:35:01.501 INFO:tasks.workunit.client.1.smithi157.stdout:3/395: chown d0/d3/d13/c3e 196369 1 2024-04-22T23:35:01.502 INFO:tasks.workunit.client.1.smithi157.stdout:3/396: write d0/d3/d34/d3d/d57/f64 [1179149,26927] 0 2024-04-22T23:35:01.503 INFO:tasks.workunit.client.0.smithi107.stdout:8/84: rename d0/d7/df/d17/f1c to d0/d7/df/d18/f1f 0 2024-04-22T23:35:01.503 INFO:tasks.workunit.client.0.smithi107.stdout:8/85: fdatasync d0/f3 0 2024-04-22T23:35:01.503 INFO:tasks.workunit.client.0.smithi107.stdout:7/112: unlink l6 0 2024-04-22T23:35:01.504 INFO:tasks.workunit.client.0.smithi107.stdout:2/210: truncate d1/d12/d35/d48/f29 455234 0 2024-04-22T23:35:01.506 INFO:tasks.workunit.client.0.smithi107.stdout:2/211: unlink d1/d12/d35/d48/d13/f23 0 2024-04-22T23:35:01.508 INFO:tasks.workunit.client.0.smithi107.stdout:2/212: truncate d1/f6 3312907 0 2024-04-22T23:35:01.508 INFO:tasks.workunit.client.0.smithi107.stdout:7/113: write d7/fd [2663392,100999] 0 2024-04-22T23:35:01.509 INFO:tasks.workunit.client.1.smithi157.stdout:3/397: read d0/f41 [734119,89610] 0 2024-04-22T23:35:01.511 INFO:tasks.workunit.client.0.smithi107.stdout:7/114: rename d7/d1b/c1c to d7/c1f 0 2024-04-22T23:35:01.511 INFO:tasks.workunit.client.1.smithi157.stdout:3/398: symlink d0/d3/d34/d67/l77 0 2024-04-22T23:35:01.511 INFO:tasks.workunit.client.1.smithi157.stdout:9/423: dwrite d4/f50 [0,4194304] 0 2024-04-22T23:35:01.511 INFO:tasks.workunit.client.1.smithi157.stdout:9/424: creat d4/d15/d23/d27/d47/f7f x:0 0 0 2024-04-22T23:35:01.512 INFO:tasks.workunit.client.1.smithi157.stdout:2/573: dwrite d7/d8/d30/d23/d27/d57/fa7 [0,4194304] 0 2024-04-22T23:35:01.512 INFO:tasks.workunit.client.1.smithi157.stdout:2/574: stat d7/d1f/d26/c91 0 2024-04-22T23:35:01.513 INFO:tasks.workunit.client.0.smithi107.stdout:7/115: rename d7/c12 to d7/c20 0 2024-04-22T23:35:01.513 INFO:tasks.workunit.client.0.smithi107.stdout:4/146: write d1/d8/f1f [3540177,92894] 0 2024-04-22T23:35:01.513 INFO:tasks.workunit.client.0.smithi107.stdout:4/147: chown d1/d7/f11 62472 1 2024-04-22T23:35:01.513 INFO:tasks.workunit.client.0.smithi107.stdout:7/116: mkdir d7/d21 0 2024-04-22T23:35:01.513 INFO:tasks.workunit.client.0.smithi107.stdout:7/117: readlink d7/le 0 2024-04-22T23:35:01.514 INFO:tasks.workunit.client.1.smithi157.stdout:3/399: creat d0/d3/d34/d3d/d57/d5c/f78 x:0 0 0 2024-04-22T23:35:01.515 INFO:tasks.workunit.client.1.smithi157.stdout:6/611: dwrite d1/f2c [0,4194304] 0 2024-04-22T23:35:01.516 INFO:tasks.workunit.client.0.smithi107.stdout:7/118: creat d7/d21/f22 x:0 0 0 2024-04-22T23:35:01.516 INFO:tasks.workunit.client.0.smithi107.stdout:8/86: dread d0/f5 [4194304,4194304] 0 2024-04-22T23:35:01.517 INFO:tasks.workunit.client.1.smithi157.stdout:9/425: unlink d4/d15/d23/d3f/c59 0 2024-04-22T23:35:01.517 INFO:tasks.workunit.client.1.smithi157.stdout:9/426: truncate d4/df/d52/f7d 929592 0 2024-04-22T23:35:01.517 INFO:tasks.workunit.client.1.smithi157.stdout:9/427: read - d4/d15/d23/d27/d47/f7f zero size 2024-04-22T23:35:01.517 INFO:tasks.workunit.client.1.smithi157.stdout:9/428: fsync d4/d15/d23/d27/f33 0 2024-04-22T23:35:01.517 INFO:tasks.workunit.client.1.smithi157.stdout:9/429: chown d4/df/d57/f44 0 1 2024-04-22T23:35:01.518 INFO:tasks.workunit.client.0.smithi107.stdout:7/119: mknod d7/d21/c23 0 2024-04-22T23:35:01.518 INFO:tasks.workunit.client.1.smithi157.stdout:2/575: mknod d7/d1f/d26/d8f/cc6 0 2024-04-22T23:35:01.519 INFO:tasks.workunit.client.1.smithi157.stdout:3/400: creat d0/d3/d34/d3d/d5b/d72/f79 x:0 0 0 2024-04-22T23:35:01.519 INFO:tasks.workunit.client.0.smithi107.stdout:8/87: mkdir d0/d7/d20 0 2024-04-22T23:35:01.520 INFO:tasks.workunit.client.0.smithi107.stdout:8/88: chown d0/d7/df/d18/f1f 12708 1 2024-04-22T23:35:01.520 INFO:tasks.workunit.client.1.smithi157.stdout:6/612: creat d1/d2f/d3b/d6c/d83/d93/fbf x:0 0 0 2024-04-22T23:35:01.521 INFO:tasks.workunit.client.1.smithi157.stdout:2/576: truncate d7/d1f/d26/d4a/f60 6854212 0 2024-04-22T23:35:01.521 INFO:tasks.workunit.client.1.smithi157.stdout:2/577: truncate d7/d8/fa4 858953 0 2024-04-22T23:35:01.521 INFO:tasks.workunit.client.0.smithi107.stdout:7/120: creat d7/d1b/f24 x:0 0 0 2024-04-22T23:35:01.522 INFO:tasks.workunit.client.1.smithi157.stdout:3/401: creat d0/d3/d13/d21/f7a x:0 0 0 2024-04-22T23:35:01.522 INFO:tasks.workunit.client.0.smithi107.stdout:8/89: getdents d0/d7/df 0 2024-04-22T23:35:01.523 INFO:tasks.workunit.client.1.smithi157.stdout:6/613: mkdir d1/d2f/d3b/d39/d4e/da8/dac/dc0 0 2024-04-22T23:35:01.523 INFO:tasks.workunit.client.1.smithi157.stdout:2/578: getdents d7/d8/d8e 0 2024-04-22T23:35:01.523 INFO:tasks.workunit.client.1.smithi157.stdout:2/579: fdatasync d7/d8/d30/d5b/f72 0 2024-04-22T23:35:01.525 INFO:tasks.workunit.client.1.smithi157.stdout:6/614: creat d1/d2f/d54/fc1 x:0 0 0 2024-04-22T23:35:01.525 INFO:tasks.workunit.client.1.smithi157.stdout:6/615: fsync d1/d2f/d3b/d6c/d83/f71 0 2024-04-22T23:35:01.525 INFO:tasks.workunit.client.1.smithi157.stdout:6/616: dread - d1/d38/d97/fb4 zero size 2024-04-22T23:35:01.525 INFO:tasks.workunit.client.1.smithi157.stdout:6/617: write d1/d2f/d54/d9b/f6e [3570997,36006] 0 2024-04-22T23:35:01.525 INFO:tasks.workunit.client.0.smithi107.stdout:4/148: write d1/d21/d29/f34 [4176764,2757] 0 2024-04-22T23:35:01.525 INFO:tasks.workunit.client.0.smithi107.stdout:2/213: dwrite d1/d2b/d49/f50 [0,4194304] 0 2024-04-22T23:35:01.527 INFO:tasks.workunit.client.1.smithi157.stdout:6/618: creat d1/fc2 x:0 0 0 2024-04-22T23:35:01.528 INFO:tasks.workunit.client.1.smithi157.stdout:9/430: dread d4/d15/d55/f58 [0,4194304] 0 2024-04-22T23:35:01.530 INFO:tasks.workunit.client.1.smithi157.stdout:2/580: write d7/d1f/d26/d8f/fab [4087981,26502] 0 2024-04-22T23:35:01.530 INFO:tasks.workunit.client.1.smithi157.stdout:2/581: stat d7/d8/d8e/d94/cbd 0 2024-04-22T23:35:01.530 INFO:tasks.workunit.client.1.smithi157.stdout:9/431: mkdir d4/df/d80 0 2024-04-22T23:35:01.531 INFO:tasks.workunit.client.0.smithi107.stdout:7/121: unlink d7/le 0 2024-04-22T23:35:01.532 INFO:tasks.workunit.client.1.smithi157.stdout:2/582: dread d7/d8/d30/fb3 [0,4194304] 0 2024-04-22T23:35:01.532 INFO:tasks.workunit.client.1.smithi157.stdout:9/432: rename d4/d15/d23/c4d to d4/df/d80/c81 0 2024-04-22T23:35:01.534 INFO:tasks.workunit.client.1.smithi157.stdout:2/583: mkdir d7/d9f/dc7 0 2024-04-22T23:35:01.534 INFO:tasks.workunit.client.0.smithi107.stdout:8/90: mknod d0/d7/df/d18/c21 0 2024-04-22T23:35:01.538 INFO:tasks.workunit.client.1.smithi157.stdout:9/433: rename d4/f50 to d4/d15/d23/d27/d75/f82 0 2024-04-22T23:35:01.539 INFO:tasks.workunit.client.0.smithi107.stdout:2/214: rmdir d1/d12/d35/d48 39 2024-04-22T23:35:01.541 INFO:tasks.workunit.client.0.smithi107.stdout:7/122: unlink d7/l19 0 2024-04-22T23:35:01.541 INFO:tasks.workunit.client.0.smithi107.stdout:7/123: creat d7/d1b/f25 x:0 0 0 2024-04-22T23:35:01.542 INFO:tasks.workunit.client.0.smithi107.stdout:4/149: mkdir d1/d21/d29/d2e/d3b 0 2024-04-22T23:35:01.543 INFO:tasks.workunit.client.1.smithi157.stdout:2/584: link d7/d8/d30/d23/l42 d7/d8/d30/d5b/d8a/lc8 0 2024-04-22T23:35:01.543 INFO:tasks.workunit.client.1.smithi157.stdout:2/585: stat d7/d8/d25/d55/f6c 0 2024-04-22T23:35:01.543 INFO:tasks.workunit.client.1.smithi157.stdout:2/586: chown d7/d8/d30/db8 0 1 2024-04-22T23:35:01.544 INFO:tasks.workunit.client.1.smithi157.stdout:2/587: chown d7/d8/lbb 3553540 1 2024-04-22T23:35:01.544 INFO:tasks.workunit.client.1.smithi157.stdout:2/588: write d7/d8/d25/d55/f6c [349390,56874] 0 2024-04-22T23:35:01.545 INFO:tasks.workunit.client.0.smithi107.stdout:7/124: write d7/f18 [413279,95321] 0 2024-04-22T23:35:01.545 INFO:tasks.workunit.client.0.smithi107.stdout:7/125: stat d7/c1d 0 2024-04-22T23:35:01.545 INFO:tasks.workunit.client.0.smithi107.stdout:7/126: dread - d7/d21/f22 zero size 2024-04-22T23:35:01.545 INFO:tasks.workunit.client.0.smithi107.stdout:7/127: truncate d7/d21/f22 813137 0 2024-04-22T23:35:01.545 INFO:tasks.workunit.client.0.smithi107.stdout:4/150: dread d1/f28 [0,4194304] 0 2024-04-22T23:35:01.545 INFO:tasks.workunit.client.1.smithi157.stdout:9/434: symlink d4/df/d72/l83 0 2024-04-22T23:35:01.545 INFO:tasks.workunit.client.1.smithi157.stdout:9/435: readlink d4/df/d57/l7b 0 2024-04-22T23:35:01.545 INFO:tasks.workunit.client.1.smithi157.stdout:9/436: creat d4/df/d61/f84 x:0 0 0 2024-04-22T23:35:01.546 INFO:tasks.workunit.client.1.smithi157.stdout:9/437: write d4/d15/d23/d27/f53 [1279009,23359] 0 2024-04-22T23:35:01.546 INFO:tasks.workunit.client.1.smithi157.stdout:9/438: stat d4/d15/d55/f6e 0 2024-04-22T23:35:01.546 INFO:tasks.workunit.client.1.smithi157.stdout:9/439: fdatasync d4/d15/f73 0 2024-04-22T23:35:01.546 INFO:tasks.workunit.client.1.smithi157.stdout:2/589: creat d7/d1f/d26/d4a/fc9 x:0 0 0 2024-04-22T23:35:01.547 INFO:tasks.workunit.client.1.smithi157.stdout:2/590: chown d7/d8/d30/d23/c32 13 1 2024-04-22T23:35:01.547 INFO:tasks.workunit.client.1.smithi157.stdout:2/591: stat d7/d8/dd/d4d/c84 0 2024-04-22T23:35:01.547 INFO:tasks.workunit.client.1.smithi157.stdout:2/592: chown d7/d8/d6b/c81 13615383 1 2024-04-22T23:35:01.547 INFO:tasks.workunit.client.1.smithi157.stdout:2/593: read - d7/d8/d95/f63 zero size 2024-04-22T23:35:01.547 INFO:tasks.workunit.client.0.smithi107.stdout:2/215: getdents d1 0 2024-04-22T23:35:01.547 INFO:tasks.workunit.client.0.smithi107.stdout:2/216: truncate d1/d12/f27 1019978 0 2024-04-22T23:35:01.547 INFO:tasks.workunit.client.0.smithi107.stdout:2/217: dread - d1/d2b/d49/f51 zero size 2024-04-22T23:35:01.549 INFO:tasks.workunit.client.1.smithi157.stdout:9/440: rename d4/l14 to d4/d15/d23/l85 0 2024-04-22T23:35:01.549 INFO:tasks.workunit.client.0.smithi107.stdout:4/151: rename d1/d8/l2c to d1/d8/d23/l3c 0 2024-04-22T23:35:01.551 INFO:tasks.workunit.client.1.smithi157.stdout:2/594: mkdir d7/d8/dca 0 2024-04-22T23:35:01.551 INFO:tasks.workunit.client.1.smithi157.stdout:2/595: creat d7/d8/d95/fcb x:0 0 0 2024-04-22T23:35:01.551 INFO:tasks.workunit.client.0.smithi107.stdout:4/152: creat d1/d21/d29/d2e/f3d x:0 0 0 2024-04-22T23:35:01.554 INFO:tasks.workunit.client.1.smithi157.stdout:2/596: mknod d7/ccc 0 2024-04-22T23:35:01.561 INFO:tasks.workunit.client.1.smithi157.stdout:3/402: dwrite d0/d3/d34/d39/f66 [0,4194304] 0 2024-04-22T23:35:01.562 INFO:tasks.workunit.client.1.smithi157.stdout:3/403: write d0/d3/f1d [1826753,19992] 0 2024-04-22T23:35:01.562 INFO:tasks.workunit.client.1.smithi157.stdout:3/404: creat d0/d3/d13/d21/f7b x:0 0 0 2024-04-22T23:35:01.564 INFO:tasks.workunit.client.1.smithi157.stdout:6/619: dwrite d1/d2f/d54/f95 [4194304,4194304] 0 2024-04-22T23:35:01.564 INFO:tasks.workunit.client.1.smithi157.stdout:6/620: chown d1/d2f/d3b/d6c/d83/c91 386 1 2024-04-22T23:35:01.564 INFO:tasks.workunit.client.1.smithi157.stdout:6/621: fsync d1/d17/f94 0 2024-04-22T23:35:01.565 INFO:tasks.workunit.client.0.smithi107.stdout:8/91: dwrite d0/f4 [0,4194304] 0 2024-04-22T23:35:01.565 INFO:tasks.workunit.client.0.smithi107.stdout:8/92: write d0/fb [3746167,35090] 0 2024-04-22T23:35:01.566 INFO:tasks.workunit.client.1.smithi157.stdout:3/405: rmdir d0/d3/d34/d47 39 2024-04-22T23:35:01.567 INFO:tasks.workunit.client.1.smithi157.stdout:6/622: rename d1/d36/l3d to d1/d2f/d54/d86/lc3 0 2024-04-22T23:35:01.568 INFO:tasks.workunit.client.0.smithi107.stdout:8/93: rename d0/d7/df/d18/c21 to d0/d7/d8/c22 0 2024-04-22T23:35:01.568 INFO:tasks.workunit.client.0.smithi107.stdout:2/218: dwrite d1/d12/f27 [0,4194304] 0 2024-04-22T23:35:01.568 INFO:tasks.workunit.client.1.smithi157.stdout:3/406: mknod d0/d3/d34/d3d/d57/d5c/c7c 0 2024-04-22T23:35:01.569 INFO:tasks.workunit.client.1.smithi157.stdout:3/407: chown d0/f1c 11 1 2024-04-22T23:35:01.569 INFO:tasks.workunit.client.1.smithi157.stdout:3/408: chown d0/d3/d34/d3d/c54 1814189 1 2024-04-22T23:35:01.570 INFO:tasks.workunit.client.1.smithi157.stdout:6/623: mkdir d1/d2f/d54/dc4 0 2024-04-22T23:35:01.572 INFO:tasks.workunit.client.1.smithi157.stdout:3/409: creat d0/d7/f7d x:0 0 0 2024-04-22T23:35:01.572 INFO:tasks.workunit.client.0.smithi107.stdout:7/128: dread d7/fd [0,4194304] 0 2024-04-22T23:35:01.572 INFO:tasks.workunit.client.0.smithi107.stdout:7/129: read d7/f13 [713109,115258] 0 2024-04-22T23:35:01.577 INFO:tasks.workunit.client.1.smithi157.stdout:9/441: dwrite d4/d15/d23/f69 [0,4194304] 0 2024-04-22T23:35:01.577 INFO:tasks.workunit.client.1.smithi157.stdout:9/442: fdatasync d4/d15/d23/d3f/f45 0 2024-04-22T23:35:01.586 INFO:tasks.workunit.client.1.smithi157.stdout:2/597: dwrite d7/d8/d6b/f97 [0,4194304] 0 2024-04-22T23:35:01.586 INFO:tasks.workunit.client.1.smithi157.stdout:2/598: chown d7/d8/d6b/f89 2810 1 2024-04-22T23:35:01.591 INFO:tasks.workunit.client.0.smithi107.stdout:2/219: dwrite d1/d12/d35/d48/f2a [0,4194304] 0 2024-04-22T23:35:01.593 INFO:tasks.workunit.client.0.smithi107.stdout:2/220: symlink d1/d12/d35/d48/d13/l52 0 2024-04-22T23:35:01.594 INFO:tasks.workunit.client.0.smithi107.stdout:2/221: creat d1/d12/d35/d48/d13/d4f/f53 x:0 0 0 2024-04-22T23:35:01.596 INFO:tasks.workunit.client.1.smithi157.stdout:3/410: dwrite d0/d3/d34/d47/f6d [0,4194304] 0 2024-04-22T23:35:01.596 INFO:tasks.workunit.client.1.smithi157.stdout:3/411: dread - d0/d3/d34/d47/f70 zero size 2024-04-22T23:35:01.597 INFO:tasks.workunit.client.1.smithi157.stdout:6/624: dwrite d1/d2f/d3b/f2e [0,4194304] 0 2024-04-22T23:35:01.599 INFO:tasks.workunit.client.1.smithi157.stdout:3/412: symlink d0/d7/d6c/l7e 0 2024-04-22T23:35:01.599 INFO:tasks.workunit.client.1.smithi157.stdout:3/413: fsync d0/d3/d13/d21/f3f 0 2024-04-22T23:35:01.599 INFO:tasks.workunit.client.1.smithi157.stdout:9/443: dwrite d4/d15/d23/d27/f64 [0,4194304] 0 2024-04-22T23:35:01.605 INFO:tasks.workunit.client.1.smithi157.stdout:9/444: unlink d4/df/f16 0 2024-04-22T23:35:01.607 INFO:tasks.workunit.client.1.smithi157.stdout:3/414: dread d0/d3/d34/d39/f32 [0,4194304] 0 2024-04-22T23:35:01.607 INFO:tasks.workunit.client.1.smithi157.stdout:3/415: read - d0/d3/d34/d39/f76 zero size 2024-04-22T23:35:01.607 INFO:tasks.workunit.client.1.smithi157.stdout:3/416: read - d0/d3/d34/d3d/d5b/d72/f79 zero size 2024-04-22T23:35:01.607 INFO:tasks.workunit.client.1.smithi157.stdout:3/417: creat d0/d3/d34/d3d/d5b/d72/f7f x:0 0 0 2024-04-22T23:35:01.607 INFO:tasks.workunit.client.1.smithi157.stdout:3/418: chown d0/d3/d13/d21/f7b 358220 1 2024-04-22T23:35:01.607 INFO:tasks.workunit.client.1.smithi157.stdout:3/419: chown d0/d3/d34/d3d/d57/l74 69353 1 2024-04-22T23:35:01.607 INFO:tasks.workunit.client.1.smithi157.stdout:3/420: chown d0/d7/l38 7 1 2024-04-22T23:35:01.610 INFO:tasks.workunit.client.1.smithi157.stdout:6/625: dwrite d1/d2f/d3b/d6c/f98 [0,4194304] 0 2024-04-22T23:35:01.612 INFO:tasks.workunit.client.1.smithi157.stdout:6/626: mkdir d1/d2f/dc5 0 2024-04-22T23:35:01.614 INFO:tasks.workunit.client.1.smithi157.stdout:6/627: mknod d1/d38/cc6 0 2024-04-22T23:35:01.621 INFO:tasks.workunit.client.1.smithi157.stdout:3/421: dwrite d0/d3/d34/d3d/f63 [0,4194304] 0 2024-04-22T23:35:01.623 INFO:tasks.workunit.client.1.smithi157.stdout:3/422: symlink d0/d7/l80 0 2024-04-22T23:35:01.623 INFO:tasks.workunit.client.1.smithi157.stdout:3/423: chown d0/d3/d34/d3d/d5b/d72 105289 1 2024-04-22T23:35:01.624 INFO:tasks.workunit.client.1.smithi157.stdout:6/628: dread d1/f18 [0,4194304] 0 2024-04-22T23:35:01.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:35:01 smithi157 ceph-mon[33877]: pgmap v257: 65 pgs: 65 active+clean; 984 MiB data, 4.4 GiB used, 532 GiB / 536 GiB avail; 101 MiB/s rd, 123 MiB/s wr, 754 op/s 2024-04-22T23:35:01.644 INFO:tasks.workunit.client.1.smithi157.stdout:3/424: dwrite d0/d3/d34/d3d/f42 [0,4194304] 0 2024-04-22T23:35:01.644 INFO:tasks.workunit.client.1.smithi157.stdout:3/425: chown d0/d3/l12 185420 1 2024-04-22T23:35:01.644 INFO:tasks.workunit.client.1.smithi157.stdout:3/426: chown d0/d3/d34/d47/f70 0 1 2024-04-22T23:35:01.644 INFO:tasks.workunit.client.1.smithi157.stdout:3/427: dread - d0/d3/d34/d3d/d57/f65 zero size 2024-04-22T23:35:01.646 INFO:tasks.workunit.client.1.smithi157.stdout:3/428: link d0/d3/d34/d3d/d57/d60/c75 d0/d7/d6c/c81 0 2024-04-22T23:35:01.647 INFO:tasks.workunit.client.1.smithi157.stdout:3/429: mknod d0/d3/d34/c82 0 2024-04-22T23:35:01.647 INFO:tasks.workunit.client.1.smithi157.stdout:6/629: dwrite d1/d36/fa3 [0,4194304] 0 2024-04-22T23:35:01.647 INFO:tasks.workunit.client.1.smithi157.stdout:6/630: readlink d1/l7b 0 2024-04-22T23:35:01.647 INFO:tasks.workunit.client.1.smithi157.stdout:3/430: mkdir d0/d83 0 2024-04-22T23:35:01.648 INFO:tasks.workunit.client.1.smithi157.stdout:3/431: write d0/d3/d34/d47/f70 [51508,108433] 0 2024-04-22T23:35:01.648 INFO:tasks.workunit.client.1.smithi157.stdout:6/631: mkdir d1/d2f/dc7 0 2024-04-22T23:35:01.649 INFO:tasks.workunit.client.1.smithi157.stdout:3/432: creat d0/d3/d34/d3d/d5b/f84 x:0 0 0 2024-04-22T23:35:01.649 INFO:tasks.workunit.client.1.smithi157.stdout:3/433: stat d0/d7/d6c/c81 0 2024-04-22T23:35:01.649 INFO:tasks.workunit.client.1.smithi157.stdout:3/434: dread - d0/d3/d34/d3d/d5b/d72/f79 zero size 2024-04-22T23:35:01.650 INFO:tasks.workunit.client.1.smithi157.stdout:3/435: creat d0/d3/d34/d3d/d57/d60/d6b/f85 x:0 0 0 2024-04-22T23:35:01.657 INFO:tasks.workunit.client.1.smithi157.stdout:3/436: dread d0/d3/d34/d3d/f42 [0,4194304] 0 2024-04-22T23:35:01.658 INFO:tasks.workunit.client.1.smithi157.stdout:3/437: symlink d0/d3/d34/d3d/d57/d60/l86 0 2024-04-22T23:35:01.665 INFO:tasks.workunit.client.1.smithi157.stdout:3/438: dread d0/d3/f1f [0,4194304] 0 2024-04-22T23:35:01.666 INFO:tasks.workunit.client.1.smithi157.stdout:3/439: mknod d0/d3/d34/d3d/d5b/d72/c87 0 2024-04-22T23:35:01.667 INFO:tasks.workunit.client.1.smithi157.stdout:6/632: dwrite d1/d2f/d54/d9b/f6e [0,4194304] 0 2024-04-22T23:35:01.670 INFO:tasks.workunit.client.1.smithi157.stdout:6/633: dread d1/d17/f24 [0,4194304] 0 2024-04-22T23:35:01.673 INFO:tasks.workunit.client.1.smithi157.stdout:3/440: dread d0/d3/d13/d21/f3f [0,4194304] 0 2024-04-22T23:35:01.675 INFO:tasks.workunit.client.1.smithi157.stdout:3/441: rmdir d0/d3/d34/d3d/d57 39 2024-04-22T23:35:01.677 INFO:tasks.workunit.client.1.smithi157.stdout:3/442: rename d0/d3/d34/d67/l77 to d0/d7/l88 0 2024-04-22T23:35:01.678 INFO:tasks.workunit.client.1.smithi157.stdout:3/443: creat d0/d3/d34/d3d/d57/d60/f89 x:0 0 0 2024-04-22T23:35:01.680 INFO:tasks.workunit.client.1.smithi157.stdout:3/444: unlink d0/d3/d34/d67/l6e 0 2024-04-22T23:35:01.681 INFO:tasks.workunit.client.1.smithi157.stdout:3/445: mknod d0/d3/d34/d3d/d57/d60/c8a 0 2024-04-22T23:35:01.682 INFO:tasks.workunit.client.0.smithi107.stdout:1/136: sync 2024-04-22T23:35:01.682 INFO:tasks.workunit.client.0.smithi107.stdout:5/160: sync 2024-04-22T23:35:01.684 INFO:tasks.workunit.client.0.smithi107.stdout:1/137: write d7/d19/d1c/f29 [98101,73693] 0 2024-04-22T23:35:01.684 INFO:tasks.workunit.client.0.smithi107.stdout:5/161: unlink d9/db/l2f 0 2024-04-22T23:35:01.684 INFO:tasks.workunit.client.0.smithi107.stdout:1/138: mkdir d7/d19/d2d 0 2024-04-22T23:35:01.685 INFO:tasks.workunit.client.0.smithi107.stdout:5/162: symlink d9/db/de/l34 0 2024-04-22T23:35:01.687 INFO:tasks.workunit.client.0.smithi107.stdout:5/163: mknod d9/db/c35 0 2024-04-22T23:35:01.687 INFO:tasks.workunit.client.0.smithi107.stdout:5/164: creat d9/f36 x:0 0 0 2024-04-22T23:35:01.688 INFO:tasks.workunit.client.1.smithi157.stdout:6/634: dwrite d1/d17/fa2 [0,4194304] 0 2024-04-22T23:35:01.688 INFO:tasks.workunit.client.1.smithi157.stdout:6/635: chown d1/d2f/d54/cb8 11 1 2024-04-22T23:35:01.690 INFO:tasks.workunit.client.0.smithi107.stdout:1/139: getdents d7/d19 0 2024-04-22T23:35:01.691 INFO:tasks.workunit.client.0.smithi107.stdout:1/140: creat d7/d19/f2e x:0 0 0 2024-04-22T23:35:01.692 INFO:tasks.workunit.client.0.smithi107.stdout:1/141: readlink d7/d19/l1f 0 2024-04-22T23:35:01.693 INFO:tasks.workunit.client.0.smithi107.stdout:1/142: symlink d7/l2f 0 2024-04-22T23:35:01.694 INFO:tasks.workunit.client.1.smithi157.stdout:6/636: dread d1/d2f/d3b/d6c/f98 [0,4194304] 0 2024-04-22T23:35:01.694 INFO:tasks.workunit.client.0.smithi107.stdout:5/165: dread d9/db/f13 [4194304,4194304] 0 2024-04-22T23:35:01.695 INFO:tasks.workunit.client.1.smithi157.stdout:6/637: mknod d1/d2f/d3b/d39/d4e/da8/dac/dc0/cc8 0 2024-04-22T23:35:01.696 INFO:tasks.workunit.client.0.smithi107.stdout:5/166: rename d9/d26/f30 to d9/f37 0 2024-04-22T23:35:01.696 INFO:tasks.workunit.client.0.smithi107.stdout:5/167: chown d9/l21 334 1 2024-04-22T23:35:01.696 INFO:tasks.workunit.client.0.smithi107.stdout:5/168: creat d9/d26/f38 x:0 0 0 2024-04-22T23:35:01.697 INFO:tasks.workunit.client.1.smithi157.stdout:6/638: symlink d1/d2f/lc9 0 2024-04-22T23:35:01.697 INFO:tasks.workunit.client.1.smithi157.stdout:6/639: write d1/d36/f37 [357162,103829] 0 2024-04-22T23:35:01.698 INFO:tasks.workunit.client.1.smithi157.stdout:6/640: symlink d1/d2f/d3b/d6c/da7/lca 0 2024-04-22T23:35:01.700 INFO:tasks.workunit.client.1.smithi157.stdout:6/641: mknod d1/d38/d97/ccb 0 2024-04-22T23:35:01.700 INFO:tasks.workunit.client.1.smithi157.stdout:6/642: dread - d1/fc2 zero size 2024-04-22T23:35:01.724 INFO:tasks.workunit.client.0.smithi107.stdout:1/143: dwrite d7/d19/f26 [0,4194304] 0 2024-04-22T23:35:01.725 INFO:tasks.workunit.client.0.smithi107.stdout:5/169: dwrite d9/f24 [0,4194304] 0 2024-04-22T23:35:01.725 INFO:tasks.workunit.client.0.smithi107.stdout:5/170: truncate d9/d26/f38 959209 0 2024-04-22T23:35:01.726 INFO:tasks.workunit.client.1.smithi157.stdout:6/643: dwrite d1/f2c [0,4194304] 0 2024-04-22T23:35:01.729 INFO:tasks.workunit.client.0.smithi107.stdout:5/171: creat d9/db/f39 x:0 0 0 2024-04-22T23:35:01.731 INFO:tasks.workunit.client.0.smithi107.stdout:5/172: truncate d9/db/de/f23 279847 0 2024-04-22T23:35:01.731 INFO:tasks.workunit.client.0.smithi107.stdout:5/173: write d9/d26/f32 [478726,85738] 0 2024-04-22T23:35:01.734 INFO:tasks.workunit.client.1.smithi157.stdout:6/644: dread d1/f18 [0,4194304] 0 2024-04-22T23:35:01.734 INFO:tasks.workunit.client.1.smithi157.stdout:6/645: fsync d1/d17/f31 0 2024-04-22T23:35:01.735 INFO:tasks.workunit.client.1.smithi157.stdout:6/646: creat d1/d2f/d3b/d39/d4e/da8/dac/dbe/fcc x:0 0 0 2024-04-22T23:35:01.738 INFO:tasks.workunit.client.0.smithi107.stdout:1/144: dwrite d7/f2a [0,4194304] 0 2024-04-22T23:35:01.747 INFO:tasks.workunit.client.1.smithi157.stdout:6/647: write d1/d2f/d54/d9b/f6e [2386669,2049] 0 2024-04-22T23:35:01.749 INFO:tasks.workunit.client.1.smithi157.stdout:6/648: symlink d1/d36/lcd 0 2024-04-22T23:35:01.751 INFO:tasks.workunit.client.1.smithi157.stdout:6/649: mkdir d1/d2f/dc7/dce 0 2024-04-22T23:35:01.756 INFO:tasks.workunit.client.1.smithi157.stdout:6/650: dread d1/d2f/d3b/d6c/d83/f73 [0,4194304] 0 2024-04-22T23:35:01.756 INFO:tasks.workunit.client.1.smithi157.stdout:6/651: stat d1/d2f/d3b/d39/d4e/c78 0 2024-04-22T23:35:01.757 INFO:tasks.workunit.client.1.smithi157.stdout:6/652: creat d1/d36/fcf x:0 0 0 2024-04-22T23:35:01.758 INFO:tasks.workunit.client.1.smithi157.stdout:6/653: rmdir d1/d2f/d3b/d6c/d83/d93 39 2024-04-22T23:35:01.762 INFO:tasks.workunit.client.0.smithi107.stdout:3/214: sync 2024-04-22T23:35:01.762 INFO:tasks.workunit.client.0.smithi107.stdout:0/161: sync 2024-04-22T23:35:01.764 INFO:tasks.workunit.client.0.smithi107.stdout:3/215: rename db/f18 to db/d21/d27/d22/f4e 0 2024-04-22T23:35:01.764 INFO:tasks.workunit.client.0.smithi107.stdout:3/216: write db/f1c [1696996,55439] 0 2024-04-22T23:35:01.765 INFO:tasks.workunit.client.0.smithi107.stdout:1/145: dwrite d7/f10 [0,4194304] 0 2024-04-22T23:35:01.765 INFO:tasks.workunit.client.0.smithi107.stdout:1/146: write d7/f17 [773681,12610] 0 2024-04-22T23:35:01.770 INFO:tasks.workunit.client.0.smithi107.stdout:1/147: rmdir d7 39 2024-04-22T23:35:01.774 INFO:tasks.workunit.client.0.smithi107.stdout:0/162: write d1/d27/f21 [4169974,95918] 0 2024-04-22T23:35:01.775 INFO:tasks.workunit.client.0.smithi107.stdout:1/148: symlink d7/d19/d20/l30 0 2024-04-22T23:35:01.784 INFO:tasks.workunit.client.0.smithi107.stdout:0/163: write d1/d9/f29 [3349044,124124] 0 2024-04-22T23:35:01.784 INFO:tasks.workunit.client.0.smithi107.stdout:0/164: fdatasync d1/d9/fc 0 2024-04-22T23:35:01.786 INFO:tasks.workunit.client.0.smithi107.stdout:0/165: link d1/d27/c26 d1/d27/d20/d2c/c3d 0 2024-04-22T23:35:01.787 INFO:tasks.workunit.client.1.smithi157.stdout:6/654: dwrite d1/d38/faa [0,4194304] 0 2024-04-22T23:35:01.787 INFO:tasks.workunit.client.1.smithi157.stdout:6/655: truncate d1/d2f/d54/d9b/fb7 88413 0 2024-04-22T23:35:01.788 INFO:tasks.workunit.client.0.smithi107.stdout:3/217: dwrite db/d21/d27/f23 [0,4194304] 0 2024-04-22T23:35:01.788 INFO:tasks.workunit.client.0.smithi107.stdout:1/149: write d7/f10 [999674,26539] 0 2024-04-22T23:35:01.788 INFO:tasks.workunit.client.0.smithi107.stdout:0/166: creat d1/d9/f3e x:0 0 0 2024-04-22T23:35:01.788 INFO:tasks.workunit.client.0.smithi107.stdout:0/167: write d1/d9/fc [1640787,10837] 0 2024-04-22T23:35:01.789 INFO:tasks.workunit.client.0.smithi107.stdout:0/168: chown d1/d9/d14/l19 0 1 2024-04-22T23:35:01.791 INFO:tasks.workunit.client.0.smithi107.stdout:1/150: creat d7/d19/d2d/f31 x:0 0 0 2024-04-22T23:35:01.794 INFO:tasks.workunit.client.0.smithi107.stdout:1/151: mkdir d7/d19/d32 0 2024-04-22T23:35:01.795 INFO:tasks.workunit.client.0.smithi107.stdout:3/218: dread f0 [0,4194304] 0 2024-04-22T23:35:01.797 INFO:tasks.workunit.client.0.smithi107.stdout:3/219: mkdir db/d21/d27/d15/d29/d4f 0 2024-04-22T23:35:01.797 INFO:tasks.workunit.client.0.smithi107.stdout:1/152: creat d7/d19/d1c/f33 x:0 0 0 2024-04-22T23:35:01.799 INFO:tasks.workunit.client.0.smithi107.stdout:3/220: rename db/c1b to db/d21/d27/d15/d29/d2a/d45/c50 0 2024-04-22T23:35:01.800 INFO:tasks.workunit.client.0.smithi107.stdout:3/221: write db/f1c [521634,127472] 0 2024-04-22T23:35:01.806 INFO:tasks.workunit.client.0.smithi107.stdout:1/153: fdatasync d7/f10 0 2024-04-22T23:35:01.807 INFO:tasks.workunit.client.1.smithi157.stdout:6/656: dwrite d1/f88 [0,4194304] 0 2024-04-22T23:35:01.808 INFO:tasks.workunit.client.0.smithi107.stdout:1/154: rmdir d7/d19 39 2024-04-22T23:35:01.808 INFO:tasks.workunit.client.0.smithi107.stdout:1/155: fsync d7/f10 0 2024-04-22T23:35:01.809 INFO:tasks.workunit.client.0.smithi107.stdout:1/156: symlink d7/l34 0 2024-04-22T23:35:01.809 INFO:tasks.workunit.client.0.smithi107.stdout:1/157: creat d7/d19/d2d/f35 x:0 0 0 2024-04-22T23:35:01.812 INFO:tasks.workunit.client.0.smithi107.stdout:0/169: dwrite d1/d9/f29 [0,4194304] 0 2024-04-22T23:35:01.812 INFO:tasks.workunit.client.0.smithi107.stdout:0/170: write d1/d27/d20/d2c/d2e/f39 [1638637,5581] 0 2024-04-22T23:35:01.814 INFO:tasks.workunit.client.0.smithi107.stdout:1/158: getdents d7/d19/d20 0 2024-04-22T23:35:01.815 INFO:tasks.workunit.client.0.smithi107.stdout:1/159: stat f2 0 2024-04-22T23:35:01.816 INFO:tasks.workunit.client.1.smithi157.stdout:6/657: write d1/d17/f94 [3226797,35931] 0 2024-04-22T23:35:01.816 INFO:tasks.workunit.client.1.smithi157.stdout:6/658: truncate d1/d2f/d54/fa5 669144 0 2024-04-22T23:35:01.816 INFO:tasks.workunit.client.1.smithi157.stdout:6/659: truncate d1/d38/d97/fb4 541247 0 2024-04-22T23:35:01.817 INFO:tasks.workunit.client.0.smithi107.stdout:1/160: chown d7/d19/c28 27 1 2024-04-22T23:35:01.817 INFO:tasks.workunit.client.0.smithi107.stdout:1/161: fsync d7/f15 0 2024-04-22T23:35:01.819 INFO:tasks.workunit.client.0.smithi107.stdout:1/162: rename l6 to d7/d19/d1c/l36 0 2024-04-22T23:35:01.821 INFO:tasks.workunit.client.0.smithi107.stdout:1/163: mknod d7/c37 0 2024-04-22T23:35:01.829 INFO:tasks.workunit.client.1.smithi157.stdout:6/660: dread d1/d36/fa3 [0,4194304] 0 2024-04-22T23:35:01.829 INFO:tasks.workunit.client.1.smithi157.stdout:6/661: fsync d1/d2f/d54/d9b/f79 0 2024-04-22T23:35:01.829 INFO:tasks.workunit.client.1.smithi157.stdout:6/662: truncate d1/d2f/d3b/d6c/d83/f9c 5192417 0 2024-04-22T23:35:01.832 INFO:tasks.workunit.client.0.smithi107.stdout:1/164: write f2 [741736,91637] 0 2024-04-22T23:35:01.834 INFO:tasks.workunit.client.0.smithi107.stdout:1/165: unlink d7/d19/f2e 0 2024-04-22T23:35:01.834 INFO:tasks.workunit.client.0.smithi107.stdout:1/166: fsync d7/d19/d1c/f29 0 2024-04-22T23:35:01.834 INFO:tasks.workunit.client.0.smithi107.stdout:1/167: readlink d7/l16 0 2024-04-22T23:35:01.836 INFO:tasks.workunit.client.0.smithi107.stdout:1/168: link f2 d7/d19/d20/f38 0 2024-04-22T23:35:01.836 INFO:tasks.workunit.client.0.smithi107.stdout:0/171: dwrite d1/d27/d20/d2c/d2e/f39 [0,4194304] 0 2024-04-22T23:35:01.838 INFO:tasks.workunit.client.0.smithi107.stdout:0/172: symlink d1/d27/l3f 0 2024-04-22T23:35:01.839 INFO:tasks.workunit.client.0.smithi107.stdout:0/173: mkdir d1/d27/d20/d40 0 2024-04-22T23:35:01.840 INFO:tasks.workunit.client.0.smithi107.stdout:0/174: mkdir d1/d27/d20/d2c/d41 0 2024-04-22T23:35:01.841 INFO:tasks.workunit.client.1.smithi157.stdout:6/663: dwrite d1/d2f/d54/d9b/f79 [0,4194304] 0 2024-04-22T23:35:01.841 INFO:tasks.workunit.client.1.smithi157.stdout:6/664: creat d1/d2f/d3b/d6c/d83/fd0 x:0 0 0 2024-04-22T23:35:01.841 INFO:tasks.workunit.client.1.smithi157.stdout:6/665: write d1/d17/f27 [4468626,73693] 0 2024-04-22T23:35:01.842 INFO:tasks.workunit.client.1.smithi157.stdout:6/666: readlink d1/d38/daf/lb2 0 2024-04-22T23:35:01.842 INFO:tasks.workunit.client.1.smithi157.stdout:6/667: creat d1/d36/fd1 x:0 0 0 2024-04-22T23:35:01.842 INFO:tasks.workunit.client.0.smithi107.stdout:0/175: rename d1/d27/d20/l3c to d1/d27/d20/d2c/d41/l42 0 2024-04-22T23:35:01.842 INFO:tasks.workunit.client.0.smithi107.stdout:0/176: write d1/d9/fc [3558842,16098] 0 2024-04-22T23:35:01.843 INFO:tasks.workunit.client.0.smithi107.stdout:0/177: truncate d1/d27/f38 4750456 0 2024-04-22T23:35:01.844 INFO:tasks.workunit.client.0.smithi107.stdout:0/178: mknod d1/d9/c43 0 2024-04-22T23:35:01.846 INFO:tasks.workunit.client.0.smithi107.stdout:0/179: mknod d1/d9/d14/c44 0 2024-04-22T23:35:01.846 INFO:tasks.workunit.client.0.smithi107.stdout:0/180: chown d1/d27/d20/d2c/c3d 0 1 2024-04-22T23:35:01.847 INFO:tasks.workunit.client.0.smithi107.stdout:1/169: read d7/d19/f2c [108480,91610] 0 2024-04-22T23:35:01.847 INFO:tasks.workunit.client.0.smithi107.stdout:1/170: readlink d7/l34 0 2024-04-22T23:35:01.847 INFO:tasks.workunit.client.1.smithi157.stdout:6/668: dread d1/d2f/d3b/d39/f46 [0,4194304] 0 2024-04-22T23:35:01.848 INFO:tasks.workunit.client.1.smithi157.stdout:6/669: chown d1/d2f/d3b/d6c/d83/f59 0 1 2024-04-22T23:35:01.848 INFO:tasks.workunit.client.1.smithi157.stdout:6/670: dread d1/d2f/d54/f6b [0,4194304] 0 2024-04-22T23:35:01.848 INFO:tasks.workunit.client.1.smithi157.stdout:6/671: read - d1/d2f/d54/f7e zero size 2024-04-22T23:35:01.848 INFO:tasks.workunit.client.1.smithi157.stdout:6/672: fdatasync d1/d17/f24 0 2024-04-22T23:35:01.848 INFO:tasks.workunit.client.0.smithi107.stdout:0/181: rename d1/d27/c28 to d1/d2a/c45 0 2024-04-22T23:35:01.851 INFO:tasks.workunit.client.0.smithi107.stdout:1/171: truncate d7/f2a 1995128 0 2024-04-22T23:35:01.851 INFO:tasks.workunit.client.0.smithi107.stdout:1/172: chown d7/d19/d32 1020654 1 2024-04-22T23:35:01.851 INFO:tasks.workunit.client.0.smithi107.stdout:1/173: write d7/d19/d1c/f27 [5657,4771] 0 2024-04-22T23:35:01.852 INFO:tasks.workunit.client.0.smithi107.stdout:0/182: mknod d1/d27/d20/c46 0 2024-04-22T23:35:01.852 INFO:tasks.workunit.client.0.smithi107.stdout:1/174: mkdir d7/d19/d20/d39 0 2024-04-22T23:35:01.877 INFO:tasks.workunit.client.1.smithi157.stdout:6/673: dwrite d1/d2f/f4a [0,4194304] 0 2024-04-22T23:35:01.878 INFO:tasks.workunit.client.0.smithi107.stdout:0/183: dwrite d1/d9/f3e [0,4194304] 0 2024-04-22T23:35:01.878 INFO:tasks.workunit.client.1.smithi157.stdout:6/674: creat d1/d2f/dc7/dce/fd2 x:0 0 0 2024-04-22T23:35:01.880 INFO:tasks.workunit.client.0.smithi107.stdout:1/175: dwrite d7/d19/f2c [0,4194304] 0 2024-04-22T23:35:01.882 INFO:tasks.workunit.client.0.smithi107.stdout:9/207: sync 2024-04-22T23:35:01.890 INFO:tasks.workunit.client.0.smithi107.stdout:1/176: creat d7/d19/d32/f3a x:0 0 0 2024-04-22T23:35:01.892 INFO:tasks.workunit.client.1.smithi157.stdout:6/675: dread d1/d17/f43 [0,4194304] 0 2024-04-22T23:35:01.892 INFO:tasks.workunit.client.1.smithi157.stdout:6/676: chown d1/d2f/lc9 90 1 2024-04-22T23:35:01.893 INFO:tasks.workunit.client.1.smithi157.stdout:6/677: getdents d1/d2f/d3b/d39/d4e 0 2024-04-22T23:35:01.893 INFO:tasks.workunit.client.0.smithi107.stdout:9/208: dread d5/d7/f1d [0,4194304] 0 2024-04-22T23:35:01.893 INFO:tasks.workunit.client.0.smithi107.stdout:1/177: creat d7/f3b x:0 0 0 2024-04-22T23:35:01.893 INFO:tasks.workunit.client.0.smithi107.stdout:1/178: chown d7/d19/d2d/f31 1974 1 2024-04-22T23:35:01.893 INFO:tasks.workunit.client.0.smithi107.stdout:1/179: chown d7/d19/d20 16981004 1 2024-04-22T23:35:01.894 INFO:tasks.workunit.client.1.smithi157.stdout:6/678: mkdir d1/d2f/d3b/d39/dd3 0 2024-04-22T23:35:01.894 INFO:tasks.workunit.client.1.smithi157.stdout:6/679: stat d1/d2f/d3b/d6c/d83/f73 0 2024-04-22T23:35:01.894 INFO:tasks.workunit.client.1.smithi157.stdout:6/680: mknod d1/d2f/d3b/d6c/d83/d93/cd4 0 2024-04-22T23:35:01.896 INFO:tasks.workunit.client.0.smithi107.stdout:9/209: creat d5/d7/d17/d1b/f48 x:0 0 0 2024-04-22T23:35:01.896 INFO:tasks.workunit.client.0.smithi107.stdout:9/210: truncate d5/fc 1058969 0 2024-04-22T23:35:01.896 INFO:tasks.workunit.client.0.smithi107.stdout:9/211: chown d5/d7/d27/c45 33992 1 2024-04-22T23:35:01.896 INFO:tasks.workunit.client.0.smithi107.stdout:9/212: chown f4 9 1 2024-04-22T23:35:01.897 INFO:tasks.workunit.client.0.smithi107.stdout:1/180: creat d7/d19/d20/d39/f3c x:0 0 0 2024-04-22T23:35:01.898 INFO:tasks.workunit.client.0.smithi107.stdout:1/181: creat d7/d19/d2d/f3d x:0 0 0 2024-04-22T23:35:01.898 INFO:tasks.workunit.client.0.smithi107.stdout:1/182: creat d7/d19/d20/d39/f3e x:0 0 0 2024-04-22T23:35:01.898 INFO:tasks.workunit.client.0.smithi107.stdout:1/183: creat d7/d19/d32/f3f x:0 0 0 2024-04-22T23:35:01.903 INFO:tasks.workunit.client.0.smithi107.stdout:1/184: mknod d7/d19/d2d/c40 0 2024-04-22T23:35:01.905 INFO:tasks.workunit.client.1.smithi157.stdout:6/681: dread d1/d2f/f4a [0,4194304] 0 2024-04-22T23:35:01.905 INFO:tasks.workunit.client.1.smithi157.stdout:6/682: stat d1/l7b 0 2024-04-22T23:35:01.906 INFO:tasks.workunit.client.0.smithi107.stdout:1/185: creat d7/d19/d20/f41 x:0 0 0 2024-04-22T23:35:01.915 INFO:tasks.workunit.client.0.smithi107.stdout:9/213: dwrite d5/f10 [0,4194304] 0 2024-04-22T23:35:01.915 INFO:tasks.workunit.client.0.smithi107.stdout:9/214: fsync d5/d7/f16 0 2024-04-22T23:35:01.928 INFO:tasks.workunit.client.0.smithi107.stdout:7/130: sync 2024-04-22T23:35:01.928 INFO:tasks.workunit.client.0.smithi107.stdout:8/94: sync 2024-04-22T23:35:01.928 INFO:tasks.workunit.client.0.smithi107.stdout:2/222: sync 2024-04-22T23:35:01.928 INFO:tasks.workunit.client.0.smithi107.stdout:4/153: sync 2024-04-22T23:35:01.929 INFO:tasks.workunit.client.0.smithi107.stdout:2/223: chown d1/d12/d35/d48/d13/d4f 1022290277 1 2024-04-22T23:35:01.929 INFO:tasks.workunit.client.0.smithi107.stdout:5/174: sync 2024-04-22T23:35:01.930 INFO:tasks.workunit.client.0.smithi107.stdout:7/131: unlink d7/d1b/f25 0 2024-04-22T23:35:01.930 INFO:tasks.workunit.client.0.smithi107.stdout:6/176: sync 2024-04-22T23:35:01.931 INFO:tasks.workunit.client.0.smithi107.stdout:2/224: mkdir d1/d12/d35/d54 0 2024-04-22T23:35:01.932 INFO:tasks.workunit.client.0.smithi107.stdout:2/225: mknod d1/d12/d35/d48/d13/c55 0 2024-04-22T23:35:01.934 INFO:tasks.workunit.client.0.smithi107.stdout:2/226: symlink d1/d2b/d49/l56 0 2024-04-22T23:35:01.935 INFO:tasks.workunit.client.0.smithi107.stdout:2/227: mkdir d1/d2b/d3d/d57 0 2024-04-22T23:35:01.935 INFO:tasks.workunit.client.0.smithi107.stdout:2/228: chown d1/d12/d35/d48/d13/l28 17234994 1 2024-04-22T23:35:01.936 INFO:tasks.workunit.client.0.smithi107.stdout:0/184: dwrite d1/d9/f3e [4194304,4194304] 0 2024-04-22T23:35:01.936 INFO:tasks.workunit.client.0.smithi107.stdout:2/229: symlink d1/l58 0 2024-04-22T23:35:01.938 INFO:tasks.workunit.client.0.smithi107.stdout:2/230: mkdir d1/d2b/d59 0 2024-04-22T23:35:01.953 INFO:tasks.workunit.client.0.smithi107.stdout:4/154: dwrite d1/d7/fc [0,4194304] 0 2024-04-22T23:35:01.956 INFO:tasks.workunit.client.0.smithi107.stdout:4/155: rename d1/l26 to d1/d21/d29/d2e/d3b/l3e 0 2024-04-22T23:35:01.958 INFO:tasks.workunit.client.0.smithi107.stdout:4/156: getdents d1/d21 0 2024-04-22T23:35:01.960 INFO:tasks.workunit.client.0.smithi107.stdout:4/157: creat d1/d8/f3f x:0 0 0 2024-04-22T23:35:01.961 INFO:tasks.workunit.client.0.smithi107.stdout:0/185: dread d1/d9/f3e [4194304,4194304] 0 2024-04-22T23:35:01.961 INFO:tasks.workunit.client.0.smithi107.stdout:2/231: dread d1/f11 [0,4194304] 0 2024-04-22T23:35:01.963 INFO:tasks.workunit.client.0.smithi107.stdout:2/232: truncate d1/f3c 765240 0 2024-04-22T23:35:01.963 INFO:tasks.workunit.client.0.smithi107.stdout:2/233: creat d1/d2b/d49/f5a x:0 0 0 2024-04-22T23:35:01.963 INFO:tasks.workunit.client.0.smithi107.stdout:2/234: write d1/d12/d35/d48/d13/f32 [905994,66568] 0 2024-04-22T23:35:01.963 INFO:tasks.workunit.client.0.smithi107.stdout:2/235: creat d1/d12/d35/d48/d13/d4f/f5b x:0 0 0 2024-04-22T23:35:01.964 INFO:tasks.workunit.client.0.smithi107.stdout:2/236: dread d1/d12/d35/d48/f29 [0,4194304] 0 2024-04-22T23:35:01.964 INFO:tasks.workunit.client.0.smithi107.stdout:2/237: write d1/d2b/f39 [12406,64993] 0 2024-04-22T23:35:01.967 INFO:tasks.workunit.client.0.smithi107.stdout:4/158: unlink d1/d21/d29/d2e/d3b/l3e 0 2024-04-22T23:35:01.967 INFO:tasks.workunit.client.0.smithi107.stdout:4/159: write d1/d8/f3f [34853,49364] 0 2024-04-22T23:35:01.968 INFO:tasks.workunit.client.0.smithi107.stdout:8/95: dwrite d0/f3 [0,4194304] 0 2024-04-22T23:35:01.968 INFO:tasks.workunit.client.0.smithi107.stdout:0/186: dread d1/f15 [0,4194304] 0 2024-04-22T23:35:01.968 INFO:tasks.workunit.client.0.smithi107.stdout:0/187: getdents d1/d27/d20/d40 0 2024-04-22T23:35:01.968 INFO:tasks.workunit.client.0.smithi107.stdout:2/238: creat d1/d2b/d3d/d57/f5c x:0 0 0 2024-04-22T23:35:01.968 INFO:tasks.workunit.client.0.smithi107.stdout:2/239: creat d1/d2f/f5d x:0 0 0 2024-04-22T23:35:01.971 INFO:tasks.workunit.client.0.smithi107.stdout:8/96: creat d0/d7/df/f23 x:0 0 0 2024-04-22T23:35:01.972 INFO:tasks.workunit.client.0.smithi107.stdout:0/188: mknod d1/d27/d20/c47 0 2024-04-22T23:35:01.974 INFO:tasks.workunit.client.0.smithi107.stdout:8/97: mkdir d0/d7/d24 0 2024-04-22T23:35:01.982 INFO:tasks.workunit.client.0.smithi107.stdout:4/160: dread d1/d7/f11 [0,4194304] 0 2024-04-22T23:35:01.983 INFO:tasks.workunit.client.1.smithi157.stdout:6/683: dwrite d1/d2f/d54/f95 [4194304,4194304] 0 2024-04-22T23:35:01.984 INFO:tasks.workunit.client.1.smithi157.stdout:6/684: read d1/d17/f24 [1161185,74439] 0 2024-04-22T23:35:01.985 INFO:tasks.workunit.client.1.smithi157.stdout:6/685: dread d1/d2f/d54/f6b [0,4194304] 0 2024-04-22T23:35:01.985 INFO:tasks.workunit.client.1.smithi157.stdout:6/686: creat d1/d36/fd5 x:0 0 0 2024-04-22T23:35:01.985 INFO:tasks.workunit.client.1.smithi157.stdout:6/687: truncate d1/d2f/d54/f7e 865129 0 2024-04-22T23:35:01.989 INFO:tasks.workunit.client.0.smithi107.stdout:0/189: dread d1/d9/f29 [0,4194304] 0 2024-04-22T23:35:01.990 INFO:tasks.workunit.client.0.smithi107.stdout:0/190: rename d1/cf to d1/d27/d20/d2c/d2e/c48 0 2024-04-22T23:35:01.991 INFO:tasks.workunit.client.0.smithi107.stdout:0/191: mknod d1/c49 0 2024-04-22T23:35:01.992 INFO:tasks.workunit.client.0.smithi107.stdout:4/161: dread d1/f13 [0,4194304] 0 2024-04-22T23:35:01.992 INFO:tasks.workunit.client.0.smithi107.stdout:4/162: truncate d1/d8/f3f 210906 0 2024-04-22T23:35:01.992 INFO:tasks.workunit.client.0.smithi107.stdout:0/192: mkdir d1/d27/d20/d4a 0 2024-04-22T23:35:01.993 INFO:tasks.workunit.client.0.smithi107.stdout:4/163: mknod d1/d21/d29/d2e/d3b/c40 0 2024-04-22T23:35:01.994 INFO:tasks.workunit.client.0.smithi107.stdout:8/98: dread d0/f4 [0,4194304] 0 2024-04-22T23:35:01.994 INFO:tasks.workunit.client.0.smithi107.stdout:0/193: rename d1/l22 to d1/d27/l4b 0 2024-04-22T23:35:01.995 INFO:tasks.workunit.client.0.smithi107.stdout:8/99: mknod d0/d7/df/c25 0 2024-04-22T23:35:01.996 INFO:tasks.workunit.client.0.smithi107.stdout:4/164: link d1/d21/d29/d2e/f3d d1/d21/d29/f41 0 2024-04-22T23:35:01.997 INFO:tasks.workunit.client.0.smithi107.stdout:8/100: symlink d0/d7/d8/l26 0 2024-04-22T23:35:01.998 INFO:tasks.workunit.client.0.smithi107.stdout:4/165: mknod d1/d21/d29/d2e/c42 0 2024-04-22T23:35:02.005 INFO:tasks.workunit.client.0.smithi107.stdout:9/215: dwrite d5/f10 [0,4194304] 0 2024-04-22T23:35:02.005 INFO:tasks.workunit.client.0.smithi107.stdout:8/101: dread d0/f3 [0,4194304] 0 2024-04-22T23:35:02.005 INFO:tasks.workunit.client.0.smithi107.stdout:6/177: dwrite db/f1c [0,4194304] 0 2024-04-22T23:35:02.005 INFO:tasks.workunit.client.0.smithi107.stdout:8/102: chown d0/fc 298 1 2024-04-22T23:35:02.006 INFO:tasks.workunit.client.0.smithi107.stdout:5/175: dwrite d9/fd [0,4194304] 0 2024-04-22T23:35:02.006 INFO:tasks.workunit.client.0.smithi107.stdout:5/176: read - d9/db/f27 zero size 2024-04-22T23:35:02.006 INFO:tasks.workunit.client.0.smithi107.stdout:5/177: rename d9 to d9/db/de/d3a 22 2024-04-22T23:35:02.006 INFO:tasks.workunit.client.0.smithi107.stdout:2/240: dwrite d1/d12/d35/d48/f3a [0,4194304] 0 2024-04-22T23:35:02.007 INFO:tasks.workunit.client.0.smithi107.stdout:9/216: unlink d5/l25 0 2024-04-22T23:35:02.007 INFO:tasks.workunit.client.0.smithi107.stdout:9/217: stat d5/d7/f16 0 2024-04-22T23:35:02.007 INFO:tasks.workunit.client.0.smithi107.stdout:4/166: dread d1/d8/f3f [0,4194304] 0 2024-04-22T23:35:02.007 INFO:tasks.workunit.client.0.smithi107.stdout:6/178: write db/f12 [1323782,69728] 0 2024-04-22T23:35:02.007 INFO:tasks.workunit.client.0.smithi107.stdout:6/179: chown db/d20/l28 0 1 2024-04-22T23:35:02.010 INFO:tasks.workunit.client.0.smithi107.stdout:8/103: truncate d0/fb 3543607 0 2024-04-22T23:35:02.010 INFO:tasks.workunit.client.0.smithi107.stdout:8/104: dread - d0/d7/df/d18/f1f zero size 2024-04-22T23:35:02.010 INFO:tasks.workunit.client.0.smithi107.stdout:8/105: fdatasync d0/d7/df/f23 0 2024-04-22T23:35:02.017 INFO:tasks.workunit.client.0.smithi107.stdout:6/180: dread db/f1c [0,4194304] 0 2024-04-22T23:35:02.024 INFO:tasks.workunit.client.0.smithi107.stdout:2/241: creat d1/d2b/d3d/f5e x:0 0 0 2024-04-22T23:35:02.025 INFO:tasks.workunit.client.0.smithi107.stdout:0/194: getdents d1/d27 0 2024-04-22T23:35:02.031 INFO:tasks.workunit.client.0.smithi107.stdout:5/178: dwrite f8 [4194304,4194304] 0 2024-04-22T23:35:02.032 INFO:tasks.workunit.client.0.smithi107.stdout:9/218: dwrite f1 [0,4194304] 0 2024-04-22T23:35:02.032 INFO:tasks.workunit.client.0.smithi107.stdout:9/219: dread - d5/d14/f20 zero size 2024-04-22T23:35:02.033 INFO:tasks.workunit.client.0.smithi107.stdout:2/242: creat d1/d12/d35/d54/f5f x:0 0 0 2024-04-22T23:35:02.034 INFO:tasks.workunit.client.0.smithi107.stdout:8/106: getdents d0/d7/df 0 2024-04-22T23:35:02.037 INFO:tasks.workunit.client.0.smithi107.stdout:9/220: mknod d5/d14/d2e/d43/c49 0 2024-04-22T23:35:02.037 INFO:tasks.workunit.client.0.smithi107.stdout:9/221: chown d5/d14 201540 1 2024-04-22T23:35:02.040 INFO:tasks.workunit.client.0.smithi107.stdout:9/222: rename d5/d7/d17/d1b/l23 to d5/d14/d28/l4a 0 2024-04-22T23:35:02.041 INFO:tasks.workunit.client.0.smithi107.stdout:9/223: mkdir d5/d7/d27/d46/d4b 0 2024-04-22T23:35:02.043 INFO:tasks.workunit.client.0.smithi107.stdout:9/224: symlink d5/d14/d2e/d43/l4c 0 2024-04-22T23:35:02.044 INFO:tasks.workunit.client.0.smithi107.stdout:4/167: dwrite d1/f5 [4194304,4194304] 0 2024-04-22T23:35:02.044 INFO:tasks.workunit.client.0.smithi107.stdout:6/181: dwrite db/f1e [0,4194304] 0 2024-04-22T23:35:02.045 INFO:tasks.workunit.client.0.smithi107.stdout:9/225: unlink d5/d7/ff 0 2024-04-22T23:35:02.045 INFO:tasks.workunit.client.0.smithi107.stdout:9/226: chown d5/d7/d27/d46 0 1 2024-04-22T23:35:02.045 INFO:tasks.workunit.client.0.smithi107.stdout:9/227: fsync d5/f13 0 2024-04-22T23:35:02.046 INFO:tasks.workunit.client.0.smithi107.stdout:9/228: write d5/d7/fd [1704661,45192] 0 2024-04-22T23:35:02.047 INFO:tasks.workunit.client.0.smithi107.stdout:4/168: symlink d1/d21/d29/d2e/d3b/l43 0 2024-04-22T23:35:02.047 INFO:tasks.workunit.client.0.smithi107.stdout:3/222: sync 2024-04-22T23:35:02.047 INFO:tasks.workunit.client.0.smithi107.stdout:3/223: truncate db/d21/d27/d15/d29/f42 517215 0 2024-04-22T23:35:02.047 INFO:tasks.workunit.client.0.smithi107.stdout:2/243: dread d1/f6 [0,4194304] 0 2024-04-22T23:35:02.051 INFO:tasks.workunit.client.0.smithi107.stdout:5/179: dread d9/fd [0,4194304] 0 2024-04-22T23:35:02.051 INFO:tasks.workunit.client.0.smithi107.stdout:5/180: dread - d9/f31 zero size 2024-04-22T23:35:02.051 INFO:tasks.workunit.client.0.smithi107.stdout:9/229: dread d5/f11 [0,4194304] 0 2024-04-22T23:35:02.052 INFO:tasks.workunit.client.0.smithi107.stdout:6/182: unlink db/c18 0 2024-04-22T23:35:02.053 INFO:tasks.workunit.client.0.smithi107.stdout:0/195: dwrite d1/d27/d20/f36 [0,4194304] 0 2024-04-22T23:35:02.055 INFO:tasks.workunit.client.0.smithi107.stdout:9/230: write d5/d7/f16 [1366232,17405] 0 2024-04-22T23:35:02.055 INFO:tasks.workunit.client.0.smithi107.stdout:9/231: chown d5/l6 24 1 2024-04-22T23:35:02.055 INFO:tasks.workunit.client.0.smithi107.stdout:9/232: creat d5/d29/f4d x:0 0 0 2024-04-22T23:35:02.056 INFO:tasks.workunit.client.0.smithi107.stdout:4/169: truncate d1/d7/f11 1164354 0 2024-04-22T23:35:02.056 INFO:tasks.workunit.client.0.smithi107.stdout:4/170: write d1/d7/f32 [5046752,55678] 0 2024-04-22T23:35:02.056 INFO:tasks.workunit.client.0.smithi107.stdout:4/171: stat d1/d7/c1a 0 2024-04-22T23:35:02.057 INFO:tasks.workunit.client.0.smithi107.stdout:3/224: creat db/f51 x:0 0 0 2024-04-22T23:35:02.059 INFO:tasks.workunit.client.0.smithi107.stdout:6/183: link db/le db/d20/l2b 0 2024-04-22T23:35:02.064 INFO:tasks.workunit.client.0.smithi107.stdout:9/233: creat d5/d7/d17/f4e x:0 0 0 2024-04-22T23:35:02.064 INFO:tasks.workunit.client.0.smithi107.stdout:6/184: dread db/f11 [0,4194304] 0 2024-04-22T23:35:02.065 INFO:tasks.workunit.client.0.smithi107.stdout:0/196: dread d1/d27/d20/d2c/d2e/f39 [0,4194304] 0 2024-04-22T23:35:02.066 INFO:tasks.workunit.client.0.smithi107.stdout:2/244: dwrite d1/f1a [0,4194304] 0 2024-04-22T23:35:02.066 INFO:tasks.workunit.client.0.smithi107.stdout:0/197: stat d1/d2a/c37 0 2024-04-22T23:35:02.066 INFO:tasks.workunit.client.0.smithi107.stdout:4/172: mkdir d1/d1b/d44 0 2024-04-22T23:35:02.070 INFO:tasks.workunit.client.0.smithi107.stdout:4/173: dread d1/f28 [0,4194304] 0 2024-04-22T23:35:02.070 INFO:tasks.workunit.client.0.smithi107.stdout:3/225: mkdir db/d21/d27/d15/d24/d52 0 2024-04-22T23:35:02.071 INFO:tasks.workunit.client.0.smithi107.stdout:9/234: mkdir d5/d14/d2e/d4f 0 2024-04-22T23:35:02.071 INFO:tasks.workunit.client.0.smithi107.stdout:9/235: stat d5/f13 0 2024-04-22T23:35:02.071 INFO:tasks.workunit.client.0.smithi107.stdout:9/236: dread - d5/d14/f2f zero size 2024-04-22T23:35:02.071 INFO:tasks.workunit.client.0.smithi107.stdout:9/237: creat d5/d7/d17/d1b/d1f/d36/f50 x:0 0 0 2024-04-22T23:35:02.071 INFO:tasks.workunit.client.0.smithi107.stdout:9/238: write d5/d7/d17/f4e [372354,117302] 0 2024-04-22T23:35:02.073 INFO:tasks.workunit.client.0.smithi107.stdout:5/181: dwrite d9/f37 [0,4194304] 0 2024-04-22T23:35:02.077 INFO:tasks.workunit.client.0.smithi107.stdout:0/198: symlink d1/l4c 0 2024-04-22T23:35:02.078 INFO:tasks.workunit.client.0.smithi107.stdout:4/174: creat d1/d21/d29/f45 x:0 0 0 2024-04-22T23:35:02.079 INFO:tasks.workunit.client.0.smithi107.stdout:6/185: dwrite db/f12 [0,4194304] 0 2024-04-22T23:35:02.079 INFO:tasks.workunit.client.0.smithi107.stdout:3/226: getdents db/d21/d27/d36 0 2024-04-22T23:35:02.084 INFO:tasks.workunit.client.0.smithi107.stdout:2/245: rmdir d1/d12/d35/d48/d13/d4f 39 2024-04-22T23:35:02.086 INFO:tasks.workunit.client.0.smithi107.stdout:4/175: dread d1/d7/fc [0,4194304] 0 2024-04-22T23:35:02.086 INFO:tasks.workunit.client.0.smithi107.stdout:4/176: write d1/d8/f1f [3803512,129354] 0 2024-04-22T23:35:02.088 INFO:tasks.workunit.client.0.smithi107.stdout:1/186: sync 2024-04-22T23:35:02.088 INFO:tasks.workunit.client.0.smithi107.stdout:7/132: sync 2024-04-22T23:35:02.089 INFO:tasks.workunit.client.0.smithi107.stdout:6/186: mkdir db/d20/d2c 0 2024-04-22T23:35:02.091 INFO:tasks.workunit.client.0.smithi107.stdout:2/246: write d1/d12/d35/d48/d13/f40 [905506,73831] 0 2024-04-22T23:35:02.091 INFO:tasks.workunit.client.0.smithi107.stdout:2/247: write d1/d2b/d49/f50 [5034237,83644] 0 2024-04-22T23:35:02.091 INFO:tasks.workunit.client.0.smithi107.stdout:3/227: mkdir db/d21/d27/d22/d25/d53 0 2024-04-22T23:35:02.091 INFO:tasks.workunit.client.0.smithi107.stdout:3/228: readlink db/d21/d27/d15/d29/d2a/l1e 0 2024-04-22T23:35:02.093 INFO:tasks.workunit.client.0.smithi107.stdout:9/239: rename d5/d7/d17/d1b/d1f/d36/f50 to d5/d29/f51 0 2024-04-22T23:35:02.093 INFO:tasks.workunit.client.0.smithi107.stdout:5/182: getdents d9/db 0 2024-04-22T23:35:02.093 INFO:tasks.workunit.client.0.smithi107.stdout:5/183: creat d9/d26/f3b x:0 0 0 2024-04-22T23:35:02.094 INFO:tasks.workunit.client.0.smithi107.stdout:0/199: getdents d1/d27/d20/d2c/d2e 0 2024-04-22T23:35:02.096 INFO:tasks.workunit.client.0.smithi107.stdout:7/133: creat d7/f26 x:0 0 0 2024-04-22T23:35:02.097 INFO:tasks.workunit.client.0.smithi107.stdout:2/248: symlink d1/d12/d35/d48/d13/d4f/l60 0 2024-04-22T23:35:02.098 INFO:tasks.workunit.client.0.smithi107.stdout:1/187: dread f2 [0,4194304] 0 2024-04-22T23:35:02.098 INFO:tasks.workunit.client.0.smithi107.stdout:1/188: readlink d7/d19/l1f 0 2024-04-22T23:35:02.099 INFO:tasks.workunit.client.0.smithi107.stdout:6/187: dread db/f1c [0,4194304] 0 2024-04-22T23:35:02.100 INFO:tasks.workunit.client.0.smithi107.stdout:9/240: mknod d5/d14/d2e/c52 0 2024-04-22T23:35:02.100 INFO:tasks.workunit.client.0.smithi107.stdout:4/177: dwrite d1/f13 [0,4194304] 0 2024-04-22T23:35:02.100 INFO:tasks.workunit.client.0.smithi107.stdout:4/178: write d1/d8/f25 [73934,17250] 0 2024-04-22T23:35:02.102 INFO:tasks.workunit.client.0.smithi107.stdout:0/200: truncate d1/d9/f3e 5651642 0 2024-04-22T23:35:02.103 INFO:tasks.workunit.client.0.smithi107.stdout:7/134: symlink d7/l27 0 2024-04-22T23:35:02.103 INFO:tasks.workunit.client.0.smithi107.stdout:5/184: dread d9/f29 [0,4194304] 0 2024-04-22T23:35:02.104 INFO:tasks.workunit.client.0.smithi107.stdout:1/189: stat d7/cd 0 2024-04-22T23:35:02.104 INFO:tasks.workunit.client.0.smithi107.stdout:5/185: dread d9/d26/f38 [0,4194304] 0 2024-04-22T23:35:02.106 INFO:tasks.workunit.client.0.smithi107.stdout:6/188: creat db/f2d x:0 0 0 2024-04-22T23:35:02.106 INFO:tasks.workunit.client.0.smithi107.stdout:4/179: write d1/d21/d29/f39 [234763,7749] 0 2024-04-22T23:35:02.108 INFO:tasks.workunit.client.0.smithi107.stdout:0/201: creat d1/d9/d14/f4d x:0 0 0 2024-04-22T23:35:02.110 INFO:tasks.workunit.client.0.smithi107.stdout:6/189: creat db/d20/d2a/f2e x:0 0 0 2024-04-22T23:35:02.110 INFO:tasks.workunit.client.0.smithi107.stdout:6/190: read - db/f1d zero size 2024-04-22T23:35:02.110 INFO:tasks.workunit.client.0.smithi107.stdout:6/191: creat db/d20/d2a/f2f x:0 0 0 2024-04-22T23:35:02.110 INFO:tasks.workunit.client.0.smithi107.stdout:6/192: write db/f15 [2561976,65281] 0 2024-04-22T23:35:02.112 INFO:tasks.workunit.client.0.smithi107.stdout:1/190: mkdir d7/d19/d42 0 2024-04-22T23:35:02.113 INFO:tasks.workunit.client.0.smithi107.stdout:0/202: symlink d1/d9/l4e 0 2024-04-22T23:35:02.114 INFO:tasks.workunit.client.0.smithi107.stdout:3/229: rename db/d21/d27/d22 to db/d21/d27/d54 0 2024-04-22T23:35:02.115 INFO:tasks.workunit.client.0.smithi107.stdout:3/230: chown db/d21/d27/d15/d24/c4d 18939 1 2024-04-22T23:35:02.119 INFO:tasks.workunit.client.0.smithi107.stdout:2/249: dwrite d1/d12/d35/f3f [0,4194304] 0 2024-04-22T23:35:02.120 INFO:tasks.workunit.client.0.smithi107.stdout:5/186: dread d9/db/de/f1f [0,4194304] 0 2024-04-22T23:35:02.120 INFO:tasks.workunit.client.0.smithi107.stdout:6/193: mknod db/d20/c30 0 2024-04-22T23:35:02.120 INFO:tasks.workunit.client.0.smithi107.stdout:6/194: write db/f1f [2783676,46732] 0 2024-04-22T23:35:02.120 INFO:tasks.workunit.client.0.smithi107.stdout:6/195: creat db/d20/d2a/f31 x:0 0 0 2024-04-22T23:35:02.121 INFO:tasks.workunit.client.0.smithi107.stdout:1/191: mkdir d7/d19/d1c/d43 0 2024-04-22T23:35:02.121 INFO:tasks.workunit.client.0.smithi107.stdout:1/192: dread - d7/d19/d32/f3f zero size 2024-04-22T23:35:02.121 INFO:tasks.workunit.client.0.smithi107.stdout:1/193: dread - d7/f3b zero size 2024-04-22T23:35:02.122 INFO:tasks.workunit.client.0.smithi107.stdout:9/241: dwrite d5/d29/f51 [0,4194304] 0 2024-04-22T23:35:02.124 INFO:tasks.workunit.client.0.smithi107.stdout:7/135: dwrite d7/f13 [4194304,4194304] 0 2024-04-22T23:35:02.124 INFO:tasks.workunit.client.0.smithi107.stdout:7/136: truncate d7/d1b/f24 376677 0 2024-04-22T23:35:02.125 INFO:tasks.workunit.client.0.smithi107.stdout:7/137: write d7/f11 [1472102,5096] 0 2024-04-22T23:35:02.125 INFO:tasks.workunit.client.0.smithi107.stdout:4/180: dread d1/d8/f1f [0,4194304] 0 2024-04-22T23:35:02.133 INFO:tasks.workunit.client.0.smithi107.stdout:7/138: dread d7/f13 [0,4194304] 0 2024-04-22T23:35:02.133 INFO:tasks.workunit.client.0.smithi107.stdout:7/139: write d7/f26 [610975,81835] 0 2024-04-22T23:35:02.133 INFO:tasks.workunit.client.0.smithi107.stdout:2/250: link d1/d12/d35/d48/d13/c55 d1/d2b/d49/c61 0 2024-04-22T23:35:02.133 INFO:tasks.workunit.client.0.smithi107.stdout:2/251: chown d1/d2b/f39 4415688 1 2024-04-22T23:35:02.136 INFO:tasks.workunit.client.0.smithi107.stdout:6/196: creat db/d20/d2c/f32 x:0 0 0 2024-04-22T23:35:02.137 INFO:tasks.workunit.client.0.smithi107.stdout:5/187: dwrite d9/f29 [0,4194304] 0 2024-04-22T23:35:02.137 INFO:tasks.workunit.client.0.smithi107.stdout:1/194: mknod d7/d19/d42/c44 0 2024-04-22T23:35:02.137 INFO:tasks.workunit.client.0.smithi107.stdout:1/195: write d7/d19/d20/f41 [422337,109869] 0 2024-04-22T23:35:02.137 INFO:tasks.workunit.client.0.smithi107.stdout:1/196: stat d7/d19/d1c/f33 0 2024-04-22T23:35:02.141 INFO:tasks.workunit.client.0.smithi107.stdout:9/242: mkdir d5/d7/d17/d1b/d1f/d36/d53 0 2024-04-22T23:35:02.141 INFO:tasks.workunit.client.0.smithi107.stdout:7/140: write d7/f13 [6711354,73105] 0 2024-04-22T23:35:02.142 INFO:tasks.workunit.client.0.smithi107.stdout:0/203: getdents d1/d9 0 2024-04-22T23:35:02.143 INFO:tasks.workunit.client.0.smithi107.stdout:4/181: truncate d1/d7/f11 960837 0 2024-04-22T23:35:02.143 INFO:tasks.workunit.client.0.smithi107.stdout:4/182: readlink d1/l14 0 2024-04-22T23:35:02.146 INFO:tasks.workunit.client.0.smithi107.stdout:2/252: rename d1/fb to d1/d12/d35/d48/d13/d4f/f62 0 2024-04-22T23:35:02.146 INFO:tasks.workunit.client.0.smithi107.stdout:2/253: read - d1/d12/f46 zero size 2024-04-22T23:35:02.146 INFO:tasks.workunit.client.0.smithi107.stdout:1/197: dread d7/d19/f26 [0,4194304] 0 2024-04-22T23:35:02.146 INFO:tasks.workunit.client.0.smithi107.stdout:6/197: mknod db/c33 0 2024-04-22T23:35:02.148 INFO:tasks.workunit.client.0.smithi107.stdout:5/188: creat d9/db/f3c x:0 0 0 2024-04-22T23:35:02.148 INFO:tasks.workunit.client.0.smithi107.stdout:8/107: sync 2024-04-22T23:35:02.148 INFO:tasks.workunit.client.0.smithi107.stdout:8/108: readlink d0/l12 0 2024-04-22T23:35:02.149 INFO:tasks.workunit.client.0.smithi107.stdout:9/243: write d5/d29/f51 [3200837,22072] 0 2024-04-22T23:35:02.149 INFO:tasks.workunit.client.0.smithi107.stdout:9/244: stat d5/d7/d27/d46/d4b 0 2024-04-22T23:35:02.149 INFO:tasks.workunit.client.0.smithi107.stdout:9/245: write d5/f13 [868102,472] 0 2024-04-22T23:35:02.149 INFO:tasks.workunit.client.0.smithi107.stdout:9/246: fdatasync d5/d29/f3a 0 2024-04-22T23:35:02.149 INFO:tasks.workunit.client.0.smithi107.stdout:9/247: fsync d5/d7/d17/f4e 0 2024-04-22T23:35:02.149 INFO:tasks.workunit.client.0.smithi107.stdout:9/248: read d5/f13 [670313,87483] 0 2024-04-22T23:35:02.151 INFO:tasks.workunit.client.0.smithi107.stdout:0/204: dread d1/d27/f38 [0,4194304] 0 2024-04-22T23:35:02.152 INFO:tasks.workunit.client.0.smithi107.stdout:2/254: truncate d1/d12/d35/f3b 2949375 0 2024-04-22T23:35:02.154 INFO:tasks.workunit.client.0.smithi107.stdout:1/198: symlink d7/d19/d1c/l45 0 2024-04-22T23:35:02.154 INFO:tasks.workunit.client.0.smithi107.stdout:1/199: chown d7/d19/d2d 399227212 1 2024-04-22T23:35:02.154 INFO:tasks.workunit.client.0.smithi107.stdout:6/198: dread db/fd [0,4194304] 0 2024-04-22T23:35:02.158 INFO:tasks.workunit.client.0.smithi107.stdout:2/255: dread d1/d12/d35/f3f [0,4194304] 0 2024-04-22T23:35:02.159 INFO:tasks.workunit.client.0.smithi107.stdout:9/249: symlink d5/d14/d2e/d43/l54 0 2024-04-22T23:35:02.163 INFO:tasks.workunit.client.0.smithi107.stdout:7/141: rmdir d7 39 2024-04-22T23:35:02.164 INFO:tasks.workunit.client.0.smithi107.stdout:4/183: rmdir d1/d21/d29/d2e/d3b 39 2024-04-22T23:35:02.164 INFO:tasks.workunit.client.0.smithi107.stdout:4/184: chown d1/d21/d29/f45 308 1 2024-04-22T23:35:02.164 INFO:tasks.workunit.client.0.smithi107.stdout:4/185: creat d1/d21/d29/d2e/f46 x:0 0 0 2024-04-22T23:35:02.166 INFO:tasks.workunit.client.0.smithi107.stdout:0/205: mkdir d1/d9/d14/d4f 0 2024-04-22T23:35:02.167 INFO:tasks.workunit.client.0.smithi107.stdout:8/109: dwrite d0/f3 [0,4194304] 0 2024-04-22T23:35:02.168 INFO:tasks.workunit.client.0.smithi107.stdout:1/200: mkdir d7/d19/d1c/d43/d46 0 2024-04-22T23:35:02.168 INFO:tasks.workunit.client.0.smithi107.stdout:1/201: chown d7/f3b 24 1 2024-04-22T23:35:02.168 INFO:tasks.workunit.client.0.smithi107.stdout:2/256: dread d1/f2d [0,4194304] 0 2024-04-22T23:35:02.168 INFO:tasks.workunit.client.0.smithi107.stdout:2/257: dread - d1/d12/d35/d48/d13/d4f/f53 zero size 2024-04-22T23:35:02.169 INFO:tasks.workunit.client.0.smithi107.stdout:6/199: symlink db/d20/d2a/l34 0 2024-04-22T23:35:02.169 INFO:tasks.workunit.client.0.smithi107.stdout:6/200: readlink db/l19 0 2024-04-22T23:35:02.169 INFO:tasks.workunit.client.0.smithi107.stdout:6/201: dread - db/f1d zero size 2024-04-22T23:35:02.170 INFO:tasks.workunit.client.0.smithi107.stdout:5/189: rmdir d9/db 39 2024-04-22T23:35:02.173 INFO:tasks.workunit.client.0.smithi107.stdout:7/142: creat d7/d1b/f28 x:0 0 0 2024-04-22T23:35:02.177 INFO:tasks.workunit.client.0.smithi107.stdout:0/206: rename d1/d27/d20/d2c/d2e/c48 to d1/d27/d20/c50 0 2024-04-22T23:35:02.177 INFO:tasks.workunit.client.0.smithi107.stdout:0/207: write d1/f15 [3789951,83823] 0 2024-04-22T23:35:02.178 INFO:tasks.workunit.client.0.smithi107.stdout:0/208: stat d1/l4c 0 2024-04-22T23:35:02.181 INFO:tasks.workunit.client.0.smithi107.stdout:8/110: symlink d0/d7/df/l27 0 2024-04-22T23:35:02.181 INFO:tasks.workunit.client.0.smithi107.stdout:8/111: stat d0/d7/df/f23 0 2024-04-22T23:35:02.181 INFO:tasks.workunit.client.0.smithi107.stdout:8/112: chown d0/d7/df/c25 40 1 2024-04-22T23:35:02.181 INFO:tasks.workunit.client.0.smithi107.stdout:8/113: chown d0 18896 1 2024-04-22T23:35:02.181 INFO:tasks.workunit.client.0.smithi107.stdout:8/114: readlink d0/d7/df/l27 0 2024-04-22T23:35:02.183 INFO:tasks.workunit.client.0.smithi107.stdout:6/202: mkdir db/d20/d2a/d35 0 2024-04-22T23:35:02.184 INFO:tasks.workunit.client.0.smithi107.stdout:5/190: mknod d9/db/de/c3d 0 2024-04-22T23:35:02.185 INFO:tasks.workunit.client.0.smithi107.stdout:7/143: unlink d7/c10 0 2024-04-22T23:35:02.186 INFO:tasks.workunit.client.0.smithi107.stdout:4/186: dwrite d1/d21/d29/f39 [0,4194304] 0 2024-04-22T23:35:02.186 INFO:tasks.workunit.client.0.smithi107.stdout:0/209: mknod d1/d2a/c51 0 2024-04-22T23:35:02.186 INFO:tasks.workunit.client.0.smithi107.stdout:0/210: write d1/d27/f21 [607302,112393] 0 2024-04-22T23:35:02.187 INFO:tasks.workunit.client.0.smithi107.stdout:8/115: link d0/d7/df/c1a d0/d7/c28 0 2024-04-22T23:35:02.187 INFO:tasks.workunit.client.0.smithi107.stdout:8/116: truncate d0/d7/f11 4743690 0 2024-04-22T23:35:02.188 INFO:tasks.workunit.client.0.smithi107.stdout:6/203: symlink db/d20/l36 0 2024-04-22T23:35:02.189 INFO:tasks.workunit.client.0.smithi107.stdout:5/191: truncate d9/d26/f32 515451 0 2024-04-22T23:35:02.190 INFO:tasks.workunit.client.0.smithi107.stdout:7/144: symlink d7/d21/l29 0 2024-04-22T23:35:02.194 INFO:tasks.workunit.client.0.smithi107.stdout:7/145: creat d7/f2a x:0 0 0 2024-04-22T23:35:02.194 INFO:tasks.workunit.client.0.smithi107.stdout:7/146: rename d7 to d7/d2b 22 2024-04-22T23:35:02.195 INFO:tasks.workunit.client.0.smithi107.stdout:1/202: dwrite d7/d19/d2d/f31 [0,4194304] 0 2024-04-22T23:35:02.195 INFO:tasks.workunit.client.0.smithi107.stdout:1/203: stat d7/d19/d1c/l45 0 2024-04-22T23:35:02.195 INFO:tasks.workunit.client.0.smithi107.stdout:4/187: write d1/f13 [867331,9808] 0 2024-04-22T23:35:02.196 INFO:tasks.workunit.client.0.smithi107.stdout:4/188: dread - d1/d8/d23/d27/f30 zero size 2024-04-22T23:35:02.197 INFO:tasks.workunit.client.0.smithi107.stdout:7/147: rename d7/d21/l29 to d7/l2c 0 2024-04-22T23:35:02.197 INFO:tasks.workunit.client.0.smithi107.stdout:7/148: chown d7/f26 305775416 1 2024-04-22T23:35:02.202 INFO:tasks.workunit.client.0.smithi107.stdout:5/192: dread d9/f37 [0,4194304] 0 2024-04-22T23:35:02.202 INFO:tasks.workunit.client.0.smithi107.stdout:2/258: dwrite d1/d12/d35/d48/d13/f40 [0,4194304] 0 2024-04-22T23:35:02.202 INFO:tasks.workunit.client.0.smithi107.stdout:5/193: chown d9/db/de/c16 47212 1 2024-04-22T23:35:02.206 INFO:tasks.workunit.client.0.smithi107.stdout:1/204: rename d7/d19/d32/f3f to d7/d19/d1c/d43/d46/f47 0 2024-04-22T23:35:02.206 INFO:tasks.workunit.client.0.smithi107.stdout:1/205: chown d7/d19/l1f 239 1 2024-04-22T23:35:02.212 INFO:tasks.workunit.client.0.smithi107.stdout:8/117: dwrite d0/f19 [0,4194304] 0 2024-04-22T23:35:02.212 INFO:tasks.workunit.client.0.smithi107.stdout:8/118: readlink d0/d7/d8/l26 0 2024-04-22T23:35:02.212 INFO:tasks.workunit.client.0.smithi107.stdout:8/119: creat d0/d7/d8/f29 x:0 0 0 2024-04-22T23:35:02.212 INFO:tasks.workunit.client.0.smithi107.stdout:8/120: fdatasync d0/d7/f11 0 2024-04-22T23:35:02.213 INFO:tasks.workunit.client.0.smithi107.stdout:8/121: write d0/d7/df/f23 [290855,48749] 0 2024-04-22T23:35:02.213 INFO:tasks.workunit.client.0.smithi107.stdout:2/259: unlink d1/d2b/f4e 0 2024-04-22T23:35:02.216 INFO:tasks.workunit.client.0.smithi107.stdout:1/206: creat d7/d19/d1c/d43/f48 x:0 0 0 2024-04-22T23:35:02.222 INFO:tasks.workunit.client.0.smithi107.stdout:0/211: dwrite d1/f15 [0,4194304] 0 2024-04-22T23:35:02.224 INFO:tasks.workunit.client.0.smithi107.stdout:5/194: dwrite d9/db/f1d [0,4194304] 0 2024-04-22T23:35:02.224 INFO:tasks.workunit.client.0.smithi107.stdout:6/204: dwrite db/f1e [0,4194304] 0 2024-04-22T23:35:02.225 INFO:tasks.workunit.client.0.smithi107.stdout:2/260: mkdir d1/d63 0 2024-04-22T23:35:02.225 INFO:tasks.workunit.client.0.smithi107.stdout:7/149: dwrite d7/d21/f22 [0,4194304] 0 2024-04-22T23:35:02.227 INFO:tasks.workunit.client.0.smithi107.stdout:6/205: write db/f11 [1345129,103499] 0 2024-04-22T23:35:02.227 INFO:tasks.workunit.client.0.smithi107.stdout:6/206: write db/f1e [4683151,120064] 0 2024-04-22T23:35:02.227 INFO:tasks.workunit.client.0.smithi107.stdout:1/207: unlink d7/d19/d1c/d43/f48 0 2024-04-22T23:35:02.227 INFO:tasks.workunit.client.0.smithi107.stdout:1/208: truncate d7/d19/d2d/f35 244073 0 2024-04-22T23:35:02.227 INFO:tasks.workunit.client.0.smithi107.stdout:1/209: write d7/f15 [2010192,35968] 0 2024-04-22T23:35:02.227 INFO:tasks.workunit.client.0.smithi107.stdout:1/210: fdatasync d7/d19/d20/f41 0 2024-04-22T23:35:02.229 INFO:tasks.workunit.client.0.smithi107.stdout:0/212: truncate d1/d9/f29 1964040 0 2024-04-22T23:35:02.230 INFO:tasks.workunit.client.0.smithi107.stdout:5/195: symlink d9/d26/l3e 0 2024-04-22T23:35:02.236 INFO:tasks.workunit.client.0.smithi107.stdout:2/261: mknod d1/d12/d35/d54/c64 0 2024-04-22T23:35:02.239 INFO:tasks.workunit.client.0.smithi107.stdout:5/196: write d9/f37 [2112363,70055] 0 2024-04-22T23:35:02.240 INFO:tasks.workunit.client.0.smithi107.stdout:7/150: chown d7/cc 2509 1 2024-04-22T23:35:02.243 INFO:tasks.workunit.client.0.smithi107.stdout:6/207: mkdir db/d37 0 2024-04-22T23:35:02.243 INFO:tasks.workunit.client.0.smithi107.stdout:6/208: truncate db/f26 501628 0 2024-04-22T23:35:02.245 INFO:tasks.workunit.client.0.smithi107.stdout:0/213: mkdir d1/d27/d20/d2c/d41/d52 0 2024-04-22T23:35:02.245 INFO:tasks.workunit.client.0.smithi107.stdout:0/214: fsync d1/d27/d20/f36 0 2024-04-22T23:35:02.245 INFO:tasks.workunit.client.0.smithi107.stdout:0/215: stat d1/c49 0 2024-04-22T23:35:02.246 INFO:tasks.workunit.client.0.smithi107.stdout:2/262: creat d1/d63/f65 x:0 0 0 2024-04-22T23:35:02.247 INFO:tasks.workunit.client.0.smithi107.stdout:1/211: dwrite d7/d19/f2c [0,4194304] 0 2024-04-22T23:35:02.247 INFO:tasks.workunit.client.0.smithi107.stdout:1/212: chown d7/d19/f26 302 1 2024-04-22T23:35:02.247 INFO:tasks.workunit.client.0.smithi107.stdout:1/213: write d7/d19/d20/f41 [121765,66280] 0 2024-04-22T23:35:02.248 INFO:tasks.workunit.client.0.smithi107.stdout:7/151: dread d7/f18 [0,4194304] 0 2024-04-22T23:35:02.253 INFO:tasks.workunit.client.0.smithi107.stdout:0/216: symlink d1/d27/d20/d2c/d2e/l53 0 2024-04-22T23:35:02.255 INFO:tasks.workunit.client.0.smithi107.stdout:2/263: symlink d1/d2b/d49/l66 0 2024-04-22T23:35:02.259 INFO:tasks.workunit.client.0.smithi107.stdout:1/214: rename d7/f3b to d7/d19/d1c/d43/f49 0 2024-04-22T23:35:02.260 INFO:tasks.workunit.client.0.smithi107.stdout:6/209: dwrite db/d20/d2c/f32 [0,4194304] 0 2024-04-22T23:35:02.263 INFO:tasks.workunit.client.0.smithi107.stdout:0/217: write d1/d27/d20/f36 [1558987,110529] 0 2024-04-22T23:35:02.273 INFO:tasks.workunit.client.0.smithi107.stdout:7/152: mkdir d7/d2d 0 2024-04-22T23:35:02.273 INFO:tasks.workunit.client.0.smithi107.stdout:1/215: symlink d7/d19/l4a 0 2024-04-22T23:35:02.274 INFO:tasks.workunit.client.0.smithi107.stdout:0/218: creat d1/d27/d20/d2c/d41/f54 x:0 0 0 2024-04-22T23:35:02.276 INFO:tasks.workunit.client.0.smithi107.stdout:2/264: dwrite d1/d12/d35/f3f [0,4194304] 0 2024-04-22T23:35:02.276 INFO:tasks.workunit.client.0.smithi107.stdout:6/210: dwrite db/f2d [0,4194304] 0 2024-04-22T23:35:02.277 INFO:tasks.workunit.client.0.smithi107.stdout:0/219: truncate d1/d27/d20/d2c/d2e/f39 545162 0 2024-04-22T23:35:02.277 INFO:tasks.workunit.client.0.smithi107.stdout:0/220: readlink d1/d9/l4e 0 2024-04-22T23:35:02.278 INFO:tasks.workunit.client.0.smithi107.stdout:0/221: stat d1/d27/d20/d2c/d2e 0 2024-04-22T23:35:02.278 INFO:tasks.workunit.client.0.smithi107.stdout:0/222: readlink d1/d27/d20/l33 0 2024-04-22T23:35:02.278 INFO:tasks.workunit.client.0.smithi107.stdout:0/223: write d1/d27/f21 [3851335,122264] 0 2024-04-22T23:35:02.278 INFO:tasks.workunit.client.0.smithi107.stdout:0/224: stat d1/d9/cd 0 2024-04-22T23:35:02.279 INFO:tasks.workunit.client.0.smithi107.stdout:7/153: rename d7/c1d to d7/d2d/c2e 0 2024-04-22T23:35:02.289 INFO:tasks.workunit.client.0.smithi107.stdout:0/225: symlink d1/d2a/l55 0 2024-04-22T23:35:02.289 INFO:tasks.workunit.client.0.smithi107.stdout:3/231: sync 2024-04-22T23:35:02.290 INFO:tasks.workunit.client.0.smithi107.stdout:3/232: write db/d21/d27/f39 [622260,30955] 0 2024-04-22T23:35:02.290 INFO:tasks.workunit.client.0.smithi107.stdout:1/216: dwrite d7/d19/f2c [0,4194304] 0 2024-04-22T23:35:02.293 INFO:tasks.workunit.client.0.smithi107.stdout:1/217: chown d7/c12 30101924 1 2024-04-22T23:35:02.293 INFO:tasks.workunit.client.0.smithi107.stdout:1/218: creat d7/d19/d20/d39/f4b x:0 0 0 2024-04-22T23:35:02.294 INFO:tasks.workunit.client.0.smithi107.stdout:0/226: dread d1/d9/f29 [0,4194304] 0 2024-04-22T23:35:02.294 INFO:tasks.workunit.client.0.smithi107.stdout:0/227: chown d1/d9/d14/f4d 245 1 2024-04-22T23:35:02.297 INFO:tasks.workunit.client.0.smithi107.stdout:6/211: dwrite db/f1d [0,4194304] 0 2024-04-22T23:35:02.302 INFO:tasks.workunit.client.0.smithi107.stdout:3/233: dread db/f1c [0,4194304] 0 2024-04-22T23:35:02.302 INFO:tasks.workunit.client.0.smithi107.stdout:3/234: readlink db/d21/d27/d54/l3d 0 2024-04-22T23:35:02.303 INFO:tasks.workunit.client.0.smithi107.stdout:1/219: dread d7/d19/f2c [0,4194304] 0 2024-04-22T23:35:02.303 INFO:tasks.workunit.client.0.smithi107.stdout:1/220: readlink d7/d19/l4a 0 2024-04-22T23:35:02.303 INFO:tasks.workunit.client.0.smithi107.stdout:6/212: mkdir db/d20/d2a/d38 0 2024-04-22T23:35:02.305 INFO:tasks.workunit.client.0.smithi107.stdout:3/235: symlink db/d21/d27/d15/d29/d2a/l55 0 2024-04-22T23:35:02.305 INFO:tasks.workunit.client.0.smithi107.stdout:1/221: dread d7/f17 [0,4194304] 0 2024-04-22T23:35:02.306 INFO:tasks.workunit.client.0.smithi107.stdout:6/213: mknod db/d20/c39 0 2024-04-22T23:35:02.306 INFO:tasks.workunit.client.0.smithi107.stdout:6/214: creat db/f3a x:0 0 0 2024-04-22T23:35:02.306 INFO:tasks.workunit.client.0.smithi107.stdout:6/215: chown db/le 106 1 2024-04-22T23:35:02.307 INFO:tasks.workunit.client.0.smithi107.stdout:1/222: creat d7/d19/d1c/d43/d46/f4c x:0 0 0 2024-04-22T23:35:02.307 INFO:tasks.workunit.client.0.smithi107.stdout:1/223: write d7/d19/d1c/f27 [5241079,120865] 0 2024-04-22T23:35:02.310 INFO:tasks.workunit.client.0.smithi107.stdout:1/224: rename d7/d19/d2d/c40 to d7/d19/d1c/c4d 0 2024-04-22T23:35:02.313 INFO:tasks.workunit.client.0.smithi107.stdout:1/225: rename d7/d19/d20/d39/f3e to d7/d19/d1c/f4e 0 2024-04-22T23:35:02.315 INFO:tasks.workunit.client.0.smithi107.stdout:0/228: dwrite d1/d9/fc [0,4194304] 0 2024-04-22T23:35:02.316 INFO:tasks.workunit.client.0.smithi107.stdout:1/226: getdents d7/d19 0 2024-04-22T23:35:02.317 INFO:tasks.workunit.client.0.smithi107.stdout:1/227: creat d7/d19/d32/f4f x:0 0 0 2024-04-22T23:35:02.317 INFO:tasks.workunit.client.0.smithi107.stdout:1/228: fsync d7/d19/d32/f3a 0 2024-04-22T23:35:02.317 INFO:tasks.workunit.client.0.smithi107.stdout:1/229: write d7/d19/d20/f24 [642894,106556] 0 2024-04-22T23:35:02.317 INFO:tasks.workunit.client.0.smithi107.stdout:3/236: dread db/ff [0,4194304] 0 2024-04-22T23:35:02.317 INFO:tasks.workunit.client.0.smithi107.stdout:3/237: chown db/d21/d27/d15/d29 4202 1 2024-04-22T23:35:02.318 INFO:tasks.workunit.client.0.smithi107.stdout:3/238: write db/d21/d27/d54/f4e [1461011,8265] 0 2024-04-22T23:35:02.319 INFO:tasks.workunit.client.0.smithi107.stdout:1/230: mknod d7/d19/d42/c50 0 2024-04-22T23:35:02.324 INFO:tasks.workunit.client.0.smithi107.stdout:3/239: unlink db/d21/d27/d15/l1f 0 2024-04-22T23:35:02.324 INFO:tasks.workunit.client.0.smithi107.stdout:3/240: readlink db/d21/d27/d54/l31 0 2024-04-22T23:35:02.324 INFO:tasks.workunit.client.0.smithi107.stdout:3/241: dread - db/d21/d27/d15/d29/d2a/f43 zero size 2024-04-22T23:35:02.324 INFO:tasks.workunit.client.0.smithi107.stdout:3/242: chown db/l3f 137 1 2024-04-22T23:35:02.329 INFO:tasks.workunit.client.0.smithi107.stdout:6/216: dwrite db/f1e [0,4194304] 0 2024-04-22T23:35:02.332 INFO:tasks.workunit.client.0.smithi107.stdout:6/217: mkdir db/d20/d2c/d3b 0 2024-04-22T23:35:02.334 INFO:tasks.workunit.client.0.smithi107.stdout:3/243: dread db/d21/d27/d54/d25/f34 [0,4194304] 0 2024-04-22T23:35:02.334 INFO:tasks.workunit.client.0.smithi107.stdout:3/244: dread - db/d21/d27/d15/d29/d2a/f4c zero size 2024-04-22T23:35:02.337 INFO:tasks.workunit.client.0.smithi107.stdout:3/245: read db/ff [1195273,42000] 0 2024-04-22T23:35:02.341 INFO:tasks.workunit.client.0.smithi107.stdout:6/218: link db/f11 db/d20/d2a/d38/f3c 0 2024-04-22T23:35:02.342 INFO:tasks.workunit.client.0.smithi107.stdout:6/219: chown db/f2d 246933 1 2024-04-22T23:35:02.343 INFO:tasks.workunit.client.0.smithi107.stdout:9/250: sync 2024-04-22T23:35:02.344 INFO:tasks.workunit.client.0.smithi107.stdout:9/251: read d5/d29/f51 [1110931,103148] 0 2024-04-22T23:35:02.346 INFO:tasks.workunit.client.0.smithi107.stdout:0/229: truncate d1/d9/fc 3562023 0 2024-04-22T23:35:02.355 INFO:tasks.workunit.client.0.smithi107.stdout:3/246: dwrite db/f1c [0,4194304] 0 2024-04-22T23:35:02.355 INFO:tasks.workunit.client.0.smithi107.stdout:3/247: dread - db/d21/d27/d15/d29/d2a/d45/f49 zero size 2024-04-22T23:35:02.357 INFO:tasks.workunit.client.0.smithi107.stdout:0/230: read d1/f15 [2275147,15875] 0 2024-04-22T23:35:02.362 INFO:tasks.workunit.client.0.smithi107.stdout:3/248: rename db/d21/d27/d54/l48 to db/d21/d27/d36/l56 0 2024-04-22T23:35:02.364 INFO:tasks.workunit.client.0.smithi107.stdout:0/231: rename d1/d27/l3f to d1/l56 0 2024-04-22T23:35:02.364 INFO:tasks.workunit.client.0.smithi107.stdout:3/249: dread db/d21/d27/f20 [0,4194304] 0 2024-04-22T23:35:02.365 INFO:tasks.workunit.client.0.smithi107.stdout:8/122: sync 2024-04-22T23:35:02.365 INFO:tasks.workunit.client.0.smithi107.stdout:5/197: sync 2024-04-22T23:35:02.366 INFO:tasks.workunit.client.0.smithi107.stdout:4/189: sync 2024-04-22T23:35:02.366 INFO:tasks.workunit.client.0.smithi107.stdout:4/190: chown d1/d8/d23/l3c 3884 1 2024-04-22T23:35:02.367 INFO:tasks.workunit.client.0.smithi107.stdout:0/232: creat d1/d27/f57 x:0 0 0 2024-04-22T23:35:02.367 INFO:tasks.workunit.client.0.smithi107.stdout:0/233: creat d1/d27/d20/d2c/d41/f58 x:0 0 0 2024-04-22T23:35:02.367 INFO:tasks.workunit.client.0.smithi107.stdout:0/234: stat d1/d27 0 2024-04-22T23:35:02.368 INFO:tasks.workunit.client.0.smithi107.stdout:9/252: dwrite d5/d7/f16 [0,4194304] 0 2024-04-22T23:35:02.369 INFO:tasks.workunit.client.0.smithi107.stdout:6/220: dwrite db/f2d [0,4194304] 0 2024-04-22T23:35:02.369 INFO:tasks.workunit.client.0.smithi107.stdout:3/250: mknod db/d21/d27/d15/d24/c57 0 2024-04-22T23:35:02.370 INFO:tasks.workunit.client.0.smithi107.stdout:3/251: chown db/d21 101752 1 2024-04-22T23:35:02.370 INFO:tasks.workunit.client.0.smithi107.stdout:8/123: truncate d0/fb 1899638 0 2024-04-22T23:35:02.370 INFO:tasks.workunit.client.0.smithi107.stdout:8/124: read - d0/d7/d8/f1d zero size 2024-04-22T23:35:02.370 INFO:tasks.workunit.client.0.smithi107.stdout:8/125: write d0/d6/f10 [4780698,100007] 0 2024-04-22T23:35:02.371 INFO:tasks.workunit.client.0.smithi107.stdout:5/198: getdents d9/db 0 2024-04-22T23:35:02.371 INFO:tasks.workunit.client.0.smithi107.stdout:4/191: write d1/d7/fc [1375415,47605] 0 2024-04-22T23:35:02.371 INFO:tasks.workunit.client.0.smithi107.stdout:4/192: chown d1/d8 638 1 2024-04-22T23:35:02.375 INFO:tasks.workunit.client.0.smithi107.stdout:0/235: mkdir d1/d9/d14/d4f/d59 0 2024-04-22T23:35:02.375 INFO:tasks.workunit.client.0.smithi107.stdout:0/236: truncate d1/d27/d20/d2c/d41/f58 398007 0 2024-04-22T23:35:02.376 INFO:tasks.workunit.client.0.smithi107.stdout:5/199: write f4 [2231360,76316] 0 2024-04-22T23:35:02.378 INFO:tasks.workunit.client.0.smithi107.stdout:3/252: rename db/f51 to db/d21/d27/d15/d29/d2a/f58 0 2024-04-22T23:35:02.379 INFO:tasks.workunit.client.0.smithi107.stdout:0/237: write d1/d9/f29 [340922,116666] 0 2024-04-22T23:35:02.383 INFO:tasks.workunit.client.0.smithi107.stdout:5/200: rmdir d9/db/de 39 2024-04-22T23:35:02.385 INFO:tasks.workunit.client.0.smithi107.stdout:3/253: unlink db/d21/d27/d15/d24/l32 0 2024-04-22T23:35:02.385 INFO:tasks.workunit.client.0.smithi107.stdout:3/254: fdatasync f0 0 2024-04-22T23:35:02.385 INFO:tasks.workunit.client.0.smithi107.stdout:3/255: read db/d21/d27/f20 [489747,110625] 0 2024-04-22T23:35:02.387 INFO:tasks.workunit.client.0.smithi107.stdout:8/126: rename d0/d7/d8/f1d to d0/d7/df/d17/f2a 0 2024-04-22T23:35:02.388 INFO:tasks.workunit.client.0.smithi107.stdout:0/238: creat d1/d27/d20/d2c/d2e/f5a x:0 0 0 2024-04-22T23:35:02.389 INFO:tasks.workunit.client.0.smithi107.stdout:3/256: mknod db/d21/d27/d54/d25/c59 0 2024-04-22T23:35:02.391 INFO:tasks.workunit.client.0.smithi107.stdout:3/257: unlink db/c3e 0 2024-04-22T23:35:02.391 INFO:tasks.workunit.client.0.smithi107.stdout:4/193: dwrite d1/d21/d29/d2e/f46 [0,4194304] 0 2024-04-22T23:35:02.392 INFO:tasks.workunit.client.0.smithi107.stdout:3/258: rename db/c11 to db/d21/c5a 0 2024-04-22T23:35:02.393 INFO:tasks.workunit.client.0.smithi107.stdout:9/253: dwrite d5/d29/f2c [0,4194304] 0 2024-04-22T23:35:02.393 INFO:tasks.workunit.client.0.smithi107.stdout:9/254: stat d5/d7/l37 0 2024-04-22T23:35:02.393 INFO:tasks.workunit.client.0.smithi107.stdout:9/255: readlink d5/d7/l32 0 2024-04-22T23:35:02.396 INFO:tasks.workunit.client.0.smithi107.stdout:5/201: dread f4 [4194304,4194304] 0 2024-04-22T23:35:02.396 INFO:tasks.workunit.client.0.smithi107.stdout:5/202: fsync d9/db/f39 0 2024-04-22T23:35:02.398 INFO:tasks.workunit.client.0.smithi107.stdout:4/194: unlink d1/d21/d29/l2d 0 2024-04-22T23:35:02.399 INFO:tasks.workunit.client.0.smithi107.stdout:5/203: creat d9/db/f3f x:0 0 0 2024-04-22T23:35:02.399 INFO:tasks.workunit.client.0.smithi107.stdout:5/204: fdatasync d9/f37 0 2024-04-22T23:35:02.400 INFO:tasks.workunit.client.0.smithi107.stdout:4/195: mknod d1/d21/c47 0 2024-04-22T23:35:02.400 INFO:tasks.workunit.client.0.smithi107.stdout:4/196: fsync d1/d21/d29/f39 0 2024-04-22T23:35:02.401 INFO:tasks.workunit.client.0.smithi107.stdout:3/259: dread db/d21/d27/f23 [0,4194304] 0 2024-04-22T23:35:02.402 INFO:tasks.workunit.client.0.smithi107.stdout:3/260: chown db/d21/d27/d54/d25 3697 1 2024-04-22T23:35:02.403 INFO:tasks.workunit.client.0.smithi107.stdout:4/197: mknod d1/d21/d29/c48 0 2024-04-22T23:35:02.404 INFO:tasks.workunit.client.0.smithi107.stdout:3/261: truncate db/f1c 3450159 0 2024-04-22T23:35:02.405 INFO:tasks.workunit.client.0.smithi107.stdout:4/198: creat d1/d1b/f49 x:0 0 0 2024-04-22T23:35:02.407 INFO:tasks.workunit.client.0.smithi107.stdout:4/199: symlink d1/d8/l4a 0 2024-04-22T23:35:02.411 INFO:tasks.workunit.client.0.smithi107.stdout:8/127: dwrite d0/f4 [0,4194304] 0 2024-04-22T23:35:02.411 INFO:tasks.workunit.client.0.smithi107.stdout:8/128: readlink d0/d7/d8/l26 0 2024-04-22T23:35:02.428 INFO:tasks.workunit.client.0.smithi107.stdout:0/239: dwrite d1/d27/f57 [0,4194304] 0 2024-04-22T23:35:02.428 INFO:tasks.workunit.client.0.smithi107.stdout:9/256: dwrite d5/d7/fb [0,4194304] 0 2024-04-22T23:35:02.429 INFO:tasks.workunit.client.0.smithi107.stdout:5/205: dwrite d9/f24 [0,4194304] 0 2024-04-22T23:35:02.429 INFO:tasks.workunit.client.0.smithi107.stdout:3/262: dwrite db/d21/d27/d15/d29/d2a/f43 [0,4194304] 0 2024-04-22T23:35:02.434 INFO:tasks.workunit.client.0.smithi107.stdout:5/206: mknod d9/db/de/c40 0 2024-04-22T23:35:02.437 INFO:tasks.workunit.client.0.smithi107.stdout:8/129: link d0/d7/df/d17/f2a d0/d6/f2b 0 2024-04-22T23:35:02.437 INFO:tasks.workunit.client.0.smithi107.stdout:0/240: dread d1/d9/f3e [0,4194304] 0 2024-04-22T23:35:02.440 INFO:tasks.workunit.client.0.smithi107.stdout:9/257: dread d5/d7/f16 [0,4194304] 0 2024-04-22T23:35:02.440 INFO:tasks.workunit.client.0.smithi107.stdout:9/258: stat d5/d7/d17/d1b/d1f/d36 0 2024-04-22T23:35:02.444 INFO:tasks.workunit.client.0.smithi107.stdout:3/263: link db/d21/d27/f23 db/d21/d27/d15/d29/d2a/f5b 0 2024-04-22T23:35:02.446 INFO:tasks.workunit.client.0.smithi107.stdout:8/130: creat d0/d7/d24/f2c x:0 0 0 2024-04-22T23:35:02.448 INFO:tasks.workunit.client.0.smithi107.stdout:3/264: symlink db/d21/d27/d15/d24/d52/l5c 0 2024-04-22T23:35:02.448 INFO:tasks.workunit.client.0.smithi107.stdout:3/265: chown db/d21/d27/d54/d25/l2b 980 1 2024-04-22T23:35:02.449 INFO:tasks.workunit.client.0.smithi107.stdout:3/266: dread - db/d21/d27/d15/d29/d2a/d45/f49 zero size 2024-04-22T23:35:02.453 INFO:tasks.workunit.client.0.smithi107.stdout:9/259: write d5/d7/fb [4888572,73824] 0 2024-04-22T23:35:02.453 INFO:tasks.workunit.client.0.smithi107.stdout:9/260: fdatasync d5/d7/d17/d1b/f30 0 2024-04-22T23:35:02.454 INFO:tasks.workunit.client.0.smithi107.stdout:9/261: mkdir d5/d7/d17/d1b/d1f/d36/d53/d55 0 2024-04-22T23:35:02.458 INFO:tasks.workunit.client.0.smithi107.stdout:0/241: dwrite d1/d27/d20/d2c/d41/f54 [0,4194304] 0 2024-04-22T23:35:02.458 INFO:tasks.workunit.client.0.smithi107.stdout:0/242: write d1/d27/d20/d2c/d2e/f39 [1438086,1456] 0 2024-04-22T23:35:02.458 INFO:tasks.workunit.client.0.smithi107.stdout:0/243: chown d1/d2a/c37 2762 1 2024-04-22T23:35:02.458 INFO:tasks.workunit.client.0.smithi107.stdout:0/244: truncate d1/d9/f29 2967242 0 2024-04-22T23:35:02.458 INFO:tasks.workunit.client.0.smithi107.stdout:0/245: write d1/d27/f21 [3750968,98975] 0 2024-04-22T23:35:02.459 INFO:tasks.workunit.client.0.smithi107.stdout:7/154: sync 2024-04-22T23:35:02.459 INFO:tasks.workunit.client.0.smithi107.stdout:2/265: sync 2024-04-22T23:35:02.459 INFO:tasks.workunit.client.0.smithi107.stdout:2/266: fsync d1/f5 0 2024-04-22T23:35:02.459 INFO:tasks.workunit.client.0.smithi107.stdout:2/267: fsync d1/d12/f26 0 2024-04-22T23:35:02.460 INFO:tasks.workunit.client.0.smithi107.stdout:2/268: write d1/d12/d35/d48/f1f [2443500,105377] 0 2024-04-22T23:35:02.460 INFO:tasks.workunit.client.0.smithi107.stdout:2/269: readlink d1/d12/d35/d48/l24 0 2024-04-22T23:35:02.460 INFO:tasks.workunit.client.0.smithi107.stdout:2/270: truncate d1/d12/d35/f45 615800 0 2024-04-22T23:35:02.460 INFO:tasks.workunit.client.0.smithi107.stdout:2/271: dread - d1/f4c zero size 2024-04-22T23:35:02.460 INFO:tasks.workunit.client.0.smithi107.stdout:9/262: dread d5/d14/f15 [0,4194304] 0 2024-04-22T23:35:02.460 INFO:tasks.workunit.client.0.smithi107.stdout:5/207: dwrite d9/d26/f3b [0,4194304] 0 2024-04-22T23:35:02.460 INFO:tasks.workunit.client.0.smithi107.stdout:5/208: readlink d9/l33 0 2024-04-22T23:35:02.460 INFO:tasks.workunit.client.0.smithi107.stdout:9/263: write d5/d7/fd [2048245,65091] 0 2024-04-22T23:35:02.460 INFO:tasks.workunit.client.0.smithi107.stdout:9/264: fdatasync d5/d14/f20 0 2024-04-22T23:35:02.460 INFO:tasks.workunit.client.0.smithi107.stdout:5/209: creat d9/d26/f41 x:0 0 0 2024-04-22T23:35:02.462 INFO:tasks.workunit.client.0.smithi107.stdout:8/131: rename d0/d7/d24 to d0/d7/d20/d2d 0 2024-04-22T23:35:02.463 INFO:tasks.workunit.client.0.smithi107.stdout:7/155: creat d7/f2f x:0 0 0 2024-04-22T23:35:02.465 INFO:tasks.workunit.client.0.smithi107.stdout:8/132: dread d0/d7/df/f23 [0,4194304] 0 2024-04-22T23:35:02.469 INFO:tasks.workunit.client.0.smithi107.stdout:3/267: dwrite db/d21/d27/f39 [0,4194304] 0 2024-04-22T23:35:02.470 INFO:tasks.workunit.client.0.smithi107.stdout:3/268: truncate db/d21/d27/d15/d29/d2a/f4c 1040835 0 2024-04-22T23:35:02.470 INFO:tasks.workunit.client.0.smithi107.stdout:9/265: creat d5/d7/d27/d46/d4b/f56 x:0 0 0 2024-04-22T23:35:02.474 INFO:tasks.workunit.client.0.smithi107.stdout:5/210: dread f7 [0,4194304] 0 2024-04-22T23:35:02.474 INFO:tasks.workunit.client.0.smithi107.stdout:5/211: chown c6 12256 1 2024-04-22T23:35:02.474 INFO:tasks.workunit.client.0.smithi107.stdout:7/156: write d7/d1b/f28 [594895,72717] 0 2024-04-22T23:35:02.474 INFO:tasks.workunit.client.0.smithi107.stdout:8/133: rename d0/d7/df/c13 to d0/d7/c2e 0 2024-04-22T23:35:02.475 INFO:tasks.workunit.client.0.smithi107.stdout:8/134: read d0/d7/df/f23 [233359,2499] 0 2024-04-22T23:35:02.479 INFO:tasks.workunit.client.0.smithi107.stdout:0/246: dwrite d1/d27/d20/d2c/d2e/f5a [0,4194304] 0 2024-04-22T23:35:02.479 INFO:tasks.workunit.client.0.smithi107.stdout:0/247: readlink d1/d27/d20/d2c/l30 0 2024-04-22T23:35:02.479 INFO:tasks.workunit.client.0.smithi107.stdout:3/269: link db/d21/d27/d15/d24/c57 db/d21/d27/c5d 0 2024-04-22T23:35:02.479 INFO:tasks.workunit.client.0.smithi107.stdout:3/270: write db/d21/d27/d15/d29/d2a/f58 [707749,99566] 0 2024-04-22T23:35:02.479 INFO:tasks.workunit.client.0.smithi107.stdout:5/212: mkdir d9/db/d42 0 2024-04-22T23:35:02.479 INFO:tasks.workunit.client.0.smithi107.stdout:5/213: creat d9/f43 x:0 0 0 2024-04-22T23:35:02.480 INFO:tasks.workunit.client.0.smithi107.stdout:2/272: rename d1/d2f to d1/d67 0 2024-04-22T23:35:02.481 INFO:tasks.workunit.client.0.smithi107.stdout:2/273: dread - d1/d12/d35/d48/d13/d4f/f5b zero size 2024-04-22T23:35:02.482 INFO:tasks.workunit.client.0.smithi107.stdout:7/157: symlink d7/d2d/l30 0 2024-04-22T23:35:02.482 INFO:tasks.workunit.client.0.smithi107.stdout:7/158: truncate d7/d1b/f24 1036502 0 2024-04-22T23:35:02.482 INFO:tasks.workunit.client.0.smithi107.stdout:8/135: rmdir d0/d6 39 2024-04-22T23:35:02.482 INFO:tasks.workunit.client.0.smithi107.stdout:8/136: creat d0/f2f x:0 0 0 2024-04-22T23:35:02.486 INFO:tasks.workunit.client.0.smithi107.stdout:0/248: symlink d1/d27/d20/l5b 0 2024-04-22T23:35:02.486 INFO:tasks.workunit.client.0.smithi107.stdout:3/271: creat db/d21/d27/d15/d29/d2a/d40/f5e x:0 0 0 2024-04-22T23:35:02.488 INFO:tasks.workunit.client.0.smithi107.stdout:5/214: mknod d9/d26/c44 0 2024-04-22T23:35:02.489 INFO:tasks.workunit.client.0.smithi107.stdout:2/274: creat d1/d12/d35/d48/d13/f68 x:0 0 0 2024-04-22T23:35:02.490 INFO:tasks.workunit.client.0.smithi107.stdout:7/159: fsync d7/f13 0 2024-04-22T23:35:02.492 INFO:tasks.workunit.client.0.smithi107.stdout:8/137: getdents d0/d7/df/d18 0 2024-04-22T23:35:02.494 INFO:tasks.workunit.client.0.smithi107.stdout:0/249: creat d1/d9/d14/d4f/d59/f5c x:0 0 0 2024-04-22T23:35:02.496 INFO:tasks.workunit.client.0.smithi107.stdout:3/272: link db/d21/d27/d15/d29/c3b db/d21/d27/d15/d29/c5f 0 2024-04-22T23:35:02.496 INFO:tasks.workunit.client.0.smithi107.stdout:3/273: write db/d21/d27/d15/d29/d2a/f43 [4195224,127601] 0 2024-04-22T23:35:02.496 INFO:tasks.workunit.client.0.smithi107.stdout:3/274: chown db/d21/d27/d54/f4e 1 1 2024-04-22T23:35:02.497 INFO:tasks.workunit.client.0.smithi107.stdout:5/215: truncate f7 1862686 0 2024-04-22T23:35:02.497 INFO:tasks.workunit.client.0.smithi107.stdout:9/266: dwrite d5/fc [0,4194304] 0 2024-04-22T23:35:02.497 INFO:tasks.workunit.client.0.smithi107.stdout:9/267: creat d5/d7/d17/d1b/d1f/d36/f57 x:0 0 0 2024-04-22T23:35:02.498 INFO:tasks.workunit.client.0.smithi107.stdout:7/160: creat d7/d2d/f31 x:0 0 0 2024-04-22T23:35:02.502 INFO:tasks.workunit.client.0.smithi107.stdout:2/275: write d1/f5 [1475233,115559] 0 2024-04-22T23:35:02.502 INFO:tasks.workunit.client.0.smithi107.stdout:2/276: readlink d1/d12/d35/l41 0 2024-04-22T23:35:02.502 INFO:tasks.workunit.client.0.smithi107.stdout:3/275: mkdir db/d21/d27/d15/d24/d52/d60 0 2024-04-22T23:35:02.502 INFO:tasks.workunit.client.0.smithi107.stdout:3/276: write db/d21/d27/d15/d29/d2a/f4c [1904614,26080] 0 2024-04-22T23:35:02.503 INFO:tasks.workunit.client.0.smithi107.stdout:8/138: read d0/f19 [34263,54607] 0 2024-04-22T23:35:02.503 INFO:tasks.workunit.client.0.smithi107.stdout:5/216: link d9/d26/f38 d9/f45 0 2024-04-22T23:35:02.511 INFO:tasks.workunit.client.0.smithi107.stdout:6/221: sync 2024-04-22T23:35:02.511 INFO:tasks.workunit.client.0.smithi107.stdout:1/231: sync 2024-04-22T23:35:02.511 INFO:tasks.workunit.client.0.smithi107.stdout:6/222: dread - db/d20/d2a/f31 zero size 2024-04-22T23:35:02.513 INFO:tasks.workunit.client.0.smithi107.stdout:0/250: dwrite d1/d9/f3e [4194304,4194304] 0 2024-04-22T23:35:02.513 INFO:tasks.workunit.client.0.smithi107.stdout:2/277: unlink d1/d12/d35/d48/d13/l44 0 2024-04-22T23:35:02.516 INFO:tasks.workunit.client.0.smithi107.stdout:8/139: creat d0/d7/d20/d2d/f30 x:0 0 0 2024-04-22T23:35:02.518 INFO:tasks.workunit.client.0.smithi107.stdout:5/217: mknod d9/d26/c46 0 2024-04-22T23:35:02.518 INFO:tasks.workunit.client.0.smithi107.stdout:5/218: chown d9/l21 1059 1 2024-04-22T23:35:02.518 INFO:tasks.workunit.client.0.smithi107.stdout:5/219: read - d9/d26/f41 zero size 2024-04-22T23:35:02.519 INFO:tasks.workunit.client.0.smithi107.stdout:7/161: creat d7/d21/f32 x:0 0 0 2024-04-22T23:35:02.519 INFO:tasks.workunit.client.0.smithi107.stdout:7/162: creat d7/d2d/f33 x:0 0 0 2024-04-22T23:35:02.520 INFO:tasks.workunit.client.0.smithi107.stdout:9/268: dwrite d5/d29/f3a [0,4194304] 0 2024-04-22T23:35:02.520 INFO:tasks.workunit.client.0.smithi107.stdout:9/269: write d5/f13 [1781030,114610] 0 2024-04-22T23:35:02.520 INFO:tasks.workunit.client.0.smithi107.stdout:1/232: dread d7/f1b [0,4194304] 0 2024-04-22T23:35:02.521 INFO:tasks.workunit.client.0.smithi107.stdout:1/233: fdatasync d7/d19/d20/d39/f3c 0 2024-04-22T23:35:02.521 INFO:tasks.workunit.client.0.smithi107.stdout:3/277: rename db/d21/d27/d15/c1d to db/d21/d27/d54/d25/c61 0 2024-04-22T23:35:02.527 INFO:tasks.workunit.client.0.smithi107.stdout:6/223: dread db/f12 [0,4194304] 0 2024-04-22T23:35:02.527 INFO:tasks.workunit.client.0.smithi107.stdout:6/224: chown db/c16 119747701 1 2024-04-22T23:35:02.527 INFO:tasks.workunit.client.0.smithi107.stdout:8/140: link d0/d7/d8/f29 d0/d7/f31 0 2024-04-22T23:35:02.528 INFO:tasks.workunit.client.0.smithi107.stdout:5/220: mknod d9/c47 0 2024-04-22T23:35:02.528 INFO:tasks.workunit.client.0.smithi107.stdout:5/221: write d9/db/f27 [322994,125770] 0 2024-04-22T23:35:02.533 INFO:tasks.workunit.client.0.smithi107.stdout:4/200: sync 2024-04-22T23:35:02.533 INFO:tasks.workunit.client.0.smithi107.stdout:4/201: creat d1/d8/f4b x:0 0 0 2024-04-22T23:35:02.533 INFO:tasks.workunit.client.0.smithi107.stdout:4/202: fdatasync d1/d21/f36 0 2024-04-22T23:35:02.534 INFO:tasks.workunit.client.0.smithi107.stdout:9/270: mknod d5/d7/d17/d1b/d1f/c58 0 2024-04-22T23:35:02.534 INFO:tasks.workunit.client.0.smithi107.stdout:9/271: readlink d5/l6 0 2024-04-22T23:35:02.535 INFO:tasks.workunit.client.0.smithi107.stdout:9/272: chown d5/d14/f15 0 1 2024-04-22T23:35:02.535 INFO:tasks.workunit.client.0.smithi107.stdout:1/234: mknod d7/d19/d20/c51 0 2024-04-22T23:35:02.537 INFO:tasks.workunit.client.0.smithi107.stdout:6/225: symlink db/d20/l3d 0 2024-04-22T23:35:02.538 INFO:tasks.workunit.client.0.smithi107.stdout:3/278: read db/d21/d27/d15/d29/d2a/f5b [2481650,126083] 0 2024-04-22T23:35:02.538 INFO:tasks.workunit.client.0.smithi107.stdout:0/251: dwrite d1/f15 [0,4194304] 0 2024-04-22T23:35:02.538 INFO:tasks.workunit.client.0.smithi107.stdout:0/252: write d1/f15 [4601051,114800] 0 2024-04-22T23:35:02.539 INFO:tasks.workunit.client.0.smithi107.stdout:8/141: dread d0/f19 [0,4194304] 0 2024-04-22T23:35:02.540 INFO:tasks.workunit.client.0.smithi107.stdout:7/163: symlink d7/l34 0 2024-04-22T23:35:02.540 INFO:tasks.workunit.client.0.smithi107.stdout:7/164: dread - d7/d2d/f31 zero size 2024-04-22T23:35:02.542 INFO:tasks.workunit.client.0.smithi107.stdout:4/203: getdents d1/d21/d29/d2e/d3b 0 2024-04-22T23:35:02.544 INFO:tasks.workunit.client.0.smithi107.stdout:9/273: creat d5/d14/d28/f59 x:0 0 0 2024-04-22T23:35:02.546 INFO:tasks.workunit.client.0.smithi107.stdout:6/226: link db/f21 db/d20/d2a/d38/f3e 0 2024-04-22T23:35:02.549 INFO:tasks.workunit.client.0.smithi107.stdout:0/253: link d1/c6 d1/d9/d14/d4f/c5d 0 2024-04-22T23:35:02.550 INFO:tasks.workunit.client.0.smithi107.stdout:8/142: symlink d0/d7/l32 0 2024-04-22T23:35:02.551 INFO:tasks.workunit.client.0.smithi107.stdout:8/143: dread - d0/d7/df/d17/f2a zero size 2024-04-22T23:35:02.551 INFO:tasks.workunit.client.0.smithi107.stdout:3/279: dread db/f1c [0,4194304] 0 2024-04-22T23:35:02.551 INFO:tasks.workunit.client.0.smithi107.stdout:3/280: creat db/d21/d27/d15/d29/d2a/d45/f62 x:0 0 0 2024-04-22T23:35:02.551 INFO:tasks.workunit.client.0.smithi107.stdout:7/165: dread - d7/f2a zero size 2024-04-22T23:35:02.552 INFO:tasks.workunit.client.0.smithi107.stdout:2/278: dwrite d1/f3c [0,4194304] 0 2024-04-22T23:35:02.552 INFO:tasks.workunit.client.0.smithi107.stdout:2/279: chown d1/d12/d35/d48/d13/d4f/f62 34765974 1 2024-04-22T23:35:02.555 INFO:tasks.workunit.client.0.smithi107.stdout:9/274: link d5/d29/c41 d5/c5a 0 2024-04-22T23:35:02.555 INFO:tasks.workunit.client.0.smithi107.stdout:9/275: write f4 [820908,105552] 0 2024-04-22T23:35:02.555 INFO:tasks.workunit.client.0.smithi107.stdout:9/276: fdatasync d5/d29/f2c 0 2024-04-22T23:35:02.555 INFO:tasks.workunit.client.0.smithi107.stdout:9/277: getdents d5/d14/d2e/d4f 0 2024-04-22T23:35:02.556 INFO:tasks.workunit.client.0.smithi107.stdout:5/222: dwrite d9/d26/f41 [0,4194304] 0 2024-04-22T23:35:02.556 INFO:tasks.workunit.client.0.smithi107.stdout:5/223: fdatasync d9/f37 0 2024-04-22T23:35:02.556 INFO:tasks.workunit.client.0.smithi107.stdout:6/227: write db/f1e [63381,128295] 0 2024-04-22T23:35:02.556 INFO:tasks.workunit.client.0.smithi107.stdout:5/224: truncate d9/db/de/f23 979904 0 2024-04-22T23:35:02.557 INFO:tasks.workunit.client.0.smithi107.stdout:6/228: readlink db/d20/d2a/l34 0 2024-04-22T23:35:02.557 INFO:tasks.workunit.client.0.smithi107.stdout:0/254: creat d1/d9/f5e x:0 0 0 2024-04-22T23:35:02.558 INFO:tasks.workunit.client.0.smithi107.stdout:8/144: mkdir d0/d7/df/d33 0 2024-04-22T23:35:02.558 INFO:tasks.workunit.client.0.smithi107.stdout:8/145: readlink d0/d7/d8/l26 0 2024-04-22T23:35:02.560 INFO:tasks.workunit.client.0.smithi107.stdout:3/281: mknod db/d21/d27/d54/d25/d53/c63 0 2024-04-22T23:35:02.561 INFO:tasks.workunit.client.0.smithi107.stdout:3/282: truncate db/d21/d27/d54/d25/f34 4654950 0 2024-04-22T23:35:02.561 INFO:tasks.workunit.client.0.smithi107.stdout:0/255: read d1/d27/f38 [438903,15895] 0 2024-04-22T23:35:02.561 INFO:tasks.workunit.client.0.smithi107.stdout:0/256: fdatasync d1/d9/fc 0 2024-04-22T23:35:02.569 INFO:tasks.workunit.client.0.smithi107.stdout:9/278: truncate d5/d7/f1d 3992492 0 2024-04-22T23:35:02.575 INFO:tasks.workunit.client.0.smithi107.stdout:5/225: symlink d9/db/d42/l48 0 2024-04-22T23:35:02.576 INFO:tasks.workunit.client.0.smithi107.stdout:5/226: chown d9/db/c11 0 1 2024-04-22T23:35:02.577 INFO:tasks.workunit.client.0.smithi107.stdout:7/166: symlink d7/l35 0 2024-04-22T23:35:02.578 INFO:tasks.workunit.client.0.smithi107.stdout:2/280: getdents d1/d12 0 2024-04-22T23:35:02.581 INFO:tasks.workunit.client.0.smithi107.stdout:9/279: creat d5/d7/d17/d1b/d1f/d36/d53/f5b x:0 0 0 2024-04-22T23:35:02.583 INFO:tasks.workunit.client.0.smithi107.stdout:3/283: symlink db/d21/d27/d15/l64 0 2024-04-22T23:35:02.583 INFO:tasks.workunit.client.0.smithi107.stdout:3/284: write db/ff [2092635,38989] 0 2024-04-22T23:35:02.584 INFO:tasks.workunit.client.0.smithi107.stdout:0/257: dwrite d1/d27/d20/d2c/d2e/f39 [0,4194304] 0 2024-04-22T23:35:02.584 INFO:tasks.workunit.client.0.smithi107.stdout:0/258: creat d1/d9/f5f x:0 0 0 2024-04-22T23:35:02.584 INFO:tasks.workunit.client.0.smithi107.stdout:0/259: creat d1/d27/d20/f60 x:0 0 0 2024-04-22T23:35:02.584 INFO:tasks.workunit.client.0.smithi107.stdout:0/260: write d1/d9/d14/f4d [796336,58116] 0 2024-04-22T23:35:02.584 INFO:tasks.workunit.client.0.smithi107.stdout:7/167: mknod d7/d2d/c36 0 2024-04-22T23:35:02.585 INFO:tasks.workunit.client.0.smithi107.stdout:3/285: write db/d21/d27/d15/d29/d2a/f4c [1847002,88557] 0 2024-04-22T23:35:02.586 INFO:tasks.workunit.client.0.smithi107.stdout:6/229: dwrite db/d20/d2a/d38/f3c [0,4194304] 0 2024-04-22T23:35:02.586 INFO:tasks.workunit.client.0.smithi107.stdout:6/230: chown db/d20/d2a/d38/f3e 15 1 2024-04-22T23:35:02.586 INFO:tasks.workunit.client.0.smithi107.stdout:6/231: fsync db/d20/d2c/f32 0 2024-04-22T23:35:02.586 INFO:tasks.workunit.client.0.smithi107.stdout:6/232: creat db/f3f x:0 0 0 2024-04-22T23:35:02.586 INFO:tasks.workunit.client.0.smithi107.stdout:9/280: symlink d5/d7/d17/d1b/l5c 0 2024-04-22T23:35:02.586 INFO:tasks.workunit.client.0.smithi107.stdout:9/281: chown f4 759056250 1 2024-04-22T23:35:02.586 INFO:tasks.workunit.client.0.smithi107.stdout:9/282: dread - d5/d14/f2f zero size 2024-04-22T23:35:02.587 INFO:tasks.workunit.client.0.smithi107.stdout:7/168: dread d7/f18 [0,4194304] 0 2024-04-22T23:35:02.589 INFO:tasks.workunit.client.0.smithi107.stdout:5/227: dread d9/fd [0,4194304] 0 2024-04-22T23:35:02.590 INFO:tasks.workunit.client.0.smithi107.stdout:0/261: mknod d1/d27/d20/d40/c61 0 2024-04-22T23:35:02.591 INFO:tasks.workunit.client.0.smithi107.stdout:2/281: dread d1/d12/d35/d48/f2c [0,4194304] 0 2024-04-22T23:35:02.591 INFO:tasks.workunit.client.0.smithi107.stdout:2/282: write d1/d2b/d49/f51 [879756,25269] 0 2024-04-22T23:35:02.592 INFO:tasks.workunit.client.0.smithi107.stdout:9/283: symlink d5/d7/d17/l5d 0 2024-04-22T23:35:02.592 INFO:tasks.workunit.client.0.smithi107.stdout:7/169: readlink d7/l2c 0 2024-04-22T23:35:02.593 INFO:tasks.workunit.client.0.smithi107.stdout:5/228: rename d9/db/f10 to d9/db/de/f49 0 2024-04-22T23:35:02.594 INFO:tasks.workunit.client.0.smithi107.stdout:0/262: stat d1/d2a/c45 0 2024-04-22T23:35:02.594 INFO:tasks.workunit.client.0.smithi107.stdout:0/263: chown d1/d27/d20/d2c/d41/d52 3527044 1 2024-04-22T23:35:02.594 INFO:tasks.workunit.client.0.smithi107.stdout:0/264: chown d1/f15 38663 1 2024-04-22T23:35:02.595 INFO:tasks.workunit.client.0.smithi107.stdout:7/170: write d7/f1a [271172,61840] 0 2024-04-22T23:35:02.596 INFO:tasks.workunit.client.0.smithi107.stdout:0/265: rename d1/d9/d14/d4f/c5d to d1/d9/d14/d4f/c62 0 2024-04-22T23:35:02.597 INFO:tasks.workunit.client.0.smithi107.stdout:2/283: rmdir d1/d2b/d3d 39 2024-04-22T23:35:02.598 INFO:tasks.workunit.client.0.smithi107.stdout:2/284: creat d1/d63/f69 x:0 0 0 2024-04-22T23:35:02.598 INFO:tasks.workunit.client.0.smithi107.stdout:2/285: readlink d1/d12/d35/d48/d13/l28 0 2024-04-22T23:35:02.599 INFO:tasks.workunit.client.0.smithi107.stdout:6/233: write f6 [2382604,114065] 0 2024-04-22T23:35:02.601 INFO:tasks.workunit.client.0.smithi107.stdout:0/266: symlink d1/d27/d20/d4a/l63 0 2024-04-22T23:35:02.601 INFO:tasks.workunit.client.0.smithi107.stdout:0/267: creat d1/d27/d20/d2c/d41/f64 x:0 0 0 2024-04-22T23:35:02.601 INFO:tasks.workunit.client.0.smithi107.stdout:8/146: dwrite d0/fb [0,4194304] 0 2024-04-22T23:35:02.601 INFO:tasks.workunit.client.0.smithi107.stdout:8/147: truncate d0/d7/d20/d2d/f2c 393691 0 2024-04-22T23:35:02.601 INFO:tasks.workunit.client.0.smithi107.stdout:2/286: truncate d1/f30 472363 0 2024-04-22T23:35:02.601 INFO:tasks.workunit.client.0.smithi107.stdout:9/284: write d5/f10 [2794063,41784] 0 2024-04-22T23:35:02.602 INFO:tasks.workunit.client.0.smithi107.stdout:4/204: dwrite d1/d21/d29/f34 [0,4194304] 0 2024-04-22T23:35:02.603 INFO:tasks.workunit.client.0.smithi107.stdout:0/268: mknod d1/d9/d14/d4f/c65 0 2024-04-22T23:35:02.607 INFO:tasks.workunit.client.0.smithi107.stdout:9/285: symlink d5/d14/d28/l5e 0 2024-04-22T23:35:02.612 INFO:tasks.workunit.client.0.smithi107.stdout:7/171: dread d7/ff [0,4194304] 0 2024-04-22T23:35:02.612 INFO:tasks.workunit.client.0.smithi107.stdout:0/269: link d1/d9/d14/l19 d1/d9/d14/d4f/l66 0 2024-04-22T23:35:02.613 INFO:tasks.workunit.client.0.smithi107.stdout:8/148: rmdir d0/d7/df/d17 39 2024-04-22T23:35:02.613 INFO:tasks.workunit.client.0.smithi107.stdout:9/286: mknod d5/d7/d17/d1b/d1f/c5f 0 2024-04-22T23:35:02.614 INFO:tasks.workunit.client.0.smithi107.stdout:4/205: rmdir d1/d21/d29/d2e 39 2024-04-22T23:35:02.615 INFO:tasks.workunit.client.0.smithi107.stdout:5/229: dwrite d9/db/f27 [0,4194304] 0 2024-04-22T23:35:02.615 INFO:tasks.workunit.client.0.smithi107.stdout:5/230: creat d9/f4a x:0 0 0 2024-04-22T23:35:02.615 INFO:tasks.workunit.client.0.smithi107.stdout:8/149: creat d0/d7/f34 x:0 0 0 2024-04-22T23:35:02.617 INFO:tasks.workunit.client.0.smithi107.stdout:9/287: rename d5/l2d to d5/d7/l60 0 2024-04-22T23:35:02.619 INFO:tasks.workunit.client.0.smithi107.stdout:2/287: read d1/f1a [1780442,19785] 0 2024-04-22T23:35:02.623 INFO:tasks.workunit.client.0.smithi107.stdout:0/270: read d1/d9/fc [2129035,102004] 0 2024-04-22T23:35:02.624 INFO:tasks.workunit.client.0.smithi107.stdout:9/288: dread f4 [0,4194304] 0 2024-04-22T23:35:02.625 INFO:tasks.workunit.client.0.smithi107.stdout:9/289: creat d5/d29/f61 x:0 0 0 2024-04-22T23:35:02.627 INFO:tasks.workunit.client.0.smithi107.stdout:4/206: creat d1/d21/d29/d2e/d3b/f4c x:0 0 0 2024-04-22T23:35:02.627 INFO:tasks.workunit.client.0.smithi107.stdout:4/207: dread - d1/d8/d23/d27/f30 zero size 2024-04-22T23:35:02.628 INFO:tasks.workunit.client.0.smithi107.stdout:0/271: mkdir d1/d67 0 2024-04-22T23:35:02.631 INFO:tasks.workunit.client.0.smithi107.stdout:9/290: symlink d5/d7/d17/d1b/d1f/d36/d53/d55/l62 0 2024-04-22T23:35:02.633 INFO:tasks.workunit.client.0.smithi107.stdout:4/208: chown d1/d21/d29/l2b 7082109 1 2024-04-22T23:35:02.636 INFO:tasks.workunit.client.0.smithi107.stdout:9/291: rename d5/d24/l34 to d5/d14/d2e/d4f/l63 0 2024-04-22T23:35:02.636 INFO:tasks.workunit.client.0.smithi107.stdout:9/292: write d5/d7/d17/d1b/f30 [4635165,99733] 0 2024-04-22T23:35:02.638 INFO:tasks.workunit.client.0.smithi107.stdout:5/231: rmdir d9/db 39 2024-04-22T23:35:02.638 INFO:tasks.workunit.client.0.smithi107.stdout:5/232: dread - d9/f43 zero size 2024-04-22T23:35:02.639 INFO:tasks.workunit.client.0.smithi107.stdout:8/150: dwrite d0/d6/f2b [0,4194304] 0 2024-04-22T23:35:02.639 INFO:tasks.workunit.client.0.smithi107.stdout:8/151: dread - d0/d7/df/d18/f1f zero size 2024-04-22T23:35:02.639 INFO:tasks.workunit.client.0.smithi107.stdout:8/152: chown d0/d7/d8/c22 22878592 1 2024-04-22T23:35:02.640 INFO:tasks.workunit.client.0.smithi107.stdout:8/153: write d0/d7/f34 [1016707,58755] 0 2024-04-22T23:35:02.640 INFO:tasks.workunit.client.0.smithi107.stdout:2/288: dwrite d1/d12/d35/d48/f1f [0,4194304] 0 2024-04-22T23:35:02.641 INFO:tasks.workunit.client.0.smithi107.stdout:7/172: dwrite d7/f18 [0,4194304] 0 2024-04-22T23:35:02.641 INFO:tasks.workunit.client.0.smithi107.stdout:9/293: link d5/d14/d28/l5e d5/d7/l64 0 2024-04-22T23:35:02.644 INFO:tasks.workunit.client.0.smithi107.stdout:5/233: creat d9/db/d42/f4b x:0 0 0 2024-04-22T23:35:02.644 INFO:tasks.workunit.client.0.smithi107.stdout:5/234: dread - d9/db/f3c zero size 2024-04-22T23:35:02.648 INFO:tasks.workunit.client.0.smithi107.stdout:2/289: symlink d1/d2b/d59/l6a 0 2024-04-22T23:35:02.652 INFO:tasks.workunit.client.0.smithi107.stdout:5/235: symlink d9/l4c 0 2024-04-22T23:35:02.653 INFO:tasks.workunit.client.0.smithi107.stdout:5/236: write d9/db/f3f [874798,72781] 0 2024-04-22T23:35:02.660 INFO:tasks.workunit.client.0.smithi107.stdout:2/290: read d1/d2b/d49/f50 [566960,533] 0 2024-04-22T23:35:02.661 INFO:tasks.workunit.client.0.smithi107.stdout:2/291: rmdir d1/d2b/d3d/d57 39 2024-04-22T23:35:02.661 INFO:tasks.workunit.client.0.smithi107.stdout:7/173: dread d7/fd [0,4194304] 0 2024-04-22T23:35:02.661 INFO:tasks.workunit.client.0.smithi107.stdout:1/235: sync 2024-04-22T23:35:02.661 INFO:tasks.workunit.client.0.smithi107.stdout:1/236: creat d7/d19/d20/f52 x:0 0 0 2024-04-22T23:35:02.661 INFO:tasks.workunit.client.0.smithi107.stdout:1/237: readlink d7/d19/l1f 0 2024-04-22T23:35:02.662 INFO:tasks.workunit.client.0.smithi107.stdout:0/272: dwrite d1/d27/f21 [0,4194304] 0 2024-04-22T23:35:02.662 INFO:tasks.workunit.client.0.smithi107.stdout:0/273: creat d1/d9/f68 x:0 0 0 2024-04-22T23:35:02.662 INFO:tasks.workunit.client.0.smithi107.stdout:2/292: creat d1/d2b/d59/f6b x:0 0 0 2024-04-22T23:35:02.663 INFO:tasks.workunit.client.0.smithi107.stdout:7/174: truncate d7/f1a 139486 0 2024-04-22T23:35:02.663 INFO:tasks.workunit.client.0.smithi107.stdout:4/209: dwrite d1/d21/d29/d2e/d3b/f4c [0,4194304] 0 2024-04-22T23:35:02.663 INFO:tasks.workunit.client.0.smithi107.stdout:8/154: dwrite d0/f4 [0,4194304] 0 2024-04-22T23:35:02.663 INFO:tasks.workunit.client.0.smithi107.stdout:5/237: dread d9/db/f1d [4194304,4194304] 0 2024-04-22T23:35:02.663 INFO:tasks.workunit.client.0.smithi107.stdout:5/238: write d9/f43 [621060,23926] 0 2024-04-22T23:35:02.667 INFO:tasks.workunit.client.0.smithi107.stdout:4/210: dread d1/d8/f25 [0,4194304] 0 2024-04-22T23:35:02.667 INFO:tasks.workunit.client.0.smithi107.stdout:0/274: creat d1/d9/d14/d4f/f69 x:0 0 0 2024-04-22T23:35:02.667 INFO:tasks.workunit.client.0.smithi107.stdout:0/275: dread - d1/d9/d14/d4f/d59/f5c zero size 2024-04-22T23:35:02.669 INFO:tasks.workunit.client.0.smithi107.stdout:2/293: link d1/f4c d1/d2b/d3d/f6c 0 2024-04-22T23:35:02.675 INFO:tasks.workunit.client.0.smithi107.stdout:7/175: mknod d7/d21/c37 0 2024-04-22T23:35:02.675 INFO:tasks.workunit.client.0.smithi107.stdout:7/176: creat d7/d1b/f38 x:0 0 0 2024-04-22T23:35:02.675 INFO:tasks.workunit.client.0.smithi107.stdout:7/177: stat d7/l34 0 2024-04-22T23:35:02.675 INFO:tasks.workunit.client.0.smithi107.stdout:4/211: dread d1/d21/d29/d2e/f46 [0,4194304] 0 2024-04-22T23:35:02.680 INFO:tasks.workunit.client.0.smithi107.stdout:5/239: dwrite f7 [0,4194304] 0 2024-04-22T23:35:02.680 INFO:tasks.workunit.client.0.smithi107.stdout:0/276: link d1/d9/d14/d4f/c62 d1/d27/d20/d4a/c6a 0 2024-04-22T23:35:02.682 INFO:tasks.workunit.client.0.smithi107.stdout:7/178: mknod d7/d21/c39 0 2024-04-22T23:35:02.682 INFO:tasks.workunit.client.0.smithi107.stdout:7/179: truncate d7/f2f 890375 0 2024-04-22T23:35:02.685 INFO:tasks.workunit.client.0.smithi107.stdout:4/212: unlink d1/d21/d29/f34 0 2024-04-22T23:35:02.686 INFO:tasks.workunit.client.0.smithi107.stdout:1/238: rename d7/d19/d1c to d7/d19/d20/d39/d53 0 2024-04-22T23:35:02.689 INFO:tasks.workunit.client.0.smithi107.stdout:4/213: mkdir d1/d21/d29/d2e/d4d 0 2024-04-22T23:35:02.690 INFO:tasks.workunit.client.0.smithi107.stdout:0/277: write d1/d27/f57 [3175569,70379] 0 2024-04-22T23:35:02.690 INFO:tasks.workunit.client.0.smithi107.stdout:0/278: write d1/d27/f57 [3001447,123203] 0 2024-04-22T23:35:02.691 INFO:tasks.workunit.client.0.smithi107.stdout:8/155: rename d0/d7/d8/f29 to d0/d7/f35 0 2024-04-22T23:35:02.691 INFO:tasks.workunit.client.0.smithi107.stdout:5/240: rmdir d9/db/d42 39 2024-04-22T23:35:02.692 INFO:tasks.workunit.client.0.smithi107.stdout:7/180: mknod d7/c3a 0 2024-04-22T23:35:02.692 INFO:tasks.workunit.client.0.smithi107.stdout:4/214: dread d1/d21/d29/d2e/f46 [0,4194304] 0 2024-04-22T23:35:02.694 INFO:tasks.workunit.client.0.smithi107.stdout:0/279: rename d1/d9/f29 to d1/d27/d20/f6b 0 2024-04-22T23:35:02.695 INFO:tasks.workunit.client.0.smithi107.stdout:7/181: creat d7/f3b x:0 0 0 2024-04-22T23:35:02.697 INFO:tasks.workunit.client.0.smithi107.stdout:0/280: truncate d1/d27/f38 1367716 0 2024-04-22T23:35:02.700 INFO:tasks.workunit.client.0.smithi107.stdout:8/156: dread d0/f5 [0,4194304] 0 2024-04-22T23:35:02.705 INFO:tasks.workunit.client.0.smithi107.stdout:7/182: rename d7/d2d/f31 to d7/d2d/f3c 0 2024-04-22T23:35:02.707 INFO:tasks.workunit.client.0.smithi107.stdout:8/157: mknod d0/d6/c36 0 2024-04-22T23:35:02.707 INFO:tasks.workunit.client.0.smithi107.stdout:4/215: dwrite d1/d1b/f49 [0,4194304] 0 2024-04-22T23:35:02.707 INFO:tasks.workunit.client.0.smithi107.stdout:4/216: write d1/d21/f36 [571300,71475] 0 2024-04-22T23:35:02.708 INFO:tasks.workunit.client.0.smithi107.stdout:7/183: rename d7/d21/c39 to d7/d2d/c3d 0 2024-04-22T23:35:02.712 INFO:tasks.workunit.client.0.smithi107.stdout:7/184: mknod d7/d1b/c3e 0 2024-04-22T23:35:02.712 INFO:tasks.workunit.client.0.smithi107.stdout:7/185: fdatasync d7/f2f 0 2024-04-22T23:35:02.714 INFO:tasks.workunit.client.0.smithi107.stdout:0/281: dwrite d1/d27/d20/f6b [0,4194304] 0 2024-04-22T23:35:02.714 INFO:tasks.workunit.client.0.smithi107.stdout:8/158: dread d0/d7/fa [0,4194304] 0 2024-04-22T23:35:02.715 INFO:tasks.workunit.client.0.smithi107.stdout:8/159: chown d0/d7/df/f23 48 1 2024-04-22T23:35:02.719 INFO:tasks.workunit.client.0.smithi107.stdout:4/217: rmdir d1 39 2024-04-22T23:35:02.719 INFO:tasks.workunit.client.0.smithi107.stdout:4/218: chown d1/d1b/d44 5985 1 2024-04-22T23:35:02.720 INFO:tasks.workunit.client.0.smithi107.stdout:7/186: creat d7/f3f x:0 0 0 2024-04-22T23:35:02.720 INFO:tasks.workunit.client.0.smithi107.stdout:3/286: sync 2024-04-22T23:35:02.721 INFO:tasks.workunit.client.0.smithi107.stdout:8/160: truncate d0/d7/df/d17/f2a 3748609 0 2024-04-22T23:35:02.721 INFO:tasks.workunit.client.0.smithi107.stdout:8/161: chown d0/d7/df/l27 6276 1 2024-04-22T23:35:02.721 INFO:tasks.workunit.client.0.smithi107.stdout:8/162: write d0/f2f [575017,84196] 0 2024-04-22T23:35:02.721 INFO:tasks.workunit.client.0.smithi107.stdout:8/163: fdatasync d0/f5 0 2024-04-22T23:35:02.722 INFO:tasks.workunit.client.0.smithi107.stdout:4/219: creat d1/d21/d29/d2e/d4d/f4e x:0 0 0 2024-04-22T23:35:02.724 INFO:tasks.workunit.client.0.smithi107.stdout:4/220: getdents d1/d21/d29 0 2024-04-22T23:35:02.725 INFO:tasks.workunit.client.0.smithi107.stdout:8/164: dread d0/d7/fa [0,4194304] 0 2024-04-22T23:35:02.725 INFO:tasks.workunit.client.0.smithi107.stdout:8/165: getdents d0/d7/df/d18 0 2024-04-22T23:35:02.727 INFO:tasks.workunit.client.0.smithi107.stdout:7/187: rename d7/l14 to d7/d21/l40 0 2024-04-22T23:35:02.727 INFO:tasks.workunit.client.0.smithi107.stdout:7/188: write d7/d2d/f33 [883137,74498] 0 2024-04-22T23:35:02.727 INFO:tasks.workunit.client.0.smithi107.stdout:7/189: creat d7/d2d/f41 x:0 0 0 2024-04-22T23:35:02.727 INFO:tasks.workunit.client.0.smithi107.stdout:7/190: stat d7/cc 0 2024-04-22T23:35:02.727 INFO:tasks.workunit.client.0.smithi107.stdout:4/221: unlink d1/d7/l31 0 2024-04-22T23:35:02.727 INFO:tasks.workunit.client.0.smithi107.stdout:4/222: dread - d1/d21/d29/f41 zero size 2024-04-22T23:35:02.729 INFO:tasks.workunit.client.0.smithi107.stdout:7/191: mknod d7/d21/c42 0 2024-04-22T23:35:02.730 INFO:tasks.workunit.client.0.smithi107.stdout:4/223: rmdir d1/d8/d23/d27 39 2024-04-22T23:35:02.731 INFO:tasks.workunit.client.0.smithi107.stdout:3/287: write db/ff [4768280,86662] 0 2024-04-22T23:35:02.731 INFO:tasks.workunit.client.0.smithi107.stdout:3/288: read - db/d21/d27/d15/d29/d2a/d40/f5e zero size 2024-04-22T23:35:02.731 INFO:tasks.workunit.client.0.smithi107.stdout:3/289: fsync db/d21/d27/d15/d29/d2a/f5b 0 2024-04-22T23:35:02.732 INFO:tasks.workunit.client.0.smithi107.stdout:7/192: unlink d7/f1a 0 2024-04-22T23:35:02.732 INFO:tasks.workunit.client.0.smithi107.stdout:7/193: chown d7/d1b/f24 195 1 2024-04-22T23:35:02.732 INFO:tasks.workunit.client.0.smithi107.stdout:7/194: creat d7/d2d/f43 x:0 0 0 2024-04-22T23:35:02.732 INFO:tasks.workunit.client.0.smithi107.stdout:7/195: chown d7/c3a 43202 1 2024-04-22T23:35:02.732 INFO:tasks.workunit.client.0.smithi107.stdout:7/196: chown d7/d21 3785 1 2024-04-22T23:35:02.733 INFO:tasks.workunit.client.0.smithi107.stdout:0/282: truncate d1/d27/d20/f6b 1948465 0 2024-04-22T23:35:02.733 INFO:tasks.workunit.client.0.smithi107.stdout:4/224: creat d1/d21/d29/d2e/f4f x:0 0 0 2024-04-22T23:35:02.735 INFO:tasks.workunit.client.0.smithi107.stdout:3/290: mknod db/d21/d27/d15/d29/d2a/c65 0 2024-04-22T23:35:02.738 INFO:tasks.workunit.client.0.smithi107.stdout:0/283: rename d1/d9/fc to d1/d27/d20/d2c/d41/d52/f6c 0 2024-04-22T23:35:02.738 INFO:tasks.workunit.client.0.smithi107.stdout:4/225: mknod d1/d21/d29/c50 0 2024-04-22T23:35:02.739 INFO:tasks.workunit.client.0.smithi107.stdout:4/226: chown d1/d7/f11 401 1 2024-04-22T23:35:02.739 INFO:tasks.workunit.client.0.smithi107.stdout:4/227: write d1/d7/fc [2230133,55623] 0 2024-04-22T23:35:02.739 INFO:tasks.workunit.client.0.smithi107.stdout:4/228: chown d1/d1b/c1d 581528660 1 2024-04-22T23:35:02.739 INFO:tasks.workunit.client.0.smithi107.stdout:8/166: dwrite d0/d7/f31 [0,4194304] 0 2024-04-22T23:35:02.739 INFO:tasks.workunit.client.0.smithi107.stdout:8/167: write d0/f4 [5809153,46313] 0 2024-04-22T23:35:02.740 INFO:tasks.workunit.client.0.smithi107.stdout:8/168: write d0/d7/fa [4573480,12034] 0 2024-04-22T23:35:02.740 INFO:tasks.workunit.client.0.smithi107.stdout:7/197: dread d7/fd [0,4194304] 0 2024-04-22T23:35:02.748 INFO:tasks.workunit.client.0.smithi107.stdout:8/169: mkdir d0/d7/df/d33/d37 0 2024-04-22T23:35:02.750 INFO:tasks.workunit.client.0.smithi107.stdout:7/198: symlink d7/l44 0 2024-04-22T23:35:02.751 INFO:tasks.workunit.client.0.smithi107.stdout:7/199: truncate d7/d21/f32 643240 0 2024-04-22T23:35:02.751 INFO:tasks.workunit.client.0.smithi107.stdout:0/284: dread d1/d27/d20/d2c/d2e/f5a [0,4194304] 0 2024-04-22T23:35:02.752 INFO:tasks.workunit.client.0.smithi107.stdout:0/285: mknod d1/d67/c6d 0 2024-04-22T23:35:02.755 INFO:tasks.workunit.client.0.smithi107.stdout:4/229: dwrite d1/d21/f36 [0,4194304] 0 2024-04-22T23:35:02.759 INFO:tasks.workunit.client.0.smithi107.stdout:4/230: mknod d1/d8/c51 0 2024-04-22T23:35:02.761 INFO:tasks.workunit.client.0.smithi107.stdout:3/291: dwrite db/d21/d27/d54/f4e [0,4194304] 0 2024-04-22T23:35:02.761 INFO:tasks.workunit.client.0.smithi107.stdout:8/170: dread d0/d6/f10 [0,4194304] 0 2024-04-22T23:35:02.761 INFO:tasks.workunit.client.0.smithi107.stdout:4/231: mkdir d1/d21/d52 0 2024-04-22T23:35:02.761 INFO:tasks.workunit.client.0.smithi107.stdout:3/292: write db/ff [981419,68663] 0 2024-04-22T23:35:02.766 INFO:tasks.workunit.client.0.smithi107.stdout:3/293: mkdir db/d21/d27/d15/d24/d66 0 2024-04-22T23:35:02.771 INFO:tasks.workunit.client.1.smithi157.stdout:4/585: sync 2024-04-22T23:35:02.771 INFO:tasks.workunit.client.0.smithi107.stdout:4/232: dread d1/d21/d29/d2e/d3b/f4c [0,4194304] 0 2024-04-22T23:35:02.772 INFO:tasks.workunit.client.1.smithi157.stdout:4/586: mknod d3/d11/d34/da4/db1/cc4 0 2024-04-22T23:35:02.773 INFO:tasks.workunit.client.0.smithi107.stdout:4/233: mkdir d1/d7/d53 0 2024-04-22T23:35:02.773 INFO:tasks.workunit.client.0.smithi107.stdout:4/234: mknod d1/d21/d29/d2e/c54 0 2024-04-22T23:35:02.773 INFO:tasks.workunit.client.0.smithi107.stdout:4/235: fsync d1/d8/d23/d27/f2f 0 2024-04-22T23:35:02.774 INFO:tasks.workunit.client.1.smithi157.stdout:4/587: creat d3/d11/d1d/d73/da7/fc5 x:0 0 0 2024-04-22T23:35:02.775 INFO:tasks.workunit.client.0.smithi107.stdout:7/200: dwrite d7/f18 [0,4194304] 0 2024-04-22T23:35:02.775 INFO:tasks.workunit.client.0.smithi107.stdout:7/201: fsync d7/d1b/f28 0 2024-04-22T23:35:02.775 INFO:tasks.workunit.client.0.smithi107.stdout:4/236: link d1/l4 d1/d1b/d44/l55 0 2024-04-22T23:35:02.775 INFO:tasks.workunit.client.1.smithi157.stdout:4/588: truncate d3/d11/d34/f48 2083163 0 2024-04-22T23:35:02.776 INFO:tasks.workunit.client.0.smithi107.stdout:0/286: dwrite d1/d27/f57 [0,4194304] 0 2024-04-22T23:35:02.779 INFO:tasks.workunit.client.0.smithi107.stdout:0/287: mkdir d1/d9/d6e 0 2024-04-22T23:35:02.780 INFO:tasks.workunit.client.0.smithi107.stdout:7/202: getdents d7/d21 0 2024-04-22T23:35:02.781 INFO:tasks.workunit.client.1.smithi157.stdout:4/589: dread d3/d11/d1d/d73/da3/d31/f6f [0,4194304] 0 2024-04-22T23:35:02.782 INFO:tasks.workunit.client.0.smithi107.stdout:7/203: mknod d7/c45 0 2024-04-22T23:35:02.783 INFO:tasks.workunit.client.1.smithi157.stdout:4/590: dread d3/d96/d40/f72 [0,4194304] 0 2024-04-22T23:35:02.785 INFO:tasks.workunit.client.0.smithi107.stdout:7/204: symlink d7/d2d/l46 0 2024-04-22T23:35:02.786 INFO:tasks.workunit.client.0.smithi107.stdout:7/205: chown d7/c17 5 1 2024-04-22T23:35:02.786 INFO:tasks.workunit.client.0.smithi107.stdout:3/294: dwrite db/d21/d27/d15/d29/d2a/d40/f5e [0,4194304] 0 2024-04-22T23:35:02.789 INFO:tasks.workunit.client.0.smithi107.stdout:7/206: rename d7/c3a to d7/d2d/c47 0 2024-04-22T23:35:02.793 INFO:tasks.workunit.client.0.smithi107.stdout:7/207: mkdir d7/d21/d48 0 2024-04-22T23:35:02.793 INFO:tasks.workunit.client.0.smithi107.stdout:6/234: sync 2024-04-22T23:35:02.793 INFO:tasks.workunit.client.0.smithi107.stdout:6/235: truncate db/f3f 600722 0 2024-04-22T23:35:02.793 INFO:tasks.workunit.client.0.smithi107.stdout:4/237: dwrite d1/d21/d29/d2e/d3b/f4c [4194304,4194304] 0 2024-04-22T23:35:02.796 INFO:tasks.workunit.client.0.smithi107.stdout:4/238: getdents d1 0 2024-04-22T23:35:02.796 INFO:tasks.workunit.client.0.smithi107.stdout:7/208: dread d7/d1b/f28 [0,4194304] 0 2024-04-22T23:35:02.797 INFO:tasks.workunit.client.0.smithi107.stdout:7/209: write d7/d2d/f43 [744005,7907] 0 2024-04-22T23:35:02.797 INFO:tasks.workunit.client.0.smithi107.stdout:4/239: rename d1/d8/d23 to d1/d8/d56 0 2024-04-22T23:35:02.799 INFO:tasks.workunit.client.0.smithi107.stdout:7/210: mkdir d7/d21/d49 0 2024-04-22T23:35:02.800 INFO:tasks.workunit.client.0.smithi107.stdout:7/211: mknod d7/d21/d49/c4a 0 2024-04-22T23:35:02.802 INFO:tasks.workunit.client.0.smithi107.stdout:7/212: rename d7/l2c to d7/l4b 0 2024-04-22T23:35:02.803 INFO:tasks.workunit.client.0.smithi107.stdout:6/236: write db/d20/d2c/f32 [3973809,81493] 0 2024-04-22T23:35:02.804 INFO:tasks.workunit.client.0.smithi107.stdout:7/213: dread d7/d1b/f28 [0,4194304] 0 2024-04-22T23:35:02.805 INFO:tasks.workunit.client.1.smithi157.stdout:4/591: dwrite d3/d4b/d7a/db7/f65 [0,4194304] 0 2024-04-22T23:35:02.806 INFO:tasks.workunit.client.0.smithi107.stdout:6/237: mkdir db/d20/d2c/d3b/d40 0 2024-04-22T23:35:02.811 INFO:tasks.workunit.client.0.smithi107.stdout:7/214: dread d7/f11 [0,4194304] 0 2024-04-22T23:35:02.813 INFO:tasks.workunit.client.0.smithi107.stdout:7/215: mkdir d7/d4c 0 2024-04-22T23:35:02.814 INFO:tasks.workunit.client.0.smithi107.stdout:4/240: dwrite d1/d7/f11 [0,4194304] 0 2024-04-22T23:35:02.815 INFO:tasks.workunit.client.1.smithi157.stdout:4/592: getdents d3/d11/d1d/d73/da7 0 2024-04-22T23:35:02.816 INFO:tasks.workunit.client.0.smithi107.stdout:4/241: mknod d1/d7/c57 0 2024-04-22T23:35:02.817 INFO:tasks.workunit.client.1.smithi157.stdout:4/593: creat d3/d11/d34/da6/fc6 x:0 0 0 2024-04-22T23:35:02.818 INFO:tasks.workunit.client.0.smithi107.stdout:4/242: creat d1/d1b/d44/f58 x:0 0 0 2024-04-22T23:35:02.818 INFO:tasks.workunit.client.0.smithi107.stdout:6/238: dread db/f29 [4194304,4194304] 0 2024-04-22T23:35:02.820 INFO:tasks.workunit.client.0.smithi107.stdout:6/239: rmdir db/d37 0 2024-04-22T23:35:02.820 INFO:tasks.workunit.client.0.smithi107.stdout:4/243: getdents d1/d21/d29/d2e/d4d 0 2024-04-22T23:35:02.821 INFO:tasks.workunit.client.1.smithi157.stdout:4/594: rmdir d3/d96/d81/db3 0 2024-04-22T23:35:02.823 INFO:tasks.workunit.client.1.smithi157.stdout:4/595: link d3/d11/d34/da4/db1/cc4 d3/d11/d34/da4/cc7 0 2024-04-22T23:35:02.825 INFO:tasks.workunit.client.1.smithi157.stdout:4/596: mkdir d3/d11/d1d/d73/d80/dac/db5/dc8 0 2024-04-22T23:35:02.825 INFO:tasks.workunit.client.1.smithi157.stdout:4/597: chown d3/d11/d34/c7e 12148399 1 2024-04-22T23:35:02.826 INFO:tasks.workunit.client.1.smithi157.stdout:4/598: stat d3/d4b/d5d 0 2024-04-22T23:35:02.827 INFO:tasks.workunit.client.1.smithi157.stdout:4/599: truncate d3/d4b/f4f 1555621 0 2024-04-22T23:35:02.827 INFO:tasks.workunit.client.0.smithi107.stdout:7/216: dwrite d7/d21/f32 [0,4194304] 0 2024-04-22T23:35:02.827 INFO:tasks.workunit.client.0.smithi107.stdout:7/217: read - d7/d2d/f41 zero size 2024-04-22T23:35:02.832 INFO:tasks.workunit.client.0.smithi107.stdout:7/218: getdents d7/d21/d49 0 2024-04-22T23:35:02.834 INFO:tasks.workunit.client.0.smithi107.stdout:7/219: dread d7/f2f [0,4194304] 0 2024-04-22T23:35:02.836 INFO:tasks.workunit.client.0.smithi107.stdout:9/294: sync 2024-04-22T23:35:02.836 INFO:tasks.workunit.client.0.smithi107.stdout:9/295: stat f4 0 2024-04-22T23:35:02.838 INFO:tasks.workunit.client.0.smithi107.stdout:9/296: mknod d5/d7/d27/c65 0 2024-04-22T23:35:02.841 INFO:tasks.workunit.client.0.smithi107.stdout:4/244: dwrite d1/f28 [0,4194304] 0 2024-04-22T23:35:02.841 INFO:tasks.workunit.client.0.smithi107.stdout:4/245: write d1/d21/d29/f45 [848802,81375] 0 2024-04-22T23:35:02.844 INFO:tasks.workunit.client.0.smithi107.stdout:4/246: getdents d1/d21 0 2024-04-22T23:35:02.848 INFO:tasks.workunit.client.0.smithi107.stdout:4/247: symlink d1/d1b/l59 0 2024-04-22T23:35:02.849 INFO:tasks.workunit.client.0.smithi107.stdout:4/248: dread - d1/d7/f33 zero size 2024-04-22T23:35:02.849 INFO:tasks.workunit.client.0.smithi107.stdout:4/249: write d1/d21/d29/f45 [1170242,35506] 0 2024-04-22T23:35:02.849 INFO:tasks.workunit.client.0.smithi107.stdout:4/250: write d1/d8/f3f [730633,59853] 0 2024-04-22T23:35:02.850 INFO:tasks.workunit.client.0.smithi107.stdout:4/251: mknod d1/d7/d53/c5a 0 2024-04-22T23:35:02.851 INFO:tasks.workunit.client.0.smithi107.stdout:4/252: truncate d1/d21/d29/d2e/f46 3863799 0 2024-04-22T23:35:02.852 INFO:tasks.workunit.client.0.smithi107.stdout:9/297: read d5/d7/fb [3257704,70713] 0 2024-04-22T23:35:02.852 INFO:tasks.workunit.client.0.smithi107.stdout:6/240: dwrite f0 [0,4194304] 0 2024-04-22T23:35:02.854 INFO:tasks.workunit.client.1.smithi157.stdout:4/600: dwrite d3/d11/d1d/d73/d80/dac/db5/f94 [0,4194304] 0 2024-04-22T23:35:02.855 INFO:tasks.workunit.client.0.smithi107.stdout:9/298: creat d5/d7/d27/d46/f66 x:0 0 0 2024-04-22T23:35:02.856 INFO:tasks.workunit.client.0.smithi107.stdout:7/220: dwrite d7/f26 [0,4194304] 0 2024-04-22T23:35:02.856 INFO:tasks.workunit.client.0.smithi107.stdout:7/221: chown d7/d2d/c47 11 1 2024-04-22T23:35:02.857 INFO:tasks.workunit.client.0.smithi107.stdout:6/241: mknod db/d20/d2a/c41 0 2024-04-22T23:35:02.857 INFO:tasks.workunit.client.1.smithi157.stdout:4/601: creat d3/d11/d34/d9d/fc9 x:0 0 0 2024-04-22T23:35:02.858 INFO:tasks.workunit.client.0.smithi107.stdout:9/299: mknod d5/d7/d27/d46/d4b/c67 0 2024-04-22T23:35:02.858 INFO:tasks.workunit.client.1.smithi157.stdout:4/602: creat d3/d5c/fca x:0 0 0 2024-04-22T23:35:02.859 INFO:tasks.workunit.client.1.smithi157.stdout:4/603: write d3/d11/d30/f57 [1099185,5574] 0 2024-04-22T23:35:02.862 INFO:tasks.workunit.client.0.smithi107.stdout:7/222: truncate d7/fd 2400193 0 2024-04-22T23:35:02.864 INFO:tasks.workunit.client.0.smithi107.stdout:9/300: creat d5/d14/d2e/d43/f68 x:0 0 0 2024-04-22T23:35:02.866 INFO:tasks.workunit.client.0.smithi107.stdout:9/301: truncate d5/d29/f3a 2253399 0 2024-04-22T23:35:02.870 INFO:tasks.workunit.client.0.smithi107.stdout:9/302: mkdir d5/d7/d27/d69 0 2024-04-22T23:35:02.870 INFO:tasks.workunit.client.0.smithi107.stdout:9/303: creat d5/d24/f6a x:0 0 0 2024-04-22T23:35:02.873 INFO:tasks.workunit.client.0.smithi107.stdout:9/304: truncate d5/f11 509385 0 2024-04-22T23:35:02.873 INFO:tasks.workunit.client.0.smithi107.stdout:9/305: write d5/d14/d2e/d43/f68 [512990,69194] 0 2024-04-22T23:35:02.873 INFO:tasks.workunit.client.0.smithi107.stdout:9/306: chown d5/l6 938 1 2024-04-22T23:35:02.877 INFO:tasks.workunit.client.0.smithi107.stdout:9/307: rmdir d5/d7/d27/d46 39 2024-04-22T23:35:02.878 INFO:tasks.workunit.client.0.smithi107.stdout:9/308: symlink d5/d7/d17/l6b 0 2024-04-22T23:35:02.881 INFO:tasks.workunit.client.0.smithi107.stdout:9/309: truncate d5/f11 1088827 0 2024-04-22T23:35:02.882 INFO:tasks.workunit.client.0.smithi107.stdout:9/310: mkdir d5/d6c 0 2024-04-22T23:35:02.882 INFO:tasks.workunit.client.0.smithi107.stdout:9/311: stat d5/d7/d17/d1b/d1f/d36/d53/d55/l62 0 2024-04-22T23:35:02.883 INFO:tasks.workunit.client.0.smithi107.stdout:9/312: creat d5/d29/f6d x:0 0 0 2024-04-22T23:35:02.884 INFO:tasks.workunit.client.0.smithi107.stdout:9/313: mknod d5/d7/d27/c6e 0 2024-04-22T23:35:02.885 INFO:tasks.workunit.client.0.smithi107.stdout:9/314: truncate f1 1161516 0 2024-04-22T23:35:02.885 INFO:tasks.workunit.client.0.smithi107.stdout:9/315: fsync d5/d7/d17/d1b/f30 0 2024-04-22T23:35:02.885 INFO:tasks.workunit.client.0.smithi107.stdout:9/316: stat d5/d7/d27/l42 0 2024-04-22T23:35:02.886 INFO:tasks.workunit.client.0.smithi107.stdout:9/317: mkdir d5/d7/d27/d6f 0 2024-04-22T23:35:02.886 INFO:tasks.workunit.client.0.smithi107.stdout:9/318: write d5/d7/f16 [5170355,129802] 0 2024-04-22T23:35:02.887 INFO:tasks.workunit.client.0.smithi107.stdout:9/319: creat d5/d7/f70 x:0 0 0 2024-04-22T23:35:02.888 INFO:tasks.workunit.client.0.smithi107.stdout:9/320: write d5/f10 [4690954,114306] 0 2024-04-22T23:35:02.896 INFO:tasks.workunit.client.0.smithi107.stdout:9/321: write d5/fc [199301,111700] 0 2024-04-22T23:35:02.898 INFO:tasks.workunit.client.0.smithi107.stdout:9/322: mknod d5/d7/d17/d1b/d1f/d36/c71 0 2024-04-22T23:35:02.898 INFO:tasks.workunit.client.0.smithi107.stdout:9/323: fsync f4 0 2024-04-22T23:35:02.898 INFO:tasks.workunit.client.0.smithi107.stdout:9/324: write d5/d14/f2f [134955,98138] 0 2024-04-22T23:35:02.901 INFO:tasks.workunit.client.0.smithi107.stdout:4/253: dwrite d1/d7/f32 [0,4194304] 0 2024-04-22T23:35:02.903 INFO:tasks.workunit.client.0.smithi107.stdout:4/254: mkdir d1/d7/d5b 0 2024-04-22T23:35:02.906 INFO:tasks.workunit.client.1.smithi157.stdout:4/604: read d3/d5c/d7b/fc2 [2349311,86712] 0 2024-04-22T23:35:02.906 INFO:tasks.workunit.client.1.smithi157.stdout:4/605: read - d3/d5c/f95 zero size 2024-04-22T23:35:02.909 INFO:tasks.workunit.client.1.smithi157.stdout:1/527: sync 2024-04-22T23:35:02.909 INFO:tasks.workunit.client.1.smithi157.stdout:1/528: truncate d4/db/fe 4692023 0 2024-04-22T23:35:02.909 INFO:tasks.workunit.client.1.smithi157.stdout:7/366: sync 2024-04-22T23:35:02.909 INFO:tasks.workunit.client.1.smithi157.stdout:5/478: sync 2024-04-22T23:35:02.911 INFO:tasks.workunit.client.1.smithi157.stdout:2/599: sync 2024-04-22T23:35:02.911 INFO:tasks.workunit.client.1.smithi157.stdout:8/314: sync 2024-04-22T23:35:02.911 INFO:tasks.workunit.client.1.smithi157.stdout:2/600: dread - d7/d1f/d26/d4a/fc9 zero size 2024-04-22T23:35:02.911 INFO:tasks.workunit.client.1.smithi157.stdout:3/446: sync 2024-04-22T23:35:02.911 INFO:tasks.workunit.client.1.smithi157.stdout:6/688: sync 2024-04-22T23:35:02.912 INFO:tasks.workunit.client.1.smithi157.stdout:9/445: sync 2024-04-22T23:35:02.912 INFO:tasks.workunit.client.1.smithi157.stdout:0/397: sync 2024-04-22T23:35:02.914 INFO:tasks.workunit.client.0.smithi107.stdout:4/255: dread d1/d1b/f49 [0,4194304] 0 2024-04-22T23:35:02.916 INFO:tasks.workunit.client.1.smithi157.stdout:3/447: mknod d0/c8b 0 2024-04-22T23:35:02.918 INFO:tasks.workunit.client.0.smithi107.stdout:4/256: mknod d1/d7/d5b/c5c 0 2024-04-22T23:35:02.919 INFO:tasks.workunit.client.1.smithi157.stdout:9/446: symlink d4/d15/l86 0 2024-04-22T23:35:02.920 INFO:tasks.workunit.client.1.smithi157.stdout:0/398: truncate f11 3234103 0 2024-04-22T23:35:02.920 INFO:tasks.workunit.client.0.smithi107.stdout:6/242: dwrite db/f11 [0,4194304] 0 2024-04-22T23:35:02.920 INFO:tasks.workunit.client.0.smithi107.stdout:6/243: fsync db/f12 0 2024-04-22T23:35:02.920 INFO:tasks.workunit.client.0.smithi107.stdout:6/244: creat db/d20/d2a/f42 x:0 0 0 2024-04-22T23:35:02.921 INFO:tasks.workunit.client.0.smithi107.stdout:6/245: dread - db/d20/d2a/f2e zero size 2024-04-22T23:35:02.921 INFO:tasks.workunit.client.0.smithi107.stdout:9/325: dwrite d5/d7/fd [0,4194304] 0 2024-04-22T23:35:02.921 INFO:tasks.workunit.client.1.smithi157.stdout:3/448: link d0/d3/d34/d3d/d5b/d72/f7f d0/d3/d13/d21/f8c 0 2024-04-22T23:35:02.923 INFO:tasks.workunit.client.0.smithi107.stdout:4/257: symlink d1/d8/d56/d27/l5d 0 2024-04-22T23:35:02.923 INFO:tasks.workunit.client.1.smithi157.stdout:6/689: dread d1/d2f/d3b/f2a [0,4194304] 0 2024-04-22T23:35:02.923 INFO:tasks.workunit.client.1.smithi157.stdout:9/447: rename d4/d15/d55/f58 to d4/d15/d23/d27/f87 0 2024-04-22T23:35:02.926 INFO:tasks.workunit.client.1.smithi157.stdout:7/367: write d0/f4 [723036,121127] 0 2024-04-22T23:35:02.926 INFO:tasks.workunit.client.1.smithi157.stdout:0/399: creat d17/d23/d34/d67/d7f/d69/f89 x:0 0 0 2024-04-22T23:35:02.926 INFO:tasks.workunit.client.1.smithi157.stdout:0/400: chown d17/d2b/d4f/l75 1261 1 2024-04-22T23:35:02.927 INFO:tasks.workunit.client.1.smithi157.stdout:3/449: mkdir d0/d3/d8d 0 2024-04-22T23:35:02.927 INFO:tasks.workunit.client.1.smithi157.stdout:3/450: chown d0/d3/d34/d3d/d57/d60/d6b/f85 37913336 1 2024-04-22T23:35:02.930 INFO:tasks.workunit.client.1.smithi157.stdout:6/690: link d1/d38/fba d1/d2f/d3b/d6c/fd6 0 2024-04-22T23:35:02.931 INFO:tasks.workunit.client.0.smithi107.stdout:6/246: creat db/d20/d2c/d3b/f43 x:0 0 0 2024-04-22T23:35:02.931 INFO:tasks.workunit.client.0.smithi107.stdout:6/247: creat db/d20/d2c/f44 x:0 0 0 2024-04-22T23:35:02.931 INFO:tasks.workunit.client.0.smithi107.stdout:6/248: creat db/d20/d2c/d3b/f45 x:0 0 0 2024-04-22T23:35:02.931 INFO:tasks.workunit.client.0.smithi107.stdout:6/249: chown f0 2847 1 2024-04-22T23:35:02.932 INFO:tasks.workunit.client.1.smithi157.stdout:9/448: creat d4/d15/d23/f88 x:0 0 0 2024-04-22T23:35:02.932 INFO:tasks.workunit.client.1.smithi157.stdout:9/449: fsync d4/d2d/f78 0 2024-04-22T23:35:02.932 INFO:tasks.workunit.client.1.smithi157.stdout:9/450: stat d4/d15/d55/l5b 0 2024-04-22T23:35:02.933 INFO:tasks.workunit.client.1.smithi157.stdout:9/451: dread d4/d15/d23/d27/f53 [0,4194304] 0 2024-04-22T23:35:02.934 INFO:tasks.workunit.client.1.smithi157.stdout:7/368: mkdir d0/d9/d29/d33/d7c 0 2024-04-22T23:35:02.934 INFO:tasks.workunit.client.0.smithi107.stdout:6/250: symlink db/d20/d2a/d35/l46 0 2024-04-22T23:35:02.935 INFO:tasks.workunit.client.1.smithi157.stdout:0/401: symlink d17/d2b/d4f/d5a/d7c/l8a 0 2024-04-22T23:35:02.935 INFO:tasks.workunit.client.1.smithi157.stdout:0/402: fdatasync d17/d38/f73 0 2024-04-22T23:35:02.936 INFO:tasks.workunit.client.0.smithi107.stdout:6/251: symlink db/d20/d2c/d3b/l47 0 2024-04-22T23:35:02.937 INFO:tasks.workunit.client.1.smithi157.stdout:3/451: mknod d0/d3/d34/d39/c8e 0 2024-04-22T23:35:02.937 INFO:tasks.workunit.client.1.smithi157.stdout:3/452: write d0/d3/d34/d47/f70 [94018,38522] 0 2024-04-22T23:35:02.937 INFO:tasks.workunit.client.1.smithi157.stdout:3/453: fsync d0/d3/d34/d3d/d5b/f84 0 2024-04-22T23:35:02.938 INFO:tasks.workunit.client.1.smithi157.stdout:6/691: mkdir d1/d2f/d3b/d6c/d83/d93/dd7 0 2024-04-22T23:35:02.939 INFO:tasks.workunit.client.1.smithi157.stdout:6/692: creat d1/d2f/d3b/d6c/d83/fd8 x:0 0 0 2024-04-22T23:35:02.939 INFO:tasks.workunit.client.1.smithi157.stdout:7/369: readlink d0/d9/d10/l3f 0 2024-04-22T23:35:02.940 INFO:tasks.workunit.client.1.smithi157.stdout:0/403: mknod d17/d23/d34/d67/d78/c8b 0 2024-04-22T23:35:02.940 INFO:tasks.workunit.client.1.smithi157.stdout:3/454: rmdir d0/d3/d13 39 2024-04-22T23:35:02.943 INFO:tasks.workunit.client.1.smithi157.stdout:6/693: symlink d1/d2f/dc7/dce/ld9 0 2024-04-22T23:35:02.943 INFO:tasks.workunit.client.1.smithi157.stdout:6/694: read - d1/d2f/d3b/d6c/d83/fd8 zero size 2024-04-22T23:35:02.943 INFO:tasks.workunit.client.1.smithi157.stdout:6/695: write d1/d36/f57 [1962657,59198] 0 2024-04-22T23:35:02.943 INFO:tasks.workunit.client.1.smithi157.stdout:6/696: readlink d1/d2f/d3b/l15 0 2024-04-22T23:35:02.944 INFO:tasks.workunit.client.1.smithi157.stdout:7/370: creat d0/d9/f7d x:0 0 0 2024-04-22T23:35:02.944 INFO:tasks.workunit.client.1.smithi157.stdout:6/697: write d1/d2f/d54/f6b [122586,121483] 0 2024-04-22T23:35:02.945 INFO:tasks.workunit.client.1.smithi157.stdout:7/371: write d0/d9/de/f5f [976116,29987] 0 2024-04-22T23:35:02.945 INFO:tasks.workunit.client.1.smithi157.stdout:3/455: mknod d0/d3/d13/d21/c8f 0 2024-04-22T23:35:02.947 INFO:tasks.workunit.client.1.smithi157.stdout:3/456: rmdir d0 39 2024-04-22T23:35:02.947 INFO:tasks.workunit.client.1.smithi157.stdout:3/457: fsync d0/d3/d34/d39/f49 0 2024-04-22T23:35:02.947 INFO:tasks.workunit.client.1.smithi157.stdout:3/458: write d0/d3/d34/d3d/d57/f65 [495249,89556] 0 2024-04-22T23:35:02.948 INFO:tasks.workunit.client.1.smithi157.stdout:7/372: getdents d0/d9/db/d35 0 2024-04-22T23:35:02.948 INFO:tasks.workunit.client.1.smithi157.stdout:7/373: write d0/d9/d29/d5e/f6a [830227,24400] 0 2024-04-22T23:35:02.950 INFO:tasks.workunit.client.1.smithi157.stdout:7/374: dread d0/d9/db/f57 [0,4194304] 0 2024-04-22T23:35:02.951 INFO:tasks.workunit.client.1.smithi157.stdout:7/375: mknod d0/d9/d29/d33/d37/c7e 0 2024-04-22T23:35:02.952 INFO:tasks.workunit.client.1.smithi157.stdout:9/452: dwrite d4/df/f6a [0,4194304] 0 2024-04-22T23:35:02.956 INFO:tasks.workunit.client.1.smithi157.stdout:6/698: write d1/d38/faa [2641542,49521] 0 2024-04-22T23:35:02.956 INFO:tasks.workunit.client.1.smithi157.stdout:6/699: write d1/d2f/dc7/dce/fd2 [818137,119723] 0 2024-04-22T23:35:02.957 INFO:tasks.workunit.client.1.smithi157.stdout:8/315: dwrite d2/d6/d21/d5f/f6f [0,4194304] 0 2024-04-22T23:35:02.957 INFO:tasks.workunit.client.1.smithi157.stdout:1/529: dwrite d4/db/d3d/f82 [4194304,4194304] 0 2024-04-22T23:35:02.957 INFO:tasks.workunit.client.1.smithi157.stdout:1/530: truncate d4/d17/d1d/d33/fa2 6268 0 2024-04-22T23:35:02.958 INFO:tasks.workunit.client.1.smithi157.stdout:6/700: mkdir d1/d2f/d54/d9b/dda 0 2024-04-22T23:35:02.960 INFO:tasks.workunit.client.0.smithi107.stdout:6/252: truncate f0 2069152 0 2024-04-22T23:35:02.960 INFO:tasks.workunit.client.0.smithi107.stdout:6/253: chown db/d20/d2a/f42 191322305 1 2024-04-22T23:35:02.961 INFO:tasks.workunit.client.0.smithi107.stdout:6/254: write db/f1e [1244473,20346] 0 2024-04-22T23:35:02.961 INFO:tasks.workunit.client.1.smithi157.stdout:9/453: dread d4/d15/d23/d27/d47/f4a [0,4194304] 0 2024-04-22T23:35:02.962 INFO:tasks.workunit.client.1.smithi157.stdout:6/701: truncate d1/f34 255969 0 2024-04-22T23:35:02.964 INFO:tasks.workunit.client.0.smithi107.stdout:8/171: sync 2024-04-22T23:35:02.964 INFO:tasks.workunit.client.0.smithi107.stdout:1/239: sync 2024-04-22T23:35:02.964 INFO:tasks.workunit.client.0.smithi107.stdout:1/240: write d7/d19/d20/d39/d53/f29 [721520,94185] 0 2024-04-22T23:35:02.964 INFO:tasks.workunit.client.0.smithi107.stdout:1/241: chown d7/d19/d20/d39/d53/f33 54419 1 2024-04-22T23:35:02.964 INFO:tasks.workunit.client.0.smithi107.stdout:1/242: chown d7/d19/d42 38 1 2024-04-22T23:35:02.964 INFO:tasks.workunit.client.0.smithi107.stdout:5/241: sync 2024-04-22T23:35:02.964 INFO:tasks.workunit.client.0.smithi107.stdout:2/294: sync 2024-04-22T23:35:02.966 INFO:tasks.workunit.client.1.smithi157.stdout:6/702: link d1/d2f/d3b/d6c/d83/f71 d1/d2f/d3b/d39/d4e/da8/dac/db0/fdb 0 2024-04-22T23:35:02.966 INFO:tasks.workunit.client.0.smithi107.stdout:6/255: dread db/f1c [0,4194304] 0 2024-04-22T23:35:02.966 INFO:tasks.workunit.client.0.smithi107.stdout:6/256: dread - db/d20/d2a/f42 zero size 2024-04-22T23:35:02.966 INFO:tasks.workunit.client.0.smithi107.stdout:6/257: creat db/f48 x:0 0 0 2024-04-22T23:35:02.967 INFO:tasks.workunit.client.1.smithi157.stdout:0/404: dwrite d17/d23/d34/d67/d7f/f88 [0,4194304] 0 2024-04-22T23:35:02.967 INFO:tasks.workunit.client.1.smithi157.stdout:7/376: fsync d0/d9/d29/d5e/f6a 0 2024-04-22T23:35:02.967 INFO:tasks.workunit.client.0.smithi107.stdout:4/258: dwrite d1/d1b/f49 [0,4194304] 0 2024-04-22T23:35:02.967 INFO:tasks.workunit.client.0.smithi107.stdout:9/326: dwrite d5/d14/d28/f59 [0,4194304] 0 2024-04-22T23:35:02.968 INFO:tasks.workunit.client.0.smithi107.stdout:4/259: stat d1/d7/l18 0 2024-04-22T23:35:02.968 INFO:tasks.workunit.client.0.smithi107.stdout:4/260: chown d1/d21/d29/d2e/d4d/f4e 109 1 2024-04-22T23:35:02.968 INFO:tasks.workunit.client.0.smithi107.stdout:5/242: mknod d9/db/c4d 0 2024-04-22T23:35:02.969 INFO:tasks.workunit.client.0.smithi107.stdout:2/295: link d1/c1d d1/d12/c6d 0 2024-04-22T23:35:02.969 INFO:tasks.workunit.client.1.smithi157.stdout:9/454: read d4/d15/d23/d27/f33 [764610,109791] 0 2024-04-22T23:35:02.969 INFO:tasks.workunit.client.1.smithi157.stdout:5/479: dwrite d1/d5/d7/d2f/d45/da4/fac [0,4194304] 0 2024-04-22T23:35:02.976 INFO:tasks.workunit.client.1.smithi157.stdout:7/377: dread d0/f3e [0,4194304] 0 2024-04-22T23:35:02.977 INFO:tasks.workunit.client.1.smithi157.stdout:8/316: dread d2/d6/f46 [0,4194304] 0 2024-04-22T23:35:02.977 INFO:tasks.workunit.client.1.smithi157.stdout:8/317: write d2/f48 [906636,76697] 0 2024-04-22T23:35:02.980 INFO:tasks.workunit.client.0.smithi107.stdout:9/327: dread d5/d7/d17/d1b/f30 [0,4194304] 0 2024-04-22T23:35:02.981 INFO:tasks.workunit.client.1.smithi157.stdout:0/405: symlink d17/d2b/d52/d7e/l8c 0 2024-04-22T23:35:02.982 INFO:tasks.workunit.client.0.smithi107.stdout:5/243: link d9/db/f39 d9/f4e 0 2024-04-22T23:35:02.987 INFO:tasks.workunit.client.0.smithi107.stdout:2/296: rename d1/d12/f27 to d1/d2b/f6e 0 2024-04-22T23:35:02.987 INFO:tasks.workunit.client.0.smithi107.stdout:2/297: write d1/d63/f65 [447025,105180] 0 2024-04-22T23:35:02.989 INFO:tasks.workunit.client.0.smithi107.stdout:1/243: rename d7/d19/d32 to d7/d19/d54 0 2024-04-22T23:35:02.991 INFO:tasks.workunit.client.0.smithi107.stdout:9/328: link d5/d7/d17/d1b/d1f/l21 d5/d14/d2e/d43/l72 0 2024-04-22T23:35:02.992 INFO:tasks.workunit.client.1.smithi157.stdout:5/480: mknod d1/d1f/cbb 0 2024-04-22T23:35:02.993 INFO:tasks.workunit.client.0.smithi107.stdout:5/244: getdents d9/db 0 2024-04-22T23:35:02.993 INFO:tasks.workunit.client.0.smithi107.stdout:5/245: fdatasync d9/db/f13 0 2024-04-22T23:35:02.993 INFO:tasks.workunit.client.0.smithi107.stdout:8/172: dwrite d0/d7/f34 [0,4194304] 0 2024-04-22T23:35:02.994 INFO:tasks.workunit.client.0.smithi107.stdout:8/173: write d0/d7/df/f23 [1033229,35666] 0 2024-04-22T23:35:02.994 INFO:tasks.workunit.client.0.smithi107.stdout:8/174: read d0/d7/d20/d2d/f2c [252202,59939] 0 2024-04-22T23:35:02.995 INFO:tasks.workunit.client.1.smithi157.stdout:8/318: symlink d2/d6/d21/d5f/d17/l70 0 2024-04-22T23:35:02.995 INFO:tasks.workunit.client.1.smithi157.stdout:8/319: write d2/f48 [1438545,71337] 0 2024-04-22T23:35:02.995 INFO:tasks.workunit.client.1.smithi157.stdout:8/320: creat d2/f71 x:0 0 0 2024-04-22T23:35:02.995 INFO:tasks.workunit.client.1.smithi157.stdout:8/321: chown d2/d4/f52 1099665 1 2024-04-22T23:35:02.995 INFO:tasks.workunit.client.1.smithi157.stdout:8/322: write d2/d6/d21/d5f/d17/f4e [293209,10962] 0 2024-04-22T23:35:02.995 INFO:tasks.workunit.client.0.smithi107.stdout:1/244: symlink d7/d19/d2d/l55 0 2024-04-22T23:35:02.996 INFO:tasks.workunit.client.1.smithi157.stdout:0/406: creat d17/d23/d34/d67/f8d x:0 0 0 2024-04-22T23:35:02.996 INFO:tasks.workunit.client.1.smithi157.stdout:0/407: readlink d17/d2b/d52/l59 0 2024-04-22T23:35:02.997 INFO:tasks.workunit.client.1.smithi157.stdout:9/455: rmdir d4/d15/d55 39 2024-04-22T23:35:02.998 INFO:tasks.workunit.client.0.smithi107.stdout:5/246: mknod d9/db/d42/c4f 0 2024-04-22T23:35:03.000 INFO:tasks.workunit.client.1.smithi157.stdout:5/481: creat d1/d27/fbc x:0 0 0 2024-04-22T23:35:03.000 INFO:tasks.workunit.client.1.smithi157.stdout:5/482: read d1/d5/dc/f1b [4720065,115458] 0 2024-04-22T23:35:03.001 INFO:tasks.workunit.client.0.smithi107.stdout:8/175: rmdir d0/d7/df/d18 39 2024-04-22T23:35:03.001 INFO:tasks.workunit.client.0.smithi107.stdout:8/176: fdatasync d0/fb 0 2024-04-22T23:35:03.002 INFO:tasks.workunit.client.0.smithi107.stdout:9/329: getdents d5/d7/d17/d1b/d1f/d36 0 2024-04-22T23:35:03.003 INFO:tasks.workunit.client.0.smithi107.stdout:9/330: mkdir d5/d14/d2e/d4f/d73 0 2024-04-22T23:35:03.005 INFO:tasks.workunit.client.1.smithi157.stdout:8/323: rename d2/d6/d21/d5f/d17/l70 to d2/d6/d21/d5f/d11/d13/l72 0 2024-04-22T23:35:03.008 INFO:tasks.workunit.client.0.smithi107.stdout:8/177: write d0/d6/f10 [2719555,4450] 0 2024-04-22T23:35:03.010 INFO:tasks.workunit.client.1.smithi157.stdout:6/703: getdents d1/d2f/d3b/d39/d4e/da8/dac/db0 0 2024-04-22T23:35:03.010 INFO:tasks.workunit.client.1.smithi157.stdout:6/704: read d1/f7 [1434012,122166] 0 2024-04-22T23:35:03.011 INFO:tasks.workunit.client.1.smithi157.stdout:0/408: mknod d17/d38/d6a/c8e 0 2024-04-22T23:35:03.011 INFO:tasks.workunit.client.1.smithi157.stdout:0/409: read - d17/d23/d34/d67/d7f/f74 zero size 2024-04-22T23:35:03.012 INFO:tasks.workunit.client.0.smithi107.stdout:8/178: creat d0/d7/df/d33/d37/f38 x:0 0 0 2024-04-22T23:35:03.012 INFO:tasks.workunit.client.0.smithi107.stdout:8/179: stat d0/d7/fa 0 2024-04-22T23:35:03.012 INFO:tasks.workunit.client.1.smithi157.stdout:9/456: mknod d4/d15/d23/c89 0 2024-04-22T23:35:03.013 INFO:tasks.workunit.client.0.smithi107.stdout:6/258: dwrite db/f11 [0,4194304] 0 2024-04-22T23:35:03.014 INFO:tasks.workunit.client.0.smithi107.stdout:8/180: creat d0/d7/d20/f39 x:0 0 0 2024-04-22T23:35:03.015 INFO:tasks.workunit.client.1.smithi157.stdout:8/324: mkdir d2/d4/d73 0 2024-04-22T23:35:03.015 INFO:tasks.workunit.client.1.smithi157.stdout:8/325: fsync d2/d6/d21/d5f/fd 0 2024-04-22T23:35:03.015 INFO:tasks.workunit.client.0.smithi107.stdout:6/259: mkdir db/d20/d49 0 2024-04-22T23:35:03.015 INFO:tasks.workunit.client.0.smithi107.stdout:6/260: getdents db/d20/d49 0 2024-04-22T23:35:03.015 INFO:tasks.workunit.client.0.smithi107.stdout:6/261: fdatasync db/f1f 0 2024-04-22T23:35:03.015 INFO:tasks.workunit.client.0.smithi107.stdout:6/262: write db/d20/d2c/f44 [661346,91943] 0 2024-04-22T23:35:03.015 INFO:tasks.workunit.client.0.smithi107.stdout:5/247: dread d9/d26/f41 [0,4194304] 0 2024-04-22T23:35:03.016 INFO:tasks.workunit.client.0.smithi107.stdout:5/248: creat d9/db/f50 x:0 0 0 2024-04-22T23:35:03.016 INFO:tasks.workunit.client.0.smithi107.stdout:5/249: write d9/f36 [1046040,37150] 0 2024-04-22T23:35:03.017 INFO:tasks.workunit.client.1.smithi157.stdout:6/705: dread d1/d2f/d3b/d6c/d83/fa6 [0,4194304] 0 2024-04-22T23:35:03.017 INFO:tasks.workunit.client.0.smithi107.stdout:8/181: getdents d0/d7 0 2024-04-22T23:35:03.018 INFO:tasks.workunit.client.0.smithi107.stdout:8/182: readlink d0/d7/d8/l26 0 2024-04-22T23:35:03.018 INFO:tasks.workunit.client.1.smithi157.stdout:0/410: symlink d17/d2b/d4f/d5a/d7c/l8f 0 2024-04-22T23:35:03.019 INFO:tasks.workunit.client.0.smithi107.stdout:5/250: mknod d9/db/c51 0 2024-04-22T23:35:03.019 INFO:tasks.workunit.client.0.smithi107.stdout:9/331: read d5/d14/d28/f59 [1885113,70478] 0 2024-04-22T23:35:03.020 INFO:tasks.workunit.client.0.smithi107.stdout:9/332: dread - d5/d29/f61 zero size 2024-04-22T23:35:03.020 INFO:tasks.workunit.client.0.smithi107.stdout:5/251: symlink d9/db/de/l52 0 2024-04-22T23:35:03.020 INFO:tasks.workunit.client.1.smithi157.stdout:9/457: creat d4/d15/d23/d27/f8a x:0 0 0 2024-04-22T23:35:03.020 INFO:tasks.workunit.client.1.smithi157.stdout:9/458: write d4/df/d61/f71 [159507,98889] 0 2024-04-22T23:35:03.020 INFO:tasks.workunit.client.1.smithi157.stdout:9/459: dread - d4/d15/d23/d3f/f45 zero size 2024-04-22T23:35:03.021 INFO:tasks.workunit.client.0.smithi107.stdout:5/252: dread d9/f45 [0,4194304] 0 2024-04-22T23:35:03.021 INFO:tasks.workunit.client.0.smithi107.stdout:5/253: write d9/d26/f38 [490119,114391] 0 2024-04-22T23:35:03.022 INFO:tasks.workunit.client.1.smithi157.stdout:6/706: creat d1/d2f/d3b/d39/fdc x:0 0 0 2024-04-22T23:35:03.022 INFO:tasks.workunit.client.1.smithi157.stdout:0/411: rmdir d17/d2b/d71 39 2024-04-22T23:35:03.023 INFO:tasks.workunit.client.0.smithi107.stdout:5/254: creat d9/db/de/f53 x:0 0 0 2024-04-22T23:35:03.023 INFO:tasks.workunit.client.0.smithi107.stdout:5/255: fdatasync d9/db/f39 0 2024-04-22T23:35:03.024 INFO:tasks.workunit.client.1.smithi157.stdout:0/412: dread d17/d38/f65 [0,4194304] 0 2024-04-22T23:35:03.024 INFO:tasks.workunit.client.1.smithi157.stdout:9/460: getdents d4/d15 0 2024-04-22T23:35:03.025 INFO:tasks.workunit.client.1.smithi157.stdout:9/461: rmdir d4 39 2024-04-22T23:35:03.025 INFO:tasks.workunit.client.1.smithi157.stdout:9/462: dread - d4/d15/d23/d3f/f45 zero size 2024-04-22T23:35:03.030 INFO:tasks.workunit.client.0.smithi107.stdout:5/256: dread f4 [4194304,4194304] 0 2024-04-22T23:35:03.030 INFO:tasks.workunit.client.0.smithi107.stdout:5/257: rename d9/db to d9/db/d42/d54 22 2024-04-22T23:35:03.031 INFO:tasks.workunit.client.0.smithi107.stdout:9/333: write d5/d29/f2c [2431439,81074] 0 2024-04-22T23:35:03.031 INFO:tasks.workunit.client.0.smithi107.stdout:9/334: chown d5/d7/d17/d1b/d1f/c58 3391462 1 2024-04-22T23:35:03.032 INFO:tasks.workunit.client.1.smithi157.stdout:8/326: dread d2/d6/d21/f54 [0,4194304] 0 2024-04-22T23:35:03.033 INFO:tasks.workunit.client.0.smithi107.stdout:1/245: dwrite d7/d19/d20/d39/d53/f29 [0,4194304] 0 2024-04-22T23:35:03.034 INFO:tasks.workunit.client.1.smithi157.stdout:9/463: dread d4/df/f26 [0,4194304] 0 2024-04-22T23:35:03.034 INFO:tasks.workunit.client.1.smithi157.stdout:6/707: write d1/d2f/d3b/d39/d4e/da8/dac/db0/fdb [3586330,25109] 0 2024-04-22T23:35:03.035 INFO:tasks.workunit.client.0.smithi107.stdout:2/298: dwrite d1/d12/f26 [0,4194304] 0 2024-04-22T23:35:03.035 INFO:tasks.workunit.client.1.smithi157.stdout:6/708: fdatasync d1/d2f/d54/f6b 0 2024-04-22T23:35:03.036 INFO:tasks.workunit.client.1.smithi157.stdout:7/378: dwrite d0/d9/db/f5a [4194304,4194304] 0 2024-04-22T23:35:03.036 INFO:tasks.workunit.client.1.smithi157.stdout:5/483: dwrite d1/d5/d7/d2f/d45/da4/fac [0,4194304] 0 2024-04-22T23:35:03.039 INFO:tasks.workunit.client.0.smithi107.stdout:1/246: creat d7/d19/d54/f56 x:0 0 0 2024-04-22T23:35:03.040 INFO:tasks.workunit.client.1.smithi157.stdout:6/709: dread d1/f2c [0,4194304] 0 2024-04-22T23:35:03.043 INFO:tasks.workunit.client.1.smithi157.stdout:7/379: link d0/d9/d42/d55/d56/l58 d0/l7f 0 2024-04-22T23:35:03.045 INFO:tasks.workunit.client.0.smithi107.stdout:3/295: sync 2024-04-22T23:35:03.046 INFO:tasks.workunit.client.0.smithi107.stdout:0/288: sync 2024-04-22T23:35:03.046 INFO:tasks.workunit.client.0.smithi107.stdout:3/296: dread - db/d21/d27/d15/d29/d2a/d45/f62 zero size 2024-04-22T23:35:03.046 INFO:tasks.workunit.client.0.smithi107.stdout:8/183: dwrite d0/f2f [0,4194304] 0 2024-04-22T23:35:03.046 INFO:tasks.workunit.client.1.smithi157.stdout:0/413: dwrite d17/d23/d34/d67/d7f/f77 [0,4194304] 0 2024-04-22T23:35:03.046 INFO:tasks.workunit.client.1.smithi157.stdout:5/484: unlink d1/d1f/d22/l42 0 2024-04-22T23:35:03.047 INFO:tasks.workunit.client.0.smithi107.stdout:2/299: creat d1/d67/f6f x:0 0 0 2024-04-22T23:35:03.047 INFO:tasks.workunit.client.0.smithi107.stdout:2/300: chown d1/d67/f33 26294 1 2024-04-22T23:35:03.047 INFO:tasks.workunit.client.0.smithi107.stdout:2/301: fsync d1/d12/d35/f45 0 2024-04-22T23:35:03.051 INFO:tasks.workunit.client.0.smithi107.stdout:1/247: rmdir d7/d19/d54 39 2024-04-22T23:35:03.052 INFO:tasks.workunit.client.0.smithi107.stdout:8/184: dread d0/d7/df/f23 [0,4194304] 0 2024-04-22T23:35:03.052 INFO:tasks.workunit.client.0.smithi107.stdout:0/289: mknod d1/d2a/c6f 0 2024-04-22T23:35:03.058 INFO:tasks.workunit.client.0.smithi107.stdout:2/302: read d1/f11 [4029600,126926] 0 2024-04-22T23:35:03.060 INFO:tasks.workunit.client.1.smithi157.stdout:0/414: mknod d17/d2b/d4f/d5a/c90 0 2024-04-22T23:35:03.061 INFO:tasks.workunit.client.1.smithi157.stdout:5/485: mkdir d1/d5/d7/dbd 0 2024-04-22T23:35:03.061 INFO:tasks.workunit.client.1.smithi157.stdout:0/415: chown d17/d2b/l79 114662 1 2024-04-22T23:35:03.061 INFO:tasks.workunit.client.1.smithi157.stdout:5/486: chown d1/d5/dc/d43/l9b 608 1 2024-04-22T23:35:03.061 INFO:tasks.workunit.client.1.smithi157.stdout:7/380: getdents d0/d9/d29/d5e 0 2024-04-22T23:35:03.061 INFO:tasks.workunit.client.1.smithi157.stdout:7/381: fdatasync d0/d9/de/f5f 0 2024-04-22T23:35:03.062 INFO:tasks.workunit.client.1.smithi157.stdout:8/327: dwrite d2/d6/d21/d5f/fd [4194304,4194304] 0 2024-04-22T23:35:03.062 INFO:tasks.workunit.client.1.smithi157.stdout:8/328: fsync d2/d6/d21/d5f/d11/d13/f26 0 2024-04-22T23:35:03.063 INFO:tasks.workunit.client.1.smithi157.stdout:8/329: fsync d2/d4/f5b 0 2024-04-22T23:35:03.063 INFO:tasks.workunit.client.1.smithi157.stdout:0/416: symlink d17/d2b/d52/d7e/l91 0 2024-04-22T23:35:03.066 INFO:tasks.workunit.client.1.smithi157.stdout:5/487: unlink d1/d27/d29/d4b/l81 0 2024-04-22T23:35:03.067 INFO:tasks.workunit.client.0.smithi107.stdout:1/248: dread d7/f2a [0,4194304] 0 2024-04-22T23:35:03.069 INFO:tasks.workunit.client.1.smithi157.stdout:0/417: mknod d17/d23/d34/d67/d78/c92 0 2024-04-22T23:35:03.071 INFO:tasks.workunit.client.0.smithi107.stdout:1/249: truncate d7/d19/d20/f38 1425384 0 2024-04-22T23:35:03.071 INFO:tasks.workunit.client.0.smithi107.stdout:1/250: write d7/d19/d20/d39/d53/d43/d46/f47 [371060,35137] 0 2024-04-22T23:35:03.075 INFO:tasks.workunit.client.0.smithi107.stdout:1/251: rmdir d7/d19/d2d 39 2024-04-22T23:35:03.075 INFO:tasks.workunit.client.0.smithi107.stdout:1/252: fdatasync d7/d19/f2c 0 2024-04-22T23:35:03.075 INFO:tasks.workunit.client.0.smithi107.stdout:1/253: dread - d7/d19/d54/f4f zero size 2024-04-22T23:35:03.075 INFO:tasks.workunit.client.0.smithi107.stdout:1/254: dread - d7/d19/d54/f3a zero size 2024-04-22T23:35:03.083 INFO:tasks.workunit.client.0.smithi107.stdout:0/290: dwrite d1/f15 [0,4194304] 0 2024-04-22T23:35:03.090 INFO:tasks.workunit.client.1.smithi157.stdout:8/330: dread d2/d6/d21/d5f/d11/f4b [0,4194304] 0 2024-04-22T23:35:03.090 INFO:tasks.workunit.client.1.smithi157.stdout:8/331: dread - d2/d6/d21/d5f/f56 zero size 2024-04-22T23:35:03.094 INFO:tasks.workunit.client.0.smithi107.stdout:5/258: dwrite d9/f45 [0,4194304] 0 2024-04-22T23:35:03.095 INFO:tasks.workunit.client.0.smithi107.stdout:5/259: creat d9/d26/f55 x:0 0 0 2024-04-22T23:35:03.095 INFO:tasks.workunit.client.1.smithi157.stdout:8/332: symlink d2/d4/d30/l74 0 2024-04-22T23:35:03.095 INFO:tasks.workunit.client.1.smithi157.stdout:8/333: fsync d2/d6/d21/d5f/d11/d13/f6c 0 2024-04-22T23:35:03.099 INFO:tasks.workunit.client.1.smithi157.stdout:8/334: dread d2/d6/f46 [0,4194304] 0 2024-04-22T23:35:03.099 INFO:tasks.workunit.client.1.smithi157.stdout:8/335: getdents d2/d6/d6b 0 2024-04-22T23:35:03.102 INFO:tasks.workunit.client.1.smithi157.stdout:8/336: rename d2/d6/d21/d5f/d11/d13/c22 to d2/d6/d21/d33/d42/d58/c75 0 2024-04-22T23:35:03.102 INFO:tasks.workunit.client.1.smithi157.stdout:8/337: chown d2/f69 246 1 2024-04-22T23:35:03.103 INFO:tasks.workunit.client.0.smithi107.stdout:5/260: read d9/f12 [2894288,122185] 0 2024-04-22T23:35:03.104 INFO:tasks.workunit.client.0.smithi107.stdout:5/261: stat d9/db/c4d 0 2024-04-22T23:35:03.104 INFO:tasks.workunit.client.0.smithi107.stdout:5/262: read f7 [2922907,46273] 0 2024-04-22T23:35:03.104 INFO:tasks.workunit.client.0.smithi107.stdout:5/263: truncate f7 4268949 0 2024-04-22T23:35:03.104 INFO:tasks.workunit.client.0.smithi107.stdout:5/264: chown d9/db/d42/c4f 1616 1 2024-04-22T23:35:03.104 INFO:tasks.workunit.client.0.smithi107.stdout:5/265: read f3 [2961422,84411] 0 2024-04-22T23:35:03.105 INFO:tasks.workunit.client.0.smithi107.stdout:5/266: creat d9/db/d42/f56 x:0 0 0 2024-04-22T23:35:03.105 INFO:tasks.workunit.client.0.smithi107.stdout:5/267: fdatasync d9/db/f27 0 2024-04-22T23:35:03.105 INFO:tasks.workunit.client.0.smithi107.stdout:5/268: chown d9/c2d 17012042 1 2024-04-22T23:35:03.107 INFO:tasks.workunit.client.0.smithi107.stdout:3/297: dwrite db/ff [8388608,4194304] 0 2024-04-22T23:35:03.108 INFO:tasks.workunit.client.1.smithi157.stdout:9/464: dwrite d4/d15/d23/f69 [0,4194304] 0 2024-04-22T23:35:03.108 INFO:tasks.workunit.client.0.smithi107.stdout:8/185: dwrite d0/d7/d20/f39 [0,4194304] 0 2024-04-22T23:35:03.109 INFO:tasks.workunit.client.1.smithi157.stdout:9/465: write d4/d15/d55/f76 [57652,40426] 0 2024-04-22T23:35:03.109 INFO:tasks.workunit.client.1.smithi157.stdout:7/382: dwrite d0/d9/d29/f3b [0,4194304] 0 2024-04-22T23:35:03.109 INFO:tasks.workunit.client.1.smithi157.stdout:6/710: dwrite d1/f2c [0,4194304] 0 2024-04-22T23:35:03.109 INFO:tasks.workunit.client.1.smithi157.stdout:6/711: write d1/d2f/d3b/d39/d4e/f5f [481373,107834] 0 2024-04-22T23:35:03.109 INFO:tasks.workunit.client.1.smithi157.stdout:6/712: chown d1/d17/f28 104604 1 2024-04-22T23:35:03.110 INFO:tasks.workunit.client.1.smithi157.stdout:5/488: dwrite d1/d5/d7/d2f/d51/f97 [0,4194304] 0 2024-04-22T23:35:03.111 INFO:tasks.workunit.client.0.smithi107.stdout:2/303: dwrite d1/d12/d35/d48/d13/f40 [0,4194304] 0 2024-04-22T23:35:03.111 INFO:tasks.workunit.client.0.smithi107.stdout:2/304: chown d1 62 1 2024-04-22T23:35:03.111 INFO:tasks.workunit.client.0.smithi107.stdout:3/298: rename db/d21/d27/d54/f4e to db/d21/d27/d15/d29/f67 0 2024-04-22T23:35:03.111 INFO:tasks.workunit.client.0.smithi107.stdout:3/299: write db/d21/d27/d15/d29/f42 [892887,100693] 0 2024-04-22T23:35:03.111 INFO:tasks.workunit.client.0.smithi107.stdout:0/291: dwrite d1/d27/d20/d2c/d41/d52/f6c [0,4194304] 0 2024-04-22T23:35:03.111 INFO:tasks.workunit.client.0.smithi107.stdout:0/292: chown d1/d27/d20/d2c/c3a 1 1 2024-04-22T23:35:03.111 INFO:tasks.workunit.client.0.smithi107.stdout:1/255: dread d7/d19/d20/d39/d53/d43/d46/f47 [0,4194304] 0 2024-04-22T23:35:03.111 INFO:tasks.workunit.client.1.smithi157.stdout:9/466: creat d4/df/f8b x:0 0 0 2024-04-22T23:35:03.113 INFO:tasks.workunit.client.1.smithi157.stdout:7/383: mkdir d0/d9/d42/d55/d80 0 2024-04-22T23:35:03.113 INFO:tasks.workunit.client.1.smithi157.stdout:7/384: dread - d0/d9/d29/d33/f4e zero size 2024-04-22T23:35:03.113 INFO:tasks.workunit.client.0.smithi107.stdout:8/186: mkdir d0/d3a 0 2024-04-22T23:35:03.115 INFO:tasks.workunit.client.1.smithi157.stdout:6/713: mkdir d1/d2f/d3b/d39/d4e/da8/dac/ddd 0 2024-04-22T23:35:03.115 INFO:tasks.workunit.client.1.smithi157.stdout:6/714: stat d1/d38/cc6 0 2024-04-22T23:35:03.118 INFO:tasks.workunit.client.0.smithi107.stdout:2/305: truncate d1/d12/f34 892947 0 2024-04-22T23:35:03.118 INFO:tasks.workunit.client.0.smithi107.stdout:2/306: read d1/d12/d35/d48/f2c [1498668,68979] 0 2024-04-22T23:35:03.118 INFO:tasks.workunit.client.0.smithi107.stdout:2/307: stat d1/d12/d35/d48/c20 0 2024-04-22T23:35:03.123 INFO:tasks.workunit.client.1.smithi157.stdout:9/467: truncate d4/d15/d23/d27/f53 997469 0 2024-04-22T23:35:03.126 INFO:tasks.workunit.client.1.smithi157.stdout:7/385: truncate d0/f2 11938647 0 2024-04-22T23:35:03.130 INFO:tasks.workunit.client.1.smithi157.stdout:7/386: symlink d0/d9/d10/l81 0 2024-04-22T23:35:03.131 INFO:tasks.workunit.client.0.smithi107.stdout:0/293: symlink d1/d27/d20/d2c/d41/d52/l70 0 2024-04-22T23:35:03.131 INFO:tasks.workunit.client.0.smithi107.stdout:0/294: creat d1/d27/d20/d2c/d2e/f71 x:0 0 0 2024-04-22T23:35:03.134 INFO:tasks.workunit.client.1.smithi157.stdout:7/387: dread d0/d9/de/f23 [0,4194304] 0 2024-04-22T23:35:03.134 INFO:tasks.workunit.client.1.smithi157.stdout:5/489: dwrite d1/d5/d7/f72 [0,4194304] 0 2024-04-22T23:35:03.134 INFO:tasks.workunit.client.1.smithi157.stdout:5/490: chown d1/d5/d7/f67 433701 1 2024-04-22T23:35:03.135 INFO:tasks.workunit.client.1.smithi157.stdout:5/491: chown d1/d27/d29/d4b 7051 1 2024-04-22T23:35:03.135 INFO:tasks.workunit.client.0.smithi107.stdout:2/308: mknod d1/d2b/d49/c70 0 2024-04-22T23:35:03.135 INFO:tasks.workunit.client.1.smithi157.stdout:7/388: mkdir d0/d9/d42/d55/d80/d82 0 2024-04-22T23:35:03.136 INFO:tasks.workunit.client.1.smithi157.stdout:7/389: readlink d0/d9/d10/l81 0 2024-04-22T23:35:03.137 INFO:tasks.workunit.client.0.smithi107.stdout:8/187: dwrite d0/d7/d8/f1e [0,4194304] 0 2024-04-22T23:35:03.137 INFO:tasks.workunit.client.0.smithi107.stdout:8/188: dread - d0/d7/df/d18/f1f zero size 2024-04-22T23:35:03.137 INFO:tasks.workunit.client.0.smithi107.stdout:8/189: write d0/d6/f2b [3913040,22334] 0 2024-04-22T23:35:03.138 INFO:tasks.workunit.client.0.smithi107.stdout:1/256: getdents d7/d19/d20/d39 0 2024-04-22T23:35:03.138 INFO:tasks.workunit.client.0.smithi107.stdout:1/257: write d7/d19/d20/f38 [735045,17593] 0 2024-04-22T23:35:03.140 INFO:tasks.workunit.client.0.smithi107.stdout:2/309: link d1/d12/d35/d48/d13/c55 d1/d12/d35/d54/c71 0 2024-04-22T23:35:03.140 INFO:tasks.workunit.client.0.smithi107.stdout:2/310: truncate d1/f3c 4758395 0 2024-04-22T23:35:03.140 INFO:tasks.workunit.client.0.smithi107.stdout:2/311: readlink d1/d12/d35/d48/l24 0 2024-04-22T23:35:03.141 INFO:tasks.workunit.client.1.smithi157.stdout:5/492: dread d1/d5/d7/d10/f49 [0,4194304] 0 2024-04-22T23:35:03.141 INFO:tasks.workunit.client.1.smithi157.stdout:7/390: symlink d0/d9/d10/l83 0 2024-04-22T23:35:03.144 INFO:tasks.workunit.client.0.smithi107.stdout:0/295: dread d1/d27/d20/d2c/d41/f54 [0,4194304] 0 2024-04-22T23:35:03.145 INFO:tasks.workunit.client.1.smithi157.stdout:7/391: getdents d0 0 2024-04-22T23:35:03.146 INFO:tasks.workunit.client.0.smithi107.stdout:8/190: rename d0/d6 to d0/d7/df/d3b 0 2024-04-22T23:35:03.148 INFO:tasks.workunit.client.0.smithi107.stdout:1/258: unlink d7/d19/d20/d39/d53/l45 0 2024-04-22T23:35:03.148 INFO:tasks.workunit.client.0.smithi107.stdout:1/259: readlink d7/l34 0 2024-04-22T23:35:03.150 INFO:tasks.workunit.client.0.smithi107.stdout:1/260: mknod d7/d19/d20/d39/d53/c57 0 2024-04-22T23:35:03.151 INFO:tasks.workunit.client.0.smithi107.stdout:0/296: getdents d1/d9/d14/d4f 0 2024-04-22T23:35:03.157 INFO:tasks.workunit.client.0.smithi107.stdout:8/191: dread d0/d7/f11 [0,4194304] 0 2024-04-22T23:35:03.157 INFO:tasks.workunit.client.0.smithi107.stdout:8/192: truncate d0/d7/df/d33/d37/f38 636780 0 2024-04-22T23:35:03.160 INFO:tasks.workunit.client.1.smithi157.stdout:0/418: dwrite d17/f20 [0,4194304] 0 2024-04-22T23:35:03.160 INFO:tasks.workunit.client.1.smithi157.stdout:0/419: readlink d17/l26 0 2024-04-22T23:35:03.160 INFO:tasks.workunit.client.1.smithi157.stdout:9/468: dwrite d4/d15/d23/d27/f64 [0,4194304] 0 2024-04-22T23:35:03.160 INFO:tasks.workunit.client.1.smithi157.stdout:7/392: dread d0/d9/f1d [0,4194304] 0 2024-04-22T23:35:03.160 INFO:tasks.workunit.client.0.smithi107.stdout:8/193: creat d0/d7/df/d3b/f3c x:0 0 0 2024-04-22T23:35:03.161 INFO:tasks.workunit.client.1.smithi157.stdout:6/715: dwrite d1/d17/fa2 [0,4194304] 0 2024-04-22T23:35:03.161 INFO:tasks.workunit.client.1.smithi157.stdout:6/716: chown d1/d2f/d3b/d6c/da7/lca 34 1 2024-04-22T23:35:03.162 INFO:tasks.workunit.client.1.smithi157.stdout:0/420: mkdir d17/d2b/d52/d93 0 2024-04-22T23:35:03.162 INFO:tasks.workunit.client.1.smithi157.stdout:0/421: fsync f11 0 2024-04-22T23:35:03.165 INFO:tasks.workunit.client.0.smithi107.stdout:8/194: creat d0/d7/df/d3b/f3d x:0 0 0 2024-04-22T23:35:03.165 INFO:tasks.workunit.client.0.smithi107.stdout:3/300: dwrite f0 [4194304,4194304] 0 2024-04-22T23:35:03.165 INFO:tasks.workunit.client.1.smithi157.stdout:6/717: creat d1/d2f/d54/d5c/fde x:0 0 0 2024-04-22T23:35:03.166 INFO:tasks.workunit.client.0.smithi107.stdout:7/223: sync 2024-04-22T23:35:03.169 INFO:tasks.workunit.client.1.smithi157.stdout:0/422: symlink d17/d2b/l94 0 2024-04-22T23:35:03.170 INFO:tasks.workunit.client.0.smithi107.stdout:8/195: symlink d0/d7/df/l3e 0 2024-04-22T23:35:03.172 INFO:tasks.workunit.client.1.smithi157.stdout:6/718: rename d1/d2f/d3b/d6c/d83/d93/dd7 to d1/d2f/d3b/d39/d4e/da8/dac/ddd/ddf 0 2024-04-22T23:35:03.173 INFO:tasks.workunit.client.1.smithi157.stdout:0/423: creat d17/d2b/d52/d7e/f95 x:0 0 0 2024-04-22T23:35:03.174 INFO:tasks.workunit.client.1.smithi157.stdout:0/424: fsync d17/d23/d34/d67/d7f/f5f 0 2024-04-22T23:35:03.174 INFO:tasks.workunit.client.1.smithi157.stdout:0/425: fdatasync d17/d2b/f84 0 2024-04-22T23:35:03.174 INFO:tasks.workunit.client.1.smithi157.stdout:0/426: write d17/d2b/d52/f53 [1132334,3920] 0 2024-04-22T23:35:03.175 INFO:tasks.workunit.client.0.smithi107.stdout:1/261: dwrite d7/f15 [0,4194304] 0 2024-04-22T23:35:03.175 INFO:tasks.workunit.client.0.smithi107.stdout:1/262: dread - d7/d19/d54/f3a zero size 2024-04-22T23:35:03.176 INFO:tasks.workunit.client.0.smithi107.stdout:8/196: truncate d0/d7/fa 2344520 0 2024-04-22T23:35:03.176 INFO:tasks.workunit.client.0.smithi107.stdout:8/197: getdents d0/d7/df/d18 0 2024-04-22T23:35:03.178 INFO:tasks.workunit.client.1.smithi157.stdout:0/427: mknod d17/d2b/d52/d93/c96 0 2024-04-22T23:35:03.179 INFO:tasks.workunit.client.1.smithi157.stdout:0/428: chown d17/d2b/f86 26155889 1 2024-04-22T23:35:03.179 INFO:tasks.workunit.client.1.smithi157.stdout:0/429: creat d17/d2b/d52/d7e/f97 x:0 0 0 2024-04-22T23:35:03.180 INFO:tasks.workunit.client.1.smithi157.stdout:6/719: symlink d1/d2f/d3b/d39/le0 0 2024-04-22T23:35:03.180 INFO:tasks.workunit.client.1.smithi157.stdout:7/393: read d0/f2 [9259027,1008] 0 2024-04-22T23:35:03.181 INFO:tasks.workunit.client.0.smithi107.stdout:1/263: rename f2 to d7/d19/d20/f58 0 2024-04-22T23:35:03.181 INFO:tasks.workunit.client.1.smithi157.stdout:0/430: creat d17/d2b/d4f/d5a/d7c/f98 x:0 0 0 2024-04-22T23:35:03.181 INFO:tasks.workunit.client.1.smithi157.stdout:0/431: chown d17/f22 393100 1 2024-04-22T23:35:03.182 INFO:tasks.workunit.client.1.smithi157.stdout:7/394: creat d0/d9/d29/d49/f84 x:0 0 0 2024-04-22T23:35:03.182 INFO:tasks.workunit.client.0.smithi107.stdout:1/264: mknod d7/d19/d2d/c59 0 2024-04-22T23:35:03.183 INFO:tasks.workunit.client.0.smithi107.stdout:1/265: write d7/d19/d2d/f35 [258788,75407] 0 2024-04-22T23:35:03.183 INFO:tasks.workunit.client.1.smithi157.stdout:0/432: symlink d17/d2b/d4f/d5a/d7c/l99 0 2024-04-22T23:35:03.184 INFO:tasks.workunit.client.1.smithi157.stdout:7/395: mkdir d0/d9/d29/d5e/d85 0 2024-04-22T23:35:03.184 INFO:tasks.workunit.client.1.smithi157.stdout:7/396: read - d0/d9/d29/d49/d4a/f67 zero size 2024-04-22T23:35:03.185 INFO:tasks.workunit.client.1.smithi157.stdout:0/433: mknod d17/d2b/d52/d93/c9a 0 2024-04-22T23:35:03.185 INFO:tasks.workunit.client.1.smithi157.stdout:0/434: write d17/d2b/f5c [1471783,21323] 0 2024-04-22T23:35:03.186 INFO:tasks.workunit.client.0.smithi107.stdout:1/266: creat d7/d19/d20/d39/f5a x:0 0 0 2024-04-22T23:35:03.186 INFO:tasks.workunit.client.1.smithi157.stdout:7/397: rename d0/d9/l1a to d0/d9/d29/d33/d37/l86 0 2024-04-22T23:35:03.187 INFO:tasks.workunit.client.0.smithi107.stdout:0/297: truncate d1/d27/d20/d2c/d41/d52/f6c 2916103 0 2024-04-22T23:35:03.187 INFO:tasks.workunit.client.1.smithi157.stdout:0/435: mkdir d17/d2b/d4f/d9b 0 2024-04-22T23:35:03.188 INFO:tasks.workunit.client.0.smithi107.stdout:8/198: write d0/d7/f31 [2420215,14299] 0 2024-04-22T23:35:03.188 INFO:tasks.workunit.client.0.smithi107.stdout:8/199: dread - d0/d7/df/d3b/f3c zero size 2024-04-22T23:35:03.188 INFO:tasks.workunit.client.0.smithi107.stdout:1/267: mknod d7/d19/d42/c5b 0 2024-04-22T23:35:03.189 INFO:tasks.workunit.client.1.smithi157.stdout:9/469: dwrite d4/df/d57/f2b [0,4194304] 0 2024-04-22T23:35:03.189 INFO:tasks.workunit.client.0.smithi107.stdout:7/224: dwrite d7/f18 [0,4194304] 0 2024-04-22T23:35:03.189 INFO:tasks.workunit.client.0.smithi107.stdout:7/225: creat d7/d1b/f4d x:0 0 0 2024-04-22T23:35:03.192 INFO:tasks.workunit.client.0.smithi107.stdout:8/200: mknod d0/d3a/c3f 0 2024-04-22T23:35:03.192 INFO:tasks.workunit.client.0.smithi107.stdout:8/201: fsync d0/d7/df/d33/d37/f38 0 2024-04-22T23:35:03.194 INFO:tasks.workunit.client.0.smithi107.stdout:1/268: mkdir d7/d19/d20/d39/d53/d43/d46/d5c 0 2024-04-22T23:35:03.199 INFO:tasks.workunit.client.0.smithi107.stdout:7/226: symlink d7/d4c/l4e 0 2024-04-22T23:35:03.200 INFO:tasks.workunit.client.0.smithi107.stdout:1/269: mknod d7/d19/d20/d39/d53/d43/d46/c5d 0 2024-04-22T23:35:03.200 INFO:tasks.workunit.client.0.smithi107.stdout:0/298: write d1/d27/f21 [1705254,116645] 0 2024-04-22T23:35:03.201 INFO:tasks.workunit.client.0.smithi107.stdout:3/301: truncate f0 5824913 0 2024-04-22T23:35:03.203 INFO:tasks.workunit.client.0.smithi107.stdout:1/270: creat d7/d19/d20/d39/d53/d43/d46/d5c/f5e x:0 0 0 2024-04-22T23:35:03.205 INFO:tasks.workunit.client.0.smithi107.stdout:8/202: write d0/d7/d20/f39 [1830324,114432] 0 2024-04-22T23:35:03.205 INFO:tasks.workunit.client.1.smithi157.stdout:6/720: dwrite d1/d17/d5d/f64 [0,4194304] 0 2024-04-22T23:35:03.208 INFO:tasks.workunit.client.1.smithi157.stdout:9/470: dread d4/df/f12 [0,4194304] 0 2024-04-22T23:35:03.208 INFO:tasks.workunit.client.1.smithi157.stdout:0/436: rename d17/d2b/d52/d93 to d17/d23/d9c 0 2024-04-22T23:35:03.208 INFO:tasks.workunit.client.1.smithi157.stdout:0/437: stat d17/d23/f7a 0 2024-04-22T23:35:03.208 INFO:tasks.workunit.client.1.smithi157.stdout:0/438: readlink d17/l7d 0 2024-04-22T23:35:03.208 INFO:tasks.workunit.client.1.smithi157.stdout:0/439: write d17/d38/f6d [939083,18913] 0 2024-04-22T23:35:03.208 INFO:tasks.workunit.client.1.smithi157.stdout:6/721: dread d1/f88 [4194304,4194304] 0 2024-04-22T23:35:03.208 INFO:tasks.workunit.client.1.smithi157.stdout:6/722: fdatasync d1/d2f/d54/d9b/f61 0 2024-04-22T23:35:03.208 INFO:tasks.workunit.client.1.smithi157.stdout:7/398: dwrite d0/d9/d42/f7a [0,4194304] 0 2024-04-22T23:35:03.209 INFO:tasks.workunit.client.0.smithi107.stdout:8/203: creat d0/d7/df/d17/f40 x:0 0 0 2024-04-22T23:35:03.209 INFO:tasks.workunit.client.0.smithi107.stdout:8/204: dread - d0/d7/df/d17/f40 zero size 2024-04-22T23:35:03.210 INFO:tasks.workunit.client.1.smithi157.stdout:9/471: unlink d4/d15/d23/d27/d47/c63 0 2024-04-22T23:35:03.210 INFO:tasks.workunit.client.0.smithi107.stdout:7/227: write d7/f26 [3695907,113291] 0 2024-04-22T23:35:03.210 INFO:tasks.workunit.client.0.smithi107.stdout:7/228: fsync d7/d1b/f38 0 2024-04-22T23:35:03.211 INFO:tasks.workunit.client.0.smithi107.stdout:8/205: mkdir d0/d7/d8/d41 0 2024-04-22T23:35:03.213 INFO:tasks.workunit.client.1.smithi157.stdout:7/399: getdents d0/d9/d29/d49/d64 0 2024-04-22T23:35:03.216 INFO:tasks.workunit.client.1.smithi157.stdout:7/400: unlink d0/d9/db/d35/f3a 0 2024-04-22T23:35:03.217 INFO:tasks.workunit.client.1.smithi157.stdout:7/401: rename d0/d9/d29/d33/f4e to d0/d9/d29/d33/d37/f87 0 2024-04-22T23:35:03.219 INFO:tasks.workunit.client.0.smithi107.stdout:8/206: mknod d0/d7/d20/d2d/c42 0 2024-04-22T23:35:03.220 INFO:tasks.workunit.client.1.smithi157.stdout:7/402: unlink d0/d9/d29/d49/c6b 0 2024-04-22T23:35:03.222 INFO:tasks.workunit.client.1.smithi157.stdout:6/723: dread d1/d2f/d3b/d6c/f6d [0,4194304] 0 2024-04-22T23:35:03.222 INFO:tasks.workunit.client.1.smithi157.stdout:6/724: chown d1/d2f/d3b/d39/d4e/da8/dac/dbe 7 1 2024-04-22T23:35:03.224 INFO:tasks.workunit.client.1.smithi157.stdout:1/531: sync 2024-04-22T23:35:03.224 INFO:tasks.workunit.client.1.smithi157.stdout:2/601: sync 2024-04-22T23:35:03.224 INFO:tasks.workunit.client.1.smithi157.stdout:3/459: sync 2024-04-22T23:35:03.224 INFO:tasks.workunit.client.1.smithi157.stdout:4/606: sync 2024-04-22T23:35:03.225 INFO:tasks.workunit.client.0.smithi107.stdout:4/261: sync 2024-04-22T23:35:03.225 INFO:tasks.workunit.client.0.smithi107.stdout:4/262: fsync d1/d21/f36 0 2024-04-22T23:35:03.226 INFO:tasks.workunit.client.1.smithi157.stdout:6/725: rename d1/d2f/d54/f6b to d1/d2f/d3b/d39/fe1 0 2024-04-22T23:35:03.226 INFO:tasks.workunit.client.1.smithi157.stdout:6/726: truncate d1/d2f/d3b/d6c/d83/f9f 922705 0 2024-04-22T23:35:03.226 INFO:tasks.workunit.client.1.smithi157.stdout:6/727: chown d1/d36/f37 548 1 2024-04-22T23:35:03.226 INFO:tasks.workunit.client.1.smithi157.stdout:6/728: write d1/d17/f94 [1449032,126293] 0 2024-04-22T23:35:03.226 INFO:tasks.workunit.client.0.smithi107.stdout:4/263: rmdir d1/d21 39 2024-04-22T23:35:03.226 INFO:tasks.workunit.client.0.smithi107.stdout:4/264: dread - d1/d8/f4b zero size 2024-04-22T23:35:03.226 INFO:tasks.workunit.client.0.smithi107.stdout:4/265: chown d1/l15 53 1 2024-04-22T23:35:03.228 INFO:tasks.workunit.client.1.smithi157.stdout:2/602: mknod d7/d8/d30/d5b/d8a/ccd 0 2024-04-22T23:35:03.228 INFO:tasks.workunit.client.1.smithi157.stdout:2/603: creat d7/d8/d30/d5b/fce x:0 0 0 2024-04-22T23:35:03.228 INFO:tasks.workunit.client.1.smithi157.stdout:2/604: truncate d7/d8/d8e/f7e 617087 0 2024-04-22T23:35:03.229 INFO:tasks.workunit.client.1.smithi157.stdout:0/440: dread d17/d23/d34/f7b [0,4194304] 0 2024-04-22T23:35:03.229 INFO:tasks.workunit.client.1.smithi157.stdout:0/441: chown d17/d23/d34/d67/d78/c8b 13633 1 2024-04-22T23:35:03.229 INFO:tasks.workunit.client.1.smithi157.stdout:0/442: write d17/d38/f65 [1695721,24848] 0 2024-04-22T23:35:03.230 INFO:tasks.workunit.client.1.smithi157.stdout:3/460: unlink d0/d3/d34/d39/l1a 0 2024-04-22T23:35:03.231 INFO:tasks.workunit.client.0.smithi107.stdout:4/266: link d1/d1b/f49 d1/d7/d5b/f5e 0 2024-04-22T23:35:03.231 INFO:tasks.workunit.client.0.smithi107.stdout:4/267: chown d1/d21/d29/d2e/d3b/c40 3412019 1 2024-04-22T23:35:03.232 INFO:tasks.workunit.client.1.smithi157.stdout:4/607: mkdir d3/d11/d1d/d73/da3/d31/d91/dcb 0 2024-04-22T23:35:03.233 INFO:tasks.workunit.client.1.smithi157.stdout:6/729: rmdir d1/d2f/d3b/d39/d4e/da8/dac/ddd 39 2024-04-22T23:35:03.234 INFO:tasks.workunit.client.0.smithi107.stdout:4/268: symlink d1/d8/d56/d27/l5f 0 2024-04-22T23:35:03.234 INFO:tasks.workunit.client.1.smithi157.stdout:2/605: getdents d7/d8/d25 0 2024-04-22T23:35:03.236 INFO:tasks.workunit.client.0.smithi107.stdout:4/269: symlink d1/d21/d29/d2e/d4d/l60 0 2024-04-22T23:35:03.236 INFO:tasks.workunit.client.1.smithi157.stdout:0/443: mknod d17/d2b/c9d 0 2024-04-22T23:35:03.238 INFO:tasks.workunit.client.0.smithi107.stdout:1/271: dwrite d7/d19/d20/d39/d53/d43/f49 [0,4194304] 0 2024-04-22T23:35:03.238 INFO:tasks.workunit.client.1.smithi157.stdout:4/608: rename d3/d11/d1d/d73/d80/dac/db5/c8e to d3/d96/d40/ccc 0 2024-04-22T23:35:03.239 INFO:tasks.workunit.client.0.smithi107.stdout:3/302: dread db/d21/d27/d15/d29/d2a/d40/f5e [0,4194304] 0 2024-04-22T23:35:03.239 INFO:tasks.workunit.client.0.smithi107.stdout:3/303: fdatasync db/d21/d27/f23 0 2024-04-22T23:35:03.239 INFO:tasks.workunit.client.0.smithi107.stdout:3/304: read db/d21/d27/d15/d29/d2a/f4c [398034,127443] 0 2024-04-22T23:35:03.240 INFO:tasks.workunit.client.1.smithi157.stdout:6/730: link d1/d2f/f4a d1/d2f/d3b/fe2 0 2024-04-22T23:35:03.240 INFO:tasks.workunit.client.1.smithi157.stdout:0/444: truncate d17/f22 2562084 0 2024-04-22T23:35:03.241 INFO:tasks.workunit.client.0.smithi107.stdout:1/272: rename d7/d19/d20/d39/d53/f27 to d7/d19/d20/d39/d53/f5f 0 2024-04-22T23:35:03.242 INFO:tasks.workunit.client.0.smithi107.stdout:1/273: write d7/d19/d20/d39/d53/d43/d46/f47 [1036969,10603] 0 2024-04-22T23:35:03.242 INFO:tasks.workunit.client.1.smithi157.stdout:4/609: mkdir d3/d11/d1d/d73/da3/d31/d91/dcb/dcd 0 2024-04-22T23:35:03.242 INFO:tasks.workunit.client.1.smithi157.stdout:4/610: read - d3/d5c/f95 zero size 2024-04-22T23:35:03.243 INFO:tasks.workunit.client.1.smithi157.stdout:9/472: dread d4/d15/d23/f69 [0,4194304] 0 2024-04-22T23:35:03.243 INFO:tasks.workunit.client.1.smithi157.stdout:9/473: creat d4/d15/d23/f8c x:0 0 0 2024-04-22T23:35:03.243 INFO:tasks.workunit.client.1.smithi157.stdout:6/731: creat d1/d2f/d3b/d39/dd3/fe3 x:0 0 0 2024-04-22T23:35:03.243 INFO:tasks.workunit.client.1.smithi157.stdout:3/461: dread d0/f1c [0,4194304] 0 2024-04-22T23:35:03.243 INFO:tasks.workunit.client.0.smithi107.stdout:1/274: rename d7/c23 to d7/d19/d42/c60 0 2024-04-22T23:35:03.246 INFO:tasks.workunit.client.1.smithi157.stdout:4/611: link d3/d11/d34/da4/fbc d3/d11/d1d/fce 0 2024-04-22T23:35:03.247 INFO:tasks.workunit.client.0.smithi107.stdout:7/229: dwrite d7/d1b/f4d [0,4194304] 0 2024-04-22T23:35:03.247 INFO:tasks.workunit.client.0.smithi107.stdout:0/299: dwrite d1/d27/d20/d2c/d2e/f71 [0,4194304] 0 2024-04-22T23:35:03.247 INFO:tasks.workunit.client.0.smithi107.stdout:0/300: dread - d1/d9/f5f zero size 2024-04-22T23:35:03.247 INFO:tasks.workunit.client.0.smithi107.stdout:0/301: chown d1/f15 102626642 1 2024-04-22T23:35:03.247 INFO:tasks.workunit.client.0.smithi107.stdout:8/207: dread d0/fc [0,4194304] 0 2024-04-22T23:35:03.247 INFO:tasks.workunit.client.1.smithi157.stdout:7/403: dwrite d0/d9/d29/f4d [0,4194304] 0 2024-04-22T23:35:03.247 INFO:tasks.workunit.client.1.smithi157.stdout:7/404: stat d0/d9/de 0 2024-04-22T23:35:03.248 INFO:tasks.workunit.client.0.smithi107.stdout:4/270: dread d1/d7/d5b/f5e [0,4194304] 0 2024-04-22T23:35:03.248 INFO:tasks.workunit.client.0.smithi107.stdout:3/305: dread db/d21/d27/f20 [0,4194304] 0 2024-04-22T23:35:03.249 INFO:tasks.workunit.client.1.smithi157.stdout:9/474: unlink d4/d15/f73 0 2024-04-22T23:35:03.250 INFO:tasks.workunit.client.1.smithi157.stdout:6/732: symlink d1/d2f/d3b/d39/d4e/da8/dac/dbe/le4 0 2024-04-22T23:35:03.251 INFO:tasks.workunit.client.0.smithi107.stdout:0/302: symlink d1/d27/l72 0 2024-04-22T23:35:03.251 INFO:tasks.workunit.client.0.smithi107.stdout:0/303: fsync d1/f15 0 2024-04-22T23:35:03.253 INFO:tasks.workunit.client.1.smithi157.stdout:3/462: truncate d0/d3/d34/d3d/f63 1715450 0 2024-04-22T23:35:03.254 INFO:tasks.workunit.client.0.smithi107.stdout:4/271: symlink d1/d21/d29/d2e/d4d/l61 0 2024-04-22T23:35:03.256 INFO:tasks.workunit.client.1.smithi157.stdout:0/445: getdents d17/d2b/d4f/d5a/d7c 0 2024-04-22T23:35:03.256 INFO:tasks.workunit.client.0.smithi107.stdout:3/306: truncate db/d21/d27/d15/d29/d2a/f4c 251520 0 2024-04-22T23:35:03.258 INFO:tasks.workunit.client.0.smithi107.stdout:7/230: symlink d7/l4f 0 2024-04-22T23:35:03.259 INFO:tasks.workunit.client.1.smithi157.stdout:7/405: truncate d0/d9/d10/f30 246991 0 2024-04-22T23:35:03.259 INFO:tasks.workunit.client.1.smithi157.stdout:7/406: fsync d0/d9/db/d35/f76 0 2024-04-22T23:35:03.261 INFO:tasks.workunit.client.1.smithi157.stdout:9/475: dread d4/d2d/f78 [0,4194304] 0 2024-04-22T23:35:03.262 INFO:tasks.workunit.client.0.smithi107.stdout:0/304: link d1/d9/d14/d4f/f69 d1/d27/d20/d2c/d41/d52/f73 0 2024-04-22T23:35:03.262 INFO:tasks.workunit.client.0.smithi107.stdout:0/305: write d1/d27/d20/f36 [4091303,65847] 0 2024-04-22T23:35:03.264 INFO:tasks.workunit.client.1.smithi157.stdout:3/463: symlink d0/d3/d34/d47/l90 0 2024-04-22T23:35:03.266 INFO:tasks.workunit.client.1.smithi157.stdout:0/446: write d17/d23/d34/d67/d7f/f5f [2149767,22838] 0 2024-04-22T23:35:03.266 INFO:tasks.workunit.client.0.smithi107.stdout:4/272: unlink d1/d7/c1c 0 2024-04-22T23:35:03.267 INFO:tasks.workunit.client.1.smithi157.stdout:7/407: symlink d0/d9/db/df/l88 0 2024-04-22T23:35:03.269 INFO:tasks.workunit.client.1.smithi157.stdout:6/733: rename d1/d2f/d3b/d6c to d1/d2f/d3b/d39/d4e/da8/de5 0 2024-04-22T23:35:03.269 INFO:tasks.workunit.client.1.smithi157.stdout:2/606: dwrite d7/d8/d6b/f97 [0,4194304] 0 2024-04-22T23:35:03.269 INFO:tasks.workunit.client.1.smithi157.stdout:3/464: dread d0/d3/d34/d3d/f44 [0,4194304] 0 2024-04-22T23:35:03.269 INFO:tasks.workunit.client.0.smithi107.stdout:7/231: mknod d7/d21/c50 0 2024-04-22T23:35:03.270 INFO:tasks.workunit.client.0.smithi107.stdout:7/232: fdatasync d7/d2d/f41 0 2024-04-22T23:35:03.270 INFO:tasks.workunit.client.0.smithi107.stdout:7/233: chown d7/d2d/c47 0 1 2024-04-22T23:35:03.270 INFO:tasks.workunit.client.0.smithi107.stdout:7/234: chown d7/l35 114202455 1 2024-04-22T23:35:03.270 INFO:tasks.workunit.client.1.smithi157.stdout:0/447: creat d17/d23/d34/f9e x:0 0 0 2024-04-22T23:35:03.271 INFO:tasks.workunit.client.1.smithi157.stdout:0/448: fsync d17/d23/f51 0 2024-04-22T23:35:03.276 INFO:tasks.workunit.client.1.smithi157.stdout:0/449: dread d17/d2b/d52/f3f [0,4194304] 0 2024-04-22T23:35:03.276 INFO:tasks.workunit.client.1.smithi157.stdout:0/450: fsync d17/d2b/f66 0 2024-04-22T23:35:03.276 INFO:tasks.workunit.client.1.smithi157.stdout:0/451: chown d17/d38/f6d 838 1 2024-04-22T23:35:03.276 INFO:tasks.workunit.client.1.smithi157.stdout:0/452: chown d17/d2b/c55 249 1 2024-04-22T23:35:03.278 INFO:tasks.workunit.client.0.smithi107.stdout:3/307: symlink db/d21/d27/d15/l68 0 2024-04-22T23:35:03.278 INFO:tasks.workunit.client.0.smithi107.stdout:3/308: write db/d21/d27/f23 [5037627,43053] 0 2024-04-22T23:35:03.278 INFO:tasks.workunit.client.0.smithi107.stdout:3/309: fsync db/d21/d27/d15/d29/d2a/f58 0 2024-04-22T23:35:03.278 INFO:tasks.workunit.client.0.smithi107.stdout:3/310: fsync db/ff 0 2024-04-22T23:35:03.278 INFO:tasks.workunit.client.0.smithi107.stdout:3/311: chown db/c17 205634230 1 2024-04-22T23:35:03.278 INFO:tasks.workunit.client.0.smithi107.stdout:3/312: rename db/d21/d27 to db/d21/d27/d15/d29/d2a/d69 22 2024-04-22T23:35:03.279 INFO:tasks.workunit.client.1.smithi157.stdout:9/476: dread d4/d15/d23/d27/f4f [4194304,4194304] 0 2024-04-22T23:35:03.280 INFO:tasks.workunit.client.1.smithi157.stdout:9/477: mkdir d4/d15/d55/d8d 0 2024-04-22T23:35:03.281 INFO:tasks.workunit.client.1.smithi157.stdout:9/478: creat d4/d2d/f8e x:0 0 0 2024-04-22T23:35:03.281 INFO:tasks.workunit.client.0.smithi107.stdout:7/235: mkdir d7/d1b/d51 0 2024-04-22T23:35:03.283 INFO:tasks.workunit.client.1.smithi157.stdout:7/408: dread d0/d9/d29/f4d [0,4194304] 0 2024-04-22T23:35:03.283 INFO:tasks.workunit.client.0.smithi107.stdout:1/275: dwrite d7/d19/d20/d39/d53/f5f [4194304,4194304] 0 2024-04-22T23:35:03.284 INFO:tasks.workunit.client.0.smithi107.stdout:3/313: symlink db/d21/d27/d15/d24/d52/d60/l6a 0 2024-04-22T23:35:03.285 INFO:tasks.workunit.client.0.smithi107.stdout:8/208: dwrite d0/d7/d20/f39 [0,4194304] 0 2024-04-22T23:35:03.285 INFO:tasks.workunit.client.0.smithi107.stdout:1/276: mkdir d7/d19/d20/d39/d53/d43/d61 0 2024-04-22T23:35:03.285 INFO:tasks.workunit.client.0.smithi107.stdout:1/277: stat d7/d19 0 2024-04-22T23:35:03.286 INFO:tasks.workunit.client.0.smithi107.stdout:1/278: write d7/d19/d20/d39/d53/f4e [667829,13372] 0 2024-04-22T23:35:03.287 INFO:tasks.workunit.client.1.smithi157.stdout:7/409: symlink d0/d9/d42/d55/d56/l89 0 2024-04-22T23:35:03.288 INFO:tasks.workunit.client.0.smithi107.stdout:4/273: dwrite d1/d8/d56/d27/f2f [0,4194304] 0 2024-04-22T23:35:03.288 INFO:tasks.workunit.client.1.smithi157.stdout:3/465: dread d0/d3/d34/d39/f49 [0,4194304] 0 2024-04-22T23:35:03.289 INFO:tasks.workunit.client.1.smithi157.stdout:3/466: dread - d0/d3/d13/d21/f7a zero size 2024-04-22T23:35:03.289 INFO:tasks.workunit.client.1.smithi157.stdout:3/467: fsync d0/d3/d34/d39/f4c 0 2024-04-22T23:35:03.289 INFO:tasks.workunit.client.0.smithi107.stdout:7/236: symlink d7/l52 0 2024-04-22T23:35:03.289 INFO:tasks.workunit.client.0.smithi107.stdout:7/237: truncate d7/d1b/f24 1123024 0 2024-04-22T23:35:03.290 INFO:tasks.workunit.client.1.smithi157.stdout:4/612: dwrite d3/d5c/fca [0,4194304] 0 2024-04-22T23:35:03.292 INFO:tasks.workunit.client.0.smithi107.stdout:6/263: sync 2024-04-22T23:35:03.292 INFO:tasks.workunit.client.0.smithi107.stdout:9/335: sync 2024-04-22T23:35:03.294 INFO:tasks.workunit.client.0.smithi107.stdout:8/209: symlink d0/d7/df/d3b/l43 0 2024-04-22T23:35:03.294 INFO:tasks.workunit.client.0.smithi107.stdout:5/269: sync 2024-04-22T23:35:03.298 INFO:tasks.workunit.client.0.smithi107.stdout:1/279: rename d7/d19/d2d/f35 to d7/d19/d20/d39/d53/f62 0 2024-04-22T23:35:03.298 INFO:tasks.workunit.client.0.smithi107.stdout:1/280: chown d7/d19/d20/d39/d53/f29 82430316 1 2024-04-22T23:35:03.298 INFO:tasks.workunit.client.0.smithi107.stdout:1/281: chown d7/f10 222 1 2024-04-22T23:35:03.298 INFO:tasks.workunit.client.0.smithi107.stdout:1/282: fdatasync d7/d19/f2c 0 2024-04-22T23:35:03.299 INFO:tasks.workunit.client.1.smithi157.stdout:3/468: rmdir d0/d3/d8d 0 2024-04-22T23:35:03.299 INFO:tasks.workunit.client.1.smithi157.stdout:3/469: fdatasync d0/d7/f7d 0 2024-04-22T23:35:03.304 INFO:tasks.workunit.client.0.smithi107.stdout:4/274: symlink d1/d8/l62 0 2024-04-22T23:35:03.305 INFO:tasks.workunit.client.0.smithi107.stdout:9/336: dread d5/d7/d17/d1b/f30 [0,4194304] 0 2024-04-22T23:35:03.305 INFO:tasks.workunit.client.1.smithi157.stdout:3/470: dread d0/d3/d34/d39/f49 [0,4194304] 0 2024-04-22T23:35:03.306 INFO:tasks.workunit.client.1.smithi157.stdout:6/734: dwrite d1/d2f/d54/fa5 [0,4194304] 0 2024-04-22T23:35:03.307 INFO:tasks.workunit.client.0.smithi107.stdout:8/210: unlink d0/f3 0 2024-04-22T23:35:03.309 INFO:tasks.workunit.client.1.smithi157.stdout:2/607: dwrite d7/d8/d25/f4f [0,4194304] 0 2024-04-22T23:35:03.309 INFO:tasks.workunit.client.1.smithi157.stdout:2/608: getdents d7/d8/d8e/d80 0 2024-04-22T23:35:03.309 INFO:tasks.workunit.client.0.smithi107.stdout:5/270: symlink d9/db/l57 0 2024-04-22T23:35:03.310 INFO:tasks.workunit.client.0.smithi107.stdout:1/283: mkdir d7/d19/d20/d39/d53/d43/d63 0 2024-04-22T23:35:03.311 INFO:tasks.workunit.client.0.smithi107.stdout:4/275: dread d1/d8/f3f [0,4194304] 0 2024-04-22T23:35:03.313 INFO:tasks.workunit.client.0.smithi107.stdout:1/284: dread d7/d19/d20/d39/d53/f62 [0,4194304] 0 2024-04-22T23:35:03.314 INFO:tasks.workunit.client.0.smithi107.stdout:8/211: unlink d0/fc 0 2024-04-22T23:35:03.314 INFO:tasks.workunit.client.0.smithi107.stdout:8/212: chown d0/f4 13613 1 2024-04-22T23:35:03.315 INFO:tasks.workunit.client.0.smithi107.stdout:4/276: mkdir d1/d7/d63 0 2024-04-22T23:35:03.316 INFO:tasks.workunit.client.0.smithi107.stdout:1/285: rmdir d7/d19/d20/d39/d53/d43/d46 39 2024-04-22T23:35:03.318 INFO:tasks.workunit.client.0.smithi107.stdout:8/213: symlink d0/d7/d8/l44 0 2024-04-22T23:35:03.320 INFO:tasks.workunit.client.0.smithi107.stdout:4/277: rename d1/d21/d29/f45 to d1/d1b/f64 0 2024-04-22T23:35:03.321 INFO:tasks.workunit.client.1.smithi157.stdout:9/479: getdents d4/d2d 0 2024-04-22T23:35:03.322 INFO:tasks.workunit.client.0.smithi107.stdout:1/286: unlink d7/d19/d42/c5b 0 2024-04-22T23:35:03.323 INFO:tasks.workunit.client.0.smithi107.stdout:8/214: creat d0/d3a/f45 x:0 0 0 2024-04-22T23:35:03.323 INFO:tasks.workunit.client.0.smithi107.stdout:8/215: chown d0/d7/df/d17 14047 1 2024-04-22T23:35:03.323 INFO:tasks.workunit.client.0.smithi107.stdout:8/216: truncate d0/d7/df/d3b/f3d 819246 0 2024-04-22T23:35:03.324 INFO:tasks.workunit.client.0.smithi107.stdout:4/278: getdents d1/d21/d29/d2e/d4d 0 2024-04-22T23:35:03.325 INFO:tasks.workunit.client.1.smithi157.stdout:2/609: mkdir d7/d8/d30/d23/d27/d57/dcf 0 2024-04-22T23:35:03.327 INFO:tasks.workunit.client.0.smithi107.stdout:1/287: creat d7/d19/d42/f64 x:0 0 0 2024-04-22T23:35:03.327 INFO:tasks.workunit.client.1.smithi157.stdout:9/480: unlink d4/d15/l43 0 2024-04-22T23:35:03.327 INFO:tasks.workunit.client.1.smithi157.stdout:2/610: stat d7/d8/d95/l65 0 2024-04-22T23:35:03.329 INFO:tasks.workunit.client.1.smithi157.stdout:6/735: rename d1/d2f/d3b/d39/dd3 to d1/d2f/d3b/d39/d4e/da8/dac/de6 0 2024-04-22T23:35:03.329 INFO:tasks.workunit.client.1.smithi157.stdout:6/736: creat d1/d2f/dc7/dce/fe7 x:0 0 0 2024-04-22T23:35:03.330 INFO:tasks.workunit.client.0.smithi107.stdout:8/217: mkdir d0/d3a/d46 0 2024-04-22T23:35:03.331 INFO:tasks.workunit.client.0.smithi107.stdout:3/314: rmdir db 39 2024-04-22T23:35:03.331 INFO:tasks.workunit.client.0.smithi107.stdout:3/315: chown db/d21 960426 1 2024-04-22T23:35:03.331 INFO:tasks.workunit.client.1.smithi157.stdout:9/481: rmdir d4/d15/d23/d67 39 2024-04-22T23:35:03.331 INFO:tasks.workunit.client.1.smithi157.stdout:2/611: dread d7/d8/d30/f76 [0,4194304] 0 2024-04-22T23:35:03.331 INFO:tasks.workunit.client.1.smithi157.stdout:2/612: creat d7/d8/d30/d23/d27/d57/fd0 x:0 0 0 2024-04-22T23:35:03.331 INFO:tasks.workunit.client.0.smithi107.stdout:1/288: symlink d7/d19/d20/d39/d53/d43/l65 0 2024-04-22T23:35:03.332 INFO:tasks.workunit.client.0.smithi107.stdout:8/218: symlink d0/d3a/d46/l47 0 2024-04-22T23:35:03.333 INFO:tasks.workunit.client.1.smithi157.stdout:9/482: unlink d4/df/f1f 0 2024-04-22T23:35:03.333 INFO:tasks.workunit.client.1.smithi157.stdout:9/483: chown d4/d15/d55/l6d 64461303 1 2024-04-22T23:35:03.333 INFO:tasks.workunit.client.1.smithi157.stdout:9/484: creat d4/d15/d23/f8f x:0 0 0 2024-04-22T23:35:03.333 INFO:tasks.workunit.client.0.smithi107.stdout:7/238: rmdir d7 39 2024-04-22T23:35:03.333 INFO:tasks.workunit.client.0.smithi107.stdout:7/239: write d7/ff [4863411,117475] 0 2024-04-22T23:35:03.334 INFO:tasks.workunit.client.0.smithi107.stdout:3/316: link db/d21/d27/d15/d29/d2a/f43 db/d21/d27/d15/d24/d52/d60/f6b 0 2024-04-22T23:35:03.336 INFO:tasks.workunit.client.0.smithi107.stdout:7/240: creat d7/d1b/f53 x:0 0 0 2024-04-22T23:35:03.336 INFO:tasks.workunit.client.1.smithi157.stdout:9/485: unlink d4/l2a 0 2024-04-22T23:35:03.336 INFO:tasks.workunit.client.1.smithi157.stdout:9/486: write d4/d15/d55/f76 [338243,49037] 0 2024-04-22T23:35:03.337 INFO:tasks.workunit.client.0.smithi107.stdout:1/289: getdents d7/d19/d20 0 2024-04-22T23:35:03.337 INFO:tasks.workunit.client.1.smithi157.stdout:9/487: mkdir d4/d15/d23/d90 0 2024-04-22T23:35:03.338 INFO:tasks.workunit.client.0.smithi107.stdout:7/241: mkdir d7/d21/d54 0 2024-04-22T23:35:03.339 INFO:tasks.workunit.client.1.smithi157.stdout:0/453: dwrite d17/d2b/d52/f53 [0,4194304] 0 2024-04-22T23:35:03.340 INFO:tasks.workunit.client.1.smithi157.stdout:6/737: dread d1/d2f/dc7/dce/fd2 [0,4194304] 0 2024-04-22T23:35:03.340 INFO:tasks.workunit.client.1.smithi157.stdout:6/738: write d1/d2f/f40 [4362011,86041] 0 2024-04-22T23:35:03.340 INFO:tasks.workunit.client.0.smithi107.stdout:6/264: dwrite db/f2d [0,4194304] 0 2024-04-22T23:35:03.340 INFO:tasks.workunit.client.0.smithi107.stdout:6/265: read db/f1e [2549509,25899] 0 2024-04-22T23:35:03.342 INFO:tasks.workunit.client.0.smithi107.stdout:6/266: link f6 db/d20/f4a 0 2024-04-22T23:35:03.343 INFO:tasks.workunit.client.1.smithi157.stdout:0/454: link d17/d23/d9c/c96 d17/d2b/d5b/c9f 0 2024-04-22T23:35:03.344 INFO:tasks.workunit.client.0.smithi107.stdout:6/267: mkdir db/d20/d2a/d38/d4b 0 2024-04-22T23:35:03.344 INFO:tasks.workunit.client.0.smithi107.stdout:6/268: write f6 [7989057,86752] 0 2024-04-22T23:35:03.344 INFO:tasks.workunit.client.1.smithi157.stdout:6/739: symlink d1/d2f/d3b/d49/le8 0 2024-04-22T23:35:03.344 INFO:tasks.workunit.client.1.smithi157.stdout:6/740: truncate d1/d2f/d3b/d39/d4e/d74/fb5 522648 0 2024-04-22T23:35:03.347 INFO:tasks.workunit.client.0.smithi107.stdout:3/317: dread db/d21/f33 [4194304,4194304] 0 2024-04-22T23:35:03.347 INFO:tasks.workunit.client.0.smithi107.stdout:3/318: chown db/ff 4065 1 2024-04-22T23:35:03.350 INFO:tasks.workunit.client.0.smithi107.stdout:3/319: creat db/d21/d27/d15/d29/d2a/d40/f6c x:0 0 0 2024-04-22T23:35:03.350 INFO:tasks.workunit.client.0.smithi107.stdout:3/320: fdatasync db/d21/d27/d15/d29/d2a/f5b 0 2024-04-22T23:35:03.350 INFO:tasks.workunit.client.0.smithi107.stdout:6/269: read db/f1c [1987311,17127] 0 2024-04-22T23:35:03.351 INFO:tasks.workunit.client.1.smithi157.stdout:6/741: read d1/d17/d5d/f64 [1604169,28123] 0 2024-04-22T23:35:03.352 INFO:tasks.workunit.client.0.smithi107.stdout:6/270: mknod db/d20/d2a/d38/c4c 0 2024-04-22T23:35:03.352 INFO:tasks.workunit.client.0.smithi107.stdout:6/271: truncate db/d20/d2c/d3b/f43 869065 0 2024-04-22T23:35:03.353 INFO:tasks.workunit.client.0.smithi107.stdout:6/272: stat db/f15 0 2024-04-22T23:35:03.356 INFO:tasks.workunit.client.0.smithi107.stdout:8/219: dread d0/d7/d20/f39 [0,4194304] 0 2024-04-22T23:35:03.356 INFO:tasks.workunit.client.0.smithi107.stdout:8/220: chown d0/d7/df/d18/f1f 67399092 1 2024-04-22T23:35:03.356 INFO:tasks.workunit.client.0.smithi107.stdout:8/221: creat d0/d7/df/d17/f48 x:0 0 0 2024-04-22T23:35:03.360 INFO:tasks.workunit.client.0.smithi107.stdout:8/222: rename d0/d3a/f45 to d0/d7/df/d33/d37/f49 0 2024-04-22T23:35:03.360 INFO:tasks.workunit.client.0.smithi107.stdout:8/223: creat d0/d3a/f4a x:0 0 0 2024-04-22T23:35:03.363 INFO:tasks.workunit.client.1.smithi157.stdout:2/613: write d7/fb1 [3586628,45846] 0 2024-04-22T23:35:03.363 INFO:tasks.workunit.client.1.smithi157.stdout:2/614: truncate d7/d8/d30/d5b/f73 561372 0 2024-04-22T23:35:03.364 INFO:tasks.workunit.client.1.smithi157.stdout:2/615: write d7/d1f/d3e/db4/fc4 [397839,86637] 0 2024-04-22T23:35:03.367 INFO:tasks.workunit.client.1.smithi157.stdout:2/616: mkdir d7/d8/d95/dd1 0 2024-04-22T23:35:03.367 INFO:tasks.workunit.client.1.smithi157.stdout:2/617: chown d7 61091 1 2024-04-22T23:35:03.367 INFO:tasks.workunit.client.1.smithi157.stdout:2/618: chown d7/d8/d25/d55/f6c 91 1 2024-04-22T23:35:03.367 INFO:tasks.workunit.client.1.smithi157.stdout:2/619: truncate d7/d8/d30/fc1 916575 0 2024-04-22T23:35:03.368 INFO:tasks.workunit.client.0.smithi107.stdout:9/337: dwrite d5/d7/d17/d1b/f48 [0,4194304] 0 2024-04-22T23:35:03.368 INFO:tasks.workunit.client.0.smithi107.stdout:9/338: dread - d5/d14/d2e/f3f zero size 2024-04-22T23:35:03.368 INFO:tasks.workunit.client.1.smithi157.stdout:7/410: dwrite d0/f22 [4194304,4194304] 0 2024-04-22T23:35:03.371 INFO:tasks.workunit.client.1.smithi157.stdout:8/338: sync 2024-04-22T23:35:03.374 INFO:tasks.workunit.client.1.smithi157.stdout:4/613: dwrite d3/d11/d34/d9d/fc9 [0,4194304] 0 2024-04-22T23:35:03.375 INFO:tasks.workunit.client.0.smithi107.stdout:8/224: dread d0/d7/d8/f1e [0,4194304] 0 2024-04-22T23:35:03.376 INFO:tasks.workunit.client.0.smithi107.stdout:8/225: dread d0/d7/d20/d2d/f2c [0,4194304] 0 2024-04-22T23:35:03.379 INFO:tasks.workunit.client.0.smithi107.stdout:8/226: rmdir d0/d7/d20 39 2024-04-22T23:35:03.380 INFO:tasks.workunit.client.1.smithi157.stdout:4/614: creat d3/d11/d1d/d73/d80/dac/fcf x:0 0 0 2024-04-22T23:35:03.381 INFO:tasks.workunit.client.1.smithi157.stdout:7/411: write d0/d9/db/f5a [6372033,4610] 0 2024-04-22T23:35:03.381 INFO:tasks.workunit.client.1.smithi157.stdout:4/615: mknod d3/d11/d1d/d73/da3/d31/d91/dcb/cd0 0 2024-04-22T23:35:03.382 INFO:tasks.workunit.client.0.smithi107.stdout:2/312: sync 2024-04-22T23:35:03.383 INFO:tasks.workunit.client.1.smithi157.stdout:7/412: unlink d0/f3e 0 2024-04-22T23:35:03.385 INFO:tasks.workunit.client.1.smithi157.stdout:4/616: rmdir d3/d11/dc1 0 2024-04-22T23:35:03.385 INFO:tasks.workunit.client.0.smithi107.stdout:5/271: dwrite d9/db/f27 [0,4194304] 0 2024-04-22T23:35:03.386 INFO:tasks.workunit.client.0.smithi107.stdout:5/272: readlink d9/d26/l3e 0 2024-04-22T23:35:03.386 INFO:tasks.workunit.client.0.smithi107.stdout:5/273: chown d9/d26/f3b 27619192 1 2024-04-22T23:35:03.386 INFO:tasks.workunit.client.0.smithi107.stdout:5/274: stat d9/c47 0 2024-04-22T23:35:03.386 INFO:tasks.workunit.client.0.smithi107.stdout:5/275: chown d9/db/de/c16 148 1 2024-04-22T23:35:03.386 INFO:tasks.workunit.client.0.smithi107.stdout:5/276: stat d9/f29 0 2024-04-22T23:35:03.386 INFO:tasks.workunit.client.0.smithi107.stdout:5/277: dread - d9/db/f3c zero size 2024-04-22T23:35:03.386 INFO:tasks.workunit.client.0.smithi107.stdout:5/278: creat d9/f58 x:0 0 0 2024-04-22T23:35:03.387 INFO:tasks.workunit.client.1.smithi157.stdout:3/471: dwrite d0/d3/d34/d39/f18 [0,4194304] 0 2024-04-22T23:35:03.387 INFO:tasks.workunit.client.1.smithi157.stdout:7/413: link d0/c14 d0/d9/d42/c8a 0 2024-04-22T23:35:03.387 INFO:tasks.workunit.client.1.smithi157.stdout:7/414: chown d0/d9/d42/d55/d56 442150 1 2024-04-22T23:35:03.387 INFO:tasks.workunit.client.1.smithi157.stdout:4/617: rmdir d3 39 2024-04-22T23:35:03.389 INFO:tasks.workunit.client.0.smithi107.stdout:8/227: dread d0/d7/fa [0,4194304] 0 2024-04-22T23:35:03.390 INFO:tasks.workunit.client.1.smithi157.stdout:9/488: dwrite d4/d15/d23/d27/d47/f74 [0,4194304] 0 2024-04-22T23:35:03.390 INFO:tasks.workunit.client.0.smithi107.stdout:4/279: dwrite d1/f13 [4194304,4194304] 0 2024-04-22T23:35:03.390 INFO:tasks.workunit.client.0.smithi107.stdout:4/280: dread - d1/d1b/d44/f58 zero size 2024-04-22T23:35:03.391 INFO:tasks.workunit.client.0.smithi107.stdout:5/279: link d9/db/c4d d9/db/de/c59 0 2024-04-22T23:35:03.393 INFO:tasks.workunit.client.0.smithi107.stdout:7/242: dwrite d7/d2d/f3c [0,4194304] 0 2024-04-22T23:35:03.396 INFO:tasks.workunit.client.1.smithi157.stdout:2/620: dwrite d7/d8/d30/f76 [0,4194304] 0 2024-04-22T23:35:03.396 INFO:tasks.workunit.client.1.smithi157.stdout:6/742: dwrite d1/d2f/d3b/d39/d4e/da8/dac/dbe/fcc [0,4194304] 0 2024-04-22T23:35:03.396 INFO:tasks.workunit.client.1.smithi157.stdout:6/743: write d1/d2f/d3b/f3e [1249195,79858] 0 2024-04-22T23:35:03.396 INFO:tasks.workunit.client.1.smithi157.stdout:6/744: write d1/d2f/d3b/d39/d4e/f52 [287906,60128] 0 2024-04-22T23:35:03.396 INFO:tasks.workunit.client.1.smithi157.stdout:6/745: write d1/d2f/d54/f7e [1504960,27285] 0 2024-04-22T23:35:03.396 INFO:tasks.workunit.client.1.smithi157.stdout:6/746: readlink d1/l23 0 2024-04-22T23:35:03.397 INFO:tasks.workunit.client.0.smithi107.stdout:2/313: dread d1/f11 [0,4194304] 0 2024-04-22T23:35:03.397 INFO:tasks.workunit.client.1.smithi157.stdout:3/472: mknod d0/d3/d34/d3d/d57/d60/d6b/c91 0 2024-04-22T23:35:03.398 INFO:tasks.workunit.client.1.smithi157.stdout:7/415: mknod d0/d9/d42/d55/d80/c8b 0 2024-04-22T23:35:03.398 INFO:tasks.workunit.client.0.smithi107.stdout:4/281: truncate d1/d21/d29/d2e/f46 3547617 0 2024-04-22T23:35:03.400 INFO:tasks.workunit.client.0.smithi107.stdout:9/339: dwrite d5/d14/d2e/f3f [0,4194304] 0 2024-04-22T23:35:03.400 INFO:tasks.workunit.client.0.smithi107.stdout:9/340: chown d5/d14/d2e/d4f/l63 1586656 1 2024-04-22T23:35:03.400 INFO:tasks.workunit.client.0.smithi107.stdout:6/273: dwrite db/f1c [0,4194304] 0 2024-04-22T23:35:03.401 INFO:tasks.workunit.client.1.smithi157.stdout:9/489: getdents d4/d15/d23/d3f 0 2024-04-22T23:35:03.402 INFO:tasks.workunit.client.1.smithi157.stdout:8/339: dwrite d2/d4/f52 [0,4194304] 0 2024-04-22T23:35:03.402 INFO:tasks.workunit.client.1.smithi157.stdout:8/340: write d2/d6/d21/d5f/d11/f5a [450218,104127] 0 2024-04-22T23:35:03.402 INFO:tasks.workunit.client.1.smithi157.stdout:8/341: creat d2/d34/d6a/f76 x:0 0 0 2024-04-22T23:35:03.403 INFO:tasks.workunit.client.0.smithi107.stdout:5/280: dread d9/db/de/f1f [0,4194304] 0 2024-04-22T23:35:03.403 INFO:tasks.workunit.client.0.smithi107.stdout:5/281: write d9/f31 [926901,21304] 0 2024-04-22T23:35:03.404 INFO:tasks.workunit.client.0.smithi107.stdout:2/314: symlink d1/d12/d35/d54/l72 0 2024-04-22T23:35:03.404 INFO:tasks.workunit.client.0.smithi107.stdout:2/315: stat d1/f3c 0 2024-04-22T23:35:03.404 INFO:tasks.workunit.client.0.smithi107.stdout:2/316: fsync d1/d2b/d49/f5a 0 2024-04-22T23:35:03.404 INFO:tasks.workunit.client.0.smithi107.stdout:2/317: readlink d1/l58 0 2024-04-22T23:35:03.404 INFO:tasks.workunit.client.0.smithi107.stdout:7/243: dread d7/ff [4194304,4194304] 0 2024-04-22T23:35:03.406 INFO:tasks.workunit.client.0.smithi107.stdout:4/282: link d1/d7/d5b/c5c d1/d21/d29/c65 0 2024-04-22T23:35:03.407 INFO:tasks.workunit.client.1.smithi157.stdout:3/473: dread d0/d3/f1d [0,4194304] 0 2024-04-22T23:35:03.408 INFO:tasks.workunit.client.1.smithi157.stdout:7/416: dread d0/d9/d29/d33/d37/f4c [0,4194304] 0 2024-04-22T23:35:03.408 INFO:tasks.workunit.client.1.smithi157.stdout:7/417: dread - d0/d9/f7d zero size 2024-04-22T23:35:03.409 INFO:tasks.workunit.client.0.smithi107.stdout:2/318: dread d1/d12/d35/d48/d13/d4f/f62 [0,4194304] 0 2024-04-22T23:35:03.410 INFO:tasks.workunit.client.0.smithi107.stdout:8/228: dwrite d0/d7/df/d17/f2a [0,4194304] 0 2024-04-22T23:35:03.411 INFO:tasks.workunit.client.1.smithi157.stdout:2/621: mkdir d7/d8/d95/dd2 0 2024-04-22T23:35:03.411 INFO:tasks.workunit.client.1.smithi157.stdout:2/622: chown d7/d8/d8e/d94/cbd 6048127 1 2024-04-22T23:35:03.411 INFO:tasks.workunit.client.0.smithi107.stdout:9/341: rename d5/d7/fb to d5/d7/d27/d46/f74 0 2024-04-22T23:35:03.411 INFO:tasks.workunit.client.0.smithi107.stdout:9/342: truncate d5/d7/f70 740351 0 2024-04-22T23:35:03.412 INFO:tasks.workunit.client.0.smithi107.stdout:1/290: rmdir d7/d19/d20/d39/d53 39 2024-04-22T23:35:03.412 INFO:tasks.workunit.client.1.smithi157.stdout:6/747: creat d1/d2f/d3b/d39/d4e/da8/fe9 x:0 0 0 2024-04-22T23:35:03.415 INFO:tasks.workunit.client.0.smithi107.stdout:5/282: unlink d9/db/de/f23 0 2024-04-22T23:35:03.415 INFO:tasks.workunit.client.1.smithi157.stdout:9/490: rename d4/d15/d23/d3f/f45 to d4/d15/f91 0 2024-04-22T23:35:03.417 INFO:tasks.workunit.client.0.smithi107.stdout:7/244: chown d7/l27 37908 1 2024-04-22T23:35:03.424 INFO:tasks.workunit.client.0.smithi107.stdout:2/319: unlink d1/d2b/d49/l66 0 2024-04-22T23:35:03.424 INFO:tasks.workunit.client.0.smithi107.stdout:8/229: creat d0/d7/df/f4b x:0 0 0 2024-04-22T23:35:03.424 INFO:tasks.workunit.client.0.smithi107.stdout:2/320: write d1/d12/d35/d48/d13/f32 [2001926,124543] 0 2024-04-22T23:35:03.426 INFO:tasks.workunit.client.1.smithi157.stdout:7/418: dread d0/d9/d42/f51 [0,4194304] 0 2024-04-22T23:35:03.426 INFO:tasks.workunit.client.1.smithi157.stdout:7/419: creat d0/d9/d42/f8c x:0 0 0 2024-04-22T23:35:03.426 INFO:tasks.workunit.client.0.smithi107.stdout:2/321: dread d1/d12/d35/d48/f10 [0,4194304] 0 2024-04-22T23:35:03.426 INFO:tasks.workunit.client.0.smithi107.stdout:2/322: fdatasync d1/d12/d35/d54/f5f 0 2024-04-22T23:35:03.429 INFO:tasks.workunit.client.1.smithi157.stdout:3/474: mknod d0/d3/d13/d21/c92 0 2024-04-22T23:35:03.429 INFO:tasks.workunit.client.1.smithi157.stdout:3/475: dread - d0/d7/d6c/f6f zero size 2024-04-22T23:35:03.429 INFO:tasks.workunit.client.0.smithi107.stdout:6/274: rename db/d20/d2a/f2e to db/d20/d2c/d3b/f4d 0 2024-04-22T23:35:03.432 INFO:tasks.workunit.client.0.smithi107.stdout:1/291: unlink d7/d19/d20/d39/d53/c57 0 2024-04-22T23:35:03.433 INFO:tasks.workunit.client.0.smithi107.stdout:8/230: mknod d0/d3a/d46/c4c 0 2024-04-22T23:35:03.435 INFO:tasks.workunit.client.0.smithi107.stdout:5/283: rename d9/f45 to d9/db/d42/f5a 0 2024-04-22T23:35:03.437 INFO:tasks.workunit.client.1.smithi157.stdout:3/476: creat d0/d3/d34/f93 x:0 0 0 2024-04-22T23:35:03.437 INFO:tasks.workunit.client.1.smithi157.stdout:3/477: chown d0/d3/d34/d39/f4c 10 1 2024-04-22T23:35:03.438 INFO:tasks.workunit.client.1.smithi157.stdout:3/478: truncate d0/d3/d34/d47/f70 204818 0 2024-04-22T23:35:03.438 INFO:tasks.workunit.client.0.smithi107.stdout:6/275: truncate db/f1e 1626930 0 2024-04-22T23:35:03.438 INFO:tasks.workunit.client.0.smithi107.stdout:6/276: write db/d20/d2c/f32 [1233582,22523] 0 2024-04-22T23:35:03.438 INFO:tasks.workunit.client.0.smithi107.stdout:6/277: truncate f6 10239513 0 2024-04-22T23:35:03.438 INFO:tasks.workunit.client.1.smithi157.stdout:9/491: mknod d4/d2d/c92 0 2024-04-22T23:35:03.439 INFO:tasks.workunit.client.0.smithi107.stdout:1/292: creat d7/d19/d54/f66 x:0 0 0 2024-04-22T23:35:03.439 INFO:tasks.workunit.client.0.smithi107.stdout:1/293: write d7/f1b [4097805,21985] 0 2024-04-22T23:35:03.440 INFO:tasks.workunit.client.1.smithi157.stdout:9/492: truncate d4/d15/f91 43845 0 2024-04-22T23:35:03.441 INFO:tasks.workunit.client.0.smithi107.stdout:8/231: link d0/d3a/d46/c4c d0/d7/df/d33/c4d 0 2024-04-22T23:35:03.441 INFO:tasks.workunit.client.1.smithi157.stdout:7/420: write d0/d9/d29/f3b [3173302,119401] 0 2024-04-22T23:35:03.441 INFO:tasks.workunit.client.1.smithi157.stdout:7/421: chown d0/d9/f3c 1 1 2024-04-22T23:35:03.441 INFO:tasks.workunit.client.1.smithi157.stdout:5/493: sync 2024-04-22T23:35:03.442 INFO:tasks.workunit.client.1.smithi157.stdout:9/493: chown d4/df/d61/f84 17724 1 2024-04-22T23:35:03.442 INFO:tasks.workunit.client.1.smithi157.stdout:9/494: chown d4/d15/d23/d27/d47 932878 1 2024-04-22T23:35:03.442 INFO:tasks.workunit.client.0.smithi107.stdout:5/284: truncate d9/d26/f32 10901 0 2024-04-22T23:35:03.442 INFO:tasks.workunit.client.1.smithi157.stdout:7/422: dread d0/f45 [0,4194304] 0 2024-04-22T23:35:03.445 INFO:tasks.workunit.client.0.smithi107.stdout:2/323: write d1/d2b/f6e [2545031,31746] 0 2024-04-22T23:35:03.445 INFO:tasks.workunit.client.0.smithi107.stdout:9/343: dwrite d5/d7/d17/d1b/d1f/d36/d53/f5b [0,4194304] 0 2024-04-22T23:35:03.445 INFO:tasks.workunit.client.0.smithi107.stdout:2/324: read - d1/d2b/d3d/f6c zero size 2024-04-22T23:35:03.446 INFO:tasks.workunit.client.1.smithi157.stdout:5/494: mknod d1/d5/d7/d2f/d45/cbe 0 2024-04-22T23:35:03.446 INFO:tasks.workunit.client.1.smithi157.stdout:5/495: stat d1/d5/d7/d2f/d51 0 2024-04-22T23:35:03.446 INFO:tasks.workunit.client.1.smithi157.stdout:5/496: read - d1/d5/f4e zero size 2024-04-22T23:35:03.446 INFO:tasks.workunit.client.0.smithi107.stdout:6/278: rename db/d20/c23 to db/d20/d2c/d3b/c4e 0 2024-04-22T23:35:03.448 INFO:tasks.workunit.client.1.smithi157.stdout:7/423: symlink d0/d9/d29/d5e/d85/l8d 0 2024-04-22T23:35:03.448 INFO:tasks.workunit.client.1.smithi157.stdout:7/424: chown d0/d9/db/f74 7813259 1 2024-04-22T23:35:03.449 INFO:tasks.workunit.client.0.smithi107.stdout:1/294: unlink d7/d19/f26 0 2024-04-22T23:35:03.449 INFO:tasks.workunit.client.0.smithi107.stdout:1/295: chown d7/d19/d20/d39/d53/f29 213572 1 2024-04-22T23:35:03.449 INFO:tasks.workunit.client.0.smithi107.stdout:1/296: chown d7/d19/c28 0 1 2024-04-22T23:35:03.455 INFO:tasks.workunit.client.1.smithi157.stdout:6/748: dwrite d1/d2f/d3b/d39/fe1 [0,4194304] 0 2024-04-22T23:35:03.456 INFO:tasks.workunit.client.1.smithi157.stdout:6/749: fdatasync d1/d17/f43 0 2024-04-22T23:35:03.456 INFO:tasks.workunit.client.1.smithi157.stdout:6/750: chown d1/d38/d97 0 1 2024-04-22T23:35:03.456 INFO:tasks.workunit.client.1.smithi157.stdout:6/751: chown d1/d2f/d3b/d39/d4e/da8/de5/d83/f9f 892 1 2024-04-22T23:35:03.456 INFO:tasks.workunit.client.0.smithi107.stdout:7/245: dwrite d7/d1b/f24 [0,4194304] 0 2024-04-22T23:35:03.456 INFO:tasks.workunit.client.0.smithi107.stdout:7/246: write d7/f26 [2570247,62981] 0 2024-04-22T23:35:03.457 INFO:tasks.workunit.client.1.smithi157.stdout:5/497: getdents d1/d5/d7/d2f/d45/da4 0 2024-04-22T23:35:03.464 INFO:tasks.workunit.client.0.smithi107.stdout:1/297: write d7/d19/d20/d39/d53/f29 [1722394,3260] 0 2024-04-22T23:35:03.466 INFO:tasks.workunit.client.1.smithi157.stdout:6/752: mkdir d1/d36/dea 0 2024-04-22T23:35:03.466 INFO:tasks.workunit.client.1.smithi157.stdout:7/425: dread d0/d9/d72/f44 [0,4194304] 0 2024-04-22T23:35:03.469 INFO:tasks.workunit.client.1.smithi157.stdout:1/532: sync 2024-04-22T23:35:03.471 INFO:tasks.workunit.client.1.smithi157.stdout:7/426: creat d0/d9/db/df/d68/f8e x:0 0 0 2024-04-22T23:35:03.472 INFO:tasks.workunit.client.1.smithi157.stdout:1/533: rename d4/d5f/l62 to d4/db/lb5 0 2024-04-22T23:35:03.473 INFO:tasks.workunit.client.1.smithi157.stdout:1/534: fsync d4/d17/d34/d3b/d52/d65/dae/f99 0 2024-04-22T23:35:03.473 INFO:tasks.workunit.client.1.smithi157.stdout:8/342: rmdir d2/d34 39 2024-04-22T23:35:03.476 INFO:tasks.workunit.client.1.smithi157.stdout:1/535: getdents d4/d17/d1d/d33 0 2024-04-22T23:35:03.476 INFO:tasks.workunit.client.1.smithi157.stdout:1/536: stat d4/d17/d34/f38 0 2024-04-22T23:35:03.476 INFO:tasks.workunit.client.1.smithi157.stdout:1/537: write d4/d77/f93 [145563,79724] 0 2024-04-22T23:35:03.476 INFO:tasks.workunit.client.1.smithi157.stdout:1/538: chown d4/db/d71/c7c 27936 1 2024-04-22T23:35:03.476 INFO:tasks.workunit.client.1.smithi157.stdout:2/623: dwrite d7/d8/d30/d23/fbe [0,4194304] 0 2024-04-22T23:35:03.476 INFO:tasks.workunit.client.1.smithi157.stdout:1/539: creat d4/d17/d34/d3b/d50/fb6 x:0 0 0 2024-04-22T23:35:03.476 INFO:tasks.workunit.client.1.smithi157.stdout:1/540: chown d4/d17/d1d/d33/d41/f67 2 1 2024-04-22T23:35:03.477 INFO:tasks.workunit.client.0.smithi107.stdout:1/298: creat d7/f67 x:0 0 0 2024-04-22T23:35:03.478 INFO:tasks.workunit.client.0.smithi107.stdout:5/285: getdents d9/db/de 0 2024-04-22T23:35:03.480 INFO:tasks.workunit.client.0.smithi107.stdout:7/247: read d7/d1b/f4d [3935469,103852] 0 2024-04-22T23:35:03.480 INFO:tasks.workunit.client.0.smithi107.stdout:7/248: read - d7/f2a zero size 2024-04-22T23:35:03.482 INFO:tasks.workunit.client.1.smithi157.stdout:7/427: dread d0/f8 [0,4194304] 0 2024-04-22T23:35:03.482 INFO:tasks.workunit.client.1.smithi157.stdout:2/624: rename d7/d8/d30/d5b/d8a/c9e to d7/d8/d30/d23/d27/d57/dcf/cd3 0 2024-04-22T23:35:03.482 INFO:tasks.workunit.client.1.smithi157.stdout:2/625: fdatasync d7/d8/d30/d23/d27/f7b 0 2024-04-22T23:35:03.482 INFO:tasks.workunit.client.1.smithi157.stdout:1/541: mknod d4/d17/d1d/cb7 0 2024-04-22T23:35:03.482 INFO:tasks.workunit.client.1.smithi157.stdout:1/542: write d4/f26 [406592,88753] 0 2024-04-22T23:35:03.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:35:03 smithi107 ceph-mon[26355]: pgmap v258: 65 pgs: 65 active+clean; 1.2 GiB data, 5.0 GiB used, 531 GiB / 536 GiB avail; 133 MiB/s rd, 165 MiB/s wr, 679 op/s 2024-04-22T23:35:03.485 INFO:tasks.workunit.client.0.smithi107.stdout:5/286: truncate d9/db/de/f49 3301813 0 2024-04-22T23:35:03.485 INFO:tasks.workunit.client.0.smithi107.stdout:5/287: chown d9/f31 1506 1 2024-04-22T23:35:03.486 INFO:tasks.workunit.client.1.smithi157.stdout:2/626: creat d7/d8/dd/fd4 x:0 0 0 2024-04-22T23:35:03.488 INFO:tasks.workunit.client.0.smithi107.stdout:8/232: dwrite d0/d7/df/d3b/f3d [0,4194304] 0 2024-04-22T23:35:03.488 INFO:tasks.workunit.client.0.smithi107.stdout:8/233: readlink d0/l12 0 2024-04-22T23:35:03.488 INFO:tasks.workunit.client.0.smithi107.stdout:8/234: fsync d0/d7/df/d17/f48 0 2024-04-22T23:35:03.489 INFO:tasks.workunit.client.1.smithi157.stdout:1/543: mkdir d4/d5f/d7e/db8 0 2024-04-22T23:35:03.489 INFO:tasks.workunit.client.0.smithi107.stdout:7/249: creat d7/d21/d48/f55 x:0 0 0 2024-04-22T23:35:03.490 INFO:tasks.workunit.client.0.smithi107.stdout:7/250: write d7/f26 [421095,35301] 0 2024-04-22T23:35:03.492 INFO:tasks.workunit.client.0.smithi107.stdout:9/344: dwrite d5/d7/f70 [0,4194304] 0 2024-04-22T23:35:03.493 INFO:tasks.workunit.client.1.smithi157.stdout:3/479: dwrite d0/d3/f1e [0,4194304] 0 2024-04-22T23:35:03.495 INFO:tasks.workunit.client.1.smithi157.stdout:3/480: creat d0/d3/d34/d3d/d5b/f94 x:0 0 0 2024-04-22T23:35:03.500 INFO:tasks.workunit.client.0.smithi107.stdout:1/299: rmdir d7/d19/d54 39 2024-04-22T23:35:03.501 INFO:tasks.workunit.client.1.smithi157.stdout:3/481: rmdir d0/d3/d34/d67 0 2024-04-22T23:35:03.501 INFO:tasks.workunit.client.1.smithi157.stdout:3/482: chown d0/d7/d6c/f6f 3 1 2024-04-22T23:35:03.503 INFO:tasks.workunit.client.1.smithi157.stdout:3/483: mkdir d0/d7/d95 0 2024-04-22T23:35:03.504 INFO:tasks.workunit.client.1.smithi157.stdout:9/495: dwrite d4/d15/d23/d27/f38 [0,4194304] 0 2024-04-22T23:35:03.505 INFO:tasks.workunit.client.0.smithi107.stdout:6/279: dwrite db/d20/d2a/f42 [0,4194304] 0 2024-04-22T23:35:03.505 INFO:tasks.workunit.client.0.smithi107.stdout:1/300: mkdir d7/d19/d20/d39/d53/d43/d46/d68 0 2024-04-22T23:35:03.505 INFO:tasks.workunit.client.0.smithi107.stdout:1/301: creat d7/d19/d20/d39/f69 x:0 0 0 2024-04-22T23:35:03.506 INFO:tasks.workunit.client.1.smithi157.stdout:3/484: rmdir d0/d3/d34/d47 39 2024-04-22T23:35:03.506 INFO:tasks.workunit.client.0.smithi107.stdout:2/325: dwrite d1/d12/d35/f3b [0,4194304] 0 2024-04-22T23:35:03.506 INFO:tasks.workunit.client.0.smithi107.stdout:2/326: creat d1/d12/d35/d54/f73 x:0 0 0 2024-04-22T23:35:03.506 INFO:tasks.workunit.client.0.smithi107.stdout:2/327: creat d1/d12/d35/d48/d13/f74 x:0 0 0 2024-04-22T23:35:03.507 INFO:tasks.workunit.client.0.smithi107.stdout:2/328: write d1/d12/f26 [1929755,66666] 0 2024-04-22T23:35:03.507 INFO:tasks.workunit.client.0.smithi107.stdout:2/329: read - d1/d2b/d3d/d57/f5c zero size 2024-04-22T23:35:03.507 INFO:tasks.workunit.client.0.smithi107.stdout:2/330: truncate d1/d12/d35/d48/d13/f68 896 0 2024-04-22T23:35:03.507 INFO:tasks.workunit.client.0.smithi107.stdout:2/331: write d1/d2b/f6e [1634815,66008] 0 2024-04-22T23:35:03.509 INFO:tasks.workunit.client.0.smithi107.stdout:6/280: creat db/d20/f4f x:0 0 0 2024-04-22T23:35:03.512 INFO:tasks.workunit.client.1.smithi157.stdout:9/496: creat d4/d15/d23/d67/f93 x:0 0 0 2024-04-22T23:35:03.512 INFO:tasks.workunit.client.1.smithi157.stdout:9/497: chown d4/d15/d23/d27/d75/f82 0 1 2024-04-22T23:35:03.512 INFO:tasks.workunit.client.1.smithi157.stdout:9/498: chown d4/d15/d23/d90 1279 1 2024-04-22T23:35:03.513 INFO:tasks.workunit.client.0.smithi107.stdout:1/302: mknod d7/d19/d20/d39/d53/d43/d46/d5c/c6a 0 2024-04-22T23:35:03.515 INFO:tasks.workunit.client.1.smithi157.stdout:9/499: symlink d4/df/l94 0 2024-04-22T23:35:03.516 INFO:tasks.workunit.client.1.smithi157.stdout:9/500: chown d4/d15/l70 606057 1 2024-04-22T23:35:03.516 INFO:tasks.workunit.client.1.smithi157.stdout:9/501: dread - d4/d15/d23/f88 zero size 2024-04-22T23:35:03.516 INFO:tasks.workunit.client.1.smithi157.stdout:9/502: creat d4/d15/d23/d67/f95 x:0 0 0 2024-04-22T23:35:03.516 INFO:tasks.workunit.client.1.smithi157.stdout:9/503: chown d4/df/f12 42397729 1 2024-04-22T23:35:03.516 INFO:tasks.workunit.client.1.smithi157.stdout:9/504: chown d4/d2d/f78 13630 1 2024-04-22T23:35:03.516 INFO:tasks.workunit.client.1.smithi157.stdout:3/485: dread d0/d3/f1f [0,4194304] 0 2024-04-22T23:35:03.519 INFO:tasks.workunit.client.0.smithi107.stdout:2/332: symlink d1/d12/d35/d48/l75 0 2024-04-22T23:35:03.519 INFO:tasks.workunit.client.0.smithi107.stdout:2/333: chown d1/d12/f46 7407 1 2024-04-22T23:35:03.519 INFO:tasks.workunit.client.0.smithi107.stdout:2/334: write d1/d2b/d3d/f6c [194284,129667] 0 2024-04-22T23:35:03.519 INFO:tasks.workunit.client.0.smithi107.stdout:2/335: write d1/d2b/f6e [3726798,118142] 0 2024-04-22T23:35:03.520 INFO:tasks.workunit.client.0.smithi107.stdout:6/281: symlink db/d20/d2c/l50 0 2024-04-22T23:35:03.520 INFO:tasks.workunit.client.0.smithi107.stdout:0/306: sync 2024-04-22T23:35:03.521 INFO:tasks.workunit.client.1.smithi157.stdout:3/486: dread d0/d3/d34/d3d/f63 [0,4194304] 0 2024-04-22T23:35:03.521 INFO:tasks.workunit.client.1.smithi157.stdout:3/487: rename d0/d3/d13/d21/f4e to d0/d3/d34/d39/f96 0 2024-04-22T23:35:03.522 INFO:tasks.workunit.client.0.smithi107.stdout:2/336: link d1/lf d1/d12/d35/d48/d13/l76 0 2024-04-22T23:35:03.522 INFO:tasks.workunit.client.1.smithi157.stdout:3/488: mkdir d0/d3/d34/d3d/d57/d60/d6b/d97 0 2024-04-22T23:35:03.523 INFO:tasks.workunit.client.1.smithi157.stdout:3/489: creat d0/d3/d34/d62/f98 x:0 0 0 2024-04-22T23:35:03.524 INFO:tasks.workunit.client.0.smithi107.stdout:0/307: symlink d1/d27/d20/l74 0 2024-04-22T23:35:03.524 INFO:tasks.workunit.client.0.smithi107.stdout:0/308: dread - d1/d27/d20/d2c/d41/d52/f73 zero size 2024-04-22T23:35:03.524 INFO:tasks.workunit.client.0.smithi107.stdout:0/309: fsync d1/d9/d14/d4f/d59/f5c 0 2024-04-22T23:35:03.525 INFO:tasks.workunit.client.0.smithi107.stdout:2/337: symlink d1/d12/d35/l77 0 2024-04-22T23:35:03.525 INFO:tasks.workunit.client.0.smithi107.stdout:2/338: chown d1/d12/d35/d48/d13/l28 79 1 2024-04-22T23:35:03.526 INFO:tasks.workunit.client.1.smithi157.stdout:3/490: mkdir d0/d3/d34/d3d/d5b/d72/d99 0 2024-04-22T23:35:03.526 INFO:tasks.workunit.client.1.smithi157.stdout:9/505: dread d4/f8 [0,4194304] 0 2024-04-22T23:35:03.526 INFO:tasks.workunit.client.0.smithi107.stdout:2/339: link d1/d2b/d59/f6b d1/f78 0 2024-04-22T23:35:03.528 INFO:tasks.workunit.client.0.smithi107.stdout:2/340: mkdir d1/d12/d35/d79 0 2024-04-22T23:35:03.530 INFO:tasks.workunit.client.0.smithi107.stdout:2/341: write d1/d12/d35/d48/f29 [279397,43227] 0 2024-04-22T23:35:03.531 INFO:tasks.workunit.client.1.smithi157.stdout:5/498: dwrite d1/d27/d29/d4b/fae [0,4194304] 0 2024-04-22T23:35:03.535 INFO:tasks.workunit.client.0.smithi107.stdout:2/342: rmdir d1/d12/d35/d48 39 2024-04-22T23:35:03.535 INFO:tasks.workunit.client.0.smithi107.stdout:2/343: creat d1/d12/d35/f7a x:0 0 0 2024-04-22T23:35:03.535 INFO:tasks.workunit.client.0.smithi107.stdout:2/344: fsync d1/d12/d35/d54/f5f 0 2024-04-22T23:35:03.535 INFO:tasks.workunit.client.0.smithi107.stdout:2/345: creat d1/d67/f7b x:0 0 0 2024-04-22T23:35:03.535 INFO:tasks.workunit.client.0.smithi107.stdout:2/346: readlink d1/d2b/d49/l56 0 2024-04-22T23:35:03.537 INFO:tasks.workunit.client.1.smithi157.stdout:9/506: read d4/d15/d23/d27/f87 [1089394,100909] 0 2024-04-22T23:35:03.538 INFO:tasks.workunit.client.0.smithi107.stdout:2/347: creat d1/f7c x:0 0 0 2024-04-22T23:35:03.539 INFO:tasks.workunit.client.1.smithi157.stdout:9/507: getdents d4/df/d72 0 2024-04-22T23:35:03.540 INFO:tasks.workunit.client.0.smithi107.stdout:0/310: dread d1/d27/d20/d2c/d2e/f5a [0,4194304] 0 2024-04-22T23:35:03.540 INFO:tasks.workunit.client.0.smithi107.stdout:0/311: readlink d1/d27/d20/d2c/d41/l42 0 2024-04-22T23:35:03.540 INFO:tasks.workunit.client.1.smithi157.stdout:9/508: mknod d4/df/d80/c96 0 2024-04-22T23:35:03.540 INFO:tasks.workunit.client.1.smithi157.stdout:9/509: stat d4/d15/d23/d90 0 2024-04-22T23:35:03.542 INFO:tasks.workunit.client.1.smithi157.stdout:9/510: unlink d4/d15/d55/l6d 0 2024-04-22T23:35:03.542 INFO:tasks.workunit.client.1.smithi157.stdout:9/511: read - d4/d2d/f8e zero size 2024-04-22T23:35:03.542 INFO:tasks.workunit.client.1.smithi157.stdout:9/512: creat d4/d15/d23/f97 x:0 0 0 2024-04-22T23:35:03.542 INFO:tasks.workunit.client.1.smithi157.stdout:8/343: dwrite d2/d6/f3c [0,4194304] 0 2024-04-22T23:35:03.543 INFO:tasks.workunit.client.1.smithi157.stdout:6/753: dwrite d1/d2f/d54/f87 [0,4194304] 0 2024-04-22T23:35:03.545 INFO:tasks.workunit.client.1.smithi157.stdout:9/513: mknod d4/df/d61/c98 0 2024-04-22T23:35:03.545 INFO:tasks.workunit.client.1.smithi157.stdout:1/544: dwrite d4/d17/d1d/d33/d41/f85 [0,4194304] 0 2024-04-22T23:35:03.545 INFO:tasks.workunit.client.1.smithi157.stdout:1/545: chown d4/db/d3d/l8b 0 1 2024-04-22T23:35:03.546 INFO:tasks.workunit.client.1.smithi157.stdout:2/627: dwrite d7/d8/d25/f4f [0,4194304] 0 2024-04-22T23:35:03.546 INFO:tasks.workunit.client.1.smithi157.stdout:7/428: dwrite d0/d9/d10/f5d [0,4194304] 0 2024-04-22T23:35:03.552 INFO:tasks.workunit.client.1.smithi157.stdout:9/514: chown d4/l30 117987987 1 2024-04-22T23:35:03.552 INFO:tasks.workunit.client.1.smithi157.stdout:9/515: write d4/d15/d23/f88 [334650,60607] 0 2024-04-22T23:35:03.552 INFO:tasks.workunit.client.1.smithi157.stdout:6/754: dread d1/f34 [0,4194304] 0 2024-04-22T23:35:03.552 INFO:tasks.workunit.client.1.smithi157.stdout:6/755: getdents d1/d2f/d54/dc4 0 2024-04-22T23:35:03.554 INFO:tasks.workunit.client.0.smithi107.stdout:6/282: dwrite db/d20/d2a/d38/f3c [0,4194304] 0 2024-04-22T23:35:03.555 INFO:tasks.workunit.client.0.smithi107.stdout:1/303: dwrite d7/d19/d2d/f3d [0,4194304] 0 2024-04-22T23:35:03.555 INFO:tasks.workunit.client.0.smithi107.stdout:1/304: dread - d7/d19/d42/f64 zero size 2024-04-22T23:35:03.556 INFO:tasks.workunit.client.0.smithi107.stdout:1/305: dread - d7/d19/d20/d39/f69 zero size 2024-04-22T23:35:03.556 INFO:tasks.workunit.client.1.smithi157.stdout:1/546: getdents d4/d17/d1d 0 2024-04-22T23:35:03.556 INFO:tasks.workunit.client.0.smithi107.stdout:8/235: dwrite d0/f19 [0,4194304] 0 2024-04-22T23:35:03.557 INFO:tasks.workunit.client.1.smithi157.stdout:3/491: dwrite d0/d3/f1e [4194304,4194304] 0 2024-04-22T23:35:03.557 INFO:tasks.workunit.client.0.smithi107.stdout:6/283: rmdir db/d20/d2a/d35 39 2024-04-22T23:35:03.557 INFO:tasks.workunit.client.0.smithi107.stdout:6/284: write db/f1e [2349115,116213] 0 2024-04-22T23:35:03.560 INFO:tasks.workunit.client.1.smithi157.stdout:5/499: dwrite d1/d5/dc/d5d/d77/fad [0,4194304] 0 2024-04-22T23:35:03.561 INFO:tasks.workunit.client.0.smithi107.stdout:1/306: creat d7/d19/d20/d39/d53/d43/d46/d5c/f6b x:0 0 0 2024-04-22T23:35:03.563 INFO:tasks.workunit.client.1.smithi157.stdout:2/628: write d7/d1f/d26/f44 [3661944,82141] 0 2024-04-22T23:35:03.568 INFO:tasks.workunit.client.0.smithi107.stdout:8/236: getdents d0/d7/df 0 2024-04-22T23:35:03.570 INFO:tasks.workunit.client.1.smithi157.stdout:3/492: dread d0/d3/d34/d3d/f44 [0,4194304] 0 2024-04-22T23:35:03.571 INFO:tasks.workunit.client.0.smithi107.stdout:0/312: dwrite d1/d9/d14/d4f/f69 [0,4194304] 0 2024-04-22T23:35:03.572 INFO:tasks.workunit.client.1.smithi157.stdout:5/500: dread d1/d1f/faf [4194304,4194304] 0 2024-04-22T23:35:03.572 INFO:tasks.workunit.client.1.smithi157.stdout:6/756: truncate d1/d2f/d3b/d39/d4e/da8/de5/d83/f9f 449914 0 2024-04-22T23:35:03.572 INFO:tasks.workunit.client.1.smithi157.stdout:8/344: rename d2/d6/d21/d33 to d2/d34/d6a/d77 0 2024-04-22T23:35:03.575 INFO:tasks.workunit.client.1.smithi157.stdout:3/493: mkdir d0/d3/d34/d62/d9a 0 2024-04-22T23:35:03.575 INFO:tasks.workunit.client.0.smithi107.stdout:8/237: write d0/d7/fa [2228298,80322] 0 2024-04-22T23:35:03.578 INFO:tasks.workunit.client.0.smithi107.stdout:2/348: rmdir d1/d2b/d59 39 2024-04-22T23:35:03.578 INFO:tasks.workunit.client.0.smithi107.stdout:0/313: dread d1/d9/d14/f4d [0,4194304] 0 2024-04-22T23:35:03.579 INFO:tasks.workunit.client.1.smithi157.stdout:6/757: truncate d1/d36/fa3 494171 0 2024-04-22T23:35:03.579 INFO:tasks.workunit.client.1.smithi157.stdout:2/629: rename d7/d8/d30/d23/l42 to d7/d1f/d3e/db4/ld5 0 2024-04-22T23:35:03.580 INFO:tasks.workunit.client.0.smithi107.stdout:8/238: symlink d0/d7/d20/l4e 0 2024-04-22T23:35:03.580 INFO:tasks.workunit.client.1.smithi157.stdout:7/429: dread d0/d9/d10/f18 [0,4194304] 0 2024-04-22T23:35:03.581 INFO:tasks.workunit.client.0.smithi107.stdout:2/349: mknod d1/d67/c7d 0 2024-04-22T23:35:03.582 INFO:tasks.workunit.client.1.smithi157.stdout:3/494: symlink d0/d3/d34/d62/l9b 0 2024-04-22T23:35:03.582 INFO:tasks.workunit.client.1.smithi157.stdout:3/495: truncate d0/d3/f35 349453 0 2024-04-22T23:35:03.582 INFO:tasks.workunit.client.1.smithi157.stdout:3/496: write d0/d3/d34/d3d/d5b/d72/f7f [628136,43561] 0 2024-04-22T23:35:03.582 INFO:tasks.workunit.client.0.smithi107.stdout:8/239: truncate d0/f4 2035014 0 2024-04-22T23:35:03.584 INFO:tasks.workunit.client.0.smithi107.stdout:6/285: dwrite db/f1d [0,4194304] 0 2024-04-22T23:35:03.584 INFO:tasks.workunit.client.1.smithi157.stdout:6/758: getdents d1/d17 0 2024-04-22T23:35:03.585 INFO:tasks.workunit.client.0.smithi107.stdout:2/350: mkdir d1/d2b/d49/d7e 0 2024-04-22T23:35:03.585 INFO:tasks.workunit.client.0.smithi107.stdout:2/351: stat d1/d12/d35/d48/d13/l28 0 2024-04-22T23:35:03.587 INFO:tasks.workunit.client.1.smithi157.stdout:2/630: creat d7/d1f/d26/fd6 x:0 0 0 2024-04-22T23:35:03.587 INFO:tasks.workunit.client.1.smithi157.stdout:2/631: fdatasync d7/d1f/f52 0 2024-04-22T23:35:03.587 INFO:tasks.workunit.client.1.smithi157.stdout:5/501: dread d1/d5/d7/f2e [0,4194304] 0 2024-04-22T23:35:03.587 INFO:tasks.workunit.client.1.smithi157.stdout:2/632: chown d7/d8/d30/d5b/d8a/lac 0 1 2024-04-22T23:35:03.588 INFO:tasks.workunit.client.0.smithi107.stdout:2/352: mknod d1/d2b/d59/c7f 0 2024-04-22T23:35:03.588 INFO:tasks.workunit.client.0.smithi107.stdout:0/314: write d1/f15 [108095,104767] 0 2024-04-22T23:35:03.588 INFO:tasks.workunit.client.0.smithi107.stdout:0/315: fdatasync d1/d27/f21 0 2024-04-22T23:35:03.589 INFO:tasks.workunit.client.1.smithi157.stdout:7/430: rmdir d0/d9/d29/d5e 39 2024-04-22T23:35:03.589 INFO:tasks.workunit.client.1.smithi157.stdout:7/431: stat d0/d9/db/df 0 2024-04-22T23:35:03.592 INFO:tasks.workunit.client.0.smithi107.stdout:0/316: mkdir d1/d27/d20/d40/d75 0 2024-04-22T23:35:03.592 INFO:tasks.workunit.client.0.smithi107.stdout:0/317: write d1/d27/d20/d2c/d41/f64 [858085,119133] 0 2024-04-22T23:35:03.595 INFO:tasks.workunit.client.0.smithi107.stdout:8/240: dread d0/d7/df/d3b/f3d [0,4194304] 0 2024-04-22T23:35:03.596 INFO:tasks.workunit.client.0.smithi107.stdout:0/318: dread d1/d27/f38 [0,4194304] 0 2024-04-22T23:35:03.596 INFO:tasks.workunit.client.0.smithi107.stdout:3/321: sync 2024-04-22T23:35:03.596 INFO:tasks.workunit.client.0.smithi107.stdout:3/322: write db/d21/d27/d15/d29/d2a/f43 [5351754,7591] 0 2024-04-22T23:35:03.596 INFO:tasks.workunit.client.1.smithi157.stdout:3/497: unlink d0/d3/f1d 0 2024-04-22T23:35:03.597 INFO:tasks.workunit.client.1.smithi157.stdout:6/759: link d1/d17/d5d/f64 d1/d2f/d3b/feb 0 2024-04-22T23:35:03.597 INFO:tasks.workunit.client.1.smithi157.stdout:7/432: dread d0/f46 [0,4194304] 0 2024-04-22T23:35:03.597 INFO:tasks.workunit.client.1.smithi157.stdout:7/433: truncate d0/d9/d29/d33/d37/f87 68494 0 2024-04-22T23:35:03.599 INFO:tasks.workunit.client.0.smithi107.stdout:3/323: creat db/d21/d27/d15/d29/f6d x:0 0 0 2024-04-22T23:35:03.599 INFO:tasks.workunit.client.0.smithi107.stdout:3/324: write db/d21/d27/f23 [6001493,89437] 0 2024-04-22T23:35:03.601 INFO:tasks.workunit.client.1.smithi157.stdout:5/502: creat d1/d27/d29/fbf x:0 0 0 2024-04-22T23:35:03.602 INFO:tasks.workunit.client.1.smithi157.stdout:6/760: dread d1/f88 [4194304,4194304] 0 2024-04-22T23:35:03.602 INFO:tasks.workunit.client.0.smithi107.stdout:3/325: mkdir db/d21/d27/d54/d25/d6e 0 2024-04-22T23:35:03.603 INFO:tasks.workunit.client.1.smithi157.stdout:3/498: mkdir d0/d3/d34/d3d/d57/d60/d9c 0 2024-04-22T23:35:03.605 INFO:tasks.workunit.client.1.smithi157.stdout:6/761: creat d1/d2f/fec x:0 0 0 2024-04-22T23:35:03.605 INFO:tasks.workunit.client.0.smithi107.stdout:3/326: unlink db/d21/d27/d15/d29/f67 0 2024-04-22T23:35:03.606 INFO:tasks.workunit.client.0.smithi107.stdout:3/327: creat db/f6f x:0 0 0 2024-04-22T23:35:03.606 INFO:tasks.workunit.client.1.smithi157.stdout:0/455: sync 2024-04-22T23:35:03.610 INFO:tasks.workunit.client.0.smithi107.stdout:3/328: write db/d21/d27/d15/d29/d2a/f58 [194711,22640] 0 2024-04-22T23:35:03.611 INFO:tasks.workunit.client.1.smithi157.stdout:4/618: sync 2024-04-22T23:35:03.611 INFO:tasks.workunit.client.1.smithi157.stdout:4/619: write d3/f55 [756437,51834] 0 2024-04-22T23:35:03.615 INFO:tasks.workunit.client.0.smithi107.stdout:3/329: rename db/d21/d27/d15/d29/d2a/d45/c50 to db/d21/d27/d54/d25/c70 0 2024-04-22T23:35:03.615 INFO:tasks.workunit.client.0.smithi107.stdout:3/330: truncate db/f6f 1036135 0 2024-04-22T23:35:03.617 INFO:tasks.workunit.client.0.smithi107.stdout:3/331: mknod db/d21/d27/d36/c71 0 2024-04-22T23:35:03.618 INFO:tasks.workunit.client.0.smithi107.stdout:3/332: truncate db/d21/f33 9086330 0 2024-04-22T23:35:03.620 INFO:tasks.workunit.client.1.smithi157.stdout:5/503: dread d1/fe [0,4194304] 0 2024-04-22T23:35:03.621 INFO:tasks.workunit.client.1.smithi157.stdout:5/504: creat d1/d5/d7/d2f/d45/d54/fc0 x:0 0 0 2024-04-22T23:35:03.622 INFO:tasks.workunit.client.1.smithi157.stdout:5/505: write d1/d5/d7/d2f/d45/d54/fc0 [573612,59035] 0 2024-04-22T23:35:03.622 INFO:tasks.workunit.client.0.smithi107.stdout:3/333: mknod db/d21/d27/d15/c72 0 2024-04-22T23:35:03.622 INFO:tasks.workunit.client.0.smithi107.stdout:3/334: fdatasync db/f6f 0 2024-04-22T23:35:03.623 INFO:tasks.workunit.client.1.smithi157.stdout:6/762: dread d1/d2f/d3b/f3e [0,4194304] 0 2024-04-22T23:35:03.623 INFO:tasks.workunit.client.1.smithi157.stdout:6/763: dread - d1/d2f/dc7/dce/fe7 zero size 2024-04-22T23:35:03.625 INFO:tasks.workunit.client.0.smithi107.stdout:3/335: truncate db/ff 1493792 0 2024-04-22T23:35:03.625 INFO:tasks.workunit.client.0.smithi107.stdout:3/336: chown db/d21/d27/d54/d25/d53/c63 1 1 2024-04-22T23:35:03.625 INFO:tasks.workunit.client.1.smithi157.stdout:5/506: dread d1/d5/fb5 [0,4194304] 0 2024-04-22T23:35:03.625 INFO:tasks.workunit.client.1.smithi157.stdout:5/507: write d1/d5/f14 [8895631,92991] 0 2024-04-22T23:35:03.625 INFO:tasks.workunit.client.1.smithi157.stdout:6/764: symlink d1/d2f/d54/d9b/dda/led 0 2024-04-22T23:35:03.625 INFO:tasks.workunit.client.1.smithi157.stdout:6/765: creat d1/d2f/d3b/d39/d4e/d74/fee x:0 0 0 2024-04-22T23:35:03.626 INFO:tasks.workunit.client.1.smithi157.stdout:6/766: chown d1/d2f/d3b/d49/c67 64707688 1 2024-04-22T23:35:03.626 INFO:tasks.workunit.client.1.smithi157.stdout:6/767: stat d1/d17/c1f 0 2024-04-22T23:35:03.626 INFO:tasks.workunit.client.0.smithi107.stdout:6/286: dwrite db/d20/d2a/f2f [0,4194304] 0 2024-04-22T23:35:03.626 INFO:tasks.workunit.client.0.smithi107.stdout:6/287: fdatasync db/d20/d2a/f2f 0 2024-04-22T23:35:03.626 INFO:tasks.workunit.client.0.smithi107.stdout:3/337: symlink db/d21/d27/d15/d29/l73 0 2024-04-22T23:35:03.627 INFO:tasks.workunit.client.1.smithi157.stdout:6/768: mkdir d1/d2f/d3b/d39/d4e/da8/dac/dbe/def 0 2024-04-22T23:35:03.630 INFO:tasks.workunit.client.1.smithi157.stdout:4/620: write d3/d11/d1d/f41 [2279224,43851] 0 2024-04-22T23:35:03.630 INFO:tasks.workunit.client.1.smithi157.stdout:8/345: dwrite d2/d6/d21/d5f/d17/f4f [0,4194304] 0 2024-04-22T23:35:03.630 INFO:tasks.workunit.client.0.smithi107.stdout:3/338: rename db/d21/d27/d54/d25/c61 to db/d21/d27/d54/d25/c74 0 2024-04-22T23:35:03.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:35:03 smithi157 ceph-mon[33877]: pgmap v258: 65 pgs: 65 active+clean; 1.2 GiB data, 5.0 GiB used, 531 GiB / 536 GiB avail; 133 MiB/s rd, 165 MiB/s wr, 679 op/s 2024-04-22T23:35:03.632 INFO:tasks.workunit.client.1.smithi157.stdout:6/769: symlink d1/d2f/d3b/d39/d4e/da8/dac/ddd/ddf/lf0 0 2024-04-22T23:35:03.633 INFO:tasks.workunit.client.1.smithi157.stdout:6/770: chown d1/d2f/d54/d5c/fde 2 1 2024-04-22T23:35:03.633 INFO:tasks.workunit.client.1.smithi157.stdout:6/771: write d1/d2f/d3b/d39/d4e/da8/de5/d83/f59 [836319,83610] 0 2024-04-22T23:35:03.636 INFO:tasks.workunit.client.1.smithi157.stdout:8/346: creat d2/d34/f78 x:0 0 0 2024-04-22T23:35:03.638 INFO:tasks.workunit.client.0.smithi107.stdout:2/353: dwrite d1/d2b/f6e [0,4194304] 0 2024-04-22T23:35:03.638 INFO:tasks.workunit.client.0.smithi107.stdout:2/354: truncate d1/d67/f7b 503213 0 2024-04-22T23:35:03.638 INFO:tasks.workunit.client.0.smithi107.stdout:2/355: write d1/f78 [748572,71420] 0 2024-04-22T23:35:03.638 INFO:tasks.workunit.client.0.smithi107.stdout:2/356: creat d1/d2b/f80 x:0 0 0 2024-04-22T23:35:03.638 INFO:tasks.workunit.client.0.smithi107.stdout:2/357: chown d1/d67/f5d 666587 1 2024-04-22T23:35:03.640 INFO:tasks.workunit.client.0.smithi107.stdout:6/288: write db/d20/d2a/d38/f3c [2961416,88056] 0 2024-04-22T23:35:03.641 INFO:tasks.workunit.client.0.smithi107.stdout:2/358: mknod d1/d2b/d59/c81 0 2024-04-22T23:35:03.641 INFO:tasks.workunit.client.0.smithi107.stdout:2/359: fsync d1/d12/d35/d48/d13/f32 0 2024-04-22T23:35:03.641 INFO:tasks.workunit.client.0.smithi107.stdout:2/360: truncate d1/d67/f6f 794078 0 2024-04-22T23:35:03.642 INFO:tasks.workunit.client.0.smithi107.stdout:2/361: truncate d1/d12/d35/d48/f2c 346229 0 2024-04-22T23:35:03.648 INFO:tasks.workunit.client.0.smithi107.stdout:8/241: dwrite d0/d7/df/d17/f40 [0,4194304] 0 2024-04-22T23:35:03.650 INFO:tasks.workunit.client.1.smithi157.stdout:7/434: dwrite d0/d9/d29/d5e/f6a [0,4194304] 0 2024-04-22T23:35:03.650 INFO:tasks.workunit.client.1.smithi157.stdout:7/435: fsync d0/f46 0 2024-04-22T23:35:03.650 INFO:tasks.workunit.client.1.smithi157.stdout:7/436: readlink d0/d9/d29/l2d 0 2024-04-22T23:35:03.650 INFO:tasks.workunit.client.1.smithi157.stdout:7/437: fsync d0/d9/db/f15 0 2024-04-22T23:35:03.653 INFO:tasks.workunit.client.1.smithi157.stdout:7/438: truncate d0/d9/d10/f18 943835 0 2024-04-22T23:35:03.654 INFO:tasks.workunit.client.1.smithi157.stdout:7/439: chown d0/d9/d36 1 1 2024-04-22T23:35:03.654 INFO:tasks.workunit.client.1.smithi157.stdout:7/440: write d0/d9/f17 [907053,108560] 0 2024-04-22T23:35:03.655 INFO:tasks.workunit.client.1.smithi157.stdout:7/441: dread d0/d9/de/f23 [0,4194304] 0 2024-04-22T23:35:03.656 INFO:tasks.workunit.client.0.smithi107.stdout:8/242: symlink d0/d7/df/d33/l4f 0 2024-04-22T23:35:03.656 INFO:tasks.workunit.client.0.smithi107.stdout:8/243: dread - d0/d7/d20/d2d/f30 zero size 2024-04-22T23:35:03.657 INFO:tasks.workunit.client.1.smithi157.stdout:0/456: dwrite d17/f25 [0,4194304] 0 2024-04-22T23:35:03.657 INFO:tasks.workunit.client.1.smithi157.stdout:0/457: dread - d17/d23/d34/d67/d7f/f85 zero size 2024-04-22T23:35:03.658 INFO:tasks.workunit.client.0.smithi107.stdout:0/319: dwrite d1/d9/d14/d4f/d59/f5c [0,4194304] 0 2024-04-22T23:35:03.658 INFO:tasks.workunit.client.0.smithi107.stdout:8/244: truncate d0/d7/df/d17/f2a 2426617 0 2024-04-22T23:35:03.659 INFO:tasks.workunit.client.1.smithi157.stdout:7/442: rename d0/d9/f17 to d0/d9/d42/d55/f8f 0 2024-04-22T23:35:03.660 INFO:tasks.workunit.client.1.smithi157.stdout:0/458: truncate d17/d38/f65 1011095 0 2024-04-22T23:35:03.662 INFO:tasks.workunit.client.1.smithi157.stdout:0/459: rmdir d17/d2b/d4f 39 2024-04-22T23:35:03.662 INFO:tasks.workunit.client.1.smithi157.stdout:7/443: symlink d0/d9/db/l90 0 2024-04-22T23:35:03.663 INFO:tasks.workunit.client.0.smithi107.stdout:8/245: rename d0/d7/df/d18/f1f to d0/d7/df/d18/f50 0 2024-04-22T23:35:03.664 INFO:tasks.workunit.client.1.smithi157.stdout:0/460: symlink d17/d2b/la0 0 2024-04-22T23:35:03.664 INFO:tasks.workunit.client.1.smithi157.stdout:7/444: mkdir d0/d9/d42/d55/d56/d91 0 2024-04-22T23:35:03.664 INFO:tasks.workunit.client.1.smithi157.stdout:7/445: write d0/d9/f7d [16160,65464] 0 2024-04-22T23:35:03.664 INFO:tasks.workunit.client.1.smithi157.stdout:7/446: readlink d0/d9/d42/d55/d56/l58 0 2024-04-22T23:35:03.664 INFO:tasks.workunit.client.1.smithi157.stdout:7/447: fsync d0/f7 0 2024-04-22T23:35:03.666 INFO:tasks.workunit.client.1.smithi157.stdout:7/448: rename d0/d9/d42/d55/d56/l58 to d0/d9/l92 0 2024-04-22T23:35:03.672 INFO:tasks.workunit.client.1.smithi157.stdout:7/449: creat d0/d9/db/d35/f93 x:0 0 0 2024-04-22T23:35:03.672 INFO:tasks.workunit.client.1.smithi157.stdout:7/450: write d0/d9/db/df/d68/f8e [785875,57253] 0 2024-04-22T23:35:03.673 INFO:tasks.workunit.client.1.smithi157.stdout:7/451: creat d0/d9/d42/d55/f94 x:0 0 0 2024-04-22T23:35:03.674 INFO:tasks.workunit.client.1.smithi157.stdout:7/452: mkdir d0/d9/d42/d55/d80/d95 0 2024-04-22T23:35:03.676 INFO:tasks.workunit.client.1.smithi157.stdout:5/508: dwrite d1/d5/f46 [0,4194304] 0 2024-04-22T23:35:03.677 INFO:tasks.workunit.client.0.smithi107.stdout:0/320: write d1/d27/f57 [83419,115177] 0 2024-04-22T23:35:03.677 INFO:tasks.workunit.client.1.smithi157.stdout:7/453: symlink d0/d9/d10/l96 0 2024-04-22T23:35:03.681 INFO:tasks.workunit.client.1.smithi157.stdout:5/509: mkdir d1/db0/dc1 0 2024-04-22T23:35:03.684 INFO:tasks.workunit.client.1.smithi157.stdout:6/772: dwrite d1/fc2 [0,4194304] 0 2024-04-22T23:35:03.687 INFO:tasks.workunit.client.0.smithi107.stdout:0/321: dread d1/d27/d20/d2c/d41/d52/f6c [0,4194304] 0 2024-04-22T23:35:03.687 INFO:tasks.workunit.client.0.smithi107.stdout:0/322: read d1/f15 [3027120,74446] 0 2024-04-22T23:35:03.688 INFO:tasks.workunit.client.0.smithi107.stdout:0/323: unlink d1/d27/d20/d2c/d41/d52/f73 0 2024-04-22T23:35:03.689 INFO:tasks.workunit.client.0.smithi107.stdout:0/324: mknod d1/d27/d20/d40/d75/c76 0 2024-04-22T23:35:03.689 INFO:tasks.workunit.client.0.smithi107.stdout:0/325: truncate d1/d9/d14/f4d 954879 0 2024-04-22T23:35:03.690 INFO:tasks.workunit.client.0.smithi107.stdout:0/326: creat d1/d2a/f77 x:0 0 0 2024-04-22T23:35:03.692 INFO:tasks.workunit.client.1.smithi157.stdout:6/773: getdents d1 0 2024-04-22T23:35:03.693 INFO:tasks.workunit.client.1.smithi157.stdout:6/774: rmdir d1/d2f/d54/d9b/dda 39 2024-04-22T23:35:03.693 INFO:tasks.workunit.client.1.smithi157.stdout:6/775: fdatasync d1/d2f/d3b/d49/fb3 0 2024-04-22T23:35:03.693 INFO:tasks.workunit.client.1.smithi157.stdout:6/776: stat d1/d17/f94 0 2024-04-22T23:35:03.693 INFO:tasks.workunit.client.1.smithi157.stdout:6/777: read - d1/d2f/d3b/d39/fdc zero size 2024-04-22T23:35:03.693 INFO:tasks.workunit.client.0.smithi107.stdout:0/327: dread d1/d27/d20/d2c/d41/d52/f6c [0,4194304] 0 2024-04-22T23:35:03.698 INFO:tasks.workunit.client.1.smithi157.stdout:4/621: write d3/d11/d34/f48 [1615341,44795] 0 2024-04-22T23:35:03.698 INFO:tasks.workunit.client.1.smithi157.stdout:4/622: fsync d3/d11/d1d/f26 0 2024-04-22T23:35:03.699 INFO:tasks.workunit.client.1.smithi157.stdout:4/623: mknod d3/d11/d1d/d73/da3/d31/cd1 0 2024-04-22T23:35:03.701 INFO:tasks.workunit.client.1.smithi157.stdout:4/624: rename d3/d96/d81/c8a to d3/d11/d1d/d73/da3/d31/da2/cd2 0 2024-04-22T23:35:03.701 INFO:tasks.workunit.client.1.smithi157.stdout:4/625: readlink d3/d96/d40/lb2 0 2024-04-22T23:35:03.702 INFO:tasks.workunit.client.0.smithi107.stdout:6/289: dwrite db/f21 [0,4194304] 0 2024-04-22T23:35:03.705 INFO:tasks.workunit.client.0.smithi107.stdout:6/290: dread db/d20/d2c/f44 [0,4194304] 0 2024-04-22T23:35:03.706 INFO:tasks.workunit.client.1.smithi157.stdout:8/347: dwrite d2/d6/d21/f54 [0,4194304] 0 2024-04-22T23:35:03.706 INFO:tasks.workunit.client.1.smithi157.stdout:8/348: fsync d2/d6/d21/d5f/f6f 0 2024-04-22T23:35:03.707 INFO:tasks.workunit.client.0.smithi107.stdout:6/291: rmdir db/d20/d2a/d35 39 2024-04-22T23:35:03.708 INFO:tasks.workunit.client.1.smithi157.stdout:6/778: write d1/f18 [3989348,108300] 0 2024-04-22T23:35:03.709 INFO:tasks.workunit.client.0.smithi107.stdout:6/292: rmdir db/d20/d2c/d3b 39 2024-04-22T23:35:03.709 INFO:tasks.workunit.client.1.smithi157.stdout:8/349: link d2/d6/d21/d5f/f1b d2/d6/d66/d67/f79 0 2024-04-22T23:35:03.709 INFO:tasks.workunit.client.1.smithi157.stdout:0/461: dwrite d17/f1a [4194304,4194304] 0 2024-04-22T23:35:03.710 INFO:tasks.workunit.client.0.smithi107.stdout:3/339: dwrite db/d21/d27/d15/d24/d52/d60/f6b [4194304,4194304] 0 2024-04-22T23:35:03.710 INFO:tasks.workunit.client.0.smithi107.stdout:8/246: dwrite d0/d7/df/d33/d37/f49 [0,4194304] 0 2024-04-22T23:35:03.710 INFO:tasks.workunit.client.1.smithi157.stdout:4/626: read d3/d96/f3d [3304255,68363] 0 2024-04-22T23:35:03.710 INFO:tasks.workunit.client.1.smithi157.stdout:4/627: fdatasync d3/f58 0 2024-04-22T23:35:03.711 INFO:tasks.workunit.client.0.smithi107.stdout:6/293: unlink db/d20/d2a/f31 0 2024-04-22T23:35:03.711 INFO:tasks.workunit.client.0.smithi107.stdout:6/294: chown db/f1d 16021552 1 2024-04-22T23:35:03.712 INFO:tasks.workunit.client.1.smithi157.stdout:8/350: rename d2/d4/d30/l65 to d2/d34/d6a/d77/d42/l7a 0 2024-04-22T23:35:03.712 INFO:tasks.workunit.client.1.smithi157.stdout:7/454: dread d0/d9/d42/d55/f8f [0,4194304] 0 2024-04-22T23:35:03.713 INFO:tasks.workunit.client.1.smithi157.stdout:4/628: mkdir d3/d5c/dd3 0 2024-04-22T23:35:03.714 INFO:tasks.workunit.client.0.smithi107.stdout:3/340: rename db/d21/d27/d15/d24/c4d to db/d21/c75 0 2024-04-22T23:35:03.714 INFO:tasks.workunit.client.0.smithi107.stdout:3/341: fdatasync db/d21/d27/d15/d29/d2a/f5b 0 2024-04-22T23:35:03.715 INFO:tasks.workunit.client.0.smithi107.stdout:0/328: dwrite d1/d27/d20/d2c/d2e/f71 [0,4194304] 0 2024-04-22T23:35:03.715 INFO:tasks.workunit.client.0.smithi107.stdout:3/342: dread db/d21/d27/d15/d29/d2a/f4c [0,4194304] 0 2024-04-22T23:35:03.715 INFO:tasks.workunit.client.1.smithi157.stdout:8/351: symlink d2/d6/d66/l7b 0 2024-04-22T23:35:03.715 INFO:tasks.workunit.client.1.smithi157.stdout:8/352: chown d2/d6/c16 317931 1 2024-04-22T23:35:03.715 INFO:tasks.workunit.client.1.smithi157.stdout:8/353: chown d2/d4/d73 1 1 2024-04-22T23:35:03.716 INFO:tasks.workunit.client.1.smithi157.stdout:5/510: dwrite d1/d5/dc/d43/f44 [0,4194304] 0 2024-04-22T23:35:03.716 INFO:tasks.workunit.client.1.smithi157.stdout:5/511: creat d1/d27/d29/fc2 x:0 0 0 2024-04-22T23:35:03.717 INFO:tasks.workunit.client.0.smithi107.stdout:2/362: fsync d1/d12/d35/d48/f2c 0 2024-04-22T23:35:03.717 INFO:tasks.workunit.client.0.smithi107.stdout:2/363: chown d1/d12/d35/d48/l24 1114101 1 2024-04-22T23:35:03.717 INFO:tasks.workunit.client.0.smithi107.stdout:2/364: chown d1/d67/c7d 19751 1 2024-04-22T23:35:03.717 INFO:tasks.workunit.client.1.smithi157.stdout:6/779: write d1/d2f/d54/d9b/f79 [3733523,8004] 0 2024-04-22T23:35:03.717 INFO:tasks.workunit.client.1.smithi157.stdout:7/455: mknod d0/d9/d29/d33/d37/c97 0 2024-04-22T23:35:03.717 INFO:tasks.workunit.client.1.smithi157.stdout:7/456: readlink d0/d9/db/d35/l5c 0 2024-04-22T23:35:03.722 INFO:tasks.workunit.client.0.smithi107.stdout:6/295: unlink db/d20/d2c/l50 0 2024-04-22T23:35:03.722 INFO:tasks.workunit.client.0.smithi107.stdout:3/343: write db/f1c [4051846,130612] 0 2024-04-22T23:35:03.722 INFO:tasks.workunit.client.0.smithi107.stdout:3/344: truncate db/d21/d27/f41 919045 0 2024-04-22T23:35:03.723 INFO:tasks.workunit.client.1.smithi157.stdout:4/629: dread d3/d11/d1d/d73/da3/f2b [0,4194304] 0 2024-04-22T23:35:03.723 INFO:tasks.workunit.client.1.smithi157.stdout:8/354: rename d2/f71 to d2/d6/d21/d5f/d11/d13/f7c 0 2024-04-22T23:35:03.723 INFO:tasks.workunit.client.1.smithi157.stdout:8/355: stat d2/d6/d21/d5f/d11/f4a 0 2024-04-22T23:35:03.723 INFO:tasks.workunit.client.1.smithi157.stdout:8/356: fsync d2/d34/f41 0 2024-04-22T23:35:03.723 INFO:tasks.workunit.client.1.smithi157.stdout:8/357: fdatasync d2/d6/d21/d5f/f56 0 2024-04-22T23:35:03.723 INFO:tasks.workunit.client.1.smithi157.stdout:8/358: read - d2/d6/d21/d5f/d11/f44 zero size 2024-04-22T23:35:03.723 INFO:tasks.workunit.client.1.smithi157.stdout:0/462: dread d17/f20 [0,4194304] 0 2024-04-22T23:35:03.723 INFO:tasks.workunit.client.1.smithi157.stdout:0/463: chown d17/d23/d34 143 1 2024-04-22T23:35:03.724 INFO:tasks.workunit.client.0.smithi107.stdout:0/329: mknod d1/d27/d20/c78 0 2024-04-22T23:35:03.727 INFO:tasks.workunit.client.0.smithi107.stdout:6/296: creat db/d20/d2c/d3b/f51 x:0 0 0 2024-04-22T23:35:03.727 INFO:tasks.workunit.client.0.smithi107.stdout:6/297: fdatasync db/f21 0 2024-04-22T23:35:03.728 INFO:tasks.workunit.client.0.smithi107.stdout:3/345: mkdir db/d21/d27/d54/d25/d76 0 2024-04-22T23:35:03.728 INFO:tasks.workunit.client.0.smithi107.stdout:3/346: readlink db/d21/d27/d36/l46 0 2024-04-22T23:35:03.731 INFO:tasks.workunit.client.0.smithi107.stdout:2/365: rmdir d1/d12/d35/d48/d13 39 2024-04-22T23:35:03.733 INFO:tasks.workunit.client.0.smithi107.stdout:6/298: rename db/c16 to db/d20/d2c/d3b/c52 0 2024-04-22T23:35:03.735 INFO:tasks.workunit.client.1.smithi157.stdout:8/359: write d2/d6/d21/d5f/f2f [214337,36633] 0 2024-04-22T23:35:03.736 INFO:tasks.workunit.client.1.smithi157.stdout:4/630: truncate d3/d11/d1d/d73/da3/d31/f6f 412053 0 2024-04-22T23:35:03.738 INFO:tasks.workunit.client.1.smithi157.stdout:0/464: mkdir d17/d2b/d4f/d5a/da1 0 2024-04-22T23:35:03.739 INFO:tasks.workunit.client.1.smithi157.stdout:6/780: getdents d1/d2f/d3b/d39/d4e/da8 0 2024-04-22T23:35:03.741 INFO:tasks.workunit.client.1.smithi157.stdout:7/457: dwrite d0/d9/d29/f4d [4194304,4194304] 0 2024-04-22T23:35:03.741 INFO:tasks.workunit.client.1.smithi157.stdout:8/360: creat d2/d6/d21/d5f/d11/d13/f7d x:0 0 0 2024-04-22T23:35:03.742 INFO:tasks.workunit.client.0.smithi107.stdout:4/283: sync 2024-04-22T23:35:03.742 INFO:tasks.workunit.client.0.smithi107.stdout:5/288: sync 2024-04-22T23:35:03.742 INFO:tasks.workunit.client.0.smithi107.stdout:2/366: write d1/d12/d35/d48/f2a [1740052,128306] 0 2024-04-22T23:35:03.743 INFO:tasks.workunit.client.1.smithi157.stdout:5/512: dwrite d1/d5/dc/f1d [0,4194304] 0 2024-04-22T23:35:03.744 INFO:tasks.workunit.client.0.smithi107.stdout:6/299: creat db/d20/d2a/d38/f53 x:0 0 0 2024-04-22T23:35:03.744 INFO:tasks.workunit.client.1.smithi157.stdout:4/631: creat d3/d96/d4c/fd4 x:0 0 0 2024-04-22T23:35:03.746 INFO:tasks.workunit.client.1.smithi157.stdout:0/465: unlink d17/d2b/d52/f44 0 2024-04-22T23:35:03.746 INFO:tasks.workunit.client.0.smithi107.stdout:4/284: mkdir d1/d7/d53/d66 0 2024-04-22T23:35:03.749 INFO:tasks.workunit.client.1.smithi157.stdout:8/361: mkdir d2/d6/d21/d5f/d17/d7e 0 2024-04-22T23:35:03.753 INFO:tasks.workunit.client.0.smithi107.stdout:5/289: link d9/db/f1d d9/d26/f5b 0 2024-04-22T23:35:03.753 INFO:tasks.workunit.client.0.smithi107.stdout:3/347: dread db/f1c [0,4194304] 0 2024-04-22T23:35:03.754 INFO:tasks.workunit.client.0.smithi107.stdout:7/251: sync 2024-04-22T23:35:03.755 INFO:tasks.workunit.client.1.smithi157.stdout:0/466: creat d17/d2b/d52/fa2 x:0 0 0 2024-04-22T23:35:03.755 INFO:tasks.workunit.client.0.smithi107.stdout:7/252: dread d7/f2f [0,4194304] 0 2024-04-22T23:35:03.756 INFO:tasks.workunit.client.1.smithi157.stdout:7/458: getdents d0/d9/db/df/d68 0 2024-04-22T23:35:03.757 INFO:tasks.workunit.client.0.smithi107.stdout:2/367: rename d1/d12/d35/d48/d13/c21 to d1/d67/c82 0 2024-04-22T23:35:03.757 INFO:tasks.workunit.client.1.smithi157.stdout:8/362: mkdir d2/d6/d21/d5f/d17/d7f 0 2024-04-22T23:35:03.758 INFO:tasks.workunit.client.1.smithi157.stdout:8/363: getdents d2/d6/d21/d5f/d17/d7f 0 2024-04-22T23:35:03.758 INFO:tasks.workunit.client.1.smithi157.stdout:6/781: dwrite d1/d17/d5d/f64 [0,4194304] 0 2024-04-22T23:35:03.759 INFO:tasks.workunit.client.1.smithi157.stdout:8/364: write d2/f48 [488165,2182] 0 2024-04-22T23:35:03.759 INFO:tasks.workunit.client.1.smithi157.stdout:8/365: write d2/d6/d21/d5f/d11/d13/f6c [1566688,7500] 0 2024-04-22T23:35:03.759 INFO:tasks.workunit.client.1.smithi157.stdout:8/366: write d2/d6/d21/d5f/f56 [705552,98678] 0 2024-04-22T23:35:03.759 INFO:tasks.workunit.client.0.smithi107.stdout:6/300: creat db/d20/d2a/d38/d4b/f54 x:0 0 0 2024-04-22T23:35:03.760 INFO:tasks.workunit.client.0.smithi107.stdout:4/285: mkdir d1/d8/d67 0 2024-04-22T23:35:03.760 INFO:tasks.workunit.client.0.smithi107.stdout:2/368: dread d1/d67/f7b [0,4194304] 0 2024-04-22T23:35:03.760 INFO:tasks.workunit.client.0.smithi107.stdout:2/369: write d1/d2b/d3d/d57/f5c [827429,112912] 0 2024-04-22T23:35:03.763 INFO:tasks.workunit.client.0.smithi107.stdout:3/348: rename db/d21/d27/d15/d29/d2a/d40/f5e to db/d21/d27/d15/d29/d4f/f77 0 2024-04-22T23:35:03.763 INFO:tasks.workunit.client.0.smithi107.stdout:3/349: chown db/d21/d27/d15/d29/d2a/f4c 1843986193 1 2024-04-22T23:35:03.763 INFO:tasks.workunit.client.0.smithi107.stdout:3/350: write db/d21/d27/d15/d29/d2a/d40/f6c [407807,64797] 0 2024-04-22T23:35:03.764 INFO:tasks.workunit.client.0.smithi107.stdout:4/286: truncate d1/d8/d56/d27/f2f 2451503 0 2024-04-22T23:35:03.765 INFO:tasks.workunit.client.1.smithi157.stdout:6/782: symlink d1/d2f/d3b/d39/d4e/lf1 0 2024-04-22T23:35:03.768 INFO:tasks.workunit.client.0.smithi107.stdout:0/330: rmdir d1 39 2024-04-22T23:35:03.769 INFO:tasks.workunit.client.1.smithi157.stdout:8/367: creat d2/d4/f80 x:0 0 0 2024-04-22T23:35:03.770 INFO:tasks.workunit.client.0.smithi107.stdout:9/345: sync 2024-04-22T23:35:03.770 INFO:tasks.workunit.client.0.smithi107.stdout:9/346: truncate f1 1399194 0 2024-04-22T23:35:03.771 INFO:tasks.workunit.client.0.smithi107.stdout:9/347: write d5/d7/f16 [5389958,22318] 0 2024-04-22T23:35:03.773 INFO:tasks.workunit.client.1.smithi157.stdout:6/783: mknod d1/d2f/d3b/d39/d4e/da8/de5/da7/cf2 0 2024-04-22T23:35:03.774 INFO:tasks.workunit.client.0.smithi107.stdout:7/253: rename d7/c20 to d7/d21/d54/c56 0 2024-04-22T23:35:03.775 INFO:tasks.workunit.client.1.smithi157.stdout:6/784: symlink d1/d2f/d54/d5c/lf3 0 2024-04-22T23:35:03.776 INFO:tasks.workunit.client.1.smithi157.stdout:1/547: sync 2024-04-22T23:35:03.776 INFO:tasks.workunit.client.1.smithi157.stdout:3/499: sync 2024-04-22T23:35:03.776 INFO:tasks.workunit.client.1.smithi157.stdout:3/500: dread - d0/d3/d34/f93 zero size 2024-04-22T23:35:03.776 INFO:tasks.workunit.client.1.smithi157.stdout:2/633: sync 2024-04-22T23:35:03.776 INFO:tasks.workunit.client.1.smithi157.stdout:9/516: sync 2024-04-22T23:35:03.777 INFO:tasks.workunit.client.0.smithi107.stdout:4/287: write d1/d7/f32 [4513022,45515] 0 2024-04-22T23:35:03.778 INFO:tasks.workunit.client.1.smithi157.stdout:6/785: unlink d1/d17/d5d/f64 0 2024-04-22T23:35:03.779 INFO:tasks.workunit.client.0.smithi107.stdout:9/348: creat d5/d7/d27/d46/f75 x:0 0 0 2024-04-22T23:35:03.780 INFO:tasks.workunit.client.1.smithi157.stdout:7/459: write d0/d9/d10/f1f [1800846,82822] 0 2024-04-22T23:35:03.780 INFO:tasks.workunit.client.1.smithi157.stdout:5/513: dwrite d1/d5/f15 [0,4194304] 0 2024-04-22T23:35:03.781 INFO:tasks.workunit.client.1.smithi157.stdout:9/517: getdents d4 0 2024-04-22T23:35:03.781 INFO:tasks.workunit.client.1.smithi157.stdout:6/786: symlink d1/d2f/d54/dc4/lf4 0 2024-04-22T23:35:03.782 INFO:tasks.workunit.client.0.smithi107.stdout:4/288: mknod d1/d21/d29/c68 0 2024-04-22T23:35:03.782 INFO:tasks.workunit.client.0.smithi107.stdout:4/289: readlink d1/d8/l3a 0 2024-04-22T23:35:03.782 INFO:tasks.workunit.client.0.smithi107.stdout:4/290: dread - d1/d8/d56/d27/f30 zero size 2024-04-22T23:35:03.783 INFO:tasks.workunit.client.0.smithi107.stdout:4/291: rmdir d1/d21/d29/d2e/d4d 39 2024-04-22T23:35:03.784 INFO:tasks.workunit.client.0.smithi107.stdout:4/292: chown d1/d8/l62 216063 1 2024-04-22T23:35:03.784 INFO:tasks.workunit.client.0.smithi107.stdout:4/293: write d1/d21/d29/d2e/f46 [3657424,61319] 0 2024-04-22T23:35:03.784 INFO:tasks.workunit.client.1.smithi157.stdout:2/634: dread d7/d8/d30/d23/d27/f58 [4194304,4194304] 0 2024-04-22T23:35:03.787 INFO:tasks.workunit.client.1.smithi157.stdout:9/518: truncate d4/f29 1804488 0 2024-04-22T23:35:03.788 INFO:tasks.workunit.client.1.smithi157.stdout:3/501: write d0/d3/d34/d39/f66 [621741,67057] 0 2024-04-22T23:35:03.788 INFO:tasks.workunit.client.1.smithi157.stdout:3/502: chown d0/d7/d95 6 1 2024-04-22T23:35:03.789 INFO:tasks.workunit.client.0.smithi107.stdout:4/294: unlink d1/d7/d5b/f5e 0 2024-04-22T23:35:03.790 INFO:tasks.workunit.client.1.smithi157.stdout:1/548: dread d4/d17/d1d/d33/d41/f85 [0,4194304] 0 2024-04-22T23:35:03.790 INFO:tasks.workunit.client.1.smithi157.stdout:6/787: mkdir d1/d2f/d3b/d39/d4e/da8/dac/dbe/df5 0 2024-04-22T23:35:03.791 INFO:tasks.workunit.client.1.smithi157.stdout:2/635: rmdir d7/d8/d95/d92 0 2024-04-22T23:35:03.791 INFO:tasks.workunit.client.0.smithi107.stdout:6/301: dwrite db/f29 [0,4194304] 0 2024-04-22T23:35:03.791 INFO:tasks.workunit.client.0.smithi107.stdout:6/302: fsync db/f1d 0 2024-04-22T23:35:03.792 INFO:tasks.workunit.client.1.smithi157.stdout:3/503: dread d0/d3/f51 [0,4194304] 0 2024-04-22T23:35:03.792 INFO:tasks.workunit.client.1.smithi157.stdout:3/504: write d0/d3/d34/d39/f66 [2175629,50629] 0 2024-04-22T23:35:03.792 INFO:tasks.workunit.client.1.smithi157.stdout:3/505: chown d0/d3/d34/d39/f49 19946349 1 2024-04-22T23:35:03.793 INFO:tasks.workunit.client.0.smithi107.stdout:4/295: rename d1/d21/d29/c35 to d1/d7/d63/c69 0 2024-04-22T23:35:03.793 INFO:tasks.workunit.client.0.smithi107.stdout:4/296: dread - d1/d8/d56/d27/f30 zero size 2024-04-22T23:35:03.793 INFO:tasks.workunit.client.0.smithi107.stdout:9/349: dread d5/d14/f15 [0,4194304] 0 2024-04-22T23:35:03.793 INFO:tasks.workunit.client.0.smithi107.stdout:9/350: write d5/f13 [4257659,55732] 0 2024-04-22T23:35:03.793 INFO:tasks.workunit.client.0.smithi107.stdout:9/351: fdatasync d5/fc 0 2024-04-22T23:35:03.793 INFO:tasks.workunit.client.0.smithi107.stdout:6/303: mkdir db/d20/d55 0 2024-04-22T23:35:03.793 INFO:tasks.workunit.client.0.smithi107.stdout:6/304: fsync db/d20/d2a/f2f 0 2024-04-22T23:35:03.794 INFO:tasks.workunit.client.1.smithi157.stdout:7/460: rmdir d0/d9/db/d35 39 2024-04-22T23:35:03.794 INFO:tasks.workunit.client.1.smithi157.stdout:9/519: symlink d4/d15/d23/d27/l99 0 2024-04-22T23:35:03.795 INFO:tasks.workunit.client.0.smithi107.stdout:4/297: link d1/d21/d29/f41 d1/d21/f6a 0 2024-04-22T23:35:03.796 INFO:tasks.workunit.client.1.smithi157.stdout:3/506: dread d0/d3/d34/d3d/d57/f64 [0,4194304] 0 2024-04-22T23:35:03.796 INFO:tasks.workunit.client.1.smithi157.stdout:1/549: mkdir d4/d17/db9 0 2024-04-22T23:35:03.797 INFO:tasks.workunit.client.1.smithi157.stdout:1/550: write d4/d17/d34/d3b/d50/f7b [476077,59335] 0 2024-04-22T23:35:03.797 INFO:tasks.workunit.client.1.smithi157.stdout:2/636: symlink d7/d8/dca/ld7 0 2024-04-22T23:35:03.798 INFO:tasks.workunit.client.1.smithi157.stdout:7/461: mkdir d0/d9/d29/d98 0 2024-04-22T23:35:03.798 INFO:tasks.workunit.client.0.smithi107.stdout:9/352: rename d5/d7/d27/c6e to d5/d6c/c76 0 2024-04-22T23:35:03.799 INFO:tasks.workunit.client.1.smithi157.stdout:2/637: symlink d7/d8/d8e/d80/ld8 0 2024-04-22T23:35:03.799 INFO:tasks.workunit.client.1.smithi157.stdout:2/638: fsync d7/d1f/d3e/f5e 0 2024-04-22T23:35:03.800 INFO:tasks.workunit.client.1.smithi157.stdout:0/467: dwrite d17/d23/d34/d67/d7f/f77 [4194304,4194304] 0 2024-04-22T23:35:03.800 INFO:tasks.workunit.client.1.smithi157.stdout:0/468: readlink d17/l7d 0 2024-04-22T23:35:03.802 INFO:tasks.workunit.client.1.smithi157.stdout:2/639: unlink d7/d8/l51 0 2024-04-22T23:35:03.803 INFO:tasks.workunit.client.1.smithi157.stdout:0/469: mknod d17/d38/ca3 0 2024-04-22T23:35:03.804 INFO:tasks.workunit.client.1.smithi157.stdout:2/640: mknod d7/d8/d25/cd9 0 2024-04-22T23:35:03.804 INFO:tasks.workunit.client.1.smithi157.stdout:6/788: dread d1/d17/f27 [0,4194304] 0 2024-04-22T23:35:03.806 INFO:tasks.workunit.client.0.smithi107.stdout:6/305: dread db/f1d [0,4194304] 0 2024-04-22T23:35:03.807 INFO:tasks.workunit.client.1.smithi157.stdout:5/514: write d1/d5/d7/d2f/d51/f97 [860636,14077] 0 2024-04-22T23:35:03.807 INFO:tasks.workunit.client.1.smithi157.stdout:0/470: unlink d17/d23/c60 0 2024-04-22T23:35:03.807 INFO:tasks.workunit.client.0.smithi107.stdout:3/351: dwrite db/d21/d27/d15/d29/d2a/d45/f49 [0,4194304] 0 2024-04-22T23:35:03.807 INFO:tasks.workunit.client.0.smithi107.stdout:7/254: dwrite d7/d1b/f28 [0,4194304] 0 2024-04-22T23:35:03.808 INFO:tasks.workunit.client.0.smithi107.stdout:2/370: dwrite d1/d12/d35/d48/d13/d4f/f62 [0,4194304] 0 2024-04-22T23:35:03.808 INFO:tasks.workunit.client.0.smithi107.stdout:7/255: write d7/f2f [408015,32665] 0 2024-04-22T23:35:03.808 INFO:tasks.workunit.client.0.smithi107.stdout:3/352: dread db/f6f [0,4194304] 0 2024-04-22T23:35:03.811 INFO:tasks.workunit.client.0.smithi107.stdout:6/306: rename db/f1e to db/d20/d2a/d38/f56 0 2024-04-22T23:35:03.811 INFO:tasks.workunit.client.0.smithi107.stdout:6/307: write db/d20/d2a/d38/f53 [856855,48994] 0 2024-04-22T23:35:03.812 INFO:tasks.workunit.client.1.smithi157.stdout:3/507: dread d0/d3/d34/d3d/f69 [0,4194304] 0 2024-04-22T23:35:03.814 INFO:tasks.workunit.client.0.smithi107.stdout:5/290: dwrite d9/f12 [0,4194304] 0 2024-04-22T23:35:03.814 INFO:tasks.workunit.client.0.smithi107.stdout:9/353: dread d5/d7/f1d [0,4194304] 0 2024-04-22T23:35:03.815 INFO:tasks.workunit.client.1.smithi157.stdout:5/515: read d1/d5/d7/f2e [2253032,13248] 0 2024-04-22T23:35:03.815 INFO:tasks.workunit.client.1.smithi157.stdout:6/789: unlink d1/d17/c1f 0 2024-04-22T23:35:03.818 INFO:tasks.workunit.client.1.smithi157.stdout:3/508: symlink d0/d83/l9d 0 2024-04-22T23:35:03.819 INFO:tasks.workunit.client.0.smithi107.stdout:2/371: truncate d1/d12/d35/d48/d13/f32 1297255 0 2024-04-22T23:35:03.819 INFO:tasks.workunit.client.0.smithi107.stdout:9/354: dread d5/d7/d17/f4e [0,4194304] 0 2024-04-22T23:35:03.821 INFO:tasks.workunit.client.1.smithi157.stdout:9/520: dwrite d4/d15/f4c [0,4194304] 0 2024-04-22T23:35:03.822 INFO:tasks.workunit.client.1.smithi157.stdout:2/641: rename d7/d8/d6b to d7/d8/d95/dd2/dda 0 2024-04-22T23:35:03.822 INFO:tasks.workunit.client.0.smithi107.stdout:7/256: symlink d7/d21/d54/l57 0 2024-04-22T23:35:03.822 INFO:tasks.workunit.client.0.smithi107.stdout:7/257: dread - d7/d2d/f41 zero size 2024-04-22T23:35:03.822 INFO:tasks.workunit.client.0.smithi107.stdout:7/258: stat d7/f2f 0 2024-04-22T23:35:03.823 INFO:tasks.workunit.client.1.smithi157.stdout:3/509: mkdir d0/d3/d34/d62/d9a/d9e 0 2024-04-22T23:35:03.823 INFO:tasks.workunit.client.1.smithi157.stdout:0/471: fsync d17/d2b/d52/fa2 0 2024-04-22T23:35:03.823 INFO:tasks.workunit.client.1.smithi157.stdout:0/472: chown d17/d23/d34 9880 1 2024-04-22T23:35:03.823 INFO:tasks.workunit.client.1.smithi157.stdout:0/473: write d17/f2d [258339,34678] 0 2024-04-22T23:35:03.823 INFO:tasks.workunit.client.1.smithi157.stdout:9/521: mknod d4/df/d57/c9a 0 2024-04-22T23:35:03.824 INFO:tasks.workunit.client.0.smithi107.stdout:3/353: rename db/d21/d27/d15/d29/d2a/l55 to db/d21/d27/d36/l78 0 2024-04-22T23:35:03.824 INFO:tasks.workunit.client.1.smithi157.stdout:2/642: mkdir d7/d1f/d3e/db4/ddb 0 2024-04-22T23:35:03.825 INFO:tasks.workunit.client.1.smithi157.stdout:6/790: rename d1/d36/l5e to d1/d2f/d3b/d49/lf6 0 2024-04-22T23:35:03.829 INFO:tasks.workunit.client.1.smithi157.stdout:3/510: mkdir d0/d3/d13/d21/d9f 0 2024-04-22T23:35:03.829 INFO:tasks.workunit.client.1.smithi157.stdout:3/511: readlink d0/d3/d34/d4a/l5a 0 2024-04-22T23:35:03.829 INFO:tasks.workunit.client.0.smithi107.stdout:5/291: unlink d9/db/d42/f56 0 2024-04-22T23:35:03.829 INFO:tasks.workunit.client.1.smithi157.stdout:0/474: stat d17/d38/l6b 0 2024-04-22T23:35:03.830 INFO:tasks.workunit.client.1.smithi157.stdout:1/551: dwrite d4/d17/d1d/d33/d41/d54/f5b [0,4194304] 0 2024-04-22T23:35:03.830 INFO:tasks.workunit.client.1.smithi157.stdout:1/552: creat d4/d17/d34/d3b/d50/fba x:0 0 0 2024-04-22T23:35:03.830 INFO:tasks.workunit.client.0.smithi107.stdout:2/372: link d1/d2b/d3d/d57/f5c d1/d2b/d3d/d57/f83 0 2024-04-22T23:35:03.831 INFO:tasks.workunit.client.0.smithi107.stdout:2/373: fsync d1/d2b/d59/f6b 0 2024-04-22T23:35:03.831 INFO:tasks.workunit.client.0.smithi107.stdout:2/374: write d1/d12/d35/f45 [1103345,48547] 0 2024-04-22T23:35:03.831 INFO:tasks.workunit.client.1.smithi157.stdout:9/522: link d4/f79 d4/d15/d55/f9b 0 2024-04-22T23:35:03.831 INFO:tasks.workunit.client.1.smithi157.stdout:9/523: dread - d4/df/f8b zero size 2024-04-22T23:35:03.832 INFO:tasks.workunit.client.1.smithi157.stdout:9/524: write d4/df/d52/f7d [569092,87230] 0 2024-04-22T23:35:03.832 INFO:tasks.workunit.client.0.smithi107.stdout:6/308: dwrite db/d20/f4f [0,4194304] 0 2024-04-22T23:35:03.832 INFO:tasks.workunit.client.1.smithi157.stdout:5/516: dwrite d1/d27/d29/d4b/f63 [0,4194304] 0 2024-04-22T23:35:03.835 INFO:tasks.workunit.client.0.smithi107.stdout:7/259: creat d7/d21/d54/f58 x:0 0 0 2024-04-22T23:35:03.836 INFO:tasks.workunit.client.0.smithi107.stdout:3/354: mknod db/d21/d27/d36/c79 0 2024-04-22T23:35:03.836 INFO:tasks.workunit.client.0.smithi107.stdout:5/292: mknod d9/db/d42/c5c 0 2024-04-22T23:35:03.836 INFO:tasks.workunit.client.1.smithi157.stdout:3/512: unlink d0/d3/d34/f93 0 2024-04-22T23:35:03.837 INFO:tasks.workunit.client.0.smithi107.stdout:4/298: dread d1/f28 [0,4194304] 0 2024-04-22T23:35:03.838 INFO:tasks.workunit.client.0.smithi107.stdout:2/375: rmdir d1/d12/d35/d79 0 2024-04-22T23:35:03.840 INFO:tasks.workunit.client.1.smithi157.stdout:0/475: creat d17/d2b/d4f/d9b/fa4 x:0 0 0 2024-04-22T23:35:03.843 INFO:tasks.workunit.client.0.smithi107.stdout:9/355: rmdir d5/d7/d17 39 2024-04-22T23:35:03.845 INFO:tasks.workunit.client.1.smithi157.stdout:1/553: creat d4/d17/d34/d75/fbb x:0 0 0 2024-04-22T23:35:03.847 INFO:tasks.workunit.client.1.smithi157.stdout:9/525: mkdir d4/df/d61/d9c 0 2024-04-22T23:35:03.847 INFO:tasks.workunit.client.1.smithi157.stdout:9/526: write d4/d15/f7a [623685,111454] 0 2024-04-22T23:35:03.849 INFO:tasks.workunit.client.1.smithi157.stdout:2/643: rename d7/d8/d30/d23/d27/d57/c75 to d7/d8/dd/cdc 0 2024-04-22T23:35:03.850 INFO:tasks.workunit.client.1.smithi157.stdout:2/644: write d7/d8/d30/d23/d27/f58 [5729133,10140] 0 2024-04-22T23:35:03.850 INFO:tasks.workunit.client.1.smithi157.stdout:2/645: dread - d7/d8/d30/d5b/fa5 zero size 2024-04-22T23:35:03.850 INFO:tasks.workunit.client.0.smithi107.stdout:3/355: symlink db/d21/d27/d15/d29/d2a/d45/l7a 0 2024-04-22T23:35:03.851 INFO:tasks.workunit.client.0.smithi107.stdout:3/356: fdatasync db/ff 0 2024-04-22T23:35:03.853 INFO:tasks.workunit.client.0.smithi107.stdout:5/293: rename d9/db/de/l1a to d9/l5d 0 2024-04-22T23:35:03.853 INFO:tasks.workunit.client.1.smithi157.stdout:3/513: mknod d0/d7/d6c/ca0 0 2024-04-22T23:35:03.854 INFO:tasks.workunit.client.1.smithi157.stdout:1/554: read d4/d17/d34/d84/d94/fa5 [6723864,87083] 0 2024-04-22T23:35:03.854 INFO:tasks.workunit.client.0.smithi107.stdout:6/309: dwrite db/d20/d2a/d38/f56 [0,4194304] 0 2024-04-22T23:35:03.854 INFO:tasks.workunit.client.1.smithi157.stdout:0/476: mkdir d17/d2b/d71/da5 0 2024-04-22T23:35:03.855 INFO:tasks.workunit.client.1.smithi157.stdout:5/517: dwrite d1/f90 [0,4194304] 0 2024-04-22T23:35:03.855 INFO:tasks.workunit.client.0.smithi107.stdout:4/299: unlink d1/d1b/c1d 0 2024-04-22T23:35:03.857 INFO:tasks.workunit.client.0.smithi107.stdout:2/376: mknod d1/d12/d35/d48/d13/d4f/c84 0 2024-04-22T23:35:03.859 INFO:tasks.workunit.client.0.smithi107.stdout:9/356: link d5/d29/f3a d5/d24/f77 0 2024-04-22T23:35:03.861 INFO:tasks.workunit.client.1.smithi157.stdout:9/527: creat d4/df/d80/f9d x:0 0 0 2024-04-22T23:35:03.861 INFO:tasks.workunit.client.0.smithi107.stdout:3/357: creat db/d21/d27/d54/d25/d53/f7b x:0 0 0 2024-04-22T23:35:03.861 INFO:tasks.workunit.client.0.smithi107.stdout:3/358: fsync db/d21/d27/d15/d29/d2a/f43 0 2024-04-22T23:35:03.862 INFO:tasks.workunit.client.0.smithi107.stdout:5/294: mkdir d9/db/de/d5e 0 2024-04-22T23:35:03.863 INFO:tasks.workunit.client.1.smithi157.stdout:5/518: read d1/db0/d7a/f87 [3736092,89249] 0 2024-04-22T23:35:03.865 INFO:tasks.workunit.client.1.smithi157.stdout:2/646: unlink d7/d8/d30/d5b/fce 0 2024-04-22T23:35:03.866 INFO:tasks.workunit.client.0.smithi107.stdout:6/310: symlink db/d20/d2a/d35/l57 0 2024-04-22T23:35:03.867 INFO:tasks.workunit.client.1.smithi157.stdout:6/791: rename d1/d2f/d3b/d39/d4e/da8/dac/dbe/df5 to d1/d2f/d3b/d39/d4e/df7 0 2024-04-22T23:35:03.867 INFO:tasks.workunit.client.1.smithi157.stdout:6/792: readlink d1/d2f/d54/d9b/l53 0 2024-04-22T23:35:03.868 INFO:tasks.workunit.client.0.smithi107.stdout:4/300: rename d1/d21/d29/d2e/d4d to d1/d7/d53/d66/d6b 0 2024-04-22T23:35:03.868 INFO:tasks.workunit.client.0.smithi107.stdout:4/301: fdatasync d1/d1b/f64 0 2024-04-22T23:35:03.868 INFO:tasks.workunit.client.1.smithi157.stdout:3/514: symlink d0/d3/d34/d47/la1 0 2024-04-22T23:35:03.869 INFO:tasks.workunit.client.1.smithi157.stdout:1/555: truncate d4/db/f3c 3304803 0 2024-04-22T23:35:03.870 INFO:tasks.workunit.client.0.smithi107.stdout:9/357: creat d5/d7/d27/d69/f78 x:0 0 0 2024-04-22T23:35:03.871 INFO:tasks.workunit.client.0.smithi107.stdout:7/260: dwrite d7/d1b/f24 [4194304,4194304] 0 2024-04-22T23:35:03.871 INFO:tasks.workunit.client.0.smithi107.stdout:7/261: fdatasync d7/d21/f32 0 2024-04-22T23:35:03.871 INFO:tasks.workunit.client.0.smithi107.stdout:7/262: read - d7/f2a zero size 2024-04-22T23:35:03.872 INFO:tasks.workunit.client.1.smithi157.stdout:0/477: dwrite d17/d23/d34/d67/d7f/f5f [4194304,4194304] 0 2024-04-22T23:35:03.872 INFO:tasks.workunit.client.1.smithi157.stdout:0/478: write d17/f1a [2981577,90457] 0 2024-04-22T23:35:03.873 INFO:tasks.workunit.client.0.smithi107.stdout:3/359: mknod db/d21/d27/d15/d24/d52/c7c 0 2024-04-22T23:35:03.873 INFO:tasks.workunit.client.0.smithi107.stdout:5/295: dread f8 [4194304,4194304] 0 2024-04-22T23:35:03.875 INFO:tasks.workunit.client.1.smithi157.stdout:9/528: unlink d4/d15/d55/f6e 0 2024-04-22T23:35:03.875 INFO:tasks.workunit.client.1.smithi157.stdout:5/519: dread d1/d5/d7/f72 [0,4194304] 0 2024-04-22T23:35:03.875 INFO:tasks.workunit.client.1.smithi157.stdout:5/520: fdatasync d1/d5/d7/f2e 0 2024-04-22T23:35:03.875 INFO:tasks.workunit.client.0.smithi107.stdout:4/302: creat d1/d7/d5b/f6c x:0 0 0 2024-04-22T23:35:03.877 INFO:tasks.workunit.client.1.smithi157.stdout:6/793: read - d1/d2f/d3b/d39/d4e/da8/dac/de6/fe3 zero size 2024-04-22T23:35:03.877 INFO:tasks.workunit.client.1.smithi157.stdout:6/794: chown d1/d2f/d3b/d39/d4e/da8/de5 79907599 1 2024-04-22T23:35:03.878 INFO:tasks.workunit.client.0.smithi107.stdout:7/263: mknod d7/d21/d54/c59 0 2024-04-22T23:35:03.878 INFO:tasks.workunit.client.0.smithi107.stdout:7/264: write d7/d1b/f4d [426388,83960] 0 2024-04-22T23:35:03.879 INFO:tasks.workunit.client.1.smithi157.stdout:3/515: creat d0/d3/d34/d3d/d57/d60/d9c/fa2 x:0 0 0 2024-04-22T23:35:03.879 INFO:tasks.workunit.client.1.smithi157.stdout:3/516: fsync d0/d3/d34/d3d/d5b/d72/f79 0 2024-04-22T23:35:03.879 INFO:tasks.workunit.client.1.smithi157.stdout:3/517: write d0/d3/d34/d3d/d57/d60/f89 [623620,109462] 0 2024-04-22T23:35:03.879 INFO:tasks.workunit.client.1.smithi157.stdout:9/529: write d4/f8 [3951720,48308] 0 2024-04-22T23:35:03.879 INFO:tasks.workunit.client.0.smithi107.stdout:3/360: truncate db/d21/d27/d15/d29/d2a/d45/f49 4088456 0 2024-04-22T23:35:03.879 INFO:tasks.workunit.client.0.smithi107.stdout:1/307: sync 2024-04-22T23:35:03.879 INFO:tasks.workunit.client.0.smithi107.stdout:1/308: dread - d7/d19/d54/f56 zero size 2024-04-22T23:35:03.881 INFO:tasks.workunit.client.0.smithi107.stdout:4/303: mknod d1/d8/d67/c6d 0 2024-04-22T23:35:03.881 INFO:tasks.workunit.client.1.smithi157.stdout:0/479: write f3 [3216007,82457] 0 2024-04-22T23:35:03.881 INFO:tasks.workunit.client.1.smithi157.stdout:0/480: creat d17/d2b/d52/d7e/fa6 x:0 0 0 2024-04-22T23:35:03.882 INFO:tasks.workunit.client.0.smithi107.stdout:9/358: dread d5/d14/d28/f59 [0,4194304] 0 2024-04-22T23:35:03.885 INFO:tasks.workunit.client.0.smithi107.stdout:7/265: link d7/d2d/f41 d7/d1b/d51/f5a 0 2024-04-22T23:35:03.885 INFO:tasks.workunit.client.0.smithi107.stdout:7/266: truncate d7/d21/f32 5189322 0 2024-04-22T23:35:03.886 INFO:tasks.workunit.client.0.smithi107.stdout:3/361: mkdir db/d21/d27/d54/d25/d6e/d7d 0 2024-04-22T23:35:03.886 INFO:tasks.workunit.client.0.smithi107.stdout:5/296: write d9/db/de/f49 [1867245,24901] 0 2024-04-22T23:35:03.886 INFO:tasks.workunit.client.0.smithi107.stdout:5/297: creat d9/db/f5f x:0 0 0 2024-04-22T23:35:03.886 INFO:tasks.workunit.client.0.smithi107.stdout:5/298: read - d9/f4a zero size 2024-04-22T23:35:03.888 INFO:tasks.workunit.client.0.smithi107.stdout:9/359: mknod d5/d14/d2e/d43/c79 0 2024-04-22T23:35:03.888 INFO:tasks.workunit.client.1.smithi157.stdout:5/521: mknod d1/d5/dc/d43/d5e/cc3 0 2024-04-22T23:35:03.891 INFO:tasks.workunit.client.1.smithi157.stdout:3/518: mknod d0/d3/d34/d62/d9a/d9e/ca3 0 2024-04-22T23:35:03.895 INFO:tasks.workunit.client.0.smithi107.stdout:6/311: dwrite db/f3f [0,4194304] 0 2024-04-22T23:35:03.896 INFO:tasks.workunit.client.0.smithi107.stdout:9/360: rename d5/d7/l32 to d5/d7/d17/d1b/d1f/d36/d53/l7a 0 2024-04-22T23:35:03.896 INFO:tasks.workunit.client.0.smithi107.stdout:9/361: read - d5/d7/d27/d46/f66 zero size 2024-04-22T23:35:03.896 INFO:tasks.workunit.client.0.smithi107.stdout:9/362: chown d5/d14/l2a 1 1 2024-04-22T23:35:03.896 INFO:tasks.workunit.client.0.smithi107.stdout:9/363: truncate d5/d7/fd 5161495 0 2024-04-22T23:35:03.897 INFO:tasks.workunit.client.0.smithi107.stdout:3/362: getdents db 0 2024-04-22T23:35:03.897 INFO:tasks.workunit.client.0.smithi107.stdout:3/363: read db/d21/d27/f39 [2939724,117348] 0 2024-04-22T23:35:03.898 INFO:tasks.workunit.client.0.smithi107.stdout:4/304: read d1/f13 [4424780,97962] 0 2024-04-22T23:35:03.899 INFO:tasks.workunit.client.1.smithi157.stdout:9/530: read d4/df/d57/f22 [3398688,9145] 0 2024-04-22T23:35:03.899 INFO:tasks.workunit.client.0.smithi107.stdout:7/267: dread d7/d1b/f24 [0,4194304] 0 2024-04-22T23:35:03.903 INFO:tasks.workunit.client.0.smithi107.stdout:9/364: unlink f1 0 2024-04-22T23:35:03.903 INFO:tasks.workunit.client.0.smithi107.stdout:9/365: read - d5/d7/d27/d46/d4b/f56 zero size 2024-04-22T23:35:03.903 INFO:tasks.workunit.client.1.smithi157.stdout:3/519: write d0/d3/d34/d47/f6d [2257887,40096] 0 2024-04-22T23:35:03.903 INFO:tasks.workunit.client.1.smithi157.stdout:3/520: chown d0/f2f 69 1 2024-04-22T23:35:03.904 INFO:tasks.workunit.client.0.smithi107.stdout:3/364: creat db/d21/d27/d36/f7e x:0 0 0 2024-04-22T23:35:03.905 INFO:tasks.workunit.client.1.smithi157.stdout:5/522: unlink d1/cd 0 2024-04-22T23:35:03.905 INFO:tasks.workunit.client.0.smithi107.stdout:4/305: mknod d1/d21/d29/d2e/d3b/c6e 0 2024-04-22T23:35:03.907 INFO:tasks.workunit.client.1.smithi157.stdout:1/556: rename d4/db/d45 to d4/d17/d34/d3b/d52/d65/dbc 0 2024-04-22T23:35:03.910 INFO:tasks.workunit.client.1.smithi157.stdout:9/531: link d4/c36 d4/d15/d55/d8d/c9e 0 2024-04-22T23:35:03.910 INFO:tasks.workunit.client.0.smithi107.stdout:3/365: unlink db/d21/l44 0 2024-04-22T23:35:03.910 INFO:tasks.workunit.client.0.smithi107.stdout:4/306: link d1/d1b/f49 d1/d7/d53/f6f 0 2024-04-22T23:35:03.910 INFO:tasks.workunit.client.0.smithi107.stdout:9/366: getdents d5/d14 0 2024-04-22T23:35:03.912 INFO:tasks.workunit.client.0.smithi107.stdout:3/366: rename db/d21/d27/d54/d25/d6e/d7d to db/d21/d7f 0 2024-04-22T23:35:03.912 INFO:tasks.workunit.client.0.smithi107.stdout:4/307: rmdir d1/d21/d29 39 2024-04-22T23:35:03.913 INFO:tasks.workunit.client.1.smithi157.stdout:3/521: link d0/d3/d34/d39/l2d d0/la4 0 2024-04-22T23:35:03.913 INFO:tasks.workunit.client.1.smithi157.stdout:3/522: fsync d0/d3/f1e 0 2024-04-22T23:35:03.914 INFO:tasks.workunit.client.0.smithi107.stdout:6/312: write db/f2d [688380,93710] 0 2024-04-22T23:35:03.915 INFO:tasks.workunit.client.0.smithi107.stdout:9/367: mkdir d5/d29/d7b 0 2024-04-22T23:35:03.915 INFO:tasks.workunit.client.0.smithi107.stdout:9/368: write d5/d7/d27/d46/f74 [179000,101380] 0 2024-04-22T23:35:03.916 INFO:tasks.workunit.client.1.smithi157.stdout:1/557: creat d4/d17/d34/d3b/d52/d65/dae/fbd x:0 0 0 2024-04-22T23:35:03.917 INFO:tasks.workunit.client.0.smithi107.stdout:3/367: getdents db/d21/d27/d15/d29/d2a/d45 0 2024-04-22T23:35:03.918 INFO:tasks.workunit.client.1.smithi157.stdout:9/532: symlink d4/l9f 0 2024-04-22T23:35:03.918 INFO:tasks.workunit.client.1.smithi157.stdout:9/533: chown d4/df/d57/l3c 64065 1 2024-04-22T23:35:03.919 INFO:tasks.workunit.client.0.smithi107.stdout:4/308: creat d1/d21/d52/f70 x:0 0 0 2024-04-22T23:35:03.919 INFO:tasks.workunit.client.0.smithi107.stdout:4/309: write d1/d7/fc [5221657,13248] 0 2024-04-22T23:35:03.919 INFO:tasks.workunit.client.0.smithi107.stdout:4/310: dread - d1/d8/f4b zero size 2024-04-22T23:35:03.921 INFO:tasks.workunit.client.0.smithi107.stdout:9/369: link d5/d7/d17/d1b/d1f/d36/d53/d55/l62 d5/d14/d2e/d4f/d73/l7c 0 2024-04-22T23:35:03.922 INFO:tasks.workunit.client.0.smithi107.stdout:9/370: chown d5/d7/d17/l6b 305035 1 2024-04-22T23:35:03.922 INFO:tasks.workunit.client.0.smithi107.stdout:9/371: truncate d5/d7/d17/f4e 1353706 0 2024-04-22T23:35:03.922 INFO:tasks.workunit.client.0.smithi107.stdout:9/372: creat d5/d24/f7d x:0 0 0 2024-04-22T23:35:03.922 INFO:tasks.workunit.client.1.smithi157.stdout:5/523: rename d1/d5/dc/l5b to d1/d27/lc4 0 2024-04-22T23:35:03.922 INFO:tasks.workunit.client.1.smithi157.stdout:5/524: readlink d1/d5/l71 0 2024-04-22T23:35:03.923 INFO:tasks.workunit.client.1.smithi157.stdout:9/534: symlink d4/df/d80/la0 0 2024-04-22T23:35:03.923 INFO:tasks.workunit.client.0.smithi107.stdout:5/299: dwrite d9/f4a [0,4194304] 0 2024-04-22T23:35:03.925 INFO:tasks.workunit.client.1.smithi157.stdout:9/535: symlink d4/d15/d23/d27/d75/la1 0 2024-04-22T23:35:03.929 INFO:tasks.workunit.client.0.smithi107.stdout:4/311: creat d1/d21/d29/f71 x:0 0 0 2024-04-22T23:35:03.929 INFO:tasks.workunit.client.0.smithi107.stdout:4/312: write d1/d1b/f49 [4813516,59977] 0 2024-04-22T23:35:03.929 INFO:tasks.workunit.client.0.smithi107.stdout:4/313: write d1/d7/d53/d66/d6b/f4e [440446,41738] 0 2024-04-22T23:35:03.931 INFO:tasks.workunit.client.0.smithi107.stdout:6/313: rmdir db/d20/d2a 39 2024-04-22T23:35:03.932 INFO:tasks.workunit.client.0.smithi107.stdout:6/314: chown db/d20/d2c/d3b/f45 8098836 1 2024-04-22T23:35:03.934 INFO:tasks.workunit.client.0.smithi107.stdout:9/373: rename d5/d7/d27/c65 to d5/d7/d27/d46/c7e 0 2024-04-22T23:35:03.935 INFO:tasks.workunit.client.1.smithi157.stdout:2/647: write d7/d8/fa4 [1494083,121058] 0 2024-04-22T23:35:03.935 INFO:tasks.workunit.client.1.smithi157.stdout:9/536: write d4/d15/d23/d27/d75/f82 [3764855,75395] 0 2024-04-22T23:35:03.935 INFO:tasks.workunit.client.1.smithi157.stdout:9/537: truncate d4/d15/d23/f97 996385 0 2024-04-22T23:35:03.936 INFO:tasks.workunit.client.0.smithi107.stdout:3/368: rmdir db/d21/d27/d15/d29/d2a/d40 39 2024-04-22T23:35:03.937 INFO:tasks.workunit.client.1.smithi157.stdout:9/538: mknod d4/d15/d23/d67/ca2 0 2024-04-22T23:35:03.937 INFO:tasks.workunit.client.1.smithi157.stdout:9/539: write d4/d15/d23/f8f [324627,13643] 0 2024-04-22T23:35:03.937 INFO:tasks.workunit.client.1.smithi157.stdout:9/540: read - d4/d15/d55/f9b zero size 2024-04-22T23:35:03.938 INFO:tasks.workunit.client.1.smithi157.stdout:9/541: mkdir d4/d15/da3 0 2024-04-22T23:35:03.938 INFO:tasks.workunit.client.1.smithi157.stdout:9/542: stat d4/df/ce 0 2024-04-22T23:35:03.939 INFO:tasks.workunit.client.0.smithi107.stdout:3/369: creat db/d21/d27/d15/d29/d2a/d40/f80 x:0 0 0 2024-04-22T23:35:03.939 INFO:tasks.workunit.client.1.smithi157.stdout:9/543: mknod d4/d15/d23/ca4 0 2024-04-22T23:35:03.941 INFO:tasks.workunit.client.0.smithi107.stdout:3/370: truncate db/d21/d27/f41 282931 0 2024-04-22T23:35:03.942 INFO:tasks.workunit.client.1.smithi157.stdout:9/544: symlink d4/df/la5 0 2024-04-22T23:35:03.944 INFO:tasks.workunit.client.1.smithi157.stdout:6/795: dwrite d1/d2f/d3b/d39/d4e/da8/de5/d83/f59 [0,4194304] 0 2024-04-22T23:35:03.944 INFO:tasks.workunit.client.1.smithi157.stdout:6/796: dread - d1/d36/fd5 zero size 2024-04-22T23:35:03.944 INFO:tasks.workunit.client.0.smithi107.stdout:1/309: dwrite d7/d19/d20/f38 [0,4194304] 0 2024-04-22T23:35:03.945 INFO:tasks.workunit.client.1.smithi157.stdout:6/797: creat d1/d2f/d3b/d39/d4e/ff8 x:0 0 0 2024-04-22T23:35:03.945 INFO:tasks.workunit.client.1.smithi157.stdout:6/798: stat d1/d2f/d3b/d39/d4e/c72 0 2024-04-22T23:35:03.945 INFO:tasks.workunit.client.1.smithi157.stdout:0/481: dwrite d17/d38/f80 [4194304,4194304] 0 2024-04-22T23:35:03.945 INFO:tasks.workunit.client.1.smithi157.stdout:0/482: fsync d17/d38/f56 0 2024-04-22T23:35:03.945 INFO:tasks.workunit.client.1.smithi157.stdout:0/483: chown d17/d2b/d52/l5d 0 1 2024-04-22T23:35:03.946 INFO:tasks.workunit.client.1.smithi157.stdout:6/799: creat d1/d17/d5d/ff9 x:0 0 0 2024-04-22T23:35:03.946 INFO:tasks.workunit.client.0.smithi107.stdout:1/310: unlink d7/d19/d54/f56 0 2024-04-22T23:35:03.947 INFO:tasks.workunit.client.0.smithi107.stdout:1/311: symlink d7/d19/d2d/l6c 0 2024-04-22T23:35:03.947 INFO:tasks.workunit.client.0.smithi107.stdout:1/312: write d7/d19/d20/f38 [4970883,110354] 0 2024-04-22T23:35:03.947 INFO:tasks.workunit.client.0.smithi107.stdout:1/313: write d7/f67 [247963,1574] 0 2024-04-22T23:35:03.947 INFO:tasks.workunit.client.0.smithi107.stdout:1/314: write d7/d19/d20/d39/d53/d43/d46/d5c/f5e [51626,130216] 0 2024-04-22T23:35:03.950 INFO:tasks.workunit.client.0.smithi107.stdout:5/300: dwrite d9/f58 [0,4194304] 0 2024-04-22T23:35:03.951 INFO:tasks.workunit.client.1.smithi157.stdout:3/523: dwrite d0/d3/d34/f5d [0,4194304] 0 2024-04-22T23:35:03.951 INFO:tasks.workunit.client.1.smithi157.stdout:5/525: dwrite d1/d5/d7/f72 [0,4194304] 0 2024-04-22T23:35:03.951 INFO:tasks.workunit.client.1.smithi157.stdout:5/526: write d1/d5/f46 [2502457,102281] 0 2024-04-22T23:35:03.956 INFO:tasks.workunit.client.1.smithi157.stdout:0/484: symlink d17/d2b/d4f/d5a/da1/la7 0 2024-04-22T23:35:03.960 INFO:tasks.workunit.client.1.smithi157.stdout:6/800: creat d1/d17/ffa x:0 0 0 2024-04-22T23:35:03.962 INFO:tasks.workunit.client.1.smithi157.stdout:0/485: truncate d17/d38/f73 4021389 0 2024-04-22T23:35:03.964 INFO:tasks.workunit.client.0.smithi107.stdout:5/301: link d9/db/f27 d9/f60 0 2024-04-22T23:35:03.971 INFO:tasks.workunit.client.0.smithi107.stdout:6/315: dwrite db/f29 [4194304,4194304] 0 2024-04-22T23:35:03.982 INFO:tasks.workunit.client.1.smithi157.stdout:9/545: dwrite d4/d15/d23/f8f [0,4194304] 0 2024-04-22T23:35:03.984 INFO:tasks.workunit.client.0.smithi107.stdout:5/302: rename d9/db/de/f49 to d9/db/de/d5e/f61 0 2024-04-22T23:35:03.985 INFO:tasks.workunit.client.1.smithi157.stdout:9/546: getdents d4/df/d57 0 2024-04-22T23:35:03.985 INFO:tasks.workunit.client.1.smithi157.stdout:9/547: write d4/d15/d23/d27/d75/f82 [2680794,102436] 0 2024-04-22T23:35:03.985 INFO:tasks.workunit.client.0.smithi107.stdout:1/315: dwrite d7/d19/d20/f41 [0,4194304] 0 2024-04-22T23:35:03.986 INFO:tasks.workunit.client.0.smithi107.stdout:1/316: chown d7/d19/d2d/l6c 430517 1 2024-04-22T23:35:03.986 INFO:tasks.workunit.client.0.smithi107.stdout:1/317: creat d7/d19/d42/f6d x:0 0 0 2024-04-22T23:35:03.986 INFO:tasks.workunit.client.0.smithi107.stdout:1/318: chown d7/d19/d2d/f3d 479 1 2024-04-22T23:35:03.987 INFO:tasks.workunit.client.1.smithi157.stdout:9/548: symlink d4/df/d80/la6 0 2024-04-22T23:35:03.988 INFO:tasks.workunit.client.0.smithi107.stdout:6/316: mkdir db/d20/d58 0 2024-04-22T23:35:03.990 INFO:tasks.workunit.client.1.smithi157.stdout:9/549: rename d4/l49 to d4/d15/d23/la7 0 2024-04-22T23:35:03.990 INFO:tasks.workunit.client.1.smithi157.stdout:9/550: fdatasync d4/df/d57/f2b 0 2024-04-22T23:35:03.990 INFO:tasks.workunit.client.1.smithi157.stdout:9/551: chown d4/f8 4134 1 2024-04-22T23:35:03.990 INFO:tasks.workunit.client.1.smithi157.stdout:9/552: fsync d4/df/d61/f84 0 2024-04-22T23:35:03.991 INFO:tasks.workunit.client.0.smithi107.stdout:4/314: truncate d1/d7/d53/d66/d6b/f4e 348030 0 2024-04-22T23:35:03.991 INFO:tasks.workunit.client.1.smithi157.stdout:3/524: dwrite d0/d3/d34/d3d/f44 [0,4194304] 0 2024-04-22T23:35:03.992 INFO:tasks.workunit.client.1.smithi157.stdout:9/553: mknod d4/df/d57/ca8 0 2024-04-22T23:35:03.993 INFO:tasks.workunit.client.0.smithi107.stdout:6/317: dread db/f1d [0,4194304] 0 2024-04-22T23:35:03.993 INFO:tasks.workunit.client.0.smithi107.stdout:6/318: stat db/d20/d2c/d3b/f4d 0 2024-04-22T23:35:03.993 INFO:tasks.workunit.client.0.smithi107.stdout:6/319: getdents db/d20/d2c/d3b/d40 0 2024-04-22T23:35:03.994 INFO:tasks.workunit.client.0.smithi107.stdout:4/315: dread d1/f28 [0,4194304] 0 2024-04-22T23:35:03.994 INFO:tasks.workunit.client.0.smithi107.stdout:4/316: write d1/d21/d29/f71 [979338,128221] 0 2024-04-22T23:35:03.994 INFO:tasks.workunit.client.0.smithi107.stdout:4/317: write d1/f13 [3284093,93465] 0 2024-04-22T23:35:03.994 INFO:tasks.workunit.client.0.smithi107.stdout:1/319: link d7/d19/d20/d39/d53/d43/d46/f47 d7/d19/d42/f6e 0 2024-04-22T23:35:03.995 INFO:tasks.workunit.client.1.smithi157.stdout:0/486: dwrite d17/d23/f32 [0,4194304] 0 2024-04-22T23:35:03.995 INFO:tasks.workunit.client.1.smithi157.stdout:0/487: readlink d17/d2b/d52/l5d 0 2024-04-22T23:35:03.995 INFO:tasks.workunit.client.1.smithi157.stdout:0/488: fdatasync d17/f20 0 2024-04-22T23:35:03.995 INFO:tasks.workunit.client.1.smithi157.stdout:0/489: chown d17/f2d 1 1 2024-04-22T23:35:03.995 INFO:tasks.workunit.client.0.smithi107.stdout:6/320: truncate db/d20/d2a/d38/f56 3380293 0 2024-04-22T23:35:03.995 INFO:tasks.workunit.client.0.smithi107.stdout:6/321: truncate db/f48 358293 0 2024-04-22T23:35:03.997 INFO:tasks.workunit.client.0.smithi107.stdout:1/320: link d7/f1b d7/d19/d42/f6f 0 2024-04-22T23:35:03.998 INFO:tasks.workunit.client.0.smithi107.stdout:5/303: write d9/db/f13 [335841,10639] 0 2024-04-22T23:35:03.999 INFO:tasks.workunit.client.1.smithi157.stdout:6/801: dwrite d1/d2f/d54/d9b/f79 [0,4194304] 0 2024-04-22T23:35:03.999 INFO:tasks.workunit.client.1.smithi157.stdout:5/527: dwrite d1/d5/f46 [0,4194304] 0 2024-04-22T23:35:04.000 INFO:tasks.workunit.client.0.smithi107.stdout:1/321: creat d7/d19/d54/f70 x:0 0 0 2024-04-22T23:35:04.000 INFO:tasks.workunit.client.0.smithi107.stdout:1/322: getdents d7/d19/d20/d39/d53/d43/d46/d68 0 2024-04-22T23:35:04.000 INFO:tasks.workunit.client.0.smithi107.stdout:1/323: write d7/d19/d42/f6d [1044568,9723] 0 2024-04-22T23:35:04.000 INFO:tasks.workunit.client.0.smithi107.stdout:1/324: write d7/d19/d20/d39/f4b [998674,14338] 0 2024-04-22T23:35:04.000 INFO:tasks.workunit.client.0.smithi107.stdout:5/304: symlink d9/db/de/l62 0 2024-04-22T23:35:04.002 INFO:tasks.workunit.client.0.smithi107.stdout:1/325: mkdir d7/d19/d71 0 2024-04-22T23:35:04.002 INFO:tasks.workunit.client.0.smithi107.stdout:1/326: write d7/f10 [4868498,87241] 0 2024-04-22T23:35:04.002 INFO:tasks.workunit.client.1.smithi157.stdout:9/554: unlink d4/d15/d23/l85 0 2024-04-22T23:35:04.002 INFO:tasks.workunit.client.1.smithi157.stdout:9/555: getdents d4/df/d57/d6b 0 2024-04-22T23:35:04.003 INFO:tasks.workunit.client.0.smithi107.stdout:5/305: creat d9/db/de/f63 x:0 0 0 2024-04-22T23:35:04.006 INFO:tasks.workunit.client.0.smithi107.stdout:5/306: unlink d9/l17 0 2024-04-22T23:35:04.006 INFO:tasks.workunit.client.0.smithi107.stdout:5/307: write d9/db/f1d [1950618,16285] 0 2024-04-22T23:35:04.006 INFO:tasks.workunit.client.1.smithi157.stdout:0/490: rename d17/d38/d6a to d17/d2b/d4f/d5a/da1/da8 0 2024-04-22T23:35:04.006 INFO:tasks.workunit.client.1.smithi157.stdout:0/491: readlink d17/d2b/d52/l5d 0 2024-04-22T23:35:04.006 INFO:tasks.workunit.client.1.smithi157.stdout:0/492: write d17/d2b/d4f/d9b/fa4 [539838,20047] 0 2024-04-22T23:35:04.007 INFO:tasks.workunit.client.0.smithi107.stdout:5/308: rename d9/db/f39 to d9/d26/f64 0 2024-04-22T23:35:04.008 INFO:tasks.workunit.client.0.smithi107.stdout:5/309: write d9/f36 [444276,57551] 0 2024-04-22T23:35:04.009 INFO:tasks.workunit.client.0.smithi107.stdout:5/310: symlink d9/db/d42/l65 0 2024-04-22T23:35:04.010 INFO:tasks.workunit.client.1.smithi157.stdout:6/802: mknod d1/d2f/d3b/d39/d4e/da8/dac/de6/cfb 0 2024-04-22T23:35:04.010 INFO:tasks.workunit.client.1.smithi157.stdout:6/803: fsync d1/d36/f57 0 2024-04-22T23:35:04.010 INFO:tasks.workunit.client.1.smithi157.stdout:6/804: read d1/f34 [135821,67111] 0 2024-04-22T23:35:04.011 INFO:tasks.workunit.client.0.smithi107.stdout:5/311: creat d9/db/de/d5e/f66 x:0 0 0 2024-04-22T23:35:04.014 INFO:tasks.workunit.client.1.smithi157.stdout:9/556: symlink d4/d15/la9 0 2024-04-22T23:35:04.015 INFO:tasks.workunit.client.0.smithi107.stdout:4/318: dwrite d1/d7/f33 [0,4194304] 0 2024-04-22T23:35:04.016 INFO:tasks.workunit.client.1.smithi157.stdout:0/493: symlink d17/d23/d9c/la9 0 2024-04-22T23:35:04.018 INFO:tasks.workunit.client.0.smithi107.stdout:1/327: dread d7/d19/d42/f6d [0,4194304] 0 2024-04-22T23:35:04.018 INFO:tasks.workunit.client.0.smithi107.stdout:1/328: truncate d7/d19/d20/d39/d53/f33 106443 0 2024-04-22T23:35:04.018 INFO:tasks.workunit.client.0.smithi107.stdout:1/329: chown d7/d19/d54/f70 550126716 1 2024-04-22T23:35:04.018 INFO:tasks.workunit.client.0.smithi107.stdout:1/330: fsync d7/d19/d20/d39/d53/f5f 0 2024-04-22T23:35:04.019 INFO:tasks.workunit.client.1.smithi157.stdout:6/805: rename d1/f18 to d1/d2f/d54/d9b/dda/ffc 0 2024-04-22T23:35:04.019 INFO:tasks.workunit.client.1.smithi157.stdout:5/528: dread d1/d5/d7/d10/f49 [0,4194304] 0 2024-04-22T23:35:04.020 INFO:tasks.workunit.client.1.smithi157.stdout:9/557: unlink d4/d15/d23/d67/l6c 0 2024-04-22T23:35:04.020 INFO:tasks.workunit.client.1.smithi157.stdout:9/558: fdatasync d4/d15/d23/f8c 0 2024-04-22T23:35:04.021 INFO:tasks.workunit.client.1.smithi157.stdout:6/806: symlink d1/d2f/d3b/d39/d4e/da8/dac/db0/lfd 0 2024-04-22T23:35:04.026 INFO:tasks.workunit.client.1.smithi157.stdout:9/559: rename d4/df/d57/f22 to d4/d15/faa 0 2024-04-22T23:35:04.026 INFO:tasks.workunit.client.1.smithi157.stdout:6/807: rmdir d1/d2f/d54/d9b/dda 39 2024-04-22T23:35:04.028 INFO:tasks.workunit.client.0.smithi107.stdout:4/319: creat d1/d7/d63/f72 x:0 0 0 2024-04-22T23:35:04.035 INFO:tasks.workunit.client.1.smithi157.stdout:4/632: sync 2024-04-22T23:35:04.035 INFO:tasks.workunit.client.1.smithi157.stdout:7/462: sync 2024-04-22T23:35:04.035 INFO:tasks.workunit.client.1.smithi157.stdout:8/368: sync 2024-04-22T23:35:04.037 INFO:tasks.workunit.client.1.smithi157.stdout:6/808: symlink d1/d2f/d3b/d39/d4e/da8/dac/ddd/ddf/lfe 0 2024-04-22T23:35:04.039 INFO:tasks.workunit.client.1.smithi157.stdout:4/633: mknod d3/d11/d1d/d73/d80/dac/db5/cd5 0 2024-04-22T23:35:04.041 INFO:tasks.workunit.client.0.smithi107.stdout:8/247: sync 2024-04-22T23:35:04.042 INFO:tasks.workunit.client.1.smithi157.stdout:7/463: rename d0/d9/db/df/l88 to d0/d9/db/df/l99 0 2024-04-22T23:35:04.042 INFO:tasks.workunit.client.0.smithi107.stdout:8/248: mkdir d0/d7/df/d18/d51 0 2024-04-22T23:35:04.042 INFO:tasks.workunit.client.0.smithi107.stdout:8/249: write d0/f2f [4728216,108977] 0 2024-04-22T23:35:04.044 INFO:tasks.workunit.client.1.smithi157.stdout:4/634: mknod d3/d11/d1d/d73/d80/dac/db5/dc8/cd6 0 2024-04-22T23:35:04.044 INFO:tasks.workunit.client.1.smithi157.stdout:6/809: dread d1/d2f/d3b/d39/d4e/da8/dac/db0/fdb [0,4194304] 0 2024-04-22T23:35:04.044 INFO:tasks.workunit.client.1.smithi157.stdout:6/810: fsync d1/d17/f43 0 2024-04-22T23:35:04.045 INFO:tasks.workunit.client.0.smithi107.stdout:8/250: creat d0/d7/df/f52 x:0 0 0 2024-04-22T23:35:04.045 INFO:tasks.workunit.client.1.smithi157.stdout:9/560: write d4/df/fb [3247922,32021] 0 2024-04-22T23:35:04.045 INFO:tasks.workunit.client.1.smithi157.stdout:7/464: symlink d0/d9/d29/d98/l9a 0 2024-04-22T23:35:04.046 INFO:tasks.workunit.client.1.smithi157.stdout:7/465: chown d0/f2 7 1 2024-04-22T23:35:04.046 INFO:tasks.workunit.client.0.smithi107.stdout:8/251: mkdir d0/d7/d8/d41/d53 0 2024-04-22T23:35:04.046 INFO:tasks.workunit.client.0.smithi107.stdout:8/252: chown d0/d7/df/d33/c4d 14880 1 2024-04-22T23:35:04.047 INFO:tasks.workunit.client.1.smithi157.stdout:6/811: creat d1/d36/dea/fff x:0 0 0 2024-04-22T23:35:04.053 INFO:tasks.workunit.client.1.smithi157.stdout:3/525: truncate d0/d3/d34/f5d 6044733 0 2024-04-22T23:35:04.054 INFO:tasks.workunit.client.1.smithi157.stdout:8/369: dread d2/d6/d21/d5f/d11/f4a [0,4194304] 0 2024-04-22T23:35:04.054 INFO:tasks.workunit.client.1.smithi157.stdout:8/370: write d2/d6/d21/d5f/f1b [86948,109584] 0 2024-04-22T23:35:04.058 INFO:tasks.workunit.client.1.smithi157.stdout:3/526: creat d0/d7/d95/fa5 x:0 0 0 2024-04-22T23:35:04.059 INFO:tasks.workunit.client.1.smithi157.stdout:3/527: dread - d0/d3/d34/d3d/d5b/d72/f79 zero size 2024-04-22T23:35:04.059 INFO:tasks.workunit.client.1.smithi157.stdout:9/561: symlink d4/df/lab 0 2024-04-22T23:35:04.060 INFO:tasks.workunit.client.1.smithi157.stdout:8/371: rmdir d2/d6/d21/d5f/d17/d7e 0 2024-04-22T23:35:04.061 INFO:tasks.workunit.client.1.smithi157.stdout:8/372: creat d2/d34/d6a/f81 x:0 0 0 2024-04-22T23:35:04.063 INFO:tasks.workunit.client.1.smithi157.stdout:3/528: dread d0/d3/d34/d3d/d57/f65 [0,4194304] 0 2024-04-22T23:35:04.063 INFO:tasks.workunit.client.1.smithi157.stdout:3/529: read d0/d3/d34/d3d/f69 [225970,80526] 0 2024-04-22T23:35:04.063 INFO:tasks.workunit.client.1.smithi157.stdout:3/530: truncate d0/d3/d34/d3d/d5b/f84 411374 0 2024-04-22T23:35:04.063 INFO:tasks.workunit.client.1.smithi157.stdout:3/531: write d0/d3/d34/d3d/d57/d60/f89 [1354032,76726] 0 2024-04-22T23:35:04.063 INFO:tasks.workunit.client.1.smithi157.stdout:8/373: creat d2/d34/d6a/d77/f82 x:0 0 0 2024-04-22T23:35:04.063 INFO:tasks.workunit.client.1.smithi157.stdout:8/374: stat d2/d6/d21/d5f/d11/d13 0 2024-04-22T23:35:04.064 INFO:tasks.workunit.client.1.smithi157.stdout:3/532: link d0/d3/d34/d3d/d57/d60/c71 d0/d3/d34/d3d/ca6 0 2024-04-22T23:35:04.065 INFO:tasks.workunit.client.1.smithi157.stdout:8/375: symlink d2/d4/l83 0 2024-04-22T23:35:04.067 INFO:tasks.workunit.client.1.smithi157.stdout:3/533: link d0/d3/d34/d3d/d57/d60/c71 d0/d3/d34/d3d/d57/d60/ca7 0 2024-04-22T23:35:04.069 INFO:tasks.workunit.client.1.smithi157.stdout:3/534: creat d0/d3/d34/d4a/fa8 x:0 0 0 2024-04-22T23:35:04.074 INFO:tasks.workunit.client.1.smithi157.stdout:9/562: dread d4/f20 [0,4194304] 0 2024-04-22T23:35:04.075 INFO:tasks.workunit.client.0.smithi107.stdout:1/331: dwrite d7/d19/d20/f38 [0,4194304] 0 2024-04-22T23:35:04.076 INFO:tasks.workunit.client.1.smithi157.stdout:9/563: rename d4/d15/f4c to d4/df/d52/fac 0 2024-04-22T23:35:04.076 INFO:tasks.workunit.client.0.smithi107.stdout:1/332: mknod d7/d19/d20/d39/d53/d43/d46/c72 0 2024-04-22T23:35:04.076 INFO:tasks.workunit.client.1.smithi157.stdout:0/494: dwrite d17/f1b [0,4194304] 0 2024-04-22T23:35:04.077 INFO:tasks.workunit.client.1.smithi157.stdout:9/564: mkdir d4/d15/d23/dad 0 2024-04-22T23:35:04.077 INFO:tasks.workunit.client.1.smithi157.stdout:9/565: chown d4/d15/d23/d27/d75/la1 213 1 2024-04-22T23:35:04.078 INFO:tasks.workunit.client.1.smithi157.stdout:9/566: readlink d4/l28 0 2024-04-22T23:35:04.078 INFO:tasks.workunit.client.1.smithi157.stdout:9/567: fsync d4/df/f26 0 2024-04-22T23:35:04.078 INFO:tasks.workunit.client.1.smithi157.stdout:9/568: read d4/d15/d23/d27/f33 [3982770,81527] 0 2024-04-22T23:35:04.078 INFO:tasks.workunit.client.1.smithi157.stdout:9/569: chown d4/df/d52/f7d 0 1 2024-04-22T23:35:04.078 INFO:tasks.workunit.client.0.smithi107.stdout:4/320: dwrite d1/f28 [0,4194304] 0 2024-04-22T23:35:04.078 INFO:tasks.workunit.client.0.smithi107.stdout:1/333: truncate d7/d19/d20/d39/d53/f4e 661428 0 2024-04-22T23:35:04.079 INFO:tasks.workunit.client.0.smithi107.stdout:1/334: unlink d7/d19/d20/d39/d53/c1e 0 2024-04-22T23:35:04.079 INFO:tasks.workunit.client.0.smithi107.stdout:1/335: chown d7/d19/d20 705829843 1 2024-04-22T23:35:04.079 INFO:tasks.workunit.client.1.smithi157.stdout:5/529: dwrite d1/d27/d29/d4b/f5c [0,4194304] 0 2024-04-22T23:35:04.079 INFO:tasks.workunit.client.1.smithi157.stdout:9/570: mknod d4/d15/d23/cae 0 2024-04-22T23:35:04.080 INFO:tasks.workunit.client.0.smithi107.stdout:5/312: dwrite d9/db/de/d5e/f61 [0,4194304] 0 2024-04-22T23:35:04.081 INFO:tasks.workunit.client.1.smithi157.stdout:6/812: dwrite d1/f12 [0,4194304] 0 2024-04-22T23:35:04.081 INFO:tasks.workunit.client.0.smithi107.stdout:4/321: getdents d1/d7 0 2024-04-22T23:35:04.081 INFO:tasks.workunit.client.1.smithi157.stdout:7/466: dwrite d0/d9/f7d [0,4194304] 0 2024-04-22T23:35:04.083 INFO:tasks.workunit.client.0.smithi107.stdout:1/336: rename d7/d19/d20/d39/d53/f62 to d7/f73 0 2024-04-22T23:35:04.083 INFO:tasks.workunit.client.0.smithi107.stdout:1/337: fsync d7/d19/d20/d39/f5a 0 2024-04-22T23:35:04.083 INFO:tasks.workunit.client.0.smithi107.stdout:8/253: dwrite d0/d7/df/d3b/f3c [0,4194304] 0 2024-04-22T23:35:04.084 INFO:tasks.workunit.client.1.smithi157.stdout:8/376: dwrite d2/d6/d21/d5f/d11/d13/f7d [0,4194304] 0 2024-04-22T23:35:04.084 INFO:tasks.workunit.client.1.smithi157.stdout:8/377: write d2/d6/d66/d67/f79 [611254,126271] 0 2024-04-22T23:35:04.086 INFO:tasks.workunit.client.1.smithi157.stdout:5/530: symlink d1/d27/d29/d4b/lc5 0 2024-04-22T23:35:04.087 INFO:tasks.workunit.client.0.smithi107.stdout:1/338: rename d7/d19/d20/f24 to d7/d19/d20/d39/d53/d43/d63/f74 0 2024-04-22T23:35:04.087 INFO:tasks.workunit.client.0.smithi107.stdout:1/339: fsync d7/d19/d20/d39/d53/d43/d46/f4c 0 2024-04-22T23:35:04.091 INFO:tasks.workunit.client.1.smithi157.stdout:9/571: rename d4/d15/d23/ca4 to d4/d15/d23/d27/d75/caf 0 2024-04-22T23:35:04.093 INFO:tasks.workunit.client.1.smithi157.stdout:6/813: link d1/f3 d1/d2f/d3b/d39/d4e/da8/dac/dc0/f100 0 2024-04-22T23:35:04.100 INFO:tasks.workunit.client.0.smithi107.stdout:8/254: dread d0/d7/df/d17/f40 [0,4194304] 0 2024-04-22T23:35:04.100 INFO:tasks.workunit.client.0.smithi107.stdout:8/255: fdatasync d0/d7/df/f52 0 2024-04-22T23:35:04.101 INFO:tasks.workunit.client.0.smithi107.stdout:8/256: symlink d0/d7/df/d18/l54 0 2024-04-22T23:35:04.101 INFO:tasks.workunit.client.1.smithi157.stdout:7/467: creat d0/d9/d29/d33/d37/f9b x:0 0 0 2024-04-22T23:35:04.102 INFO:tasks.workunit.client.0.smithi107.stdout:8/257: link d0/d7/d20/d2d/c42 d0/d7/d20/d2d/c55 0 2024-04-22T23:35:04.102 INFO:tasks.workunit.client.0.smithi107.stdout:8/258: creat d0/d3a/f56 x:0 0 0 2024-04-22T23:35:04.102 INFO:tasks.workunit.client.0.smithi107.stdout:8/259: chown d0/d7/df/f52 1 1 2024-04-22T23:35:04.103 INFO:tasks.workunit.client.0.smithi107.stdout:0/331: sync 2024-04-22T23:35:04.106 INFO:tasks.workunit.client.1.smithi157.stdout:8/378: link d2/d34/d6a/d77/d42/l7a d2/d4/d73/l84 0 2024-04-22T23:35:04.107 INFO:tasks.workunit.client.1.smithi157.stdout:5/531: rmdir d1/d5/dc/d43/d5e/db6 0 2024-04-22T23:35:04.107 INFO:tasks.workunit.client.1.smithi157.stdout:5/532: truncate d1/d27/d29/d4b/fae 4376786 0 2024-04-22T23:35:04.107 INFO:tasks.workunit.client.1.smithi157.stdout:5/533: stat d1/d5/dc/d43/d5e/lb3 0 2024-04-22T23:35:04.109 INFO:tasks.workunit.client.1.smithi157.stdout:9/572: creat d4/df/d57/fb0 x:0 0 0 2024-04-22T23:35:04.110 INFO:tasks.workunit.client.0.smithi107.stdout:8/260: write d0/f4 [1257855,51839] 0 2024-04-22T23:35:04.114 INFO:tasks.workunit.client.1.smithi157.stdout:6/814: mkdir d1/d2f/d3b/d39/d4e/da8/de5/d83/d93/d101 0 2024-04-22T23:35:04.114 INFO:tasks.workunit.client.0.smithi107.stdout:8/261: creat d0/d7/d8/d41/d53/f57 x:0 0 0 2024-04-22T23:35:04.114 INFO:tasks.workunit.client.0.smithi107.stdout:8/262: write d0/d7/f35 [2097219,107941] 0 2024-04-22T23:35:04.114 INFO:tasks.workunit.client.0.smithi107.stdout:5/313: dwrite d9/f43 [0,4194304] 0 2024-04-22T23:35:04.114 INFO:tasks.workunit.client.0.smithi107.stdout:5/314: chown f7 5790222 1 2024-04-22T23:35:04.115 INFO:tasks.workunit.client.0.smithi107.stdout:1/340: dwrite d7/d19/d20/d39/f3c [0,4194304] 0 2024-04-22T23:35:04.115 INFO:tasks.workunit.client.0.smithi107.stdout:1/341: truncate d7/d19/d20/d39/d53/f33 817913 0 2024-04-22T23:35:04.115 INFO:tasks.workunit.client.0.smithi107.stdout:1/342: rename d7/d19/d20/d39/d53/d43 to d7/d19/d20/d39/d53/d43/d63/d75 22 2024-04-22T23:35:04.115 INFO:tasks.workunit.client.0.smithi107.stdout:1/343: readlink d7/d19/l1f 0 2024-04-22T23:35:04.115 INFO:tasks.workunit.client.1.smithi157.stdout:7/468: rename d0/d9/d29/d49/d4a/f59 to d0/d9/d29/d33/d7c/f9c 0 2024-04-22T23:35:04.116 INFO:tasks.workunit.client.0.smithi107.stdout:0/332: dread d1/d27/d20/d2c/d2e/f39 [0,4194304] 0 2024-04-22T23:35:04.116 INFO:tasks.workunit.client.0.smithi107.stdout:0/333: chown d1/d9/d14/d4f/f69 8199 1 2024-04-22T23:35:04.116 INFO:tasks.workunit.client.0.smithi107.stdout:0/334: fsync d1/d27/f21 0 2024-04-22T23:35:04.116 INFO:tasks.workunit.client.0.smithi107.stdout:0/335: read - d1/d9/f5f zero size 2024-04-22T23:35:04.116 INFO:tasks.workunit.client.1.smithi157.stdout:8/379: truncate d2/d6/d21/f54 2413290 0 2024-04-22T23:35:04.117 INFO:tasks.workunit.client.0.smithi107.stdout:4/322: dwrite d1/d8/f4b [0,4194304] 0 2024-04-22T23:35:04.117 INFO:tasks.workunit.client.1.smithi157.stdout:6/815: getdents d1/d2f/d3b/d39/d4e/da8/dac/de6 0 2024-04-22T23:35:04.117 INFO:tasks.workunit.client.1.smithi157.stdout:6/816: stat d1/d2f/d3b/d39/d4e/da8/dac/dc0/f100 0 2024-04-22T23:35:04.120 INFO:tasks.workunit.client.0.smithi107.stdout:5/315: link d9/d26/f32 d9/db/de/d5e/f67 0 2024-04-22T23:35:04.120 INFO:tasks.workunit.client.0.smithi107.stdout:5/316: chown d9/db/d42/l65 1367913 1 2024-04-22T23:35:04.122 INFO:tasks.workunit.client.1.smithi157.stdout:8/380: rename d2/l45 to d2/d34/d6a/d77/d42/l85 0 2024-04-22T23:35:04.125 INFO:tasks.workunit.client.0.smithi107.stdout:1/344: unlink d7/d19/d20/d39/d53/d43/l65 0 2024-04-22T23:35:04.125 INFO:tasks.workunit.client.0.smithi107.stdout:1/345: read - d7/d19/d20/d39/d53/d43/d46/d5c/f6b zero size 2024-04-22T23:35:04.125 INFO:tasks.workunit.client.0.smithi107.stdout:8/263: dread d0/d7/df/d3b/f3d [0,4194304] 0 2024-04-22T23:35:04.126 INFO:tasks.workunit.client.0.smithi107.stdout:0/336: symlink d1/d9/d14/l79 0 2024-04-22T23:35:04.126 INFO:tasks.workunit.client.0.smithi107.stdout:0/337: write d1/d27/d20/f60 [908041,26455] 0 2024-04-22T23:35:04.127 INFO:tasks.workunit.client.1.smithi157.stdout:6/817: rename d1/d17/fa2 to d1/d38/f102 0 2024-04-22T23:35:04.127 INFO:tasks.workunit.client.1.smithi157.stdout:6/818: chown d1/d2f/d3b/d39/d4e/da8 470 1 2024-04-22T23:35:04.134 INFO:tasks.workunit.client.0.smithi107.stdout:4/323: unlink d1/d1b/d44/f58 0 2024-04-22T23:35:04.134 INFO:tasks.workunit.client.0.smithi107.stdout:5/317: dread d9/f29 [0,4194304] 0 2024-04-22T23:35:04.134 INFO:tasks.workunit.client.0.smithi107.stdout:5/318: dread - d9/d26/f64 zero size 2024-04-22T23:35:04.134 INFO:tasks.workunit.client.0.smithi107.stdout:4/324: truncate d1/d21/d29/f71 1614072 0 2024-04-22T23:35:04.134 INFO:tasks.workunit.client.0.smithi107.stdout:5/319: creat d9/d26/f68 x:0 0 0 2024-04-22T23:35:04.134 INFO:tasks.workunit.client.0.smithi107.stdout:4/325: creat d1/d21/d52/f73 x:0 0 0 2024-04-22T23:35:04.134 INFO:tasks.workunit.client.0.smithi107.stdout:1/346: mknod d7/d19/d2d/c76 0 2024-04-22T23:35:04.134 INFO:tasks.workunit.client.0.smithi107.stdout:1/347: write d7/d19/d20/d39/d53/f29 [5113637,15422] 0 2024-04-22T23:35:04.135 INFO:tasks.workunit.client.0.smithi107.stdout:0/338: link d1/d9/d14/c44 d1/d67/c7a 0 2024-04-22T23:35:04.137 INFO:tasks.workunit.client.0.smithi107.stdout:0/339: dread d1/d9/d14/f4d [0,4194304] 0 2024-04-22T23:35:04.137 INFO:tasks.workunit.client.0.smithi107.stdout:5/320: mknod d9/db/d42/c69 0 2024-04-22T23:35:04.138 INFO:tasks.workunit.client.0.smithi107.stdout:1/348: dread d7/f67 [0,4194304] 0 2024-04-22T23:35:04.138 INFO:tasks.workunit.client.0.smithi107.stdout:8/264: write d0/d7/df/d3b/f3c [1187486,49957] 0 2024-04-22T23:35:04.139 INFO:tasks.workunit.client.0.smithi107.stdout:4/326: rename d1/d8/c51 to d1/d7/d53/d66/c74 0 2024-04-22T23:35:04.140 INFO:tasks.workunit.client.0.smithi107.stdout:4/327: chown d1/d8/l3a 35 1 2024-04-22T23:35:04.141 INFO:tasks.workunit.client.0.smithi107.stdout:0/340: mkdir d1/d7b 0 2024-04-22T23:35:04.142 INFO:tasks.workunit.client.0.smithi107.stdout:5/321: mkdir d9/d26/d6a 0 2024-04-22T23:35:04.142 INFO:tasks.workunit.client.0.smithi107.stdout:5/322: chown d9/l21 320 1 2024-04-22T23:35:04.143 INFO:tasks.workunit.client.0.smithi107.stdout:5/323: chown d9/db/c28 2994 1 2024-04-22T23:35:04.143 INFO:tasks.workunit.client.0.smithi107.stdout:1/349: mknod d7/d19/d20/d39/c77 0 2024-04-22T23:35:04.143 INFO:tasks.workunit.client.0.smithi107.stdout:1/350: stat d7/d19/d54/f70 0 2024-04-22T23:35:04.144 INFO:tasks.workunit.client.0.smithi107.stdout:8/265: symlink d0/d7/df/d18/d51/l58 0 2024-04-22T23:35:04.144 INFO:tasks.workunit.client.0.smithi107.stdout:4/328: creat d1/f75 x:0 0 0 2024-04-22T23:35:04.146 INFO:tasks.workunit.client.0.smithi107.stdout:5/324: rename d9/d26/f32 to d9/db/de/f6b 0 2024-04-22T23:35:04.146 INFO:tasks.workunit.client.0.smithi107.stdout:5/325: truncate d9/d26/f64 948686 0 2024-04-22T23:35:04.147 INFO:tasks.workunit.client.0.smithi107.stdout:8/266: truncate d0/d7/d8/f1e 2379530 0 2024-04-22T23:35:04.147 INFO:tasks.workunit.client.0.smithi107.stdout:8/267: chown d0/d7/df/l3e 4 1 2024-04-22T23:35:04.148 INFO:tasks.workunit.client.0.smithi107.stdout:4/329: creat d1/d21/f76 x:0 0 0 2024-04-22T23:35:04.149 INFO:tasks.workunit.client.0.smithi107.stdout:8/268: unlink d0/d7/d8/cd 0 2024-04-22T23:35:04.149 INFO:tasks.workunit.client.0.smithi107.stdout:5/326: dread d9/db/f3f [0,4194304] 0 2024-04-22T23:35:04.150 INFO:tasks.workunit.client.0.smithi107.stdout:4/330: symlink d1/d8/d67/l77 0 2024-04-22T23:35:04.151 INFO:tasks.workunit.client.0.smithi107.stdout:8/269: mkdir d0/d3a/d46/d59 0 2024-04-22T23:35:04.154 INFO:tasks.workunit.client.1.smithi157.stdout:5/534: dwrite d1/d5/dc/d43/f88 [0,4194304] 0 2024-04-22T23:35:04.155 INFO:tasks.workunit.client.0.smithi107.stdout:0/341: write d1/d27/d20/d2c/d41/f54 [936695,22019] 0 2024-04-22T23:35:04.155 INFO:tasks.workunit.client.0.smithi107.stdout:0/342: write d1/d27/f38 [1318036,15986] 0 2024-04-22T23:35:04.157 INFO:tasks.workunit.client.0.smithi107.stdout:0/343: mkdir d1/d7c 0 2024-04-22T23:35:04.172 INFO:tasks.workunit.client.0.smithi107.stdout:2/377: sync 2024-04-22T23:35:04.173 INFO:tasks.workunit.client.0.smithi107.stdout:7/268: sync 2024-04-22T23:35:04.173 INFO:tasks.workunit.client.0.smithi107.stdout:9/374: sync 2024-04-22T23:35:04.173 INFO:tasks.workunit.client.0.smithi107.stdout:2/378: chown d1/d12/d35/d48/d13/l28 12 1 2024-04-22T23:35:04.173 INFO:tasks.workunit.client.0.smithi107.stdout:7/269: creat d7/d21/d48/f5b x:0 0 0 2024-04-22T23:35:04.173 INFO:tasks.workunit.client.0.smithi107.stdout:7/270: fsync d7/d21/d48/f55 0 2024-04-22T23:35:04.173 INFO:tasks.workunit.client.0.smithi107.stdout:2/379: write d1/d12/d35/d48/d13/f68 [472499,116589] 0 2024-04-22T23:35:04.173 INFO:tasks.workunit.client.0.smithi107.stdout:2/380: fsync d1/d2b/d49/f50 0 2024-04-22T23:35:04.173 INFO:tasks.workunit.client.0.smithi107.stdout:2/381: truncate d1/d12/f46 1039709 0 2024-04-22T23:35:04.173 INFO:tasks.workunit.client.0.smithi107.stdout:2/382: write d1/d12/d35/d48/f2a [525037,128095] 0 2024-04-22T23:35:04.174 INFO:tasks.workunit.client.0.smithi107.stdout:9/375: rename d5/d7/d17/d1b/d1f/c58 to d5/d7/d17/d1b/d1f/c7f 0 2024-04-22T23:35:04.174 INFO:tasks.workunit.client.0.smithi107.stdout:9/376: stat d5/d7/d17/l6b 0 2024-04-22T23:35:04.174 INFO:tasks.workunit.client.0.smithi107.stdout:9/377: read d5/fc [1136346,16788] 0 2024-04-22T23:35:04.175 INFO:tasks.workunit.client.1.smithi157.stdout:9/573: dwrite d4/d15/d55/f9b [0,4194304] 0 2024-04-22T23:35:04.175 INFO:tasks.workunit.client.1.smithi157.stdout:9/574: stat d4/l30 0 2024-04-22T23:35:04.175 INFO:tasks.workunit.client.1.smithi157.stdout:7/469: dwrite d0/d9/d29/d5e/f65 [0,4194304] 0 2024-04-22T23:35:04.175 INFO:tasks.workunit.client.1.smithi157.stdout:7/470: chown d0/d9/d42/d55/d80/d95 1041962 1 2024-04-22T23:35:04.175 INFO:tasks.workunit.client.0.smithi107.stdout:7/271: mkdir d7/d21/d48/d5c 0 2024-04-22T23:35:04.176 INFO:tasks.workunit.client.1.smithi157.stdout:9/575: rmdir d4/d15/d23/d90 0 2024-04-22T23:35:04.176 INFO:tasks.workunit.client.1.smithi157.stdout:6/819: dwrite d1/d2f/d3b/d39/d4e/da8/fe9 [0,4194304] 0 2024-04-22T23:35:04.177 INFO:tasks.workunit.client.0.smithi107.stdout:2/383: unlink d1/d12/l1b 0 2024-04-22T23:35:04.178 INFO:tasks.workunit.client.1.smithi157.stdout:6/820: mknod d1/d2f/d3b/d39/d4e/da8/dac/dbe/c103 0 2024-04-22T23:35:04.179 INFO:tasks.workunit.client.1.smithi157.stdout:7/471: getdents d0/d9/d72 0 2024-04-22T23:35:04.179 INFO:tasks.workunit.client.1.smithi157.stdout:7/472: write d0/f2 [11924493,16668] 0 2024-04-22T23:35:04.179 INFO:tasks.workunit.client.0.smithi107.stdout:9/378: unlink d5/d14/d28/l5e 0 2024-04-22T23:35:04.180 INFO:tasks.workunit.client.1.smithi157.stdout:8/381: dwrite d2/d6/f37 [0,4194304] 0 2024-04-22T23:35:04.180 INFO:tasks.workunit.client.1.smithi157.stdout:8/382: chown d2/d4/d30/f62 171 1 2024-04-22T23:35:04.180 INFO:tasks.workunit.client.1.smithi157.stdout:9/576: dread d4/f20 [0,4194304] 0 2024-04-22T23:35:04.180 INFO:tasks.workunit.client.1.smithi157.stdout:9/577: creat d4/d15/d23/d27/fb1 x:0 0 0 2024-04-22T23:35:04.181 INFO:tasks.workunit.client.0.smithi107.stdout:0/344: dwrite d1/d27/d20/d2c/d41/f58 [0,4194304] 0 2024-04-22T23:35:04.181 INFO:tasks.workunit.client.0.smithi107.stdout:7/272: creat d7/d21/d48/d5c/f5d x:0 0 0 2024-04-22T23:35:04.181 INFO:tasks.workunit.client.0.smithi107.stdout:7/273: write d7/d1b/f4d [1286095,39220] 0 2024-04-22T23:35:04.181 INFO:tasks.workunit.client.0.smithi107.stdout:7/274: stat d7/f3b 0 2024-04-22T23:35:04.182 INFO:tasks.workunit.client.0.smithi107.stdout:1/351: dwrite d7/d19/d20/d39/d53/f4e [0,4194304] 0 2024-04-22T23:35:04.182 INFO:tasks.workunit.client.0.smithi107.stdout:4/331: dwrite d1/d21/d52/f70 [0,4194304] 0 2024-04-22T23:35:04.183 INFO:tasks.workunit.client.1.smithi157.stdout:5/535: dwrite d1/d27/d29/f58 [0,4194304] 0 2024-04-22T23:35:04.183 INFO:tasks.workunit.client.0.smithi107.stdout:9/379: link d5/d7/d17/d1b/d1f/c5f d5/c80 0 2024-04-22T23:35:04.183 INFO:tasks.workunit.client.0.smithi107.stdout:9/380: creat d5/d14/d28/f81 x:0 0 0 2024-04-22T23:35:04.187 INFO:tasks.workunit.client.1.smithi157.stdout:8/383: rename d2/d34/d6a/d77/d42/d58/c75 to d2/d4/d30/c86 0 2024-04-22T23:35:04.188 INFO:tasks.workunit.client.1.smithi157.stdout:9/578: chown d4/d2d/l2f 10031711 1 2024-04-22T23:35:04.190 INFO:tasks.workunit.client.0.smithi107.stdout:0/345: symlink d1/d67/l7d 0 2024-04-22T23:35:04.192 INFO:tasks.workunit.client.0.smithi107.stdout:6/322: sync 2024-04-22T23:35:04.192 INFO:tasks.workunit.client.0.smithi107.stdout:3/371: sync 2024-04-22T23:35:04.192 INFO:tasks.workunit.client.0.smithi107.stdout:3/372: truncate db/d21/d27/d15/d29/d2a/d45/f62 599152 0 2024-04-22T23:35:04.192 INFO:tasks.workunit.client.0.smithi107.stdout:2/384: write d1/d12/d35/d48/f3a [965345,44472] 0 2024-04-22T23:35:04.192 INFO:tasks.workunit.client.0.smithi107.stdout:3/373: fsync db/f1c 0 2024-04-22T23:35:04.193 INFO:tasks.workunit.client.1.smithi157.stdout:5/536: mkdir d1/d1f/dc6 0 2024-04-22T23:35:04.193 INFO:tasks.workunit.client.0.smithi107.stdout:7/275: getdents d7/d4c 0 2024-04-22T23:35:04.195 INFO:tasks.workunit.client.1.smithi157.stdout:6/821: link d1/d2f/d3b/d39/l77 d1/d2f/d3b/d39/d4e/da8/dac/dc0/l104 0 2024-04-22T23:35:04.197 INFO:tasks.workunit.client.1.smithi157.stdout:8/384: mknod d2/d34/d6a/d77/d42/d58/c87 0 2024-04-22T23:35:04.199 INFO:tasks.workunit.client.1.smithi157.stdout:7/473: dwrite d0/d9/f3c [4194304,4194304] 0 2024-04-22T23:35:04.200 INFO:tasks.workunit.client.1.smithi157.stdout:9/579: rename d4/d15/faa to d4/df/d61/d9c/fb2 0 2024-04-22T23:35:04.202 INFO:tasks.workunit.client.0.smithi107.stdout:0/346: dread d1/d27/f21 [0,4194304] 0 2024-04-22T23:35:04.202 INFO:tasks.workunit.client.0.smithi107.stdout:0/347: fsync d1/f15 0 2024-04-22T23:35:04.203 INFO:tasks.workunit.client.0.smithi107.stdout:0/348: chown d1/d27/d20/c47 313 1 2024-04-22T23:35:04.204 INFO:tasks.workunit.client.0.smithi107.stdout:9/381: mknod d5/d7/d17/d1b/d1f/d36/d53/c82 0 2024-04-22T23:35:04.205 INFO:tasks.workunit.client.1.smithi157.stdout:6/822: symlink d1/d2f/d54/dc4/l105 0 2024-04-22T23:35:04.205 INFO:tasks.workunit.client.0.smithi107.stdout:7/276: read d7/ff [3607254,90121] 0 2024-04-22T23:35:04.207 INFO:tasks.workunit.client.0.smithi107.stdout:6/323: mkdir db/d20/d2a/d38/d59 0 2024-04-22T23:35:04.208 INFO:tasks.workunit.client.1.smithi157.stdout:7/474: creat d0/d9/db/df/f9d x:0 0 0 2024-04-22T23:35:04.209 INFO:tasks.workunit.client.0.smithi107.stdout:3/374: getdents db/d21/d27/d54/d25/d6e 0 2024-04-22T23:35:04.210 INFO:tasks.workunit.client.1.smithi157.stdout:9/580: truncate d4/d15/d23/f97 210102 0 2024-04-22T23:35:04.210 INFO:tasks.workunit.client.0.smithi107.stdout:1/352: truncate d7/d19/d20/d39/d53/f29 4013032 0 2024-04-22T23:35:04.211 INFO:tasks.workunit.client.0.smithi107.stdout:1/353: readlink d7/d19/d20/d39/d53/l36 0 2024-04-22T23:35:04.212 INFO:tasks.workunit.client.1.smithi157.stdout:7/475: unlink d0/c4f 0 2024-04-22T23:35:04.212 INFO:tasks.workunit.client.1.smithi157.stdout:7/476: stat d0/d9/d29/d49/d64 0 2024-04-22T23:35:04.212 INFO:tasks.workunit.client.1.smithi157.stdout:9/581: mkdir d4/d15/d23/d27/d47/db3 0 2024-04-22T23:35:04.213 INFO:tasks.workunit.client.0.smithi107.stdout:4/332: getdents d1/d21 0 2024-04-22T23:35:04.215 INFO:tasks.workunit.client.1.smithi157.stdout:7/477: mkdir d0/d9/d29/d5e/d9e 0 2024-04-22T23:35:04.215 INFO:tasks.workunit.client.1.smithi157.stdout:6/823: dread d1/d17/f21 [0,4194304] 0 2024-04-22T23:35:04.215 INFO:tasks.workunit.client.1.smithi157.stdout:8/385: write d2/d6/d21/d5f/f6f [4121903,76441] 0 2024-04-22T23:35:04.215 INFO:tasks.workunit.client.1.smithi157.stdout:7/478: write d0/d9/db/f5a [9216995,89609] 0 2024-04-22T23:35:04.216 INFO:tasks.workunit.client.1.smithi157.stdout:7/479: getdents d0/d9/d42/d55/d56/d91 0 2024-04-22T23:35:04.216 INFO:tasks.workunit.client.1.smithi157.stdout:8/386: write d2/d6/d21/d5f/d11/f5a [514767,102164] 0 2024-04-22T23:35:04.216 INFO:tasks.workunit.client.1.smithi157.stdout:8/387: read - d2/d4/f50 zero size 2024-04-22T23:35:04.216 INFO:tasks.workunit.client.1.smithi157.stdout:8/388: dread - d2/d4/f80 zero size 2024-04-22T23:35:04.216 INFO:tasks.workunit.client.1.smithi157.stdout:8/389: write d2/d34/d6a/d77/d42/f5c [156680,5296] 0 2024-04-22T23:35:04.216 INFO:tasks.workunit.client.1.smithi157.stdout:8/390: write d2/d4/f50 [33142,73791] 0 2024-04-22T23:35:04.217 INFO:tasks.workunit.client.1.smithi157.stdout:1/558: sync 2024-04-22T23:35:04.217 INFO:tasks.workunit.client.1.smithi157.stdout:2/648: sync 2024-04-22T23:35:04.217 INFO:tasks.workunit.client.1.smithi157.stdout:5/537: dwrite d1/fe [0,4194304] 0 2024-04-22T23:35:04.217 INFO:tasks.workunit.client.1.smithi157.stdout:7/480: write d0/d9/db/f5a [8892516,54604] 0 2024-04-22T23:35:04.218 INFO:tasks.workunit.client.0.smithi107.stdout:0/349: symlink d1/d2a/l7e 0 2024-04-22T23:35:04.218 INFO:tasks.workunit.client.0.smithi107.stdout:8/270: dread d0/d7/d8/f1e [0,4194304] 0 2024-04-22T23:35:04.218 INFO:tasks.workunit.client.0.smithi107.stdout:8/271: fsync d0/f4 0 2024-04-22T23:35:04.218 INFO:tasks.workunit.client.0.smithi107.stdout:8/272: fsync d0/d7/fa 0 2024-04-22T23:35:04.218 INFO:tasks.workunit.client.0.smithi107.stdout:8/273: write d0/f19 [4919648,31019] 0 2024-04-22T23:35:04.218 INFO:tasks.workunit.client.0.smithi107.stdout:8/274: write d0/d7/d8/f1e [3299117,71715] 0 2024-04-22T23:35:04.218 INFO:tasks.workunit.client.0.smithi107.stdout:9/382: mkdir d5/d14/d2e/d4f/d83 0 2024-04-22T23:35:04.218 INFO:tasks.workunit.client.1.smithi157.stdout:6/824: write d1/d2f/d3b/f2a [727149,93651] 0 2024-04-22T23:35:04.220 INFO:tasks.workunit.client.1.smithi157.stdout:1/559: creat d4/d17/d34/d3b/d52/fbe x:0 0 0 2024-04-22T23:35:04.220 INFO:tasks.workunit.client.1.smithi157.stdout:1/560: write d4/f1b [3163637,47390] 0 2024-04-22T23:35:04.221 INFO:tasks.workunit.client.0.smithi107.stdout:7/277: creat d7/f5e x:0 0 0 2024-04-22T23:35:04.221 INFO:tasks.workunit.client.0.smithi107.stdout:2/385: dwrite d1/d63/f69 [0,4194304] 0 2024-04-22T23:35:04.226 INFO:tasks.workunit.client.0.smithi107.stdout:6/324: rename db/f48 to db/d20/d2a/d38/d59/f5a 0 2024-04-22T23:35:04.227 INFO:tasks.workunit.client.1.smithi157.stdout:1/561: creat d4/d17/d1d/d33/d41/d54/db3/fbf x:0 0 0 2024-04-22T23:35:04.227 INFO:tasks.workunit.client.1.smithi157.stdout:1/562: chown d4/d17/d34/d3b/d52/fbe 23341 1 2024-04-22T23:35:04.227 INFO:tasks.workunit.client.1.smithi157.stdout:1/563: readlink d4/d17/d6b/l80 0 2024-04-22T23:35:04.229 INFO:tasks.workunit.client.1.smithi157.stdout:6/825: mkdir d1/d2f/d3b/d106 0 2024-04-22T23:35:04.232 INFO:tasks.workunit.client.0.smithi107.stdout:3/375: unlink db/d21/d27/d15/d29/d2a/d45/f49 0 2024-04-22T23:35:04.232 INFO:tasks.workunit.client.1.smithi157.stdout:2/649: dread d7/d8/f2f [0,4194304] 0 2024-04-22T23:35:04.233 INFO:tasks.workunit.client.1.smithi157.stdout:2/650: stat d7/d8/dd/d4d 0 2024-04-22T23:35:04.233 INFO:tasks.workunit.client.1.smithi157.stdout:2/651: write d7/d8/d30/d23/d27/f49 [826513,89144] 0 2024-04-22T23:35:04.233 INFO:tasks.workunit.client.1.smithi157.stdout:1/564: creat d4/d17/d34/d75/d8a/fc0 x:0 0 0 2024-04-22T23:35:04.234 INFO:tasks.workunit.client.1.smithi157.stdout:2/652: mknod d7/d9f/dc7/cdd 0 2024-04-22T23:35:04.234 INFO:tasks.workunit.client.0.smithi107.stdout:1/354: creat d7/d19/d71/f78 x:0 0 0 2024-04-22T23:35:04.235 INFO:tasks.workunit.client.1.smithi157.stdout:1/565: mkdir d4/db/d3d/dc1 0 2024-04-22T23:35:04.235 INFO:tasks.workunit.client.1.smithi157.stdout:1/566: write d4/d17/d6b/f8d [4162103,104912] 0 2024-04-22T23:35:04.236 INFO:tasks.workunit.client.1.smithi157.stdout:2/653: read d7/d1f/d3e/db4/fc4 [1879351,89828] 0 2024-04-22T23:35:04.236 INFO:tasks.workunit.client.1.smithi157.stdout:6/826: rename d1/d2f/d3b/c26 to d1/d2f/d3b/d39/d4e/da8/de5/da7/c107 0 2024-04-22T23:35:04.236 INFO:tasks.workunit.client.1.smithi157.stdout:2/654: chown d7/d8/d30/db8 992814 1 2024-04-22T23:35:04.237 INFO:tasks.workunit.client.1.smithi157.stdout:1/567: mkdir d4/db/d3d/dc2 0 2024-04-22T23:35:04.238 INFO:tasks.workunit.client.1.smithi157.stdout:9/582: dwrite d4/df/f12 [0,4194304] 0 2024-04-22T23:35:04.239 INFO:tasks.workunit.client.1.smithi157.stdout:5/538: dwrite d1/d5/d7/f67 [0,4194304] 0 2024-04-22T23:35:04.239 INFO:tasks.workunit.client.1.smithi157.stdout:6/827: write d1/d2f/d3b/d49/f7d [534624,65794] 0 2024-04-22T23:35:04.239 INFO:tasks.workunit.client.0.smithi107.stdout:6/325: write db/d20/f4f [2854818,126205] 0 2024-04-22T23:35:04.239 INFO:tasks.workunit.client.0.smithi107.stdout:4/333: mkdir d1/d7/d63/d78 0 2024-04-22T23:35:04.239 INFO:tasks.workunit.client.0.smithi107.stdout:4/334: getdents d1/d21 0 2024-04-22T23:35:04.240 INFO:tasks.workunit.client.0.smithi107.stdout:4/335: write d1/d7/d53/d66/d6b/f4e [516873,73084] 0 2024-04-22T23:35:04.240 INFO:tasks.workunit.client.0.smithi107.stdout:4/336: chown d1/d8/l62 16888572 1 2024-04-22T23:35:04.240 INFO:tasks.workunit.client.1.smithi157.stdout:1/568: unlink d4/d5f/d7e/c9e 0 2024-04-22T23:35:04.242 INFO:tasks.workunit.client.0.smithi107.stdout:0/350: symlink d1/d7b/l7f 0 2024-04-22T23:35:04.242 INFO:tasks.workunit.client.1.smithi157.stdout:5/539: mknod d1/d27/cc7 0 2024-04-22T23:35:04.243 INFO:tasks.workunit.client.1.smithi157.stdout:6/828: creat d1/d2f/d3b/d39/d4e/da8/dac/dbe/def/f108 x:0 0 0 2024-04-22T23:35:04.245 INFO:tasks.workunit.client.1.smithi157.stdout:1/569: symlink d4/d17/d34/d84/lc3 0 2024-04-22T23:35:04.246 INFO:tasks.workunit.client.1.smithi157.stdout:1/570: readlink d4/d17/d34/d3b/d52/lac 0 2024-04-22T23:35:04.246 INFO:tasks.workunit.client.1.smithi157.stdout:1/571: truncate d4/d17/d34/d3b/d52/d65/dbc/faa 386082 0 2024-04-22T23:35:04.246 INFO:tasks.workunit.client.1.smithi157.stdout:1/572: dread - d4/d17/d34/d75/fbb zero size 2024-04-22T23:35:04.246 INFO:tasks.workunit.client.1.smithi157.stdout:9/583: dread d4/d15/d23/d27/d75/f82 [0,4194304] 0 2024-04-22T23:35:04.247 INFO:tasks.workunit.client.1.smithi157.stdout:5/540: unlink d1/d1f/l37 0 2024-04-22T23:35:04.247 INFO:tasks.workunit.client.1.smithi157.stdout:5/541: dread - d1/d27/d29/fbf zero size 2024-04-22T23:35:04.248 INFO:tasks.workunit.client.1.smithi157.stdout:7/481: dwrite d0/f2 [0,4194304] 0 2024-04-22T23:35:04.248 INFO:tasks.workunit.client.0.smithi107.stdout:4/337: write d1/d8/d56/d27/f2f [351966,119167] 0 2024-04-22T23:35:04.249 INFO:tasks.workunit.client.0.smithi107.stdout:6/326: dread db/fd [0,4194304] 0 2024-04-22T23:35:04.249 INFO:tasks.workunit.client.0.smithi107.stdout:4/338: read - d1/d7/d5b/f6c zero size 2024-04-22T23:35:04.250 INFO:tasks.workunit.client.0.smithi107.stdout:9/383: truncate d5/fc 1197331 0 2024-04-22T23:35:04.251 INFO:tasks.workunit.client.1.smithi157.stdout:1/573: creat d4/d17/fc4 x:0 0 0 2024-04-22T23:35:04.251 INFO:tasks.workunit.client.1.smithi157.stdout:1/574: chown d4/d17/d34/l55 51249 1 2024-04-22T23:35:04.251 INFO:tasks.workunit.client.1.smithi157.stdout:9/584: creat d4/d2d/fb4 x:0 0 0 2024-04-22T23:35:04.252 INFO:tasks.workunit.client.1.smithi157.stdout:5/542: mknod d1/d27/d29/cc8 0 2024-04-22T23:35:04.252 INFO:tasks.workunit.client.0.smithi107.stdout:7/278: creat d7/f5f x:0 0 0 2024-04-22T23:35:04.253 INFO:tasks.workunit.client.1.smithi157.stdout:7/482: symlink d0/d9/d29/d33/l9f 0 2024-04-22T23:35:04.254 INFO:tasks.workunit.client.0.smithi107.stdout:2/386: mknod d1/d2b/d3d/d57/c85 0 2024-04-22T23:35:04.257 INFO:tasks.workunit.client.0.smithi107.stdout:8/275: rename d0/d7/df/f52 to d0/f5a 0 2024-04-22T23:35:04.257 INFO:tasks.workunit.client.1.smithi157.stdout:2/655: dwrite d7/d8/d30/f76 [0,4194304] 0 2024-04-22T23:35:04.258 INFO:tasks.workunit.client.1.smithi157.stdout:2/656: chown d7/d1f/d26/c5a 1261547191 1 2024-04-22T23:35:04.258 INFO:tasks.workunit.client.1.smithi157.stdout:2/657: chown d7/d8/d30/d5b/d8a 3 1 2024-04-22T23:35:04.259 INFO:tasks.workunit.client.0.smithi107.stdout:3/376: mkdir db/d21/d27/d54/d25/d53/d81 0 2024-04-22T23:35:04.259 INFO:tasks.workunit.client.0.smithi107.stdout:3/377: write db/d21/d27/f20 [1255425,80750] 0 2024-04-22T23:35:04.260 INFO:tasks.workunit.client.1.smithi157.stdout:9/585: mkdir d4/d15/d23/d27/db5 0 2024-04-22T23:35:04.261 INFO:tasks.workunit.client.0.smithi107.stdout:4/339: dread d1/d7/d53/f6f [0,4194304] 0 2024-04-22T23:35:04.262 INFO:tasks.workunit.client.1.smithi157.stdout:5/543: mknod d1/db0/d7a/cc9 0 2024-04-22T23:35:04.263 INFO:tasks.workunit.client.1.smithi157.stdout:6/829: rename d1/d2f/d54/d5c to d1/d2f/d54/d9b/d109 0 2024-04-22T23:35:04.263 INFO:tasks.workunit.client.1.smithi157.stdout:6/830: creat d1/d36/dea/f10a x:0 0 0 2024-04-22T23:35:04.264 INFO:tasks.workunit.client.1.smithi157.stdout:6/831: dread - d1/d2f/d3b/d39/d4e/da8/de5/d83/fd8 zero size 2024-04-22T23:35:04.264 INFO:tasks.workunit.client.1.smithi157.stdout:6/832: fsync d1/d36/dea/fff 0 2024-04-22T23:35:04.264 INFO:tasks.workunit.client.0.smithi107.stdout:6/327: symlink db/d20/d2c/l5b 0 2024-04-22T23:35:04.265 INFO:tasks.workunit.client.0.smithi107.stdout:6/328: write db/f29 [9124131,28427] 0 2024-04-22T23:35:04.266 INFO:tasks.workunit.client.0.smithi107.stdout:2/387: creat d1/d12/d35/d48/d13/d4f/f86 x:0 0 0 2024-04-22T23:35:04.267 INFO:tasks.workunit.client.0.smithi107.stdout:2/388: chown d1/d2b/d3d/f5e 420 1 2024-04-22T23:35:04.268 INFO:tasks.workunit.client.0.smithi107.stdout:1/355: rename d7/l16 to d7/d19/d20/d39/d53/d43/d46/d68/l79 0 2024-04-22T23:35:04.269 INFO:tasks.workunit.client.0.smithi107.stdout:8/276: dread d0/d7/f31 [0,4194304] 0 2024-04-22T23:35:04.269 INFO:tasks.workunit.client.0.smithi107.stdout:8/277: write d0/d7/df/f4b [277136,1004] 0 2024-04-22T23:35:04.270 INFO:tasks.workunit.client.0.smithi107.stdout:3/378: creat db/d21/d27/d54/d25/d6e/f82 x:0 0 0 2024-04-22T23:35:04.272 INFO:tasks.workunit.client.0.smithi107.stdout:4/340: unlink d1/d8/d56/d27/l38 0 2024-04-22T23:35:04.272 INFO:tasks.workunit.client.0.smithi107.stdout:4/341: dread - d1/f75 zero size 2024-04-22T23:35:04.273 INFO:tasks.workunit.client.1.smithi157.stdout:1/575: write d4/db/f5a [545729,30574] 0 2024-04-22T23:35:04.274 INFO:tasks.workunit.client.1.smithi157.stdout:2/658: mkdir d7/d8/d95/dde 0 2024-04-22T23:35:04.274 INFO:tasks.workunit.client.1.smithi157.stdout:9/586: stat d4/d15/d23/d3f/l48 0 2024-04-22T23:35:04.275 INFO:tasks.workunit.client.0.smithi107.stdout:6/329: mknod db/d20/d2c/c5c 0 2024-04-22T23:35:04.276 INFO:tasks.workunit.client.0.smithi107.stdout:2/389: truncate d1/d12/d35/f3f 168780 0 2024-04-22T23:35:04.276 INFO:tasks.workunit.client.0.smithi107.stdout:2/390: stat d1/d2b/f6e 0 2024-04-22T23:35:04.277 INFO:tasks.workunit.client.1.smithi157.stdout:6/833: unlink d1/d2f/d3b/d39/d4e/d74/d8c/fa9 0 2024-04-22T23:35:04.277 INFO:tasks.workunit.client.1.smithi157.stdout:1/576: symlink d4/d17/d1d/db2/lc5 0 2024-04-22T23:35:04.277 INFO:tasks.workunit.client.1.smithi157.stdout:9/587: symlink d4/df/d61/d9c/lb6 0 2024-04-22T23:35:04.279 INFO:tasks.workunit.client.0.smithi107.stdout:0/351: rename d1/d9/f5f to d1/d9/d14/f80 0 2024-04-22T23:35:04.279 INFO:tasks.workunit.client.1.smithi157.stdout:9/588: rename d4/d15/da3 to d4/df/d57/db7 0 2024-04-22T23:35:04.281 INFO:tasks.workunit.client.1.smithi157.stdout:5/544: dwrite d1/d27/d29/fc2 [0,4194304] 0 2024-04-22T23:35:04.282 INFO:tasks.workunit.client.0.smithi107.stdout:1/356: creat d7/d19/d20/d39/d53/d43/f7a x:0 0 0 2024-04-22T23:35:04.283 INFO:tasks.workunit.client.1.smithi157.stdout:9/589: creat d4/d15/d23/d27/d47/db3/fb8 x:0 0 0 2024-04-22T23:35:04.284 INFO:tasks.workunit.client.0.smithi107.stdout:4/342: creat d1/d7/d53/d66/f79 x:0 0 0 2024-04-22T23:35:04.285 INFO:tasks.workunit.client.1.smithi157.stdout:1/577: dread d4/f5 [0,4194304] 0 2024-04-22T23:35:04.287 INFO:tasks.workunit.client.0.smithi107.stdout:6/330: rmdir db/d20/d2a/d38/d4b 39 2024-04-22T23:35:04.289 INFO:tasks.workunit.client.1.smithi157.stdout:9/590: getdents d4/d15/d55/d8d 0 2024-04-22T23:35:04.289 INFO:tasks.workunit.client.1.smithi157.stdout:9/591: dread - d4/df/d80/f9d zero size 2024-04-22T23:35:04.289 INFO:tasks.workunit.client.1.smithi157.stdout:4/635: sync 2024-04-22T23:35:04.289 INFO:tasks.workunit.client.1.smithi157.stdout:3/535: sync 2024-04-22T23:35:04.289 INFO:tasks.workunit.client.1.smithi157.stdout:3/536: truncate d0/d7/d95/fa5 591615 0 2024-04-22T23:35:04.289 INFO:tasks.workunit.client.1.smithi157.stdout:4/636: write d3/d5c/fca [5088452,93803] 0 2024-04-22T23:35:04.290 INFO:tasks.workunit.client.0.smithi107.stdout:2/391: unlink d1/d12/d35/f3b 0 2024-04-22T23:35:04.291 INFO:tasks.workunit.client.1.smithi157.stdout:0/495: sync 2024-04-22T23:35:04.291 INFO:tasks.workunit.client.0.smithi107.stdout:9/384: rename d5/d24/f6a to d5/d14/d2e/f84 0 2024-04-22T23:35:04.292 INFO:tasks.workunit.client.0.smithi107.stdout:9/385: dread - d5/d24/f7d zero size 2024-04-22T23:35:04.294 INFO:tasks.workunit.client.1.smithi157.stdout:1/578: mkdir d4/d17/d34/dc6 0 2024-04-22T23:35:04.295 INFO:tasks.workunit.client.1.smithi157.stdout:5/545: getdents d1/d27/d29/d4b 0 2024-04-22T23:35:04.295 INFO:tasks.workunit.client.1.smithi157.stdout:3/537: creat d0/fa9 x:0 0 0 2024-04-22T23:35:04.295 INFO:tasks.workunit.client.1.smithi157.stdout:3/538: chown d0/l68 15959332 1 2024-04-22T23:35:04.295 INFO:tasks.workunit.client.1.smithi157.stdout:3/539: truncate d0/d3/d34/d3d/d5b/f84 568345 0 2024-04-22T23:35:04.295 INFO:tasks.workunit.client.1.smithi157.stdout:6/834: read d1/d2f/d3b/d39/d4e/da8/fe9 [3797037,42023] 0 2024-04-22T23:35:04.296 INFO:tasks.workunit.client.0.smithi107.stdout:0/352: rmdir d1/d27/d20/d40/d75 39 2024-04-22T23:35:04.298 INFO:tasks.workunit.client.0.smithi107.stdout:4/343: creat d1/d1b/f7a x:0 0 0 2024-04-22T23:35:04.298 INFO:tasks.workunit.client.1.smithi157.stdout:5/546: rename d1/d5/d7/d10/l1e to d1/d5/d7/d2f/d9c/lca 0 2024-04-22T23:35:04.299 INFO:tasks.workunit.client.1.smithi157.stdout:5/547: write d1/d5/dc/f1b [8478479,79960] 0 2024-04-22T23:35:04.299 INFO:tasks.workunit.client.1.smithi157.stdout:5/548: chown d1/d5/d7/d2f/d45/d54/la6 578007 1 2024-04-22T23:35:04.299 INFO:tasks.workunit.client.1.smithi157.stdout:5/549: fdatasync d1/d5/d7/d2f/d9c/faa 0 2024-04-22T23:35:04.300 INFO:tasks.workunit.client.0.smithi107.stdout:2/392: mkdir d1/d2b/d49/d87 0 2024-04-22T23:35:04.301 INFO:tasks.workunit.client.1.smithi157.stdout:3/540: getdents d0/d3/d34/d3d/d57/d5c 0 2024-04-22T23:35:04.301 INFO:tasks.workunit.client.1.smithi157.stdout:3/541: rename d0/d3 to d0/d3/d34/d3d/d57/d60/d9c/daa 22 2024-04-22T23:35:04.301 INFO:tasks.workunit.client.1.smithi157.stdout:3/542: dread - d0/d3/d34/d3d/d57/d60/d6b/f85 zero size 2024-04-22T23:35:04.302 INFO:tasks.workunit.client.0.smithi107.stdout:2/393: write d1/d12/f46 [83459,51830] 0 2024-04-22T23:35:04.303 INFO:tasks.workunit.client.0.smithi107.stdout:8/278: getdents d0/d7/df 0 2024-04-22T23:35:04.303 INFO:tasks.workunit.client.0.smithi107.stdout:8/279: dread - d0/d3a/f56 zero size 2024-04-22T23:35:04.303 INFO:tasks.workunit.client.0.smithi107.stdout:8/280: chown d0/d7/d8 170338 1 2024-04-22T23:35:04.304 INFO:tasks.workunit.client.0.smithi107.stdout:8/281: creat d0/d7/df/d17/f5b x:0 0 0 2024-04-22T23:35:04.304 INFO:tasks.workunit.client.1.smithi157.stdout:2/659: dwrite d7/d8/d30/fa6 [0,4194304] 0 2024-04-22T23:35:04.304 INFO:tasks.workunit.client.1.smithi157.stdout:1/579: dread d4/d17/d34/d84/d94/fa5 [4194304,4194304] 0 2024-04-22T23:35:04.305 INFO:tasks.workunit.client.1.smithi157.stdout:1/580: fdatasync d4/db/f13 0 2024-04-22T23:35:04.305 INFO:tasks.workunit.client.0.smithi107.stdout:7/279: rename d7/d21/c50 to d7/d1b/d51/c60 0 2024-04-22T23:35:04.307 INFO:tasks.workunit.client.1.smithi157.stdout:1/581: mkdir d4/d17/d34/d75/dc7 0 2024-04-22T23:35:04.308 INFO:tasks.workunit.client.1.smithi157.stdout:1/582: write d4/db/f4a [1011033,81857] 0 2024-04-22T23:35:04.308 INFO:tasks.workunit.client.1.smithi157.stdout:0/496: write d17/d38/f73 [127075,33719] 0 2024-04-22T23:35:04.310 INFO:tasks.workunit.client.1.smithi157.stdout:1/583: truncate d4/db/fe 520584 0 2024-04-22T23:35:04.310 INFO:tasks.workunit.client.1.smithi157.stdout:5/550: mknod d1/d1f/dc6/ccb 0 2024-04-22T23:35:04.310 INFO:tasks.workunit.client.1.smithi157.stdout:5/551: stat d1/d5/d7/d2f/d45/d54/f82 0 2024-04-22T23:35:04.312 INFO:tasks.workunit.client.0.smithi107.stdout:0/353: write d1/d9/d14/d4f/d59/f5c [3023819,55400] 0 2024-04-22T23:35:04.312 INFO:tasks.workunit.client.0.smithi107.stdout:4/344: dread d1/f13 [0,4194304] 0 2024-04-22T23:35:04.312 INFO:tasks.workunit.client.1.smithi157.stdout:6/835: dread d1/f2c [0,4194304] 0 2024-04-22T23:35:04.313 INFO:tasks.workunit.client.1.smithi157.stdout:0/497: mknod d17/d2b/d4f/d5a/da1/caa 0 2024-04-22T23:35:04.313 INFO:tasks.workunit.client.1.smithi157.stdout:1/584: truncate d4/d17/d34/d84/d94/fa5 8435448 0 2024-04-22T23:35:04.317 INFO:tasks.workunit.client.0.smithi107.stdout:8/282: link d0/d7/df/f23 d0/d7/df/f5c 0 2024-04-22T23:35:04.318 INFO:tasks.workunit.client.0.smithi107.stdout:2/394: dread d1/f1a [0,4194304] 0 2024-04-22T23:35:04.318 INFO:tasks.workunit.client.0.smithi107.stdout:2/395: truncate d1/d63/f65 923056 0 2024-04-22T23:35:04.318 INFO:tasks.workunit.client.0.smithi107.stdout:2/396: write d1/d12/d35/f7a [104837,72447] 0 2024-04-22T23:35:04.318 INFO:tasks.workunit.client.0.smithi107.stdout:2/397: readlink d1/d12/l38 0 2024-04-22T23:35:04.318 INFO:tasks.workunit.client.0.smithi107.stdout:3/379: rename db/d21/d27/d15/d29/l73 to db/d21/d27/d15/d24/d52/l83 0 2024-04-22T23:35:04.320 INFO:tasks.workunit.client.0.smithi107.stdout:7/280: link d7/f2a d7/d21/d48/f61 0 2024-04-22T23:35:04.320 INFO:tasks.workunit.client.1.smithi157.stdout:5/552: symlink d1/d5/dc/d43/d4f/d7f/dab/lcc 0 2024-04-22T23:35:04.320 INFO:tasks.workunit.client.1.smithi157.stdout:5/553: fsync d1/d27/d29/d4b/fae 0 2024-04-22T23:35:04.320 INFO:tasks.workunit.client.1.smithi157.stdout:4/637: dwrite d3/d96/d40/fab [0,4194304] 0 2024-04-22T23:35:04.320 INFO:tasks.workunit.client.1.smithi157.stdout:3/543: getdents d0 0 2024-04-22T23:35:04.321 INFO:tasks.workunit.client.0.smithi107.stdout:9/386: symlink d5/d7/l85 0 2024-04-22T23:35:04.322 INFO:tasks.workunit.client.0.smithi107.stdout:6/331: dwrite db/f1d [0,4194304] 0 2024-04-22T23:35:04.323 INFO:tasks.workunit.client.0.smithi107.stdout:1/357: dwrite d7/d19/d20/d39/d53/f4e [0,4194304] 0 2024-04-22T23:35:04.323 INFO:tasks.workunit.client.0.smithi107.stdout:3/380: dread f0 [4194304,4194304] 0 2024-04-22T23:35:04.324 INFO:tasks.workunit.client.1.smithi157.stdout:0/498: symlink d17/d2b/lab 0 2024-04-22T23:35:04.327 INFO:tasks.workunit.client.0.smithi107.stdout:0/354: mknod d1/d27/d20/d2c/d41/d52/c81 0 2024-04-22T23:35:04.327 INFO:tasks.workunit.client.0.smithi107.stdout:0/355: write d1/f15 [701525,40815] 0 2024-04-22T23:35:04.327 INFO:tasks.workunit.client.0.smithi107.stdout:2/398: mkdir d1/d2b/d88 0 2024-04-22T23:35:04.327 INFO:tasks.workunit.client.0.smithi107.stdout:2/399: chown d1/d12/d35/d48 646 1 2024-04-22T23:35:04.328 INFO:tasks.workunit.client.0.smithi107.stdout:2/400: fdatasync d1/d63/f69 0 2024-04-22T23:35:04.328 INFO:tasks.workunit.client.0.smithi107.stdout:4/345: rename d1/d21/d29/d2e/c42 to d1/d1b/c7b 0 2024-04-22T23:35:04.329 INFO:tasks.workunit.client.1.smithi157.stdout:1/585: rename d4/d17/d34/dc6 to d4/d17/d34/dc8 0 2024-04-22T23:35:04.330 INFO:tasks.workunit.client.1.smithi157.stdout:6/836: rmdir d1/d2f/d54 39 2024-04-22T23:35:04.331 INFO:tasks.workunit.client.0.smithi107.stdout:0/356: dread d1/d27/f38 [0,4194304] 0 2024-04-22T23:35:04.331 INFO:tasks.workunit.client.0.smithi107.stdout:0/357: read d1/d9/d14/d4f/d59/f5c [632494,73757] 0 2024-04-22T23:35:04.331 INFO:tasks.workunit.client.0.smithi107.stdout:0/358: fsync d1/d27/d20/d2c/d2e/f5a 0 2024-04-22T23:35:04.331 INFO:tasks.workunit.client.0.smithi107.stdout:0/359: chown d1/d9/l4e 14992 1 2024-04-22T23:35:04.333 INFO:tasks.workunit.client.1.smithi157.stdout:3/544: getdents d0/d3/d34/d3d/d57 0 2024-04-22T23:35:04.338 INFO:tasks.workunit.client.1.smithi157.stdout:0/499: creat d17/d2b/d71/fac x:0 0 0 2024-04-22T23:35:04.341 INFO:tasks.workunit.client.1.smithi157.stdout:1/586: rename d4/d17/d34/d3b/d50/fba to d4/d17/d34/d3b/d52/fc9 0 2024-04-22T23:35:04.343 INFO:tasks.workunit.client.1.smithi157.stdout:5/554: mknod d1/d5/dc/ccd 0 2024-04-22T23:35:04.343 INFO:tasks.workunit.client.1.smithi157.stdout:5/555: chown d1/d1f/dc6/ccb 3419 1 2024-04-22T23:35:04.344 INFO:tasks.workunit.client.1.smithi157.stdout:6/837: unlink d1/d2f/d3b/d39/d4e/da8/de5/da7/lbb 0 2024-04-22T23:35:04.346 INFO:tasks.workunit.client.1.smithi157.stdout:0/500: mkdir d17/d2b/d4f/d5a/da1/da8/dad 0 2024-04-22T23:35:04.346 INFO:tasks.workunit.client.1.smithi157.stdout:0/501: write d17/d23/d34/d67/d7f/f74 [11146,42760] 0 2024-04-22T23:35:04.346 INFO:tasks.workunit.client.1.smithi157.stdout:1/587: dread d4/f5 [0,4194304] 0 2024-04-22T23:35:04.346 INFO:tasks.workunit.client.1.smithi157.stdout:5/556: symlink d1/db0/dc1/lce 0 2024-04-22T23:35:04.348 INFO:tasks.workunit.client.1.smithi157.stdout:6/838: creat d1/d2f/d3b/d39/d4e/da8/f10b x:0 0 0 2024-04-22T23:35:04.348 INFO:tasks.workunit.client.1.smithi157.stdout:1/588: mknod d4/d5f/d7e/cca 0 2024-04-22T23:35:04.349 INFO:tasks.workunit.client.1.smithi157.stdout:6/839: mknod d1/d2f/d54/d9b/dda/c10c 0 2024-04-22T23:35:04.349 INFO:tasks.workunit.client.1.smithi157.stdout:6/840: rename d1 to d1/d2f/d3b/d39/d4e/d74/d8c/d10d 22 2024-04-22T23:35:04.350 INFO:tasks.workunit.client.1.smithi157.stdout:6/841: chown d1/d2f/d3b/d39/d4e/da8/dac/ddd/ddf/lf0 39012 1 2024-04-22T23:35:04.350 INFO:tasks.workunit.client.1.smithi157.stdout:1/589: mknod d4/d17/d1d/ccb 0 2024-04-22T23:35:04.351 INFO:tasks.workunit.client.0.smithi107.stdout:3/381: mknod db/d21/d27/d54/d25/c84 0 2024-04-22T23:35:04.351 INFO:tasks.workunit.client.1.smithi157.stdout:6/842: dread d1/f34 [0,4194304] 0 2024-04-22T23:35:04.351 INFO:tasks.workunit.client.1.smithi157.stdout:6/843: fsync d1/f7 0 2024-04-22T23:35:04.352 INFO:tasks.workunit.client.1.smithi157.stdout:1/590: mkdir d4/d17/d34/d75/dcc 0 2024-04-22T23:35:04.352 INFO:tasks.workunit.client.1.smithi157.stdout:1/591: chown d4/d17/d1d/d33/d41/la7 403 1 2024-04-22T23:35:04.353 INFO:tasks.workunit.client.1.smithi157.stdout:6/844: mknod d1/d2f/d54/c10e 0 2024-04-22T23:35:04.353 INFO:tasks.workunit.client.0.smithi107.stdout:0/360: creat d1/d9/d14/d4f/f82 x:0 0 0 2024-04-22T23:35:04.354 INFO:tasks.workunit.client.1.smithi157.stdout:1/592: creat d4/d77/fcd x:0 0 0 2024-04-22T23:35:04.354 INFO:tasks.workunit.client.0.smithi107.stdout:9/387: symlink d5/d7/d17/d1b/d1f/d36/d53/d55/l86 0 2024-04-22T23:35:04.356 INFO:tasks.workunit.client.0.smithi107.stdout:1/358: getdents d7/d19/d20/d39/d53 0 2024-04-22T23:35:04.357 INFO:tasks.workunit.client.1.smithi157.stdout:6/845: rename d1/d17/f21 to d1/d2f/d54/d9b/dda/f10f 0 2024-04-22T23:35:04.357 INFO:tasks.workunit.client.1.smithi157.stdout:6/846: fsync d1/d2f/d3b/d39/d4e/f52 0 2024-04-22T23:35:04.357 INFO:tasks.workunit.client.0.smithi107.stdout:3/382: symlink db/d21/d27/d15/d24/d52/d60/l85 0 2024-04-22T23:35:04.358 INFO:tasks.workunit.client.1.smithi157.stdout:5/557: write d1/db0/d7a/f87 [4000296,21753] 0 2024-04-22T23:35:04.358 INFO:tasks.workunit.client.1.smithi157.stdout:1/593: creat d4/d5f/fce x:0 0 0 2024-04-22T23:35:04.358 INFO:tasks.workunit.client.0.smithi107.stdout:0/361: symlink d1/d27/d20/d2c/d41/l83 0 2024-04-22T23:35:04.359 INFO:tasks.workunit.client.0.smithi107.stdout:9/388: creat d5/d6c/f87 x:0 0 0 2024-04-22T23:35:04.359 INFO:tasks.workunit.client.1.smithi157.stdout:6/847: link d1/d2f/d3b/d39/d4e/da8/de5/f98 d1/d2f/d3b/d39/d4e/da8/dac/f110 0 2024-04-22T23:35:04.360 INFO:tasks.workunit.client.0.smithi107.stdout:1/359: rename d7/d19/l1f to d7/d19/d20/l7b 0 2024-04-22T23:35:04.361 INFO:tasks.workunit.client.1.smithi157.stdout:5/558: creat d1/d5/dc/d43/d4f/fcf x:0 0 0 2024-04-22T23:35:04.363 INFO:tasks.workunit.client.1.smithi157.stdout:1/594: link d4/d17/f59 d4/d17/d34/fcf 0 2024-04-22T23:35:04.363 INFO:tasks.workunit.client.1.smithi157.stdout:1/595: readlink d4/d17/lb0 0 2024-04-22T23:35:04.363 INFO:tasks.workunit.client.1.smithi157.stdout:1/596: chown d4/d17/d1d/d33/d41/c4f 3978341 1 2024-04-22T23:35:04.365 INFO:tasks.workunit.client.1.smithi157.stdout:5/559: creat d1/d5/fd0 x:0 0 0 2024-04-22T23:35:04.367 INFO:tasks.workunit.client.1.smithi157.stdout:1/597: rename d4/d17/d1d/d33/d41/f67 to d4/db/d3d/dc1/fd0 0 2024-04-22T23:35:04.368 INFO:tasks.workunit.client.1.smithi157.stdout:5/560: mknod d1/d5/d7/dbd/cd1 0 2024-04-22T23:35:04.369 INFO:tasks.workunit.client.0.smithi107.stdout:3/383: dread db/d21/d27/d15/d29/d4f/f77 [0,4194304] 0 2024-04-22T23:35:04.369 INFO:tasks.workunit.client.0.smithi107.stdout:7/281: dwrite d7/d1b/f4d [0,4194304] 0 2024-04-22T23:35:04.369 INFO:tasks.workunit.client.0.smithi107.stdout:7/282: fsync d7/d21/d54/f58 0 2024-04-22T23:35:04.369 INFO:tasks.workunit.client.1.smithi157.stdout:5/561: mkdir d1/db0/dd2 0 2024-04-22T23:35:04.371 INFO:tasks.workunit.client.0.smithi107.stdout:3/384: creat db/d21/d27/d15/f86 x:0 0 0 2024-04-22T23:35:04.371 INFO:tasks.workunit.client.0.smithi107.stdout:7/283: creat d7/d1b/d51/f62 x:0 0 0 2024-04-22T23:35:04.373 INFO:tasks.workunit.client.0.smithi107.stdout:3/385: mkdir db/d21/d27/d15/d24/d87 0 2024-04-22T23:35:04.373 INFO:tasks.workunit.client.0.smithi107.stdout:3/386: write db/d21/d27/d54/d25/d53/f7b [110948,43910] 0 2024-04-22T23:35:04.373 INFO:tasks.workunit.client.0.smithi107.stdout:3/387: readlink db/d21/d27/d54/d25/l47 0 2024-04-22T23:35:04.374 INFO:tasks.workunit.client.0.smithi107.stdout:7/284: creat d7/f63 x:0 0 0 2024-04-22T23:35:04.374 INFO:tasks.workunit.client.0.smithi107.stdout:7/285: creat d7/d21/d48/d5c/f64 x:0 0 0 2024-04-22T23:35:04.374 INFO:tasks.workunit.client.0.smithi107.stdout:7/286: dread - d7/d21/d54/f58 zero size 2024-04-22T23:35:04.374 INFO:tasks.workunit.client.0.smithi107.stdout:7/287: write d7/d1b/f38 [452405,123687] 0 2024-04-22T23:35:04.374 INFO:tasks.workunit.client.0.smithi107.stdout:7/288: dread - d7/d1b/d51/f5a zero size 2024-04-22T23:35:04.374 INFO:tasks.workunit.client.0.smithi107.stdout:3/388: rmdir db/d21/d27/d54 39 2024-04-22T23:35:04.376 INFO:tasks.workunit.client.0.smithi107.stdout:7/289: link d7/l4f d7/d1b/d51/l65 0 2024-04-22T23:35:04.376 INFO:tasks.workunit.client.0.smithi107.stdout:7/290: truncate d7/f3f 688505 0 2024-04-22T23:35:04.376 INFO:tasks.workunit.client.0.smithi107.stdout:7/291: chown d7/d21/d48/f55 5398219 1 2024-04-22T23:35:04.376 INFO:tasks.workunit.client.0.smithi107.stdout:7/292: write d7/f3f [421989,22741] 0 2024-04-22T23:35:04.376 INFO:tasks.workunit.client.0.smithi107.stdout:7/293: dread - d7/d21/d48/f61 zero size 2024-04-22T23:35:04.385 INFO:tasks.workunit.client.1.smithi157.stdout:1/598: write d4/db/f14 [2196830,87570] 0 2024-04-22T23:35:04.385 INFO:tasks.workunit.client.1.smithi157.stdout:1/599: readlink d4/d17/d34/l53 0 2024-04-22T23:35:04.385 INFO:tasks.workunit.client.1.smithi157.stdout:1/600: creat d4/fd1 x:0 0 0 2024-04-22T23:35:04.387 INFO:tasks.workunit.client.1.smithi157.stdout:2/660: dwrite d7/fb1 [0,4194304] 0 2024-04-22T23:35:04.387 INFO:tasks.workunit.client.1.smithi157.stdout:4/638: dwrite d3/d11/f2a [0,4194304] 0 2024-04-22T23:35:04.389 INFO:tasks.workunit.client.1.smithi157.stdout:1/601: link d4/d17/d34/f49 d4/d17/d1d/fd2 0 2024-04-22T23:35:04.389 INFO:tasks.workunit.client.1.smithi157.stdout:1/602: readlink d4/d17/d34/d3b/l60 0 2024-04-22T23:35:04.392 INFO:tasks.workunit.client.1.smithi157.stdout:5/562: dread d1/d27/d29/d4b/fae [0,4194304] 0 2024-04-22T23:35:04.392 INFO:tasks.workunit.client.1.smithi157.stdout:5/563: chown d1/f90 87302 1 2024-04-22T23:35:04.394 INFO:tasks.workunit.client.0.smithi107.stdout:8/283: dwrite d0/d7/df/d17/f40 [0,4194304] 0 2024-04-22T23:35:04.397 INFO:tasks.workunit.client.0.smithi107.stdout:6/332: dwrite db/f2d [0,4194304] 0 2024-04-22T23:35:04.398 INFO:tasks.workunit.client.1.smithi157.stdout:2/661: mknod d7/d1f/d26/cdf 0 2024-04-22T23:35:04.398 INFO:tasks.workunit.client.1.smithi157.stdout:3/545: dwrite d0/d3/f1e [0,4194304] 0 2024-04-22T23:35:04.398 INFO:tasks.workunit.client.1.smithi157.stdout:3/546: fsync d0/d3/d34/d4a/fa8 0 2024-04-22T23:35:04.398 INFO:tasks.workunit.client.0.smithi107.stdout:9/389: dwrite d5/d7/d17/d1b/f48 [0,4194304] 0 2024-04-22T23:35:04.398 INFO:tasks.workunit.client.0.smithi107.stdout:9/390: stat d5/fc 0 2024-04-22T23:35:04.401 INFO:tasks.workunit.client.1.smithi157.stdout:5/564: creat d1/d5/dc/d5d/fd3 x:0 0 0 2024-04-22T23:35:04.401 INFO:tasks.workunit.client.1.smithi157.stdout:6/848: dwrite d1/d2f/d3b/d39/d4e/ff8 [0,4194304] 0 2024-04-22T23:35:04.402 INFO:tasks.workunit.client.0.smithi107.stdout:8/284: symlink d0/d7/l5d 0 2024-04-22T23:35:04.402 INFO:tasks.workunit.client.1.smithi157.stdout:3/547: unlink d0/f55 0 2024-04-22T23:35:04.402 INFO:tasks.workunit.client.1.smithi157.stdout:3/548: chown d0/d3/d34/d62/d9a 123507454 1 2024-04-22T23:35:04.404 INFO:tasks.workunit.client.1.smithi157.stdout:3/549: mknod d0/d3/d34/d3d/d57/d5c/cab 0 2024-04-22T23:35:04.406 INFO:tasks.workunit.client.0.smithi107.stdout:6/333: truncate db/d20/f4f 486128 0 2024-04-22T23:35:04.406 INFO:tasks.workunit.client.0.smithi107.stdout:3/389: dwrite db/d21/d27/d54/d25/f34 [0,4194304] 0 2024-04-22T23:35:04.407 INFO:tasks.workunit.client.1.smithi157.stdout:6/849: rename d1/d2f/d3b/d39/d4e/da8/de5/da7/lca to d1/d2f/d3b/d39/l111 0 2024-04-22T23:35:04.407 INFO:tasks.workunit.client.1.smithi157.stdout:0/502: dwrite d17/d2b/f66 [0,4194304] 0 2024-04-22T23:35:04.407 INFO:tasks.workunit.client.1.smithi157.stdout:0/503: fdatasync f11 0 2024-04-22T23:35:04.408 INFO:tasks.workunit.client.1.smithi157.stdout:3/550: rmdir d0/d3/d13 39 2024-04-22T23:35:04.408 INFO:tasks.workunit.client.1.smithi157.stdout:3/551: write d0/d3/f6 [6101959,97286] 0 2024-04-22T23:35:04.408 INFO:tasks.workunit.client.0.smithi107.stdout:0/362: rename d1/d27/d20/d2c/d41/d52 to d1/d84 0 2024-04-22T23:35:04.409 INFO:tasks.workunit.client.0.smithi107.stdout:0/363: write d1/d9/d14/f80 [796375,108729] 0 2024-04-22T23:35:04.409 INFO:tasks.workunit.client.0.smithi107.stdout:9/391: creat d5/d7/d27/d46/f88 x:0 0 0 2024-04-22T23:35:04.411 INFO:tasks.workunit.client.1.smithi157.stdout:6/850: creat d1/d2f/d3b/d39/d4e/da8/dac/db0/f112 x:0 0 0 2024-04-22T23:35:04.411 INFO:tasks.workunit.client.1.smithi157.stdout:5/565: write d1/d5/d7/d2f/d45/d54/d59/f70 [2415225,86916] 0 2024-04-22T23:35:04.412 INFO:tasks.workunit.client.1.smithi157.stdout:5/566: dread - d1/d27/fbc zero size 2024-04-22T23:35:04.412 INFO:tasks.workunit.client.1.smithi157.stdout:0/504: mkdir d17/d23/d34/dae 0 2024-04-22T23:35:04.413 INFO:tasks.workunit.client.1.smithi157.stdout:3/552: mkdir d0/d3/d34/d62/d9a/d9e/dac 0 2024-04-22T23:35:04.414 INFO:tasks.workunit.client.1.smithi157.stdout:6/851: truncate d1/d2f/d54/d9b/dda/f10f 2422534 0 2024-04-22T23:35:04.414 INFO:tasks.workunit.client.1.smithi157.stdout:6/852: chown d1/d2f/d54/fa5 2031239 1 2024-04-22T23:35:04.414 INFO:tasks.workunit.client.0.smithi107.stdout:8/285: rename d0/d7/df/d3b/f3c to d0/d7/f5e 0 2024-04-22T23:35:04.415 INFO:tasks.workunit.client.0.smithi107.stdout:0/364: getdents d1/d27/d20/d2c/d2e 0 2024-04-22T23:35:04.416 INFO:tasks.workunit.client.0.smithi107.stdout:0/365: read d1/d9/d14/f80 [824459,72347] 0 2024-04-22T23:35:04.416 INFO:tasks.workunit.client.0.smithi107.stdout:0/366: write d1/d27/f57 [2580084,63106] 0 2024-04-22T23:35:04.417 INFO:tasks.workunit.client.0.smithi107.stdout:9/392: truncate d5/d7/d17/f4e 250793 0 2024-04-22T23:35:04.418 INFO:tasks.workunit.client.1.smithi157.stdout:3/553: creat d0/d3/d34/d47/fad x:0 0 0 2024-04-22T23:35:04.419 INFO:tasks.workunit.client.0.smithi107.stdout:0/367: mkdir d1/d7b/d85 0 2024-04-22T23:35:04.419 INFO:tasks.workunit.client.0.smithi107.stdout:0/368: truncate d1/d9/d14/f4d 1140702 0 2024-04-22T23:35:04.422 INFO:tasks.workunit.client.0.smithi107.stdout:9/393: mknod d5/c89 0 2024-04-22T23:35:04.422 INFO:tasks.workunit.client.0.smithi107.stdout:5/327: sync 2024-04-22T23:35:04.422 INFO:tasks.workunit.client.0.smithi107.stdout:3/390: read db/d21/d27/d15/d24/d52/d60/f6b [1253665,65711] 0 2024-04-22T23:35:04.422 INFO:tasks.workunit.client.0.smithi107.stdout:3/391: truncate db/d21/d27/d15/d29/f6d 720536 0 2024-04-22T23:35:04.423 INFO:tasks.workunit.client.0.smithi107.stdout:1/360: rmdir d7 39 2024-04-22T23:35:04.423 INFO:tasks.workunit.client.0.smithi107.stdout:1/361: dread - d7/d19/d20/d39/d53/d43/d46/f4c zero size 2024-04-22T23:35:04.423 INFO:tasks.workunit.client.1.smithi157.stdout:6/853: creat d1/d2f/d54/dc4/f113 x:0 0 0 2024-04-22T23:35:04.423 INFO:tasks.workunit.client.1.smithi157.stdout:3/554: dread d0/d3/d13/d21/f8c [0,4194304] 0 2024-04-22T23:35:04.424 INFO:tasks.workunit.client.1.smithi157.stdout:3/555: creat d0/d3/d34/d62/fae x:0 0 0 2024-04-22T23:35:04.424 INFO:tasks.workunit.client.1.smithi157.stdout:3/556: read - d0/d3/d34/d3d/d57/d5c/f78 zero size 2024-04-22T23:35:04.425 INFO:tasks.workunit.client.1.smithi157.stdout:5/567: rename d1/d5/d7/c8d to d1/d5/dc/d43/d4f/cd4 0 2024-04-22T23:35:04.425 INFO:tasks.workunit.client.0.smithi107.stdout:9/394: mknod d5/d7/d27/d6f/c8a 0 2024-04-22T23:35:04.425 INFO:tasks.workunit.client.0.smithi107.stdout:5/328: truncate d9/f12 3942685 0 2024-04-22T23:35:04.425 INFO:tasks.workunit.client.0.smithi107.stdout:5/329: creat d9/db/d42/f6c x:0 0 0 2024-04-22T23:35:04.425 INFO:tasks.workunit.client.0.smithi107.stdout:5/330: read d9/db/f1d [1699947,11484] 0 2024-04-22T23:35:04.426 INFO:tasks.workunit.client.0.smithi107.stdout:3/392: mknod db/d21/d27/d15/d24/d52/c88 0 2024-04-22T23:35:04.426 INFO:tasks.workunit.client.1.smithi157.stdout:6/854: dread d1/f34 [0,4194304] 0 2024-04-22T23:35:04.427 INFO:tasks.workunit.client.0.smithi107.stdout:6/334: dread db/d20/d2a/d38/f3e [0,4194304] 0 2024-04-22T23:35:04.427 INFO:tasks.workunit.client.0.smithi107.stdout:1/362: symlink d7/d19/d20/l7c 0 2024-04-22T23:35:04.427 INFO:tasks.workunit.client.0.smithi107.stdout:1/363: readlink d7/d19/d20/d39/d53/d43/d46/d68/l79 0 2024-04-22T23:35:04.428 INFO:tasks.workunit.client.0.smithi107.stdout:9/395: symlink d5/d14/d2e/d4f/d83/l8b 0 2024-04-22T23:35:04.428 INFO:tasks.workunit.client.0.smithi107.stdout:9/396: creat d5/d29/f8c x:0 0 0 2024-04-22T23:35:04.430 INFO:tasks.workunit.client.1.smithi157.stdout:0/505: rename d17/d2b/d5b/c9f to d17/d2b/d4f/d5a/d7c/caf 0 2024-04-22T23:35:04.430 INFO:tasks.workunit.client.0.smithi107.stdout:7/294: dwrite d7/f3b [0,4194304] 0 2024-04-22T23:35:04.432 INFO:tasks.workunit.client.1.smithi157.stdout:5/568: truncate d1/d27/d29/d4b/fae 491045 0 2024-04-22T23:35:04.433 INFO:tasks.workunit.client.0.smithi107.stdout:5/331: mknod d9/db/de/d5e/c6d 0 2024-04-22T23:35:04.433 INFO:tasks.workunit.client.1.smithi157.stdout:6/855: creat d1/d2f/d3b/d39/d4e/da8/dac/dc0/f114 x:0 0 0 2024-04-22T23:35:04.434 INFO:tasks.workunit.client.0.smithi107.stdout:3/393: rmdir db/d21/d27/d54/d25/d76 0 2024-04-22T23:35:04.434 INFO:tasks.workunit.client.0.smithi107.stdout:3/394: write db/d21/d27/d15/d29/d2a/d40/f6c [970154,100347] 0 2024-04-22T23:35:04.437 INFO:tasks.workunit.client.1.smithi157.stdout:5/569: link d1/d27/d29/f2a d1/d5/d7/d2f/d9c/fd5 0 2024-04-22T23:35:04.440 INFO:tasks.workunit.client.0.smithi107.stdout:5/332: mknod d9/db/de/c6e 0 2024-04-22T23:35:04.442 INFO:tasks.workunit.client.1.smithi157.stdout:1/603: dwrite d4/d17/d34/d3b/d52/d65/dae/f99 [0,4194304] 0 2024-04-22T23:35:04.442 INFO:tasks.workunit.client.1.smithi157.stdout:1/604: chown d4/fd1 1605 1 2024-04-22T23:35:04.442 INFO:tasks.workunit.client.1.smithi157.stdout:1/605: read d4/d17/f35 [1438666,96506] 0 2024-04-22T23:35:04.442 INFO:tasks.workunit.client.1.smithi157.stdout:1/606: chown d4/d17/d1d/d33/d41/la7 2777 1 2024-04-22T23:35:04.442 INFO:tasks.workunit.client.1.smithi157.stdout:4/639: dwrite d3/d11/d1d/d73/d80/dac/db5/f94 [0,4194304] 0 2024-04-22T23:35:04.442 INFO:tasks.workunit.client.1.smithi157.stdout:1/607: creat d4/d17/d34/d75/d8a/fd3 x:0 0 0 2024-04-22T23:35:04.442 INFO:tasks.workunit.client.1.smithi157.stdout:1/608: readlink d4/d17/l21 0 2024-04-22T23:35:04.442 INFO:tasks.workunit.client.1.smithi157.stdout:4/640: creat d3/d11/d1d/d73/d80/fd7 x:0 0 0 2024-04-22T23:35:04.443 INFO:tasks.workunit.client.0.smithi107.stdout:6/335: dread db/f1d [0,4194304] 0 2024-04-22T23:35:04.444 INFO:tasks.workunit.client.1.smithi157.stdout:2/662: dwrite d7/d8/d25/f4f [0,4194304] 0 2024-04-22T23:35:04.444 INFO:tasks.workunit.client.1.smithi157.stdout:6/856: rename d1/d2f/dc7/dce/fd2 to d1/d2f/d3b/d39/d4e/da8/dac/f115 0 2024-04-22T23:35:04.445 INFO:tasks.workunit.client.0.smithi107.stdout:5/333: unlink d9/c47 0 2024-04-22T23:35:04.445 INFO:tasks.workunit.client.0.smithi107.stdout:5/334: truncate d9/d26/f41 4422649 0 2024-04-22T23:35:04.446 INFO:tasks.workunit.client.0.smithi107.stdout:6/336: mkdir db/d20/d2a/d5d 0 2024-04-22T23:35:04.447 INFO:tasks.workunit.client.0.smithi107.stdout:5/335: truncate d9/f29 3778265 0 2024-04-22T23:35:04.449 INFO:tasks.workunit.client.1.smithi157.stdout:5/570: creat d1/d5/d7/d2f/d45/fd6 x:0 0 0 2024-04-22T23:35:04.449 INFO:tasks.workunit.client.0.smithi107.stdout:5/336: rename d9/db/de/f63 to d9/d26/d6a/f6f 0 2024-04-22T23:35:04.452 INFO:tasks.workunit.client.0.smithi107.stdout:5/337: symlink d9/d26/l70 0 2024-04-22T23:35:04.454 INFO:tasks.workunit.client.0.smithi107.stdout:6/337: dread f0 [0,4194304] 0 2024-04-22T23:35:04.454 INFO:tasks.workunit.client.0.smithi107.stdout:1/364: dwrite d7/d19/d20/d39/d53/f33 [0,4194304] 0 2024-04-22T23:35:04.457 INFO:tasks.workunit.client.1.smithi157.stdout:2/663: dread d7/d1f/f52 [0,4194304] 0 2024-04-22T23:35:04.457 INFO:tasks.workunit.client.1.smithi157.stdout:2/664: read - d7/d8/dd/f71 zero size 2024-04-22T23:35:04.458 INFO:tasks.workunit.client.0.smithi107.stdout:1/365: truncate d7/d19/d2d/f3d 4129376 0 2024-04-22T23:35:04.459 INFO:tasks.workunit.client.1.smithi157.stdout:6/857: rmdir d1/d2f/dc5 0 2024-04-22T23:35:04.459 INFO:tasks.workunit.client.1.smithi157.stdout:6/858: creat d1/d2f/f116 x:0 0 0 2024-04-22T23:35:04.459 INFO:tasks.workunit.client.1.smithi157.stdout:5/571: write d1/d5/f15 [2028914,7400] 0 2024-04-22T23:35:04.459 INFO:tasks.workunit.client.1.smithi157.stdout:5/572: dread - d1/d5/dc/d43/d4f/fcf zero size 2024-04-22T23:35:04.459 INFO:tasks.workunit.client.1.smithi157.stdout:5/573: creat d1/d27/fd7 x:0 0 0 2024-04-22T23:35:04.460 INFO:tasks.workunit.client.0.smithi107.stdout:1/366: rename d7/d19/d42/c60 to d7/d19/d42/c7d 0 2024-04-22T23:35:04.460 INFO:tasks.workunit.client.0.smithi107.stdout:1/367: chown d7/d19/d20/d39/d53/f5f 455 1 2024-04-22T23:35:04.462 INFO:tasks.workunit.client.1.smithi157.stdout:2/665: unlink d7/d8/dd/cdc 0 2024-04-22T23:35:04.462 INFO:tasks.workunit.client.1.smithi157.stdout:2/666: chown d7/d8/dd/d4d 80539888 1 2024-04-22T23:35:04.462 INFO:tasks.workunit.client.1.smithi157.stdout:2/667: dread - d7/d8/d30/fc5 zero size 2024-04-22T23:35:04.463 INFO:tasks.workunit.client.1.smithi157.stdout:1/609: getdents d4/d77 0 2024-04-22T23:35:04.465 INFO:tasks.workunit.client.0.smithi107.stdout:1/368: dread d7/f10 [4194304,4194304] 0 2024-04-22T23:35:04.467 INFO:tasks.workunit.client.0.smithi107.stdout:6/338: dread db/f1f [0,4194304] 0 2024-04-22T23:35:04.467 INFO:tasks.workunit.client.0.smithi107.stdout:6/339: chown db/d20/d2a/f2f 15 1 2024-04-22T23:35:04.467 INFO:tasks.workunit.client.0.smithi107.stdout:6/340: write db/f3a [823111,50980] 0 2024-04-22T23:35:04.467 INFO:tasks.workunit.client.0.smithi107.stdout:6/341: dread - db/d20/d2c/d3b/f51 zero size 2024-04-22T23:35:04.469 INFO:tasks.workunit.client.0.smithi107.stdout:6/342: unlink db/d20/d2c/d3b/f4d 0 2024-04-22T23:35:04.469 INFO:tasks.workunit.client.0.smithi107.stdout:6/343: creat db/f5e x:0 0 0 2024-04-22T23:35:04.470 INFO:tasks.workunit.client.1.smithi157.stdout:5/574: unlink d1/d5/dc/d5d/d77/fb2 0 2024-04-22T23:35:04.471 INFO:tasks.workunit.client.1.smithi157.stdout:2/668: creat d7/d8/dca/fe0 x:0 0 0 2024-04-22T23:35:04.471 INFO:tasks.workunit.client.1.smithi157.stdout:2/669: fsync d7/d1f/d3e/f5e 0 2024-04-22T23:35:04.471 INFO:tasks.workunit.client.1.smithi157.stdout:2/670: write d7/d8/d95/dd2/dda/f89 [959220,101730] 0 2024-04-22T23:35:04.471 INFO:tasks.workunit.client.1.smithi157.stdout:2/671: creat d7/d1f/d26/fe1 x:0 0 0 2024-04-22T23:35:04.471 INFO:tasks.workunit.client.1.smithi157.stdout:6/859: rmdir d1/d38/d97 39 2024-04-22T23:35:04.471 INFO:tasks.workunit.client.0.smithi107.stdout:6/344: truncate db/f1c 3152793 0 2024-04-22T23:35:04.472 INFO:tasks.workunit.client.0.smithi107.stdout:9/397: dwrite d5/d24/f3c [0,4194304] 0 2024-04-22T23:35:04.473 INFO:tasks.workunit.client.0.smithi107.stdout:9/398: write d5/d7/d27/d46/d4b/f56 [337039,25033] 0 2024-04-22T23:35:04.473 INFO:tasks.workunit.client.0.smithi107.stdout:9/399: dread - d5/f47 zero size 2024-04-22T23:35:04.473 INFO:tasks.workunit.client.1.smithi157.stdout:0/506: dwrite d17/d2b/d52/fa2 [0,4194304] 0 2024-04-22T23:35:04.473 INFO:tasks.workunit.client.1.smithi157.stdout:0/507: chown d17/d2b 2091633182 1 2024-04-22T23:35:04.474 INFO:tasks.workunit.client.1.smithi157.stdout:0/508: dread - d17/d2b/d4f/d5a/d7c/f98 zero size 2024-04-22T23:35:04.474 INFO:tasks.workunit.client.0.smithi107.stdout:6/345: mkdir db/d20/d2a/d5f 0 2024-04-22T23:35:04.474 INFO:tasks.workunit.client.0.smithi107.stdout:6/346: write db/d20/d2a/d38/f3e [5099710,17379] 0 2024-04-22T23:35:04.476 INFO:tasks.workunit.client.0.smithi107.stdout:8/286: dwrite d0/d7/df/d3b/f3d [0,4194304] 0 2024-04-22T23:35:04.476 INFO:tasks.workunit.client.1.smithi157.stdout:5/575: dread d1/d27/d29/d4b/f4d [0,4194304] 0 2024-04-22T23:35:04.476 INFO:tasks.workunit.client.1.smithi157.stdout:2/672: mknod d7/d8/d8e/ce2 0 2024-04-22T23:35:04.476 INFO:tasks.workunit.client.1.smithi157.stdout:2/673: chown d7/d8/d30/d23/d27 11030236 1 2024-04-22T23:35:04.477 INFO:tasks.workunit.client.0.smithi107.stdout:0/369: dwrite d1/d9/f68 [0,4194304] 0 2024-04-22T23:35:04.477 INFO:tasks.workunit.client.1.smithi157.stdout:3/557: dwrite d0/d3/d34/d47/f70 [0,4194304] 0 2024-04-22T23:35:04.480 INFO:tasks.workunit.client.0.smithi107.stdout:8/287: dread d0/d7/df/f4b [0,4194304] 0 2024-04-22T23:35:04.480 INFO:tasks.workunit.client.0.smithi107.stdout:8/288: fsync d0/f5a 0 2024-04-22T23:35:04.480 INFO:tasks.workunit.client.0.smithi107.stdout:8/289: chown d0/d7/c2e 1649 1 2024-04-22T23:35:04.480 INFO:tasks.workunit.client.0.smithi107.stdout:8/290: creat d0/f5f x:0 0 0 2024-04-22T23:35:04.480 INFO:tasks.workunit.client.0.smithi107.stdout:9/400: symlink d5/d7/d17/l8d 0 2024-04-22T23:35:04.481 INFO:tasks.workunit.client.1.smithi157.stdout:2/674: truncate d7/d8/f2f 1940173 0 2024-04-22T23:35:04.481 INFO:tasks.workunit.client.1.smithi157.stdout:2/675: creat d7/d8/dd/d4d/fe3 x:0 0 0 2024-04-22T23:35:04.481 INFO:tasks.workunit.client.1.smithi157.stdout:2/676: readlink d7/d8/d30/l6f 0 2024-04-22T23:35:04.481 INFO:tasks.workunit.client.1.smithi157.stdout:1/610: dread d4/d17/d6b/f86 [0,4194304] 0 2024-04-22T23:35:04.482 INFO:tasks.workunit.client.1.smithi157.stdout:6/860: link d1/d2f/d3b/d39/d4e/da8/dac/dbe/le4 d1/d38/l117 0 2024-04-22T23:35:04.482 INFO:tasks.workunit.client.1.smithi157.stdout:6/861: read - d1/d2f/d3b/d39/d4e/da8/de5/d83/d93/fbf zero size 2024-04-22T23:35:04.483 INFO:tasks.workunit.client.1.smithi157.stdout:6/862: write d1/d17/f28 [1942076,73861] 0 2024-04-22T23:35:04.483 INFO:tasks.workunit.client.1.smithi157.stdout:0/509: dread d17/d23/d34/d67/d7f/f88 [0,4194304] 0 2024-04-22T23:35:04.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:35:04 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:35:04.484 INFO:tasks.workunit.client.1.smithi157.stdout:2/677: dread d7/d8/d30/f79 [0,4194304] 0 2024-04-22T23:35:04.485 INFO:tasks.workunit.client.0.smithi107.stdout:6/347: link db/d20/d2c/f44 db/d20/d2c/d3b/f60 0 2024-04-22T23:35:04.485 INFO:tasks.workunit.client.0.smithi107.stdout:6/348: chown db/d20/d2c/d3b 12 1 2024-04-22T23:35:04.488 INFO:tasks.workunit.client.1.smithi157.stdout:0/510: write d17/d23/d34/d67/d7f/f88 [946769,6284] 0 2024-04-22T23:35:04.491 INFO:tasks.workunit.client.0.smithi107.stdout:0/370: mkdir d1/d67/d86 0 2024-04-22T23:35:04.492 INFO:tasks.workunit.client.1.smithi157.stdout:3/558: symlink d0/d3/d34/d3d/d57/d60/d9c/laf 0 2024-04-22T23:35:04.493 INFO:tasks.workunit.client.0.smithi107.stdout:8/291: symlink d0/d7/df/d17/l60 0 2024-04-22T23:35:04.496 INFO:tasks.workunit.client.0.smithi107.stdout:9/401: mkdir d5/d7/d17/d1b/d1f/d8e 0 2024-04-22T23:35:04.496 INFO:tasks.workunit.client.0.smithi107.stdout:9/402: write d5/d14/d2e/f84 [295418,42169] 0 2024-04-22T23:35:04.497 INFO:tasks.workunit.client.0.smithi107.stdout:6/349: truncate db/d20/f4f 98945 0 2024-04-22T23:35:04.499 INFO:tasks.workunit.client.1.smithi157.stdout:6/863: rename d1/d2f/d54/d9b/cbd to d1/d2f/c118 0 2024-04-22T23:35:04.499 INFO:tasks.workunit.client.0.smithi107.stdout:0/371: rename d1/d27/d20/d2c/d41/l42 to d1/d27/d20/d40/l87 0 2024-04-22T23:35:04.500 INFO:tasks.workunit.client.0.smithi107.stdout:8/292: mknod d0/d7/df/d3b/c61 0 2024-04-22T23:35:04.501 INFO:tasks.workunit.client.1.smithi157.stdout:2/678: truncate d7/d8/d30/d5b/f73 242829 0 2024-04-22T23:35:04.501 INFO:tasks.workunit.client.0.smithi107.stdout:6/350: mkdir db/d20/d2a/d38/d61 0 2024-04-22T23:35:04.502 INFO:tasks.workunit.client.0.smithi107.stdout:6/351: chown db/d20/d2a/d5f 1644 1 2024-04-22T23:35:04.502 INFO:tasks.workunit.client.0.smithi107.stdout:1/369: dwrite d7/d19/d20/d39/f69 [0,4194304] 0 2024-04-22T23:35:04.503 INFO:tasks.workunit.client.1.smithi157.stdout:3/559: mknod d0/d7/d6c/cb0 0 2024-04-22T23:35:04.504 INFO:tasks.workunit.client.0.smithi107.stdout:8/293: truncate d0/d7/f31 1557068 0 2024-04-22T23:35:04.504 INFO:tasks.workunit.client.1.smithi157.stdout:2/679: creat d7/d8/d30/fe4 x:0 0 0 2024-04-22T23:35:04.506 INFO:tasks.workunit.client.1.smithi157.stdout:6/864: truncate d1/d2f/d3b/f32 2909024 0 2024-04-22T23:35:04.506 INFO:tasks.workunit.client.0.smithi107.stdout:6/352: symlink db/l62 0 2024-04-22T23:35:04.506 INFO:tasks.workunit.client.0.smithi107.stdout:6/353: readlink db/d20/d2a/d35/l57 0 2024-04-22T23:35:04.507 INFO:tasks.workunit.client.1.smithi157.stdout:2/680: truncate d7/d1f/d26/f44 3338990 0 2024-04-22T23:35:04.507 INFO:tasks.workunit.client.1.smithi157.stdout:2/681: fsync d7/d8/d25/d55/f6c 0 2024-04-22T23:35:04.508 INFO:tasks.workunit.client.1.smithi157.stdout:4/641: rmdir d3/d11/d1d/d73 39 2024-04-22T23:35:04.511 INFO:tasks.workunit.client.1.smithi157.stdout:6/865: symlink d1/d2f/d3b/d39/d4e/da8/de5/d83/d93/d101/l119 0 2024-04-22T23:35:04.513 INFO:tasks.workunit.client.0.smithi107.stdout:1/370: creat d7/d19/d20/d39/f7e x:0 0 0 2024-04-22T23:35:04.514 INFO:tasks.workunit.client.0.smithi107.stdout:1/371: fdatasync d7/d19/d20/d39/f69 0 2024-04-22T23:35:04.514 INFO:tasks.workunit.client.0.smithi107.stdout:1/372: chown d7/d19/d71 248 1 2024-04-22T23:35:04.514 INFO:tasks.workunit.client.1.smithi157.stdout:5/576: dwrite d1/d5/d7/d2f/d51/f97 [0,4194304] 0 2024-04-22T23:35:04.514 INFO:tasks.workunit.client.1.smithi157.stdout:1/611: dwrite d4/d5f/d7e/fab [0,4194304] 0 2024-04-22T23:35:04.514 INFO:tasks.workunit.client.1.smithi157.stdout:3/560: dread d0/f11 [0,4194304] 0 2024-04-22T23:35:04.514 INFO:tasks.workunit.client.1.smithi157.stdout:4/642: creat d3/d11/d1d/d73/da3/d31/fd8 x:0 0 0 2024-04-22T23:35:04.515 INFO:tasks.workunit.client.0.smithi107.stdout:8/294: rename d0/d7/c2e to d0/d7/d8/c62 0 2024-04-22T23:35:04.516 INFO:tasks.workunit.client.1.smithi157.stdout:6/866: mknod d1/d17/c11a 0 2024-04-22T23:35:04.516 INFO:tasks.workunit.client.1.smithi157.stdout:6/867: write d1/d2f/d3b/d39/d4e/da8/fe9 [2907607,117608] 0 2024-04-22T23:35:04.516 INFO:tasks.workunit.client.1.smithi157.stdout:0/511: dwrite d17/d2b/d52/d7e/fa6 [0,4194304] 0 2024-04-22T23:35:04.516 INFO:tasks.workunit.client.1.smithi157.stdout:0/512: dread - d17/d2b/f84 zero size 2024-04-22T23:35:04.517 INFO:tasks.workunit.client.0.smithi107.stdout:6/354: rename db/d20/d2c/d3b/l47 to db/d20/d2c/d3b/l63 0 2024-04-22T23:35:04.517 INFO:tasks.workunit.client.0.smithi107.stdout:6/355: chown db/d20/d2a/d38/f3e 0 1 2024-04-22T23:35:04.519 INFO:tasks.workunit.client.0.smithi107.stdout:1/373: getdents d7/d19/d20/d39/d53/d43/d63 0 2024-04-22T23:35:04.520 INFO:tasks.workunit.client.1.smithi157.stdout:3/561: truncate d0/d3/d34/d39/f56 310841 0 2024-04-22T23:35:04.520 INFO:tasks.workunit.client.0.smithi107.stdout:8/295: unlink d0/d3a/c3f 0 2024-04-22T23:35:04.526 INFO:tasks.workunit.client.0.smithi107.stdout:6/356: symlink db/d20/d58/l64 0 2024-04-22T23:35:04.526 INFO:tasks.workunit.client.1.smithi157.stdout:4/643: mkdir d3/d11/d30/dd9 0 2024-04-22T23:35:04.527 INFO:tasks.workunit.client.1.smithi157.stdout:3/562: write d0/d3/d34/d3d/d57/d60/f89 [47208,113207] 0 2024-04-22T23:35:04.527 INFO:tasks.workunit.client.1.smithi157.stdout:3/563: chown d0/d7/d6c/cb0 238355 1 2024-04-22T23:35:04.527 INFO:tasks.workunit.client.1.smithi157.stdout:3/564: creat d0/d3/d34/d3d/d5b/d72/fb1 x:0 0 0 2024-04-22T23:35:04.527 INFO:tasks.workunit.client.1.smithi157.stdout:3/565: creat d0/d3/d34/d47/fb2 x:0 0 0 2024-04-22T23:35:04.527 INFO:tasks.workunit.client.1.smithi157.stdout:3/566: read d0/d3/d34/d3d/d57/f65 [378797,74693] 0 2024-04-22T23:35:04.527 INFO:tasks.workunit.client.1.smithi157.stdout:3/567: chown d0/d3/d34/d3d/d57/d60/d6b 98 1 2024-04-22T23:35:04.527 INFO:tasks.workunit.client.1.smithi157.stdout:3/568: truncate d0/d3/d34/d3d/d57/d5c/f78 868918 0 2024-04-22T23:35:04.527 INFO:tasks.workunit.client.0.smithi107.stdout:8/296: unlink d0/d7/df/d33/d37/f38 0 2024-04-22T23:35:04.528 INFO:tasks.workunit.client.1.smithi157.stdout:2/682: dwrite d7/d8/d30/d23/fbe [0,4194304] 0 2024-04-22T23:35:04.528 INFO:tasks.workunit.client.1.smithi157.stdout:0/513: symlink d17/d2b/d71/lb0 0 2024-04-22T23:35:04.528 INFO:tasks.workunit.client.1.smithi157.stdout:5/577: dread d1/d27/d29/f2a [0,4194304] 0 2024-04-22T23:35:04.528 INFO:tasks.workunit.client.1.smithi157.stdout:5/578: fdatasync d1/d5/dc/f1b 0 2024-04-22T23:35:04.529 INFO:tasks.workunit.client.0.smithi107.stdout:6/357: creat db/d20/d58/f65 x:0 0 0 2024-04-22T23:35:04.529 INFO:tasks.workunit.client.0.smithi107.stdout:9/403: dwrite d5/f10 [0,4194304] 0 2024-04-22T23:35:04.531 INFO:tasks.workunit.client.0.smithi107.stdout:8/297: symlink d0/d7/d20/l63 0 2024-04-22T23:35:04.531 INFO:tasks.workunit.client.1.smithi157.stdout:4/644: creat d3/fda x:0 0 0 2024-04-22T23:35:04.531 INFO:tasks.workunit.client.1.smithi157.stdout:4/645: stat d3/d11/f1f 0 2024-04-22T23:35:04.532 INFO:tasks.workunit.client.1.smithi157.stdout:7/483: sync 2024-04-22T23:35:04.532 INFO:tasks.workunit.client.1.smithi157.stdout:7/484: fsync d0/d9/d29/d33/d37/f9b 0 2024-04-22T23:35:04.532 INFO:tasks.workunit.client.1.smithi157.stdout:7/485: chown d0/d9/db/df/d60/l7b 7811 1 2024-04-22T23:35:04.532 INFO:tasks.workunit.client.0.smithi107.stdout:6/358: mknod db/d20/d2a/d38/d4b/c66 0 2024-04-22T23:35:04.533 INFO:tasks.workunit.client.1.smithi157.stdout:9/592: sync 2024-04-22T23:35:04.534 INFO:tasks.workunit.client.1.smithi157.stdout:3/569: creat d0/d3/fb3 x:0 0 0 2024-04-22T23:35:04.534 INFO:tasks.workunit.client.1.smithi157.stdout:8/391: sync 2024-04-22T23:35:04.537 INFO:tasks.workunit.client.1.smithi157.stdout:2/683: link d7/d8/d95/dd2/dda/f97 d7/d1f/d3e/db4/fe5 0 2024-04-22T23:35:04.538 INFO:tasks.workunit.client.1.smithi157.stdout:0/514: symlink d17/lb1 0 2024-04-22T23:35:04.538 INFO:tasks.workunit.client.1.smithi157.stdout:0/515: chown d17/d2b/d4f/d5a/d7c/l99 0 1 2024-04-22T23:35:04.538 INFO:tasks.workunit.client.0.smithi107.stdout:5/338: dwrite d9/f29 [0,4194304] 0 2024-04-22T23:35:04.539 INFO:tasks.workunit.client.1.smithi157.stdout:1/612: dwrite d4/db/f36 [0,4194304] 0 2024-04-22T23:35:04.539 INFO:tasks.workunit.client.1.smithi157.stdout:1/613: write d4/db/f7d [258816,79192] 0 2024-04-22T23:35:04.539 INFO:tasks.workunit.client.0.smithi107.stdout:9/404: unlink d5/d7/d27/d69/f78 0 2024-04-22T23:35:04.540 INFO:tasks.workunit.client.1.smithi157.stdout:5/579: symlink d1/d5/dc/d43/d4f/d7f/dab/ld8 0 2024-04-22T23:35:04.541 INFO:tasks.workunit.client.0.smithi107.stdout:9/405: symlink d5/d7/d17/d1b/l8f 0 2024-04-22T23:35:04.543 INFO:tasks.workunit.client.0.smithi107.stdout:0/372: write d1/d9/f68 [4277232,1856] 0 2024-04-22T23:35:04.548 INFO:tasks.workunit.client.1.smithi157.stdout:7/486: rmdir d0/d9/de 39 2024-04-22T23:35:04.550 INFO:tasks.workunit.client.1.smithi157.stdout:3/570: symlink d0/lb4 0 2024-04-22T23:35:04.551 INFO:tasks.workunit.client.1.smithi157.stdout:3/571: fsync d0/d3/d13/d21/f3f 0 2024-04-22T23:35:04.551 INFO:tasks.workunit.client.1.smithi157.stdout:4/646: dread d3/d96/f67 [0,4194304] 0 2024-04-22T23:35:04.553 INFO:tasks.workunit.client.0.smithi107.stdout:1/374: dwrite d7/d19/d20/d39/f5a [0,4194304] 0 2024-04-22T23:35:04.553 INFO:tasks.workunit.client.1.smithi157.stdout:8/392: mknod d2/d6/d21/d5f/c88 0 2024-04-22T23:35:04.556 INFO:tasks.workunit.client.1.smithi157.stdout:2/684: mknod d7/d8/d30/d23/d27/d57/ce6 0 2024-04-22T23:35:04.559 INFO:tasks.workunit.client.1.smithi157.stdout:0/516: rename d17/d2b/d52/d7e/l91 to d17/d2b/d71/lb2 0 2024-04-22T23:35:04.562 INFO:tasks.workunit.client.1.smithi157.stdout:1/614: rmdir d4/d17/d1d/d33/d41 39 2024-04-22T23:35:04.562 INFO:tasks.workunit.client.1.smithi157.stdout:1/615: stat d4/d17/d34/d3b/d52/d65/dbc/la4 0 2024-04-22T23:35:04.562 INFO:tasks.workunit.client.0.smithi107.stdout:1/375: mknod d7/d19/d20/c7f 0 2024-04-22T23:35:04.564 INFO:tasks.workunit.client.1.smithi157.stdout:7/487: unlink d0/d9/db/d35/f93 0 2024-04-22T23:35:04.567 INFO:tasks.workunit.client.0.smithi107.stdout:1/376: creat d7/d19/f80 x:0 0 0 2024-04-22T23:35:04.568 INFO:tasks.workunit.client.0.smithi107.stdout:1/377: fsync d7/d19/d42/f6f 0 2024-04-22T23:35:04.568 INFO:tasks.workunit.client.1.smithi157.stdout:3/572: creat d0/d3/d34/d3d/d57/d60/d6b/d97/fb5 x:0 0 0 2024-04-22T23:35:04.570 INFO:tasks.workunit.client.1.smithi157.stdout:4/647: creat d3/d96/d40/dc3/fdb x:0 0 0 2024-04-22T23:35:04.570 INFO:tasks.workunit.client.1.smithi157.stdout:4/648: chown d3/d5c/d7b/fa8 907921 1 2024-04-22T23:35:04.573 INFO:tasks.workunit.client.1.smithi157.stdout:5/580: rename d1/d1f/d22/l57 to d1/d5/dc/d5d/d77/ld9 0 2024-04-22T23:35:04.576 INFO:tasks.workunit.client.0.smithi107.stdout:6/359: dread db/d20/f4f [0,4194304] 0 2024-04-22T23:35:04.577 INFO:tasks.workunit.client.1.smithi157.stdout:1/616: creat d4/d17/d34/d75/dc7/fd4 x:0 0 0 2024-04-22T23:35:04.578 INFO:tasks.workunit.client.1.smithi157.stdout:7/488: unlink d0/f21 0 2024-04-22T23:35:04.578 INFO:tasks.workunit.client.1.smithi157.stdout:2/685: write d7/d8/d25/f4f [3573054,93469] 0 2024-04-22T23:35:04.578 INFO:tasks.workunit.client.1.smithi157.stdout:2/686: creat d7/d8/d8e/d94/fe7 x:0 0 0 2024-04-22T23:35:04.580 INFO:tasks.workunit.client.1.smithi157.stdout:7/489: write d0/d9/d29/d33/d37/f50 [326164,126454] 0 2024-04-22T23:35:04.581 INFO:tasks.workunit.client.1.smithi157.stdout:3/573: symlink d0/d3/d34/d3d/d5b/d72/lb6 0 2024-04-22T23:35:04.581 INFO:tasks.workunit.client.1.smithi157.stdout:4/649: link d3/fda d3/fdc 0 2024-04-22T23:35:04.583 INFO:tasks.workunit.client.1.smithi157.stdout:5/581: rename d1/d5/d7/d2f/d51/f97 to d1/d5/dc/d43/d5e/fda 0 2024-04-22T23:35:04.583 INFO:tasks.workunit.client.1.smithi157.stdout:5/582: fdatasync d1/d5/d7/d2f/d45/fd6 0 2024-04-22T23:35:04.588 INFO:tasks.workunit.client.1.smithi157.stdout:1/617: dread d4/d17/d34/d3b/d50/f56 [0,4194304] 0 2024-04-22T23:35:04.591 INFO:tasks.workunit.client.1.smithi157.stdout:4/650: link d3/d11/d1d/d73/da3/c78 d3/d4b/d7a/cdd 0 2024-04-22T23:35:04.594 INFO:tasks.workunit.client.1.smithi157.stdout:1/618: unlink d4/f5 0 2024-04-22T23:35:04.597 INFO:tasks.workunit.client.1.smithi157.stdout:1/619: symlink d4/d5f/ld5 0 2024-04-22T23:35:04.597 INFO:tasks.workunit.client.1.smithi157.stdout:1/620: creat d4/d17/fd6 x:0 0 0 2024-04-22T23:35:04.597 INFO:tasks.workunit.client.1.smithi157.stdout:1/621: write d4/d17/d34/d3b/f42 [975677,23728] 0 2024-04-22T23:35:04.597 INFO:tasks.workunit.client.1.smithi157.stdout:1/622: write d4/d17/d1d/d33/d41/f69 [924767,79922] 0 2024-04-22T23:35:04.597 INFO:tasks.workunit.client.1.smithi157.stdout:1/623: chown d4/d17/d34/ca0 1600904 1 2024-04-22T23:35:04.597 INFO:tasks.workunit.client.1.smithi157.stdout:1/624: fdatasync d4/d17/d6b/f86 0 2024-04-22T23:35:04.597 INFO:tasks.workunit.client.1.smithi157.stdout:1/625: write d4/d17/f35 [2315123,45193] 0 2024-04-22T23:35:04.597 INFO:tasks.workunit.client.1.smithi157.stdout:1/626: fsync d4/d77/f93 0 2024-04-22T23:35:04.598 INFO:tasks.workunit.client.1.smithi157.stdout:4/651: getdents d3/d11/d34/da6 0 2024-04-22T23:35:04.599 INFO:tasks.workunit.client.0.smithi107.stdout:6/360: dread db/d20/d2c/d3b/f60 [0,4194304] 0 2024-04-22T23:35:04.600 INFO:tasks.workunit.client.1.smithi157.stdout:4/652: creat d3/d11/fde x:0 0 0 2024-04-22T23:35:04.600 INFO:tasks.workunit.client.1.smithi157.stdout:4/653: fsync d3/d96/d40/f72 0 2024-04-22T23:35:04.602 INFO:tasks.workunit.client.1.smithi157.stdout:4/654: rename d3/f55 to d3/d4b/d7a/db7/fdf 0 2024-04-22T23:35:04.612 INFO:tasks.workunit.client.0.smithi107.stdout:6/361: dread db/f3f [0,4194304] 0 2024-04-22T23:35:04.613 INFO:tasks.workunit.client.0.smithi107.stdout:6/362: symlink db/d20/d2a/d38/d4b/l67 0 2024-04-22T23:35:04.615 INFO:tasks.workunit.client.0.smithi107.stdout:6/363: symlink db/d20/l68 0 2024-04-22T23:35:04.615 INFO:tasks.workunit.client.0.smithi107.stdout:6/364: write f6 [4981663,126965] 0 2024-04-22T23:35:04.615 INFO:tasks.workunit.client.0.smithi107.stdout:9/406: dwrite d5/d7/d17/d1b/f48 [0,4194304] 0 2024-04-22T23:35:04.615 INFO:tasks.workunit.client.0.smithi107.stdout:9/407: fsync d5/d7/d17/f4e 0 2024-04-22T23:35:04.615 INFO:tasks.workunit.client.0.smithi107.stdout:6/365: mkdir db/d20/d2a/d35/d69 0 2024-04-22T23:35:04.617 INFO:tasks.workunit.client.0.smithi107.stdout:9/408: mknod d5/d14/d2e/c90 0 2024-04-22T23:35:04.619 INFO:tasks.workunit.client.0.smithi107.stdout:9/409: mkdir d5/d29/d7b/d91 0 2024-04-22T23:35:04.619 INFO:tasks.workunit.client.0.smithi107.stdout:9/410: chown d5/d14/l2a 12790576 1 2024-04-22T23:35:04.619 INFO:tasks.workunit.client.0.smithi107.stdout:9/411: truncate d5/f31 2155027 0 2024-04-22T23:35:04.621 INFO:tasks.workunit.client.0.smithi107.stdout:9/412: mknod d5/d7/d17/c92 0 2024-04-22T23:35:04.624 INFO:tasks.workunit.client.1.smithi157.stdout:9/593: dwrite d4/df/f26 [0,4194304] 0 2024-04-22T23:35:04.624 INFO:tasks.workunit.client.1.smithi157.stdout:9/594: stat d4/df/f6a 0 2024-04-22T23:35:04.625 INFO:tasks.workunit.client.0.smithi107.stdout:8/298: dwrite d0/f2f [4194304,4194304] 0 2024-04-22T23:35:04.625 INFO:tasks.workunit.client.0.smithi107.stdout:8/299: write d0/d7/f35 [552121,64256] 0 2024-04-22T23:35:04.627 INFO:tasks.workunit.client.1.smithi157.stdout:9/595: mkdir d4/d15/d23/d27/db5/db9 0 2024-04-22T23:35:04.628 INFO:tasks.workunit.client.0.smithi107.stdout:8/300: mknod d0/d7/df/d18/c64 0 2024-04-22T23:35:04.628 INFO:tasks.workunit.client.0.smithi107.stdout:8/301: stat d0/f19 0 2024-04-22T23:35:04.632 INFO:tasks.workunit.client.1.smithi157.stdout:9/596: rename d4/d15/f91 to d4/d15/d23/d27/fba 0 2024-04-22T23:35:04.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:35:04 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:35:04.632 INFO:tasks.workunit.client.0.smithi107.stdout:5/339: dwrite d9/db/f5f [0,4194304] 0 2024-04-22T23:35:04.632 INFO:tasks.workunit.client.0.smithi107.stdout:5/340: readlink d9/l21 0 2024-04-22T23:35:04.635 INFO:tasks.workunit.client.1.smithi157.stdout:9/597: dread d4/df/d61/f71 [0,4194304] 0 2024-04-22T23:35:04.637 INFO:tasks.workunit.client.1.smithi157.stdout:0/517: dwrite d17/d23/d34/f7b [4194304,4194304] 0 2024-04-22T23:35:04.637 INFO:tasks.workunit.client.1.smithi157.stdout:9/598: truncate d4/f29 747004 0 2024-04-22T23:35:04.637 INFO:tasks.workunit.client.1.smithi157.stdout:9/599: chown d4/d2d/l2f 820 1 2024-04-22T23:35:04.637 INFO:tasks.workunit.client.1.smithi157.stdout:9/600: write d4/df/d52/f7d [692291,100792] 0 2024-04-22T23:35:04.637 INFO:tasks.workunit.client.1.smithi157.stdout:9/601: chown d4/d15/d23/d27/fb1 29 1 2024-04-22T23:35:04.637 INFO:tasks.workunit.client.1.smithi157.stdout:9/602: readlink d4/l6f 0 2024-04-22T23:35:04.639 INFO:tasks.workunit.client.1.smithi157.stdout:0/518: mkdir d17/d2b/d71/db3 0 2024-04-22T23:35:04.639 INFO:tasks.workunit.client.1.smithi157.stdout:0/519: readlink d17/d38/l4a 0 2024-04-22T23:35:04.639 INFO:tasks.workunit.client.1.smithi157.stdout:0/520: chown d17/d2b/d71/db3 64840 1 2024-04-22T23:35:04.646 INFO:tasks.workunit.client.0.smithi107.stdout:5/341: dread d9/db/f5f [0,4194304] 0 2024-04-22T23:35:04.646 INFO:tasks.workunit.client.0.smithi107.stdout:5/342: fdatasync d9/db/de/d5e/f67 0 2024-04-22T23:35:04.646 INFO:tasks.workunit.client.0.smithi107.stdout:5/343: creat d9/db/de/d5e/f71 x:0 0 0 2024-04-22T23:35:04.647 INFO:tasks.workunit.client.1.smithi157.stdout:9/603: write d4/d2d/f78 [3495186,48579] 0 2024-04-22T23:35:04.650 INFO:tasks.workunit.client.1.smithi157.stdout:9/604: rename d4/f20 to d4/d15/d55/d8d/fbb 0 2024-04-22T23:35:04.650 INFO:tasks.workunit.client.1.smithi157.stdout:9/605: write d4/d15/d23/d27/fba [446816,128323] 0 2024-04-22T23:35:04.650 INFO:tasks.workunit.client.0.smithi107.stdout:5/344: getdents d9/d26 0 2024-04-22T23:35:04.650 INFO:tasks.workunit.client.0.smithi107.stdout:5/345: chown d9/db/c28 60 1 2024-04-22T23:35:04.650 INFO:tasks.workunit.client.0.smithi107.stdout:5/346: chown d9/c20 1 1 2024-04-22T23:35:04.651 INFO:tasks.workunit.client.1.smithi157.stdout:9/606: link d4/df/d61/c98 d4/d15/d23/d27/db5/cbc 0 2024-04-22T23:35:04.651 INFO:tasks.workunit.client.0.smithi107.stdout:5/347: mkdir d9/db/de/d72 0 2024-04-22T23:35:04.651 INFO:tasks.workunit.client.0.smithi107.stdout:5/348: mknod d9/db/c73 0 2024-04-22T23:35:04.652 INFO:tasks.workunit.client.0.smithi107.stdout:1/378: dwrite d7/d19/d2d/f3d [0,4194304] 0 2024-04-22T23:35:04.652 INFO:tasks.workunit.client.1.smithi157.stdout:8/393: dwrite d2/d6/d21/d5f/f3f [0,4194304] 0 2024-04-22T23:35:04.652 INFO:tasks.workunit.client.1.smithi157.stdout:9/607: mkdir d4/df/d57/dbd 0 2024-04-22T23:35:04.653 INFO:tasks.workunit.client.1.smithi157.stdout:8/394: symlink d2/d6/d21/d5f/d17/d7f/l89 0 2024-04-22T23:35:04.654 INFO:tasks.workunit.client.1.smithi157.stdout:7/490: dwrite d0/f7 [0,4194304] 0 2024-04-22T23:35:04.655 INFO:tasks.workunit.client.1.smithi157.stdout:8/395: getdents d2/d4/d30 0 2024-04-22T23:35:04.657 INFO:tasks.workunit.client.1.smithi157.stdout:8/396: truncate d2/d4/f10 4599839 0 2024-04-22T23:35:04.657 INFO:tasks.workunit.client.0.smithi107.stdout:0/373: dwrite d1/d27/d20/f6b [0,4194304] 0 2024-04-22T23:35:04.659 INFO:tasks.workunit.client.0.smithi107.stdout:0/374: mknod d1/d27/d20/d2c/d41/c88 0 2024-04-22T23:35:04.661 INFO:tasks.workunit.client.0.smithi107.stdout:0/375: rmdir d1/d9/d14 39 2024-04-22T23:35:04.665 INFO:tasks.workunit.client.1.smithi157.stdout:9/608: write d4/df/d52/fac [2647050,63725] 0 2024-04-22T23:35:04.665 INFO:tasks.workunit.client.1.smithi157.stdout:9/609: stat d4/df/d72 0 2024-04-22T23:35:04.666 INFO:tasks.workunit.client.1.smithi157.stdout:2/687: dwrite d7/d8/dd/f21 [0,4194304] 0 2024-04-22T23:35:04.666 INFO:tasks.workunit.client.1.smithi157.stdout:1/627: dwrite d4/db/f24 [0,4194304] 0 2024-04-22T23:35:04.666 INFO:tasks.workunit.client.1.smithi157.stdout:4/655: dwrite d3/d11/f2a [0,4194304] 0 2024-04-22T23:35:04.666 INFO:tasks.workunit.client.0.smithi107.stdout:0/376: rmdir d1/d9/d14/d4f 39 2024-04-22T23:35:04.667 INFO:tasks.workunit.client.0.smithi107.stdout:0/377: chown d1/d27/d20/d2c/c3d 1 1 2024-04-22T23:35:04.667 INFO:tasks.workunit.client.0.smithi107.stdout:0/378: read - d1/d2a/f77 zero size 2024-04-22T23:35:04.667 INFO:tasks.workunit.client.0.smithi107.stdout:5/349: dread d9/f37 [0,4194304] 0 2024-04-22T23:35:04.668 INFO:tasks.workunit.client.0.smithi107.stdout:0/379: link d1/d9/d14/c3b d1/d27/d20/d2c/d41/c89 0 2024-04-22T23:35:04.668 INFO:tasks.workunit.client.1.smithi157.stdout:9/610: rename d4/df/d57/c37 to d4/cbe 0 2024-04-22T23:35:04.669 INFO:tasks.workunit.client.0.smithi107.stdout:6/366: dwrite db/d20/d2a/d38/d59/f5a [0,4194304] 0 2024-04-22T23:35:04.669 INFO:tasks.workunit.client.0.smithi107.stdout:6/367: write db/f26 [1240190,21028] 0 2024-04-22T23:35:04.669 INFO:tasks.workunit.client.0.smithi107.stdout:9/413: dwrite d5/d29/f61 [0,4194304] 0 2024-04-22T23:35:04.669 INFO:tasks.workunit.client.0.smithi107.stdout:9/414: chown d5/d14/d28/f59 1030286653 1 2024-04-22T23:35:04.669 INFO:tasks.workunit.client.0.smithi107.stdout:9/415: stat d5/d14/d2e/d4f/d73 0 2024-04-22T23:35:04.669 INFO:tasks.workunit.client.0.smithi107.stdout:9/416: readlink d5/d7/d17/l5d 0 2024-04-22T23:35:04.669 INFO:tasks.workunit.client.0.smithi107.stdout:8/302: dwrite d0/d7/df/f23 [0,4194304] 0 2024-04-22T23:35:04.670 INFO:tasks.workunit.client.1.smithi157.stdout:3/574: dwrite d0/d3/d34/d39/f56 [0,4194304] 0 2024-04-22T23:35:04.670 INFO:tasks.workunit.client.0.smithi107.stdout:5/350: mknod d9/db/de/d72/c74 0 2024-04-22T23:35:04.673 INFO:tasks.workunit.client.0.smithi107.stdout:1/379: getdents d7/d19 0 2024-04-22T23:35:04.674 INFO:tasks.workunit.client.0.smithi107.stdout:1/380: read - d7/d19/d20/d39/d53/d43/d46/f4c zero size 2024-04-22T23:35:04.674 INFO:tasks.workunit.client.1.smithi157.stdout:0/521: dwrite d17/d2b/d52/f53 [0,4194304] 0 2024-04-22T23:35:04.675 INFO:tasks.workunit.client.0.smithi107.stdout:0/380: mkdir d1/d27/d8a 0 2024-04-22T23:35:04.676 INFO:tasks.workunit.client.1.smithi157.stdout:1/628: creat d4/d17/d34/d75/dcc/fd7 x:0 0 0 2024-04-22T23:35:04.679 INFO:tasks.workunit.client.1.smithi157.stdout:8/397: dread d2/d6/d21/d5f/d11/f27 [0,4194304] 0 2024-04-22T23:35:04.679 INFO:tasks.workunit.client.1.smithi157.stdout:8/398: fsync d2/d6/d21/d5f/f2f 0 2024-04-22T23:35:04.680 INFO:tasks.workunit.client.1.smithi157.stdout:9/611: truncate d4/d15/d23/f88 67202 0 2024-04-22T23:35:04.680 INFO:tasks.workunit.client.1.smithi157.stdout:0/522: dread d17/f20 [0,4194304] 0 2024-04-22T23:35:04.680 INFO:tasks.workunit.client.1.smithi157.stdout:6/868: dwrite d1/d2f/d3b/d39/d4e/da8/de5/d83/f9f [0,4194304] 0 2024-04-22T23:35:04.683 INFO:tasks.workunit.client.0.smithi107.stdout:6/368: dread db/d20/d2a/f2f [0,4194304] 0 2024-04-22T23:35:04.683 INFO:tasks.workunit.client.0.smithi107.stdout:6/369: write db/d20/d2c/d3b/f60 [730124,106819] 0 2024-04-22T23:35:04.683 INFO:tasks.workunit.client.0.smithi107.stdout:9/417: dread d5/d24/f3c [0,4194304] 0 2024-04-22T23:35:04.683 INFO:tasks.workunit.client.0.smithi107.stdout:9/418: chown d5/d7 2107014 1 2024-04-22T23:35:04.683 INFO:tasks.workunit.client.0.smithi107.stdout:0/381: dread d1/d9/d14/f80 [0,4194304] 0 2024-04-22T23:35:04.684 INFO:tasks.workunit.client.1.smithi157.stdout:2/688: rmdir d7/d8/dd 39 2024-04-22T23:35:04.684 INFO:tasks.workunit.client.1.smithi157.stdout:7/491: dwrite d0/d9/d29/d33/d37/f4c [0,4194304] 0 2024-04-22T23:35:04.684 INFO:tasks.workunit.client.1.smithi157.stdout:3/575: mkdir d0/d3/db7 0 2024-04-22T23:35:04.684 INFO:tasks.workunit.client.1.smithi157.stdout:3/576: chown d0/d3/d34/d39/l2d 10618 1 2024-04-22T23:35:04.684 INFO:tasks.workunit.client.1.smithi157.stdout:3/577: read d0/d3/d34/d39/f96 [42912,124165] 0 2024-04-22T23:35:04.685 INFO:tasks.workunit.client.1.smithi157.stdout:0/523: dread d17/d2b/f2c [0,4194304] 0 2024-04-22T23:35:04.685 INFO:tasks.workunit.client.1.smithi157.stdout:1/629: symlink d4/db/d3d/ld8 0 2024-04-22T23:35:04.685 INFO:tasks.workunit.client.1.smithi157.stdout:1/630: creat d4/d17/d34/d3b/d52/fd9 x:0 0 0 2024-04-22T23:35:04.685 INFO:tasks.workunit.client.1.smithi157.stdout:1/631: write d4/db/f6f [357917,2048] 0 2024-04-22T23:35:04.686 INFO:tasks.workunit.client.1.smithi157.stdout:8/399: getdents d2/d6/d21 0 2024-04-22T23:35:04.687 INFO:tasks.workunit.client.0.smithi107.stdout:6/370: dread db/d20/d2a/f2f [0,4194304] 0 2024-04-22T23:35:04.689 INFO:tasks.workunit.client.1.smithi157.stdout:4/656: dwrite d3/d11/d1d/d73/d80/fd7 [0,4194304] 0 2024-04-22T23:35:04.689 INFO:tasks.workunit.client.1.smithi157.stdout:4/657: creat d3/d11/d34/da4/db1/fe0 x:0 0 0 2024-04-22T23:35:04.689 INFO:tasks.workunit.client.1.smithi157.stdout:4/658: write d3/d11/d34/da4/db1/fe0 [756154,50709] 0 2024-04-22T23:35:04.689 INFO:tasks.workunit.client.1.smithi157.stdout:4/659: write d3/d11/d1d/d73/d80/dac/fbe [868465,62302] 0 2024-04-22T23:35:04.689 INFO:tasks.workunit.client.1.smithi157.stdout:4/660: write d3/d11/d1d/f41 [978230,63143] 0 2024-04-22T23:35:04.690 INFO:tasks.workunit.client.1.smithi157.stdout:4/661: write d3/d4b/d7a/f9c [1280569,66603] 0 2024-04-22T23:35:04.690 INFO:tasks.workunit.client.1.smithi157.stdout:4/662: readlink d3/d11/d1d/d73/l9e 0 2024-04-22T23:35:04.690 INFO:tasks.workunit.client.1.smithi157.stdout:7/492: truncate d0/d9/d10/f18 561018 0 2024-04-22T23:35:04.690 INFO:tasks.workunit.client.1.smithi157.stdout:7/493: chown d0/d9/d42/d55/f78 123100 1 2024-04-22T23:35:04.691 INFO:tasks.workunit.client.1.smithi157.stdout:3/578: symlink d0/d3/lb8 0 2024-04-22T23:35:04.691 INFO:tasks.workunit.client.1.smithi157.stdout:3/579: write d0/d3/d34/d3d/f69 [3150441,6194] 0 2024-04-22T23:35:04.691 INFO:tasks.workunit.client.1.smithi157.stdout:3/580: truncate d0/d7/d95/fa5 915088 0 2024-04-22T23:35:04.692 INFO:tasks.workunit.client.1.smithi157.stdout:0/524: link d17/d23/f7a d17/d2b/fb4 0 2024-04-22T23:35:04.695 INFO:tasks.workunit.client.0.smithi107.stdout:2/401: sync 2024-04-22T23:35:04.695 INFO:tasks.workunit.client.0.smithi107.stdout:4/346: sync 2024-04-22T23:35:04.695 INFO:tasks.workunit.client.0.smithi107.stdout:2/402: readlink d1/d12/d35/l77 0 2024-04-22T23:35:04.695 INFO:tasks.workunit.client.0.smithi107.stdout:4/347: write d1/f75 [1008673,15007] 0 2024-04-22T23:35:04.698 INFO:tasks.workunit.client.1.smithi157.stdout:2/689: dread d7/d8/d30/d23/f48 [0,4194304] 0 2024-04-22T23:35:04.699 INFO:tasks.workunit.client.0.smithi107.stdout:5/351: unlink d9/db/f13 0 2024-04-22T23:35:04.699 INFO:tasks.workunit.client.0.smithi107.stdout:5/352: read - d9/d26/d6a/f6f zero size 2024-04-22T23:35:04.700 INFO:tasks.workunit.client.0.smithi107.stdout:9/419: symlink d5/d7/d27/d69/l93 0 2024-04-22T23:35:04.701 INFO:tasks.workunit.client.1.smithi157.stdout:8/400: symlink d2/d6/l8a 0 2024-04-22T23:35:04.702 INFO:tasks.workunit.client.1.smithi157.stdout:8/401: fsync d2/d6/f3c 0 2024-04-22T23:35:04.702 INFO:tasks.workunit.client.1.smithi157.stdout:8/402: dread - d2/d34/d6a/d77/d42/f61 zero size 2024-04-22T23:35:04.702 INFO:tasks.workunit.client.0.smithi107.stdout:6/371: creat db/d20/d2a/f6a x:0 0 0 2024-04-22T23:35:04.703 INFO:tasks.workunit.client.1.smithi157.stdout:6/869: write d1/d2f/d3b/feb [2532128,120859] 0 2024-04-22T23:35:04.707 INFO:tasks.workunit.client.0.smithi107.stdout:2/403: creat d1/d12/f89 x:0 0 0 2024-04-22T23:35:04.710 INFO:tasks.workunit.client.0.smithi107.stdout:4/348: creat d1/d8/f7c x:0 0 0 2024-04-22T23:35:04.711 INFO:tasks.workunit.client.0.smithi107.stdout:5/353: dread d9/fd [0,4194304] 0 2024-04-22T23:35:04.713 INFO:tasks.workunit.client.1.smithi157.stdout:2/690: dread d7/d8/d30/d23/d27/f58 [0,4194304] 0 2024-04-22T23:35:04.713 INFO:tasks.workunit.client.1.smithi157.stdout:4/663: rename d3/d11/d1d/d73/da3/d31/cd1 to d3/d5c/ce1 0 2024-04-22T23:35:04.713 INFO:tasks.workunit.client.0.smithi107.stdout:2/404: mknod d1/c8a 0 2024-04-22T23:35:04.714 INFO:tasks.workunit.client.1.smithi157.stdout:9/612: dwrite d4/df/d61/f84 [0,4194304] 0 2024-04-22T23:35:04.714 INFO:tasks.workunit.client.1.smithi157.stdout:0/525: symlink d17/d2b/d71/db3/lb5 0 2024-04-22T23:35:04.714 INFO:tasks.workunit.client.1.smithi157.stdout:0/526: creat d17/d23/d34/fb6 x:0 0 0 2024-04-22T23:35:04.714 INFO:tasks.workunit.client.0.smithi107.stdout:1/381: dwrite d7/d19/d20/d39/f69 [0,4194304] 0 2024-04-22T23:35:04.714 INFO:tasks.workunit.client.0.smithi107.stdout:1/382: read d7/d19/d20/d39/d53/f29 [928202,35028] 0 2024-04-22T23:35:04.715 INFO:tasks.workunit.client.0.smithi107.stdout:8/303: dwrite d0/f19 [4194304,4194304] 0 2024-04-22T23:35:04.716 INFO:tasks.workunit.client.0.smithi107.stdout:2/405: symlink d1/d12/d35/d48/d13/l8b 0 2024-04-22T23:35:04.717 INFO:tasks.workunit.client.0.smithi107.stdout:6/372: write db/f2d [3322490,16353] 0 2024-04-22T23:35:04.717 INFO:tasks.workunit.client.1.smithi157.stdout:1/632: rmdir d4/db/d3d 39 2024-04-22T23:35:04.718 INFO:tasks.workunit.client.0.smithi107.stdout:1/383: mknod d7/d19/c81 0 2024-04-22T23:35:04.718 INFO:tasks.workunit.client.0.smithi107.stdout:1/384: chown d7/d19/d20/c7f 80250848 1 2024-04-22T23:35:04.719 INFO:tasks.workunit.client.0.smithi107.stdout:8/304: truncate d0/d7/df/d3b/f3d 3625263 0 2024-04-22T23:35:04.719 INFO:tasks.workunit.client.0.smithi107.stdout:9/420: dwrite d5/d6c/f87 [0,4194304] 0 2024-04-22T23:35:04.721 INFO:tasks.workunit.client.0.smithi107.stdout:2/406: rmdir d1/d2b/d3d/d57 39 2024-04-22T23:35:04.721 INFO:tasks.workunit.client.1.smithi157.stdout:8/403: creat d2/d6/d6b/f8b x:0 0 0 2024-04-22T23:35:04.724 INFO:tasks.workunit.client.0.smithi107.stdout:3/395: sync 2024-04-22T23:35:04.724 INFO:tasks.workunit.client.0.smithi107.stdout:7/295: sync 2024-04-22T23:35:04.725 INFO:tasks.workunit.client.0.smithi107.stdout:9/421: dread d5/d14/f2f [0,4194304] 0 2024-04-22T23:35:04.725 INFO:tasks.workunit.client.0.smithi107.stdout:9/422: write d5/d7/d27/d46/f74 [5938792,39377] 0 2024-04-22T23:35:04.725 INFO:tasks.workunit.client.0.smithi107.stdout:9/423: stat d5/d7/d17/d1b/f30 0 2024-04-22T23:35:04.725 INFO:tasks.workunit.client.0.smithi107.stdout:9/424: chown d5/d7/d17/d1b/d1f/c5f 7 1 2024-04-22T23:35:04.725 INFO:tasks.workunit.client.0.smithi107.stdout:8/305: mknod d0/d7/df/d18/d51/c65 0 2024-04-22T23:35:04.730 INFO:tasks.workunit.client.1.smithi157.stdout:7/494: rmdir d0/d9/d29/d49 39 2024-04-22T23:35:04.730 INFO:tasks.workunit.client.0.smithi107.stdout:2/407: rename d1/d12/l15 to d1/d12/l8c 0 2024-04-22T23:35:04.731 INFO:tasks.workunit.client.0.smithi107.stdout:1/385: symlink d7/l82 0 2024-04-22T23:35:04.733 INFO:tasks.workunit.client.0.smithi107.stdout:3/396: creat db/d21/d27/d15/d24/d87/f89 x:0 0 0 2024-04-22T23:35:04.733 INFO:tasks.workunit.client.0.smithi107.stdout:3/397: chown db/d21/d27/d15/d29/d2a/d40/f80 1 1 2024-04-22T23:35:04.733 INFO:tasks.workunit.client.0.smithi107.stdout:3/398: chown db/d21/d27/d15/d29/d2a/d45 799842956 1 2024-04-22T23:35:04.734 INFO:tasks.workunit.client.1.smithi157.stdout:0/527: truncate d17/d23/f32 20739 0 2024-04-22T23:35:04.734 INFO:tasks.workunit.client.0.smithi107.stdout:9/425: mknod d5/d7/d17/c94 0 2024-04-22T23:35:04.736 INFO:tasks.workunit.client.1.smithi157.stdout:1/633: mknod d4/d17/cda 0 2024-04-22T23:35:04.736 INFO:tasks.workunit.client.0.smithi107.stdout:3/399: write f0 [4430984,88206] 0 2024-04-22T23:35:04.736 INFO:tasks.workunit.client.0.smithi107.stdout:3/400: write db/d21/d27/d15/d29/d2a/d40/f80 [16620,93426] 0 2024-04-22T23:35:04.737 INFO:tasks.workunit.client.0.smithi107.stdout:8/306: rename d0/d7/df/l3e to d0/d7/df/d33/d37/l66 0 2024-04-22T23:35:04.737 INFO:tasks.workunit.client.0.smithi107.stdout:3/401: write db/d21/d27/f41 [42683,101551] 0 2024-04-22T23:35:04.737 INFO:tasks.workunit.client.1.smithi157.stdout:8/404: mknod d2/d34/d6a/d77/d42/c8c 0 2024-04-22T23:35:04.737 INFO:tasks.workunit.client.0.smithi107.stdout:2/408: mknod d1/d2b/d49/d87/c8d 0 2024-04-22T23:35:04.737 INFO:tasks.workunit.client.0.smithi107.stdout:2/409: read - d1/d67/f5d zero size 2024-04-22T23:35:04.738 INFO:tasks.workunit.client.0.smithi107.stdout:2/410: fdatasync d1/d12/d35/f7a 0 2024-04-22T23:35:04.738 INFO:tasks.workunit.client.1.smithi157.stdout:6/870: getdents d1/d2f/d3b/d39/d4e 0 2024-04-22T23:35:04.739 INFO:tasks.workunit.client.0.smithi107.stdout:1/386: unlink d7/d19/c28 0 2024-04-22T23:35:04.739 INFO:tasks.workunit.client.0.smithi107.stdout:7/296: getdents d7/d1b/d51 0 2024-04-22T23:35:04.739 INFO:tasks.workunit.client.1.smithi157.stdout:7/495: getdents d0/d9/d42/d55/d56 0 2024-04-22T23:35:04.739 INFO:tasks.workunit.client.1.smithi157.stdout:7/496: write d0/d9/d42/f8c [743667,27269] 0 2024-04-22T23:35:04.740 INFO:tasks.workunit.client.1.smithi157.stdout:7/497: fdatasync d0/d9/d10/f1f 0 2024-04-22T23:35:04.740 INFO:tasks.workunit.client.0.smithi107.stdout:9/426: creat d5/d29/d7b/d91/f95 x:0 0 0 2024-04-22T23:35:04.740 INFO:tasks.workunit.client.0.smithi107.stdout:9/427: fsync d5/f47 0 2024-04-22T23:35:04.740 INFO:tasks.workunit.client.0.smithi107.stdout:9/428: truncate d5/d14/d2e/d43/f68 644107 0 2024-04-22T23:35:04.741 INFO:tasks.workunit.client.0.smithi107.stdout:3/402: creat db/d21/d27/d15/d29/d2a/d45/f8a x:0 0 0 2024-04-22T23:35:04.741 INFO:tasks.workunit.client.1.smithi157.stdout:0/528: link d17/d2b/d4f/d5a/d7c/c83 d17/d23/d34/cb7 0 2024-04-22T23:35:04.741 INFO:tasks.workunit.client.1.smithi157.stdout:0/529: read - d17/d23/d34/d67/d7f/f85 zero size 2024-04-22T23:35:04.742 INFO:tasks.workunit.client.1.smithi157.stdout:1/634: creat d4/d17/d1d/d33/d41/d54/fdb x:0 0 0 2024-04-22T23:35:04.743 INFO:tasks.workunit.client.1.smithi157.stdout:1/635: stat d4/db/f14 0 2024-04-22T23:35:04.743 INFO:tasks.workunit.client.1.smithi157.stdout:1/636: dread - d4/d17/d1d/d33/f9a zero size 2024-04-22T23:35:04.743 INFO:tasks.workunit.client.1.smithi157.stdout:1/637: write d4/d17/d34/d3b/f66 [582103,123423] 0 2024-04-22T23:35:04.744 INFO:tasks.workunit.client.1.smithi157.stdout:8/405: mkdir d2/d6/d21/d8d 0 2024-04-22T23:35:04.744 INFO:tasks.workunit.client.0.smithi107.stdout:2/411: rename d1/d12/d35/d48/d13/d4f/f86 to d1/d2b/d49/d7e/f8e 0 2024-04-22T23:35:04.744 INFO:tasks.workunit.client.0.smithi107.stdout:2/412: fdatasync d1/d12/d35/d48/d13/f74 0 2024-04-22T23:35:04.744 INFO:tasks.workunit.client.0.smithi107.stdout:2/413: read - d1/f7c zero size 2024-04-22T23:35:04.744 INFO:tasks.workunit.client.0.smithi107.stdout:1/387: creat d7/d19/d20/d39/d53/d43/d63/f83 x:0 0 0 2024-04-22T23:35:04.745 INFO:tasks.workunit.client.1.smithi157.stdout:6/871: rmdir d1/d2f/d3b/d39/d4e/da8/dac/ddd/ddf 39 2024-04-22T23:35:04.745 INFO:tasks.workunit.client.1.smithi157.stdout:6/872: write d1/d2f/d3b/d49/f7d [1068832,16800] 0 2024-04-22T23:35:04.745 INFO:tasks.workunit.client.0.smithi107.stdout:7/297: mkdir d7/d2d/d66 0 2024-04-22T23:35:04.746 INFO:tasks.workunit.client.0.smithi107.stdout:7/298: fdatasync d7/d1b/f28 0 2024-04-22T23:35:04.746 INFO:tasks.workunit.client.0.smithi107.stdout:8/307: write d0/d7/d8/f1e [3263509,72925] 0 2024-04-22T23:35:04.746 INFO:tasks.workunit.client.0.smithi107.stdout:8/308: write d0/d7/f31 [1652771,22504] 0 2024-04-22T23:35:04.746 INFO:tasks.workunit.client.0.smithi107.stdout:8/309: chown d0/d7/f5e 5340 1 2024-04-22T23:35:04.747 INFO:tasks.workunit.client.1.smithi157.stdout:3/581: rename d0/d3/d13/d21 to d0/d3/d34/db9 0 2024-04-22T23:35:04.747 INFO:tasks.workunit.client.1.smithi157.stdout:3/582: write d0/d3/d34/d3d/d5b/f94 [62490,65915] 0 2024-04-22T23:35:04.748 INFO:tasks.workunit.client.1.smithi157.stdout:3/583: chown d0/d3/d34/d3d/d57/d60/c71 228966 1 2024-04-22T23:35:04.748 INFO:tasks.workunit.client.0.smithi107.stdout:5/354: dwrite d9/d26/f64 [0,4194304] 0 2024-04-22T23:35:04.749 INFO:tasks.workunit.client.0.smithi107.stdout:3/403: link db/d21/d27/d15/d29/d2a/l1e db/d21/d27/d36/l8b 0 2024-04-22T23:35:04.749 INFO:tasks.workunit.client.0.smithi107.stdout:2/414: mknod d1/d2b/d59/c8f 0 2024-04-22T23:35:04.749 INFO:tasks.workunit.client.0.smithi107.stdout:2/415: dread - d1/d12/d35/d48/d13/d4f/f53 zero size 2024-04-22T23:35:04.749 INFO:tasks.workunit.client.0.smithi107.stdout:2/416: readlink d1/d12/d35/d48/d13/l28 0 2024-04-22T23:35:04.751 INFO:tasks.workunit.client.0.smithi107.stdout:8/310: truncate d0/d7/df/d33/d37/f49 922066 0 2024-04-22T23:35:04.751 INFO:tasks.workunit.client.0.smithi107.stdout:8/311: stat d0/d7/df/d17/l60 0 2024-04-22T23:35:04.751 INFO:tasks.workunit.client.1.smithi157.stdout:1/638: unlink d4/db/f2e 0 2024-04-22T23:35:04.753 INFO:tasks.workunit.client.1.smithi157.stdout:6/873: mkdir d1/d2f/d3b/d39/d4e/da8/de5/d83/d93/d11b 0 2024-04-22T23:35:04.754 INFO:tasks.workunit.client.1.smithi157.stdout:6/874: dread - d1/d2f/d3b/d39/d4e/da8/dac/fbc zero size 2024-04-22T23:35:04.754 INFO:tasks.workunit.client.0.smithi107.stdout:2/417: rename d1/d2b/f6e to d1/d2b/d59/f90 0 2024-04-22T23:35:04.755 INFO:tasks.workunit.client.1.smithi157.stdout:2/691: dwrite d7/d8/dd/f21 [0,4194304] 0 2024-04-22T23:35:04.755 INFO:tasks.workunit.client.1.smithi157.stdout:4/664: rename d3/d4b/d7a/db7/f65 to d3/d11/d1d/daf/fe2 0 2024-04-22T23:35:04.756 INFO:tasks.workunit.client.1.smithi157.stdout:4/665: dread - d3/d11/d1d/d73/da3/d31/d91/fa5 zero size 2024-04-22T23:35:04.756 INFO:tasks.workunit.client.0.smithi107.stdout:2/418: rename d1/d12/d35/d48/l75 to d1/d12/d35/l91 0 2024-04-22T23:35:04.759 INFO:tasks.workunit.client.0.smithi107.stdout:9/429: dread d5/d29/f51 [0,4194304] 0 2024-04-22T23:35:04.759 INFO:tasks.workunit.client.0.smithi107.stdout:2/419: rename d1/d12/d35/d54/c64 to d1/d2b/d3d/c92 0 2024-04-22T23:35:04.766 INFO:tasks.workunit.client.0.smithi107.stdout:6/373: dwrite db/f1f [0,4194304] 0 2024-04-22T23:35:04.768 INFO:tasks.workunit.client.1.smithi157.stdout:3/584: write d0/d3/d34/f5d [961489,93864] 0 2024-04-22T23:35:04.770 INFO:tasks.workunit.client.0.smithi107.stdout:9/430: dread d5/d29/f51 [0,4194304] 0 2024-04-22T23:35:04.770 INFO:tasks.workunit.client.0.smithi107.stdout:9/431: chown d5/d14/d2e/d4f/d73/l7c 36566714 1 2024-04-22T23:35:04.771 INFO:tasks.workunit.client.1.smithi157.stdout:2/692: dread d7/d8/d30/d23/f48 [0,4194304] 0 2024-04-22T23:35:04.773 INFO:tasks.workunit.client.0.smithi107.stdout:3/404: write db/d21/d27/d54/d25/f34 [2224,28209] 0 2024-04-22T23:35:04.773 INFO:tasks.workunit.client.0.smithi107.stdout:4/349: dwrite d1/d8/f3f [0,4194304] 0 2024-04-22T23:35:04.778 INFO:tasks.workunit.client.1.smithi157.stdout:9/613: rename d4/df/d52/c54 to d4/df/d57/dbd/cbf 0 2024-04-22T23:35:04.779 INFO:tasks.workunit.client.1.smithi157.stdout:0/530: dwrite d17/d2b/d52/f3f [0,4194304] 0 2024-04-22T23:35:04.779 INFO:tasks.workunit.client.1.smithi157.stdout:0/531: fsync d17/d2b/f3d 0 2024-04-22T23:35:04.779 INFO:tasks.workunit.client.1.smithi157.stdout:0/532: fsync d17/d2b/f66 0 2024-04-22T23:35:04.779 INFO:tasks.workunit.client.1.smithi157.stdout:0/533: write d17/d23/d34/fb6 [307099,38433] 0 2024-04-22T23:35:04.780 INFO:tasks.workunit.client.1.smithi157.stdout:6/875: symlink d1/d38/l11c 0 2024-04-22T23:35:04.782 INFO:tasks.workunit.client.1.smithi157.stdout:7/498: rename d0/d9/db/df/d68/c71 to d0/d9/d42/d55/d56/ca0 0 2024-04-22T23:35:04.783 INFO:tasks.workunit.client.0.smithi107.stdout:3/405: truncate db/d21/d27/f39 2953262 0 2024-04-22T23:35:04.785 INFO:tasks.workunit.client.1.smithi157.stdout:6/876: symlink d1/l11d 0 2024-04-22T23:35:04.789 INFO:tasks.workunit.client.1.smithi157.stdout:7/499: unlink d0/d9/d42/d55/f94 0 2024-04-22T23:35:04.790 INFO:tasks.workunit.client.0.smithi107.stdout:9/432: dread d5/d7/f70 [0,4194304] 0 2024-04-22T23:35:04.791 INFO:tasks.workunit.client.0.smithi107.stdout:5/355: dwrite d9/db/d42/f6c [0,4194304] 0 2024-04-22T23:35:04.792 INFO:tasks.workunit.client.0.smithi107.stdout:4/350: creat d1/d21/d29/d2e/d3b/f7d x:0 0 0 2024-04-22T23:35:04.792 INFO:tasks.workunit.client.1.smithi157.stdout:0/534: rename d17/d2b/lab to d17/d2b/d4f/d5a/da1/lb8 0 2024-04-22T23:35:04.792 INFO:tasks.workunit.client.1.smithi157.stdout:0/535: creat d17/d38/fb9 x:0 0 0 2024-04-22T23:35:04.794 INFO:tasks.workunit.client.1.smithi157.stdout:6/877: creat d1/d38/daf/f11e x:0 0 0 2024-04-22T23:35:04.794 INFO:tasks.workunit.client.0.smithi107.stdout:6/374: read db/f29 [7518858,62627] 0 2024-04-22T23:35:04.796 INFO:tasks.workunit.client.1.smithi157.stdout:7/500: symlink d0/la1 0 2024-04-22T23:35:04.796 INFO:tasks.workunit.client.1.smithi157.stdout:7/501: stat d0/d9/d29/f4d 0 2024-04-22T23:35:04.798 INFO:tasks.workunit.client.1.smithi157.stdout:6/878: symlink d1/d2f/d3b/d39/d4e/da8/de5/d83/d93/d11b/l11f 0 2024-04-22T23:35:04.798 INFO:tasks.workunit.client.1.smithi157.stdout:6/879: chown d1/d36/fd1 30 1 2024-04-22T23:35:04.798 INFO:tasks.workunit.client.1.smithi157.stdout:8/406: dwrite d2/d6/d21/f54 [0,4194304] 0 2024-04-22T23:35:04.799 INFO:tasks.workunit.client.1.smithi157.stdout:8/407: truncate d2/f69 659870 0 2024-04-22T23:35:04.799 INFO:tasks.workunit.client.1.smithi157.stdout:8/408: write d2/d34/d6a/f76 [343456,19915] 0 2024-04-22T23:35:04.799 INFO:tasks.workunit.client.0.smithi107.stdout:7/299: dwrite d7/f11 [0,4194304] 0 2024-04-22T23:35:04.801 INFO:tasks.workunit.client.0.smithi107.stdout:3/406: creat db/d21/d27/d15/f8c x:0 0 0 2024-04-22T23:35:04.803 INFO:tasks.workunit.client.1.smithi157.stdout:8/409: creat d2/d6/d21/d5f/d17/f8e x:0 0 0 2024-04-22T23:35:04.803 INFO:tasks.workunit.client.0.smithi107.stdout:9/433: mkdir d5/d7/d17/d1b/d1f/d36/d96 0 2024-04-22T23:35:04.804 INFO:tasks.workunit.client.0.smithi107.stdout:9/434: readlink d5/d7/l85 0 2024-04-22T23:35:04.804 INFO:tasks.workunit.client.0.smithi107.stdout:9/435: fsync d5/d29/f4d 0 2024-04-22T23:35:04.804 INFO:tasks.workunit.client.0.smithi107.stdout:9/436: chown d5/d7/d17/l8d 219 1 2024-04-22T23:35:04.804 INFO:tasks.workunit.client.0.smithi107.stdout:4/351: dread d1/d21/d29/d2e/f46 [0,4194304] 0 2024-04-22T23:35:04.805 INFO:tasks.workunit.client.1.smithi157.stdout:8/410: symlink d2/d34/d6a/d77/d42/d58/l8f 0 2024-04-22T23:35:04.806 INFO:tasks.workunit.client.1.smithi157.stdout:7/502: dread d0/d9/db/f5a [0,4194304] 0 2024-04-22T23:35:04.807 INFO:tasks.workunit.client.1.smithi157.stdout:5/583: sync 2024-04-22T23:35:04.807 INFO:tasks.workunit.client.1.smithi157.stdout:1/639: dwrite d4/d17/f35 [0,4194304] 0 2024-04-22T23:35:04.810 INFO:tasks.workunit.client.0.smithi107.stdout:5/356: mkdir d9/db/d42/d75 0 2024-04-22T23:35:04.816 INFO:tasks.workunit.client.0.smithi107.stdout:7/300: symlink d7/d21/d48/d5c/l67 0 2024-04-22T23:35:04.816 INFO:tasks.workunit.client.0.smithi107.stdout:7/301: creat d7/d1b/f68 x:0 0 0 2024-04-22T23:35:04.816 INFO:tasks.workunit.client.1.smithi157.stdout:6/880: write d1/d2f/d3b/d39/d4e/da8/de5/d83/f59 [3347442,9192] 0 2024-04-22T23:35:04.819 INFO:tasks.workunit.client.1.smithi157.stdout:8/411: write d2/d6/f3c [356434,112902] 0 2024-04-22T23:35:04.819 INFO:tasks.workunit.client.1.smithi157.stdout:8/412: write d2/d6/d21/d5f/f2f [3511877,124873] 0 2024-04-22T23:35:04.819 INFO:tasks.workunit.client.1.smithi157.stdout:8/413: chown d2/d6/d21/d5f/l1d 3 1 2024-04-22T23:35:04.819 INFO:tasks.workunit.client.1.smithi157.stdout:8/414: chown d2/d6/d21/d5f/f6f 392205861 1 2024-04-22T23:35:04.820 INFO:tasks.workunit.client.0.smithi107.stdout:3/407: rename db/d21/d27/d54/d25/l47 to db/d21/d27/d15/d29/l8d 0 2024-04-22T23:35:04.821 INFO:tasks.workunit.client.0.smithi107.stdout:4/352: mkdir d1/d21/d29/d2e/d7e 0 2024-04-22T23:35:04.822 INFO:tasks.workunit.client.1.smithi157.stdout:2/693: dwrite d7/d8/d95/dd2/dda/f89 [0,4194304] 0 2024-04-22T23:35:04.823 INFO:tasks.workunit.client.1.smithi157.stdout:8/415: symlink d2/d6/d6b/l90 0 2024-04-22T23:35:04.827 INFO:tasks.workunit.client.1.smithi157.stdout:2/694: creat d7/d8/d95/dd1/fe8 x:0 0 0 2024-04-22T23:35:04.827 INFO:tasks.workunit.client.0.smithi107.stdout:5/357: dread d9/f58 [0,4194304] 0 2024-04-22T23:35:04.827 INFO:tasks.workunit.client.0.smithi107.stdout:9/437: rename d5/d7/l26 to d5/d14/d2e/d4f/d83/l97 0 2024-04-22T23:35:04.830 INFO:tasks.workunit.client.0.smithi107.stdout:3/408: rename db/d21/d27/f41 to db/d21/d27/d54/d25/d6e/f8e 0 2024-04-22T23:35:04.831 INFO:tasks.workunit.client.0.smithi107.stdout:9/438: dread d5/d14/d2e/f84 [0,4194304] 0 2024-04-22T23:35:04.831 INFO:tasks.workunit.client.0.smithi107.stdout:9/439: dread - d5/d14/d28/f81 zero size 2024-04-22T23:35:04.831 INFO:tasks.workunit.client.0.smithi107.stdout:9/440: write d5/f11 [1337740,55495] 0 2024-04-22T23:35:04.831 INFO:tasks.workunit.client.0.smithi107.stdout:9/441: fdatasync d5/d7/f70 0 2024-04-22T23:35:04.832 INFO:tasks.workunit.client.0.smithi107.stdout:9/442: link d5/d29/f3a d5/d24/f98 0 2024-04-22T23:35:04.834 INFO:tasks.workunit.client.0.smithi107.stdout:4/353: dread d1/d7/f33 [0,4194304] 0 2024-04-22T23:35:04.835 INFO:tasks.workunit.client.0.smithi107.stdout:9/443: mknod d5/d7/c99 0 2024-04-22T23:35:04.835 INFO:tasks.workunit.client.0.smithi107.stdout:3/409: dread db/d21/d27/f20 [0,4194304] 0 2024-04-22T23:35:04.836 INFO:tasks.workunit.client.1.smithi157.stdout:8/416: rmdir d2/d34/d6a/d77 39 2024-04-22T23:35:04.836 INFO:tasks.workunit.client.1.smithi157.stdout:8/417: write d2/f32 [2662947,96997] 0 2024-04-22T23:35:04.836 INFO:tasks.workunit.client.1.smithi157.stdout:8/418: write d2/d6/d21/d5f/f55 [986829,79604] 0 2024-04-22T23:35:04.837 INFO:tasks.workunit.client.0.smithi107.stdout:4/354: rename d1/d7/d53/d66/d6b/f4e to d1/d8/f7f 0 2024-04-22T23:35:04.837 INFO:tasks.workunit.client.1.smithi157.stdout:8/419: mknod d2/d6/d21/c91 0 2024-04-22T23:35:04.837 INFO:tasks.workunit.client.0.smithi107.stdout:9/444: symlink d5/d7/d17/d1b/d1f/d36/d96/l9a 0 2024-04-22T23:35:04.838 INFO:tasks.workunit.client.0.smithi107.stdout:3/410: mkdir db/d21/d7f/d8f 0 2024-04-22T23:35:04.839 INFO:tasks.workunit.client.1.smithi157.stdout:8/420: link d2/d6/d21/d5f/f3f d2/d34/d6a/d77/d42/d58/f92 0 2024-04-22T23:35:04.840 INFO:tasks.workunit.client.0.smithi107.stdout:4/355: getdents d1/d1b 0 2024-04-22T23:35:04.846 INFO:tasks.workunit.client.1.smithi157.stdout:2/695: write d7/d1f/f52 [3864166,116010] 0 2024-04-22T23:35:04.846 INFO:tasks.workunit.client.1.smithi157.stdout:2/696: readlink d7/d8/d8e/l98 0 2024-04-22T23:35:04.847 INFO:tasks.workunit.client.1.smithi157.stdout:2/697: mknod d7/d1f/d26/ce9 0 2024-04-22T23:35:04.848 INFO:tasks.workunit.client.1.smithi157.stdout:2/698: mkdir d7/d8/d95/dea 0 2024-04-22T23:35:04.848 INFO:tasks.workunit.client.1.smithi157.stdout:2/699: chown d7/d8/d30/fe4 14 1 2024-04-22T23:35:04.849 INFO:tasks.workunit.client.1.smithi157.stdout:2/700: symlink d7/d1f/d26/leb 0 2024-04-22T23:35:04.849 INFO:tasks.workunit.client.1.smithi157.stdout:9/614: dwrite d4/df/f62 [0,4194304] 0 2024-04-22T23:35:04.850 INFO:tasks.workunit.client.0.smithi107.stdout:2/420: dwrite d1/d12/d35/d48/d13/f40 [0,4194304] 0 2024-04-22T23:35:04.850 INFO:tasks.workunit.client.1.smithi157.stdout:0/536: dwrite d17/d23/d34/d67/d7f/f70 [0,4194304] 0 2024-04-22T23:35:04.851 INFO:tasks.workunit.client.1.smithi157.stdout:9/615: mkdir d4/d15/d23/d27/dc0 0 2024-04-22T23:35:04.851 INFO:tasks.workunit.client.1.smithi157.stdout:9/616: creat d4/df/d61/fc1 x:0 0 0 2024-04-22T23:35:04.851 INFO:tasks.workunit.client.1.smithi157.stdout:9/617: truncate d4/d15/d23/d67/f93 420469 0 2024-04-22T23:35:04.852 INFO:tasks.workunit.client.0.smithi107.stdout:2/421: truncate d1/d2b/d3d/d57/f83 591698 0 2024-04-22T23:35:04.852 INFO:tasks.workunit.client.1.smithi157.stdout:0/537: link d17/d2b/d4f/d5a/da1/da8/c6f d17/d2b/d52/cba 0 2024-04-22T23:35:04.854 INFO:tasks.workunit.client.0.smithi107.stdout:9/445: write d5/d14/d2e/f3f [3506850,38201] 0 2024-04-22T23:35:04.856 INFO:tasks.workunit.client.1.smithi157.stdout:2/701: read d7/d8/d30/f76 [1669437,20719] 0 2024-04-22T23:35:04.856 INFO:tasks.workunit.client.1.smithi157.stdout:2/702: stat d7/d8/d95/f9a 0 2024-04-22T23:35:04.856 INFO:tasks.workunit.client.0.smithi107.stdout:3/411: dread db/d21/d27/d15/d29/d2a/f43 [4194304,4194304] 0 2024-04-22T23:35:04.857 INFO:tasks.workunit.client.0.smithi107.stdout:9/446: creat d5/d14/d2e/d4f/d73/f9b x:0 0 0 2024-04-22T23:35:04.857 INFO:tasks.workunit.client.0.smithi107.stdout:9/447: chown d5/d7/d17/d1b/d1f/d36/d53/c82 2963669 1 2024-04-22T23:35:04.857 INFO:tasks.workunit.client.0.smithi107.stdout:9/448: write d5/d7/d17/d1b/d1f/d36/f57 [261665,27953] 0 2024-04-22T23:35:04.858 INFO:tasks.workunit.client.0.smithi107.stdout:2/422: dread d1/f4c [0,4194304] 0 2024-04-22T23:35:04.858 INFO:tasks.workunit.client.0.smithi107.stdout:2/423: fdatasync d1/d12/d35/d48/d13/f40 0 2024-04-22T23:35:04.858 INFO:tasks.workunit.client.0.smithi107.stdout:2/424: chown d1/c1d 6523 1 2024-04-22T23:35:04.858 INFO:tasks.workunit.client.0.smithi107.stdout:2/425: chown d1/c8a 25 1 2024-04-22T23:35:04.858 INFO:tasks.workunit.client.0.smithi107.stdout:2/426: dread - d1/d2b/d3d/f5e zero size 2024-04-22T23:35:04.858 INFO:tasks.workunit.client.0.smithi107.stdout:3/412: symlink db/d21/d27/d15/d24/d52/l90 0 2024-04-22T23:35:04.859 INFO:tasks.workunit.client.0.smithi107.stdout:3/413: fdatasync db/d21/d27/d15/d29/d2a/d40/f80 0 2024-04-22T23:35:04.859 INFO:tasks.workunit.client.0.smithi107.stdout:3/414: readlink db/d21/d27/d36/l78 0 2024-04-22T23:35:04.859 INFO:tasks.workunit.client.1.smithi157.stdout:5/584: dwrite d1/d5/dc/f1b [0,4194304] 0 2024-04-22T23:35:04.859 INFO:tasks.workunit.client.1.smithi157.stdout:6/881: dwrite d1/f2c [4194304,4194304] 0 2024-04-22T23:35:04.860 INFO:tasks.workunit.client.1.smithi157.stdout:1/640: dwrite d4/d17/d34/d75/fbb [0,4194304] 0 2024-04-22T23:35:04.860 INFO:tasks.workunit.client.0.smithi107.stdout:6/375: dwrite db/fd [0,4194304] 0 2024-04-22T23:35:04.860 INFO:tasks.workunit.client.0.smithi107.stdout:6/376: read db/d20/f4f [12235,121146] 0 2024-04-22T23:35:04.860 INFO:tasks.workunit.client.0.smithi107.stdout:7/302: dwrite d7/d21/d48/d5c/f64 [0,4194304] 0 2024-04-22T23:35:04.860 INFO:tasks.workunit.client.0.smithi107.stdout:7/303: truncate d7/d21/d48/f55 629996 0 2024-04-22T23:35:04.860 INFO:tasks.workunit.client.0.smithi107.stdout:7/304: fdatasync d7/d1b/d51/f5a 0 2024-04-22T23:35:04.860 INFO:tasks.workunit.client.0.smithi107.stdout:5/358: dwrite d9/db/d42/f6c [0,4194304] 0 2024-04-22T23:35:04.860 INFO:tasks.workunit.client.0.smithi107.stdout:3/415: dread db/d21/d27/d15/d29/f6d [0,4194304] 0 2024-04-22T23:35:04.860 INFO:tasks.workunit.client.0.smithi107.stdout:9/449: symlink d5/d14/l9c 0 2024-04-22T23:35:04.862 INFO:tasks.workunit.client.0.smithi107.stdout:2/427: mknod d1/d2b/c93 0 2024-04-22T23:35:04.862 INFO:tasks.workunit.client.0.smithi107.stdout:2/428: chown d1/d2b/d49 33 1 2024-04-22T23:35:04.863 INFO:tasks.workunit.client.0.smithi107.stdout:8/312: read d0/d7/df/d33/d37/f49 [756987,53573] 0 2024-04-22T23:35:04.863 INFO:tasks.workunit.client.0.smithi107.stdout:8/313: creat d0/d7/d8/d41/d53/f67 x:0 0 0 2024-04-22T23:35:04.863 INFO:tasks.workunit.client.0.smithi107.stdout:8/314: chown d0/f2f 360 1 2024-04-22T23:35:04.865 INFO:tasks.workunit.client.1.smithi157.stdout:9/618: mknod d4/d15/d55/cc2 0 2024-04-22T23:35:04.866 INFO:tasks.workunit.client.0.smithi107.stdout:6/377: truncate db/d20/d2a/d38/f53 811263 0 2024-04-22T23:35:04.867 INFO:tasks.workunit.client.1.smithi157.stdout:8/421: write d2/d6/d21/f54 [2332008,22192] 0 2024-04-22T23:35:04.869 INFO:tasks.workunit.client.0.smithi107.stdout:7/305: symlink d7/d21/l69 0 2024-04-22T23:35:04.869 INFO:tasks.workunit.client.0.smithi107.stdout:5/359: mkdir d9/d76 0 2024-04-22T23:35:04.870 INFO:tasks.workunit.client.1.smithi157.stdout:0/538: unlink d17/d23/d34/d67/d7f/f85 0 2024-04-22T23:35:04.871 INFO:tasks.workunit.client.1.smithi157.stdout:2/703: mkdir d7/d8/d95/dde/dec 0 2024-04-22T23:35:04.871 INFO:tasks.workunit.client.1.smithi157.stdout:0/539: dread - d17/d2b/d4f/d5a/f63 zero size 2024-04-22T23:35:04.871 INFO:tasks.workunit.client.1.smithi157.stdout:0/540: fsync d17/f19 0 2024-04-22T23:35:04.871 INFO:tasks.workunit.client.1.smithi157.stdout:2/704: readlink d7/d8/d8e/d80/ld8 0 2024-04-22T23:35:04.871 INFO:tasks.workunit.client.1.smithi157.stdout:5/585: symlink d1/d5/d7/d2f/d45/d54/ldb 0 2024-04-22T23:35:04.872 INFO:tasks.workunit.client.0.smithi107.stdout:3/416: mkdir db/d21/d27/d91 0 2024-04-22T23:35:04.872 INFO:tasks.workunit.client.0.smithi107.stdout:3/417: write db/d21/d27/d15/d24/d87/f89 [260609,121127] 0 2024-04-22T23:35:04.873 INFO:tasks.workunit.client.0.smithi107.stdout:9/450: creat d5/d7/d27/f9d x:0 0 0 2024-04-22T23:35:04.875 INFO:tasks.workunit.client.1.smithi157.stdout:1/641: rename d4/db/d3d/f91 to d4/d17/d34/d75/fdc 0 2024-04-22T23:35:04.875 INFO:tasks.workunit.client.1.smithi157.stdout:1/642: write d4/d5f/fce [247215,100677] 0 2024-04-22T23:35:04.876 INFO:tasks.workunit.client.0.smithi107.stdout:3/418: dread db/d21/d27/f23 [4194304,4194304] 0 2024-04-22T23:35:04.876 INFO:tasks.workunit.client.0.smithi107.stdout:3/419: chown db/d21/d27/d54/d25/c84 1600 1 2024-04-22T23:35:04.877 INFO:tasks.workunit.client.1.smithi157.stdout:9/619: creat d4/d15/d23/d27/db5/fc3 x:0 0 0 2024-04-22T23:35:04.879 INFO:tasks.workunit.client.0.smithi107.stdout:8/315: creat d0/d3a/d46/d59/f68 x:0 0 0 2024-04-22T23:35:04.879 INFO:tasks.workunit.client.0.smithi107.stdout:8/316: write d0/d7/df/d3b/f10 [671244,45026] 0 2024-04-22T23:35:04.879 INFO:tasks.workunit.client.1.smithi157.stdout:8/422: mknod d2/d6/d21/c93 0 2024-04-22T23:35:04.879 INFO:tasks.workunit.client.1.smithi157.stdout:8/423: truncate d2/d4/f5b 307567 0 2024-04-22T23:35:04.879 INFO:tasks.workunit.client.1.smithi157.stdout:8/424: creat d2/d6/d21/d5f/d11/d13/f94 x:0 0 0 2024-04-22T23:35:04.880 INFO:tasks.workunit.client.1.smithi157.stdout:0/541: creat d17/d23/d34/d67/d78/fbb x:0 0 0 2024-04-22T23:35:04.882 INFO:tasks.workunit.client.1.smithi157.stdout:2/705: symlink d7/d1f/led 0 2024-04-22T23:35:04.882 INFO:tasks.workunit.client.1.smithi157.stdout:2/706: stat d7/d8/d30/d5b 0 2024-04-22T23:35:04.883 INFO:tasks.workunit.client.1.smithi157.stdout:9/620: mkdir d4/df/d57/d6b/dc4 0 2024-04-22T23:35:04.885 INFO:tasks.workunit.client.1.smithi157.stdout:6/882: dwrite d1/d38/faa [0,4194304] 0 2024-04-22T23:35:04.887 INFO:tasks.workunit.client.0.smithi107.stdout:5/360: link d9/db/d42/l65 d9/db/l77 0 2024-04-22T23:35:04.889 INFO:tasks.workunit.client.1.smithi157.stdout:8/425: rename d2/d6/d21/d5f/d11/f44 to d2/d6/d66/d67/f95 0 2024-04-22T23:35:04.890 INFO:tasks.workunit.client.0.smithi107.stdout:9/451: mknod d5/d29/d7b/d91/c9e 0 2024-04-22T23:35:04.890 INFO:tasks.workunit.client.0.smithi107.stdout:9/452: dread - d5/d14/d2e/d4f/d73/f9b zero size 2024-04-22T23:35:04.890 INFO:tasks.workunit.client.0.smithi107.stdout:9/453: creat d5/d7/d27/d46/f9f x:0 0 0 2024-04-22T23:35:04.891 INFO:tasks.workunit.client.1.smithi157.stdout:1/643: dread d4/db/f36 [0,4194304] 0 2024-04-22T23:35:04.891 INFO:tasks.workunit.client.1.smithi157.stdout:1/644: read - d4/d17/d34/d75/fdc zero size 2024-04-22T23:35:04.891 INFO:tasks.workunit.client.1.smithi157.stdout:0/542: symlink d17/d2b/d52/lbc 0 2024-04-22T23:35:04.893 INFO:tasks.workunit.client.1.smithi157.stdout:2/707: link d7/d8/d30/d5b/fa5 d7/d8/d8e/fee 0 2024-04-22T23:35:04.893 INFO:tasks.workunit.client.1.smithi157.stdout:2/708: stat d7/d8/d30/f79 0 2024-04-22T23:35:04.893 INFO:tasks.workunit.client.1.smithi157.stdout:5/586: dwrite d1/d5/d7/d2f/d45/d54/d59/f70 [0,4194304] 0 2024-04-22T23:35:04.894 INFO:tasks.workunit.client.0.smithi107.stdout:5/361: symlink d9/db/l78 0 2024-04-22T23:35:04.897 INFO:tasks.workunit.client.1.smithi157.stdout:6/883: rename d1/d2f/d54/lb9 to d1/d2f/d54/dc4/l120 0 2024-04-22T23:35:04.897 INFO:tasks.workunit.client.1.smithi157.stdout:6/884: write d1/d2f/d3b/d39/f46 [1167340,16110] 0 2024-04-22T23:35:04.898 INFO:tasks.workunit.client.0.smithi107.stdout:7/306: dwrite d7/fd [0,4194304] 0 2024-04-22T23:35:04.898 INFO:tasks.workunit.client.0.smithi107.stdout:9/454: symlink d5/d14/d2e/d4f/la0 0 2024-04-22T23:35:04.898 INFO:tasks.workunit.client.0.smithi107.stdout:9/455: write d5/d14/f15 [3298661,110537] 0 2024-04-22T23:35:04.900 INFO:tasks.workunit.client.1.smithi157.stdout:0/543: creat d17/d2b/d71/db3/fbd x:0 0 0 2024-04-22T23:35:04.901 INFO:tasks.workunit.client.0.smithi107.stdout:7/307: creat d7/d21/d49/f6a x:0 0 0 2024-04-22T23:35:04.903 INFO:tasks.workunit.client.1.smithi157.stdout:5/587: creat d1/d5/d7/d2f/d45/da4/fdc x:0 0 0 2024-04-22T23:35:04.903 INFO:tasks.workunit.client.1.smithi157.stdout:5/588: fsync d1/d5/d7/d2f/d45/d54/f74 0 2024-04-22T23:35:04.905 INFO:tasks.workunit.client.1.smithi157.stdout:6/885: unlink d1/d2f/d3b/d39/d4e/da8/de5/d83/fa6 0 2024-04-22T23:35:04.906 INFO:tasks.workunit.client.0.smithi107.stdout:4/356: rmdir d1/d21/d29/d2e 39 2024-04-22T23:35:04.906 INFO:tasks.workunit.client.1.smithi157.stdout:0/544: mknod d17/d2b/d4f/d5a/da1/da8/cbe 0 2024-04-22T23:35:04.908 INFO:tasks.workunit.client.0.smithi107.stdout:9/456: mknod d5/d7/d17/d1b/d1f/d8e/ca1 0 2024-04-22T23:35:04.908 INFO:tasks.workunit.client.1.smithi157.stdout:0/545: creat d17/d23/fbf x:0 0 0 2024-04-22T23:35:04.911 INFO:tasks.workunit.client.1.smithi157.stdout:0/546: symlink d17/d2b/d52/lc0 0 2024-04-22T23:35:04.912 INFO:tasks.workunit.client.0.smithi107.stdout:4/357: dread d1/d7/f33 [0,4194304] 0 2024-04-22T23:35:04.912 INFO:tasks.workunit.client.0.smithi107.stdout:7/308: rename d7/d1b/f28 to d7/d21/d48/f6b 0 2024-04-22T23:35:04.912 INFO:tasks.workunit.client.0.smithi107.stdout:5/362: dread f7 [0,4194304] 0 2024-04-22T23:35:04.912 INFO:tasks.workunit.client.1.smithi157.stdout:6/886: dread d1/d2f/d3b/f2a [0,4194304] 0 2024-04-22T23:35:04.913 INFO:tasks.workunit.client.0.smithi107.stdout:4/358: truncate d1/f5 3541655 0 2024-04-22T23:35:04.917 INFO:tasks.workunit.client.0.smithi107.stdout:7/309: truncate d7/d2d/f43 296722 0 2024-04-22T23:35:04.917 INFO:tasks.workunit.client.0.smithi107.stdout:4/359: mkdir d1/d7/d63/d78/d80 0 2024-04-22T23:35:04.917 INFO:tasks.workunit.client.0.smithi107.stdout:4/360: write d1/d7/d53/d66/f79 [291134,45229] 0 2024-04-22T23:35:04.918 INFO:tasks.workunit.client.0.smithi107.stdout:4/361: dread - d1/d21/d29/d2e/d3b/f7d zero size 2024-04-22T23:35:04.919 INFO:tasks.workunit.client.1.smithi157.stdout:5/589: dread d1/d1f/faf [0,4194304] 0 2024-04-22T23:35:04.919 INFO:tasks.workunit.client.1.smithi157.stdout:5/590: creat d1/d27/d29/fdd x:0 0 0 2024-04-22T23:35:04.919 INFO:tasks.workunit.client.1.smithi157.stdout:5/591: write d1/d5/d7/d10/f31 [3780541,83229] 0 2024-04-22T23:35:04.920 INFO:tasks.workunit.client.0.smithi107.stdout:7/310: rename d7/d21/f32 to d7/d4c/f6c 0 2024-04-22T23:35:04.920 INFO:tasks.workunit.client.0.smithi107.stdout:7/311: chown d7/l52 8831163 1 2024-04-22T23:35:04.922 INFO:tasks.workunit.client.0.smithi107.stdout:4/362: creat d1/d1b/f81 x:0 0 0 2024-04-22T23:35:04.922 INFO:tasks.workunit.client.0.smithi107.stdout:4/363: readlink d1/d8/l62 0 2024-04-22T23:35:04.923 INFO:tasks.workunit.client.1.smithi157.stdout:5/592: rename d1/d5/d7/d2f/d51/c84 to d1/d5/d7/d2f/d51/cde 0 2024-04-22T23:35:04.923 INFO:tasks.workunit.client.1.smithi157.stdout:5/593: creat d1/d27/fdf x:0 0 0 2024-04-22T23:35:04.923 INFO:tasks.workunit.client.1.smithi157.stdout:5/594: truncate d1/d5/d7/d2f/d45/f48 899364 0 2024-04-22T23:35:04.924 INFO:tasks.workunit.client.0.smithi107.stdout:7/312: rename d7/f2a to d7/d21/f6d 0 2024-04-22T23:35:04.924 INFO:tasks.workunit.client.0.smithi107.stdout:7/313: read - d7/d2d/f41 zero size 2024-04-22T23:35:04.924 INFO:tasks.workunit.client.0.smithi107.stdout:7/314: write d7/f2f [442957,1003] 0 2024-04-22T23:35:04.924 INFO:tasks.workunit.client.0.smithi107.stdout:4/364: creat d1/d7/d63/d78/d80/f82 x:0 0 0 2024-04-22T23:35:04.925 INFO:tasks.workunit.client.0.smithi107.stdout:4/365: stat d1/d8 0 2024-04-22T23:35:04.926 INFO:tasks.workunit.client.1.smithi157.stdout:5/595: mkdir d1/d5/dc/de0 0 2024-04-22T23:35:04.927 INFO:tasks.workunit.client.0.smithi107.stdout:7/315: mkdir d7/d1b/d51/d6e 0 2024-04-22T23:35:04.927 INFO:tasks.workunit.client.0.smithi107.stdout:7/316: chown d7/d1b/f53 2 1 2024-04-22T23:35:04.928 INFO:tasks.workunit.client.0.smithi107.stdout:4/366: symlink d1/d21/d29/d2e/d7e/l83 0 2024-04-22T23:35:04.928 INFO:tasks.workunit.client.1.smithi157.stdout:5/596: stat d1/c41 0 2024-04-22T23:35:04.929 INFO:tasks.workunit.client.0.smithi107.stdout:8/317: dwrite d0/d7/df/d3b/f10 [0,4194304] 0 2024-04-22T23:35:04.929 INFO:tasks.workunit.client.1.smithi157.stdout:2/709: dwrite d7/d8/d30/d23/d27/d57/fa7 [0,4194304] 0 2024-04-22T23:35:04.929 INFO:tasks.workunit.client.0.smithi107.stdout:4/367: creat d1/d7/d53/f84 x:0 0 0 2024-04-22T23:35:04.930 INFO:tasks.workunit.client.0.smithi107.stdout:4/368: dread - d1/d7/d53/f84 zero size 2024-04-22T23:35:04.930 INFO:tasks.workunit.client.1.smithi157.stdout:8/426: dwrite d2/d34/f78 [0,4194304] 0 2024-04-22T23:35:04.930 INFO:tasks.workunit.client.0.smithi107.stdout:4/369: mkdir d1/d21/d85 0 2024-04-22T23:35:04.931 INFO:tasks.workunit.client.0.smithi107.stdout:5/363: write d9/db/de/d5e/f61 [1026922,63002] 0 2024-04-22T23:35:04.932 INFO:tasks.workunit.client.0.smithi107.stdout:3/420: dwrite f0 [4194304,4194304] 0 2024-04-22T23:35:04.932 INFO:tasks.workunit.client.0.smithi107.stdout:9/457: dwrite d5/d14/d2e/d4f/d73/f9b [0,4194304] 0 2024-04-22T23:35:04.932 INFO:tasks.workunit.client.0.smithi107.stdout:9/458: truncate d5/f47 811955 0 2024-04-22T23:35:04.932 INFO:tasks.workunit.client.1.smithi157.stdout:6/887: rename d1/d2f/d54 to d1/d121 0 2024-04-22T23:35:04.934 INFO:tasks.workunit.client.1.smithi157.stdout:2/710: truncate d7/d8/d95/dd2/dda/f89 586252 0 2024-04-22T23:35:04.938 INFO:tasks.workunit.client.1.smithi157.stdout:8/427: symlink d2/d34/d6a/d77/d42/d58/l96 0 2024-04-22T23:35:04.939 INFO:tasks.workunit.client.0.smithi107.stdout:5/364: truncate d9/d26/f2c 1251921 0 2024-04-22T23:35:04.939 INFO:tasks.workunit.client.0.smithi107.stdout:5/365: readlink d9/db/de/l62 0 2024-04-22T23:35:04.942 INFO:tasks.workunit.client.1.smithi157.stdout:6/888: mknod d1/d2f/d3b/d39/d4e/da8/dac/c122 0 2024-04-22T23:35:04.943 INFO:tasks.workunit.client.0.smithi107.stdout:4/370: dread d1/d7/f11 [0,4194304] 0 2024-04-22T23:35:04.943 INFO:tasks.workunit.client.0.smithi107.stdout:9/459: creat d5/d14/fa2 x:0 0 0 2024-04-22T23:35:04.944 INFO:tasks.workunit.client.0.smithi107.stdout:8/318: write d0/d7/df/d3b/f3d [206972,93594] 0 2024-04-22T23:35:04.944 INFO:tasks.workunit.client.0.smithi107.stdout:8/319: fdatasync d0/f19 0 2024-04-22T23:35:04.945 INFO:tasks.workunit.client.0.smithi107.stdout:5/366: mkdir d9/d26/d79 0 2024-04-22T23:35:04.946 INFO:tasks.workunit.client.0.smithi107.stdout:3/421: rmdir db/d21/d27/d15/d29/d2a/d40 39 2024-04-22T23:35:04.948 INFO:tasks.workunit.client.1.smithi157.stdout:6/889: creat d1/d2f/dc7/f123 x:0 0 0 2024-04-22T23:35:04.949 INFO:tasks.workunit.client.0.smithi107.stdout:7/317: dwrite d7/d2d/f3c [0,4194304] 0 2024-04-22T23:35:04.949 INFO:tasks.workunit.client.0.smithi107.stdout:4/371: mknod d1/d7/d63/d78/d80/c86 0 2024-04-22T23:35:04.949 INFO:tasks.workunit.client.0.smithi107.stdout:4/372: truncate d1/d7/d63/d78/d80/f82 667678 0 2024-04-22T23:35:04.949 INFO:tasks.workunit.client.0.smithi107.stdout:8/320: dread d0/d7/df/f4b [0,4194304] 0 2024-04-22T23:35:04.951 INFO:tasks.workunit.client.1.smithi157.stdout:6/890: truncate d1/d2f/d3b/d39/fe1 3704070 0 2024-04-22T23:35:04.951 INFO:tasks.workunit.client.1.smithi157.stdout:5/597: write d1/d5/d7/d2f/d45/d54/f82 [3605248,54511] 0 2024-04-22T23:35:04.953 INFO:tasks.workunit.client.0.smithi107.stdout:5/367: symlink d9/db/de/d5e/l7a 0 2024-04-22T23:35:04.953 INFO:tasks.workunit.client.0.smithi107.stdout:5/368: creat d9/db/f7b x:0 0 0 2024-04-22T23:35:04.954 INFO:tasks.workunit.client.1.smithi157.stdout:5/598: symlink d1/d27/d29/le1 0 2024-04-22T23:35:04.955 INFO:tasks.workunit.client.0.smithi107.stdout:3/422: unlink db/d21/d27/d15/d24/d52/d60/l6a 0 2024-04-22T23:35:04.956 INFO:tasks.workunit.client.0.smithi107.stdout:3/423: readlink db/d21/d27/d15/d24/d52/l83 0 2024-04-22T23:35:04.957 INFO:tasks.workunit.client.1.smithi157.stdout:5/599: link d1/db0/dc1/lce d1/db0/dd2/le2 0 2024-04-22T23:35:04.959 INFO:tasks.workunit.client.0.smithi107.stdout:4/373: unlink d1/d7/d63/d78/d80/c86 0 2024-04-22T23:35:04.960 INFO:tasks.workunit.client.0.smithi107.stdout:4/374: creat d1/f87 x:0 0 0 2024-04-22T23:35:04.961 INFO:tasks.workunit.client.0.smithi107.stdout:7/318: dread d7/d21/d48/d5c/f64 [0,4194304] 0 2024-04-22T23:35:04.961 INFO:tasks.workunit.client.0.smithi107.stdout:7/319: truncate d7/d1b/d51/f5a 250114 0 2024-04-22T23:35:04.962 INFO:tasks.workunit.client.0.smithi107.stdout:8/321: mkdir d0/d7/df/d33/d37/d69 0 2024-04-22T23:35:04.965 INFO:tasks.workunit.client.0.smithi107.stdout:5/369: truncate f7 2182881 0 2024-04-22T23:35:04.969 INFO:tasks.workunit.client.1.smithi157.stdout:1/645: dwrite d4/d17/d34/d3b/d52/d65/dbc/faa [0,4194304] 0 2024-04-22T23:35:04.969 INFO:tasks.workunit.client.1.smithi157.stdout:2/711: dread d7/d8/d95/dd2/dda/f89 [0,4194304] 0 2024-04-22T23:35:04.969 INFO:tasks.workunit.client.1.smithi157.stdout:2/712: write d7/d8/d8e/d94/fb7 [479730,23319] 0 2024-04-22T23:35:04.970 INFO:tasks.workunit.client.0.smithi107.stdout:4/375: mkdir d1/d21/d29/d2e/d88 0 2024-04-22T23:35:04.972 INFO:tasks.workunit.client.0.smithi107.stdout:7/320: mknod d7/d21/d48/c6f 0 2024-04-22T23:35:04.974 INFO:tasks.workunit.client.1.smithi157.stdout:8/428: dwrite d2/d4/f5b [0,4194304] 0 2024-04-22T23:35:04.975 INFO:tasks.workunit.client.0.smithi107.stdout:8/322: creat d0/d7/df/d3b/f6a x:0 0 0 2024-04-22T23:35:04.975 INFO:tasks.workunit.client.0.smithi107.stdout:8/323: write d0/d7/df/d18/f50 [315483,30367] 0 2024-04-22T23:35:04.975 INFO:tasks.workunit.client.1.smithi157.stdout:1/646: getdents d4/db/d3d/dc1 0 2024-04-22T23:35:04.977 INFO:tasks.workunit.client.1.smithi157.stdout:2/713: dread d7/d1f/f52 [0,4194304] 0 2024-04-22T23:35:04.977 INFO:tasks.workunit.client.1.smithi157.stdout:2/714: stat d7/d8/dd/f21 0 2024-04-22T23:35:04.977 INFO:tasks.workunit.client.1.smithi157.stdout:2/715: fsync d7/d8/d95/dd2/dda/f8b 0 2024-04-22T23:35:04.977 INFO:tasks.workunit.client.1.smithi157.stdout:5/600: dwrite d1/d27/d29/d4b/f4d [0,4194304] 0 2024-04-22T23:35:04.977 INFO:tasks.workunit.client.1.smithi157.stdout:8/429: mkdir d2/d6/d66/d97 0 2024-04-22T23:35:04.980 INFO:tasks.workunit.client.0.smithi107.stdout:4/376: mkdir d1/d1b/d89 0 2024-04-22T23:35:04.981 INFO:tasks.workunit.client.1.smithi157.stdout:6/891: dwrite d1/f12 [0,4194304] 0 2024-04-22T23:35:04.981 INFO:tasks.workunit.client.1.smithi157.stdout:1/647: creat d4/d5f/d7e/db8/fdd x:0 0 0 2024-04-22T23:35:04.981 INFO:tasks.workunit.client.1.smithi157.stdout:1/648: write d4/d17/d1d/d33/d41/d54/db3/fbf [508079,75459] 0 2024-04-22T23:35:04.981 INFO:tasks.workunit.client.1.smithi157.stdout:2/716: rename d7/d8/d95/f9a to d7/fef 0 2024-04-22T23:35:04.982 INFO:tasks.workunit.client.1.smithi157.stdout:2/717: stat d7/ca1 0 2024-04-22T23:35:04.982 INFO:tasks.workunit.client.1.smithi157.stdout:2/718: fsync d7/d8/d30/d23/d27/f49 0 2024-04-22T23:35:04.984 INFO:tasks.workunit.client.0.smithi107.stdout:7/321: creat d7/d1b/d51/d6e/f70 x:0 0 0 2024-04-22T23:35:04.984 INFO:tasks.workunit.client.1.smithi157.stdout:8/430: dread d2/d6/d21/d5f/d11/f27 [0,4194304] 0 2024-04-22T23:35:04.986 INFO:tasks.workunit.client.0.smithi107.stdout:3/424: dwrite db/d21/d27/f20 [0,4194304] 0 2024-04-22T23:35:04.987 INFO:tasks.workunit.client.1.smithi157.stdout:1/649: symlink d4/d17/d34/d3b/d50/dad/lde 0 2024-04-22T23:35:04.988 INFO:tasks.workunit.client.1.smithi157.stdout:2/719: mknod d7/cf0 0 2024-04-22T23:35:04.990 INFO:tasks.workunit.client.1.smithi157.stdout:1/650: creat d4/d77/fdf x:0 0 0 2024-04-22T23:35:04.990 INFO:tasks.workunit.client.1.smithi157.stdout:1/651: readlink d4/d17/la3 0 2024-04-22T23:35:04.991 INFO:tasks.workunit.client.1.smithi157.stdout:2/720: symlink d7/d8/d30/d23/d27/d57/dcf/lf1 0 2024-04-22T23:35:04.991 INFO:tasks.workunit.client.1.smithi157.stdout:2/721: write d7/d8/d95/dd2/dda/f89 [1277379,130569] 0 2024-04-22T23:35:04.992 INFO:tasks.workunit.client.1.smithi157.stdout:2/722: mkdir d7/d8/d30/d23/d27/d57/dcf/df2 0 2024-04-22T23:35:04.992 INFO:tasks.workunit.client.1.smithi157.stdout:1/652: dread d4/d17/d1d/fd2 [0,4194304] 0 2024-04-22T23:35:04.993 INFO:tasks.workunit.client.1.smithi157.stdout:2/723: mknod d7/d1f/d3e/db4/ddb/cf3 0 2024-04-22T23:35:04.994 INFO:tasks.workunit.client.1.smithi157.stdout:1/653: symlink d4/d17/d34/d3b/d52/d65/dbc/le0 0 2024-04-22T23:35:04.995 INFO:tasks.workunit.client.1.smithi157.stdout:2/724: truncate d7/d1f/d26/d4a/f53 822363 0 2024-04-22T23:35:04.995 INFO:tasks.workunit.client.0.smithi107.stdout:8/324: rmdir d0/d7 39 2024-04-22T23:35:04.996 INFO:tasks.workunit.client.1.smithi157.stdout:1/654: creat d4/d17/d1d/db2/fe1 x:0 0 0 2024-04-22T23:35:04.996 INFO:tasks.workunit.client.1.smithi157.stdout:1/655: write d4/d17/d34/d3b/f66 [1442010,15188] 0 2024-04-22T23:35:04.996 INFO:tasks.workunit.client.1.smithi157.stdout:8/431: dread d2/d34/d6a/d77/d42/d58/f92 [0,4194304] 0 2024-04-22T23:35:04.997 INFO:tasks.workunit.client.0.smithi107.stdout:4/377: getdents d1/d7/d63/d78 0 2024-04-22T23:35:04.997 INFO:tasks.workunit.client.0.smithi107.stdout:4/378: getdents d1/d1b/d89 0 2024-04-22T23:35:04.997 INFO:tasks.workunit.client.1.smithi157.stdout:2/725: creat d7/d1f/ff4 x:0 0 0 2024-04-22T23:35:04.999 INFO:tasks.workunit.client.1.smithi157.stdout:1/656: creat d4/d17/d34/dc8/fe2 x:0 0 0 2024-04-22T23:35:04.999 INFO:tasks.workunit.client.1.smithi157.stdout:1/657: write d4/db/f24 [5144495,36525] 0 2024-04-22T23:35:05.000 INFO:tasks.workunit.client.1.smithi157.stdout:6/892: dwrite d1/d2f/d3b/d39/d4e/da8/dac/db0/fdb [4194304,4194304] 0 2024-04-22T23:35:05.001 INFO:tasks.workunit.client.0.smithi107.stdout:3/425: creat db/d21/d27/d15/d24/d52/d60/f92 x:0 0 0 2024-04-22T23:35:05.001 INFO:tasks.workunit.client.0.smithi107.stdout:3/426: stat db/d21/d7f 0 2024-04-22T23:35:05.001 INFO:tasks.workunit.client.0.smithi107.stdout:3/427: creat db/d21/d27/d15/d24/d52/d60/f93 x:0 0 0 2024-04-22T23:35:05.001 INFO:tasks.workunit.client.0.smithi107.stdout:3/428: creat db/d21/d27/d54/d25/d6e/f94 x:0 0 0 2024-04-22T23:35:05.002 INFO:tasks.workunit.client.1.smithi157.stdout:8/432: rename d2/d34/d6a/f81 to d2/d6/d21/f98 0 2024-04-22T23:35:05.002 INFO:tasks.workunit.client.1.smithi157.stdout:8/433: chown d2/d34/d6a/f6d 198 1 2024-04-22T23:35:05.003 INFO:tasks.workunit.client.0.smithi107.stdout:4/379: creat d1/d21/d29/d2e/d3b/f8a x:0 0 0 2024-04-22T23:35:05.005 INFO:tasks.workunit.client.1.smithi157.stdout:1/658: rmdir d4/db/d3d/dc2 0 2024-04-22T23:35:05.007 INFO:tasks.workunit.client.1.smithi157.stdout:1/659: dread d4/d17/d34/f49 [0,4194304] 0 2024-04-22T23:35:05.008 INFO:tasks.workunit.client.1.smithi157.stdout:1/660: read d4/d17/d34/d75/f89 [644377,88077] 0 2024-04-22T23:35:05.010 INFO:tasks.workunit.client.0.smithi107.stdout:3/429: link db/l3f db/d21/d27/d15/d24/d52/l95 0 2024-04-22T23:35:05.010 INFO:tasks.workunit.client.1.smithi157.stdout:6/893: creat d1/d2f/d3b/d49/f124 x:0 0 0 2024-04-22T23:35:05.011 INFO:tasks.workunit.client.1.smithi157.stdout:8/434: read d2/d6/d21/d5f/d11/d13/f7d [2175860,18055] 0 2024-04-22T23:35:05.012 INFO:tasks.workunit.client.0.smithi107.stdout:3/430: mkdir db/d21/d27/d15/d24/d52/d60/d96 0 2024-04-22T23:35:05.012 INFO:tasks.workunit.client.1.smithi157.stdout:1/661: rename d4/l32 to d4/d17/db9/le3 0 2024-04-22T23:35:05.014 INFO:tasks.workunit.client.0.smithi107.stdout:3/431: symlink db/d21/d27/d15/d29/l97 0 2024-04-22T23:35:05.014 INFO:tasks.workunit.client.1.smithi157.stdout:6/894: mknod d1/d2f/d3b/d39/d4e/df7/c125 0 2024-04-22T23:35:05.015 INFO:tasks.workunit.client.1.smithi157.stdout:6/895: truncate d1/d36/fa3 1432347 0 2024-04-22T23:35:05.015 INFO:tasks.workunit.client.1.smithi157.stdout:6/896: stat d1/d2f/d3b/d39/d4e/da8/de5/f6d 0 2024-04-22T23:35:05.015 INFO:tasks.workunit.client.1.smithi157.stdout:6/897: stat d1/d17/f31 0 2024-04-22T23:35:05.015 INFO:tasks.workunit.client.1.smithi157.stdout:8/435: creat d2/d4/d73/f99 x:0 0 0 2024-04-22T23:35:05.016 INFO:tasks.workunit.client.0.smithi107.stdout:3/432: symlink db/d21/d27/d91/l98 0 2024-04-22T23:35:05.016 INFO:tasks.workunit.client.0.smithi107.stdout:3/433: chown db/d21/d27/d15/l68 43 1 2024-04-22T23:35:05.017 INFO:tasks.workunit.client.1.smithi157.stdout:1/662: symlink d4/d77/db4/le4 0 2024-04-22T23:35:05.018 INFO:tasks.workunit.client.0.smithi107.stdout:3/434: dread db/d21/d27/d15/d29/d2a/f58 [0,4194304] 0 2024-04-22T23:35:05.019 INFO:tasks.workunit.client.0.smithi107.stdout:3/435: truncate db/d21/d27/f39 2739014 0 2024-04-22T23:35:05.020 INFO:tasks.workunit.client.0.smithi107.stdout:7/322: rmdir d7/d21/d48 39 2024-04-22T23:35:05.022 INFO:tasks.workunit.client.0.smithi107.stdout:7/323: truncate d7/f26 1331506 0 2024-04-22T23:35:05.022 INFO:tasks.workunit.client.1.smithi157.stdout:1/663: mkdir d4/d17/de5 0 2024-04-22T23:35:05.022 INFO:tasks.workunit.client.1.smithi157.stdout:6/898: rmdir d1/d38/daf 39 2024-04-22T23:35:05.022 INFO:tasks.workunit.client.1.smithi157.stdout:6/899: write d1/d2f/f116 [1017724,106576] 0 2024-04-22T23:35:05.023 INFO:tasks.workunit.client.1.smithi157.stdout:6/900: truncate d1/d2f/d3b/d39/d4e/da8/dac/f110 3856271 0 2024-04-22T23:35:05.033 INFO:tasks.workunit.client.0.smithi107.stdout:4/380: dwrite d1/d7/d63/d78/d80/f82 [0,4194304] 0 2024-04-22T23:35:05.033 INFO:tasks.workunit.client.0.smithi107.stdout:4/381: stat d1/d1b/f64 0 2024-04-22T23:35:05.033 INFO:tasks.workunit.client.1.smithi157.stdout:2/726: dwrite d7/d8/d95/dd2/dda/f89 [0,4194304] 0 2024-04-22T23:35:05.035 INFO:tasks.workunit.client.1.smithi157.stdout:2/727: dread d7/d8/d8e/f78 [0,4194304] 0 2024-04-22T23:35:05.035 INFO:tasks.workunit.client.1.smithi157.stdout:2/728: fdatasync d7/d8/d25/f3a 0 2024-04-22T23:35:05.036 INFO:tasks.workunit.client.0.smithi107.stdout:4/382: dread d1/f75 [0,4194304] 0 2024-04-22T23:35:05.036 INFO:tasks.workunit.client.0.smithi107.stdout:4/383: dread - d1/d8/d56/d27/f30 zero size 2024-04-22T23:35:05.037 INFO:tasks.workunit.client.0.smithi107.stdout:8/325: dwrite d0/d7/df/d3b/f2b [0,4194304] 0 2024-04-22T23:35:05.037 INFO:tasks.workunit.client.0.smithi107.stdout:8/326: read - d0/d7/df/d17/f48 zero size 2024-04-22T23:35:05.037 INFO:tasks.workunit.client.0.smithi107.stdout:8/327: creat d0/d3a/d46/d59/f6b x:0 0 0 2024-04-22T23:35:05.037 INFO:tasks.workunit.client.0.smithi107.stdout:8/328: fdatasync d0/d7/df/f4b 0 2024-04-22T23:35:05.038 INFO:tasks.workunit.client.0.smithi107.stdout:3/436: fsync db/d21/d27/d15/d24/d52/d60/f92 0 2024-04-22T23:35:05.039 INFO:tasks.workunit.client.0.smithi107.stdout:5/370: dread f7 [0,4194304] 0 2024-04-22T23:35:05.041 INFO:tasks.workunit.client.0.smithi107.stdout:3/437: mkdir db/d21/d27/d15/d29/d2a/d99 0 2024-04-22T23:35:05.042 INFO:tasks.workunit.client.0.smithi107.stdout:4/384: rmdir d1/d21/d29/d2e/d3b 39 2024-04-22T23:35:05.044 INFO:tasks.workunit.client.1.smithi157.stdout:1/664: dwrite d4/d17/d34/d75/f89 [0,4194304] 0 2024-04-22T23:35:05.045 INFO:tasks.workunit.client.1.smithi157.stdout:2/729: mknod d7/d8/d95/dde/cf5 0 2024-04-22T23:35:05.045 INFO:tasks.workunit.client.0.smithi107.stdout:5/371: mkdir d9/db/de/d72/d7c 0 2024-04-22T23:35:05.046 INFO:tasks.workunit.client.0.smithi107.stdout:5/372: readlink d9/l21 0 2024-04-22T23:35:05.047 INFO:tasks.workunit.client.0.smithi107.stdout:3/438: symlink db/d21/d27/d15/d24/d52/d60/d96/l9a 0 2024-04-22T23:35:05.047 INFO:tasks.workunit.client.0.smithi107.stdout:3/439: write db/d21/d27/d36/f7e [759931,127413] 0 2024-04-22T23:35:05.048 INFO:tasks.workunit.client.0.smithi107.stdout:3/440: write db/d21/d27/d54/d25/f34 [4562676,35750] 0 2024-04-22T23:35:05.048 INFO:tasks.workunit.client.0.smithi107.stdout:3/441: fsync db/d21/d27/d54/d25/d6e/f82 0 2024-04-22T23:35:05.049 INFO:tasks.workunit.client.1.smithi157.stdout:2/730: creat d7/d8/d8e/ff6 x:0 0 0 2024-04-22T23:35:05.050 INFO:tasks.workunit.client.1.smithi157.stdout:6/901: rename d1/d2f/d3b/d39/d4e/df7 to d1/d121/dc4/d126 0 2024-04-22T23:35:05.050 INFO:tasks.workunit.client.1.smithi157.stdout:6/902: chown d1/d121/d9b/dda/led 184307882 1 2024-04-22T23:35:05.051 INFO:tasks.workunit.client.1.smithi157.stdout:6/903: fdatasync d1/d2f/d3b/d49/fb3 0 2024-04-22T23:35:05.051 INFO:tasks.workunit.client.0.smithi107.stdout:5/373: rename d9/db/de/f1f to d9/d76/f7d 0 2024-04-22T23:35:05.053 INFO:tasks.workunit.client.1.smithi157.stdout:6/904: getdents d1/d38 0 2024-04-22T23:35:05.054 INFO:tasks.workunit.client.0.smithi107.stdout:3/442: getdents db/d21/d7f 0 2024-04-22T23:35:05.055 INFO:tasks.workunit.client.0.smithi107.stdout:3/443: fsync db/f1c 0 2024-04-22T23:35:05.055 INFO:tasks.workunit.client.0.smithi107.stdout:3/444: read db/d21/d27/d54/d25/f34 [3044845,121739] 0 2024-04-22T23:35:05.055 INFO:tasks.workunit.client.0.smithi107.stdout:3/445: chown db/d21/d27/d15/d29/d2a/d45 417 1 2024-04-22T23:35:05.057 INFO:tasks.workunit.client.1.smithi157.stdout:6/905: mknod d1/d2f/d3b/d39/d4e/da8/de5/d83/d93/c127 0 2024-04-22T23:35:05.058 INFO:tasks.workunit.client.0.smithi107.stdout:4/385: write d1/d21/d52/f70 [777641,74943] 0 2024-04-22T23:35:05.059 INFO:tasks.workunit.client.0.smithi107.stdout:3/446: rename db/d21/d27/d54/d25/c59 to db/d21/d27/d15/d24/d87/c9b 0 2024-04-22T23:35:05.060 INFO:tasks.workunit.client.0.smithi107.stdout:8/329: dwrite d0/d7/d20/f39 [0,4194304] 0 2024-04-22T23:35:05.061 INFO:tasks.workunit.client.1.smithi157.stdout:6/906: symlink d1/d36/l128 0 2024-04-22T23:35:05.062 INFO:tasks.workunit.client.0.smithi107.stdout:3/447: dread db/d21/d27/d15/d29/d2a/d40/f6c [0,4194304] 0 2024-04-22T23:35:05.063 INFO:tasks.workunit.client.1.smithi157.stdout:6/907: dread d1/d121/d9b/fb7 [0,4194304] 0 2024-04-22T23:35:05.063 INFO:tasks.workunit.client.1.smithi157.stdout:6/908: chown d1/d2f/d3b/d39/d4e/da8/dac/ddd/ddf 281514 1 2024-04-22T23:35:05.063 INFO:tasks.workunit.client.1.smithi157.stdout:6/909: chown d1/d2f/d3b/d49/l81 393517 1 2024-04-22T23:35:05.063 INFO:tasks.workunit.client.1.smithi157.stdout:6/910: creat d1/d36/f129 x:0 0 0 2024-04-22T23:35:05.067 INFO:tasks.workunit.client.0.smithi107.stdout:8/330: link d0/d7/df/d3b/f3d d0/d7/d8/f6c 0 2024-04-22T23:35:05.068 INFO:tasks.workunit.client.0.smithi107.stdout:3/448: stat db/d21/d27/d15/d24/d52/l95 0 2024-04-22T23:35:05.068 INFO:tasks.workunit.client.0.smithi107.stdout:3/449: write f0 [437035,33392] 0 2024-04-22T23:35:05.069 INFO:tasks.workunit.client.0.smithi107.stdout:8/331: mknod d0/c6d 0 2024-04-22T23:35:05.070 INFO:tasks.workunit.client.0.smithi107.stdout:3/450: mkdir db/d21/d27/d36/d9c 0 2024-04-22T23:35:05.071 INFO:tasks.workunit.client.0.smithi107.stdout:8/332: getdents d0/d7/d8/d41 0 2024-04-22T23:35:05.073 INFO:tasks.workunit.client.0.smithi107.stdout:3/451: rename db/l3f to db/d21/d27/d15/d24/d87/l9d 0 2024-04-22T23:35:05.073 INFO:tasks.workunit.client.0.smithi107.stdout:4/386: dread d1/f13 [0,4194304] 0 2024-04-22T23:35:05.075 INFO:tasks.workunit.client.0.smithi107.stdout:8/333: rmdir d0/d7/df/d33/d37/d69 0 2024-04-22T23:35:05.076 INFO:tasks.workunit.client.0.smithi107.stdout:8/334: creat d0/d3a/d46/d59/f6e x:0 0 0 2024-04-22T23:35:05.076 INFO:tasks.workunit.client.0.smithi107.stdout:4/387: creat d1/d21/d29/d2e/d88/f8b x:0 0 0 2024-04-22T23:35:05.076 INFO:tasks.workunit.client.0.smithi107.stdout:4/388: write d1/d8/f7f [621784,46136] 0 2024-04-22T23:35:05.079 INFO:tasks.workunit.client.0.smithi107.stdout:8/335: mknod d0/d7/c6f 0 2024-04-22T23:35:05.079 INFO:tasks.workunit.client.0.smithi107.stdout:8/336: truncate d0/d7/d8/d41/d53/f67 58360 0 2024-04-22T23:35:05.081 INFO:tasks.workunit.client.0.smithi107.stdout:4/389: link d1/d21/d29/c48 d1/d8/d67/c8c 0 2024-04-22T23:35:05.081 INFO:tasks.workunit.client.0.smithi107.stdout:4/390: write d1/d21/f76 [653039,71882] 0 2024-04-22T23:35:05.081 INFO:tasks.workunit.client.0.smithi107.stdout:4/391: creat d1/f8d x:0 0 0 2024-04-22T23:35:05.082 INFO:tasks.workunit.client.0.smithi107.stdout:3/452: symlink db/d21/d27/d15/l9e 0 2024-04-22T23:35:05.093 INFO:tasks.workunit.client.0.smithi107.stdout:8/337: write d0/d7/f34 [3161009,14155] 0 2024-04-22T23:35:05.094 INFO:tasks.workunit.client.0.smithi107.stdout:5/374: dwrite d9/db/f3c [0,4194304] 0 2024-04-22T23:35:05.095 INFO:tasks.workunit.client.0.smithi107.stdout:8/338: creat d0/d3a/d46/f70 x:0 0 0 2024-04-22T23:35:05.096 INFO:tasks.workunit.client.0.smithi107.stdout:4/392: fsync d1/d21/f76 0 2024-04-22T23:35:05.096 INFO:tasks.workunit.client.1.smithi157.stdout:8/436: dwrite d2/d6/d21/d5f/d17/f1e [0,4194304] 0 2024-04-22T23:35:05.096 INFO:tasks.workunit.client.1.smithi157.stdout:8/437: truncate d2/d4/f80 931686 0 2024-04-22T23:35:05.096 INFO:tasks.workunit.client.1.smithi157.stdout:6/911: dwrite d1/d2f/d3b/f2a [0,4194304] 0 2024-04-22T23:35:05.096 INFO:tasks.workunit.client.1.smithi157.stdout:6/912: stat d1/d2f/d3b/d39/d4e/da8/de5/d83/f71 0 2024-04-22T23:35:05.099 INFO:tasks.workunit.client.0.smithi107.stdout:5/375: link d9/d26/c44 d9/db/d42/d75/c7e 0 2024-04-22T23:35:05.099 INFO:tasks.workunit.client.1.smithi157.stdout:8/438: mknod d2/d6/d21/d5f/d11/c9a 0 2024-04-22T23:35:05.100 INFO:tasks.workunit.client.1.smithi157.stdout:6/913: mkdir d1/d121/dc4/d126/d12a 0 2024-04-22T23:35:05.101 INFO:tasks.workunit.client.1.smithi157.stdout:8/439: mknod d2/d6/d66/d97/c9b 0 2024-04-22T23:35:05.102 INFO:tasks.workunit.client.0.smithi107.stdout:8/339: symlink d0/d7/df/d17/l71 0 2024-04-22T23:35:05.102 INFO:tasks.workunit.client.0.smithi107.stdout:8/340: chown d0/f2f 72373011 1 2024-04-22T23:35:05.102 INFO:tasks.workunit.client.1.smithi157.stdout:6/914: mknod d1/d36/c12b 0 2024-04-22T23:35:05.102 INFO:tasks.workunit.client.1.smithi157.stdout:6/915: dread - d1/d2f/d3b/d39/fdc zero size 2024-04-22T23:35:05.103 INFO:tasks.workunit.client.0.smithi107.stdout:4/393: mknod d1/d1b/d89/c8e 0 2024-04-22T23:35:05.103 INFO:tasks.workunit.client.1.smithi157.stdout:6/916: write d1/d2f/d3b/d39/d4e/f5f [421148,21357] 0 2024-04-22T23:35:05.103 INFO:tasks.workunit.client.1.smithi157.stdout:6/917: fdatasync d1/d2f/d3b/d39/d4e/da8/de5/d83/f9f 0 2024-04-22T23:35:05.104 INFO:tasks.workunit.client.0.smithi107.stdout:3/453: dwrite db/d21/d27/d15/d29/d2a/f58 [0,4194304] 0 2024-04-22T23:35:05.105 INFO:tasks.workunit.client.1.smithi157.stdout:6/918: rename d1/d36/dea/f10a to d1/d2f/dc7/dce/f12c 0 2024-04-22T23:35:05.111 INFO:tasks.workunit.client.0.smithi107.stdout:4/394: creat d1/d8/d67/f8f x:0 0 0 2024-04-22T23:35:05.114 INFO:tasks.workunit.client.0.smithi107.stdout:3/454: rmdir db/d21/d27/d54/d25/d53/d81 0 2024-04-22T23:35:05.115 INFO:tasks.workunit.client.0.smithi107.stdout:5/376: rmdir d9/db/de/d5e 39 2024-04-22T23:35:05.115 INFO:tasks.workunit.client.0.smithi107.stdout:8/341: rmdir d0/d7/d8/d41/d53 39 2024-04-22T23:35:05.116 INFO:tasks.workunit.client.0.smithi107.stdout:8/342: write d0/d7/f34 [37695,118964] 0 2024-04-22T23:35:05.116 INFO:tasks.workunit.client.1.smithi157.stdout:8/440: write d2/d6/d21/d5f/fd [4619298,11697] 0 2024-04-22T23:35:05.117 INFO:tasks.workunit.client.0.smithi107.stdout:4/395: creat d1/d21/d29/d2e/d3b/f90 x:0 0 0 2024-04-22T23:35:05.118 INFO:tasks.workunit.client.0.smithi107.stdout:3/455: mknod db/d21/d27/d15/d24/d52/d60/d96/c9f 0 2024-04-22T23:35:05.119 INFO:tasks.workunit.client.1.smithi157.stdout:8/441: rename d2/d34/f41 to d2/d6/d21/d8d/f9c 0 2024-04-22T23:35:05.119 INFO:tasks.workunit.client.0.smithi107.stdout:5/377: symlink d9/db/d42/l7f 0 2024-04-22T23:35:05.120 INFO:tasks.workunit.client.0.smithi107.stdout:5/378: truncate d9/db/de/d5e/f61 4438639 0 2024-04-22T23:35:05.121 INFO:tasks.workunit.client.0.smithi107.stdout:3/456: creat db/d21/d7f/d8f/fa0 x:0 0 0 2024-04-22T23:35:05.135 INFO:tasks.workunit.client.0.smithi107.stdout:0/382: sync 2024-04-22T23:35:05.136 INFO:tasks.workunit.client.0.smithi107.stdout:0/383: truncate d1/d27/f38 1877634 0 2024-04-22T23:35:05.136 INFO:tasks.workunit.client.0.smithi107.stdout:1/388: sync 2024-04-22T23:35:05.139 INFO:tasks.workunit.client.0.smithi107.stdout:1/389: unlink d7/f73 0 2024-04-22T23:35:05.139 INFO:tasks.workunit.client.0.smithi107.stdout:1/390: fsync d7/d19/d71/f78 0 2024-04-22T23:35:05.139 INFO:tasks.workunit.client.0.smithi107.stdout:1/391: write d7/d19/d71/f78 [181217,116618] 0 2024-04-22T23:35:05.140 INFO:tasks.workunit.client.1.smithi157.stdout:6/919: dwrite d1/d38/daf/f11e [0,4194304] 0 2024-04-22T23:35:05.142 INFO:tasks.workunit.client.0.smithi107.stdout:4/396: dwrite d1/d21/f6a [0,4194304] 0 2024-04-22T23:35:05.144 INFO:tasks.workunit.client.0.smithi107.stdout:8/343: dwrite d0/d7/d20/f39 [0,4194304] 0 2024-04-22T23:35:05.144 INFO:tasks.workunit.client.0.smithi107.stdout:8/344: chown d0/d7/df/d3b/c36 11857415 1 2024-04-22T23:35:05.144 INFO:tasks.workunit.client.0.smithi107.stdout:8/345: truncate d0/d3a/d46/d59/f6e 603377 0 2024-04-22T23:35:05.146 INFO:tasks.workunit.client.0.smithi107.stdout:4/397: getdents d1/d7/d53/d66 0 2024-04-22T23:35:05.150 INFO:tasks.workunit.client.0.smithi107.stdout:5/379: dwrite d9/d26/f55 [0,4194304] 0 2024-04-22T23:35:05.154 INFO:tasks.workunit.client.0.smithi107.stdout:5/380: mknod d9/db/de/d72/c80 0 2024-04-22T23:35:05.156 INFO:tasks.workunit.client.0.smithi107.stdout:8/346: dread d0/d7/df/f5c [0,4194304] 0 2024-04-22T23:35:05.156 INFO:tasks.workunit.client.0.smithi107.stdout:3/457: dwrite db/d21/d27/d36/f7e [0,4194304] 0 2024-04-22T23:35:05.157 INFO:tasks.workunit.client.0.smithi107.stdout:8/347: chown d0/d7/d8/l26 1634 1 2024-04-22T23:35:05.157 INFO:tasks.workunit.client.0.smithi107.stdout:8/348: write d0/d7/d8/d41/d53/f67 [788151,95653] 0 2024-04-22T23:35:05.157 INFO:tasks.workunit.client.0.smithi107.stdout:8/349: write d0/d7/df/d17/f48 [98294,74114] 0 2024-04-22T23:35:05.157 INFO:tasks.workunit.client.0.smithi107.stdout:8/350: fsync d0/d3a/d46/d59/f6b 0 2024-04-22T23:35:05.157 INFO:tasks.workunit.client.0.smithi107.stdout:8/351: creat d0/f72 x:0 0 0 2024-04-22T23:35:05.157 INFO:tasks.workunit.client.0.smithi107.stdout:8/352: write d0/d7/d20/d2d/f2c [378018,12165] 0 2024-04-22T23:35:05.161 INFO:tasks.workunit.client.0.smithi107.stdout:8/353: mkdir d0/d73 0 2024-04-22T23:35:05.162 INFO:tasks.workunit.client.0.smithi107.stdout:3/458: mknod db/d21/d27/d15/ca1 0 2024-04-22T23:35:05.163 INFO:tasks.workunit.client.0.smithi107.stdout:3/459: write db/f6f [344331,98400] 0 2024-04-22T23:35:05.163 INFO:tasks.workunit.client.0.smithi107.stdout:3/460: fsync db/d21/d27/d15/d29/f6d 0 2024-04-22T23:35:05.163 INFO:tasks.workunit.client.0.smithi107.stdout:8/354: symlink d0/d7/df/d18/d51/l74 0 2024-04-22T23:35:05.175 INFO:tasks.workunit.client.0.smithi107.stdout:4/398: dwrite d1/d7/d5b/f6c [0,4194304] 0 2024-04-22T23:35:05.175 INFO:tasks.workunit.client.1.smithi157.stdout:6/920: dwrite d1/d38/f102 [0,4194304] 0 2024-04-22T23:35:05.176 INFO:tasks.workunit.client.0.smithi107.stdout:1/392: dwrite d7/d19/d20/d39/f69 [0,4194304] 0 2024-04-22T23:35:05.176 INFO:tasks.workunit.client.0.smithi107.stdout:1/393: chown d7 91 1 2024-04-22T23:35:05.176 INFO:tasks.workunit.client.0.smithi107.stdout:5/381: dwrite d9/f24 [0,4194304] 0 2024-04-22T23:35:05.179 INFO:tasks.workunit.client.0.smithi107.stdout:5/382: truncate d9/f37 1159912 0 2024-04-22T23:35:05.179 INFO:tasks.workunit.client.0.smithi107.stdout:5/383: fdatasync d9/f4e 0 2024-04-22T23:35:05.180 INFO:tasks.workunit.client.1.smithi157.stdout:6/921: mkdir d1/d17/d5d/d12d 0 2024-04-22T23:35:05.182 INFO:tasks.workunit.client.0.smithi107.stdout:5/384: creat d9/db/de/d72/f81 x:0 0 0 2024-04-22T23:35:05.184 INFO:tasks.workunit.client.0.smithi107.stdout:8/355: dwrite d0/d7/df/f4b [0,4194304] 0 2024-04-22T23:35:05.185 INFO:tasks.workunit.client.0.smithi107.stdout:5/385: rename d9/db/de/c59 to d9/db/de/d5e/c82 0 2024-04-22T23:35:05.185 INFO:tasks.workunit.client.0.smithi107.stdout:5/386: truncate d9/db/d42/f4b 684693 0 2024-04-22T23:35:05.187 INFO:tasks.workunit.client.1.smithi157.stdout:6/922: dread d1/d2f/d3b/d39/d4e/da8/de5/f6d [0,4194304] 0 2024-04-22T23:35:05.190 INFO:tasks.workunit.client.1.smithi157.stdout:6/923: read d1/f3 [1094213,38380] 0 2024-04-22T23:35:05.190 INFO:tasks.workunit.client.1.smithi157.stdout:6/924: chown d1/d36/fd5 758770949 1 2024-04-22T23:35:05.191 INFO:tasks.workunit.client.1.smithi157.stdout:6/925: creat d1/d121/dc4/d126/f12e x:0 0 0 2024-04-22T23:35:05.193 INFO:tasks.workunit.client.1.smithi157.stdout:6/926: rename d1/d2f/d3b/d39/d4e/da8/dac/dc0/f100 to d1/d36/dea/f12f 0 2024-04-22T23:35:05.193 INFO:tasks.workunit.client.0.smithi107.stdout:8/356: symlink d0/d7/df/d3b/l75 0 2024-04-22T23:35:05.193 INFO:tasks.workunit.client.0.smithi107.stdout:8/357: readlink d0/d7/df/d3b/l75 0 2024-04-22T23:35:05.194 INFO:tasks.workunit.client.1.smithi157.stdout:6/927: rename d1/d121/d86/cb1 to d1/d36/dea/c130 0 2024-04-22T23:35:05.194 INFO:tasks.workunit.client.0.smithi107.stdout:5/387: getdents d9/d26 0 2024-04-22T23:35:05.194 INFO:tasks.workunit.client.0.smithi107.stdout:5/388: dread - d9/d26/d6a/f6f zero size 2024-04-22T23:35:05.196 INFO:tasks.workunit.client.0.smithi107.stdout:5/389: symlink d9/db/l83 0 2024-04-22T23:35:05.198 INFO:tasks.workunit.client.0.smithi107.stdout:8/358: getdents d0 0 2024-04-22T23:35:05.202 INFO:tasks.workunit.client.0.smithi107.stdout:4/399: dwrite d1/f13 [4194304,4194304] 0 2024-04-22T23:35:05.202 INFO:tasks.workunit.client.0.smithi107.stdout:4/400: fsync d1/d8/d56/d27/f30 0 2024-04-22T23:35:05.204 INFO:tasks.workunit.client.0.smithi107.stdout:1/394: dwrite d7/d19/d54/f70 [0,4194304] 0 2024-04-22T23:35:05.205 INFO:tasks.workunit.client.0.smithi107.stdout:4/401: unlink d1/d8/d67/c6d 0 2024-04-22T23:35:05.206 INFO:tasks.workunit.client.0.smithi107.stdout:4/402: dread - d1/d21/d29/d2e/d3b/f7d zero size 2024-04-22T23:35:05.211 INFO:tasks.workunit.client.0.smithi107.stdout:4/403: mkdir d1/d7/d63/d91 0 2024-04-22T23:35:05.211 INFO:tasks.workunit.client.0.smithi107.stdout:4/404: chown d1/d8/f1f 11413639 1 2024-04-22T23:35:05.214 INFO:tasks.workunit.client.0.smithi107.stdout:4/405: creat d1/d8/d56/f92 x:0 0 0 2024-04-22T23:35:05.216 INFO:tasks.workunit.client.0.smithi107.stdout:4/406: rename d1/d7/d5b/f6c to d1/d21/d29/d2e/d3b/f93 0 2024-04-22T23:35:05.218 INFO:tasks.workunit.client.0.smithi107.stdout:4/407: unlink d1/d7/l18 0 2024-04-22T23:35:05.218 INFO:tasks.workunit.client.1.smithi157.stdout:6/928: dwrite d1/d38/d97/fb4 [0,4194304] 0 2024-04-22T23:35:05.220 INFO:tasks.workunit.client.1.smithi157.stdout:6/929: rmdir d1/d38/daf 39 2024-04-22T23:35:05.220 INFO:tasks.workunit.client.0.smithi107.stdout:4/408: symlink d1/d7/d63/d78/l94 0 2024-04-22T23:35:05.220 INFO:tasks.workunit.client.0.smithi107.stdout:4/409: creat d1/d1b/d44/f95 x:0 0 0 2024-04-22T23:35:05.220 INFO:tasks.workunit.client.0.smithi107.stdout:4/410: truncate d1/f8d 1046702 0 2024-04-22T23:35:05.222 INFO:tasks.workunit.client.0.smithi107.stdout:5/390: dwrite d9/f4e [0,4194304] 0 2024-04-22T23:35:05.222 INFO:tasks.workunit.client.0.smithi107.stdout:4/411: mkdir d1/d7/d63/d78/d96 0 2024-04-22T23:35:05.223 INFO:tasks.workunit.client.0.smithi107.stdout:5/391: write d9/db/f1d [2265348,48743] 0 2024-04-22T23:35:05.225 INFO:tasks.workunit.client.0.smithi107.stdout:8/359: dwrite d0/d7/df/d17/f2a [0,4194304] 0 2024-04-22T23:35:05.225 INFO:tasks.workunit.client.0.smithi107.stdout:8/360: fdatasync d0/d7/f11 0 2024-04-22T23:35:05.226 INFO:tasks.workunit.client.0.smithi107.stdout:8/361: mkdir d0/d7/d8/d41/d53/d76 0 2024-04-22T23:35:05.229 INFO:tasks.workunit.client.0.smithi107.stdout:8/362: dread d0/d7/df/d33/d37/f49 [0,4194304] 0 2024-04-22T23:35:05.229 INFO:tasks.workunit.client.0.smithi107.stdout:8/363: fsync d0/d7/df/d3b/f3d 0 2024-04-22T23:35:05.232 INFO:tasks.workunit.client.0.smithi107.stdout:4/412: write d1/d8/f3f [3154728,86236] 0 2024-04-22T23:35:05.234 INFO:tasks.workunit.client.0.smithi107.stdout:5/392: dread d9/f43 [0,4194304] 0 2024-04-22T23:35:05.234 INFO:tasks.workunit.client.0.smithi107.stdout:5/393: creat d9/db/d42/f84 x:0 0 0 2024-04-22T23:35:05.236 INFO:tasks.workunit.client.0.smithi107.stdout:9/460: sync 2024-04-22T23:35:05.236 INFO:tasks.workunit.client.0.smithi107.stdout:2/429: sync 2024-04-22T23:35:05.236 INFO:tasks.workunit.client.0.smithi107.stdout:6/378: sync 2024-04-22T23:35:05.236 INFO:tasks.workunit.client.0.smithi107.stdout:2/430: write d1/d12/f34 [942381,47053] 0 2024-04-22T23:35:05.237 INFO:tasks.workunit.client.0.smithi107.stdout:9/461: symlink d5/d7/d27/d46/la3 0 2024-04-22T23:35:05.237 INFO:tasks.workunit.client.0.smithi107.stdout:9/462: fdatasync d5/d29/f4d 0 2024-04-22T23:35:05.238 INFO:tasks.workunit.client.0.smithi107.stdout:1/395: dwrite d7/f10 [0,4194304] 0 2024-04-22T23:35:05.238 INFO:tasks.workunit.client.0.smithi107.stdout:1/396: chown d7/d19/d71/f78 862904 1 2024-04-22T23:35:05.238 INFO:tasks.workunit.client.0.smithi107.stdout:1/397: fdatasync d7/d19/d20/f52 0 2024-04-22T23:35:05.239 INFO:tasks.workunit.client.0.smithi107.stdout:6/379: creat db/d20/d2a/d38/d4b/f6b x:0 0 0 2024-04-22T23:35:05.241 INFO:tasks.workunit.client.0.smithi107.stdout:9/463: truncate d5/d7/d27/d46/f74 4924787 0 2024-04-22T23:35:05.241 INFO:tasks.workunit.client.0.smithi107.stdout:9/464: dread - d5/d24/f7d zero size 2024-04-22T23:35:05.242 INFO:tasks.workunit.client.1.smithi157.stdout:6/930: dwrite d1/d36/fa3 [0,4194304] 0 2024-04-22T23:35:05.242 INFO:tasks.workunit.client.0.smithi107.stdout:4/413: getdents d1/d7 0 2024-04-22T23:35:05.244 INFO:tasks.workunit.client.1.smithi157.stdout:6/931: creat d1/d2f/d3b/d39/d4e/da8/dac/ddd/f131 x:0 0 0 2024-04-22T23:35:05.244 INFO:tasks.workunit.client.1.smithi157.stdout:6/932: creat d1/d2f/d3b/d39/d4e/da8/dac/dc0/f132 x:0 0 0 2024-04-22T23:35:05.245 INFO:tasks.workunit.client.0.smithi107.stdout:1/398: link d7/d19/d20/d39/d53/d43/d46/d5c/f6b d7/d19/d2d/f84 0 2024-04-22T23:35:05.246 INFO:tasks.workunit.client.0.smithi107.stdout:4/414: dread d1/d7/fc [4194304,4194304] 0 2024-04-22T23:35:05.247 INFO:tasks.workunit.client.0.smithi107.stdout:4/415: creat d1/d21/d52/f97 x:0 0 0 2024-04-22T23:35:05.247 INFO:tasks.workunit.client.0.smithi107.stdout:4/416: dread - d1/d21/d29/d2e/d3b/f90 zero size 2024-04-22T23:35:05.247 INFO:tasks.workunit.client.0.smithi107.stdout:6/380: symlink db/l6c 0 2024-04-22T23:35:05.251 INFO:tasks.workunit.client.0.smithi107.stdout:6/381: creat db/d20/d2a/d38/d61/f6d x:0 0 0 2024-04-22T23:35:05.253 INFO:tasks.workunit.client.0.smithi107.stdout:8/364: dwrite d0/d7/df/d3b/f3d [0,4194304] 0 2024-04-22T23:35:05.253 INFO:tasks.workunit.client.0.smithi107.stdout:9/465: dread d5/d6c/f87 [0,4194304] 0 2024-04-22T23:35:05.253 INFO:tasks.workunit.client.1.smithi157.stdout:4/666: sync 2024-04-22T23:35:05.253 INFO:tasks.workunit.client.1.smithi157.stdout:9/621: sync 2024-04-22T23:35:05.253 INFO:tasks.workunit.client.1.smithi157.stdout:3/585: sync 2024-04-22T23:35:05.253 INFO:tasks.workunit.client.1.smithi157.stdout:7/503: sync 2024-04-22T23:35:05.254 INFO:tasks.workunit.client.0.smithi107.stdout:6/382: rename db/f26 to db/d20/d49/f6e 0 2024-04-22T23:35:05.254 INFO:tasks.workunit.client.1.smithi157.stdout:3/586: write d0/d3/d34/d3d/d57/d5c/f78 [83306,56272] 0 2024-04-22T23:35:05.257 INFO:tasks.workunit.client.0.smithi107.stdout:6/383: dread db/d20/d2c/d3b/f60 [0,4194304] 0 2024-04-22T23:35:05.258 INFO:tasks.workunit.client.0.smithi107.stdout:6/384: dread db/d20/d2c/d3b/f60 [0,4194304] 0 2024-04-22T23:35:05.261 INFO:tasks.workunit.client.1.smithi157.stdout:3/587: mknod d0/d3/cba 0 2024-04-22T23:35:05.261 INFO:tasks.workunit.client.1.smithi157.stdout:3/588: dread - d0/d3/d34/db9/f7b zero size 2024-04-22T23:35:05.262 INFO:tasks.workunit.client.1.smithi157.stdout:6/933: dread d1/d2f/f4a [0,4194304] 0 2024-04-22T23:35:05.263 INFO:tasks.workunit.client.1.smithi157.stdout:9/622: dread d4/d15/d55/d8d/fbb [0,4194304] 0 2024-04-22T23:35:05.265 INFO:tasks.workunit.client.1.smithi157.stdout:6/934: rename d1/d121/d9b/d109/fde to d1/d121/dc4/f133 0 2024-04-22T23:35:05.265 INFO:tasks.workunit.client.1.smithi157.stdout:6/935: chown d1/c75 354 1 2024-04-22T23:35:05.265 INFO:tasks.workunit.client.1.smithi157.stdout:6/936: dread - d1/d2f/d3b/d39/d4e/da8/de5/fd6 zero size 2024-04-22T23:35:05.266 INFO:tasks.workunit.client.1.smithi157.stdout:9/623: truncate d4/f79 4014096 0 2024-04-22T23:35:05.266 INFO:tasks.workunit.client.1.smithi157.stdout:9/624: truncate d4/df/d61/f84 4216685 0 2024-04-22T23:35:05.266 INFO:tasks.workunit.client.1.smithi157.stdout:9/625: readlink d4/d15/l70 0 2024-04-22T23:35:05.266 INFO:tasks.workunit.client.1.smithi157.stdout:9/626: write d4/df/d61/f71 [1290523,14085] 0 2024-04-22T23:35:05.266 INFO:tasks.workunit.client.1.smithi157.stdout:9/627: readlink d4/d15/d23/l5e 0 2024-04-22T23:35:05.266 INFO:tasks.workunit.client.1.smithi157.stdout:9/628: fsync d4/d15/d23/d27/f33 0 2024-04-22T23:35:05.268 INFO:tasks.workunit.client.1.smithi157.stdout:9/629: unlink d4/d15/la9 0 2024-04-22T23:35:05.271 INFO:tasks.workunit.client.1.smithi157.stdout:9/630: creat d4/d15/fc5 x:0 0 0 2024-04-22T23:35:05.271 INFO:tasks.workunit.client.0.smithi107.stdout:1/399: dwrite d7/d19/d54/f70 [0,4194304] 0 2024-04-22T23:35:05.274 INFO:tasks.workunit.client.0.smithi107.stdout:2/431: dwrite d1/d2b/d49/f5a [0,4194304] 0 2024-04-22T23:35:05.274 INFO:tasks.workunit.client.0.smithi107.stdout:2/432: write d1/d2b/d59/f6b [1353266,72773] 0 2024-04-22T23:35:05.274 INFO:tasks.workunit.client.0.smithi107.stdout:2/433: write d1/d67/f5d [288927,3091] 0 2024-04-22T23:35:05.274 INFO:tasks.workunit.client.0.smithi107.stdout:2/434: write d1/d2b/d3d/d57/f5c [605278,32654] 0 2024-04-22T23:35:05.274 INFO:tasks.workunit.client.0.smithi107.stdout:2/435: creat d1/d2b/f94 x:0 0 0 2024-04-22T23:35:05.274 INFO:tasks.workunit.client.0.smithi107.stdout:2/436: chown d1/d2b/d59/c81 3821 1 2024-04-22T23:35:05.274 INFO:tasks.workunit.client.0.smithi107.stdout:2/437: creat d1/d12/f95 x:0 0 0 2024-04-22T23:35:05.277 INFO:tasks.workunit.client.1.smithi157.stdout:3/589: write d0/d3/d34/d47/f70 [2241539,39875] 0 2024-04-22T23:35:05.279 INFO:tasks.workunit.client.1.smithi157.stdout:3/590: rename d0/d3/d34/d47/f6d to d0/d3/d34/d3d/d5b/fbb 0 2024-04-22T23:35:05.280 INFO:tasks.workunit.client.1.smithi157.stdout:6/937: dread d1/d17/f43 [0,4194304] 0 2024-04-22T23:35:05.281 INFO:tasks.workunit.client.0.smithi107.stdout:9/466: dwrite d5/d14/f15 [4194304,4194304] 0 2024-04-22T23:35:05.281 INFO:tasks.workunit.client.1.smithi157.stdout:3/591: mkdir d0/d83/dbc 0 2024-04-22T23:35:05.281 INFO:tasks.workunit.client.1.smithi157.stdout:4/667: dwrite d3/d96/d40/fab [0,4194304] 0 2024-04-22T23:35:05.282 INFO:tasks.workunit.client.1.smithi157.stdout:4/668: chown d3/d11/d34/da4/db1/fe0 586937247 1 2024-04-22T23:35:05.282 INFO:tasks.workunit.client.1.smithi157.stdout:6/938: symlink d1/d38/daf/l134 0 2024-04-22T23:35:05.282 INFO:tasks.workunit.client.1.smithi157.stdout:6/939: creat d1/d17/f135 x:0 0 0 2024-04-22T23:35:05.284 INFO:tasks.workunit.client.1.smithi157.stdout:3/592: link d0/d3/d34/d3d/d57/d60/c75 d0/d3/d34/d3d/d5b/d72/cbd 0 2024-04-22T23:35:05.284 INFO:tasks.workunit.client.0.smithi107.stdout:9/467: truncate d5/f13 1292229 0 2024-04-22T23:35:05.287 INFO:tasks.workunit.client.1.smithi157.stdout:4/669: creat d3/d11/d1d/fe3 x:0 0 0 2024-04-22T23:35:05.287 INFO:tasks.workunit.client.1.smithi157.stdout:4/670: creat d3/d4b/d7a/fe4 x:0 0 0 2024-04-22T23:35:05.288 INFO:tasks.workunit.client.1.smithi157.stdout:7/504: dwrite d0/d9/de/f5f [0,4194304] 0 2024-04-22T23:35:05.289 INFO:tasks.workunit.client.0.smithi107.stdout:9/468: creat d5/d14/d2e/d4f/fa4 x:0 0 0 2024-04-22T23:35:05.289 INFO:tasks.workunit.client.0.smithi107.stdout:9/469: fsync d5/d7/d27/d46/f9f 0 2024-04-22T23:35:05.290 INFO:tasks.workunit.client.0.smithi107.stdout:9/470: mknod d5/d24/ca5 0 2024-04-22T23:35:05.292 INFO:tasks.workunit.client.0.smithi107.stdout:9/471: mkdir d5/d7/d27/da6 0 2024-04-22T23:35:05.293 INFO:tasks.workunit.client.1.smithi157.stdout:6/940: read d1/d121/d9b/f61 [2246589,1640] 0 2024-04-22T23:35:05.293 INFO:tasks.workunit.client.0.smithi107.stdout:9/472: getdents d5/d14/d2e/d4f/d83 0 2024-04-22T23:35:05.293 INFO:tasks.workunit.client.0.smithi107.stdout:9/473: readlink d5/d14/l9c 0 2024-04-22T23:35:05.293 INFO:tasks.workunit.client.0.smithi107.stdout:9/474: write d5/d14/d28/f81 [108198,4806] 0 2024-04-22T23:35:05.294 INFO:tasks.workunit.client.1.smithi157.stdout:4/671: truncate d3/d5c/fca 3213661 0 2024-04-22T23:35:05.297 INFO:tasks.workunit.client.1.smithi157.stdout:7/505: getdents d0/d9/d29/d98 0 2024-04-22T23:35:05.298 INFO:tasks.workunit.client.0.smithi107.stdout:6/385: dwrite f6 [4194304,4194304] 0 2024-04-22T23:35:05.299 INFO:tasks.workunit.client.1.smithi157.stdout:4/672: symlink d3/d96/d81/le5 0 2024-04-22T23:35:05.302 INFO:tasks.workunit.client.0.smithi107.stdout:6/386: mkdir db/d20/d2a/d38/d61/d6f 0 2024-04-22T23:35:05.302 INFO:tasks.workunit.client.0.smithi107.stdout:6/387: stat db/d20/d2a/d35 0 2024-04-22T23:35:05.302 INFO:tasks.workunit.client.1.smithi157.stdout:4/673: rename d3/d96/d81/f99 to d3/d11/fe6 0 2024-04-22T23:35:05.303 INFO:tasks.workunit.client.0.smithi107.stdout:6/388: mkdir db/d20/d2a/d38/d59/d70 0 2024-04-22T23:35:05.304 INFO:tasks.workunit.client.1.smithi157.stdout:4/674: mkdir d3/d96/d4c/de7 0 2024-04-22T23:35:05.304 INFO:tasks.workunit.client.1.smithi157.stdout:4/675: fsync d3/f58 0 2024-04-22T23:35:05.305 INFO:tasks.workunit.client.1.smithi157.stdout:4/676: truncate d3/fdc 633164 0 2024-04-22T23:35:05.306 INFO:tasks.workunit.client.1.smithi157.stdout:4/677: rename d3/d11/d1d/fce to d3/d11/d1d/d73/da7/fe8 0 2024-04-22T23:35:05.306 INFO:tasks.workunit.client.1.smithi157.stdout:4/678: stat d3/d4b/l5b 0 2024-04-22T23:35:05.307 INFO:tasks.workunit.client.0.smithi107.stdout:6/389: rmdir db/d20/d2c 39 2024-04-22T23:35:05.308 INFO:tasks.workunit.client.1.smithi157.stdout:4/679: rmdir d3/d11/d1d/d73/da7 39 2024-04-22T23:35:05.308 INFO:tasks.workunit.client.0.smithi107.stdout:6/390: rename db/d20/d2c/d3b/f51 to db/d20/d2c/d3b/d40/f71 0 2024-04-22T23:35:05.309 INFO:tasks.workunit.client.1.smithi157.stdout:4/680: rmdir d3/d11/d1d/d73/d80/dac 39 2024-04-22T23:35:05.310 INFO:tasks.workunit.client.0.smithi107.stdout:6/391: link db/c24 db/d20/d55/c72 0 2024-04-22T23:35:05.311 INFO:tasks.workunit.client.1.smithi157.stdout:4/681: rmdir d3/d11/d1d/d73/da3 39 2024-04-22T23:35:05.311 INFO:tasks.workunit.client.1.smithi157.stdout:4/682: chown d3/d11/d1d/d73/da3/f2b 246406746 1 2024-04-22T23:35:05.312 INFO:tasks.workunit.client.1.smithi157.stdout:4/683: readlink d3/d96/d40/lb2 0 2024-04-22T23:35:05.312 INFO:tasks.workunit.client.1.smithi157.stdout:4/684: read - d3/d11/d1d/d73/da7/fe8 zero size 2024-04-22T23:35:05.312 INFO:tasks.workunit.client.1.smithi157.stdout:4/685: write d3/d96/f9a [614863,23710] 0 2024-04-22T23:35:05.317 INFO:tasks.workunit.client.1.smithi157.stdout:3/593: dwrite d0/d3/d34/d3d/d5b/d72/f7f [0,4194304] 0 2024-04-22T23:35:05.320 INFO:tasks.workunit.client.0.smithi107.stdout:9/475: dwrite d5/d29/f2c [4194304,4194304] 0 2024-04-22T23:35:05.321 INFO:tasks.workunit.client.0.smithi107.stdout:9/476: write d5/d14/d2e/f3f [4445183,70997] 0 2024-04-22T23:35:05.322 INFO:tasks.workunit.client.1.smithi157.stdout:6/941: dwrite d1/d2f/f116 [0,4194304] 0 2024-04-22T23:35:05.322 INFO:tasks.workunit.client.0.smithi107.stdout:6/392: write db/fd [1357550,98921] 0 2024-04-22T23:35:05.323 INFO:tasks.workunit.client.1.smithi157.stdout:3/594: rmdir d0/d7 39 2024-04-22T23:35:05.326 INFO:tasks.workunit.client.1.smithi157.stdout:4/686: read d3/d4b/f16 [976736,129066] 0 2024-04-22T23:35:05.326 INFO:tasks.workunit.client.1.smithi157.stdout:4/687: readlink d3/d11/d1d/lb4 0 2024-04-22T23:35:05.326 INFO:tasks.workunit.client.1.smithi157.stdout:4/688: chown d3/d11/d1d/d73/da3/d31/d91/fa5 10059 1 2024-04-22T23:35:05.328 INFO:tasks.workunit.client.1.smithi157.stdout:3/595: write d0/d3/f1e [4778679,36304] 0 2024-04-22T23:35:05.328 INFO:tasks.workunit.client.0.smithi107.stdout:7/324: sync 2024-04-22T23:35:05.329 INFO:tasks.workunit.client.0.smithi107.stdout:6/393: getdents db/d20/d2a/d38/d61 0 2024-04-22T23:35:05.329 INFO:tasks.workunit.client.1.smithi157.stdout:6/942: rename d1/d2f/d3b/d39/d4e/da8/de5/da7/cf2 to d1/d2f/d3b/d39/d4e/da8/de5/d83/d93/c136 0 2024-04-22T23:35:05.329 INFO:tasks.workunit.client.1.smithi157.stdout:6/943: read d1/d121/d9b/f61 [3635978,1898] 0 2024-04-22T23:35:05.330 INFO:tasks.workunit.client.1.smithi157.stdout:6/944: truncate d1/d121/d9b/f79 4444226 0 2024-04-22T23:35:05.332 INFO:tasks.workunit.client.1.smithi157.stdout:6/945: creat d1/d2f/d3b/d39/d4e/da8/dac/de6/f137 x:0 0 0 2024-04-22T23:35:05.332 INFO:tasks.workunit.client.1.smithi157.stdout:6/946: stat d1/d2f/d3b/d39/d4e 0 2024-04-22T23:35:05.333 INFO:tasks.workunit.client.0.smithi107.stdout:6/394: getdents db/d20/d2c 0 2024-04-22T23:35:05.333 INFO:tasks.workunit.client.0.smithi107.stdout:6/395: stat db/f5e 0 2024-04-22T23:35:05.334 INFO:tasks.workunit.client.1.smithi157.stdout:6/947: unlink d1/d121/fa5 0 2024-04-22T23:35:05.336 INFO:tasks.workunit.client.0.smithi107.stdout:6/396: symlink db/d20/l73 0 2024-04-22T23:35:05.336 INFO:tasks.workunit.client.1.smithi157.stdout:6/948: mkdir d1/d17/d5d/d12d/d138 0 2024-04-22T23:35:05.337 INFO:tasks.workunit.client.0.smithi107.stdout:6/397: symlink db/d20/d2a/l74 0 2024-04-22T23:35:05.338 INFO:tasks.workunit.client.0.smithi107.stdout:6/398: write db/fd [162335,52960] 0 2024-04-22T23:35:05.338 INFO:tasks.workunit.client.1.smithi157.stdout:6/949: getdents d1/d2f/d3b/d39/d4e/da8/de5/d83 0 2024-04-22T23:35:05.338 INFO:tasks.workunit.client.1.smithi157.stdout:6/950: creat d1/d2f/d3b/d49/f139 x:0 0 0 2024-04-22T23:35:05.339 INFO:tasks.workunit.client.0.smithi107.stdout:6/399: mknod db/d20/d2a/d35/c75 0 2024-04-22T23:35:05.339 INFO:tasks.workunit.client.0.smithi107.stdout:6/400: read - db/f5e zero size 2024-04-22T23:35:05.339 INFO:tasks.workunit.client.0.smithi107.stdout:6/401: truncate db/d20/d2a/d38/f3c 2369812 0 2024-04-22T23:35:05.340 INFO:tasks.workunit.client.0.smithi107.stdout:6/402: mkdir db/d20/d2a/d38/d59/d76 0 2024-04-22T23:35:05.340 INFO:tasks.workunit.client.0.smithi107.stdout:6/403: chown db/d20/d49/f6e 14390243 1 2024-04-22T23:35:05.341 INFO:tasks.workunit.client.0.smithi107.stdout:9/477: dwrite d5/d7/d17/d1b/d1f/d36/d53/f5b [0,4194304] 0 2024-04-22T23:35:05.341 INFO:tasks.workunit.client.0.smithi107.stdout:9/478: fdatasync d5/d14/d2e/f84 0 2024-04-22T23:35:05.342 INFO:tasks.workunit.client.0.smithi107.stdout:6/404: rmdir db/d20/d2c/d3b 39 2024-04-22T23:35:05.343 INFO:tasks.workunit.client.0.smithi107.stdout:9/479: mknod d5/d7/d27/da6/ca7 0 2024-04-22T23:35:05.346 INFO:tasks.workunit.client.0.smithi107.stdout:6/405: rename db/l62 to db/d20/d2c/d3b/l77 0 2024-04-22T23:35:05.351 INFO:tasks.workunit.client.0.smithi107.stdout:7/325: dwrite d7/f5e [0,4194304] 0 2024-04-22T23:35:05.353 INFO:tasks.workunit.client.0.smithi107.stdout:7/326: readlink d7/l15 0 2024-04-22T23:35:05.353 INFO:tasks.workunit.client.0.smithi107.stdout:7/327: stat d7/d1b/d51/d6e 0 2024-04-22T23:35:05.354 INFO:tasks.workunit.client.1.smithi157.stdout:3/596: dwrite d0/d3/d34/d47/f70 [4194304,4194304] 0 2024-04-22T23:35:05.354 INFO:tasks.workunit.client.1.smithi157.stdout:3/597: creat d0/d3/fbe x:0 0 0 2024-04-22T23:35:05.358 INFO:tasks.workunit.client.1.smithi157.stdout:6/951: dread d1/fc2 [0,4194304] 0 2024-04-22T23:35:05.358 INFO:tasks.workunit.client.1.smithi157.stdout:6/952: creat d1/d2f/d3b/d39/d4e/da8/f13a x:0 0 0 2024-04-22T23:35:05.360 INFO:tasks.workunit.client.0.smithi107.stdout:6/406: dwrite db/d20/d2a/d38/d59/f5a [0,4194304] 0 2024-04-22T23:35:05.360 INFO:tasks.workunit.client.0.smithi107.stdout:6/407: fdatasync db/d20/d2c/d3b/f45 0 2024-04-22T23:35:05.362 INFO:tasks.workunit.client.1.smithi157.stdout:3/598: getdents d0/d7 0 2024-04-22T23:35:05.363 INFO:tasks.workunit.client.0.smithi107.stdout:7/328: read d7/fd [1870541,34851] 0 2024-04-22T23:35:05.363 INFO:tasks.workunit.client.0.smithi107.stdout:7/329: truncate d7/f5f 99726 0 2024-04-22T23:35:05.363 INFO:tasks.workunit.client.0.smithi107.stdout:7/330: chown d7/f5e 102450 1 2024-04-22T23:35:05.363 INFO:tasks.workunit.client.0.smithi107.stdout:7/331: chown d7/f18 72817 1 2024-04-22T23:35:05.363 INFO:tasks.workunit.client.1.smithi157.stdout:6/953: rmdir d1/d36 39 2024-04-22T23:35:05.364 INFO:tasks.workunit.client.0.smithi107.stdout:9/480: rename d5/d7/d27/da6 to d5/d7/d17/d1b/d1f/d8e/da8 0 2024-04-22T23:35:05.364 INFO:tasks.workunit.client.1.smithi157.stdout:6/954: read d1/d121/d9b/fb7 [39240,82627] 0 2024-04-22T23:35:05.364 INFO:tasks.workunit.client.1.smithi157.stdout:6/955: fsync d1/d121/f7e 0 2024-04-22T23:35:05.365 INFO:tasks.workunit.client.1.smithi157.stdout:3/599: mknod d0/d3/d34/db9/d9f/cbf 0 2024-04-22T23:35:05.366 INFO:tasks.workunit.client.0.smithi107.stdout:6/408: getdents db/d20/d58 0 2024-04-22T23:35:05.366 INFO:tasks.workunit.client.0.smithi107.stdout:9/481: creat d5/d7/d27/d46/d4b/fa9 x:0 0 0 2024-04-22T23:35:05.367 INFO:tasks.workunit.client.1.smithi157.stdout:3/600: symlink d0/d3/d34/d3d/d5b/d72/lc0 0 2024-04-22T23:35:05.367 INFO:tasks.workunit.client.1.smithi157.stdout:5/601: sync 2024-04-22T23:35:05.367 INFO:tasks.workunit.client.1.smithi157.stdout:0/547: sync 2024-04-22T23:35:05.367 INFO:tasks.workunit.client.1.smithi157.stdout:5/602: fdatasync d1/d27/d29/fdd 0 2024-04-22T23:35:05.367 INFO:tasks.workunit.client.1.smithi157.stdout:5/603: stat d1/d5/dc/d43/d5e 0 2024-04-22T23:35:05.368 INFO:tasks.workunit.client.1.smithi157.stdout:0/548: symlink d17/d38/lc1 0 2024-04-22T23:35:05.368 INFO:tasks.workunit.client.1.smithi157.stdout:0/549: creat d17/d2b/d71/fc2 x:0 0 0 2024-04-22T23:35:05.368 INFO:tasks.workunit.client.1.smithi157.stdout:5/604: mknod d1/d5/dc/d43/d4f/d7f/ce3 0 2024-04-22T23:35:05.368 INFO:tasks.workunit.client.0.smithi107.stdout:6/409: rename db/d20/d2a/d35/l46 to db/d20/d2a/l78 0 2024-04-22T23:35:05.369 INFO:tasks.workunit.client.1.smithi157.stdout:3/601: dread d0/d3/d34/d39/f4c [0,4194304] 0 2024-04-22T23:35:05.370 INFO:tasks.workunit.client.1.smithi157.stdout:5/605: creat d1/d1f/dc6/fe4 x:0 0 0 2024-04-22T23:35:05.370 INFO:tasks.workunit.client.0.smithi107.stdout:9/482: creat d5/d7/d17/d1b/d1f/d36/d96/faa x:0 0 0 2024-04-22T23:35:05.371 INFO:tasks.workunit.client.1.smithi157.stdout:3/602: mknod d0/d3/d34/d47/cc1 0 2024-04-22T23:35:05.373 INFO:tasks.workunit.client.1.smithi157.stdout:3/603: unlink d0/d3/d34/d3d/d57/f64 0 2024-04-22T23:35:05.373 INFO:tasks.workunit.client.1.smithi157.stdout:3/604: write d0/d3/d34/d3d/d57/f65 [84833,82957] 0 2024-04-22T23:35:05.374 INFO:tasks.workunit.client.1.smithi157.stdout:5/606: rename d1/d5/d7/c69 to d1/d5/dc/d43/ce5 0 2024-04-22T23:35:05.374 INFO:tasks.workunit.client.1.smithi157.stdout:5/607: creat d1/d5/dc/d43/d5e/fe6 x:0 0 0 2024-04-22T23:35:05.374 INFO:tasks.workunit.client.1.smithi157.stdout:5/608: write d1/d1f/dc6/fe4 [289215,6573] 0 2024-04-22T23:35:05.375 INFO:tasks.workunit.client.1.smithi157.stdout:3/605: creat d0/d3/d34/d3d/d57/fc2 x:0 0 0 2024-04-22T23:35:05.377 INFO:tasks.workunit.client.1.smithi157.stdout:5/609: symlink d1/d27/le7 0 2024-04-22T23:35:05.377 INFO:tasks.workunit.client.1.smithi157.stdout:5/610: write d1/db0/d7a/f8f [1252506,129593] 0 2024-04-22T23:35:05.377 INFO:tasks.workunit.client.0.smithi107.stdout:9/483: rename d5/d7/f1d to d5/d7/d17/d1b/d1f/d8e/fab 0 2024-04-22T23:35:05.378 INFO:tasks.workunit.client.1.smithi157.stdout:3/606: symlink d0/d3/d34/d4a/lc3 0 2024-04-22T23:35:05.378 INFO:tasks.workunit.client.1.smithi157.stdout:3/607: truncate d0/d3/d34/d62/f98 504192 0 2024-04-22T23:35:05.378 INFO:tasks.workunit.client.1.smithi157.stdout:3/608: chown d0/d3/d34/d39/f56 2706 1 2024-04-22T23:35:05.379 INFO:tasks.workunit.client.1.smithi157.stdout:5/611: symlink d1/d5/dc/d5d/le8 0 2024-04-22T23:35:05.380 INFO:tasks.workunit.client.0.smithi107.stdout:9/484: link d5/l6 d5/d7/d17/d1b/d1f/d36/lac 0 2024-04-22T23:35:05.380 INFO:tasks.workunit.client.0.smithi107.stdout:6/410: dread db/f29 [0,4194304] 0 2024-04-22T23:35:05.380 INFO:tasks.workunit.client.1.smithi157.stdout:5/612: mknod d1/d5/dc/d43/d4f/d7f/ce9 0 2024-04-22T23:35:05.381 INFO:tasks.workunit.client.0.smithi107.stdout:6/411: mkdir db/d20/d2a/d35/d69/d79 0 2024-04-22T23:35:05.381 INFO:tasks.workunit.client.0.smithi107.stdout:6/412: read - db/d20/d2a/d38/d61/f6d zero size 2024-04-22T23:35:05.381 INFO:tasks.workunit.client.0.smithi107.stdout:6/413: symlink db/d20/l7a 0 2024-04-22T23:35:05.388 INFO:tasks.workunit.client.0.smithi107.stdout:7/332: dwrite d7/f13 [4194304,4194304] 0 2024-04-22T23:35:05.389 INFO:tasks.workunit.client.0.smithi107.stdout:7/333: truncate d7/d2d/f41 66277 0 2024-04-22T23:35:05.389 INFO:tasks.workunit.client.0.smithi107.stdout:7/334: chown d7/d21/d48/f55 148980507 1 2024-04-22T23:35:05.392 INFO:tasks.workunit.client.1.smithi157.stdout:3/609: dread d0/d3/d34/d39/f32 [0,4194304] 0 2024-04-22T23:35:05.392 INFO:tasks.workunit.client.1.smithi157.stdout:0/550: dwrite d17/d23/f32 [0,4194304] 0 2024-04-22T23:35:05.392 INFO:tasks.workunit.client.1.smithi157.stdout:3/610: creat d0/d3/d34/d4a/fc4 x:0 0 0 2024-04-22T23:35:05.392 INFO:tasks.workunit.client.1.smithi157.stdout:3/611: chown d0/d3/d34/d3d/f63 31358248 1 2024-04-22T23:35:05.394 INFO:tasks.workunit.client.1.smithi157.stdout:0/551: creat d17/d23/d9c/fc3 x:0 0 0 2024-04-22T23:35:05.395 INFO:tasks.workunit.client.1.smithi157.stdout:5/613: dwrite d1/d5/f15 [0,4194304] 0 2024-04-22T23:35:05.396 INFO:tasks.workunit.client.1.smithi157.stdout:3/612: link d0/l9 d0/d3/d34/d4a/lc5 0 2024-04-22T23:35:05.397 INFO:tasks.workunit.client.0.smithi107.stdout:0/384: sync 2024-04-22T23:35:05.397 INFO:tasks.workunit.client.0.smithi107.stdout:0/385: chown d1/d27/d20/d40 4 1 2024-04-22T23:35:05.398 INFO:tasks.workunit.client.0.smithi107.stdout:7/335: read d7/d1b/f24 [3719169,21059] 0 2024-04-22T23:35:05.398 INFO:tasks.workunit.client.0.smithi107.stdout:7/336: dread - d7/d1b/f53 zero size 2024-04-22T23:35:05.398 INFO:tasks.workunit.client.1.smithi157.stdout:3/613: symlink d0/d7/d95/lc6 0 2024-04-22T23:35:05.399 INFO:tasks.workunit.client.1.smithi157.stdout:3/614: chown d0/d7/d6c/cb0 33254 1 2024-04-22T23:35:05.399 INFO:tasks.workunit.client.1.smithi157.stdout:3/615: dread - d0/d3/d34/d62/fae zero size 2024-04-22T23:35:05.399 INFO:tasks.workunit.client.0.smithi107.stdout:0/386: unlink d1/d84/l70 0 2024-04-22T23:35:05.404 INFO:tasks.workunit.client.0.smithi107.stdout:6/414: dwrite db/d20/d2a/d38/f56 [0,4194304] 0 2024-04-22T23:35:05.409 INFO:tasks.workunit.client.1.smithi157.stdout:0/552: dread d17/d2b/f47 [0,4194304] 0 2024-04-22T23:35:05.410 INFO:tasks.workunit.client.1.smithi157.stdout:5/614: read d1/d27/d29/fc2 [3267716,127115] 0 2024-04-22T23:35:05.410 INFO:tasks.workunit.client.1.smithi157.stdout:5/615: stat d1/f90 0 2024-04-22T23:35:05.410 INFO:tasks.workunit.client.1.smithi157.stdout:5/616: creat d1/d1f/dc6/fea x:0 0 0 2024-04-22T23:35:05.417 INFO:tasks.workunit.client.0.smithi107.stdout:6/415: read db/d20/d2c/d3b/f43 [170676,75073] 0 2024-04-22T23:35:05.417 INFO:tasks.workunit.client.0.smithi107.stdout:6/416: stat db/f1d 0 2024-04-22T23:35:05.417 INFO:tasks.workunit.client.0.smithi107.stdout:6/417: fdatasync db/f1d 0 2024-04-22T23:35:05.418 INFO:tasks.workunit.client.1.smithi157.stdout:3/616: dwrite d0/d3/d34/d47/f70 [4194304,4194304] 0 2024-04-22T23:35:05.419 INFO:tasks.workunit.client.0.smithi107.stdout:6/418: mkdir db/d20/d49/d7b 0 2024-04-22T23:35:05.420 INFO:tasks.workunit.client.0.smithi107.stdout:7/337: dwrite d7/d21/d48/f61 [0,4194304] 0 2024-04-22T23:35:05.421 INFO:tasks.workunit.client.1.smithi157.stdout:3/617: unlink d0/d3/lb8 0 2024-04-22T23:35:05.421 INFO:tasks.workunit.client.1.smithi157.stdout:3/618: readlink d0/d3/l61 0 2024-04-22T23:35:05.424 INFO:tasks.workunit.client.1.smithi157.stdout:3/619: rename d0/f2f to d0/d7/d6c/fc7 0 2024-04-22T23:35:05.424 INFO:tasks.workunit.client.1.smithi157.stdout:3/620: creat d0/d3/d34/d3d/d57/d60/d9c/fc8 x:0 0 0 2024-04-22T23:35:05.424 INFO:tasks.workunit.client.1.smithi157.stdout:3/621: write d0/f41 [4125661,121849] 0 2024-04-22T23:35:05.424 INFO:tasks.workunit.client.0.smithi107.stdout:6/419: symlink db/d20/d2a/d38/l7c 0 2024-04-22T23:35:05.425 INFO:tasks.workunit.client.0.smithi107.stdout:7/338: creat d7/d21/d48/d5c/f71 x:0 0 0 2024-04-22T23:35:05.425 INFO:tasks.workunit.client.0.smithi107.stdout:7/339: read - d7/d21/d49/f6a zero size 2024-04-22T23:35:05.426 INFO:tasks.workunit.client.0.smithi107.stdout:6/420: rmdir db/d20/d2a/d38 39 2024-04-22T23:35:05.426 INFO:tasks.workunit.client.0.smithi107.stdout:6/421: write db/d20/d2c/d3b/d40/f71 [230907,126420] 0 2024-04-22T23:35:05.426 INFO:tasks.workunit.client.0.smithi107.stdout:6/422: write db/d20/d2a/d38/d4b/f6b [955250,2426] 0 2024-04-22T23:35:05.427 INFO:tasks.workunit.client.1.smithi157.stdout:3/622: rmdir d0/d3/d34/d3d/d5b/d72/d99 0 2024-04-22T23:35:05.428 INFO:tasks.workunit.client.0.smithi107.stdout:6/423: read db/d20/d2a/d38/f3e [4697979,58555] 0 2024-04-22T23:35:05.433 INFO:tasks.workunit.client.1.smithi157.stdout:0/553: dwrite d17/d2b/f66 [0,4194304] 0 2024-04-22T23:35:05.436 INFO:tasks.workunit.client.1.smithi157.stdout:0/554: getdents d17/d2b/d4f/d5a 0 2024-04-22T23:35:05.437 INFO:tasks.workunit.client.1.smithi157.stdout:5/617: dwrite d1/d5/d7/d2f/d45/d54/f65 [0,4194304] 0 2024-04-22T23:35:05.437 INFO:tasks.workunit.client.1.smithi157.stdout:0/555: creat d17/d2b/d71/fc4 x:0 0 0 2024-04-22T23:35:05.437 INFO:tasks.workunit.client.1.smithi157.stdout:0/556: fdatasync d17/d2b/d4f/d9b/fa4 0 2024-04-22T23:35:05.441 INFO:tasks.workunit.client.1.smithi157.stdout:5/618: symlink d1/d5/dc/d43/d5e/leb 0 2024-04-22T23:35:05.444 INFO:tasks.workunit.client.0.smithi107.stdout:0/387: dwrite d1/d27/f38 [0,4194304] 0 2024-04-22T23:35:05.447 INFO:tasks.workunit.client.0.smithi107.stdout:0/388: mkdir d1/d27/d8a/d8b 0 2024-04-22T23:35:05.449 INFO:tasks.workunit.client.0.smithi107.stdout:0/389: creat d1/d9/d14/f8c x:0 0 0 2024-04-22T23:35:05.451 INFO:tasks.workunit.client.0.smithi107.stdout:0/390: rename d1/l2d to d1/d84/l8d 0 2024-04-22T23:35:05.451 INFO:tasks.workunit.client.0.smithi107.stdout:0/391: readlink d1/d9/lb 0 2024-04-22T23:35:05.451 INFO:tasks.workunit.client.0.smithi107.stdout:0/392: chown d1/ca 86509 1 2024-04-22T23:35:05.452 INFO:tasks.workunit.client.0.smithi107.stdout:0/393: rmdir d1/d9 39 2024-04-22T23:35:05.452 INFO:tasks.workunit.client.0.smithi107.stdout:0/394: stat d1/d7c 0 2024-04-22T23:35:05.453 INFO:tasks.workunit.client.0.smithi107.stdout:0/395: rmdir d1/d9/d6e 0 2024-04-22T23:35:05.453 INFO:tasks.workunit.client.0.smithi107.stdout:0/396: fdatasync d1/d27/d20/d2c/d2e/f39 0 2024-04-22T23:35:05.454 INFO:tasks.workunit.client.1.smithi157.stdout:3/623: getdents d0/d3/d34/d3d/d5b/d72 0 2024-04-22T23:35:05.455 INFO:tasks.workunit.client.0.smithi107.stdout:6/424: dwrite db/d20/d2c/d3b/f60 [0,4194304] 0 2024-04-22T23:35:05.463 INFO:tasks.workunit.client.1.smithi157.stdout:5/619: dwrite d1/d5/d7/f2e [4194304,4194304] 0 2024-04-22T23:35:05.463 INFO:tasks.workunit.client.1.smithi157.stdout:5/620: creat d1/d5/dc/d43/d4f/fec x:0 0 0 2024-04-22T23:35:05.463 INFO:tasks.workunit.client.1.smithi157.stdout:0/557: dwrite d17/d2b/f84 [0,4194304] 0 2024-04-22T23:35:05.465 INFO:tasks.workunit.client.0.smithi107.stdout:0/397: read d1/d27/d20/f6b [1081679,97923] 0 2024-04-22T23:35:05.465 INFO:tasks.workunit.client.0.smithi107.stdout:7/340: dwrite d7/d21/d48/d5c/f5d [0,4194304] 0 2024-04-22T23:35:05.467 INFO:tasks.workunit.client.1.smithi157.stdout:0/558: mkdir d17/d2b/d4f/dc5 0 2024-04-22T23:35:05.468 INFO:tasks.workunit.client.0.smithi107.stdout:0/398: dread d1/d9/d14/f80 [0,4194304] 0 2024-04-22T23:35:05.469 INFO:tasks.workunit.client.1.smithi157.stdout:5/621: dread d1/d5/d7/d2f/d45/f48 [0,4194304] 0 2024-04-22T23:35:05.469 INFO:tasks.workunit.client.1.smithi157.stdout:5/622: write d1/d5/dc/f38 [2214769,108458] 0 2024-04-22T23:35:05.470 INFO:tasks.workunit.client.1.smithi157.stdout:0/559: write d17/d23/f32 [3092480,93337] 0 2024-04-22T23:35:05.472 INFO:tasks.workunit.client.1.smithi157.stdout:5/623: mknod d1/d27/d29/ced 0 2024-04-22T23:35:05.472 INFO:tasks.workunit.client.1.smithi157.stdout:5/624: fsync d1/d5/d7/d2f/d45/da4/fac 0 2024-04-22T23:35:05.473 INFO:tasks.workunit.client.1.smithi157.stdout:3/624: dread d0/d3/d34/db9/f8c [0,4194304] 0 2024-04-22T23:35:05.473 INFO:tasks.workunit.client.1.smithi157.stdout:3/625: write d0/d3/d34/d3d/d57/d60/d6b/f85 [905180,129416] 0 2024-04-22T23:35:05.477 INFO:tasks.workunit.client.1.smithi157.stdout:3/626: rename d0/d3/d34/d4a/lc3 to d0/d83/lc9 0 2024-04-22T23:35:05.478 INFO:tasks.workunit.client.1.smithi157.stdout:0/560: dread d17/d2b/d4f/d5a/f81 [0,4194304] 0 2024-04-22T23:35:05.479 INFO:tasks.workunit.client.1.smithi157.stdout:0/561: creat d17/d2b/d5b/fc6 x:0 0 0 2024-04-22T23:35:05.481 INFO:tasks.workunit.client.0.smithi107.stdout:6/425: dwrite db/d20/d2c/f32 [4194304,4194304] 0 2024-04-22T23:35:05.482 INFO:tasks.workunit.client.1.smithi157.stdout:0/562: unlink d17/d2b/d52/f53 0 2024-04-22T23:35:05.482 INFO:tasks.workunit.client.1.smithi157.stdout:0/563: read - d17/d23/d9c/fc3 zero size 2024-04-22T23:35:05.482 INFO:tasks.workunit.client.1.smithi157.stdout:0/564: stat d17/f22 0 2024-04-22T23:35:05.482 INFO:tasks.workunit.client.1.smithi157.stdout:3/627: dread d0/d3/d34/d39/f58 [0,4194304] 0 2024-04-22T23:35:05.482 INFO:tasks.workunit.client.1.smithi157.stdout:3/628: truncate d0/d3/d34/d47/fb2 735667 0 2024-04-22T23:35:05.483 INFO:tasks.workunit.client.0.smithi107.stdout:6/426: mknod db/d20/d2a/d5d/c7d 0 2024-04-22T23:35:05.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:35:05 smithi107 ceph-mon[26355]: pgmap v259: 65 pgs: 65 active+clean; 1.3 GiB data, 5.3 GiB used, 531 GiB / 536 GiB avail; 134 MiB/s rd, 162 MiB/s wr, 650 op/s 2024-04-22T23:35:05.485 INFO:tasks.workunit.client.0.smithi107.stdout:0/399: dwrite d1/d9/f5e [0,4194304] 0 2024-04-22T23:35:05.486 INFO:tasks.workunit.client.0.smithi107.stdout:0/400: truncate d1/d27/f38 4246778 0 2024-04-22T23:35:05.487 INFO:tasks.workunit.client.1.smithi157.stdout:0/565: dread d17/d2b/d52/f42 [0,4194304] 0 2024-04-22T23:35:05.487 INFO:tasks.workunit.client.1.smithi157.stdout:0/566: readlink d17/l1d 0 2024-04-22T23:35:05.489 INFO:tasks.workunit.client.0.smithi107.stdout:6/427: dread db/d20/d49/f6e [0,4194304] 0 2024-04-22T23:35:05.489 INFO:tasks.workunit.client.1.smithi157.stdout:0/567: rename d17/d23/d34/dae to d17/d23/d34/d67/d78/dc7 0 2024-04-22T23:35:05.490 INFO:tasks.workunit.client.1.smithi157.stdout:3/629: write d0/d3/f1f [3766196,31906] 0 2024-04-22T23:35:05.490 INFO:tasks.workunit.client.1.smithi157.stdout:3/630: stat d0/d7/d6c/l7e 0 2024-04-22T23:35:05.492 INFO:tasks.workunit.client.1.smithi157.stdout:0/568: link d17/f1b d17/d2b/d71/db3/fc8 0 2024-04-22T23:35:05.492 INFO:tasks.workunit.client.1.smithi157.stdout:0/569: creat d17/d23/fc9 x:0 0 0 2024-04-22T23:35:05.492 INFO:tasks.workunit.client.1.smithi157.stdout:0/570: truncate d17/d2b/f2c 765120 0 2024-04-22T23:35:05.493 INFO:tasks.workunit.client.0.smithi107.stdout:7/341: dwrite d7/f26 [0,4194304] 0 2024-04-22T23:35:05.493 INFO:tasks.workunit.client.0.smithi107.stdout:7/342: chown d7/d1b/d51 188061 1 2024-04-22T23:35:05.493 INFO:tasks.workunit.client.0.smithi107.stdout:7/343: truncate d7/f63 874606 0 2024-04-22T23:35:05.493 INFO:tasks.workunit.client.0.smithi107.stdout:7/344: chown d7/f3b 1619279 1 2024-04-22T23:35:05.493 INFO:tasks.workunit.client.1.smithi157.stdout:3/631: unlink d0/cb 0 2024-04-22T23:35:05.496 INFO:tasks.workunit.client.1.smithi157.stdout:0/571: rename d17/d2b/d4f/d5a/da1/da8/dad to d17/d2b/d4f/dc5/dca 0 2024-04-22T23:35:05.496 INFO:tasks.workunit.client.1.smithi157.stdout:0/572: chown d17/d23/d9c/c96 177212320 1 2024-04-22T23:35:05.497 INFO:tasks.workunit.client.0.smithi107.stdout:3/461: sync 2024-04-22T23:35:05.499 INFO:tasks.workunit.client.0.smithi107.stdout:3/462: mknod db/d21/ca2 0 2024-04-22T23:35:05.501 INFO:tasks.workunit.client.0.smithi107.stdout:3/463: creat db/d21/d27/d36/d9c/fa3 x:0 0 0 2024-04-22T23:35:05.501 INFO:tasks.workunit.client.0.smithi107.stdout:3/464: read f0 [3293483,37837] 0 2024-04-22T23:35:05.501 INFO:tasks.workunit.client.0.smithi107.stdout:3/465: write db/f1c [684591,107892] 0 2024-04-22T23:35:05.504 INFO:tasks.workunit.client.0.smithi107.stdout:3/466: unlink db/d21/d27/d36/l46 0 2024-04-22T23:35:05.504 INFO:tasks.workunit.client.0.smithi107.stdout:3/467: chown db/d21/d27/d54/d25/d53/c63 394781051 1 2024-04-22T23:35:05.504 INFO:tasks.workunit.client.1.smithi157.stdout:3/632: write d0/d3/d34/d3d/f42 [9809,102275] 0 2024-04-22T23:35:05.506 INFO:tasks.workunit.client.0.smithi107.stdout:7/345: dread d7/d21/d48/d5c/f5d [0,4194304] 0 2024-04-22T23:35:05.506 INFO:tasks.workunit.client.0.smithi107.stdout:7/346: chown d7/d1b/f24 19159709 1 2024-04-22T23:35:05.507 INFO:tasks.workunit.client.1.smithi157.stdout:3/633: creat d0/d3/d13/fca x:0 0 0 2024-04-22T23:35:05.507 INFO:tasks.workunit.client.0.smithi107.stdout:8/365: sync 2024-04-22T23:35:05.507 INFO:tasks.workunit.client.0.smithi107.stdout:8/366: dread - d0/d3a/f56 zero size 2024-04-22T23:35:05.508 INFO:tasks.workunit.client.0.smithi107.stdout:1/400: sync 2024-04-22T23:35:05.508 INFO:tasks.workunit.client.0.smithi107.stdout:4/417: sync 2024-04-22T23:35:05.508 INFO:tasks.workunit.client.0.smithi107.stdout:5/394: sync 2024-04-22T23:35:05.508 INFO:tasks.workunit.client.0.smithi107.stdout:4/418: readlink d1/d7/d53/d66/d6b/l60 0 2024-04-22T23:35:05.508 INFO:tasks.workunit.client.0.smithi107.stdout:4/419: chown d1/d1b/f64 607467 1 2024-04-22T23:35:05.508 INFO:tasks.workunit.client.0.smithi107.stdout:5/395: write d9/f31 [1308898,129815] 0 2024-04-22T23:35:05.509 INFO:tasks.workunit.client.0.smithi107.stdout:5/396: dread - d9/db/de/d72/f81 zero size 2024-04-22T23:35:05.509 INFO:tasks.workunit.client.0.smithi107.stdout:8/367: dread d0/d7/df/d17/f48 [0,4194304] 0 2024-04-22T23:35:05.509 INFO:tasks.workunit.client.1.smithi157.stdout:3/634: unlink d0/d3/d34/d39/f32 0 2024-04-22T23:35:05.510 INFO:tasks.workunit.client.1.smithi157.stdout:3/635: dread - d0/d3/d34/d3d/d57/d60/d9c/fc8 zero size 2024-04-22T23:35:05.510 INFO:tasks.workunit.client.1.smithi157.stdout:3/636: fdatasync d0/d3/d34/d3d/f44 0 2024-04-22T23:35:05.510 INFO:tasks.workunit.client.0.smithi107.stdout:1/401: symlink d7/d19/d71/l85 0 2024-04-22T23:35:05.510 INFO:tasks.workunit.client.0.smithi107.stdout:1/402: write d7/d19/d20/f52 [438632,55008] 0 2024-04-22T23:35:05.510 INFO:tasks.workunit.client.0.smithi107.stdout:1/403: truncate d7/d19/d71/f78 744011 0 2024-04-22T23:35:05.512 INFO:tasks.workunit.client.0.smithi107.stdout:4/420: rename d1/d21/d29/d2e/c54 to d1/d21/d52/c98 0 2024-04-22T23:35:05.512 INFO:tasks.workunit.client.0.smithi107.stdout:4/421: fsync d1/f28 0 2024-04-22T23:35:05.513 INFO:tasks.workunit.client.0.smithi107.stdout:6/428: dwrite db/f2d [0,4194304] 0 2024-04-22T23:35:05.513 INFO:tasks.workunit.client.0.smithi107.stdout:0/401: dwrite d1/d9/f68 [4194304,4194304] 0 2024-04-22T23:35:05.514 INFO:tasks.workunit.client.0.smithi107.stdout:6/429: write db/d20/d2c/f32 [3308884,41695] 0 2024-04-22T23:35:05.514 INFO:tasks.workunit.client.1.smithi157.stdout:3/637: mkdir d0/d3/d34/d3d/d57/d60/d6b/d97/dcb 0 2024-04-22T23:35:05.514 INFO:tasks.workunit.client.0.smithi107.stdout:5/397: creat d9/d26/f85 x:0 0 0 2024-04-22T23:35:05.515 INFO:tasks.workunit.client.0.smithi107.stdout:4/422: mkdir d1/d21/d29/d2e/d88/d99 0 2024-04-22T23:35:05.517 INFO:tasks.workunit.client.1.smithi157.stdout:3/638: link d0/d3/d34/d3d/ca6 d0/d3/d34/d4a/ccc 0 2024-04-22T23:35:05.522 INFO:tasks.workunit.client.0.smithi107.stdout:6/430: rename db/d20/d2a/l78 to db/d20/d2a/d38/d4b/l7e 0 2024-04-22T23:35:05.523 INFO:tasks.workunit.client.0.smithi107.stdout:1/404: symlink d7/l86 0 2024-04-22T23:35:05.525 INFO:tasks.workunit.client.0.smithi107.stdout:4/423: rename d1/d21/d29/c50 to d1/d8/d56/c9a 0 2024-04-22T23:35:05.525 INFO:tasks.workunit.client.0.smithi107.stdout:4/424: dread - d1/d7/d53/f84 zero size 2024-04-22T23:35:05.525 INFO:tasks.workunit.client.0.smithi107.stdout:5/398: dread f4 [0,4194304] 0 2024-04-22T23:35:05.528 INFO:tasks.workunit.client.0.smithi107.stdout:5/399: creat d9/db/d42/d75/f86 x:0 0 0 2024-04-22T23:35:05.528 INFO:tasks.workunit.client.0.smithi107.stdout:5/400: chown d9/f4a 12 1 2024-04-22T23:35:05.529 INFO:tasks.workunit.client.0.smithi107.stdout:0/402: write d1/d9/f68 [6950457,93398] 0 2024-04-22T23:35:05.529 INFO:tasks.workunit.client.0.smithi107.stdout:0/403: chown d1/d67/d86 5 1 2024-04-22T23:35:05.529 INFO:tasks.workunit.client.0.smithi107.stdout:0/404: readlink d1/d27/d20/d2c/l30 0 2024-04-22T23:35:05.539 INFO:tasks.workunit.client.0.smithi107.stdout:4/425: dread d1/d21/d29/f41 [0,4194304] 0 2024-04-22T23:35:05.541 INFO:tasks.workunit.client.0.smithi107.stdout:6/431: dread db/d20/d2a/d38/d59/f5a [0,4194304] 0 2024-04-22T23:35:05.543 INFO:tasks.workunit.client.0.smithi107.stdout:4/426: link d1/d21/d29/c48 d1/d8/d56/d27/c9b 0 2024-04-22T23:35:05.543 INFO:tasks.workunit.client.0.smithi107.stdout:5/401: dread d9/db/d42/f6c [0,4194304] 0 2024-04-22T23:35:05.543 INFO:tasks.workunit.client.0.smithi107.stdout:5/402: write d9/db/de/d5e/f67 [327550,81650] 0 2024-04-22T23:35:05.545 INFO:tasks.workunit.client.0.smithi107.stdout:5/403: mknod d9/d26/c87 0 2024-04-22T23:35:05.545 INFO:tasks.workunit.client.0.smithi107.stdout:5/404: chown d9/d26/l3e 58 1 2024-04-22T23:35:05.545 INFO:tasks.workunit.client.0.smithi107.stdout:5/405: fdatasync d9/db/d42/f4b 0 2024-04-22T23:35:05.546 INFO:tasks.workunit.client.0.smithi107.stdout:4/427: getdents d1/d7/d53 0 2024-04-22T23:35:05.555 INFO:tasks.workunit.client.0.smithi107.stdout:6/432: write db/d20/d2c/f44 [834430,35789] 0 2024-04-22T23:35:05.556 INFO:tasks.workunit.client.0.smithi107.stdout:5/406: dread d9/db/f3c [0,4194304] 0 2024-04-22T23:35:05.557 INFO:tasks.workunit.client.0.smithi107.stdout:6/433: rename db/d20/d2a/d35/c75 to db/d20/d2a/d5d/c7f 0 2024-04-22T23:35:05.557 INFO:tasks.workunit.client.0.smithi107.stdout:6/434: mknod db/d20/d49/d7b/c80 0 2024-04-22T23:35:05.559 INFO:tasks.workunit.client.1.smithi157.stdout:0/573: dwrite d17/d2b/f3d [0,4194304] 0 2024-04-22T23:35:05.560 INFO:tasks.workunit.client.0.smithi107.stdout:3/468: dwrite db/d21/d27/d15/f86 [0,4194304] 0 2024-04-22T23:35:05.560 INFO:tasks.workunit.client.0.smithi107.stdout:7/347: dwrite d7/f5e [0,4194304] 0 2024-04-22T23:35:05.563 INFO:tasks.workunit.client.0.smithi107.stdout:3/469: symlink db/d21/d27/d54/d25/la4 0 2024-04-22T23:35:05.563 INFO:tasks.workunit.client.0.smithi107.stdout:3/470: truncate db/d21/d7f/d8f/fa0 350134 0 2024-04-22T23:35:05.564 INFO:tasks.workunit.client.0.smithi107.stdout:5/407: write d9/db/f3c [3018096,95326] 0 2024-04-22T23:35:05.570 INFO:tasks.workunit.client.0.smithi107.stdout:7/348: dread d7/d21/f6d [0,4194304] 0 2024-04-22T23:35:05.570 INFO:tasks.workunit.client.0.smithi107.stdout:7/349: dread - d7/d21/d48/f5b zero size 2024-04-22T23:35:05.570 INFO:tasks.workunit.client.0.smithi107.stdout:7/350: truncate d7/f63 1908093 0 2024-04-22T23:35:05.570 INFO:tasks.workunit.client.0.smithi107.stdout:7/351: chown d7/d1b/f4d 0 1 2024-04-22T23:35:05.571 INFO:tasks.workunit.client.0.smithi107.stdout:7/352: fdatasync d7/d2d/f43 0 2024-04-22T23:35:05.571 INFO:tasks.workunit.client.0.smithi107.stdout:7/353: truncate d7/f2f 1930140 0 2024-04-22T23:35:05.571 INFO:tasks.workunit.client.0.smithi107.stdout:7/354: write d7/d1b/d51/f5a [698219,53211] 0 2024-04-22T23:35:05.572 INFO:tasks.workunit.client.0.smithi107.stdout:3/471: dread db/d21/d27/d15/d24/d87/f89 [0,4194304] 0 2024-04-22T23:35:05.573 INFO:tasks.workunit.client.1.smithi157.stdout:0/574: dread d17/f22 [0,4194304] 0 2024-04-22T23:35:05.573 INFO:tasks.workunit.client.1.smithi157.stdout:0/575: write d17/f19 [4781600,73206] 0 2024-04-22T23:35:05.575 INFO:tasks.workunit.client.1.smithi157.stdout:0/576: rename d17/d23/d9c/la9 to d17/d23/d34/d67/lcb 0 2024-04-22T23:35:05.577 INFO:tasks.workunit.client.1.smithi157.stdout:0/577: mkdir d17/d2b/d4f/dc5/dcc 0 2024-04-22T23:35:05.577 INFO:tasks.workunit.client.0.smithi107.stdout:3/472: rename db/d21/d7f to db/d21/d27/d15/d24/d66/da5 0 2024-04-22T23:35:05.581 INFO:tasks.workunit.client.1.smithi157.stdout:0/578: write d17/d2b/f47 [3531965,57265] 0 2024-04-22T23:35:05.581 INFO:tasks.workunit.client.0.smithi107.stdout:7/355: dread d7/f11 [0,4194304] 0 2024-04-22T23:35:05.582 INFO:tasks.workunit.client.1.smithi157.stdout:3/639: dwrite d0/d3/d34/d39/f96 [0,4194304] 0 2024-04-22T23:35:05.584 INFO:tasks.workunit.client.0.smithi107.stdout:7/356: mknod d7/d2d/d66/c72 0 2024-04-22T23:35:05.584 INFO:tasks.workunit.client.0.smithi107.stdout:7/357: write d7/d21/d48/f55 [1413493,123031] 0 2024-04-22T23:35:05.586 INFO:tasks.workunit.client.1.smithi157.stdout:3/640: mkdir d0/d3/d34/d47/dcd 0 2024-04-22T23:35:05.586 INFO:tasks.workunit.client.0.smithi107.stdout:7/358: link d7/d21/d48/c6f d7/d4c/c73 0 2024-04-22T23:35:05.596 INFO:tasks.workunit.client.1.smithi157.stdout:2/731: sync 2024-04-22T23:35:05.596 INFO:tasks.workunit.client.1.smithi157.stdout:8/442: sync 2024-04-22T23:35:05.597 INFO:tasks.workunit.client.1.smithi157.stdout:1/665: sync 2024-04-22T23:35:05.597 INFO:tasks.workunit.client.1.smithi157.stdout:2/732: stat d7/d1f/d26/lbc 0 2024-04-22T23:35:05.597 INFO:tasks.workunit.client.1.smithi157.stdout:1/666: chown d4/db/f13 450309 1 2024-04-22T23:35:05.597 INFO:tasks.workunit.client.1.smithi157.stdout:8/443: creat d2/d6/d66/f9d x:0 0 0 2024-04-22T23:35:05.598 INFO:tasks.workunit.client.1.smithi157.stdout:2/733: getdents d7/d8/d95/dd2/dda 0 2024-04-22T23:35:05.599 INFO:tasks.workunit.client.1.smithi157.stdout:8/444: symlink d2/d34/d6a/l9e 0 2024-04-22T23:35:05.599 INFO:tasks.workunit.client.1.smithi157.stdout:1/667: rmdir d4/d17/d6b 39 2024-04-22T23:35:05.600 INFO:tasks.workunit.client.1.smithi157.stdout:8/445: rmdir d2/d6/d21/d8d 39 2024-04-22T23:35:05.602 INFO:tasks.workunit.client.1.smithi157.stdout:1/668: rename d4/d17/d34/d3b/c43 to d4/d17/d34/d3b/ce6 0 2024-04-22T23:35:05.604 INFO:tasks.workunit.client.1.smithi157.stdout:1/669: symlink d4/d5f/d7e/db8/le7 0 2024-04-22T23:35:05.605 INFO:tasks.workunit.client.1.smithi157.stdout:1/670: truncate d4/db/d3d/f82 1642728 0 2024-04-22T23:35:05.605 INFO:tasks.workunit.client.0.smithi107.stdout:7/359: dread d7/f5e [0,4194304] 0 2024-04-22T23:35:05.606 INFO:tasks.workunit.client.0.smithi107.stdout:7/360: readlink d7/d4c/l4e 0 2024-04-22T23:35:05.609 INFO:tasks.workunit.client.0.smithi107.stdout:4/428: dwrite d1/d21/d29/d2e/f4f [0,4194304] 0 2024-04-22T23:35:05.610 INFO:tasks.workunit.client.0.smithi107.stdout:0/405: dwrite d1/d27/d20/f36 [0,4194304] 0 2024-04-22T23:35:05.615 INFO:tasks.workunit.client.0.smithi107.stdout:5/408: dwrite d9/f31 [0,4194304] 0 2024-04-22T23:35:05.616 INFO:tasks.workunit.client.0.smithi107.stdout:8/368: dwrite d0/fb [0,4194304] 0 2024-04-22T23:35:05.616 INFO:tasks.workunit.client.0.smithi107.stdout:8/369: truncate d0/d3a/d46/d59/f6e 1602565 0 2024-04-22T23:35:05.616 INFO:tasks.workunit.client.0.smithi107.stdout:8/370: stat d0/d7/df/d18/d51/l74 0 2024-04-22T23:35:05.616 INFO:tasks.workunit.client.0.smithi107.stdout:8/371: write d0/d7/f11 [4001332,18737] 0 2024-04-22T23:35:05.616 INFO:tasks.workunit.client.0.smithi107.stdout:8/372: fsync d0/f72 0 2024-04-22T23:35:05.617 INFO:tasks.workunit.client.0.smithi107.stdout:4/429: creat d1/d21/d85/f9c x:0 0 0 2024-04-22T23:35:05.617 INFO:tasks.workunit.client.0.smithi107.stdout:4/430: creat d1/d1b/f9d x:0 0 0 2024-04-22T23:35:05.617 INFO:tasks.workunit.client.0.smithi107.stdout:4/431: write d1/d21/d29/d2e/d3b/f7d [670697,126815] 0 2024-04-22T23:35:05.617 INFO:tasks.workunit.client.0.smithi107.stdout:4/432: creat d1/d8/d67/f9e x:0 0 0 2024-04-22T23:35:05.617 INFO:tasks.workunit.client.0.smithi107.stdout:4/433: write d1/f28 [4632636,27430] 0 2024-04-22T23:35:05.617 INFO:tasks.workunit.client.0.smithi107.stdout:4/434: creat d1/d21/d52/f9f x:0 0 0 2024-04-22T23:35:05.617 INFO:tasks.workunit.client.0.smithi107.stdout:0/406: mknod d1/d67/c8e 0 2024-04-22T23:35:05.627 INFO:tasks.workunit.client.0.smithi107.stdout:5/409: dread d9/fd [0,4194304] 0 2024-04-22T23:35:05.628 INFO:tasks.workunit.client.0.smithi107.stdout:3/473: dwrite db/d21/d27/d15/d24/d87/f89 [0,4194304] 0 2024-04-22T23:35:05.628 INFO:tasks.workunit.client.0.smithi107.stdout:6/435: dwrite db/d20/d2a/d38/d4b/f54 [0,4194304] 0 2024-04-22T23:35:05.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:35:05 smithi157 ceph-mon[33877]: pgmap v259: 65 pgs: 65 active+clean; 1.3 GiB data, 5.3 GiB used, 531 GiB / 536 GiB avail; 134 MiB/s rd, 162 MiB/s wr, 650 op/s 2024-04-22T23:35:05.633 INFO:tasks.workunit.client.0.smithi107.stdout:7/361: dwrite d7/f63 [0,4194304] 0 2024-04-22T23:35:05.634 INFO:tasks.workunit.client.1.smithi157.stdout:3/641: dwrite d0/d3/d34/d3d/f5e [0,4194304] 0 2024-04-22T23:35:05.634 INFO:tasks.workunit.client.1.smithi157.stdout:3/642: dread - d0/d3/d34/d39/f76 zero size 2024-04-22T23:35:05.634 INFO:tasks.workunit.client.1.smithi157.stdout:2/734: dwrite d7/d8/dd/fd4 [0,4194304] 0 2024-04-22T23:35:05.634 INFO:tasks.workunit.client.0.smithi107.stdout:4/435: mkdir d1/d7/d63/d78/d96/da0 0 2024-04-22T23:35:05.635 INFO:tasks.workunit.client.1.smithi157.stdout:3/643: dread d0/d3/d34/d3d/d57/d60/d6b/f85 [0,4194304] 0 2024-04-22T23:35:05.635 INFO:tasks.workunit.client.1.smithi157.stdout:3/644: fsync d0/d3/fbe 0 2024-04-22T23:35:05.636 INFO:tasks.workunit.client.0.smithi107.stdout:0/407: creat d1/f8f x:0 0 0 2024-04-22T23:35:05.636 INFO:tasks.workunit.client.1.smithi157.stdout:2/735: unlink d7/d8/f3f 0 2024-04-22T23:35:05.636 INFO:tasks.workunit.client.1.smithi157.stdout:2/736: fsync d7/d8/d25/d55/f6c 0 2024-04-22T23:35:05.637 INFO:tasks.workunit.client.0.smithi107.stdout:5/410: truncate d9/f36 1019340 0 2024-04-22T23:35:05.637 INFO:tasks.workunit.client.0.smithi107.stdout:5/411: fsync d9/db/de/f53 0 2024-04-22T23:35:05.638 INFO:tasks.workunit.client.1.smithi157.stdout:3/645: mknod d0/d3/d34/d62/cce 0 2024-04-22T23:35:05.638 INFO:tasks.workunit.client.1.smithi157.stdout:3/646: write d0/d3/f1f [4673464,63399] 0 2024-04-22T23:35:05.640 INFO:tasks.workunit.client.1.smithi157.stdout:2/737: mkdir d7/d8/df7 0 2024-04-22T23:35:05.640 INFO:tasks.workunit.client.1.smithi157.stdout:2/738: creat d7/ff8 x:0 0 0 2024-04-22T23:35:05.640 INFO:tasks.workunit.client.1.smithi157.stdout:2/739: chown d7/d8/l1d 15 1 2024-04-22T23:35:05.641 INFO:tasks.workunit.client.1.smithi157.stdout:2/740: chown d7/d1f/led 1073067187 1 2024-04-22T23:35:05.642 INFO:tasks.workunit.client.0.smithi107.stdout:8/373: dwrite d0/f5 [0,4194304] 0 2024-04-22T23:35:05.642 INFO:tasks.workunit.client.0.smithi107.stdout:8/374: write d0/d3a/d46/d59/f68 [259288,83625] 0 2024-04-22T23:35:05.643 INFO:tasks.workunit.client.1.smithi157.stdout:0/579: dwrite d17/d2b/d71/fac [0,4194304] 0 2024-04-22T23:35:05.643 INFO:tasks.workunit.client.1.smithi157.stdout:0/580: fsync d17/d23/d34/d67/d7f/d69/f89 0 2024-04-22T23:35:05.643 INFO:tasks.workunit.client.1.smithi157.stdout:0/581: stat d17/d23/d9c/c9a 0 2024-04-22T23:35:05.643 INFO:tasks.workunit.client.1.smithi157.stdout:0/582: write d17/d23/d34/d67/d7f/f61 [4021,58679] 0 2024-04-22T23:35:05.643 INFO:tasks.workunit.client.1.smithi157.stdout:1/671: dwrite d4/d17/d1d/d33/fa2 [0,4194304] 0 2024-04-22T23:35:05.644 INFO:tasks.workunit.client.0.smithi107.stdout:5/412: dread f7 [0,4194304] 0 2024-04-22T23:35:05.644 INFO:tasks.workunit.client.0.smithi107.stdout:3/474: symlink db/la6 0 2024-04-22T23:35:05.644 INFO:tasks.workunit.client.0.smithi107.stdout:6/436: getdents db/d20/d2a/d38 0 2024-04-22T23:35:05.645 INFO:tasks.workunit.client.0.smithi107.stdout:4/436: dread d1/d7/d53/f6f [0,4194304] 0 2024-04-22T23:35:05.648 INFO:tasks.workunit.client.0.smithi107.stdout:8/375: dread d0/d7/df/d3b/f10 [4194304,4194304] 0 2024-04-22T23:35:05.648 INFO:tasks.workunit.client.0.smithi107.stdout:7/362: symlink d7/d21/d49/l74 0 2024-04-22T23:35:05.648 INFO:tasks.workunit.client.0.smithi107.stdout:7/363: fsync d7/d21/f6d 0 2024-04-22T23:35:05.649 INFO:tasks.workunit.client.1.smithi157.stdout:0/583: rename d17/d2b/d71/l76 to d17/d2b/d4f/lcd 0 2024-04-22T23:35:05.649 INFO:tasks.workunit.client.1.smithi157.stdout:0/584: chown d17/d23/f32 125364624 1 2024-04-22T23:35:05.649 INFO:tasks.workunit.client.1.smithi157.stdout:0/585: write d17/d2b/d71/fc2 [299726,65810] 0 2024-04-22T23:35:05.649 INFO:tasks.workunit.client.1.smithi157.stdout:0/586: dread - d17/d23/d9c/fc3 zero size 2024-04-22T23:35:05.649 INFO:tasks.workunit.client.0.smithi107.stdout:0/408: creat d1/d27/d20/d2c/d41/f90 x:0 0 0 2024-04-22T23:35:05.651 INFO:tasks.workunit.client.1.smithi157.stdout:1/672: mknod d4/d17/d1d/ce8 0 2024-04-22T23:35:05.652 INFO:tasks.workunit.client.1.smithi157.stdout:1/673: creat d4/d17/d1d/d33/d41/d54/fe9 x:0 0 0 2024-04-22T23:35:05.658 INFO:tasks.workunit.client.0.smithi107.stdout:5/413: symlink d9/db/de/d72/l88 0 2024-04-22T23:35:05.659 INFO:tasks.workunit.client.1.smithi157.stdout:1/674: mkdir d4/d17/d34/d75/dc7/dea 0 2024-04-22T23:35:05.659 INFO:tasks.workunit.client.0.smithi107.stdout:3/475: mkdir db/d21/d27/d54/da7 0 2024-04-22T23:35:05.662 INFO:tasks.workunit.client.0.smithi107.stdout:6/437: unlink db/d20/f4a 0 2024-04-22T23:35:05.662 INFO:tasks.workunit.client.0.smithi107.stdout:6/438: write db/d20/d2c/d3b/f60 [2463230,110016] 0 2024-04-22T23:35:05.662 INFO:tasks.workunit.client.0.smithi107.stdout:6/439: creat db/d20/d2a/d38/d4b/f81 x:0 0 0 2024-04-22T23:35:05.662 INFO:tasks.workunit.client.0.smithi107.stdout:6/440: read db/d20/d2c/f44 [3902113,31569] 0 2024-04-22T23:35:05.664 INFO:tasks.workunit.client.0.smithi107.stdout:8/376: link d0/d7/d20/d2d/f30 d0/d7/d8/d41/d53/f77 0 2024-04-22T23:35:05.664 INFO:tasks.workunit.client.0.smithi107.stdout:8/377: chown d0/d7/df/d33/c4d 45 1 2024-04-22T23:35:05.665 INFO:tasks.workunit.client.0.smithi107.stdout:0/409: mknod d1/d84/c91 0 2024-04-22T23:35:05.665 INFO:tasks.workunit.client.0.smithi107.stdout:0/410: chown d1/d27/d20/f60 21699 1 2024-04-22T23:35:05.666 INFO:tasks.workunit.client.1.smithi157.stdout:3/647: dwrite d0/f1c [0,4194304] 0 2024-04-22T23:35:05.666 INFO:tasks.workunit.client.1.smithi157.stdout:3/648: fdatasync d0/d3/d34/d3d/d5b/d72/f79 0 2024-04-22T23:35:05.668 INFO:tasks.workunit.client.0.smithi107.stdout:5/414: link d9/db/l78 d9/d26/d79/l89 0 2024-04-22T23:35:05.669 INFO:tasks.workunit.client.1.smithi157.stdout:1/675: write d4/db/f36 [1884882,105482] 0 2024-04-22T23:35:05.669 INFO:tasks.workunit.client.0.smithi107.stdout:7/364: dread d7/d21/d48/f55 [0,4194304] 0 2024-04-22T23:35:05.669 INFO:tasks.workunit.client.0.smithi107.stdout:7/365: creat d7/d21/d48/d5c/f75 x:0 0 0 2024-04-22T23:35:05.671 INFO:tasks.workunit.client.0.smithi107.stdout:3/476: link db/d21/d27/d15/d24/d52/d60/f92 db/d21/d27/d15/d29/d2a/d99/fa8 0 2024-04-22T23:35:05.671 INFO:tasks.workunit.client.1.smithi157.stdout:1/676: mknod d4/d17/d34/d3b/ceb 0 2024-04-22T23:35:05.671 INFO:tasks.workunit.client.1.smithi157.stdout:1/677: fdatasync d4/d17/d34/d75/f89 0 2024-04-22T23:35:05.672 INFO:tasks.workunit.client.0.smithi107.stdout:3/477: dread db/d21/d27/d15/d29/d2a/d45/f62 [0,4194304] 0 2024-04-22T23:35:05.672 INFO:tasks.workunit.client.0.smithi107.stdout:3/478: chown db/d21/d27/d36/f7e 960567 1 2024-04-22T23:35:05.673 INFO:tasks.workunit.client.1.smithi157.stdout:1/678: getdents d4/d17/d34/d75/dcc 0 2024-04-22T23:35:05.673 INFO:tasks.workunit.client.0.smithi107.stdout:3/479: write db/d21/d27/d15/d29/d2a/d45/f62 [517662,40114] 0 2024-04-22T23:35:05.673 INFO:tasks.workunit.client.1.smithi157.stdout:0/587: dwrite d17/d38/f73 [4194304,4194304] 0 2024-04-22T23:35:05.673 INFO:tasks.workunit.client.1.smithi157.stdout:0/588: chown d17/d23/f28 1 1 2024-04-22T23:35:05.674 INFO:tasks.workunit.client.1.smithi157.stdout:1/679: creat d4/d17/d34/dc8/fec x:0 0 0 2024-04-22T23:35:05.674 INFO:tasks.workunit.client.0.smithi107.stdout:9/485: sync 2024-04-22T23:35:05.675 INFO:tasks.workunit.client.0.smithi107.stdout:2/438: sync 2024-04-22T23:35:05.675 INFO:tasks.workunit.client.0.smithi107.stdout:2/439: dread - d1/d2b/d3d/f5e zero size 2024-04-22T23:35:05.675 INFO:tasks.workunit.client.0.smithi107.stdout:9/486: read - d5/d7/d27/d46/f9f zero size 2024-04-22T23:35:05.675 INFO:tasks.workunit.client.0.smithi107.stdout:0/411: rename d1/d9/d14/f80 to d1/d84/f92 0 2024-04-22T23:35:05.675 INFO:tasks.workunit.client.1.smithi157.stdout:1/680: mknod d4/d5f/d7e/ced 0 2024-04-22T23:35:05.676 INFO:tasks.workunit.client.1.smithi157.stdout:1/681: write d4/d17/d34/d3b/d52/fd9 [663572,57570] 0 2024-04-22T23:35:05.679 INFO:tasks.workunit.client.0.smithi107.stdout:5/415: creat d9/d76/f8a x:0 0 0 2024-04-22T23:35:05.682 INFO:tasks.workunit.client.0.smithi107.stdout:7/366: creat d7/f76 x:0 0 0 2024-04-22T23:35:05.682 INFO:tasks.workunit.client.0.smithi107.stdout:7/367: write d7/d1b/d51/f5a [1741461,26066] 0 2024-04-22T23:35:05.684 INFO:tasks.workunit.client.0.smithi107.stdout:9/487: dread d5/d6c/f87 [0,4194304] 0 2024-04-22T23:35:05.684 INFO:tasks.workunit.client.0.smithi107.stdout:2/440: mknod d1/d2b/d3d/c96 0 2024-04-22T23:35:05.684 INFO:tasks.workunit.client.0.smithi107.stdout:2/441: chown d1/l4d 56545610 1 2024-04-22T23:35:05.686 INFO:tasks.workunit.client.0.smithi107.stdout:4/437: dwrite d1/d7/f11 [0,4194304] 0 2024-04-22T23:35:05.686 INFO:tasks.workunit.client.0.smithi107.stdout:4/438: chown d1/d8/f3f 3 1 2024-04-22T23:35:05.686 INFO:tasks.workunit.client.0.smithi107.stdout:4/439: write d1/d1b/f7a [583167,44203] 0 2024-04-22T23:35:05.686 INFO:tasks.workunit.client.0.smithi107.stdout:4/440: fdatasync d1/d8/d56/d27/f30 0 2024-04-22T23:35:05.686 INFO:tasks.workunit.client.0.smithi107.stdout:4/441: dread - d1/d8/d56/f92 zero size 2024-04-22T23:35:05.687 INFO:tasks.workunit.client.0.smithi107.stdout:5/416: mknod d9/d76/c8b 0 2024-04-22T23:35:05.687 INFO:tasks.workunit.client.0.smithi107.stdout:5/417: readlink d9/l15 0 2024-04-22T23:35:05.687 INFO:tasks.workunit.client.0.smithi107.stdout:5/418: write f4 [5277945,22171] 0 2024-04-22T23:35:05.687 INFO:tasks.workunit.client.0.smithi107.stdout:5/419: chown d9/d76/f7d 1920 1 2024-04-22T23:35:05.687 INFO:tasks.workunit.client.1.smithi157.stdout:2/741: dwrite d7/d8/d8e/f88 [0,4194304] 0 2024-04-22T23:35:05.688 INFO:tasks.workunit.client.0.smithi107.stdout:2/442: creat d1/d2b/d3d/d57/f97 x:0 0 0 2024-04-22T23:35:05.690 INFO:tasks.workunit.client.0.smithi107.stdout:4/442: symlink d1/d7/d63/d78/d80/la1 0 2024-04-22T23:35:05.690 INFO:tasks.workunit.client.0.smithi107.stdout:4/443: fsync d1/d21/d29/d2e/f46 0 2024-04-22T23:35:05.690 INFO:tasks.workunit.client.0.smithi107.stdout:9/488: read d5/d29/f3a [43793,28273] 0 2024-04-22T23:35:05.691 INFO:tasks.workunit.client.1.smithi157.stdout:2/742: rmdir d7/d8/d95/dde/dec 0 2024-04-22T23:35:05.691 INFO:tasks.workunit.client.1.smithi157.stdout:2/743: read d7/d8/f2f [1835014,112047] 0 2024-04-22T23:35:05.691 INFO:tasks.workunit.client.1.smithi157.stdout:2/744: getdents d7/d8/d95/dd2 0 2024-04-22T23:35:05.692 INFO:tasks.workunit.client.0.smithi107.stdout:2/443: dread d1/d12/d35/f7a [0,4194304] 0 2024-04-22T23:35:05.694 INFO:tasks.workunit.client.1.smithi157.stdout:2/745: link d7/d1f/d26/f74 d7/d8/d95/ff9 0 2024-04-22T23:35:05.694 INFO:tasks.workunit.client.1.smithi157.stdout:2/746: readlink d7/d8/d30/d23/d27/d57/dcf/lf1 0 2024-04-22T23:35:05.694 INFO:tasks.workunit.client.1.smithi157.stdout:2/747: truncate d7/d8/d30/fe4 294644 0 2024-04-22T23:35:05.695 INFO:tasks.workunit.client.0.smithi107.stdout:4/444: rename d1/d7/f33 to d1/d7/d63/d78/d96/da0/fa2 0 2024-04-22T23:35:05.695 INFO:tasks.workunit.client.0.smithi107.stdout:9/489: rename d5/d7/d27/d46 to d5/d7/d27/d46/dad 22 2024-04-22T23:35:05.696 INFO:tasks.workunit.client.1.smithi157.stdout:2/748: mkdir d7/d8/d30/d23/d27/d57/dcf/dfa 0 2024-04-22T23:35:05.696 INFO:tasks.workunit.client.1.smithi157.stdout:2/749: creat d7/d8/d95/ffb x:0 0 0 2024-04-22T23:35:05.696 INFO:tasks.workunit.client.1.smithi157.stdout:2/750: fsync d7/d1f/d26/f44 0 2024-04-22T23:35:05.697 INFO:tasks.workunit.client.0.smithi107.stdout:2/444: symlink d1/d2b/d59/l98 0 2024-04-22T23:35:05.697 INFO:tasks.workunit.client.0.smithi107.stdout:2/445: truncate d1/d2b/f39 1015286 0 2024-04-22T23:35:05.697 INFO:tasks.workunit.client.0.smithi107.stdout:2/446: readlink d1/d12/d35/d48/d13/l76 0 2024-04-22T23:35:05.698 INFO:tasks.workunit.client.0.smithi107.stdout:4/445: mknod d1/d7/d53/d66/ca3 0 2024-04-22T23:35:05.701 INFO:tasks.workunit.client.0.smithi107.stdout:4/446: mkdir d1/d7/d63/d78/da4 0 2024-04-22T23:35:05.703 INFO:tasks.workunit.client.0.smithi107.stdout:5/420: dread d9/fd [0,4194304] 0 2024-04-22T23:35:05.703 INFO:tasks.workunit.client.0.smithi107.stdout:5/421: chown d9/f37 72558706 1 2024-04-22T23:35:05.703 INFO:tasks.workunit.client.0.smithi107.stdout:5/422: creat d9/db/f8c x:0 0 0 2024-04-22T23:35:05.703 INFO:tasks.workunit.client.0.smithi107.stdout:2/447: dread d1/d12/f34 [0,4194304] 0 2024-04-22T23:35:05.703 INFO:tasks.workunit.client.0.smithi107.stdout:2/448: stat d1/d2b/d59/f6b 0 2024-04-22T23:35:05.704 INFO:tasks.workunit.client.0.smithi107.stdout:5/423: mknod d9/db/de/d72/d7c/c8d 0 2024-04-22T23:35:05.705 INFO:tasks.workunit.client.0.smithi107.stdout:2/449: mknod d1/d63/c99 0 2024-04-22T23:35:05.706 INFO:tasks.workunit.client.0.smithi107.stdout:2/450: stat d1/d12/d35/d48/c2e 0 2024-04-22T23:35:05.706 INFO:tasks.workunit.client.0.smithi107.stdout:2/451: read - d1/d2b/f80 zero size 2024-04-22T23:35:05.709 INFO:tasks.workunit.client.0.smithi107.stdout:1/405: sync 2024-04-22T23:35:05.710 INFO:tasks.workunit.client.0.smithi107.stdout:2/452: dread d1/d12/d35/f45 [0,4194304] 0 2024-04-22T23:35:05.711 INFO:tasks.workunit.client.0.smithi107.stdout:3/480: dwrite db/d21/d27/f39 [0,4194304] 0 2024-04-22T23:35:05.712 INFO:tasks.workunit.client.0.smithi107.stdout:1/406: rename d7/l2f to d7/d19/d20/d39/d53/d43/d46/d68/l87 0 2024-04-22T23:35:05.712 INFO:tasks.workunit.client.0.smithi107.stdout:1/407: write d7/d19/d20/d39/d53/d43/f49 [5205462,91564] 0 2024-04-22T23:35:05.713 INFO:tasks.workunit.client.0.smithi107.stdout:1/408: write d7/d19/d20/d39/f7e [378528,90826] 0 2024-04-22T23:35:05.714 INFO:tasks.workunit.client.0.smithi107.stdout:2/453: mknod d1/d2b/d49/c9a 0 2024-04-22T23:35:05.714 INFO:tasks.workunit.client.0.smithi107.stdout:2/454: stat d1/d12 0 2024-04-22T23:35:05.714 INFO:tasks.workunit.client.0.smithi107.stdout:2/455: chown d1/d2b/d3d 9 1 2024-04-22T23:35:05.714 INFO:tasks.workunit.client.0.smithi107.stdout:2/456: dread - d1/d12/d35/d48/d13/f74 zero size 2024-04-22T23:35:05.715 INFO:tasks.workunit.client.0.smithi107.stdout:2/457: read d1/f1a [1584277,86029] 0 2024-04-22T23:35:05.715 INFO:tasks.workunit.client.0.smithi107.stdout:3/481: symlink db/d21/d27/d15/d29/la9 0 2024-04-22T23:35:05.715 INFO:tasks.workunit.client.0.smithi107.stdout:4/447: dread d1/d8/f4b [0,4194304] 0 2024-04-22T23:35:05.715 INFO:tasks.workunit.client.0.smithi107.stdout:8/378: dwrite d0/d7/df/d33/d37/f49 [0,4194304] 0 2024-04-22T23:35:05.715 INFO:tasks.workunit.client.0.smithi107.stdout:2/458: mknod d1/d2b/d3d/d57/c9b 0 2024-04-22T23:35:05.716 INFO:tasks.workunit.client.0.smithi107.stdout:3/482: rmdir db/d21/d27 39 2024-04-22T23:35:05.716 INFO:tasks.workunit.client.0.smithi107.stdout:5/424: write d9/d26/f64 [30927,87487] 0 2024-04-22T23:35:05.716 INFO:tasks.workunit.client.0.smithi107.stdout:8/379: mkdir d0/d3a/d46/d59/d78 0 2024-04-22T23:35:05.718 INFO:tasks.workunit.client.0.smithi107.stdout:2/459: symlink d1/d12/d35/d48/d13/d4f/l9c 0 2024-04-22T23:35:05.718 INFO:tasks.workunit.client.0.smithi107.stdout:2/460: chown d1/f3c 26183 1 2024-04-22T23:35:05.718 INFO:tasks.workunit.client.0.smithi107.stdout:2/461: dread - d1/d2b/f80 zero size 2024-04-22T23:35:05.719 INFO:tasks.workunit.client.0.smithi107.stdout:8/380: mknod d0/d7/df/d33/d37/c79 0 2024-04-22T23:35:05.719 INFO:tasks.workunit.client.0.smithi107.stdout:2/462: rmdir d1/d2b/d3d 39 2024-04-22T23:35:05.720 INFO:tasks.workunit.client.0.smithi107.stdout:3/483: getdents db/d21/d27/d54/d25/d6e 0 2024-04-22T23:35:05.720 INFO:tasks.workunit.client.0.smithi107.stdout:3/484: fsync db/d21/d27/f23 0 2024-04-22T23:35:05.721 INFO:tasks.workunit.client.0.smithi107.stdout:8/381: symlink d0/d7/d8/d41/d53/d76/l7a 0 2024-04-22T23:35:05.724 INFO:tasks.workunit.client.0.smithi107.stdout:2/463: creat d1/d12/d35/d48/f9d x:0 0 0 2024-04-22T23:35:05.724 INFO:tasks.workunit.client.0.smithi107.stdout:3/485: mknod db/d21/d27/d15/d29/d2a/caa 0 2024-04-22T23:35:05.726 INFO:tasks.workunit.client.0.smithi107.stdout:7/368: dwrite d7/d21/f6d [4194304,4194304] 0 2024-04-22T23:35:05.726 INFO:tasks.workunit.client.0.smithi107.stdout:7/369: readlink d7/l4f 0 2024-04-22T23:35:05.726 INFO:tasks.workunit.client.0.smithi107.stdout:6/441: dwrite db/d20/d2c/f32 [0,4194304] 0 2024-04-22T23:35:05.726 INFO:tasks.workunit.client.0.smithi107.stdout:6/442: stat db/d20/d2c/f44 0 2024-04-22T23:35:05.726 INFO:tasks.workunit.client.0.smithi107.stdout:5/425: write d9/d26/f41 [4017789,72580] 0 2024-04-22T23:35:05.726 INFO:tasks.workunit.client.0.smithi107.stdout:6/443: write db/d20/d2c/d3b/f43 [1621960,59901] 0 2024-04-22T23:35:05.727 INFO:tasks.workunit.client.0.smithi107.stdout:4/448: dread d1/d21/d29/d2e/f46 [0,4194304] 0 2024-04-22T23:35:05.728 INFO:tasks.workunit.client.0.smithi107.stdout:9/490: dwrite d5/f10 [4194304,4194304] 0 2024-04-22T23:35:05.729 INFO:tasks.workunit.client.1.smithi157.stdout:2/751: dwrite d7/d8/d25/f50 [0,4194304] 0 2024-04-22T23:35:05.729 INFO:tasks.workunit.client.1.smithi157.stdout:2/752: chown d7/d8/d8e/f7e 253 1 2024-04-22T23:35:05.729 INFO:tasks.workunit.client.0.smithi107.stdout:1/409: dread d7/d19/d20/d39/d53/f4e [0,4194304] 0 2024-04-22T23:35:05.730 INFO:tasks.workunit.client.0.smithi107.stdout:1/410: fdatasync d7/f67 0 2024-04-22T23:35:05.730 INFO:tasks.workunit.client.0.smithi107.stdout:1/411: chown d7/d19/d20/f58 59 1 2024-04-22T23:35:05.730 INFO:tasks.workunit.client.0.smithi107.stdout:1/412: chown d7/d19/d20/d39/d53/d43/d63/f74 1475 1 2024-04-22T23:35:05.730 INFO:tasks.workunit.client.0.smithi107.stdout:0/412: dwrite d1/d27/d20/d2c/d41/f54 [0,4194304] 0 2024-04-22T23:35:05.731 INFO:tasks.workunit.client.1.smithi157.stdout:2/753: rmdir d7/d8/dd/d4d 39 2024-04-22T23:35:05.731 INFO:tasks.workunit.client.1.smithi157.stdout:2/754: fdatasync d7/d8/dd/d4d/fe3 0 2024-04-22T23:35:05.744 INFO:tasks.workunit.client.0.smithi107.stdout:0/413: dread d1/d27/f38 [0,4194304] 0 2024-04-22T23:35:05.744 INFO:tasks.workunit.client.0.smithi107.stdout:0/414: write d1/d27/f57 [361584,66503] 0 2024-04-22T23:35:05.744 INFO:tasks.workunit.client.0.smithi107.stdout:0/415: write d1/d9/f68 [8844279,96519] 0 2024-04-22T23:35:05.744 INFO:tasks.workunit.client.0.smithi107.stdout:0/416: readlink d1/d9/d14/d4f/l66 0 2024-04-22T23:35:05.754 INFO:tasks.workunit.client.1.smithi157.stdout:2/755: dwrite d7/d1f/d26/d4a/f60 [4194304,4194304] 0 2024-04-22T23:35:05.754 INFO:tasks.workunit.client.1.smithi157.stdout:2/756: mknod d7/d1f/d26/cfc 0 2024-04-22T23:35:05.754 INFO:tasks.workunit.client.1.smithi157.stdout:2/757: fsync d7/d1f/d26/d4a/fc9 0 2024-04-22T23:35:05.754 INFO:tasks.workunit.client.1.smithi157.stdout:2/758: fdatasync d7/d8/d30/fb3 0 2024-04-22T23:35:05.755 INFO:tasks.workunit.client.0.smithi107.stdout:3/486: dwrite db/d21/f33 [4194304,4194304] 0 2024-04-22T23:35:05.755 INFO:tasks.workunit.client.0.smithi107.stdout:3/487: chown l4 432993 1 2024-04-22T23:35:05.755 INFO:tasks.workunit.client.0.smithi107.stdout:3/488: write db/d21/d27/d15/d29/d2a/f43 [557785,10838] 0 2024-04-22T23:35:05.756 INFO:tasks.workunit.client.0.smithi107.stdout:3/489: write db/d21/d27/d15/d29/d2a/d40/f6c [688215,105739] 0 2024-04-22T23:35:05.756 INFO:tasks.workunit.client.0.smithi107.stdout:3/490: chown db/d21/d27/d54/d25/d6e/f8e 67120926 1 2024-04-22T23:35:05.770 INFO:tasks.workunit.client.1.smithi157.stdout:2/759: dread d7/d8/d25/f4f [0,4194304] 0 2024-04-22T23:35:05.772 INFO:tasks.workunit.client.0.smithi107.stdout:3/491: dwrite db/d21/d27/d15/d29/f42 [0,4194304] 0 2024-04-22T23:35:05.772 INFO:tasks.workunit.client.1.smithi157.stdout:2/760: unlink d7/d8/d95/dd2/dda/f97 0 2024-04-22T23:35:05.774 INFO:tasks.workunit.client.1.smithi157.stdout:2/761: creat d7/d8/d30/d23/d27/d57/dcf/ffd x:0 0 0 2024-04-22T23:35:05.775 INFO:tasks.workunit.client.1.smithi157.stdout:2/762: creat d7/d8/d30/db8/ffe x:0 0 0 2024-04-22T23:35:05.775 INFO:tasks.workunit.client.1.smithi157.stdout:2/763: write d7/d8/d30/f9c [1948089,98231] 0 2024-04-22T23:35:05.775 INFO:tasks.workunit.client.1.smithi157.stdout:2/764: creat d7/d1f/d26/d4a/fff x:0 0 0 2024-04-22T23:35:05.793 INFO:tasks.workunit.client.0.smithi107.stdout:8/382: mkdir d0/d7/df/d3b/d7b 0 2024-04-22T23:35:05.794 INFO:tasks.workunit.client.0.smithi107.stdout:2/464: unlink d1/f3c 0 2024-04-22T23:35:05.795 INFO:tasks.workunit.client.0.smithi107.stdout:2/465: write d1/d12/d35/d48/d13/f32 [1326194,61778] 0 2024-04-22T23:35:05.797 INFO:tasks.workunit.client.0.smithi107.stdout:7/370: symlink d7/d1b/l77 0 2024-04-22T23:35:05.798 INFO:tasks.workunit.client.0.smithi107.stdout:5/426: symlink d9/db/d42/l8e 0 2024-04-22T23:35:05.799 INFO:tasks.workunit.client.0.smithi107.stdout:6/444: mknod db/d20/d2a/d5f/c82 0 2024-04-22T23:35:05.799 INFO:tasks.workunit.client.0.smithi107.stdout:6/445: fsync db/d20/d2a/d38/d59/f5a 0 2024-04-22T23:35:05.801 INFO:tasks.workunit.client.0.smithi107.stdout:4/449: symlink d1/d8/d56/la5 0 2024-04-22T23:35:05.802 INFO:tasks.workunit.client.0.smithi107.stdout:1/413: symlink d7/d19/d20/d39/l88 0 2024-04-22T23:35:05.802 INFO:tasks.workunit.client.0.smithi107.stdout:1/414: fsync d7/d19/d20/d39/d53/f5f 0 2024-04-22T23:35:05.803 INFO:tasks.workunit.client.0.smithi107.stdout:1/415: chown d7/d19/d42/c7d 49425948 1 2024-04-22T23:35:05.803 INFO:tasks.workunit.client.0.smithi107.stdout:8/383: write d0/f19 [7105482,31646] 0 2024-04-22T23:35:05.803 INFO:tasks.workunit.client.0.smithi107.stdout:8/384: fdatasync d0/d3a/d46/d59/f6b 0 2024-04-22T23:35:05.803 INFO:tasks.workunit.client.0.smithi107.stdout:8/385: fsync d0/d3a/f4a 0 2024-04-22T23:35:05.803 INFO:tasks.workunit.client.0.smithi107.stdout:5/427: dread d9/f58 [0,4194304] 0 2024-04-22T23:35:05.803 INFO:tasks.workunit.client.0.smithi107.stdout:5/428: creat d9/db/f8f x:0 0 0 2024-04-22T23:35:05.813 INFO:tasks.workunit.client.0.smithi107.stdout:0/417: rmdir d1/d9/d14/d4f/d59 39 2024-04-22T23:35:05.819 INFO:tasks.workunit.client.0.smithi107.stdout:6/446: dwrite db/f5e [0,4194304] 0 2024-04-22T23:35:05.820 INFO:tasks.workunit.client.0.smithi107.stdout:6/447: fdatasync db/d20/d2c/d3b/f60 0 2024-04-22T23:35:05.829 INFO:tasks.workunit.client.0.smithi107.stdout:4/450: dwrite d1/d21/f6a [0,4194304] 0 2024-04-22T23:35:05.829 INFO:tasks.workunit.client.0.smithi107.stdout:8/386: dwrite d0/d7/d8/d41/d53/f67 [0,4194304] 0 2024-04-22T23:35:05.833 INFO:tasks.workunit.client.0.smithi107.stdout:3/492: rename db/d21/d27/d15/d24/d87/l9d to db/d21/d27/d91/lab 0 2024-04-22T23:35:05.835 INFO:tasks.workunit.client.0.smithi107.stdout:2/466: mknod d1/d2b/d49/d87/c9e 0 2024-04-22T23:35:05.835 INFO:tasks.workunit.client.0.smithi107.stdout:4/451: dread d1/d8/f4b [0,4194304] 0 2024-04-22T23:35:05.835 INFO:tasks.workunit.client.0.smithi107.stdout:4/452: fsync d1/d21/d29/d2e/d88/f8b 0 2024-04-22T23:35:05.835 INFO:tasks.workunit.client.0.smithi107.stdout:7/371: rmdir d7/d21 39 2024-04-22T23:35:05.836 INFO:tasks.workunit.client.0.smithi107.stdout:1/416: rmdir d7/d19/d20/d39/d53/d43/d46 39 2024-04-22T23:35:05.837 INFO:tasks.workunit.client.0.smithi107.stdout:6/448: dwrite db/d20/d2c/d3b/f45 [0,4194304] 0 2024-04-22T23:35:05.838 INFO:tasks.workunit.client.0.smithi107.stdout:6/449: chown db/d20/l36 128430 1 2024-04-22T23:35:05.838 INFO:tasks.workunit.client.0.smithi107.stdout:6/450: readlink db/d20/d2c/d3b/l63 0 2024-04-22T23:35:05.840 INFO:tasks.workunit.client.0.smithi107.stdout:5/429: creat d9/db/de/d5e/f90 x:0 0 0 2024-04-22T23:35:05.844 INFO:tasks.workunit.client.0.smithi107.stdout:0/418: unlink d1/d27/d20/d2c/d2e/f5a 0 2024-04-22T23:35:05.844 INFO:tasks.workunit.client.1.smithi157.stdout:5/625: sync 2024-04-22T23:35:05.844 INFO:tasks.workunit.client.1.smithi157.stdout:4/689: sync 2024-04-22T23:35:05.844 INFO:tasks.workunit.client.1.smithi157.stdout:6/956: sync 2024-04-22T23:35:05.845 INFO:tasks.workunit.client.1.smithi157.stdout:7/506: sync 2024-04-22T23:35:05.845 INFO:tasks.workunit.client.1.smithi157.stdout:9/631: sync 2024-04-22T23:35:05.845 INFO:tasks.workunit.client.1.smithi157.stdout:5/626: fsync d1/d5/d7/d2f/f79 0 2024-04-22T23:35:05.845 INFO:tasks.workunit.client.1.smithi157.stdout:6/957: dread - d1/d2f/d3b/d39/d4e/da8/de5/d83/fd0 zero size 2024-04-22T23:35:05.845 INFO:tasks.workunit.client.1.smithi157.stdout:6/958: chown d1/d17 15 1 2024-04-22T23:35:05.849 INFO:tasks.workunit.client.1.smithi157.stdout:5/627: symlink d1/d5/d7/lee 0 2024-04-22T23:35:05.849 INFO:tasks.workunit.client.1.smithi157.stdout:5/628: dread - d1/d27/fbc zero size 2024-04-22T23:35:05.850 INFO:tasks.workunit.client.1.smithi157.stdout:5/629: creat d1/d5/dc/d43/d4f/d7f/dab/fef x:0 0 0 2024-04-22T23:35:05.855 INFO:tasks.workunit.client.1.smithi157.stdout:5/630: getdents d1/d5/d7/d2f/d45 0 2024-04-22T23:35:05.856 INFO:tasks.workunit.client.1.smithi157.stdout:5/631: creat d1/d5/dc/d43/d4f/d7f/dab/ff0 x:0 0 0 2024-04-22T23:35:05.859 INFO:tasks.workunit.client.0.smithi107.stdout:3/493: symlink db/d21/d27/d15/d24/d52/d60/d96/lac 0 2024-04-22T23:35:05.859 INFO:tasks.workunit.client.0.smithi107.stdout:3/494: chown db/d21/d27/d36/f7e 2072 1 2024-04-22T23:35:05.860 INFO:tasks.workunit.client.1.smithi157.stdout:5/632: creat d1/d1f/ff1 x:0 0 0 2024-04-22T23:35:05.861 INFO:tasks.workunit.client.1.smithi157.stdout:6/959: dread d1/d2f/d3b/f2a [0,4194304] 0 2024-04-22T23:35:05.861 INFO:tasks.workunit.client.0.smithi107.stdout:1/417: mkdir d7/d19/d2d/d89 0 2024-04-22T23:35:05.862 INFO:tasks.workunit.client.0.smithi107.stdout:1/418: fsync d7/d19/d20/d39/d53/d43/d46/d5c/f5e 0 2024-04-22T23:35:05.863 INFO:tasks.workunit.client.1.smithi157.stdout:5/633: getdents d1/d5/dc 0 2024-04-22T23:35:05.867 INFO:tasks.workunit.client.0.smithi107.stdout:6/451: mknod db/d20/d2a/d5f/c83 0 2024-04-22T23:35:05.867 INFO:tasks.workunit.client.0.smithi107.stdout:6/452: readlink db/d20/d2a/l34 0 2024-04-22T23:35:05.867 INFO:tasks.workunit.client.0.smithi107.stdout:6/453: write db/d20/d58/f65 [75176,130370] 0 2024-04-22T23:35:05.867 INFO:tasks.workunit.client.0.smithi107.stdout:5/430: stat d9/db/de/c16 0 2024-04-22T23:35:05.869 INFO:tasks.workunit.client.0.smithi107.stdout:0/419: truncate d1/d27/d20/d2c/d41/f64 69021 0 2024-04-22T23:35:05.872 INFO:tasks.workunit.client.1.smithi157.stdout:5/634: dread d1/d5/d7/d10/f49 [4194304,4194304] 0 2024-04-22T23:35:05.872 INFO:tasks.workunit.client.0.smithi107.stdout:6/454: mkdir db/d20/d55/d84 0 2024-04-22T23:35:05.873 INFO:tasks.workunit.client.0.smithi107.stdout:6/455: chown db/d20/d2a/d38/f3e 410 1 2024-04-22T23:35:05.873 INFO:tasks.workunit.client.0.smithi107.stdout:6/456: stat db/d20/d2a/d35/d69 0 2024-04-22T23:35:05.874 INFO:tasks.workunit.client.0.smithi107.stdout:5/431: symlink d9/d76/l91 0 2024-04-22T23:35:05.874 INFO:tasks.workunit.client.1.smithi157.stdout:5/635: rename d1/d5/d7/dbd/cd1 to d1/d5/d7/d10/cf2 0 2024-04-22T23:35:05.874 INFO:tasks.workunit.client.1.smithi157.stdout:5/636: write d1/d27/d29/fbf [354711,11504] 0 2024-04-22T23:35:05.875 INFO:tasks.workunit.client.0.smithi107.stdout:1/419: dread d7/d19/d2d/f3d [0,4194304] 0 2024-04-22T23:35:05.876 INFO:tasks.workunit.client.0.smithi107.stdout:0/420: rename d1/d27/d20/c47 to d1/d9/d14/c93 0 2024-04-22T23:35:05.876 INFO:tasks.workunit.client.0.smithi107.stdout:4/453: dwrite d1/d1b/d44/f95 [0,4194304] 0 2024-04-22T23:35:05.876 INFO:tasks.workunit.client.0.smithi107.stdout:4/454: chown d1/d8/f7c 1922 1 2024-04-22T23:35:05.877 INFO:tasks.workunit.client.1.smithi157.stdout:5/637: mkdir d1/db0/df3 0 2024-04-22T23:35:05.877 INFO:tasks.workunit.client.0.smithi107.stdout:5/432: write f7 [1227141,106474] 0 2024-04-22T23:35:05.877 INFO:tasks.workunit.client.0.smithi107.stdout:6/457: symlink db/d20/d2c/d3b/l85 0 2024-04-22T23:35:05.877 INFO:tasks.workunit.client.0.smithi107.stdout:6/458: chown db/d20/d2a/d5f/c83 0 1 2024-04-22T23:35:05.880 INFO:tasks.workunit.client.0.smithi107.stdout:1/420: creat d7/d19/d20/d39/d53/d43/d46/d5c/f8a x:0 0 0 2024-04-22T23:35:05.880 INFO:tasks.workunit.client.0.smithi107.stdout:8/387: dwrite d0/d7/d8/f1e [0,4194304] 0 2024-04-22T23:35:05.881 INFO:tasks.workunit.client.0.smithi107.stdout:0/421: truncate d1/d27/d20/d2c/d41/f58 1488057 0 2024-04-22T23:35:05.883 INFO:tasks.workunit.client.1.smithi157.stdout:9/632: dwrite d4/d2d/fb4 [0,4194304] 0 2024-04-22T23:35:05.883 INFO:tasks.workunit.client.1.smithi157.stdout:9/633: fdatasync d4/df/d61/f84 0 2024-04-22T23:35:05.884 INFO:tasks.workunit.client.0.smithi107.stdout:5/433: mkdir d9/db/d92 0 2024-04-22T23:35:05.884 INFO:tasks.workunit.client.0.smithi107.stdout:5/434: fdatasync d9/d26/f38 0 2024-04-22T23:35:05.885 INFO:tasks.workunit.client.1.smithi157.stdout:9/634: mkdir d4/df/d57/dbd/dc6 0 2024-04-22T23:35:05.886 INFO:tasks.workunit.client.0.smithi107.stdout:6/459: symlink db/d20/d2a/l86 0 2024-04-22T23:35:05.886 INFO:tasks.workunit.client.1.smithi157.stdout:7/507: dwrite d0/d9/d42/d55/f8f [0,4194304] 0 2024-04-22T23:35:05.887 INFO:tasks.workunit.client.0.smithi107.stdout:1/421: truncate d7/d19/d20/d39/d53/f33 3743793 0 2024-04-22T23:35:05.887 INFO:tasks.workunit.client.1.smithi157.stdout:4/690: dwrite d3/d11/d1d/d73/d80/f89 [0,4194304] 0 2024-04-22T23:35:05.888 INFO:tasks.workunit.client.0.smithi107.stdout:7/372: dwrite d7/d2d/f41 [0,4194304] 0 2024-04-22T23:35:05.888 INFO:tasks.workunit.client.0.smithi107.stdout:7/373: write d7/d1b/d51/d6e/f70 [392377,45234] 0 2024-04-22T23:35:05.889 INFO:tasks.workunit.client.0.smithi107.stdout:2/467: dwrite d1/d12/f26 [0,4194304] 0 2024-04-22T23:35:05.889 INFO:tasks.workunit.client.0.smithi107.stdout:2/468: fdatasync d1/d12/d35/d48/d13/d4f/f53 0 2024-04-22T23:35:05.889 INFO:tasks.workunit.client.0.smithi107.stdout:2/469: chown d1/d12/d35/d48/f3a 1 1 2024-04-22T23:35:05.889 INFO:tasks.workunit.client.0.smithi107.stdout:2/470: creat d1/d67/f9f x:0 0 0 2024-04-22T23:35:05.891 INFO:tasks.workunit.client.0.smithi107.stdout:8/388: symlink d0/d7/l7c 0 2024-04-22T23:35:05.891 INFO:tasks.workunit.client.0.smithi107.stdout:8/389: chown d0/d7/df/d17 678123063 1 2024-04-22T23:35:05.891 INFO:tasks.workunit.client.0.smithi107.stdout:1/422: write d7/d19/d2d/f3d [3698093,5454] 0 2024-04-22T23:35:05.892 INFO:tasks.workunit.client.1.smithi157.stdout:9/635: link d4/df/d57/c34 d4/df/cc7 0 2024-04-22T23:35:05.893 INFO:tasks.workunit.client.1.smithi157.stdout:9/636: chown d4/df/lab 855078253 1 2024-04-22T23:35:05.894 INFO:tasks.workunit.client.1.smithi157.stdout:5/638: write d1/d27/d29/d4b/f4d [1899839,34170] 0 2024-04-22T23:35:05.895 INFO:tasks.workunit.client.1.smithi157.stdout:4/691: symlink d3/d11/d34/da6/le9 0 2024-04-22T23:35:05.896 INFO:tasks.workunit.client.0.smithi107.stdout:5/435: symlink d9/db/de/l93 0 2024-04-22T23:35:05.896 INFO:tasks.workunit.client.1.smithi157.stdout:9/637: symlink d4/df/d57/lc8 0 2024-04-22T23:35:05.896 INFO:tasks.workunit.client.1.smithi157.stdout:9/638: stat d4/df/ce 0 2024-04-22T23:35:05.897 INFO:tasks.workunit.client.1.smithi157.stdout:9/639: write d4/df/f8b [248763,62765] 0 2024-04-22T23:35:05.897 INFO:tasks.workunit.client.1.smithi157.stdout:5/639: chown d1/d5/d7/la5 12 1 2024-04-22T23:35:05.897 INFO:tasks.workunit.client.1.smithi157.stdout:5/640: fdatasync d1/d5/d7/d2f/d9c/faa 0 2024-04-22T23:35:05.899 INFO:tasks.workunit.client.0.smithi107.stdout:6/460: mknod db/d20/d2a/d35/d69/d79/c87 0 2024-04-22T23:35:05.904 INFO:tasks.workunit.client.0.smithi107.stdout:5/436: write d9/d26/f2c [231120,84066] 0 2024-04-22T23:35:05.904 INFO:tasks.workunit.client.0.smithi107.stdout:5/437: write d9/db/f50 [679487,93073] 0 2024-04-22T23:35:05.910 INFO:tasks.workunit.client.0.smithi107.stdout:7/374: write d7/f11 [119658,43553] 0 2024-04-22T23:35:05.910 INFO:tasks.workunit.client.0.smithi107.stdout:7/375: chown d7/fd 46625102 1 2024-04-22T23:35:05.910 INFO:tasks.workunit.client.1.smithi157.stdout:9/640: write d4/d15/d23/d27/f38 [1021432,24903] 0 2024-04-22T23:35:05.911 INFO:tasks.workunit.client.0.smithi107.stdout:8/390: mknod d0/d3a/d46/d59/c7d 0 2024-04-22T23:35:05.914 INFO:tasks.workunit.client.1.smithi157.stdout:9/641: write d4/d15/d23/d3f/f42 [781382,118180] 0 2024-04-22T23:35:05.915 INFO:tasks.workunit.client.0.smithi107.stdout:1/423: mknod d7/d19/d20/d39/d53/d43/d46/d68/c8b 0 2024-04-22T23:35:05.916 INFO:tasks.workunit.client.1.smithi157.stdout:9/642: stat d4/d15/d55/d8d/c9e 0 2024-04-22T23:35:05.917 INFO:tasks.workunit.client.1.smithi157.stdout:7/508: dwrite d0/f46 [4194304,4194304] 0 2024-04-22T23:35:05.917 INFO:tasks.workunit.client.1.smithi157.stdout:9/643: dread d4/d15/d23/d27/f39 [0,4194304] 0 2024-04-22T23:35:05.921 INFO:tasks.workunit.client.1.smithi157.stdout:7/509: unlink d0/f8 0 2024-04-22T23:35:05.922 INFO:tasks.workunit.client.1.smithi157.stdout:7/510: mknod d0/d9/d72/ca2 0 2024-04-22T23:35:05.922 INFO:tasks.workunit.client.1.smithi157.stdout:7/511: stat d0/d9/d42/d55/c73 0 2024-04-22T23:35:05.923 INFO:tasks.workunit.client.1.smithi157.stdout:7/512: mkdir d0/d9/db/da3 0 2024-04-22T23:35:05.923 INFO:tasks.workunit.client.1.smithi157.stdout:7/513: write d0/d9/d29/d33/d37/f87 [1063059,65251] 0 2024-04-22T23:35:05.923 INFO:tasks.workunit.client.0.smithi107.stdout:6/461: link db/d20/d49/d7b/c80 db/d20/d2c/d3b/c88 0 2024-04-22T23:35:05.923 INFO:tasks.workunit.client.0.smithi107.stdout:6/462: fsync db/d20/d2c/d3b/f45 0 2024-04-22T23:35:05.928 INFO:tasks.workunit.client.0.smithi107.stdout:9/491: sync 2024-04-22T23:35:05.930 INFO:tasks.workunit.client.1.smithi157.stdout:3/649: sync 2024-04-22T23:35:05.930 INFO:tasks.workunit.client.1.smithi157.stdout:8/446: sync 2024-04-22T23:35:05.931 INFO:tasks.workunit.client.1.smithi157.stdout:4/692: dwrite d3/d4b/f3e [4194304,4194304] 0 2024-04-22T23:35:05.931 INFO:tasks.workunit.client.0.smithi107.stdout:3/495: dwrite db/d21/d27/d15/d29/d2a/d40/f6c [0,4194304] 0 2024-04-22T23:35:05.931 INFO:tasks.workunit.client.0.smithi107.stdout:1/424: dread d7/d19/d20/f38 [0,4194304] 0 2024-04-22T23:35:05.933 INFO:tasks.workunit.client.1.smithi157.stdout:4/693: mknod d3/d11/d1d/d73/da3/d31/d91/cea 0 2024-04-22T23:35:05.934 INFO:tasks.workunit.client.1.smithi157.stdout:7/514: truncate d0/f46 3567461 0 2024-04-22T23:35:05.934 INFO:tasks.workunit.client.1.smithi157.stdout:4/694: rmdir d3/d11/d1d/d73/d80 39 2024-04-22T23:35:05.935 INFO:tasks.workunit.client.1.smithi157.stdout:5/641: dwrite d1/d5/dc/d5d/fd3 [0,4194304] 0 2024-04-22T23:35:05.936 INFO:tasks.workunit.client.0.smithi107.stdout:2/471: dwrite d1/d12/d35/d48/d13/d4f/f5b [0,4194304] 0 2024-04-22T23:35:05.936 INFO:tasks.workunit.client.0.smithi107.stdout:2/472: creat d1/d2b/d3d/d57/fa0 x:0 0 0 2024-04-22T23:35:05.936 INFO:tasks.workunit.client.1.smithi157.stdout:7/515: rename d0/d9/d10/f5d to d0/d9/d29/d49/d4a/fa4 0 2024-04-22T23:35:05.936 INFO:tasks.workunit.client.1.smithi157.stdout:7/516: readlink d0/d9/d10/l96 0 2024-04-22T23:35:05.937 INFO:tasks.workunit.client.1.smithi157.stdout:3/650: write d0/d3/d34/db9/f3f [3693784,48422] 0 2024-04-22T23:35:05.938 INFO:tasks.workunit.client.1.smithi157.stdout:3/651: chown d0/d3/l2c 13 1 2024-04-22T23:35:05.938 INFO:tasks.workunit.client.1.smithi157.stdout:4/695: link d3/d11/f1f d3/d11/d1d/feb 0 2024-04-22T23:35:05.939 INFO:tasks.workunit.client.1.smithi157.stdout:8/447: dread d2/d6/d21/d5f/d11/d13/f6c [0,4194304] 0 2024-04-22T23:35:05.939 INFO:tasks.workunit.client.1.smithi157.stdout:8/448: write d2/d6/d21/d5f/d11/f4b [4708980,8856] 0 2024-04-22T23:35:05.939 INFO:tasks.workunit.client.1.smithi157.stdout:8/449: write d2/d34/d6a/f76 [1142108,120339] 0 2024-04-22T23:35:05.944 INFO:tasks.workunit.client.1.smithi157.stdout:3/652: mkdir d0/d3/d34/d62/d9a/d9e/dac/dcf 0 2024-04-22T23:35:05.946 INFO:tasks.workunit.client.1.smithi157.stdout:4/696: link d3/d4b/d7a/db7/ca0 d3/d11/d1d/daf/cec 0 2024-04-22T23:35:05.948 INFO:tasks.workunit.client.1.smithi157.stdout:7/517: rmdir d0/d9/d42/d55 39 2024-04-22T23:35:05.948 INFO:tasks.workunit.client.1.smithi157.stdout:5/642: write d1/d5/d7/d2f/d45/d54/f7b [268268,59178] 0 2024-04-22T23:35:05.949 INFO:tasks.workunit.client.1.smithi157.stdout:3/653: truncate d0/f1c 3769966 0 2024-04-22T23:35:05.951 INFO:tasks.workunit.client.1.smithi157.stdout:7/518: symlink d0/d9/db/d35/la5 0 2024-04-22T23:35:05.953 INFO:tasks.workunit.client.1.smithi157.stdout:5/643: creat d1/d5/d7/dbd/ff4 x:0 0 0 2024-04-22T23:35:05.955 INFO:tasks.workunit.client.1.smithi157.stdout:3/654: link d0/d3/d34/d3d/d57/d60/l86 d0/d3/d34/db9/d9f/ld0 0 2024-04-22T23:35:05.955 INFO:tasks.workunit.client.1.smithi157.stdout:3/655: creat d0/d3/d13/fd1 x:0 0 0 2024-04-22T23:35:05.957 INFO:tasks.workunit.client.1.smithi157.stdout:7/519: mkdir d0/d9/d29/d5e/da6 0 2024-04-22T23:35:05.957 INFO:tasks.workunit.client.1.smithi157.stdout:7/520: fdatasync d0/d9/f1d 0 2024-04-22T23:35:05.959 INFO:tasks.workunit.client.1.smithi157.stdout:3/656: symlink d0/d3/d34/d62/d9a/d9e/dac/dcf/ld2 0 2024-04-22T23:35:05.960 INFO:tasks.workunit.client.0.smithi107.stdout:1/425: dwrite d7/d19/d71/f78 [0,4194304] 0 2024-04-22T23:35:05.960 INFO:tasks.workunit.client.0.smithi107.stdout:1/426: chown d7/d19/d54 15174 1 2024-04-22T23:35:05.960 INFO:tasks.workunit.client.0.smithi107.stdout:1/427: truncate d7/d19/d2d/f31 4400036 0 2024-04-22T23:35:05.960 INFO:tasks.workunit.client.0.smithi107.stdout:1/428: chown d7/d19/d20/d39/d53/d43/f7a 2582873 1 2024-04-22T23:35:05.961 INFO:tasks.workunit.client.1.smithi157.stdout:7/521: creat d0/d9/d42/d55/d80/fa7 x:0 0 0 2024-04-22T23:35:05.962 INFO:tasks.workunit.client.1.smithi157.stdout:3/657: rename d0/d7/d6c/l7e to d0/d3/d34/d3d/d57/d60/d9c/ld3 0 2024-04-22T23:35:05.963 INFO:tasks.workunit.client.0.smithi107.stdout:2/473: dwrite d1/f6 [0,4194304] 0 2024-04-22T23:35:05.963 INFO:tasks.workunit.client.1.smithi157.stdout:8/450: dwrite d2/d34/d6a/d77/d42/f61 [0,4194304] 0 2024-04-22T23:35:05.966 INFO:tasks.workunit.client.1.smithi157.stdout:7/522: rename d0/c48 to d0/d9/db/df/ca8 0 2024-04-22T23:35:05.968 INFO:tasks.workunit.client.1.smithi157.stdout:3/658: creat d0/d3/d34/db9/fd4 x:0 0 0 2024-04-22T23:35:05.970 INFO:tasks.workunit.client.1.smithi157.stdout:8/451: creat d2/d34/d6a/d77/f9f x:0 0 0 2024-04-22T23:35:05.971 INFO:tasks.workunit.client.1.smithi157.stdout:5/644: dwrite d1/d5/dc/f62 [0,4194304] 0 2024-04-22T23:35:05.973 INFO:tasks.workunit.client.1.smithi157.stdout:7/523: unlink d0/d9/de/c1e 0 2024-04-22T23:35:05.973 INFO:tasks.workunit.client.1.smithi157.stdout:7/524: fdatasync d0/d9/db/f39 0 2024-04-22T23:35:05.975 INFO:tasks.workunit.client.1.smithi157.stdout:3/659: unlink d0/d7/d6c/f6f 0 2024-04-22T23:35:05.977 INFO:tasks.workunit.client.0.smithi107.stdout:7/376: mknod d7/d21/d48/d5c/c78 0 2024-04-22T23:35:05.979 INFO:tasks.workunit.client.0.smithi107.stdout:8/391: symlink d0/d7/d8/d41/l7e 0 2024-04-22T23:35:05.979 INFO:tasks.workunit.client.1.smithi157.stdout:5/645: getdents d1/db0 0 2024-04-22T23:35:05.980 INFO:tasks.workunit.client.1.smithi157.stdout:3/660: mkdir d0/d3/d34/d3d/d57/d60/d9c/dd5 0 2024-04-22T23:35:05.981 INFO:tasks.workunit.client.0.smithi107.stdout:0/422: rmdir d1 39 2024-04-22T23:35:05.982 INFO:tasks.workunit.client.1.smithi157.stdout:3/661: truncate d0/d3/f35 593015 0 2024-04-22T23:35:05.983 INFO:tasks.workunit.client.1.smithi157.stdout:3/662: mknod d0/d3/d34/d62/d9a/d9e/dac/cd6 0 2024-04-22T23:35:05.984 INFO:tasks.workunit.client.1.smithi157.stdout:3/663: dread - d0/d3/d34/d62/fae zero size 2024-04-22T23:35:05.984 INFO:tasks.workunit.client.1.smithi157.stdout:3/664: readlink d0/d3/d34/d4a/l5a 0 2024-04-22T23:35:05.984 INFO:tasks.workunit.client.1.smithi157.stdout:3/665: creat d0/d3/d34/d4a/fd7 x:0 0 0 2024-04-22T23:35:05.984 INFO:tasks.workunit.client.0.smithi107.stdout:6/463: symlink db/d20/d2a/d35/d69/d79/l89 0 2024-04-22T23:35:05.984 INFO:tasks.workunit.client.0.smithi107.stdout:6/464: stat db/c10 0 2024-04-22T23:35:05.986 INFO:tasks.workunit.client.0.smithi107.stdout:9/492: creat d5/d7/d27/d6f/fae x:0 0 0 2024-04-22T23:35:05.986 INFO:tasks.workunit.client.0.smithi107.stdout:9/493: truncate d5/f11 2313557 0 2024-04-22T23:35:05.989 INFO:tasks.workunit.client.1.smithi157.stdout:7/525: dwrite d0/f45 [0,4194304] 0 2024-04-22T23:35:05.989 INFO:tasks.workunit.client.1.smithi157.stdout:5/646: read d1/d27/d29/f58 [156973,78644] 0 2024-04-22T23:35:05.990 INFO:tasks.workunit.client.1.smithi157.stdout:5/647: stat d1/d27/d29/fdd 0 2024-04-22T23:35:05.990 INFO:tasks.workunit.client.1.smithi157.stdout:5/648: creat d1/d5/dc/d43/d4f/ff5 x:0 0 0 2024-04-22T23:35:05.990 INFO:tasks.workunit.client.1.smithi157.stdout:5/649: creat d1/d1f/dc6/ff6 x:0 0 0 2024-04-22T23:35:05.991 INFO:tasks.workunit.client.1.smithi157.stdout:8/452: dwrite d2/d4/d73/f99 [0,4194304] 0 2024-04-22T23:35:05.994 INFO:tasks.workunit.client.0.smithi107.stdout:3/496: symlink db/d21/d27/d15/d24/d52/lad 0 2024-04-22T23:35:05.994 INFO:tasks.workunit.client.0.smithi107.stdout:3/497: stat db/d21/d27/d54 0 2024-04-22T23:35:05.994 INFO:tasks.workunit.client.0.smithi107.stdout:3/498: chown db/d21/d27/d54/d25/c84 4060369 1 2024-04-22T23:35:06.000 INFO:tasks.workunit.client.1.smithi157.stdout:3/666: creat d0/d3/d34/d62/d9a/d9e/fd8 x:0 0 0 2024-04-22T23:35:06.000 INFO:tasks.workunit.client.1.smithi157.stdout:7/526: read d0/f45 [2476989,76471] 0 2024-04-22T23:35:06.000 INFO:tasks.workunit.client.1.smithi157.stdout:7/527: read d0/d9/d29/d33/d37/f50 [593130,73746] 0 2024-04-22T23:35:06.001 INFO:tasks.workunit.client.1.smithi157.stdout:7/528: truncate d0/d9/d29/d33/d37/f87 1880005 0 2024-04-22T23:35:06.003 INFO:tasks.workunit.client.1.smithi157.stdout:7/529: write d0/d9/db/df/d68/f8e [767127,18175] 0 2024-04-22T23:35:06.003 INFO:tasks.workunit.client.1.smithi157.stdout:7/530: write d0/f22 [10831727,22129] 0 2024-04-22T23:35:06.008 INFO:tasks.workunit.client.0.smithi107.stdout:1/429: rename d7/c12 to d7/d19/c8c 0 2024-04-22T23:35:06.012 INFO:tasks.workunit.client.1.smithi157.stdout:5/650: creat d1/d5/dc/d43/d4f/d7f/ff7 x:0 0 0 2024-04-22T23:35:06.014 INFO:tasks.workunit.client.0.smithi107.stdout:3/499: dwrite db/d21/d27/d15/d24/d66/da5/d8f/fa0 [0,4194304] 0 2024-04-22T23:35:06.015 INFO:tasks.workunit.client.0.smithi107.stdout:7/377: symlink d7/d2d/d66/l79 0 2024-04-22T23:35:06.015 INFO:tasks.workunit.client.0.smithi107.stdout:7/378: creat d7/d21/d48/d5c/f7a x:0 0 0 2024-04-22T23:35:06.016 INFO:tasks.workunit.client.0.smithi107.stdout:6/465: dwrite db/d20/d2a/d38/f56 [0,4194304] 0 2024-04-22T23:35:06.017 INFO:tasks.workunit.client.0.smithi107.stdout:8/392: symlink d0/d7/df/d33/d37/l7f 0 2024-04-22T23:35:06.018 INFO:tasks.workunit.client.1.smithi157.stdout:7/531: dread d0/d9/d29/d5e/f65 [0,4194304] 0 2024-04-22T23:35:06.019 INFO:tasks.workunit.client.0.smithi107.stdout:0/423: link d1/d27/l72 d1/d9/d14/l94 0 2024-04-22T23:35:06.019 INFO:tasks.workunit.client.0.smithi107.stdout:0/424: fdatasync d1/d84/f6c 0 2024-04-22T23:35:06.019 INFO:tasks.workunit.client.0.smithi107.stdout:0/425: chown d1/d2a/f77 17568 1 2024-04-22T23:35:06.021 INFO:tasks.workunit.client.0.smithi107.stdout:9/494: mkdir d5/d7/d27/d6f/daf 0 2024-04-22T23:35:06.022 INFO:tasks.workunit.client.0.smithi107.stdout:2/474: dread d1/d12/f14 [0,4194304] 0 2024-04-22T23:35:06.025 INFO:tasks.workunit.client.0.smithi107.stdout:8/393: read d0/d7/d20/f39 [756143,43460] 0 2024-04-22T23:35:06.027 INFO:tasks.workunit.client.0.smithi107.stdout:3/500: creat db/d21/d27/d15/d29/d2a/fae x:0 0 0 2024-04-22T23:35:06.027 INFO:tasks.workunit.client.0.smithi107.stdout:3/501: creat db/d21/d27/d15/d24/d87/faf x:0 0 0 2024-04-22T23:35:06.028 INFO:tasks.workunit.client.1.smithi157.stdout:8/453: symlink d2/d6/d66/d97/la0 0 2024-04-22T23:35:06.031 INFO:tasks.workunit.client.0.smithi107.stdout:1/430: dwrite d7/d19/d20/f52 [0,4194304] 0 2024-04-22T23:35:06.032 INFO:tasks.workunit.client.0.smithi107.stdout:7/379: dread d7/f13 [4194304,4194304] 0 2024-04-22T23:35:06.035 INFO:tasks.workunit.client.0.smithi107.stdout:0/426: rename d1/d27/d20/l74 to d1/d9/d14/d4f/d59/l95 0 2024-04-22T23:35:06.035 INFO:tasks.workunit.client.0.smithi107.stdout:9/495: rename d5/d14/d2e to d5/d14/d2e/db0 22 2024-04-22T23:35:06.037 INFO:tasks.workunit.client.0.smithi107.stdout:2/475: link d1/d12/d35/d48/d13/f68 d1/d63/fa1 0 2024-04-22T23:35:06.037 INFO:tasks.workunit.client.0.smithi107.stdout:2/476: chown d1/d2b/d49/f50 925491130 1 2024-04-22T23:35:06.042 INFO:tasks.workunit.client.1.smithi157.stdout:7/532: dwrite d0/d9/db/f39 [0,4194304] 0 2024-04-22T23:35:06.042 INFO:tasks.workunit.client.1.smithi157.stdout:7/533: chown d0/d9/d29/d49 1018 1 2024-04-22T23:35:06.042 INFO:tasks.workunit.client.1.smithi157.stdout:7/534: fdatasync d0/d9/d29/d5e/f6a 0 2024-04-22T23:35:06.043 INFO:tasks.workunit.client.0.smithi107.stdout:1/431: symlink d7/d19/d20/d39/d53/d43/d46/d68/l8d 0 2024-04-22T23:35:06.043 INFO:tasks.workunit.client.0.smithi107.stdout:2/477: dread d1/f78 [0,4194304] 0 2024-04-22T23:35:06.043 INFO:tasks.workunit.client.0.smithi107.stdout:2/478: stat d1/d2b/d3d/f6c 0 2024-04-22T23:35:06.045 INFO:tasks.workunit.client.0.smithi107.stdout:7/380: symlink d7/d21/l7b 0 2024-04-22T23:35:06.047 INFO:tasks.workunit.client.0.smithi107.stdout:0/427: write d1/d27/d20/d2c/d2e/f39 [1323979,33278] 0 2024-04-22T23:35:06.047 INFO:tasks.workunit.client.0.smithi107.stdout:0/428: readlink d1/d9/d14/l79 0 2024-04-22T23:35:06.048 INFO:tasks.workunit.client.1.smithi157.stdout:3/667: symlink d0/d3/d34/d3d/d57/d60/d6b/d97/dcb/ld9 0 2024-04-22T23:35:06.049 INFO:tasks.workunit.client.0.smithi107.stdout:8/394: rename d0/f5a to d0/d7/df/d33/d37/f80 0 2024-04-22T23:35:06.049 INFO:tasks.workunit.client.0.smithi107.stdout:8/395: readlink d0/d7/df/d18/l54 0 2024-04-22T23:35:06.049 INFO:tasks.workunit.client.1.smithi157.stdout:5/651: symlink d1/d1f/d22/lf8 0 2024-04-22T23:35:06.050 INFO:tasks.workunit.client.0.smithi107.stdout:6/466: dwrite db/d20/d2c/f32 [0,4194304] 0 2024-04-22T23:35:06.050 INFO:tasks.workunit.client.0.smithi107.stdout:9/496: read d5/d7/fd [2802599,27844] 0 2024-04-22T23:35:06.051 INFO:tasks.workunit.client.1.smithi157.stdout:8/454: link d2/d6/f37 d2/d6/d66/d97/fa1 0 2024-04-22T23:35:06.052 INFO:tasks.workunit.client.0.smithi107.stdout:3/502: getdents db/d21/d27/d36 0 2024-04-22T23:35:06.052 INFO:tasks.workunit.client.0.smithi107.stdout:3/503: chown db/d21/d27/d54/d25/f34 114492699 1 2024-04-22T23:35:06.056 INFO:tasks.workunit.client.1.smithi157.stdout:3/668: unlink d0/d7/l38 0 2024-04-22T23:35:06.056 INFO:tasks.workunit.client.1.smithi157.stdout:7/535: mkdir d0/d9/d29/d5e/da9 0 2024-04-22T23:35:06.057 INFO:tasks.workunit.client.0.smithi107.stdout:1/432: rmdir d7/d19/d20/d39/d53/d43/d61 0 2024-04-22T23:35:06.057 INFO:tasks.workunit.client.1.smithi157.stdout:5/652: symlink d1/db0/d7a/lf9 0 2024-04-22T23:35:06.059 INFO:tasks.workunit.client.0.smithi107.stdout:0/429: mknod d1/d9/d14/d4f/c96 0 2024-04-22T23:35:06.060 INFO:tasks.workunit.client.0.smithi107.stdout:7/381: dread d7/f3b [0,4194304] 0 2024-04-22T23:35:06.061 INFO:tasks.workunit.client.0.smithi107.stdout:5/438: sync 2024-04-22T23:35:06.061 INFO:tasks.workunit.client.0.smithi107.stdout:4/455: sync 2024-04-22T23:35:06.061 INFO:tasks.workunit.client.0.smithi107.stdout:4/456: chown d1/d8/l62 716 1 2024-04-22T23:35:06.061 INFO:tasks.workunit.client.1.smithi157.stdout:8/455: link d2/d34/d6a/d77/d42/f61 d2/d6/fa2 0 2024-04-22T23:35:06.061 INFO:tasks.workunit.client.1.smithi157.stdout:3/669: rename d0/d3/f51 to d0/d3/d34/d3d/d5b/fda 0 2024-04-22T23:35:06.062 INFO:tasks.workunit.client.1.smithi157.stdout:8/456: fdatasync d2/d6/d21/d5f/d17/f8e 0 2024-04-22T23:35:06.062 INFO:tasks.workunit.client.1.smithi157.stdout:5/653: read d1/d27/d29/fbf [157627,94071] 0 2024-04-22T23:35:06.063 INFO:tasks.workunit.client.1.smithi157.stdout:7/536: truncate d0/f7 1619732 0 2024-04-22T23:35:06.064 INFO:tasks.workunit.client.0.smithi107.stdout:2/479: rename d1/d12/f89 to d1/d2b/d3d/d57/fa2 0 2024-04-22T23:35:06.066 INFO:tasks.workunit.client.1.smithi157.stdout:8/457: unlink d2/l1f 0 2024-04-22T23:35:06.067 INFO:tasks.workunit.client.0.smithi107.stdout:2/480: dread d1/d2b/d3d/d57/f5c [0,4194304] 0 2024-04-22T23:35:06.070 INFO:tasks.workunit.client.1.smithi157.stdout:5/654: link d1/d5/d7/d2f/d45/l6c d1/d5/d7/d2f/d45/lfa 0 2024-04-22T23:35:06.070 INFO:tasks.workunit.client.1.smithi157.stdout:5/655: dread - d1/d1f/dc6/ff6 zero size 2024-04-22T23:35:06.071 INFO:tasks.workunit.client.0.smithi107.stdout:0/430: truncate d1/d27/f57 1837451 0 2024-04-22T23:35:06.072 INFO:tasks.workunit.client.0.smithi107.stdout:7/382: rename d7/d1b/c3e to d7/d1b/c7c 0 2024-04-22T23:35:06.072 INFO:tasks.workunit.client.1.smithi157.stdout:8/458: rename d2/d6/d21/d5f/d17/l6e to d2/la3 0 2024-04-22T23:35:06.073 INFO:tasks.workunit.client.0.smithi107.stdout:5/439: mkdir d9/db/d94 0 2024-04-22T23:35:06.073 INFO:tasks.workunit.client.0.smithi107.stdout:8/396: getdents d0/d3a/d46 0 2024-04-22T23:35:06.074 INFO:tasks.workunit.client.0.smithi107.stdout:6/467: rmdir db/d20/d2a/d38/d4b 39 2024-04-22T23:35:06.075 INFO:tasks.workunit.client.0.smithi107.stdout:9/497: getdents d5/d6c 0 2024-04-22T23:35:06.075 INFO:tasks.workunit.client.0.smithi107.stdout:9/498: readlink d5/la 0 2024-04-22T23:35:06.076 INFO:tasks.workunit.client.1.smithi157.stdout:7/537: write d0/d9/d42/d55/f8f [1821842,62450] 0 2024-04-22T23:35:06.077 INFO:tasks.workunit.client.1.smithi157.stdout:7/538: fdatasync d0/f45 0 2024-04-22T23:35:06.077 INFO:tasks.workunit.client.1.smithi157.stdout:8/459: symlink d2/d34/d6a/la4 0 2024-04-22T23:35:06.077 INFO:tasks.workunit.client.1.smithi157.stdout:5/656: mkdir d1/d27/d29/d4b/dfb 0 2024-04-22T23:35:06.077 INFO:tasks.workunit.client.1.smithi157.stdout:5/657: chown d1/d1f/c73 13550 1 2024-04-22T23:35:06.077 INFO:tasks.workunit.client.1.smithi157.stdout:7/539: rmdir d0/d9/d72 39 2024-04-22T23:35:06.078 INFO:tasks.workunit.client.0.smithi107.stdout:3/504: link db/d21/d27/d15/d24/d66/da5/d8f/fa0 db/d21/d27/d36/fb0 0 2024-04-22T23:35:06.078 INFO:tasks.workunit.client.0.smithi107.stdout:8/397: dread d0/d7/d20/f39 [0,4194304] 0 2024-04-22T23:35:06.079 INFO:tasks.workunit.client.0.smithi107.stdout:1/433: getdents d7/d19/d20/d39 0 2024-04-22T23:35:06.079 INFO:tasks.workunit.client.1.smithi157.stdout:8/460: creat d2/d34/d6a/d77/d42/d58/fa5 x:0 0 0 2024-04-22T23:35:06.080 INFO:tasks.workunit.client.0.smithi107.stdout:0/431: symlink d1/d67/l97 0 2024-04-22T23:35:06.080 INFO:tasks.workunit.client.1.smithi157.stdout:5/658: creat d1/db0/ffc x:0 0 0 2024-04-22T23:35:06.081 INFO:tasks.workunit.client.1.smithi157.stdout:5/659: creat d1/d5/dc/d43/d5e/ffd x:0 0 0 2024-04-22T23:35:06.081 INFO:tasks.workunit.client.0.smithi107.stdout:7/383: symlink d7/d1b/l7d 0 2024-04-22T23:35:06.082 INFO:tasks.workunit.client.0.smithi107.stdout:5/440: symlink d9/db/d94/l95 0 2024-04-22T23:35:06.082 INFO:tasks.workunit.client.0.smithi107.stdout:6/468: rmdir db/d20/d2a/d5d 39 2024-04-22T23:35:06.084 INFO:tasks.workunit.client.0.smithi107.stdout:4/457: dwrite d1/d1b/d44/f95 [0,4194304] 0 2024-04-22T23:35:06.085 INFO:tasks.workunit.client.1.smithi157.stdout:3/670: dwrite d0/d3/d34/d39/f4c [0,4194304] 0 2024-04-22T23:35:06.086 INFO:tasks.workunit.client.1.smithi157.stdout:5/660: link d1/d5/d7/la5 d1/d5/d7/d2f/d45/d54/lfe 0 2024-04-22T23:35:06.091 INFO:tasks.workunit.client.1.smithi157.stdout:5/661: readlink d1/d5/d7/d2f/d51/l98 0 2024-04-22T23:35:06.093 INFO:tasks.workunit.client.1.smithi157.stdout:5/662: mkdir d1/db0/dff 0 2024-04-22T23:35:06.094 INFO:tasks.workunit.client.0.smithi107.stdout:8/398: creat d0/f81 x:0 0 0 2024-04-22T23:35:06.095 INFO:tasks.workunit.client.0.smithi107.stdout:4/458: write d1/d21/f36 [3452761,18986] 0 2024-04-22T23:35:06.095 INFO:tasks.workunit.client.0.smithi107.stdout:7/384: mkdir d7/d21/d7e 0 2024-04-22T23:35:06.096 INFO:tasks.workunit.client.0.smithi107.stdout:5/441: mkdir d9/d26/d96 0 2024-04-22T23:35:06.098 INFO:tasks.workunit.client.1.smithi157.stdout:5/663: getdents d1/d5/d7/d2f/d45/da4 0 2024-04-22T23:35:06.098 INFO:tasks.workunit.client.0.smithi107.stdout:2/481: rmdir d1/d12 39 2024-04-22T23:35:06.098 INFO:tasks.workunit.client.0.smithi107.stdout:2/482: creat d1/d2b/d49/fa3 x:0 0 0 2024-04-22T23:35:06.098 INFO:tasks.workunit.client.0.smithi107.stdout:1/434: write d7/d19/d20/d39/f69 [240066,31215] 0 2024-04-22T23:35:06.099 INFO:tasks.workunit.client.0.smithi107.stdout:1/435: dread - d7/d19/d20/d39/d53/d43/d63/f83 zero size 2024-04-22T23:35:06.102 INFO:tasks.workunit.client.0.smithi107.stdout:8/399: creat d0/d7/df/d18/d51/f82 x:0 0 0 2024-04-22T23:35:06.102 INFO:tasks.workunit.client.1.smithi157.stdout:5/664: rename d1/db0/f85 to d1/db0/df3/f100 0 2024-04-22T23:35:06.102 INFO:tasks.workunit.client.1.smithi157.stdout:5/665: readlink d1/db0/l75 0 2024-04-22T23:35:06.102 INFO:tasks.workunit.client.1.smithi157.stdout:1/682: sync 2024-04-22T23:35:06.102 INFO:tasks.workunit.client.1.smithi157.stdout:0/589: sync 2024-04-22T23:35:06.104 INFO:tasks.workunit.client.1.smithi157.stdout:5/666: creat d1/db0/dff/f101 x:0 0 0 2024-04-22T23:35:06.104 INFO:tasks.workunit.client.0.smithi107.stdout:4/459: creat d1/d7/d63/d78/d96/fa6 x:0 0 0 2024-04-22T23:35:06.104 INFO:tasks.workunit.client.0.smithi107.stdout:4/460: read d1/d21/d52/f70 [1051369,127729] 0 2024-04-22T23:35:06.105 INFO:tasks.workunit.client.1.smithi157.stdout:1/683: mknod d4/d17/d34/d3b/cee 0 2024-04-22T23:35:06.106 INFO:tasks.workunit.client.1.smithi157.stdout:1/684: readlink d4/d17/d34/d3b/l60 0 2024-04-22T23:35:06.106 INFO:tasks.workunit.client.1.smithi157.stdout:1/685: write d4/db/f13 [4291014,47097] 0 2024-04-22T23:35:06.107 INFO:tasks.workunit.client.1.smithi157.stdout:5/667: mkdir d1/db0/dff/d102 0 2024-04-22T23:35:06.109 INFO:tasks.workunit.client.0.smithi107.stdout:2/483: creat d1/d12/fa4 x:0 0 0 2024-04-22T23:35:06.110 INFO:tasks.workunit.client.1.smithi157.stdout:5/668: unlink d1/d5/d7/d2f/f79 0 2024-04-22T23:35:06.110 INFO:tasks.workunit.client.1.smithi157.stdout:5/669: chown d1/d5/d7/d10/f49 19639 1 2024-04-22T23:35:06.111 INFO:tasks.workunit.client.0.smithi107.stdout:1/436: mknod d7/d19/d20/c8e 0 2024-04-22T23:35:06.111 INFO:tasks.workunit.client.0.smithi107.stdout:1/437: fdatasync d7/d19/d20/d39/d53/f33 0 2024-04-22T23:35:06.113 INFO:tasks.workunit.client.0.smithi107.stdout:4/461: rmdir d1/d1b/d89 39 2024-04-22T23:35:06.114 INFO:tasks.workunit.client.1.smithi157.stdout:5/670: rmdir d1/d27/d29/d4b/dfb 0 2024-04-22T23:35:06.116 INFO:tasks.workunit.client.0.smithi107.stdout:2/484: mkdir d1/d12/d35/d48/d13/d4f/da5 0 2024-04-22T23:35:06.116 INFO:tasks.workunit.client.0.smithi107.stdout:2/485: truncate d1/f7c 160767 0 2024-04-22T23:35:06.118 INFO:tasks.workunit.client.1.smithi157.stdout:8/461: dwrite d2/d4/d30/f62 [0,4194304] 0 2024-04-22T23:35:06.118 INFO:tasks.workunit.client.1.smithi157.stdout:0/590: write d17/d2b/d52/d7e/fa6 [192118,8644] 0 2024-04-22T23:35:06.118 INFO:tasks.workunit.client.1.smithi157.stdout:0/591: chown d17/d2b/d52/l57 99341837 1 2024-04-22T23:35:06.119 INFO:tasks.workunit.client.1.smithi157.stdout:8/462: write d2/d6/d6b/f8b [580,55689] 0 2024-04-22T23:35:06.119 INFO:tasks.workunit.client.1.smithi157.stdout:0/592: stat d17/d23/d34/d67/d7f/f74 0 2024-04-22T23:35:06.119 INFO:tasks.workunit.client.1.smithi157.stdout:8/463: write d2/d6/d21/d5f/d17/f4e [1222496,107860] 0 2024-04-22T23:35:06.119 INFO:tasks.workunit.client.1.smithi157.stdout:8/464: readlink d2/d34/d6a/d77/d42/l3b 0 2024-04-22T23:35:06.120 INFO:tasks.workunit.client.0.smithi107.stdout:0/432: dwrite d1/f8f [0,4194304] 0 2024-04-22T23:35:06.120 INFO:tasks.workunit.client.0.smithi107.stdout:0/433: write d1/d27/f57 [2851232,29238] 0 2024-04-22T23:35:06.121 INFO:tasks.workunit.client.0.smithi107.stdout:2/486: symlink d1/d12/d35/d54/la6 0 2024-04-22T23:35:06.121 INFO:tasks.workunit.client.0.smithi107.stdout:2/487: write d1/d2b/d3d/d57/f97 [141622,83875] 0 2024-04-22T23:35:06.121 INFO:tasks.workunit.client.0.smithi107.stdout:2/488: read - d1/d12/f31 zero size 2024-04-22T23:35:06.122 INFO:tasks.workunit.client.0.smithi107.stdout:0/434: mknod d1/d27/c98 0 2024-04-22T23:35:06.122 INFO:tasks.workunit.client.0.smithi107.stdout:1/438: write d7/f10 [1448013,17593] 0 2024-04-22T23:35:06.123 INFO:tasks.workunit.client.0.smithi107.stdout:6/469: dwrite db/d20/d2c/d3b/f43 [0,4194304] 0 2024-04-22T23:35:06.123 INFO:tasks.workunit.client.0.smithi107.stdout:1/439: fsync d7/d19/d20/f52 0 2024-04-22T23:35:06.124 INFO:tasks.workunit.client.1.smithi157.stdout:0/593: mkdir d17/d23/d34/d67/d7f/d69/dce 0 2024-04-22T23:35:06.124 INFO:tasks.workunit.client.1.smithi157.stdout:0/594: chown d17/d2b/d52 3 1 2024-04-22T23:35:06.126 INFO:tasks.workunit.client.0.smithi107.stdout:2/489: symlink d1/d67/la7 0 2024-04-22T23:35:06.126 INFO:tasks.workunit.client.0.smithi107.stdout:2/490: truncate d1/d12/d35/d48/f3e 187546 0 2024-04-22T23:35:06.127 INFO:tasks.workunit.client.1.smithi157.stdout:8/465: rename d2/d4/c2a to d2/d4/ca6 0 2024-04-22T23:35:06.127 INFO:tasks.workunit.client.0.smithi107.stdout:5/442: rmdir d9/db 39 2024-04-22T23:35:06.129 INFO:tasks.workunit.client.1.smithi157.stdout:0/595: truncate d17/d2b/f66 1875388 0 2024-04-22T23:35:06.132 INFO:tasks.workunit.client.0.smithi107.stdout:1/440: unlink d7/d19/d20/l30 0 2024-04-22T23:35:06.132 INFO:tasks.workunit.client.1.smithi157.stdout:8/466: truncate d2/d4/f5b 1883744 0 2024-04-22T23:35:06.132 INFO:tasks.workunit.client.1.smithi157.stdout:8/467: write d2/d34/d6a/d77/d42/d58/fa5 [326799,63285] 0 2024-04-22T23:35:06.132 INFO:tasks.workunit.client.1.smithi157.stdout:5/671: write d1/d5/dc/d5d/d77/fad [1530027,5974] 0 2024-04-22T23:35:06.133 INFO:tasks.workunit.client.1.smithi157.stdout:8/468: read d2/d34/d6a/d77/d42/d58/fa5 [207831,76012] 0 2024-04-22T23:35:06.133 INFO:tasks.workunit.client.0.smithi107.stdout:5/443: mknod d9/db/d92/c97 0 2024-04-22T23:35:06.134 INFO:tasks.workunit.client.1.smithi157.stdout:5/672: mknod d1/d5/dc/d43/d4f/d7f/dab/c103 0 2024-04-22T23:35:06.134 INFO:tasks.workunit.client.1.smithi157.stdout:0/596: dread d17/d23/f7a [0,4194304] 0 2024-04-22T23:35:06.135 INFO:tasks.workunit.client.1.smithi157.stdout:5/673: mkdir d1/d5/d104 0 2024-04-22T23:35:06.135 INFO:tasks.workunit.client.1.smithi157.stdout:6/960: sync 2024-04-22T23:35:06.136 INFO:tasks.workunit.client.1.smithi157.stdout:2/765: sync 2024-04-22T23:35:06.136 INFO:tasks.workunit.client.1.smithi157.stdout:2/766: truncate d7/d8/d30/d5b/f72 1543204 0 2024-04-22T23:35:06.136 INFO:tasks.workunit.client.0.smithi107.stdout:1/441: creat d7/d19/d42/f8f x:0 0 0 2024-04-22T23:35:06.138 INFO:tasks.workunit.client.1.smithi157.stdout:6/961: creat d1/d38/daf/f13b x:0 0 0 2024-04-22T23:35:06.138 INFO:tasks.workunit.client.0.smithi107.stdout:5/444: symlink d9/db/d92/l98 0 2024-04-22T23:35:06.139 INFO:tasks.workunit.client.1.smithi157.stdout:5/674: mkdir d1/d5/d7/d2f/d45/d54/d105 0 2024-04-22T23:35:06.140 INFO:tasks.workunit.client.1.smithi157.stdout:1/686: dwrite d4/db/f47 [0,4194304] 0 2024-04-22T23:35:06.141 INFO:tasks.workunit.client.0.smithi107.stdout:5/445: creat d9/db/d94/f99 x:0 0 0 2024-04-22T23:35:06.142 INFO:tasks.workunit.client.0.smithi107.stdout:4/462: dwrite d1/d21/d85/f9c [0,4194304] 0 2024-04-22T23:35:06.142 INFO:tasks.workunit.client.1.smithi157.stdout:2/767: rename d7/d8/d30/d5b/d8a/ccd to d7/d1f/d26/c100 0 2024-04-22T23:35:06.144 INFO:tasks.workunit.client.0.smithi107.stdout:2/491: getdents d1/d12 0 2024-04-22T23:35:06.144 INFO:tasks.workunit.client.0.smithi107.stdout:8/400: dwrite d0/d7/df/d33/d37/f49 [0,4194304] 0 2024-04-22T23:35:06.144 INFO:tasks.workunit.client.0.smithi107.stdout:8/401: write d0/d3a/f4a [260516,98305] 0 2024-04-22T23:35:06.144 INFO:tasks.workunit.client.0.smithi107.stdout:1/442: getdents d7/d19/d20/d39/d53/d43/d46/d5c 0 2024-04-22T23:35:06.144 INFO:tasks.workunit.client.1.smithi157.stdout:0/597: write d17/d23/f32 [2548960,74733] 0 2024-04-22T23:35:06.144 INFO:tasks.workunit.client.1.smithi157.stdout:0/598: write d17/d23/d34/d67/d7f/f74 [471162,71858] 0 2024-04-22T23:35:06.145 INFO:tasks.workunit.client.1.smithi157.stdout:6/962: mknod d1/d38/daf/c13c 0 2024-04-22T23:35:06.146 INFO:tasks.workunit.client.0.smithi107.stdout:5/446: creat d9/db/de/f9a x:0 0 0 2024-04-22T23:35:06.147 INFO:tasks.workunit.client.1.smithi157.stdout:5/675: creat d1/db0/dff/d102/f106 x:0 0 0 2024-04-22T23:35:06.150 INFO:tasks.workunit.client.0.smithi107.stdout:2/492: getdents d1/d12/d35 0 2024-04-22T23:35:06.151 INFO:tasks.workunit.client.1.smithi157.stdout:1/687: getdents d4/db/d71 0 2024-04-22T23:35:06.152 INFO:tasks.workunit.client.0.smithi107.stdout:0/435: dwrite d1/d27/d20/d2c/d2e/f39 [0,4194304] 0 2024-04-22T23:35:06.154 INFO:tasks.workunit.client.1.smithi157.stdout:2/768: creat d7/d8/f101 x:0 0 0 2024-04-22T23:35:06.156 INFO:tasks.workunit.client.1.smithi157.stdout:0/599: creat d17/d2b/d4f/d5a/da1/fcf x:0 0 0 2024-04-22T23:35:06.157 INFO:tasks.workunit.client.1.smithi157.stdout:6/963: mkdir d1/d2f/dc7/d13d 0 2024-04-22T23:35:06.157 INFO:tasks.workunit.client.0.smithi107.stdout:0/436: write d1/d9/d14/f4d [31082,84606] 0 2024-04-22T23:35:06.158 INFO:tasks.workunit.client.1.smithi157.stdout:1/688: symlink d4/d17/d1d/d33/d41/lef 0 2024-04-22T23:35:06.160 INFO:tasks.workunit.client.1.smithi157.stdout:1/689: mknod d4/db/cf0 0 2024-04-22T23:35:06.161 INFO:tasks.workunit.client.1.smithi157.stdout:1/690: dread - d4/f6e zero size 2024-04-22T23:35:06.161 INFO:tasks.workunit.client.1.smithi157.stdout:1/691: chown d4/db/d3d/dc1 264237523 1 2024-04-22T23:35:06.164 INFO:tasks.workunit.client.1.smithi157.stdout:8/469: dwrite d2/d6/d21/d5f/f55 [0,4194304] 0 2024-04-22T23:35:06.165 INFO:tasks.workunit.client.0.smithi107.stdout:6/470: dwrite db/f3f [0,4194304] 0 2024-04-22T23:35:06.165 INFO:tasks.workunit.client.0.smithi107.stdout:6/471: creat db/f8a x:0 0 0 2024-04-22T23:35:06.165 INFO:tasks.workunit.client.0.smithi107.stdout:1/443: dwrite d7/d19/d20/d39/f3c [4194304,4194304] 0 2024-04-22T23:35:06.166 INFO:tasks.workunit.client.0.smithi107.stdout:1/444: readlink d7/d19/l2b 0 2024-04-22T23:35:06.166 INFO:tasks.workunit.client.0.smithi107.stdout:5/447: symlink d9/d26/d79/l9b 0 2024-04-22T23:35:06.172 INFO:tasks.workunit.client.0.smithi107.stdout:4/463: rmdir d1/d7/d63 39 2024-04-22T23:35:06.173 INFO:tasks.workunit.client.0.smithi107.stdout:2/493: mkdir d1/d2b/d49/d7e/da8 0 2024-04-22T23:35:06.173 INFO:tasks.workunit.client.0.smithi107.stdout:2/494: fsync d1/d12/d35/d48/d13/f74 0 2024-04-22T23:35:06.181 INFO:tasks.workunit.client.0.smithi107.stdout:8/402: rmdir d0 39 2024-04-22T23:35:06.185 INFO:tasks.workunit.client.1.smithi157.stdout:1/692: dread d4/d17/d34/d3b/d52/d65/dbc/faa [0,4194304] 0 2024-04-22T23:35:06.185 INFO:tasks.workunit.client.1.smithi157.stdout:1/693: readlink d4/d17/d1d/d33/d41/lef 0 2024-04-22T23:35:06.186 INFO:tasks.workunit.client.1.smithi157.stdout:1/694: read - d4/d17/d34/dc8/fec zero size 2024-04-22T23:35:06.186 INFO:tasks.workunit.client.0.smithi107.stdout:6/472: creat db/d20/d2c/d3b/d40/f8b x:0 0 0 2024-04-22T23:35:06.186 INFO:tasks.workunit.client.0.smithi107.stdout:1/445: creat d7/d19/d2d/d89/f90 x:0 0 0 2024-04-22T23:35:06.187 INFO:tasks.workunit.client.1.smithi157.stdout:1/695: getdents d4/d5f/d7e 0 2024-04-22T23:35:06.187 INFO:tasks.workunit.client.1.smithi157.stdout:2/769: dwrite d7/d8/d30/d5b/f72 [0,4194304] 0 2024-04-22T23:35:06.187 INFO:tasks.workunit.client.1.smithi157.stdout:2/770: stat d7/d8/d8e/fee 0 2024-04-22T23:35:06.187 INFO:tasks.workunit.client.1.smithi157.stdout:6/964: dwrite d1/d2f/d3b/d49/f7d [0,4194304] 0 2024-04-22T23:35:06.188 INFO:tasks.workunit.client.0.smithi107.stdout:4/464: dwrite d1/d1b/d44/f95 [0,4194304] 0 2024-04-22T23:35:06.190 INFO:tasks.workunit.client.1.smithi157.stdout:2/771: rename d7/d1f/d3e/db4 to d7/d8/d25/d102 0 2024-04-22T23:35:06.190 INFO:tasks.workunit.client.1.smithi157.stdout:2/772: write d7/d1f/d26/fb0 [185640,55601] 0 2024-04-22T23:35:06.190 INFO:tasks.workunit.client.0.smithi107.stdout:5/448: link d9/db/de/d5e/c82 d9/d26/d79/c9c 0 2024-04-22T23:35:06.191 INFO:tasks.workunit.client.1.smithi157.stdout:2/773: stat d7/d8/d30/d23/c32 0 2024-04-22T23:35:06.192 INFO:tasks.workunit.client.0.smithi107.stdout:2/495: mknod d1/d2b/d49/d7e/ca9 0 2024-04-22T23:35:06.192 INFO:tasks.workunit.client.0.smithi107.stdout:4/465: dread d1/d1b/f64 [0,4194304] 0 2024-04-22T23:35:06.192 INFO:tasks.workunit.client.0.smithi107.stdout:4/466: write d1/d21/d29/f71 [1023139,66702] 0 2024-04-22T23:35:06.194 INFO:tasks.workunit.client.0.smithi107.stdout:1/446: mkdir d7/d19/d20/d39/d53/d43/d63/d91 0 2024-04-22T23:35:06.194 INFO:tasks.workunit.client.0.smithi107.stdout:1/447: creat d7/d19/d20/d39/d53/d43/d63/f92 x:0 0 0 2024-04-22T23:35:06.194 INFO:tasks.workunit.client.0.smithi107.stdout:1/448: chown d7/d19/d2d/f31 809 1 2024-04-22T23:35:06.194 INFO:tasks.workunit.client.0.smithi107.stdout:1/449: write d7/d19/d42/f6f [5209482,11973] 0 2024-04-22T23:35:06.195 INFO:tasks.workunit.client.1.smithi157.stdout:2/774: creat d7/d8/d25/d102/f103 x:0 0 0 2024-04-22T23:35:06.195 INFO:tasks.workunit.client.1.smithi157.stdout:2/775: stat d7/d8/d95/dd2 0 2024-04-22T23:35:06.195 INFO:tasks.workunit.client.0.smithi107.stdout:5/449: getdents d9/db/de/d72 0 2024-04-22T23:35:06.196 INFO:tasks.workunit.client.0.smithi107.stdout:5/450: creat d9/f9d x:0 0 0 2024-04-22T23:35:06.196 INFO:tasks.workunit.client.0.smithi107.stdout:5/451: chown d9/l33 30316264 1 2024-04-22T23:35:06.196 INFO:tasks.workunit.client.0.smithi107.stdout:5/452: readlink d9/d26/l3e 0 2024-04-22T23:35:06.196 INFO:tasks.workunit.client.0.smithi107.stdout:5/453: readlink d9/db/de/l1c 0 2024-04-22T23:35:06.196 INFO:tasks.workunit.client.0.smithi107.stdout:2/496: getdents d1/d2b/d49/d87 0 2024-04-22T23:35:06.196 INFO:tasks.workunit.client.0.smithi107.stdout:2/497: creat d1/d2b/d49/faa x:0 0 0 2024-04-22T23:35:06.198 INFO:tasks.workunit.client.0.smithi107.stdout:4/467: rename d1/d7/d53/d66/d6b/l61 to d1/d21/d52/la7 0 2024-04-22T23:35:06.199 INFO:tasks.workunit.client.1.smithi157.stdout:2/776: rename d7/d8/d30/d23/d27/d57/fd0 to d7/d8/d30/d23/d27/d57/dcf/df2/f104 0 2024-04-22T23:35:06.201 INFO:tasks.workunit.client.0.smithi107.stdout:5/454: mkdir d9/db/de/d72/d9e 0 2024-04-22T23:35:06.202 INFO:tasks.workunit.client.0.smithi107.stdout:8/403: dwrite d0/d3a/d46/d59/f6e [0,4194304] 0 2024-04-22T23:35:06.202 INFO:tasks.workunit.client.0.smithi107.stdout:8/404: stat d0/d3a/d46 0 2024-04-22T23:35:06.203 INFO:tasks.workunit.client.1.smithi157.stdout:6/965: write d1/d38/faa [2987233,39647] 0 2024-04-22T23:35:06.203 INFO:tasks.workunit.client.1.smithi157.stdout:6/966: creat d1/d2f/d3b/d39/d4e/da8/dac/ddd/f13e x:0 0 0 2024-04-22T23:35:06.203 INFO:tasks.workunit.client.1.smithi157.stdout:6/967: fdatasync d1/f3 0 2024-04-22T23:35:06.203 INFO:tasks.workunit.client.0.smithi107.stdout:2/498: rmdir d1/d2b/d49/d87 39 2024-04-22T23:35:06.203 INFO:tasks.workunit.client.0.smithi107.stdout:4/468: mkdir d1/d8/d56/da8 0 2024-04-22T23:35:06.205 INFO:tasks.workunit.client.0.smithi107.stdout:5/455: mknod d9/d26/c9f 0 2024-04-22T23:35:06.208 INFO:tasks.workunit.client.1.smithi157.stdout:2/777: rmdir d7/d8 39 2024-04-22T23:35:06.208 INFO:tasks.workunit.client.1.smithi157.stdout:2/778: fdatasync d7/d8/fbf 0 2024-04-22T23:35:06.208 INFO:tasks.workunit.client.0.smithi107.stdout:1/450: dread d7/d19/d71/f78 [0,4194304] 0 2024-04-22T23:35:06.209 INFO:tasks.workunit.client.0.smithi107.stdout:1/451: chown d7/d19/c8c 204 1 2024-04-22T23:35:06.209 INFO:tasks.workunit.client.0.smithi107.stdout:1/452: fdatasync d7/d19/d20/d39/d53/f29 0 2024-04-22T23:35:06.209 INFO:tasks.workunit.client.0.smithi107.stdout:2/499: creat d1/d2b/d88/fab x:0 0 0 2024-04-22T23:35:06.211 INFO:tasks.workunit.client.1.smithi157.stdout:6/968: mkdir d1/d2f/d3b/d39/d4e/da8/dac/ddd/d13f 0 2024-04-22T23:35:06.212 INFO:tasks.workunit.client.0.smithi107.stdout:5/456: dread d9/db/f3c [0,4194304] 0 2024-04-22T23:35:06.215 INFO:tasks.workunit.client.1.smithi157.stdout:1/696: dwrite d4/d17/d34/d3b/f66 [0,4194304] 0 2024-04-22T23:35:06.224 INFO:tasks.workunit.client.0.smithi107.stdout:1/453: write d7/d19/d20/d39/f5a [800621,125276] 0 2024-04-22T23:35:06.225 INFO:tasks.workunit.client.0.smithi107.stdout:4/469: dread d1/d21/d85/f9c [0,4194304] 0 2024-04-22T23:35:06.225 INFO:tasks.workunit.client.0.smithi107.stdout:4/470: creat d1/d1b/fa9 x:0 0 0 2024-04-22T23:35:06.226 INFO:tasks.workunit.client.1.smithi157.stdout:6/969: dread d1/d121/d9b/dda/ffc [0,4194304] 0 2024-04-22T23:35:06.226 INFO:tasks.workunit.client.1.smithi157.stdout:6/970: fdatasync d1/d36/fa3 0 2024-04-22T23:35:06.226 INFO:tasks.workunit.client.1.smithi157.stdout:6/971: read - d1/d2f/d3b/d39/d4e/da8/dac/db0/f112 zero size 2024-04-22T23:35:06.226 INFO:tasks.workunit.client.0.smithi107.stdout:5/457: rmdir d9/d26 39 2024-04-22T23:35:06.227 INFO:tasks.workunit.client.0.smithi107.stdout:5/458: stat d9/d26/c9f 0 2024-04-22T23:35:06.227 INFO:tasks.workunit.client.0.smithi107.stdout:1/454: mkdir d7/d19/d20/d39/d93 0 2024-04-22T23:35:06.228 INFO:tasks.workunit.client.0.smithi107.stdout:2/500: dread d1/d12/f26 [0,4194304] 0 2024-04-22T23:35:06.228 INFO:tasks.workunit.client.0.smithi107.stdout:8/405: dwrite d0/d3a/f4a [0,4194304] 0 2024-04-22T23:35:06.229 INFO:tasks.workunit.client.1.smithi157.stdout:1/697: write d4/d17/d34/d75/f89 [3142719,39578] 0 2024-04-22T23:35:06.229 INFO:tasks.workunit.client.0.smithi107.stdout:5/459: dread d9/db/de/d5e/f67 [0,4194304] 0 2024-04-22T23:35:06.230 INFO:tasks.workunit.client.0.smithi107.stdout:4/471: symlink d1/d8/d56/laa 0 2024-04-22T23:35:06.230 INFO:tasks.workunit.client.0.smithi107.stdout:4/472: write d1/d7/d63/d78/d96/fa6 [725800,93024] 0 2024-04-22T23:35:06.230 INFO:tasks.workunit.client.0.smithi107.stdout:4/473: write d1/d8/d67/f8f [176370,91066] 0 2024-04-22T23:35:06.230 INFO:tasks.workunit.client.1.smithi157.stdout:2/779: dwrite d7/d8/d30/d23/f7f [0,4194304] 0 2024-04-22T23:35:06.230 INFO:tasks.workunit.client.1.smithi157.stdout:1/698: getdents d4/d17/d34/d3b/d52/d65 0 2024-04-22T23:35:06.231 INFO:tasks.workunit.client.1.smithi157.stdout:2/780: mkdir d7/d105 0 2024-04-22T23:35:06.232 INFO:tasks.workunit.client.0.smithi107.stdout:1/455: unlink d7/d19/d20/d39/l88 0 2024-04-22T23:35:06.232 INFO:tasks.workunit.client.1.smithi157.stdout:1/699: mkdir d4/d17/d34/d3b/d52/d65/df1 0 2024-04-22T23:35:06.232 INFO:tasks.workunit.client.1.smithi157.stdout:1/700: dread - d4/d17/d34/d75/dcc/fd7 zero size 2024-04-22T23:35:06.234 INFO:tasks.workunit.client.1.smithi157.stdout:1/701: rmdir d4/d17/d1d 39 2024-04-22T23:35:06.236 INFO:tasks.workunit.client.0.smithi107.stdout:8/406: rename d0/d7/df/d3b/f6a to d0/f83 0 2024-04-22T23:35:06.236 INFO:tasks.workunit.client.1.smithi157.stdout:1/702: creat d4/d17/d34/dc8/ff2 x:0 0 0 2024-04-22T23:35:06.237 INFO:tasks.workunit.client.1.smithi157.stdout:1/703: chown d4/db/f24 0 1 2024-04-22T23:35:06.237 INFO:tasks.workunit.client.1.smithi157.stdout:1/704: creat d4/d17/d34/d3b/d50/ff3 x:0 0 0 2024-04-22T23:35:06.238 INFO:tasks.workunit.client.0.smithi107.stdout:5/460: rmdir d9/db 39 2024-04-22T23:35:06.238 INFO:tasks.workunit.client.0.smithi107.stdout:4/474: mkdir d1/d7/d63/d78/da4/dab 0 2024-04-22T23:35:06.239 INFO:tasks.workunit.client.1.smithi157.stdout:1/705: symlink d4/d5f/d7e/db8/lf4 0 2024-04-22T23:35:06.239 INFO:tasks.workunit.client.1.smithi157.stdout:1/706: dread - d4/d17/d1d/d33/f9a zero size 2024-04-22T23:35:06.239 INFO:tasks.workunit.client.0.smithi107.stdout:8/407: symlink d0/d7/l84 0 2024-04-22T23:35:06.240 INFO:tasks.workunit.client.0.smithi107.stdout:5/461: unlink d9/db/de/d5e/c6d 0 2024-04-22T23:35:06.242 INFO:tasks.workunit.client.0.smithi107.stdout:4/475: truncate d1/d7/d63/d78/d80/f82 991914 0 2024-04-22T23:35:06.245 INFO:tasks.workunit.client.0.smithi107.stdout:8/408: rename d0/d7/df/d18 to d0/d85 0 2024-04-22T23:35:06.245 INFO:tasks.workunit.client.0.smithi107.stdout:1/456: write d7/d19/d20/f38 [1189815,106462] 0 2024-04-22T23:35:06.245 INFO:tasks.workunit.client.0.smithi107.stdout:1/457: creat d7/d19/d2d/d89/f94 x:0 0 0 2024-04-22T23:35:06.245 INFO:tasks.workunit.client.0.smithi107.stdout:4/476: getdents d1/d7/d63/d78/d96 0 2024-04-22T23:35:06.248 INFO:tasks.workunit.client.0.smithi107.stdout:8/409: mknod d0/d7/df/d3b/c86 0 2024-04-22T23:35:06.248 INFO:tasks.workunit.client.0.smithi107.stdout:4/477: read d1/d1b/f7a [523850,121316] 0 2024-04-22T23:35:06.248 INFO:tasks.workunit.client.0.smithi107.stdout:4/478: fdatasync d1/d1b/f49 0 2024-04-22T23:35:06.250 INFO:tasks.workunit.client.1.smithi157.stdout:6/972: dwrite d1/f12 [0,4194304] 0 2024-04-22T23:35:06.251 INFO:tasks.workunit.client.1.smithi157.stdout:1/707: write d4/db/f47 [926332,3473] 0 2024-04-22T23:35:06.251 INFO:tasks.workunit.client.0.smithi107.stdout:1/458: creat d7/d19/f95 x:0 0 0 2024-04-22T23:35:06.251 INFO:tasks.workunit.client.0.smithi107.stdout:2/501: dwrite d1/d12/d35/d48/f1f [0,4194304] 0 2024-04-22T23:35:06.252 INFO:tasks.workunit.client.1.smithi157.stdout:6/973: symlink d1/d121/dc4/d126/l140 0 2024-04-22T23:35:06.252 INFO:tasks.workunit.client.1.smithi157.stdout:6/974: write d1/d121/d9b/f61 [1120984,109500] 0 2024-04-22T23:35:06.252 INFO:tasks.workunit.client.1.smithi157.stdout:6/975: truncate d1/d2f/d3b/d39/d4e/da8/dac/dbe/def/f108 30988 0 2024-04-22T23:35:06.253 INFO:tasks.workunit.client.0.smithi107.stdout:8/410: mkdir d0/d7/d87 0 2024-04-22T23:35:06.254 INFO:tasks.workunit.client.1.smithi157.stdout:6/976: mkdir d1/d36/dea/d141 0 2024-04-22T23:35:06.254 INFO:tasks.workunit.client.1.smithi157.stdout:6/977: chown d1/d36 9465 1 2024-04-22T23:35:06.254 INFO:tasks.workunit.client.1.smithi157.stdout:6/978: dread - d1/d2f/d3b/d39/d4e/da8/dac/ddd/f13e zero size 2024-04-22T23:35:06.255 INFO:tasks.workunit.client.0.smithi107.stdout:4/479: creat d1/d21/d29/d2e/fac x:0 0 0 2024-04-22T23:35:06.255 INFO:tasks.workunit.client.0.smithi107.stdout:4/480: write d1/d1b/f9d [949442,110282] 0 2024-04-22T23:35:06.255 INFO:tasks.workunit.client.0.smithi107.stdout:4/481: stat d1/d21/d85/f9c 0 2024-04-22T23:35:06.258 INFO:tasks.workunit.client.1.smithi157.stdout:9/644: sync 2024-04-22T23:35:06.258 INFO:tasks.workunit.client.1.smithi157.stdout:4/697: sync 2024-04-22T23:35:06.258 INFO:tasks.workunit.client.1.smithi157.stdout:9/645: truncate d4/d15/d23/d27/fba 852518 0 2024-04-22T23:35:06.258 INFO:tasks.workunit.client.1.smithi157.stdout:4/698: truncate d3/d11/d1d/d73/da3/d31/f6f 1385750 0 2024-04-22T23:35:06.258 INFO:tasks.workunit.client.1.smithi157.stdout:9/646: stat d4/d15/d23/l2c 0 2024-04-22T23:35:06.261 INFO:tasks.workunit.client.1.smithi157.stdout:4/699: getdents d3/d11/d1d/d73 0 2024-04-22T23:35:06.262 INFO:tasks.workunit.client.0.smithi107.stdout:2/502: creat d1/d2b/d59/fac x:0 0 0 2024-04-22T23:35:06.262 INFO:tasks.workunit.client.0.smithi107.stdout:2/503: creat d1/fad x:0 0 0 2024-04-22T23:35:06.264 INFO:tasks.workunit.client.0.smithi107.stdout:8/411: write d0/d7/d8/f1e [3847243,58074] 0 2024-04-22T23:35:06.264 INFO:tasks.workunit.client.0.smithi107.stdout:5/462: dwrite d9/fd [0,4194304] 0 2024-04-22T23:35:06.264 INFO:tasks.workunit.client.0.smithi107.stdout:4/482: truncate d1/d1b/f49 4572153 0 2024-04-22T23:35:06.265 INFO:tasks.workunit.client.1.smithi157.stdout:9/647: rename d4/d15/d55/d8d/c9e to d4/d2d/cc9 0 2024-04-22T23:35:06.265 INFO:tasks.workunit.client.1.smithi157.stdout:9/648: write d4/df/d61/d9c/fb2 [1809298,126586] 0 2024-04-22T23:35:06.265 INFO:tasks.workunit.client.1.smithi157.stdout:9/649: write d4/df/d57/fb0 [959044,37301] 0 2024-04-22T23:35:06.266 INFO:tasks.workunit.client.1.smithi157.stdout:4/700: unlink d3/d5c/d7b/fc2 0 2024-04-22T23:35:06.267 INFO:tasks.workunit.client.0.smithi107.stdout:2/504: mkdir d1/d2b/d49/d7e/dae 0 2024-04-22T23:35:06.267 INFO:tasks.workunit.client.1.smithi157.stdout:9/650: creat d4/d2d/fca x:0 0 0 2024-04-22T23:35:06.267 INFO:tasks.workunit.client.1.smithi157.stdout:9/651: getdents d4/df/d57/db7 0 2024-04-22T23:35:06.267 INFO:tasks.workunit.client.1.smithi157.stdout:4/701: truncate d3/d5c/f79 4051555 0 2024-04-22T23:35:06.269 INFO:tasks.workunit.client.0.smithi107.stdout:8/412: creat d0/d7/df/d3b/f88 x:0 0 0 2024-04-22T23:35:06.269 INFO:tasks.workunit.client.0.smithi107.stdout:8/413: readlink d0/d7/d8/d41/l7e 0 2024-04-22T23:35:06.273 INFO:tasks.workunit.client.1.smithi157.stdout:6/979: read d1/d2f/d3b/f2e [2854133,118067] 0 2024-04-22T23:35:06.273 INFO:tasks.workunit.client.1.smithi157.stdout:6/980: creat d1/d38/daf/f142 x:0 0 0 2024-04-22T23:35:06.274 INFO:tasks.workunit.client.0.smithi107.stdout:4/483: truncate d1/f13 6936480 0 2024-04-22T23:35:06.274 INFO:tasks.workunit.client.0.smithi107.stdout:4/484: write d1/d7/fc [2425773,72980] 0 2024-04-22T23:35:06.277 INFO:tasks.workunit.client.0.smithi107.stdout:5/463: getdents d9/d76 0 2024-04-22T23:35:06.279 INFO:tasks.workunit.client.0.smithi107.stdout:4/485: mknod d1/d7/d63/cad 0 2024-04-22T23:35:06.279 INFO:tasks.workunit.client.0.smithi107.stdout:4/486: read d1/d8/f1f [2935895,130070] 0 2024-04-22T23:35:06.279 INFO:tasks.workunit.client.1.smithi157.stdout:6/981: symlink d1/d2f/d3b/d39/d4e/da8/dac/dbe/l143 0 2024-04-22T23:35:06.279 INFO:tasks.workunit.client.1.smithi157.stdout:6/982: fsync d1/d38/daf/f11e 0 2024-04-22T23:35:06.280 INFO:tasks.workunit.client.0.smithi107.stdout:8/414: dread d0/d7/d8/f6c [0,4194304] 0 2024-04-22T23:35:06.280 INFO:tasks.workunit.client.0.smithi107.stdout:8/415: fsync d0/f2f 0 2024-04-22T23:35:06.280 INFO:tasks.workunit.client.0.smithi107.stdout:5/464: mkdir d9/d76/da0 0 2024-04-22T23:35:06.281 INFO:tasks.workunit.client.0.smithi107.stdout:5/465: chown d9/d26/f64 25459406 1 2024-04-22T23:35:06.281 INFO:tasks.workunit.client.0.smithi107.stdout:5/466: fdatasync d9/db/f5f 0 2024-04-22T23:35:06.282 INFO:tasks.workunit.client.0.smithi107.stdout:4/487: symlink d1/d8/d56/da8/lae 0 2024-04-22T23:35:06.282 INFO:tasks.workunit.client.0.smithi107.stdout:4/488: chown d1/d8/f3f 1821615767 1 2024-04-22T23:35:06.282 INFO:tasks.workunit.client.0.smithi107.stdout:4/489: chown d1/d1b/fa9 362 1 2024-04-22T23:35:06.283 INFO:tasks.workunit.client.0.smithi107.stdout:2/505: dwrite d1/d2b/f94 [0,4194304] 0 2024-04-22T23:35:06.283 INFO:tasks.workunit.client.0.smithi107.stdout:8/416: mkdir d0/d7/d20/d89 0 2024-04-22T23:35:06.288 INFO:tasks.workunit.client.1.smithi157.stdout:6/983: write d1/d2f/d3b/d39/d4e/da8/de5/d83/f73 [1448114,83716] 0 2024-04-22T23:35:06.289 INFO:tasks.workunit.client.1.smithi157.stdout:9/652: dwrite d4/df/d57/fb0 [0,4194304] 0 2024-04-22T23:35:06.292 INFO:tasks.workunit.client.1.smithi157.stdout:6/984: mkdir d1/d2f/d3b/d39/d4e/da8/dac/ddd/ddf/d144 0 2024-04-22T23:35:06.297 INFO:tasks.workunit.client.0.smithi107.stdout:2/506: rename d1/d12/d35/d48/d13/d4f/l60 to d1/d12/d35/d54/laf 0 2024-04-22T23:35:06.297 INFO:tasks.workunit.client.0.smithi107.stdout:2/507: read - d1/d2b/d49/d7e/f8e zero size 2024-04-22T23:35:06.297 INFO:tasks.workunit.client.0.smithi107.stdout:2/508: fsync d1/d12/d35/f3f 0 2024-04-22T23:35:06.297 INFO:tasks.workunit.client.0.smithi107.stdout:2/509: readlink d1/l17 0 2024-04-22T23:35:06.298 INFO:tasks.workunit.client.0.smithi107.stdout:5/467: dwrite d9/f29 [0,4194304] 0 2024-04-22T23:35:06.299 INFO:tasks.workunit.client.0.smithi107.stdout:8/417: unlink d0/d85/d51/f82 0 2024-04-22T23:35:06.301 INFO:tasks.workunit.client.0.smithi107.stdout:2/510: rename d1/d12/d35/d48/d13/f40 to d1/d12/d35/d48/d13/fb0 0 2024-04-22T23:35:06.301 INFO:tasks.workunit.client.0.smithi107.stdout:2/511: stat d1/d12/d35/d48/d13/d4f/f62 0 2024-04-22T23:35:06.303 INFO:tasks.workunit.client.0.smithi107.stdout:5/468: mknod d9/db/d92/ca1 0 2024-04-22T23:35:06.303 INFO:tasks.workunit.client.0.smithi107.stdout:5/469: write d9/d26/f64 [5170049,97205] 0 2024-04-22T23:35:06.304 INFO:tasks.workunit.client.0.smithi107.stdout:8/418: creat d0/d3a/d46/f8a x:0 0 0 2024-04-22T23:35:06.304 INFO:tasks.workunit.client.0.smithi107.stdout:8/419: chown d0/d7/df/d33/d37/f80 2217676 1 2024-04-22T23:35:06.305 INFO:tasks.workunit.client.0.smithi107.stdout:4/490: getdents d1/d7/d63 0 2024-04-22T23:35:06.306 INFO:tasks.workunit.client.0.smithi107.stdout:2/512: mknod d1/d2b/d59/cb1 0 2024-04-22T23:35:06.306 INFO:tasks.workunit.client.0.smithi107.stdout:2/513: creat d1/fb2 x:0 0 0 2024-04-22T23:35:06.308 INFO:tasks.workunit.client.0.smithi107.stdout:5/470: creat d9/db/fa2 x:0 0 0 2024-04-22T23:35:06.308 INFO:tasks.workunit.client.0.smithi107.stdout:5/471: truncate d9/db/d42/f5a 4934606 0 2024-04-22T23:35:06.308 INFO:tasks.workunit.client.0.smithi107.stdout:5/472: stat d9/d26/f64 0 2024-04-22T23:35:06.308 INFO:tasks.workunit.client.0.smithi107.stdout:5/473: readlink d9/db/d42/l48 0 2024-04-22T23:35:06.309 INFO:tasks.workunit.client.0.smithi107.stdout:8/420: rename d0/d7/df/f4b to d0/d85/f8b 0 2024-04-22T23:35:06.311 INFO:tasks.workunit.client.0.smithi107.stdout:4/491: unlink d1/d8/d56/da8/lae 0 2024-04-22T23:35:06.311 INFO:tasks.workunit.client.0.smithi107.stdout:4/492: dread - d1/d8/f7c zero size 2024-04-22T23:35:06.312 INFO:tasks.workunit.client.0.smithi107.stdout:5/474: creat d9/db/de/d72/d9e/fa3 x:0 0 0 2024-04-22T23:35:06.313 INFO:tasks.workunit.client.1.smithi157.stdout:9/653: dwrite d4/df/d61/f66 [0,4194304] 0 2024-04-22T23:35:06.313 INFO:tasks.workunit.client.1.smithi157.stdout:9/654: dread - d4/df/d61/fc1 zero size 2024-04-22T23:35:06.313 INFO:tasks.workunit.client.1.smithi157.stdout:6/985: dwrite d1/d2f/d3b/d39/d4e/da8/dac/ddd/f131 [0,4194304] 0 2024-04-22T23:35:06.313 INFO:tasks.workunit.client.0.smithi107.stdout:3/505: sync 2024-04-22T23:35:06.313 INFO:tasks.workunit.client.0.smithi107.stdout:7/385: sync 2024-04-22T23:35:06.313 INFO:tasks.workunit.client.0.smithi107.stdout:9/499: sync 2024-04-22T23:35:06.314 INFO:tasks.workunit.client.0.smithi107.stdout:9/500: write d5/f13 [77145,125174] 0 2024-04-22T23:35:06.314 INFO:tasks.workunit.client.0.smithi107.stdout:8/421: truncate d0/f5 7695163 0 2024-04-22T23:35:06.314 INFO:tasks.workunit.client.0.smithi107.stdout:8/422: stat d0/d7/d8/l44 0 2024-04-22T23:35:06.314 INFO:tasks.workunit.client.0.smithi107.stdout:8/423: getdents d0/d7/df/d3b/d7b 0 2024-04-22T23:35:06.314 INFO:tasks.workunit.client.0.smithi107.stdout:8/424: write d0/d7/df/d3b/f88 [591835,48368] 0 2024-04-22T23:35:06.314 INFO:tasks.workunit.client.0.smithi107.stdout:8/425: dread - d0/d7/d20/d2d/f30 zero size 2024-04-22T23:35:06.314 INFO:tasks.workunit.client.0.smithi107.stdout:4/493: mkdir d1/d8/daf 0 2024-04-22T23:35:06.315 INFO:tasks.workunit.client.1.smithi157.stdout:9/655: link d4/df/d61/f66 d4/df/d61/fcb 0 2024-04-22T23:35:06.315 INFO:tasks.workunit.client.1.smithi157.stdout:9/656: write d4/d15/d23/d27/db5/fc3 [883921,130225] 0 2024-04-22T23:35:06.317 INFO:tasks.workunit.client.1.smithi157.stdout:6/986: creat d1/d121/dc4/f145 x:0 0 0 2024-04-22T23:35:06.317 INFO:tasks.workunit.client.0.smithi107.stdout:8/426: unlink d0/d3a/d46/d59/f6b 0 2024-04-22T23:35:06.318 INFO:tasks.workunit.client.0.smithi107.stdout:4/494: rmdir d1/d7/d63/d91 0 2024-04-22T23:35:06.319 INFO:tasks.workunit.client.1.smithi157.stdout:9/657: mknod d4/ccc 0 2024-04-22T23:35:06.320 INFO:tasks.workunit.client.0.smithi107.stdout:5/475: mkdir d9/db/de/d72/d7c/da4 0 2024-04-22T23:35:06.322 INFO:tasks.workunit.client.1.smithi157.stdout:9/658: mkdir d4/d15/d23/d27/db5/db9/dcd 0 2024-04-22T23:35:06.323 INFO:tasks.workunit.client.0.smithi107.stdout:8/427: getdents d0/d7/d8 0 2024-04-22T23:35:06.323 INFO:tasks.workunit.client.0.smithi107.stdout:4/495: creat d1/d21/d29/d2e/d88/d99/fb0 x:0 0 0 2024-04-22T23:35:06.323 INFO:tasks.workunit.client.0.smithi107.stdout:4/496: fdatasync d1/d21/d29/d2e/d3b/f8a 0 2024-04-22T23:35:06.324 INFO:tasks.workunit.client.1.smithi157.stdout:6/987: getdents d1/d2f/d3b/d39/d4e/da8/dac/dbe 0 2024-04-22T23:35:06.324 INFO:tasks.workunit.client.1.smithi157.stdout:6/988: chown d1/d2f/d3b/d39/d4e/ff8 30 1 2024-04-22T23:35:06.324 INFO:tasks.workunit.client.0.smithi107.stdout:8/428: mknod d0/d73/c8c 0 2024-04-22T23:35:06.324 INFO:tasks.workunit.client.0.smithi107.stdout:8/429: chown d0/d7/df/d33/d37/f80 173423 1 2024-04-22T23:35:06.326 INFO:tasks.workunit.client.1.smithi157.stdout:9/659: symlink d4/d15/d23/d67/lce 0 2024-04-22T23:35:06.327 INFO:tasks.workunit.client.0.smithi107.stdout:7/386: write d7/d21/d48/f6b [2760343,112307] 0 2024-04-22T23:35:06.327 INFO:tasks.workunit.client.0.smithi107.stdout:7/387: fsync d7/d4c/f6c 0 2024-04-22T23:35:06.327 INFO:tasks.workunit.client.0.smithi107.stdout:7/388: dread - d7/d21/d48/f5b zero size 2024-04-22T23:35:06.327 INFO:tasks.workunit.client.0.smithi107.stdout:7/389: chown d7/f11 26 1 2024-04-22T23:35:06.328 INFO:tasks.workunit.client.1.smithi157.stdout:9/660: mknod d4/d15/d23/dad/ccf 0 2024-04-22T23:35:06.328 INFO:tasks.workunit.client.1.smithi157.stdout:9/661: chown d4/d15/d23/d27/d75 1 1 2024-04-22T23:35:06.329 INFO:tasks.workunit.client.0.smithi107.stdout:7/390: creat d7/d21/f7f x:0 0 0 2024-04-22T23:35:06.330 INFO:tasks.workunit.client.0.smithi107.stdout:7/391: dread - d7/d1b/f68 zero size 2024-04-22T23:35:06.330 INFO:tasks.workunit.client.1.smithi157.stdout:9/662: rmdir d4/d15/d55 39 2024-04-22T23:35:06.330 INFO:tasks.workunit.client.1.smithi157.stdout:9/663: read - d4/d15/d23/d27/d47/db3/fb8 zero size 2024-04-22T23:35:06.337 INFO:tasks.workunit.client.0.smithi107.stdout:7/392: getdents d7/d21/d54 0 2024-04-22T23:35:06.337 INFO:tasks.workunit.client.0.smithi107.stdout:7/393: chown d7/d2d/f41 3 1 2024-04-22T23:35:06.341 INFO:tasks.workunit.client.0.smithi107.stdout:3/506: dwrite db/d21/d27/d15/d29/d2a/f43 [0,4194304] 0 2024-04-22T23:35:06.342 INFO:tasks.workunit.client.0.smithi107.stdout:3/507: chown db/d21/d27/d54/d25/c84 4 1 2024-04-22T23:35:06.342 INFO:tasks.workunit.client.0.smithi107.stdout:8/430: dread d0/d7/f34 [0,4194304] 0 2024-04-22T23:35:06.342 INFO:tasks.workunit.client.0.smithi107.stdout:8/431: fdatasync d0/d3a/f56 0 2024-04-22T23:35:06.342 INFO:tasks.workunit.client.0.smithi107.stdout:8/432: fsync d0/d7/d20/f39 0 2024-04-22T23:35:06.343 INFO:tasks.workunit.client.0.smithi107.stdout:3/508: mkdir db/d21/db1 0 2024-04-22T23:35:06.345 INFO:tasks.workunit.client.0.smithi107.stdout:8/433: mknod d0/d73/c8d 0 2024-04-22T23:35:06.347 INFO:tasks.workunit.client.0.smithi107.stdout:8/434: symlink d0/d7/df/d33/d37/l8e 0 2024-04-22T23:35:06.348 INFO:tasks.workunit.client.0.smithi107.stdout:8/435: symlink d0/d3a/l8f 0 2024-04-22T23:35:06.349 INFO:tasks.workunit.client.0.smithi107.stdout:8/436: stat d0/d7/d8/d41/d53/d76 0 2024-04-22T23:35:06.351 INFO:tasks.workunit.client.0.smithi107.stdout:3/509: dread db/ff [0,4194304] 0 2024-04-22T23:35:06.351 INFO:tasks.workunit.client.0.smithi107.stdout:9/501: dwrite d5/f13 [0,4194304] 0 2024-04-22T23:35:06.351 INFO:tasks.workunit.client.0.smithi107.stdout:9/502: truncate d5/d14/d2e/f84 907529 0 2024-04-22T23:35:06.351 INFO:tasks.workunit.client.0.smithi107.stdout:9/503: creat d5/d14/d2e/fb1 x:0 0 0 2024-04-22T23:35:06.354 INFO:tasks.workunit.client.0.smithi107.stdout:7/394: read d7/f3b [3310893,114902] 0 2024-04-22T23:35:06.354 INFO:tasks.workunit.client.0.smithi107.stdout:7/395: write d7/f3b [2149533,28793] 0 2024-04-22T23:35:06.354 INFO:tasks.workunit.client.0.smithi107.stdout:3/510: truncate db/d21/d27/f39 2485755 0 2024-04-22T23:35:06.357 INFO:tasks.workunit.client.0.smithi107.stdout:7/396: symlink d7/d2d/d66/l80 0 2024-04-22T23:35:06.358 INFO:tasks.workunit.client.0.smithi107.stdout:3/511: mkdir db/d21/d27/d15/d29/d4f/db2 0 2024-04-22T23:35:06.359 INFO:tasks.workunit.client.0.smithi107.stdout:3/512: fsync db/d21/d27/d15/d24/d52/d60/f6b 0 2024-04-22T23:35:06.359 INFO:tasks.workunit.client.0.smithi107.stdout:3/513: chown db/d21/d27/d15/d24/d52/d60 764628741 1 2024-04-22T23:35:06.359 INFO:tasks.workunit.client.0.smithi107.stdout:3/514: write db/d21/d27/d15/d24/d52/d60/f92 [284603,88159] 0 2024-04-22T23:35:06.365 INFO:tasks.workunit.client.0.smithi107.stdout:5/476: dwrite d9/db/de/d5e/f67 [0,4194304] 0 2024-04-22T23:35:06.365 INFO:tasks.workunit.client.0.smithi107.stdout:5/477: stat d9/d26/d96 0 2024-04-22T23:35:06.365 INFO:tasks.workunit.client.0.smithi107.stdout:5/478: creat d9/fa5 x:0 0 0 2024-04-22T23:35:06.365 INFO:tasks.workunit.client.0.smithi107.stdout:5/479: creat d9/db/de/fa6 x:0 0 0 2024-04-22T23:35:06.366 INFO:tasks.workunit.client.1.smithi157.stdout:9/664: dwrite d4/df/d61/fcb [0,4194304] 0 2024-04-22T23:35:06.366 INFO:tasks.workunit.client.1.smithi157.stdout:9/665: chown d4/df/d80 1607 1 2024-04-22T23:35:06.366 INFO:tasks.workunit.client.1.smithi157.stdout:6/989: dwrite d1/fc2 [4194304,4194304] 0 2024-04-22T23:35:06.370 INFO:tasks.workunit.client.0.smithi107.stdout:4/497: dwrite d1/d8/f7c [0,4194304] 0 2024-04-22T23:35:06.370 INFO:tasks.workunit.client.1.smithi157.stdout:6/990: rename d1/lb to d1/d2f/d3b/d39/d4e/da8/de5/da7/l146 0 2024-04-22T23:35:06.376 INFO:tasks.workunit.client.0.smithi107.stdout:3/515: dread db/d21/d27/f20 [0,4194304] 0 2024-04-22T23:35:06.377 INFO:tasks.workunit.client.0.smithi107.stdout:5/480: dread d9/db/f27 [0,4194304] 0 2024-04-22T23:35:06.379 INFO:tasks.workunit.client.0.smithi107.stdout:7/397: dread d7/f3b [0,4194304] 0 2024-04-22T23:35:06.380 INFO:tasks.workunit.client.1.smithi157.stdout:9/666: write d4/df/f26 [836011,116666] 0 2024-04-22T23:35:06.381 INFO:tasks.workunit.client.0.smithi107.stdout:8/437: dwrite d0/d7/df/f5c [0,4194304] 0 2024-04-22T23:35:06.382 INFO:tasks.workunit.client.0.smithi107.stdout:9/504: dwrite d5/f10 [0,4194304] 0 2024-04-22T23:35:06.382 INFO:tasks.workunit.client.0.smithi107.stdout:9/505: chown d5/d14/d2e/d4f/fa4 3306 1 2024-04-22T23:35:06.383 INFO:tasks.workunit.client.0.smithi107.stdout:9/506: dread - d5/d14/d2e/fb1 zero size 2024-04-22T23:35:06.383 INFO:tasks.workunit.client.0.smithi107.stdout:9/507: creat d5/d7/d17/d1b/d1f/d36/d96/fb2 x:0 0 0 2024-04-22T23:35:06.384 INFO:tasks.workunit.client.1.smithi157.stdout:9/667: getdents d4/df/d61 0 2024-04-22T23:35:06.384 INFO:tasks.workunit.client.1.smithi157.stdout:9/668: chown d4/df/fb 1828863 1 2024-04-22T23:35:06.384 INFO:tasks.workunit.client.1.smithi157.stdout:9/669: mkdir d4/df/d57/dd0 0 2024-04-22T23:35:06.385 INFO:tasks.workunit.client.0.smithi107.stdout:4/498: mkdir d1/d1b/db1 0 2024-04-22T23:35:06.385 INFO:tasks.workunit.client.1.smithi157.stdout:9/670: rmdir d4/df/d72 39 2024-04-22T23:35:06.386 INFO:tasks.workunit.client.0.smithi107.stdout:5/481: unlink d9/db/de/c2e 0 2024-04-22T23:35:06.386 INFO:tasks.workunit.client.1.smithi157.stdout:9/671: creat d4/df/d57/d6b/dc4/fd1 x:0 0 0 2024-04-22T23:35:06.387 INFO:tasks.workunit.client.0.smithi107.stdout:7/398: mkdir d7/d2d/d81 0 2024-04-22T23:35:06.390 INFO:tasks.workunit.client.1.smithi157.stdout:6/991: dwrite d1/d2f/d3b/d39/d4e/da8/de5/fd6 [0,4194304] 0 2024-04-22T23:35:06.391 INFO:tasks.workunit.client.0.smithi107.stdout:8/438: truncate d0/d7/d8/f1e 3376608 0 2024-04-22T23:35:06.393 INFO:tasks.workunit.client.1.smithi157.stdout:6/992: link d1/d121/d9b/l53 d1/d38/l147 0 2024-04-22T23:35:06.394 INFO:tasks.workunit.client.0.smithi107.stdout:9/508: mknod d5/d14/d2e/d4f/d73/cb3 0 2024-04-22T23:35:06.394 INFO:tasks.workunit.client.0.smithi107.stdout:9/509: dread - d5/d7/d17/d1b/d1f/d36/d96/faa zero size 2024-04-22T23:35:06.394 INFO:tasks.workunit.client.0.smithi107.stdout:9/510: readlink d5/d14/d2e/d43/l4c 0 2024-04-22T23:35:06.396 INFO:tasks.workunit.client.1.smithi157.stdout:6/993: rename d1/c82 to d1/d36/c148 0 2024-04-22T23:35:06.396 INFO:tasks.workunit.client.1.smithi157.stdout:6/994: write d1/d2f/d3b/feb [3511040,54323] 0 2024-04-22T23:35:06.396 INFO:tasks.workunit.client.1.smithi157.stdout:6/995: creat d1/d17/d5d/f149 x:0 0 0 2024-04-22T23:35:06.396 INFO:tasks.workunit.client.1.smithi157.stdout:6/996: fdatasync d1/f12 0 2024-04-22T23:35:06.397 INFO:tasks.workunit.client.0.smithi107.stdout:4/499: creat d1/d1b/d89/fb2 x:0 0 0 2024-04-22T23:35:06.399 INFO:tasks.workunit.client.0.smithi107.stdout:4/500: read d1/d21/d29/f71 [29807,108651] 0 2024-04-22T23:35:06.399 INFO:tasks.workunit.client.1.smithi157.stdout:6/997: fsync d1/d2f/d3b/f3e 0 2024-04-22T23:35:06.400 INFO:tasks.workunit.client.0.smithi107.stdout:7/399: symlink d7/d1b/d51/l82 0 2024-04-22T23:35:06.400 INFO:tasks.workunit.client.0.smithi107.stdout:7/400: write d7/d2d/f43 [659661,95168] 0 2024-04-22T23:35:06.401 INFO:tasks.workunit.client.0.smithi107.stdout:3/516: rmdir db/d21/d27/d36/d9c 39 2024-04-22T23:35:06.401 INFO:tasks.workunit.client.0.smithi107.stdout:3/517: dread - db/d21/d27/d15/d24/d87/faf zero size 2024-04-22T23:35:06.401 INFO:tasks.workunit.client.0.smithi107.stdout:3/518: write f0 [1644101,20941] 0 2024-04-22T23:35:06.401 INFO:tasks.workunit.client.0.smithi107.stdout:3/519: truncate db/f6f 1786785 0 2024-04-22T23:35:06.402 INFO:tasks.workunit.client.1.smithi157.stdout:6/998: chown d1/d2f/d3b/c6f 14240 1 2024-04-22T23:35:06.403 INFO:tasks.workunit.client.1.smithi157.stdout:6/999: mkdir d1/d36/dea/d14a 0 2024-04-22T23:35:06.404 INFO:tasks.workunit.client.0.smithi107.stdout:9/511: dread d5/d14/d2e/f84 [0,4194304] 0 2024-04-22T23:35:06.404 INFO:tasks.workunit.client.0.smithi107.stdout:9/512: dread - d5/d29/f8c zero size 2024-04-22T23:35:06.404 INFO:tasks.workunit.client.0.smithi107.stdout:9/513: write d5/d29/f3a [414366,47720] 0 2024-04-22T23:35:06.407 INFO:tasks.workunit.client.0.smithi107.stdout:7/401: creat d7/d21/d54/f83 x:0 0 0 2024-04-22T23:35:06.407 INFO:tasks.workunit.client.0.smithi107.stdout:7/402: write d7/d1b/d51/f5a [4251968,119401] 0 2024-04-22T23:35:06.412 INFO:tasks.workunit.client.1.smithi157.stdout:9/672: dwrite d4/d15/d55/d8d/fbb [0,4194304] 0 2024-04-22T23:35:06.412 INFO:tasks.workunit.client.1.smithi157.stdout:9/673: creat d4/d15/d23/d27/d47/fd2 x:0 0 0 2024-04-22T23:35:06.413 INFO:tasks.workunit.client.0.smithi107.stdout:8/439: dwrite d0/d7/d8/d41/d53/f57 [0,4194304] 0 2024-04-22T23:35:06.413 INFO:tasks.workunit.client.0.smithi107.stdout:4/501: getdents d1/d8 0 2024-04-22T23:35:06.413 INFO:tasks.workunit.client.0.smithi107.stdout:5/482: dwrite d9/d26/f3b [0,4194304] 0 2024-04-22T23:35:06.413 INFO:tasks.workunit.client.0.smithi107.stdout:5/483: truncate d9/db/de/d5e/f71 904563 0 2024-04-22T23:35:06.414 INFO:tasks.workunit.client.0.smithi107.stdout:9/514: write d5/f13 [3625015,103423] 0 2024-04-22T23:35:06.415 INFO:tasks.workunit.client.1.smithi157.stdout:9/674: unlink d4/df/d57/ca8 0 2024-04-22T23:35:06.415 INFO:tasks.workunit.client.1.smithi157.stdout:9/675: write d4/d15/d23/d27/d47/f7f [520331,34210] 0 2024-04-22T23:35:06.415 INFO:tasks.workunit.client.1.smithi157.stdout:9/676: truncate d4/d15/d23/d27/d47/f7f 918620 0 2024-04-22T23:35:06.415 INFO:tasks.workunit.client.1.smithi157.stdout:9/677: truncate d4/d15/d23/d27/fb1 923156 0 2024-04-22T23:35:06.416 INFO:tasks.workunit.client.1.smithi157.stdout:9/678: rename d4/df/d61/d9c/lb6 to d4/d15/d23/ld3 0 2024-04-22T23:35:06.416 INFO:tasks.workunit.client.1.smithi157.stdout:9/679: write d4/df/d57/d6b/dc4/fd1 [523478,56913] 0 2024-04-22T23:35:06.417 INFO:tasks.workunit.client.0.smithi107.stdout:7/403: symlink d7/d4c/l84 0 2024-04-22T23:35:06.420 INFO:tasks.workunit.client.0.smithi107.stdout:8/440: symlink d0/d7/d20/d2d/l90 0 2024-04-22T23:35:06.421 INFO:tasks.workunit.client.0.smithi107.stdout:4/502: truncate d1/d21/d29/d2e/f3d 1616879 0 2024-04-22T23:35:06.423 INFO:tasks.workunit.client.0.smithi107.stdout:9/515: mknod d5/d7/d27/d46/d4b/cb4 0 2024-04-22T23:35:06.423 INFO:tasks.workunit.client.0.smithi107.stdout:7/404: mknod d7/d1b/d51/c85 0 2024-04-22T23:35:06.423 INFO:tasks.workunit.client.0.smithi107.stdout:5/484: read d9/fd [3012634,124305] 0 2024-04-22T23:35:06.424 INFO:tasks.workunit.client.0.smithi107.stdout:8/441: creat d0/d7/df/d17/f91 x:0 0 0 2024-04-22T23:35:06.425 INFO:tasks.workunit.client.0.smithi107.stdout:4/503: truncate d1/d21/d29/f71 224603 0 2024-04-22T23:35:06.425 INFO:tasks.workunit.client.0.smithi107.stdout:4/504: dread - d1/d21/d29/d2e/d3b/f8a zero size 2024-04-22T23:35:06.425 INFO:tasks.workunit.client.0.smithi107.stdout:9/516: readlink d5/d7/l37 0 2024-04-22T23:35:06.425 INFO:tasks.workunit.client.0.smithi107.stdout:9/517: write f4 [1859012,40570] 0 2024-04-22T23:35:06.425 INFO:tasks.workunit.client.0.smithi107.stdout:9/518: creat d5/d29/d7b/d91/fb5 x:0 0 0 2024-04-22T23:35:06.427 INFO:tasks.workunit.client.0.smithi107.stdout:7/405: link d7/d2d/f43 d7/d21/d49/f86 0 2024-04-22T23:35:06.431 INFO:tasks.workunit.client.0.smithi107.stdout:9/519: rename d5/d7/d17/d1b/d1f/d8e to d5/d7/d27/d69/db6 0 2024-04-22T23:35:06.431 INFO:tasks.workunit.client.0.smithi107.stdout:9/520: truncate d5/d7/d27/d46/d4b/f56 1122411 0 2024-04-22T23:35:06.431 INFO:tasks.workunit.client.0.smithi107.stdout:9/521: readlink d5/d14/d2e/d4f/l63 0 2024-04-22T23:35:06.432 INFO:tasks.workunit.client.0.smithi107.stdout:0/437: sync 2024-04-22T23:35:06.433 INFO:tasks.workunit.client.0.smithi107.stdout:0/438: mkdir d1/d84/d99 0 2024-04-22T23:35:06.435 INFO:tasks.workunit.client.0.smithi107.stdout:0/439: symlink d1/d2a/l9a 0 2024-04-22T23:35:06.437 INFO:tasks.workunit.client.0.smithi107.stdout:0/440: symlink d1/d9/d14/d4f/l9b 0 2024-04-22T23:35:06.438 INFO:tasks.workunit.client.0.smithi107.stdout:0/441: rmdir d1/d27/d20/d4a 39 2024-04-22T23:35:06.446 INFO:tasks.workunit.client.0.smithi107.stdout:4/505: dread d1/d21/f6a [0,4194304] 0 2024-04-22T23:35:06.450 INFO:tasks.workunit.client.0.smithi107.stdout:4/506: dread d1/d21/d29/d2e/f46 [0,4194304] 0 2024-04-22T23:35:06.450 INFO:tasks.workunit.client.0.smithi107.stdout:4/507: write d1/d8/d56/d27/f30 [194228,16808] 0 2024-04-22T23:35:06.450 INFO:tasks.workunit.client.0.smithi107.stdout:4/508: fsync d1/d21/d52/f73 0 2024-04-22T23:35:06.451 INFO:tasks.workunit.client.0.smithi107.stdout:4/509: write d1/d8/d67/f8f [173132,49515] 0 2024-04-22T23:35:06.455 INFO:tasks.workunit.client.0.smithi107.stdout:4/510: dread d1/d21/d85/f9c [0,4194304] 0 2024-04-22T23:35:06.457 INFO:tasks.workunit.client.0.smithi107.stdout:5/485: dwrite d9/db/de/d5e/f61 [4194304,4194304] 0 2024-04-22T23:35:06.457 INFO:tasks.workunit.client.0.smithi107.stdout:4/511: write d1/d21/f6a [387830,47634] 0 2024-04-22T23:35:06.459 INFO:tasks.workunit.client.0.smithi107.stdout:4/512: read d1/d21/f76 [363208,71122] 0 2024-04-22T23:35:06.460 INFO:tasks.workunit.client.0.smithi107.stdout:4/513: creat d1/d8/d56/d27/fb3 x:0 0 0 2024-04-22T23:35:06.460 INFO:tasks.workunit.client.0.smithi107.stdout:4/514: write d1/d21/d29/d2e/fac [1011828,110076] 0 2024-04-22T23:35:06.461 INFO:tasks.workunit.client.0.smithi107.stdout:7/406: dwrite d7/d1b/d51/f62 [0,4194304] 0 2024-04-22T23:35:06.461 INFO:tasks.workunit.client.0.smithi107.stdout:7/407: dread - d7/f76 zero size 2024-04-22T23:35:06.461 INFO:tasks.workunit.client.0.smithi107.stdout:8/442: dwrite d0/d3a/d46/d59/f6e [0,4194304] 0 2024-04-22T23:35:06.461 INFO:tasks.workunit.client.0.smithi107.stdout:8/443: fsync d0/d7/df/d17/f91 0 2024-04-22T23:35:06.462 INFO:tasks.workunit.client.0.smithi107.stdout:0/442: dwrite d1/d9/d14/d4f/f82 [0,4194304] 0 2024-04-22T23:35:06.463 INFO:tasks.workunit.client.0.smithi107.stdout:5/486: dread f7 [0,4194304] 0 2024-04-22T23:35:06.463 INFO:tasks.workunit.client.0.smithi107.stdout:5/487: write d9/db/f8c [818976,15393] 0 2024-04-22T23:35:06.463 INFO:tasks.workunit.client.0.smithi107.stdout:5/488: write f7 [603835,77786] 0 2024-04-22T23:35:06.465 INFO:tasks.workunit.client.0.smithi107.stdout:8/444: mkdir d0/d7/d20/d92 0 2024-04-22T23:35:06.465 INFO:tasks.workunit.client.0.smithi107.stdout:8/445: chown d0/d7/df/d33/d37/f49 15444397 1 2024-04-22T23:35:06.476 INFO:tasks.workunit.client.0.smithi107.stdout:8/446: dread d0/d7/df/d33/d37/f49 [0,4194304] 0 2024-04-22T23:35:06.482 INFO:tasks.workunit.client.0.smithi107.stdout:5/489: dwrite d9/f58 [0,4194304] 0 2024-04-22T23:35:06.484 INFO:tasks.workunit.client.0.smithi107.stdout:4/515: dwrite d1/d8/f7f [0,4194304] 0 2024-04-22T23:35:06.484 INFO:tasks.workunit.client.0.smithi107.stdout:4/516: write d1/f5 [3895388,102362] 0 2024-04-22T23:35:06.504 INFO:tasks.workunit.client.1.smithi157.stdout:7/540: sync 2024-04-22T23:35:06.512 INFO:tasks.workunit.client.1.smithi157.stdout:7/541: dread d0/d9/d29/d33/d7c/f9c [0,4194304] 0 2024-04-22T23:35:06.512 INFO:tasks.workunit.client.1.smithi157.stdout:7/542: truncate d0/f7 1913440 0 2024-04-22T23:35:06.513 INFO:tasks.workunit.client.1.smithi157.stdout:7/543: chown d0/d9/d29/d49/f84 1461596932 1 2024-04-22T23:35:06.513 INFO:tasks.workunit.client.1.smithi157.stdout:7/544: readlink d0/d9/d29/l79 0 2024-04-22T23:35:06.514 INFO:tasks.workunit.client.1.smithi157.stdout:7/545: creat d0/d9/d29/d98/faa x:0 0 0 2024-04-22T23:35:06.515 INFO:tasks.workunit.client.0.smithi107.stdout:0/443: creat d1/d9/d14/f9c x:0 0 0 2024-04-22T23:35:06.515 INFO:tasks.workunit.client.0.smithi107.stdout:0/444: chown d1/d27/d20/d2c/d41/c89 59032 1 2024-04-22T23:35:06.520 INFO:tasks.workunit.client.0.smithi107.stdout:8/447: symlink d0/d3a/l93 0 2024-04-22T23:35:06.521 INFO:tasks.workunit.client.1.smithi157.stdout:7/546: dread d0/d9/f3c [0,4194304] 0 2024-04-22T23:35:06.522 INFO:tasks.workunit.client.1.smithi157.stdout:7/547: truncate d0/d9/d29/d5e/f65 150294 0 2024-04-22T23:35:06.522 INFO:tasks.workunit.client.0.smithi107.stdout:5/490: symlink d9/db/la7 0 2024-04-22T23:35:06.525 INFO:tasks.workunit.client.0.smithi107.stdout:4/517: rmdir d1/d21 39 2024-04-22T23:35:06.525 INFO:tasks.workunit.client.0.smithi107.stdout:4/518: fdatasync d1/d1b/f7a 0 2024-04-22T23:35:06.533 INFO:tasks.workunit.client.0.smithi107.stdout:0/445: rmdir d1/d27/d20/d40/d75 39 2024-04-22T23:35:06.534 INFO:tasks.workunit.client.0.smithi107.stdout:8/448: truncate d0/d7/d8/f6c 2025832 0 2024-04-22T23:35:06.536 INFO:tasks.workunit.client.1.smithi157.stdout:7/548: dwrite d0/d9/de/f5f [4194304,4194304] 0 2024-04-22T23:35:06.536 INFO:tasks.workunit.client.0.smithi107.stdout:5/491: rename d9/db/de/c16 to d9/d26/d6a/ca8 0 2024-04-22T23:35:06.537 INFO:tasks.workunit.client.1.smithi157.stdout:7/549: creat d0/d9/d42/d55/d56/d91/fab x:0 0 0 2024-04-22T23:35:06.538 INFO:tasks.workunit.client.0.smithi107.stdout:4/519: mkdir d1/d7/d63/d78/db4 0 2024-04-22T23:35:06.538 INFO:tasks.workunit.client.0.smithi107.stdout:4/520: read - d1/d7/d53/f84 zero size 2024-04-22T23:35:06.538 INFO:tasks.workunit.client.1.smithi157.stdout:7/550: mkdir d0/d9/db/df/dac 0 2024-04-22T23:35:06.539 INFO:tasks.workunit.client.0.smithi107.stdout:0/446: truncate d1/d9/d14/d4f/d59/f5c 3766605 0 2024-04-22T23:35:06.540 INFO:tasks.workunit.client.1.smithi157.stdout:7/551: write d0/d9/d10/f18 [341547,98163] 0 2024-04-22T23:35:06.541 INFO:tasks.workunit.client.1.smithi157.stdout:7/552: read d0/d9/de/f23 [331951,94445] 0 2024-04-22T23:35:06.542 INFO:tasks.workunit.client.1.smithi157.stdout:7/553: fsync d0/d9/d29/f4d 0 2024-04-22T23:35:06.542 INFO:tasks.workunit.client.0.smithi107.stdout:4/521: write d1/d21/d29/d2e/f46 [214315,82563] 0 2024-04-22T23:35:06.543 INFO:tasks.workunit.client.1.smithi157.stdout:7/554: getdents d0/d9/d10 0 2024-04-22T23:35:06.543 INFO:tasks.workunit.client.1.smithi157.stdout:7/555: chown d0/d9/db/c47 21506636 1 2024-04-22T23:35:06.549 INFO:tasks.workunit.client.0.smithi107.stdout:4/522: write d1/d21/d29/f39 [3955134,92988] 0 2024-04-22T23:35:06.551 INFO:tasks.workunit.client.0.smithi107.stdout:5/492: write d9/d26/f2c [3195442,110231] 0 2024-04-22T23:35:06.552 INFO:tasks.workunit.client.0.smithi107.stdout:8/449: dread d0/d7/df/d3b/f3d [0,4194304] 0 2024-04-22T23:35:06.552 INFO:tasks.workunit.client.0.smithi107.stdout:4/523: rename d1/d21/d29/d2e/f3d to d1/d21/d29/d2e/d88/d99/fb5 0 2024-04-22T23:35:06.553 INFO:tasks.workunit.client.0.smithi107.stdout:6/473: sync 2024-04-22T23:35:06.553 INFO:tasks.workunit.client.0.smithi107.stdout:0/447: write d1/d9/d14/d4f/f69 [4010629,104894] 0 2024-04-22T23:35:06.554 INFO:tasks.workunit.client.0.smithi107.stdout:5/493: creat d9/d76/da0/fa9 x:0 0 0 2024-04-22T23:35:06.554 INFO:tasks.workunit.client.0.smithi107.stdout:8/450: creat d0/d73/f94 x:0 0 0 2024-04-22T23:35:06.557 INFO:tasks.workunit.client.0.smithi107.stdout:4/524: rename d1/d1b/c1e to d1/cb6 0 2024-04-22T23:35:06.557 INFO:tasks.workunit.client.0.smithi107.stdout:4/525: dread - d1/d21/d52/f73 zero size 2024-04-22T23:35:06.557 INFO:tasks.workunit.client.0.smithi107.stdout:4/526: chown d1/f28 193 1 2024-04-22T23:35:06.559 INFO:tasks.workunit.client.0.smithi107.stdout:6/474: rmdir db/d20/d2a/d38/d59/d76 0 2024-04-22T23:35:06.560 INFO:tasks.workunit.client.0.smithi107.stdout:5/494: symlink d9/db/de/d72/d7c/laa 0 2024-04-22T23:35:06.561 INFO:tasks.workunit.client.0.smithi107.stdout:8/451: rmdir d0/d7/d20/d92 0 2024-04-22T23:35:06.562 INFO:tasks.workunit.client.0.smithi107.stdout:8/452: read d0/f4 [1541191,119395] 0 2024-04-22T23:35:06.563 INFO:tasks.workunit.client.0.smithi107.stdout:5/495: symlink d9/d26/d79/lab 0 2024-04-22T23:35:06.565 INFO:tasks.workunit.client.0.smithi107.stdout:8/453: mknod d0/d7/df/c95 0 2024-04-22T23:35:06.566 INFO:tasks.workunit.client.1.smithi157.stdout:7/556: dwrite d0/d9/f66 [0,4194304] 0 2024-04-22T23:35:06.566 INFO:tasks.workunit.client.1.smithi157.stdout:7/557: truncate d0/d9/f66 4986599 0 2024-04-22T23:35:06.566 INFO:tasks.workunit.client.1.smithi157.stdout:7/558: getdents d0/d9/d29/d5e/d9e 0 2024-04-22T23:35:06.566 INFO:tasks.workunit.client.1.smithi157.stdout:7/559: truncate d0/d9/d10/f30 700540 0 2024-04-22T23:35:06.567 INFO:tasks.workunit.client.0.smithi107.stdout:5/496: creat d9/db/de/d72/fac x:0 0 0 2024-04-22T23:35:06.568 INFO:tasks.workunit.client.0.smithi107.stdout:4/527: dread d1/d21/d29/f39 [0,4194304] 0 2024-04-22T23:35:06.570 INFO:tasks.workunit.client.0.smithi107.stdout:8/454: unlink d0/d7/df/d33/d37/l7f 0 2024-04-22T23:35:06.570 INFO:tasks.workunit.client.0.smithi107.stdout:8/455: fdatasync d0/d7/df/d3b/f3d 0 2024-04-22T23:35:06.570 INFO:tasks.workunit.client.0.smithi107.stdout:5/497: stat d9/l5d 0 2024-04-22T23:35:06.573 INFO:tasks.workunit.client.0.smithi107.stdout:4/528: creat d1/d7/d63/d78/d80/fb7 x:0 0 0 2024-04-22T23:35:06.574 INFO:tasks.workunit.client.0.smithi107.stdout:8/456: unlink d0/d7/df/d33/d37/c79 0 2024-04-22T23:35:06.575 INFO:tasks.workunit.client.0.smithi107.stdout:5/498: symlink d9/d26/lad 0 2024-04-22T23:35:06.577 INFO:tasks.workunit.client.0.smithi107.stdout:4/529: creat d1/d8/d56/da8/fb8 x:0 0 0 2024-04-22T23:35:06.578 INFO:tasks.workunit.client.0.smithi107.stdout:5/499: symlink d9/db/d42/lae 0 2024-04-22T23:35:06.579 INFO:tasks.workunit.client.0.smithi107.stdout:4/530: mkdir d1/d8/d56/da8/db9 0 2024-04-22T23:35:06.583 INFO:tasks.workunit.client.0.smithi107.stdout:4/531: getdents d1 0 2024-04-22T23:35:06.584 INFO:tasks.workunit.client.0.smithi107.stdout:0/448: dwrite d1/d27/d20/d2c/d2e/f71 [0,4194304] 0 2024-04-22T23:35:06.585 INFO:tasks.workunit.client.0.smithi107.stdout:4/532: symlink d1/d7/d63/d78/da4/lba 0 2024-04-22T23:35:06.586 INFO:tasks.workunit.client.0.smithi107.stdout:0/449: symlink d1/d7b/d85/l9d 0 2024-04-22T23:35:06.586 INFO:tasks.workunit.client.0.smithi107.stdout:0/450: getdents d1/d7c 0 2024-04-22T23:35:06.589 INFO:tasks.workunit.client.0.smithi107.stdout:5/500: dread d9/d26/f55 [0,4194304] 0 2024-04-22T23:35:06.589 INFO:tasks.workunit.client.0.smithi107.stdout:5/501: chown d9/db/l77 7 1 2024-04-22T23:35:06.589 INFO:tasks.workunit.client.0.smithi107.stdout:4/533: dread d1/f8d [0,4194304] 0 2024-04-22T23:35:06.591 INFO:tasks.workunit.client.0.smithi107.stdout:5/502: unlink d9/d76/c8b 0 2024-04-22T23:35:06.591 INFO:tasks.workunit.client.0.smithi107.stdout:5/503: chown d9/db/de/d72/d7c/da4 44233 1 2024-04-22T23:35:06.591 INFO:tasks.workunit.client.0.smithi107.stdout:5/504: creat d9/db/de/faf x:0 0 0 2024-04-22T23:35:06.592 INFO:tasks.workunit.client.1.smithi157.stdout:7/560: dwrite d0/d9/d42/f7a [4194304,4194304] 0 2024-04-22T23:35:06.592 INFO:tasks.workunit.client.1.smithi157.stdout:7/561: chown d0/d9/db/d35 6 1 2024-04-22T23:35:06.593 INFO:tasks.workunit.client.0.smithi107.stdout:5/505: rmdir d9/d26/d79 39 2024-04-22T23:35:06.594 INFO:tasks.workunit.client.0.smithi107.stdout:5/506: stat d9/db/d42/d75/c7e 0 2024-04-22T23:35:06.596 INFO:tasks.workunit.client.0.smithi107.stdout:5/507: rename d9/l33 to d9/d26/lb0 0 2024-04-22T23:35:06.608 INFO:tasks.workunit.client.0.smithi107.stdout:0/451: dread d1/d27/d20/d2c/d2e/f39 [0,4194304] 0 2024-04-22T23:35:06.610 INFO:tasks.workunit.client.0.smithi107.stdout:0/452: getdents d1 0 2024-04-22T23:35:06.610 INFO:tasks.workunit.client.0.smithi107.stdout:0/453: dread - d1/d27/d20/d2c/d41/f90 zero size 2024-04-22T23:35:06.611 INFO:tasks.workunit.client.0.smithi107.stdout:0/454: symlink d1/d84/d99/l9e 0 2024-04-22T23:35:06.614 INFO:tasks.workunit.client.1.smithi157.stdout:3/671: sync 2024-04-22T23:35:06.614 INFO:tasks.workunit.client.1.smithi157.stdout:5/676: sync 2024-04-22T23:35:06.614 INFO:tasks.workunit.client.1.smithi157.stdout:3/672: truncate d0/d3/d34/d3d/d57/d60/d9c/fc8 615695 0 2024-04-22T23:35:06.614 INFO:tasks.workunit.client.1.smithi157.stdout:5/677: creat d1/d5/dc/d5d/d77/f107 x:0 0 0 2024-04-22T23:35:06.614 INFO:tasks.workunit.client.1.smithi157.stdout:5/678: fdatasync d1/fe 0 2024-04-22T23:35:06.614 INFO:tasks.workunit.client.1.smithi157.stdout:5/679: write d1/d5/d7/dbd/ff4 [349597,28969] 0 2024-04-22T23:35:06.614 INFO:tasks.workunit.client.1.smithi157.stdout:5/680: chown d1/d5/d7/d10/c92 3377 1 2024-04-22T23:35:06.616 INFO:tasks.workunit.client.1.smithi157.stdout:5/681: unlink d1/d5/fb5 0 2024-04-22T23:35:06.618 INFO:tasks.workunit.client.1.smithi157.stdout:5/682: unlink d1/d5/d7/d10/c1c 0 2024-04-22T23:35:06.619 INFO:tasks.workunit.client.1.smithi157.stdout:5/683: truncate d1/d27/d29/f58 679667 0 2024-04-22T23:35:06.631 INFO:tasks.workunit.client.0.smithi107.stdout:0/455: dwrite d1/f15 [0,4194304] 0 2024-04-22T23:35:06.631 INFO:tasks.workunit.client.1.smithi157.stdout:3/673: dwrite d0/d3/d13/fd1 [0,4194304] 0 2024-04-22T23:35:06.633 INFO:tasks.workunit.client.1.smithi157.stdout:3/674: truncate d0/d3/d13/f2b 3293408 0 2024-04-22T23:35:06.635 INFO:tasks.workunit.client.1.smithi157.stdout:3/675: rename d0/d3/d34/db9/f7a to d0/d3/d34/db9/d9f/fdb 0 2024-04-22T23:35:06.635 INFO:tasks.workunit.client.1.smithi157.stdout:3/676: creat d0/d3/d34/d4a/fdc x:0 0 0 2024-04-22T23:35:06.642 INFO:tasks.workunit.client.1.smithi157.stdout:3/677: dread d0/d3/d34/d39/f56 [0,4194304] 0 2024-04-22T23:35:06.642 INFO:tasks.workunit.client.0.smithi107.stdout:0/456: read d1/d9/f5e [3675443,89130] 0 2024-04-22T23:35:06.659 INFO:tasks.workunit.client.1.smithi157.stdout:7/562: dread d0/d9/d29/d5e/f6a [0,4194304] 0 2024-04-22T23:35:06.664 INFO:tasks.workunit.client.1.smithi157.stdout:7/563: dread d0/d9/db/df/d68/f8e [0,4194304] 0 2024-04-22T23:35:06.670 INFO:tasks.workunit.client.1.smithi157.stdout:7/564: write d0/d9/d42/f7a [391422,128820] 0 2024-04-22T23:35:06.672 INFO:tasks.workunit.client.1.smithi157.stdout:7/565: getdents d0 0 2024-04-22T23:35:06.672 INFO:tasks.workunit.client.1.smithi157.stdout:7/566: truncate d0/d9/d42/d55/f78 673683 0 2024-04-22T23:35:06.672 INFO:tasks.workunit.client.1.smithi157.stdout:7/567: write d0/d9/db/f15 [1611023,103166] 0 2024-04-22T23:35:06.672 INFO:tasks.workunit.client.1.smithi157.stdout:7/568: write d0/d9/f1d [1341801,104191] 0 2024-04-22T23:35:06.673 INFO:tasks.workunit.client.1.smithi157.stdout:7/569: getdents d0/d9/de 0 2024-04-22T23:35:06.674 INFO:tasks.workunit.client.1.smithi157.stdout:7/570: creat d0/d9/db/df/dac/fad x:0 0 0 2024-04-22T23:35:06.675 INFO:tasks.workunit.client.1.smithi157.stdout:7/571: unlink d0/d9/d29/d5e/l6c 0 2024-04-22T23:35:06.675 INFO:tasks.workunit.client.1.smithi157.stdout:7/572: dread - d0/d9/d42/d55/d80/fa7 zero size 2024-04-22T23:35:06.676 INFO:tasks.workunit.client.1.smithi157.stdout:7/573: symlink d0/d9/d42/d55/d80/d82/lae 0 2024-04-22T23:35:06.676 INFO:tasks.workunit.client.1.smithi157.stdout:7/574: write d0/d9/d29/d49/d4a/f67 [883226,23175] 0 2024-04-22T23:35:06.676 INFO:tasks.workunit.client.1.smithi157.stdout:7/575: truncate d0/d9/d29/d5e/f65 295524 0 2024-04-22T23:35:06.677 INFO:tasks.workunit.client.1.smithi157.stdout:3/678: dwrite d0/d7/f7d [0,4194304] 0 2024-04-22T23:35:06.677 INFO:tasks.workunit.client.1.smithi157.stdout:3/679: write d0/d3/d34/d4a/fa8 [499452,45132] 0 2024-04-22T23:35:06.677 INFO:tasks.workunit.client.0.smithi107.stdout:8/457: dwrite d0/d7/df/d33/d37/f80 [0,4194304] 0 2024-04-22T23:35:06.677 INFO:tasks.workunit.client.0.smithi107.stdout:4/534: dwrite d1/d21/d29/d2e/d88/d99/fb0 [0,4194304] 0 2024-04-22T23:35:06.679 INFO:tasks.workunit.client.1.smithi157.stdout:5/684: dwrite d1/f90 [0,4194304] 0 2024-04-22T23:35:06.681 INFO:tasks.workunit.client.0.smithi107.stdout:0/457: dwrite d1/d9/f68 [4194304,4194304] 0 2024-04-22T23:35:06.684 INFO:tasks.workunit.client.0.smithi107.stdout:0/458: mknod d1/d9/d14/d4f/c9f 0 2024-04-22T23:35:06.685 INFO:tasks.workunit.client.1.smithi157.stdout:3/680: creat d0/d3/d34/d3d/d57/d5c/fdd x:0 0 0 2024-04-22T23:35:06.687 INFO:tasks.workunit.client.1.smithi157.stdout:5/685: write d1/d27/d29/fbf [806221,39105] 0 2024-04-22T23:35:06.687 INFO:tasks.workunit.client.0.smithi107.stdout:0/459: creat d1/d27/d20/d40/fa0 x:0 0 0 2024-04-22T23:35:06.688 INFO:tasks.workunit.client.1.smithi157.stdout:3/681: unlink d0/d3/d34/f5d 0 2024-04-22T23:35:06.689 INFO:tasks.workunit.client.1.smithi157.stdout:5/686: symlink d1/d5/dc/d43/l108 0 2024-04-22T23:35:06.692 INFO:tasks.workunit.client.1.smithi157.stdout:5/687: unlink d1/d5/d7/d2f/d45/f48 0 2024-04-22T23:35:06.696 INFO:tasks.workunit.client.0.smithi107.stdout:0/460: rename d1/d9/d14/d4f/f69 to d1/d84/d99/fa1 0 2024-04-22T23:35:06.698 INFO:tasks.workunit.client.0.smithi107.stdout:0/461: creat d1/d9/d14/d4f/fa2 x:0 0 0 2024-04-22T23:35:06.699 INFO:tasks.workunit.client.0.smithi107.stdout:4/535: dwrite d1/d21/d29/d2e/d3b/f7d [0,4194304] 0 2024-04-22T23:35:06.699 INFO:tasks.workunit.client.0.smithi107.stdout:4/536: fsync d1/d7/d63/d78/d80/f82 0 2024-04-22T23:35:06.700 INFO:tasks.workunit.client.0.smithi107.stdout:0/462: creat d1/d67/fa3 x:0 0 0 2024-04-22T23:35:06.700 INFO:tasks.workunit.client.1.smithi157.stdout:0/600: sync 2024-04-22T23:35:06.702 INFO:tasks.workunit.client.0.smithi107.stdout:1/459: sync 2024-04-22T23:35:06.702 INFO:tasks.workunit.client.0.smithi107.stdout:2/514: sync 2024-04-22T23:35:06.702 INFO:tasks.workunit.client.0.smithi107.stdout:9/522: sync 2024-04-22T23:35:06.703 INFO:tasks.workunit.client.0.smithi107.stdout:1/460: readlink d7/d19/d2d/l6c 0 2024-04-22T23:35:06.703 INFO:tasks.workunit.client.0.smithi107.stdout:9/523: chown d5/d29/f4d 63 1 2024-04-22T23:35:06.703 INFO:tasks.workunit.client.0.smithi107.stdout:3/520: sync 2024-04-22T23:35:06.703 INFO:tasks.workunit.client.0.smithi107.stdout:9/524: chown d5/d7/d27/d46/f74 155 1 2024-04-22T23:35:06.703 INFO:tasks.workunit.client.0.smithi107.stdout:3/521: readlink db/d21/d27/d15/d24/d52/d60/d96/l9a 0 2024-04-22T23:35:06.703 INFO:tasks.workunit.client.0.smithi107.stdout:7/408: sync 2024-04-22T23:35:06.703 INFO:tasks.workunit.client.0.smithi107.stdout:7/409: stat d7/d21/d48 0 2024-04-22T23:35:06.704 INFO:tasks.workunit.client.1.smithi157.stdout:0/601: symlink d17/d2b/d71/da5/ld0 0 2024-04-22T23:35:06.704 INFO:tasks.workunit.client.0.smithi107.stdout:4/537: creat d1/d8/d56/da8/db9/fbb x:0 0 0 2024-04-22T23:35:06.704 INFO:tasks.workunit.client.0.smithi107.stdout:4/538: truncate d1/d8/d67/f8f 293256 0 2024-04-22T23:35:06.704 INFO:tasks.workunit.client.0.smithi107.stdout:8/458: dwrite d0/d7/df/f23 [0,4194304] 0 2024-04-22T23:35:06.704 INFO:tasks.workunit.client.0.smithi107.stdout:8/459: readlink d0/d85/l54 0 2024-04-22T23:35:06.705 INFO:tasks.workunit.client.1.smithi157.stdout:0/602: rmdir d17 39 2024-04-22T23:35:06.705 INFO:tasks.workunit.client.1.smithi157.stdout:0/603: write d17/d23/d34/d67/d7f/f74 [935633,23344] 0 2024-04-22T23:35:06.705 INFO:tasks.workunit.client.0.smithi107.stdout:3/522: rename db/d21/d27/l10 to db/d21/d27/d15/d24/d52/lb3 0 2024-04-22T23:35:06.706 INFO:tasks.workunit.client.1.smithi157.stdout:3/682: dwrite d0/d3/d34/d3d/d57/f65 [0,4194304] 0 2024-04-22T23:35:06.707 INFO:tasks.workunit.client.0.smithi107.stdout:4/539: creat d1/d7/d63/d78/d96/fbc x:0 0 0 2024-04-22T23:35:06.712 INFO:tasks.workunit.client.1.smithi157.stdout:3/683: symlink d0/d3/d34/d62/d9a/d9e/lde 0 2024-04-22T23:35:06.716 INFO:tasks.workunit.client.1.smithi157.stdout:3/684: mknod d0/d7/d6c/cdf 0 2024-04-22T23:35:06.716 INFO:tasks.workunit.client.0.smithi107.stdout:8/460: creat d0/f96 x:0 0 0 2024-04-22T23:35:06.718 INFO:tasks.workunit.client.0.smithi107.stdout:3/523: creat db/d21/d27/d15/fb4 x:0 0 0 2024-04-22T23:35:06.718 INFO:tasks.workunit.client.1.smithi157.stdout:3/685: creat d0/d7/fe0 x:0 0 0 2024-04-22T23:35:06.719 INFO:tasks.workunit.client.1.smithi157.stdout:5/688: dread d1/fe [0,4194304] 0 2024-04-22T23:35:06.721 INFO:tasks.workunit.client.0.smithi107.stdout:9/525: write d5/f10 [8363801,125989] 0 2024-04-22T23:35:06.721 INFO:tasks.workunit.client.0.smithi107.stdout:9/526: write d5/d14/d28/f59 [4950372,100309] 0 2024-04-22T23:35:06.721 INFO:tasks.workunit.client.0.smithi107.stdout:9/527: chown d5/d7/d17/d1b 1622 1 2024-04-22T23:35:06.721 INFO:tasks.workunit.client.0.smithi107.stdout:9/528: creat d5/d14/d2e/d4f/fb7 x:0 0 0 2024-04-22T23:35:06.722 INFO:tasks.workunit.client.1.smithi157.stdout:3/686: link d0/d3/d34/d3d/d57/d60/c8a d0/d3/d34/d3d/d57/d60/d6b/ce1 0 2024-04-22T23:35:06.722 INFO:tasks.workunit.client.1.smithi157.stdout:3/687: creat d0/d3/d34/d62/d9a/d9e/fe2 x:0 0 0 2024-04-22T23:35:06.722 INFO:tasks.workunit.client.0.smithi107.stdout:4/540: link d1/d8/d67/f9e d1/fbd 0 2024-04-22T23:35:06.722 INFO:tasks.workunit.client.0.smithi107.stdout:8/461: mknod d0/d3a/d46/d59/d78/c97 0 2024-04-22T23:35:06.724 INFO:tasks.workunit.client.0.smithi107.stdout:8/462: symlink d0/d7/df/d3b/d7b/l98 0 2024-04-22T23:35:06.726 INFO:tasks.workunit.client.1.smithi157.stdout:3/688: getdents d0 0 2024-04-22T23:35:06.727 INFO:tasks.workunit.client.0.smithi107.stdout:3/524: write db/d21/d27/d15/d29/f6d [560510,91662] 0 2024-04-22T23:35:06.729 INFO:tasks.workunit.client.0.smithi107.stdout:3/525: getdents db/d21/d27/d91 0 2024-04-22T23:35:06.729 INFO:tasks.workunit.client.0.smithi107.stdout:3/526: fdatasync db/d21/d27/d15/d29/d2a/d99/fa8 0 2024-04-22T23:35:06.729 INFO:tasks.workunit.client.0.smithi107.stdout:3/527: truncate db/d21/d27/d15/fb4 665351 0 2024-04-22T23:35:06.731 INFO:tasks.workunit.client.0.smithi107.stdout:8/463: dread d0/d7/df/d33/d37/f49 [0,4194304] 0 2024-04-22T23:35:06.732 INFO:tasks.workunit.client.0.smithi107.stdout:3/528: rename db/d21/d27/d15/l68 to db/d21/d27/d54/da7/lb5 0 2024-04-22T23:35:06.734 INFO:tasks.workunit.client.0.smithi107.stdout:3/529: creat db/d21/d27/d54/d25/fb6 x:0 0 0 2024-04-22T23:35:06.738 INFO:tasks.workunit.client.0.smithi107.stdout:9/529: dread d5/d29/f3a [0,4194304] 0 2024-04-22T23:35:06.739 INFO:tasks.workunit.client.0.smithi107.stdout:2/515: dwrite d1/d2b/d3d/d57/f5c [0,4194304] 0 2024-04-22T23:35:06.739 INFO:tasks.workunit.client.0.smithi107.stdout:7/410: dwrite d7/f18 [4194304,4194304] 0 2024-04-22T23:35:06.740 INFO:tasks.workunit.client.0.smithi107.stdout:7/411: creat d7/d21/f87 x:0 0 0 2024-04-22T23:35:06.740 INFO:tasks.workunit.client.0.smithi107.stdout:7/412: write d7/d2d/f33 [684630,113789] 0 2024-04-22T23:35:06.742 INFO:tasks.workunit.client.0.smithi107.stdout:9/530: creat d5/d7/d17/d1b/d1f/d36/fb8 x:0 0 0 2024-04-22T23:35:06.742 INFO:tasks.workunit.client.1.smithi157.stdout:0/604: dwrite d17/d38/f80 [4194304,4194304] 0 2024-04-22T23:35:06.744 INFO:tasks.workunit.client.0.smithi107.stdout:4/541: dread d1/d7/fc [0,4194304] 0 2024-04-22T23:35:06.744 INFO:tasks.workunit.client.0.smithi107.stdout:4/542: readlink d1/d8/l4a 0 2024-04-22T23:35:06.744 INFO:tasks.workunit.client.0.smithi107.stdout:0/463: dwrite d1/d9/d14/f4d [0,4194304] 0 2024-04-22T23:35:06.744 INFO:tasks.workunit.client.0.smithi107.stdout:0/464: read - d1/d27/d20/d2c/d41/f90 zero size 2024-04-22T23:35:06.744 INFO:tasks.workunit.client.0.smithi107.stdout:3/530: dread db/f6f [0,4194304] 0 2024-04-22T23:35:06.745 INFO:tasks.workunit.client.1.smithi157.stdout:0/605: mkdir d17/d2b/d52/dd1 0 2024-04-22T23:35:06.746 INFO:tasks.workunit.client.0.smithi107.stdout:5/508: sync 2024-04-22T23:35:06.747 INFO:tasks.workunit.client.0.smithi107.stdout:6/475: sync 2024-04-22T23:35:06.747 INFO:tasks.workunit.client.0.smithi107.stdout:1/461: dwrite d7/d19/d20/d39/d53/f33 [0,4194304] 0 2024-04-22T23:35:06.749 INFO:tasks.workunit.client.0.smithi107.stdout:2/516: creat d1/d12/d35/d48/fb3 x:0 0 0 2024-04-22T23:35:06.751 INFO:tasks.workunit.client.0.smithi107.stdout:7/413: truncate d7/d1b/d51/f62 2611792 0 2024-04-22T23:35:06.752 INFO:tasks.workunit.client.0.smithi107.stdout:9/531: rmdir d5/d7/d27/d69/db6 39 2024-04-22T23:35:06.754 INFO:tasks.workunit.client.0.smithi107.stdout:4/543: link d1/d21/d85/f9c d1/d1b/d44/fbe 0 2024-04-22T23:35:06.756 INFO:tasks.workunit.client.1.smithi157.stdout:5/689: dwrite d1/d27/d29/d4b/f4d [0,4194304] 0 2024-04-22T23:35:06.756 INFO:tasks.workunit.client.1.smithi157.stdout:5/690: dread - d1/d5/f4e zero size 2024-04-22T23:35:06.756 INFO:tasks.workunit.client.1.smithi157.stdout:5/691: chown d1/d27/d29/d4b/f63 15 1 2024-04-22T23:35:06.756 INFO:tasks.workunit.client.0.smithi107.stdout:7/414: dread d7/d2d/f43 [0,4194304] 0 2024-04-22T23:35:06.758 INFO:tasks.workunit.client.0.smithi107.stdout:0/465: creat d1/d27/d8a/d8b/fa4 x:0 0 0 2024-04-22T23:35:06.758 INFO:tasks.workunit.client.0.smithi107.stdout:0/466: chown d1/d27/d8a/d8b 341032 1 2024-04-22T23:35:06.759 INFO:tasks.workunit.client.1.smithi157.stdout:3/689: dwrite d0/d3/d34/d3d/f42 [0,4194304] 0 2024-04-22T23:35:06.760 INFO:tasks.workunit.client.0.smithi107.stdout:3/531: rename db/d21/d27/d15/d29/d2a/d45/l7a to db/d21/d27/d91/lb7 0 2024-04-22T23:35:06.764 INFO:tasks.workunit.client.0.smithi107.stdout:5/509: link f3 d9/db/d42/d75/fb1 0 2024-04-22T23:35:06.764 INFO:tasks.workunit.client.0.smithi107.stdout:5/510: read d9/f12 [1579417,25021] 0 2024-04-22T23:35:06.764 INFO:tasks.workunit.client.0.smithi107.stdout:5/511: creat d9/fb2 x:0 0 0 2024-04-22T23:35:06.764 INFO:tasks.workunit.client.1.smithi157.stdout:5/692: creat d1/d5/dc/d43/f109 x:0 0 0 2024-04-22T23:35:06.765 INFO:tasks.workunit.client.1.smithi157.stdout:5/693: symlink d1/d5/dc/l10a 0 2024-04-22T23:35:06.766 INFO:tasks.workunit.client.0.smithi107.stdout:2/517: creat d1/d2b/d49/d7e/fb4 x:0 0 0 2024-04-22T23:35:06.766 INFO:tasks.workunit.client.0.smithi107.stdout:2/518: fsync d1/d63/f69 0 2024-04-22T23:35:06.768 INFO:tasks.workunit.client.0.smithi107.stdout:8/464: dwrite d0/d3a/f56 [0,4194304] 0 2024-04-22T23:35:06.768 INFO:tasks.workunit.client.0.smithi107.stdout:9/532: creat d5/d7/fb9 x:0 0 0 2024-04-22T23:35:06.772 INFO:tasks.workunit.client.1.smithi157.stdout:5/694: rename d1/d5/dc/d5d/d77/fad to d1/d5/f10b 0 2024-04-22T23:35:06.775 INFO:tasks.workunit.client.1.smithi157.stdout:5/695: creat d1/d27/d29/f10c x:0 0 0 2024-04-22T23:35:06.775 INFO:tasks.workunit.client.1.smithi157.stdout:5/696: write d1/d5/fd0 [595158,14095] 0 2024-04-22T23:35:06.775 INFO:tasks.workunit.client.1.smithi157.stdout:5/697: dread - d1/db0/dff/d102/f106 zero size 2024-04-22T23:35:06.775 INFO:tasks.workunit.client.0.smithi107.stdout:5/512: link d9/db/d42/f5a d9/db/de/d72/d7c/da4/fb3 0 2024-04-22T23:35:06.776 INFO:tasks.workunit.client.0.smithi107.stdout:5/513: dread - d9/d26/f85 zero size 2024-04-22T23:35:06.777 INFO:tasks.workunit.client.1.smithi157.stdout:5/698: symlink d1/d27/d29/l10d 0 2024-04-22T23:35:06.777 INFO:tasks.workunit.client.1.smithi157.stdout:5/699: chown d1/db0/d7a/f87 90 1 2024-04-22T23:35:06.778 INFO:tasks.workunit.client.0.smithi107.stdout:2/519: truncate d1/d2b/f94 1872613 0 2024-04-22T23:35:06.778 INFO:tasks.workunit.client.0.smithi107.stdout:2/520: dread - d1/d2b/f80 zero size 2024-04-22T23:35:06.782 INFO:tasks.workunit.client.1.smithi157.stdout:5/700: mkdir d1/d5/dc/d43/d4f/d10e 0 2024-04-22T23:35:06.782 INFO:tasks.workunit.client.1.smithi157.stdout:5/701: rmdir d1/d27/d29/d4b 39 2024-04-22T23:35:06.782 INFO:tasks.workunit.client.0.smithi107.stdout:8/465: mknod d0/d7/df/d3b/c99 0 2024-04-22T23:35:06.785 INFO:tasks.workunit.client.0.smithi107.stdout:5/514: mknod d9/db/d92/cb4 0 2024-04-22T23:35:06.786 INFO:tasks.workunit.client.0.smithi107.stdout:2/521: link d1/d12/d35/d48/c2e d1/d67/cb5 0 2024-04-22T23:35:06.786 INFO:tasks.workunit.client.0.smithi107.stdout:2/522: creat d1/d63/fb6 x:0 0 0 2024-04-22T23:35:06.787 INFO:tasks.workunit.client.0.smithi107.stdout:2/523: read d1/d67/f6f [510465,54152] 0 2024-04-22T23:35:06.787 INFO:tasks.workunit.client.0.smithi107.stdout:8/466: creat d0/d7/d8/d41/f9a x:0 0 0 2024-04-22T23:35:06.787 INFO:tasks.workunit.client.0.smithi107.stdout:2/524: write d1/d63/f65 [1667353,46595] 0 2024-04-22T23:35:06.788 INFO:tasks.workunit.client.0.smithi107.stdout:4/544: rename d1/d8/d56 to d1/d7/d5b/dbf 0 2024-04-22T23:35:06.789 INFO:tasks.workunit.client.0.smithi107.stdout:5/515: symlink d9/db/d42/lb5 0 2024-04-22T23:35:06.792 INFO:tasks.workunit.client.0.smithi107.stdout:8/467: mkdir d0/d7/df/d33/d9b 0 2024-04-22T23:35:06.792 INFO:tasks.workunit.client.1.smithi157.stdout:0/606: dwrite d17/d23/f51 [0,4194304] 0 2024-04-22T23:35:06.794 INFO:tasks.workunit.client.0.smithi107.stdout:7/415: rename d7/d21/d48/d5c/f64 to d7/d2d/f88 0 2024-04-22T23:35:06.794 INFO:tasks.workunit.client.0.smithi107.stdout:7/416: fdatasync d7/f11 0 2024-04-22T23:35:06.794 INFO:tasks.workunit.client.0.smithi107.stdout:7/417: fsync d7/f11 0 2024-04-22T23:35:06.794 INFO:tasks.workunit.client.0.smithi107.stdout:7/418: chown d7/d1b/f68 58432070 1 2024-04-22T23:35:06.794 INFO:tasks.workunit.client.0.smithi107.stdout:7/419: creat d7/d21/f89 x:0 0 0 2024-04-22T23:35:06.794 INFO:tasks.workunit.client.0.smithi107.stdout:7/420: chown d7/d21/f7f 9175 1 2024-04-22T23:35:06.794 INFO:tasks.workunit.client.0.smithi107.stdout:7/421: write d7/d21/f87 [938264,117352] 0 2024-04-22T23:35:06.795 INFO:tasks.workunit.client.0.smithi107.stdout:2/525: getdents d1/d12/d35/d48 0 2024-04-22T23:35:06.796 INFO:tasks.workunit.client.1.smithi157.stdout:0/607: mkdir d17/d2b/d4f/d5a/dd2 0 2024-04-22T23:35:06.796 INFO:tasks.workunit.client.0.smithi107.stdout:8/468: mknod d0/d73/c9c 0 2024-04-22T23:35:06.797 INFO:tasks.workunit.client.0.smithi107.stdout:6/476: dwrite db/d20/d2a/d38/d4b/f54 [0,4194304] 0 2024-04-22T23:35:06.797 INFO:tasks.workunit.client.1.smithi157.stdout:0/608: rmdir d17/d2b/d5b 39 2024-04-22T23:35:06.798 INFO:tasks.workunit.client.1.smithi157.stdout:0/609: write d17/d23/d9c/fc3 [604995,53984] 0 2024-04-22T23:35:06.798 INFO:tasks.workunit.client.1.smithi157.stdout:0/610: chown d17/d23/d34/d67/d7f/d69/f89 9571790 1 2024-04-22T23:35:06.798 INFO:tasks.workunit.client.0.smithi107.stdout:1/462: dwrite d7/d19/d20/d39/d53/d43/f7a [0,4194304] 0 2024-04-22T23:35:06.798 INFO:tasks.workunit.client.0.smithi107.stdout:7/422: symlink d7/d4c/l8a 0 2024-04-22T23:35:06.798 INFO:tasks.workunit.client.0.smithi107.stdout:4/545: write d1/d7/f11 [655163,56377] 0 2024-04-22T23:35:06.801 INFO:tasks.workunit.client.0.smithi107.stdout:2/526: symlink d1/d12/d35/lb7 0 2024-04-22T23:35:06.805 INFO:tasks.workunit.client.0.smithi107.stdout:7/423: mknod d7/d21/d49/c8b 0 2024-04-22T23:35:06.806 INFO:tasks.workunit.client.0.smithi107.stdout:5/516: read d9/db/f5f [3529393,62433] 0 2024-04-22T23:35:06.806 INFO:tasks.workunit.client.0.smithi107.stdout:5/517: readlink d9/d26/d79/l89 0 2024-04-22T23:35:06.807 INFO:tasks.workunit.client.0.smithi107.stdout:6/477: dread db/d20/d2c/d3b/d40/f71 [0,4194304] 0 2024-04-22T23:35:06.809 INFO:tasks.workunit.client.0.smithi107.stdout:4/546: unlink d1/d8/l3a 0 2024-04-22T23:35:06.809 INFO:tasks.workunit.client.0.smithi107.stdout:2/527: creat d1/d2b/d49/d7e/da8/fb8 x:0 0 0 2024-04-22T23:35:06.811 INFO:tasks.workunit.client.0.smithi107.stdout:5/518: rename d9/d26/d96 to d9/db/d42/d75/db6 0 2024-04-22T23:35:06.812 INFO:tasks.workunit.client.0.smithi107.stdout:4/547: creat d1/d7/d63/d78/db4/fc0 x:0 0 0 2024-04-22T23:35:06.821 INFO:tasks.workunit.client.0.smithi107.stdout:5/519: dread f4 [0,4194304] 0 2024-04-22T23:35:06.821 INFO:tasks.workunit.client.0.smithi107.stdout:8/469: dread d0/d7/df/f5c [0,4194304] 0 2024-04-22T23:35:06.821 INFO:tasks.workunit.client.0.smithi107.stdout:8/470: creat d0/d7/df/d3b/f9d x:0 0 0 2024-04-22T23:35:06.822 INFO:tasks.workunit.client.0.smithi107.stdout:6/478: write db/d20/d2a/f2f [414099,7989] 0 2024-04-22T23:35:06.822 INFO:tasks.workunit.client.0.smithi107.stdout:9/533: dwrite d5/d7/f70 [0,4194304] 0 2024-04-22T23:35:06.822 INFO:tasks.workunit.client.0.smithi107.stdout:5/520: mknod d9/db/cb7 0 2024-04-22T23:35:06.823 INFO:tasks.workunit.client.0.smithi107.stdout:5/521: truncate d9/db/de/d5e/f66 985787 0 2024-04-22T23:35:06.824 INFO:tasks.workunit.client.0.smithi107.stdout:8/471: symlink d0/d7/df/l9e 0 2024-04-22T23:35:06.826 INFO:tasks.workunit.client.1.smithi157.stdout:3/690: dwrite d0/d3/d34/db9/f7b [0,4194304] 0 2024-04-22T23:35:06.827 INFO:tasks.workunit.client.0.smithi107.stdout:6/479: unlink db/d20/d58/l64 0 2024-04-22T23:35:06.829 INFO:tasks.workunit.client.0.smithi107.stdout:9/534: link d5/d14/d2e/d43/c44 d5/d7/d27/d69/cba 0 2024-04-22T23:35:06.834 INFO:tasks.workunit.client.0.smithi107.stdout:6/480: unlink db/l6c 0 2024-04-22T23:35:06.834 INFO:tasks.workunit.client.0.smithi107.stdout:6/481: chown db/d20/d49/d7b 75519 1 2024-04-22T23:35:06.834 INFO:tasks.workunit.client.0.smithi107.stdout:6/482: creat db/f8c x:0 0 0 2024-04-22T23:35:06.834 INFO:tasks.workunit.client.0.smithi107.stdout:9/535: mkdir d5/d29/dbb 0 2024-04-22T23:35:06.835 INFO:tasks.workunit.client.0.smithi107.stdout:5/522: getdents d9/d76/da0 0 2024-04-22T23:35:06.836 INFO:tasks.workunit.client.0.smithi107.stdout:8/472: dread d0/d7/df/d3b/f2b [0,4194304] 0 2024-04-22T23:35:06.837 INFO:tasks.workunit.client.0.smithi107.stdout:8/473: fsync d0/f72 0 2024-04-22T23:35:06.838 INFO:tasks.workunit.client.0.smithi107.stdout:8/474: mkdir d0/d7/df/d33/d37/d9f 0 2024-04-22T23:35:06.838 INFO:tasks.workunit.client.0.smithi107.stdout:8/475: stat d0/d7/df/d3b/d7b/l98 0 2024-04-22T23:35:06.839 INFO:tasks.workunit.client.0.smithi107.stdout:8/476: write d0/d7/df/d17/f5b [163417,112645] 0 2024-04-22T23:35:06.839 INFO:tasks.workunit.client.0.smithi107.stdout:8/477: chown d0/d7/df/d33/d37 2 1 2024-04-22T23:35:06.839 INFO:tasks.workunit.client.0.smithi107.stdout:8/478: mknod d0/d7/df/d33/d9b/ca0 0 2024-04-22T23:35:06.839 INFO:tasks.workunit.client.0.smithi107.stdout:8/479: read - d0/d3a/d46/f70 zero size 2024-04-22T23:35:06.840 INFO:tasks.workunit.client.0.smithi107.stdout:9/536: dread d5/d14/d28/f81 [0,4194304] 0 2024-04-22T23:35:06.840 INFO:tasks.workunit.client.0.smithi107.stdout:9/537: truncate d5/d24/f7d 564506 0 2024-04-22T23:35:06.840 INFO:tasks.workunit.client.0.smithi107.stdout:8/480: mknod d0/d7/df/d3b/d7b/ca1 0 2024-04-22T23:35:06.841 INFO:tasks.workunit.client.0.smithi107.stdout:8/481: chown d0/d85/d51/l58 3401 1 2024-04-22T23:35:06.841 INFO:tasks.workunit.client.0.smithi107.stdout:8/482: fdatasync d0/d7/d8/d41/d53/f77 0 2024-04-22T23:35:06.841 INFO:tasks.workunit.client.0.smithi107.stdout:8/483: truncate d0/f5f 35711 0 2024-04-22T23:35:06.841 INFO:tasks.workunit.client.0.smithi107.stdout:8/484: write d0/d7/d8/f1e [840544,128073] 0 2024-04-22T23:35:06.842 INFO:tasks.workunit.client.0.smithi107.stdout:9/538: symlink d5/d7/d27/d69/db6/da8/lbc 0 2024-04-22T23:35:06.842 INFO:tasks.workunit.client.0.smithi107.stdout:9/539: chown d5/d14/d2e/d4f/d73/f9b 16377740 1 2024-04-22T23:35:06.844 INFO:tasks.workunit.client.0.smithi107.stdout:7/424: dwrite d7/f5e [4194304,4194304] 0 2024-04-22T23:35:06.845 INFO:tasks.workunit.client.0.smithi107.stdout:9/540: unlink d5/d7/d17/d1b/d1f/d36/c3d 0 2024-04-22T23:35:06.845 INFO:tasks.workunit.client.0.smithi107.stdout:9/541: creat d5/d7/d27/d46/fbd x:0 0 0 2024-04-22T23:35:06.845 INFO:tasks.workunit.client.0.smithi107.stdout:0/467: dwrite d1/d27/d20/f36 [0,4194304] 0 2024-04-22T23:35:06.845 INFO:tasks.workunit.client.0.smithi107.stdout:3/532: dwrite db/d21/d27/d54/d25/d53/f7b [0,4194304] 0 2024-04-22T23:35:06.845 INFO:tasks.workunit.client.0.smithi107.stdout:3/533: fdatasync db/d21/d27/d15/d29/d2a/f43 0 2024-04-22T23:35:06.845 INFO:tasks.workunit.client.1.smithi157.stdout:5/702: dwrite d1/d5/dc/f62 [0,4194304] 0 2024-04-22T23:35:06.848 INFO:tasks.workunit.client.1.smithi157.stdout:5/703: getdents d1/d5/d7/dbd 0 2024-04-22T23:35:06.849 INFO:tasks.workunit.client.0.smithi107.stdout:6/483: dread db/d20/d2a/d38/d59/f5a [0,4194304] 0 2024-04-22T23:35:06.851 INFO:tasks.workunit.client.0.smithi107.stdout:4/548: dwrite d1/d21/d52/f73 [0,4194304] 0 2024-04-22T23:35:06.852 INFO:tasks.workunit.client.0.smithi107.stdout:4/549: fsync d1/d21/d52/f97 0 2024-04-22T23:35:06.853 INFO:tasks.workunit.client.0.smithi107.stdout:9/542: symlink d5/d14/d2e/d43/lbe 0 2024-04-22T23:35:06.853 INFO:tasks.workunit.client.0.smithi107.stdout:9/543: dread - d5/d29/f6d zero size 2024-04-22T23:35:06.854 INFO:tasks.workunit.client.1.smithi157.stdout:3/691: dwrite d0/d3/d34/d3d/d5b/d72/fb1 [0,4194304] 0 2024-04-22T23:35:06.856 INFO:tasks.workunit.client.0.smithi107.stdout:0/468: unlink d1/d2a/c51 0 2024-04-22T23:35:06.857 INFO:tasks.workunit.client.1.smithi157.stdout:0/611: dwrite d17/d2b/f84 [0,4194304] 0 2024-04-22T23:35:06.857 INFO:tasks.workunit.client.1.smithi157.stdout:0/612: creat d17/d2b/d52/d7e/fd3 x:0 0 0 2024-04-22T23:35:06.857 INFO:tasks.workunit.client.1.smithi157.stdout:0/613: chown d17/d38 476674172 1 2024-04-22T23:35:06.857 INFO:tasks.workunit.client.1.smithi157.stdout:0/614: fdatasync d17/d23/fbf 0 2024-04-22T23:35:06.866 INFO:tasks.workunit.client.0.smithi107.stdout:5/523: dwrite d9/f58 [4194304,4194304] 0 2024-04-22T23:35:06.867 INFO:tasks.workunit.client.1.smithi157.stdout:3/692: dread d0/d3/d13/f2b [0,4194304] 0 2024-04-22T23:35:06.867 INFO:tasks.workunit.client.1.smithi157.stdout:3/693: write d0/d3/d34/d4a/fd7 [230867,108309] 0 2024-04-22T23:35:06.867 INFO:tasks.workunit.client.1.smithi157.stdout:3/694: dread - d0/d3/d34/d3d/d5b/d72/f79 zero size 2024-04-22T23:35:06.867 INFO:tasks.workunit.client.1.smithi157.stdout:3/695: readlink d0/d3/d34/d47/la1 0 2024-04-22T23:35:06.867 INFO:tasks.workunit.client.1.smithi157.stdout:3/696: chown d0/d3/cba 121587 1 2024-04-22T23:35:06.867 INFO:tasks.workunit.client.1.smithi157.stdout:5/704: dwrite d1/d5/d7/d2f/d45/d54/d59/f70 [0,4194304] 0 2024-04-22T23:35:06.870 INFO:tasks.workunit.client.1.smithi157.stdout:3/697: rename d0/d3/d34/d62/d9a/d9e/lde to d0/d3/d34/d4a/le3 0 2024-04-22T23:35:06.870 INFO:tasks.workunit.client.0.smithi107.stdout:5/524: dread f7 [0,4194304] 0 2024-04-22T23:35:06.872 INFO:tasks.workunit.client.1.smithi157.stdout:5/705: creat d1/d5/f10f x:0 0 0 2024-04-22T23:35:06.873 INFO:tasks.workunit.client.1.smithi157.stdout:5/706: chown d1/d5/d7/d2f/d45/d54/f82 7 1 2024-04-22T23:35:06.873 INFO:tasks.workunit.client.1.smithi157.stdout:5/707: symlink d1/d5/d7/d2f/d45/d54/l110 0 2024-04-22T23:35:06.874 INFO:tasks.workunit.client.1.smithi157.stdout:3/698: rmdir d0/d7 39 2024-04-22T23:35:06.874 INFO:tasks.workunit.client.1.smithi157.stdout:3/699: truncate d0/d3/d34/d39/f58 691019 0 2024-04-22T23:35:06.875 INFO:tasks.workunit.client.1.smithi157.stdout:5/708: mkdir d1/d5/dc/d43/d5e/d111 0 2024-04-22T23:35:06.875 INFO:tasks.workunit.client.1.smithi157.stdout:3/700: truncate d0/d7/d95/fa5 518546 0 2024-04-22T23:35:06.875 INFO:tasks.workunit.client.1.smithi157.stdout:3/701: write d0/d3/d34/d39/f58 [1072298,29104] 0 2024-04-22T23:35:06.876 INFO:tasks.workunit.client.1.smithi157.stdout:5/709: creat d1/db0/df3/f112 x:0 0 0 2024-04-22T23:35:06.877 INFO:tasks.workunit.client.1.smithi157.stdout:3/702: rename d0/d3/d34/d3d/d57/f65 to d0/d3/d34/fe4 0 2024-04-22T23:35:06.878 INFO:tasks.workunit.client.1.smithi157.stdout:3/703: dread - d0/d3/d34/db9/d9f/fdb zero size 2024-04-22T23:35:06.878 INFO:tasks.workunit.client.1.smithi157.stdout:3/704: chown d0/d3/d34/db9/f8c 0 1 2024-04-22T23:35:06.879 INFO:tasks.workunit.client.1.smithi157.stdout:3/705: creat d0/d3/d34/d47/dcd/fe5 x:0 0 0 2024-04-22T23:35:06.879 INFO:tasks.workunit.client.1.smithi157.stdout:3/706: creat d0/d3/d34/d3d/d5b/d72/fe6 x:0 0 0 2024-04-22T23:35:06.880 INFO:tasks.workunit.client.1.smithi157.stdout:3/707: fsync d0/d3/f35 0 2024-04-22T23:35:06.887 INFO:tasks.workunit.client.1.smithi157.stdout:5/710: dread d1/d5/dc/d43/f44 [0,4194304] 0 2024-04-22T23:35:06.889 INFO:tasks.workunit.client.1.smithi157.stdout:5/711: rename d1/d27/d29/f58 to d1/d5/d7/d2f/d45/da4/f113 0 2024-04-22T23:35:06.900 INFO:tasks.workunit.client.0.smithi107.stdout:7/425: dwrite d7/d2d/f33 [0,4194304] 0 2024-04-22T23:35:06.905 INFO:tasks.workunit.client.0.smithi107.stdout:4/550: dwrite d1/d7/f11 [0,4194304] 0 2024-04-22T23:35:06.910 INFO:tasks.workunit.client.1.smithi157.stdout:5/712: dwrite d1/d5/dc/d5d/d77/f107 [0,4194304] 0 2024-04-22T23:35:06.912 INFO:tasks.workunit.client.1.smithi157.stdout:5/713: symlink d1/d5/dc/d43/d4f/d7f/d99/l114 0 2024-04-22T23:35:06.913 INFO:tasks.workunit.client.1.smithi157.stdout:5/714: write d1/d5/d7/d2f/d45/d54/fc0 [1030324,108659] 0 2024-04-22T23:35:06.915 INFO:tasks.workunit.client.1.smithi157.stdout:3/708: fsync d0/d3/d34/d47/dcd/fe5 0 2024-04-22T23:35:06.917 INFO:tasks.workunit.client.1.smithi157.stdout:3/709: rmdir d0/d3/d34/d3d 39 2024-04-22T23:35:06.918 INFO:tasks.workunit.client.1.smithi157.stdout:3/710: dread d0/d3/d34/d3d/d5b/f84 [0,4194304] 0 2024-04-22T23:35:06.918 INFO:tasks.workunit.client.1.smithi157.stdout:3/711: fdatasync d0/d3/d34/d39/f66 0 2024-04-22T23:35:06.921 INFO:tasks.workunit.client.1.smithi157.stdout:3/712: symlink d0/d3/d13/le7 0 2024-04-22T23:35:06.923 INFO:tasks.workunit.client.1.smithi157.stdout:3/713: getdents d0/d3/d34/d3d/d5b/d72 0 2024-04-22T23:35:06.924 INFO:tasks.workunit.client.1.smithi157.stdout:3/714: read d0/d3/d34/d3d/d5b/f94 [32003,14026] 0 2024-04-22T23:35:06.924 INFO:tasks.workunit.client.1.smithi157.stdout:3/715: fdatasync d0/d3/d34/d3d/d57/d60/d9c/fc8 0 2024-04-22T23:35:06.925 INFO:tasks.workunit.client.1.smithi157.stdout:3/716: symlink d0/d3/d34/d47/dcd/le8 0 2024-04-22T23:35:06.936 INFO:tasks.workunit.client.1.smithi157.stdout:5/715: dwrite d1/db0/ffc [0,4194304] 0 2024-04-22T23:35:06.937 INFO:tasks.workunit.client.1.smithi157.stdout:5/716: dread d1/d5/fd0 [0,4194304] 0 2024-04-22T23:35:06.938 INFO:tasks.workunit.client.1.smithi157.stdout:5/717: link d1/c40 d1/d27/c115 0 2024-04-22T23:35:06.939 INFO:tasks.workunit.client.1.smithi157.stdout:5/718: creat d1/d5/d7/d2f/f116 x:0 0 0 2024-04-22T23:35:06.941 INFO:tasks.workunit.client.1.smithi157.stdout:5/719: mknod d1/db0/dff/c117 0 2024-04-22T23:35:06.941 INFO:tasks.workunit.client.1.smithi157.stdout:5/720: write d1/d27/d29/fbf [1595204,3351] 0 2024-04-22T23:35:06.952 INFO:tasks.workunit.client.0.smithi107.stdout:6/484: mknod db/d20/d2a/d38/d61/d6f/c8d 0 2024-04-22T23:35:06.954 INFO:tasks.workunit.client.1.smithi157.stdout:5/721: dwrite d1/d1f/ff1 [0,4194304] 0 2024-04-22T23:35:06.954 INFO:tasks.workunit.client.1.smithi157.stdout:5/722: write d1/d5/d7/d10/f49 [9011790,130081] 0 2024-04-22T23:35:06.955 INFO:tasks.workunit.client.0.smithi107.stdout:9/544: creat d5/d29/dbb/fbf x:0 0 0 2024-04-22T23:35:06.955 INFO:tasks.workunit.client.1.smithi157.stdout:5/723: write d1/d5/d7/d2f/d45/d54/f74 [732958,49204] 0 2024-04-22T23:35:06.957 INFO:tasks.workunit.client.1.smithi157.stdout:5/724: mkdir d1/d1f/d22/d118 0 2024-04-22T23:35:06.957 INFO:tasks.workunit.client.1.smithi157.stdout:5/725: truncate d1/d5/f4e 513056 0 2024-04-22T23:35:06.957 INFO:tasks.workunit.client.1.smithi157.stdout:5/726: dread - d1/d5/d7/d2f/d9c/faa zero size 2024-04-22T23:35:06.958 INFO:tasks.workunit.client.1.smithi157.stdout:5/727: truncate d1/db0/d7a/f8f 1501339 0 2024-04-22T23:35:06.958 INFO:tasks.workunit.client.1.smithi157.stdout:5/728: readlink d1/d1f/d22/l25 0 2024-04-22T23:35:06.958 INFO:tasks.workunit.client.1.smithi157.stdout:5/729: creat d1/db0/dff/f119 x:0 0 0 2024-04-22T23:35:06.958 INFO:tasks.workunit.client.1.smithi157.stdout:5/730: write d1/d5/d7/d2f/d45/da4/fdc [890695,59540] 0 2024-04-22T23:35:06.958 INFO:tasks.workunit.client.0.smithi107.stdout:0/469: creat d1/d9/d14/d4f/d59/fa5 x:0 0 0 2024-04-22T23:35:06.958 INFO:tasks.workunit.client.0.smithi107.stdout:8/485: getdents d0/d7/df/d33/d9b 0 2024-04-22T23:35:06.959 INFO:tasks.workunit.client.1.smithi157.stdout:5/731: link d1/c40 d1/d5/d7/d2f/d51/c11a 0 2024-04-22T23:35:06.959 INFO:tasks.workunit.client.1.smithi157.stdout:5/732: chown d1/d27/d29/fc2 983632902 1 2024-04-22T23:35:06.978 INFO:tasks.workunit.client.1.smithi157.stdout:5/733: dwrite d1/d5/dc/d43/d4f/d7f/dab/fef [0,4194304] 0 2024-04-22T23:35:06.980 INFO:tasks.workunit.client.1.smithi157.stdout:5/734: dread d1/d5/d7/dbd/ff4 [0,4194304] 0 2024-04-22T23:35:06.980 INFO:tasks.workunit.client.1.smithi157.stdout:8/470: sync 2024-04-22T23:35:06.981 INFO:tasks.workunit.client.1.smithi157.stdout:1/708: sync 2024-04-22T23:35:06.981 INFO:tasks.workunit.client.1.smithi157.stdout:2/781: sync 2024-04-22T23:35:06.981 INFO:tasks.workunit.client.1.smithi157.stdout:4/702: sync 2024-04-22T23:35:06.981 INFO:tasks.workunit.client.1.smithi157.stdout:4/703: readlink d3/d96/d40/l54 0 2024-04-22T23:35:06.981 INFO:tasks.workunit.client.1.smithi157.stdout:2/782: chown d7/d8/d30/d23/d27/d57/fa7 2587 1 2024-04-22T23:35:06.981 INFO:tasks.workunit.client.1.smithi157.stdout:9/680: sync 2024-04-22T23:35:06.981 INFO:tasks.workunit.client.1.smithi157.stdout:5/735: mkdir d1/db0/dff/d11b 0 2024-04-22T23:35:06.982 INFO:tasks.workunit.client.1.smithi157.stdout:1/709: mkdir d4/d17/d1d/df5 0 2024-04-22T23:35:06.983 INFO:tasks.workunit.client.1.smithi157.stdout:4/704: mknod d3/d11/d1d/d73/d80/ced 0 2024-04-22T23:35:06.983 INFO:tasks.workunit.client.1.smithi157.stdout:4/705: chown d3/d11/d1d/f41 193 1 2024-04-22T23:35:06.984 INFO:tasks.workunit.client.1.smithi157.stdout:2/783: creat d7/d8/d25/d55/f106 x:0 0 0 2024-04-22T23:35:06.985 INFO:tasks.workunit.client.1.smithi157.stdout:5/736: mknod d1/db0/d7a/c11c 0 2024-04-22T23:35:06.986 INFO:tasks.workunit.client.1.smithi157.stdout:8/471: getdents d2/d6/d21/d5f/d11/d13 0 2024-04-22T23:35:06.986 INFO:tasks.workunit.client.1.smithi157.stdout:8/472: readlink d2/d6/d21/l29 0 2024-04-22T23:35:06.986 INFO:tasks.workunit.client.1.smithi157.stdout:8/473: stat d2/d6/d66/d67/f95 0 2024-04-22T23:35:06.987 INFO:tasks.workunit.client.1.smithi157.stdout:1/710: link d4/d77/f93 d4/d17/d34/d75/dcc/ff6 0 2024-04-22T23:35:06.988 INFO:tasks.workunit.client.1.smithi157.stdout:1/711: write d4/d17/d1d/d33/d41/d54/fe9 [319897,126617] 0 2024-04-22T23:35:06.988 INFO:tasks.workunit.client.1.smithi157.stdout:2/784: mknod d7/d8/c107 0 2024-04-22T23:35:06.988 INFO:tasks.workunit.client.1.smithi157.stdout:2/785: readlink d7/d8/l62 0 2024-04-22T23:35:06.990 INFO:tasks.workunit.client.1.smithi157.stdout:5/737: unlink d1/d5/dc/d5d/le8 0 2024-04-22T23:35:06.990 INFO:tasks.workunit.client.1.smithi157.stdout:5/738: dread - d1/db0/dff/f101 zero size 2024-04-22T23:35:06.995 INFO:tasks.workunit.client.1.smithi157.stdout:5/739: mknod d1/d5/d7/c11d 0 2024-04-22T23:35:06.998 INFO:tasks.workunit.client.1.smithi157.stdout:8/474: dread d2/d6/f46 [0,4194304] 0 2024-04-22T23:35:06.998 INFO:tasks.workunit.client.1.smithi157.stdout:8/475: dread - d2/d34/f60 zero size 2024-04-22T23:35:06.998 INFO:tasks.workunit.client.1.smithi157.stdout:8/476: write d2/d34/d6a/d77/d42/d58/fa5 [1260662,106713] 0 2024-04-22T23:35:07.000 INFO:tasks.workunit.client.1.smithi157.stdout:1/712: dread d4/db/fd [0,4194304] 0 2024-04-22T23:35:07.000 INFO:tasks.workunit.client.1.smithi157.stdout:8/477: unlink d2/d6/d21/d5f/f2f 0 2024-04-22T23:35:07.001 INFO:tasks.workunit.client.1.smithi157.stdout:8/478: read d2/d4/f80 [809034,15356] 0 2024-04-22T23:35:07.002 INFO:tasks.workunit.client.1.smithi157.stdout:9/681: dwrite d4/df/d57/f2b [4194304,4194304] 0 2024-04-22T23:35:07.002 INFO:tasks.workunit.client.1.smithi157.stdout:4/706: dwrite d3/d11/d1d/d73/d80/f89 [0,4194304] 0 2024-04-22T23:35:07.006 INFO:tasks.workunit.client.1.smithi157.stdout:8/479: rename d2/d6/l2b to d2/d34/d6a/d77/la7 0 2024-04-22T23:35:07.009 INFO:tasks.workunit.client.1.smithi157.stdout:8/480: creat d2/d6/d66/d67/fa8 x:0 0 0 2024-04-22T23:35:07.010 INFO:tasks.workunit.client.1.smithi157.stdout:9/682: symlink d4/df/ld4 0 2024-04-22T23:35:07.013 INFO:tasks.workunit.client.1.smithi157.stdout:8/481: unlink d2/d34/d6a/d77/d42/f61 0 2024-04-22T23:35:07.013 INFO:tasks.workunit.client.1.smithi157.stdout:9/683: symlink d4/d15/d23/ld5 0 2024-04-22T23:35:07.021 INFO:tasks.workunit.client.1.smithi157.stdout:5/740: dwrite d1/d1f/dc6/ff6 [0,4194304] 0 2024-04-22T23:35:07.027 INFO:tasks.workunit.client.1.smithi157.stdout:5/741: creat d1/db0/d7a/f11e x:0 0 0 2024-04-22T23:35:07.027 INFO:tasks.workunit.client.1.smithi157.stdout:1/713: dwrite d4/d17/d1d/d33/d41/d54/f98 [0,4194304] 0 2024-04-22T23:35:07.028 INFO:tasks.workunit.client.0.smithi107.stdout:6/485: creat db/d20/d2a/d38/d59/d70/f8e x:0 0 0 2024-04-22T23:35:07.028 INFO:tasks.workunit.client.0.smithi107.stdout:6/486: write db/f21 [4574583,52423] 0 2024-04-22T23:35:07.028 INFO:tasks.workunit.client.0.smithi107.stdout:6/487: stat db/d20/c25 0 2024-04-22T23:35:07.029 INFO:tasks.workunit.client.0.smithi107.stdout:9/545: mkdir d5/d29/d7b/d91/dc0 0 2024-04-22T23:35:07.030 INFO:tasks.workunit.client.1.smithi157.stdout:5/742: creat d1/d1f/d22/f11f x:0 0 0 2024-04-22T23:35:07.030 INFO:tasks.workunit.client.1.smithi157.stdout:5/743: creat d1/db0/dff/d102/f120 x:0 0 0 2024-04-22T23:35:07.031 INFO:tasks.workunit.client.0.smithi107.stdout:0/470: mkdir d1/d9/d14/d4f/da6 0 2024-04-22T23:35:07.031 INFO:tasks.workunit.client.0.smithi107.stdout:0/471: write d1/d27/f21 [1672041,59732] 0 2024-04-22T23:35:07.032 INFO:tasks.workunit.client.1.smithi157.stdout:1/714: link d4/d17/d6b/l80 d4/d17/d34/d84/lf7 0 2024-04-22T23:35:07.034 INFO:tasks.workunit.client.0.smithi107.stdout:9/546: dread d5/d29/f3a [0,4194304] 0 2024-04-22T23:35:07.034 INFO:tasks.workunit.client.0.smithi107.stdout:9/547: write d5/d7/d27/d46/f66 [199311,86083] 0 2024-04-22T23:35:07.034 INFO:tasks.workunit.client.0.smithi107.stdout:8/486: mkdir d0/d7/d20/da2 0 2024-04-22T23:35:07.034 INFO:tasks.workunit.client.0.smithi107.stdout:8/487: truncate d0/f83 401482 0 2024-04-22T23:35:07.034 INFO:tasks.workunit.client.0.smithi107.stdout:8/488: chown d0/d7/d20/d2d/l90 3 1 2024-04-22T23:35:07.035 INFO:tasks.workunit.client.1.smithi157.stdout:4/707: dwrite d3/d11/d1d/daf/fe2 [0,4194304] 0 2024-04-22T23:35:07.035 INFO:tasks.workunit.client.1.smithi157.stdout:4/708: truncate d3/d11/d34/da6/fc6 584782 0 2024-04-22T23:35:07.037 INFO:tasks.workunit.client.0.smithi107.stdout:5/525: symlink d9/db/de/d72/lb8 0 2024-04-22T23:35:07.037 INFO:tasks.workunit.client.0.smithi107.stdout:5/526: fdatasync d9/db/d42/f4b 0 2024-04-22T23:35:07.042 INFO:tasks.workunit.client.1.smithi157.stdout:8/482: dwrite d2/d6/d66/d67/f79 [0,4194304] 0 2024-04-22T23:35:07.042 INFO:tasks.workunit.client.1.smithi157.stdout:1/715: dread d4/d17/f35 [0,4194304] 0 2024-04-22T23:35:07.042 INFO:tasks.workunit.client.1.smithi157.stdout:1/716: creat d4/d17/d34/d3b/d50/ff8 x:0 0 0 2024-04-22T23:35:07.043 INFO:tasks.workunit.client.0.smithi107.stdout:8/489: mkdir d0/d7/d20/da2/da3 0 2024-04-22T23:35:07.045 INFO:tasks.workunit.client.0.smithi107.stdout:5/527: creat d9/db/d42/d75/db6/fb9 x:0 0 0 2024-04-22T23:35:07.047 INFO:tasks.workunit.client.0.smithi107.stdout:9/548: rmdir d5/d7/d17/d1b/d1f/d36/d96 39 2024-04-22T23:35:07.048 INFO:tasks.workunit.client.0.smithi107.stdout:9/549: write d5/d29/f6d [159496,86577] 0 2024-04-22T23:35:07.048 INFO:tasks.workunit.client.0.smithi107.stdout:9/550: fdatasync d5/f13 0 2024-04-22T23:35:07.048 INFO:tasks.workunit.client.1.smithi157.stdout:9/684: dwrite d4/df/d61/fcb [0,4194304] 0 2024-04-22T23:35:07.049 INFO:tasks.workunit.client.0.smithi107.stdout:9/551: read d5/d7/d17/f4e [80376,54113] 0 2024-04-22T23:35:07.051 INFO:tasks.workunit.client.1.smithi157.stdout:1/717: rmdir d4/d17/d34/d75/dcc 39 2024-04-22T23:35:07.051 INFO:tasks.workunit.client.1.smithi157.stdout:1/718: fdatasync d4/d17/d34/d3b/d50/fb6 0 2024-04-22T23:35:07.052 INFO:tasks.workunit.client.0.smithi107.stdout:6/488: dread f6 [4194304,4194304] 0 2024-04-22T23:35:07.052 INFO:tasks.workunit.client.0.smithi107.stdout:6/489: chown db/d20/d2a/l34 7 1 2024-04-22T23:35:07.052 INFO:tasks.workunit.client.0.smithi107.stdout:6/490: creat db/d20/d2a/d38/d59/d70/f8f x:0 0 0 2024-04-22T23:35:07.054 INFO:tasks.workunit.client.1.smithi157.stdout:8/483: truncate d2/d6/d21/d5f/d11/f27 1877663 0 2024-04-22T23:35:07.056 INFO:tasks.workunit.client.1.smithi157.stdout:9/685: rename d4/d15/d23/f8c to d4/df/d57/dbd/dc6/fd6 0 2024-04-22T23:35:07.057 INFO:tasks.workunit.client.0.smithi107.stdout:0/472: dread d1/d27/f21 [0,4194304] 0 2024-04-22T23:35:07.057 INFO:tasks.workunit.client.1.smithi157.stdout:1/719: creat d4/db/d3d/ff9 x:0 0 0 2024-04-22T23:35:07.057 INFO:tasks.workunit.client.1.smithi157.stdout:1/720: creat d4/d17/d1d/db2/ffa x:0 0 0 2024-04-22T23:35:07.057 INFO:tasks.workunit.client.1.smithi157.stdout:1/721: write d4/d17/d34/d75/dcc/fd7 [591014,2652] 0 2024-04-22T23:35:07.057 INFO:tasks.workunit.client.1.smithi157.stdout:1/722: truncate d4/d17/d34/dc8/fe2 171106 0 2024-04-22T23:35:07.059 INFO:tasks.workunit.client.1.smithi157.stdout:9/686: stat d4/df/cc7 0 2024-04-22T23:35:07.059 INFO:tasks.workunit.client.0.smithi107.stdout:0/473: truncate d1/d27/d20/f36 2196964 0 2024-04-22T23:35:07.061 INFO:tasks.workunit.client.1.smithi157.stdout:8/484: rename d2/d4/f50 to d2/fa9 0 2024-04-22T23:35:07.063 INFO:tasks.workunit.client.0.smithi107.stdout:9/552: dread d5/d7/d27/d69/db6/fab [0,4194304] 0 2024-04-22T23:35:07.063 INFO:tasks.workunit.client.0.smithi107.stdout:9/553: fdatasync d5/d14/fa2 0 2024-04-22T23:35:07.063 INFO:tasks.workunit.client.1.smithi157.stdout:1/723: creat d4/d17/d1d/d33/d41/ffb x:0 0 0 2024-04-22T23:35:07.065 INFO:tasks.workunit.client.1.smithi157.stdout:1/724: mkdir d4/d17/d34/d75/d8a/dfc 0 2024-04-22T23:35:07.065 INFO:tasks.workunit.client.1.smithi157.stdout:9/687: dread d4/d15/d23/d3f/f42 [0,4194304] 0 2024-04-22T23:35:07.065 INFO:tasks.workunit.client.1.smithi157.stdout:9/688: truncate d4/d15/d23/d27/d47/f7f 1019005 0 2024-04-22T23:35:07.067 INFO:tasks.workunit.client.0.smithi107.stdout:9/554: unlink d5/d7/d27/d46/f75 0 2024-04-22T23:35:07.067 INFO:tasks.workunit.client.1.smithi157.stdout:9/689: symlink d4/d15/d23/d27/dc0/ld7 0 2024-04-22T23:35:07.067 INFO:tasks.workunit.client.1.smithi157.stdout:9/690: creat d4/d15/d23/d27/db5/fd8 x:0 0 0 2024-04-22T23:35:07.073 INFO:tasks.workunit.client.0.smithi107.stdout:5/528: dwrite d9/db/d42/f5a [4194304,4194304] 0 2024-04-22T23:35:07.073 INFO:tasks.workunit.client.1.smithi157.stdout:5/744: dwrite d1/d5/d7/d2f/d45/d54/fc0 [0,4194304] 0 2024-04-22T23:35:07.074 INFO:tasks.workunit.client.0.smithi107.stdout:5/529: truncate d9/db/f8c 142556 0 2024-04-22T23:35:07.075 INFO:tasks.workunit.client.1.smithi157.stdout:4/709: dwrite d3/d11/d1d/d73/d80/dac/db5/f94 [0,4194304] 0 2024-04-22T23:35:07.075 INFO:tasks.workunit.client.1.smithi157.stdout:4/710: stat d3/d11/d1d/f2d 0 2024-04-22T23:35:07.075 INFO:tasks.workunit.client.1.smithi157.stdout:4/711: truncate d3/d11/d34/da6/fc6 714248 0 2024-04-22T23:35:07.075 INFO:tasks.workunit.client.1.smithi157.stdout:4/712: dread - d3/d4b/d7a/fe4 zero size 2024-04-22T23:35:07.075 INFO:tasks.workunit.client.1.smithi157.stdout:4/713: write d3/d11/d1d/d73/d80/dac/db5/f8b [573082,17516] 0 2024-04-22T23:35:07.076 INFO:tasks.workunit.client.0.smithi107.stdout:5/530: symlink d9/db/d42/d75/db6/lba 0 2024-04-22T23:35:07.077 INFO:tasks.workunit.client.0.smithi107.stdout:5/531: write d9/db/de/d5e/f71 [1351365,85234] 0 2024-04-22T23:35:07.079 INFO:tasks.workunit.client.0.smithi107.stdout:6/491: fdatasync db/d20/d2a/f2f 0 2024-04-22T23:35:07.084 INFO:tasks.workunit.client.0.smithi107.stdout:6/492: symlink db/d20/d2c/d3b/l90 0 2024-04-22T23:35:07.085 INFO:tasks.workunit.client.1.smithi157.stdout:1/725: dread d4/d17/d34/fcf [0,4194304] 0 2024-04-22T23:35:07.086 INFO:tasks.workunit.client.0.smithi107.stdout:6/493: creat db/d20/d2c/f91 x:0 0 0 2024-04-22T23:35:07.087 INFO:tasks.workunit.client.0.smithi107.stdout:0/474: dwrite d1/d27/d20/d40/fa0 [0,4194304] 0 2024-04-22T23:35:07.088 INFO:tasks.workunit.client.1.smithi157.stdout:4/714: truncate d3/d11/d1d/f41 6641119 0 2024-04-22T23:35:07.089 INFO:tasks.workunit.client.0.smithi107.stdout:2/528: sync 2024-04-22T23:35:07.089 INFO:tasks.workunit.client.0.smithi107.stdout:1/463: sync 2024-04-22T23:35:07.090 INFO:tasks.workunit.client.0.smithi107.stdout:1/464: fsync d7/d19/d54/f70 0 2024-04-22T23:35:07.091 INFO:tasks.workunit.client.0.smithi107.stdout:6/494: link db/d20/d2a/d38/d4b/l7e db/d20/d55/d84/l92 0 2024-04-22T23:35:07.098 INFO:tasks.workunit.client.1.smithi157.stdout:7/576: sync 2024-04-22T23:35:07.098 INFO:tasks.workunit.client.1.smithi157.stdout:1/726: dread d4/f23 [0,4194304] 0 2024-04-22T23:35:07.099 INFO:tasks.workunit.client.1.smithi157.stdout:1/727: dread - d4/d77/fcd zero size 2024-04-22T23:35:07.099 INFO:tasks.workunit.client.1.smithi157.stdout:9/691: dwrite d4/d15/d23/d27/db5/fc3 [0,4194304] 0 2024-04-22T23:35:07.099 INFO:tasks.workunit.client.1.smithi157.stdout:9/692: write d4/df/d80/f9d [638459,83016] 0 2024-04-22T23:35:07.099 INFO:tasks.workunit.client.1.smithi157.stdout:9/693: creat d4/df/d57/fd9 x:0 0 0 2024-04-22T23:35:07.099 INFO:tasks.workunit.client.1.smithi157.stdout:9/694: write d4/df/f8b [1118762,99157] 0 2024-04-22T23:35:07.100 INFO:tasks.workunit.client.1.smithi157.stdout:7/577: unlink d0/d9/d29/d98/faa 0 2024-04-22T23:35:07.102 INFO:tasks.workunit.client.1.smithi157.stdout:1/728: rename d4/db/d71/l74 to d4/lfd 0 2024-04-22T23:35:07.102 INFO:tasks.workunit.client.0.smithi107.stdout:6/495: truncate db/f1c 2817324 0 2024-04-22T23:35:07.102 INFO:tasks.workunit.client.0.smithi107.stdout:6/496: read db/d20/d2c/d3b/d40/f71 [292,60349] 0 2024-04-22T23:35:07.102 INFO:tasks.workunit.client.0.smithi107.stdout:6/497: chown db/f2d 21566 1 2024-04-22T23:35:07.103 INFO:tasks.workunit.client.1.smithi157.stdout:7/578: dread d0/d9/d29/d33/d37/f50 [0,4194304] 0 2024-04-22T23:35:07.103 INFO:tasks.workunit.client.1.smithi157.stdout:7/579: write d0/d9/d42/d55/d56/d91/fab [512415,13910] 0 2024-04-22T23:35:07.103 INFO:tasks.workunit.client.0.smithi107.stdout:2/529: getdents d1/d2b/d49/d7e/da8 0 2024-04-22T23:35:07.104 INFO:tasks.workunit.client.0.smithi107.stdout:2/530: stat d1/d12/d35/l41 0 2024-04-22T23:35:07.104 INFO:tasks.workunit.client.0.smithi107.stdout:1/465: write d7/d19/d54/f70 [3067432,90300] 0 2024-04-22T23:35:07.106 INFO:tasks.workunit.client.1.smithi157.stdout:9/695: symlink d4/d15/d23/d27/d47/lda 0 2024-04-22T23:35:07.107 INFO:tasks.workunit.client.1.smithi157.stdout:1/729: dread d4/d17/d34/fcf [0,4194304] 0 2024-04-22T23:35:07.107 INFO:tasks.workunit.client.1.smithi157.stdout:9/696: write d4/d15/d23/d67/f95 [692526,100560] 0 2024-04-22T23:35:07.107 INFO:tasks.workunit.client.1.smithi157.stdout:9/697: truncate d4/df/d61/f71 1977974 0 2024-04-22T23:35:07.107 INFO:tasks.workunit.client.1.smithi157.stdout:9/698: chown d4/df/d57/dd0 146 1 2024-04-22T23:35:07.107 INFO:tasks.workunit.client.1.smithi157.stdout:9/699: write d4/d15/d23/d27/d47/fd2 [953219,64240] 0 2024-04-22T23:35:07.107 INFO:tasks.workunit.client.1.smithi157.stdout:9/700: creat d4/d15/d23/fdb x:0 0 0 2024-04-22T23:35:07.108 INFO:tasks.workunit.client.0.smithi107.stdout:6/498: creat db/d20/d2a/d38/d59/f93 x:0 0 0 2024-04-22T23:35:07.108 INFO:tasks.workunit.client.0.smithi107.stdout:6/499: dread - db/f8a zero size 2024-04-22T23:35:07.108 INFO:tasks.workunit.client.0.smithi107.stdout:6/500: write db/d20/d2a/d38/d4b/f54 [4458406,52250] 0 2024-04-22T23:35:07.108 INFO:tasks.workunit.client.0.smithi107.stdout:2/531: creat d1/d2b/d3d/fb9 x:0 0 0 2024-04-22T23:35:07.110 INFO:tasks.workunit.client.1.smithi157.stdout:5/745: dwrite d1/d5/dc/d43/d5e/fe6 [0,4194304] 0 2024-04-22T23:35:07.112 INFO:tasks.workunit.client.0.smithi107.stdout:1/466: creat d7/d19/d71/f96 x:0 0 0 2024-04-22T23:35:07.112 INFO:tasks.workunit.client.0.smithi107.stdout:1/467: readlink d7/d19/d71/l85 0 2024-04-22T23:35:07.112 INFO:tasks.workunit.client.0.smithi107.stdout:1/468: chown d7/l8 30 1 2024-04-22T23:35:07.112 INFO:tasks.workunit.client.1.smithi157.stdout:1/730: creat d4/d17/db9/ffe x:0 0 0 2024-04-22T23:35:07.113 INFO:tasks.workunit.client.0.smithi107.stdout:2/532: mkdir d1/d2b/d49/dba 0 2024-04-22T23:35:07.114 INFO:tasks.workunit.client.1.smithi157.stdout:8/485: dwrite d2/d34/d6a/d77/d42/d58/fa5 [0,4194304] 0 2024-04-22T23:35:07.114 INFO:tasks.workunit.client.1.smithi157.stdout:8/486: chown d2/f48 134073 1 2024-04-22T23:35:07.115 INFO:tasks.workunit.client.1.smithi157.stdout:8/487: chown d2/d4/d30/f62 4270 1 2024-04-22T23:35:07.117 INFO:tasks.workunit.client.1.smithi157.stdout:1/731: mknod d4/d17/d34/d75/dc7/dea/cff 0 2024-04-22T23:35:07.117 INFO:tasks.workunit.client.1.smithi157.stdout:1/732: read - d4/d17/d34/d3b/d50/fb6 zero size 2024-04-22T23:35:07.118 INFO:tasks.workunit.client.0.smithi107.stdout:2/533: mkdir d1/d63/dbb 0 2024-04-22T23:35:07.119 INFO:tasks.workunit.client.0.smithi107.stdout:2/534: fsync d1/f6 0 2024-04-22T23:35:07.119 INFO:tasks.workunit.client.1.smithi157.stdout:7/580: dread d0/d9/d29/f3b [0,4194304] 0 2024-04-22T23:35:07.119 INFO:tasks.workunit.client.1.smithi157.stdout:7/581: chown d0/d9/db/df/d60 241551633 1 2024-04-22T23:35:07.119 INFO:tasks.workunit.client.1.smithi157.stdout:7/582: chown d0/d9/d42/d55/c73 55 1 2024-04-22T23:35:07.119 INFO:tasks.workunit.client.1.smithi157.stdout:8/488: dread d2/d6/d21/d8d/f9c [0,4194304] 0 2024-04-22T23:35:07.120 INFO:tasks.workunit.client.1.smithi157.stdout:7/583: symlink d0/d9/de/laf 0 2024-04-22T23:35:07.121 INFO:tasks.workunit.client.0.smithi107.stdout:2/535: mknod d1/d2b/d49/d87/cbc 0 2024-04-22T23:35:07.122 INFO:tasks.workunit.client.1.smithi157.stdout:8/489: symlink d2/d6/d21/d8d/laa 0 2024-04-22T23:35:07.123 INFO:tasks.workunit.client.1.smithi157.stdout:7/584: getdents d0/d9/d29/d33/d37 0 2024-04-22T23:35:07.125 INFO:tasks.workunit.client.0.smithi107.stdout:1/469: dread d7/d19/f2c [0,4194304] 0 2024-04-22T23:35:07.125 INFO:tasks.workunit.client.0.smithi107.stdout:1/470: write d7/d19/d42/f64 [1000563,64954] 0 2024-04-22T23:35:07.125 INFO:tasks.workunit.client.1.smithi157.stdout:7/585: symlink d0/d9/db/df/lb0 0 2024-04-22T23:35:07.126 INFO:tasks.workunit.client.1.smithi157.stdout:7/586: chown d0/d9/d10/l3f 1865 1 2024-04-22T23:35:07.127 INFO:tasks.workunit.client.1.smithi157.stdout:7/587: symlink d0/d9/d42/d55/lb1 0 2024-04-22T23:35:07.128 INFO:tasks.workunit.client.1.smithi157.stdout:1/733: read d4/d17/d6b/f86 [619037,69927] 0 2024-04-22T23:35:07.128 INFO:tasks.workunit.client.1.smithi157.stdout:9/701: dwrite d4/d15/d23/d27/f33 [0,4194304] 0 2024-04-22T23:35:07.129 INFO:tasks.workunit.client.1.smithi157.stdout:1/734: mkdir d4/db/d3d/dc1/d100 0 2024-04-22T23:35:07.133 INFO:tasks.workunit.client.1.smithi157.stdout:9/702: link d4/d15/d23/d27/d47/f4a d4/df/d57/dbd/dc6/fdc 0 2024-04-22T23:35:07.135 INFO:tasks.workunit.client.1.smithi157.stdout:1/735: truncate d4/d17/f64 675986 0 2024-04-22T23:35:07.138 INFO:tasks.workunit.client.0.smithi107.stdout:6/501: dwrite db/d20/d2c/f32 [4194304,4194304] 0 2024-04-22T23:35:07.139 INFO:tasks.workunit.client.1.smithi157.stdout:7/588: dread d0/d9/db/f15 [0,4194304] 0 2024-04-22T23:35:07.146 INFO:tasks.workunit.client.1.smithi157.stdout:5/746: dwrite d1/d1f/faf [0,4194304] 0 2024-04-22T23:35:07.146 INFO:tasks.workunit.client.1.smithi157.stdout:5/747: fsync d1/db0/df3/f112 0 2024-04-22T23:35:07.146 INFO:tasks.workunit.client.1.smithi157.stdout:5/748: creat d1/d1f/dc6/f121 x:0 0 0 2024-04-22T23:35:07.146 INFO:tasks.workunit.client.1.smithi157.stdout:5/749: chown d1/db0/d7a/f87 3 1 2024-04-22T23:35:07.146 INFO:tasks.workunit.client.1.smithi157.stdout:8/490: dwrite d2/f5 [0,4194304] 0 2024-04-22T23:35:07.146 INFO:tasks.workunit.client.1.smithi157.stdout:8/491: truncate d2/d34/d6a/d77/f9f 273804 0 2024-04-22T23:35:07.147 INFO:tasks.workunit.client.1.smithi157.stdout:9/703: dread d4/d15/d23/d27/f4f [0,4194304] 0 2024-04-22T23:35:07.149 INFO:tasks.workunit.client.1.smithi157.stdout:7/589: dread d0/d9/f1d [0,4194304] 0 2024-04-22T23:35:07.149 INFO:tasks.workunit.client.0.smithi107.stdout:6/502: dread db/f3f [0,4194304] 0 2024-04-22T23:35:07.150 INFO:tasks.workunit.client.0.smithi107.stdout:6/503: stat db/d20/d2a/d38/d61/d6f 0 2024-04-22T23:35:07.150 INFO:tasks.workunit.client.1.smithi157.stdout:8/492: mkdir d2/d6/d66/d67/dab 0 2024-04-22T23:35:07.151 INFO:tasks.workunit.client.1.smithi157.stdout:8/493: mknod d2/d6/d66/d97/cac 0 2024-04-22T23:35:07.151 INFO:tasks.workunit.client.1.smithi157.stdout:8/494: chown d2/d4/d30 67811154 1 2024-04-22T23:35:07.152 INFO:tasks.workunit.client.1.smithi157.stdout:1/736: read d4/d17/d34/d75/fbb [2637033,114495] 0 2024-04-22T23:35:07.152 INFO:tasks.workunit.client.1.smithi157.stdout:8/495: rmdir d2/d4/d30 39 2024-04-22T23:35:07.154 INFO:tasks.workunit.client.1.smithi157.stdout:1/737: truncate d4/f1b 2117890 0 2024-04-22T23:35:07.154 INFO:tasks.workunit.client.0.smithi107.stdout:6/504: dread db/d20/d49/f6e [0,4194304] 0 2024-04-22T23:35:07.155 INFO:tasks.workunit.client.0.smithi107.stdout:6/505: mknod db/d20/d2a/d38/d61/d6f/c94 0 2024-04-22T23:35:07.156 INFO:tasks.workunit.client.1.smithi157.stdout:5/750: dread d1/d5/dc/f1b [0,4194304] 0 2024-04-22T23:35:07.157 INFO:tasks.workunit.client.1.smithi157.stdout:8/496: mkdir d2/d4/dad 0 2024-04-22T23:35:07.157 INFO:tasks.workunit.client.1.smithi157.stdout:8/497: creat d2/d34/fae x:0 0 0 2024-04-22T23:35:07.158 INFO:tasks.workunit.client.1.smithi157.stdout:9/704: dread d4/d2d/f78 [0,4194304] 0 2024-04-22T23:35:07.159 INFO:tasks.workunit.client.0.smithi107.stdout:6/506: link db/d20/d49/f6e db/d20/d2a/d35/d69/d79/f95 0 2024-04-22T23:35:07.159 INFO:tasks.workunit.client.0.smithi107.stdout:6/507: readlink db/d20/d2a/d38/l7c 0 2024-04-22T23:35:07.160 INFO:tasks.workunit.client.1.smithi157.stdout:5/751: rename d1/d5/d7/d2f/f116 to d1/db0/dc1/f122 0 2024-04-22T23:35:07.160 INFO:tasks.workunit.client.1.smithi157.stdout:5/752: write d1/d1f/d22/f11f [263877,24182] 0 2024-04-22T23:35:07.161 INFO:tasks.workunit.client.1.smithi157.stdout:8/498: write d2/d6/d6b/f8b [1084647,66342] 0 2024-04-22T23:35:07.162 INFO:tasks.workunit.client.0.smithi107.stdout:1/471: unlink c0 0 2024-04-22T23:35:07.163 INFO:tasks.workunit.client.1.smithi157.stdout:8/499: write d2/d6/d6b/f8b [111976,81904] 0 2024-04-22T23:35:07.163 INFO:tasks.workunit.client.1.smithi157.stdout:1/738: link d4/d17/d34/d3b/d52/d65/dbc/le0 d4/d17/d34/d75/l101 0 2024-04-22T23:35:07.164 INFO:tasks.workunit.client.0.smithi107.stdout:6/508: symlink db/d20/d49/d7b/l96 0 2024-04-22T23:35:07.164 INFO:tasks.workunit.client.0.smithi107.stdout:0/475: dread d1/d27/d20/f36 [0,4194304] 0 2024-04-22T23:35:07.164 INFO:tasks.workunit.client.1.smithi157.stdout:9/705: mkdir d4/d15/d23/d27/d47/db3/ddd 0 2024-04-22T23:35:07.164 INFO:tasks.workunit.client.1.smithi157.stdout:9/706: chown d4/c36 15939 1 2024-04-22T23:35:07.166 INFO:tasks.workunit.client.1.smithi157.stdout:5/753: creat d1/d5/dc/d43/d4f/d7f/d99/f123 x:0 0 0 2024-04-22T23:35:07.166 INFO:tasks.workunit.client.1.smithi157.stdout:5/754: chown d1/d27/d29 1687 1 2024-04-22T23:35:07.166 INFO:tasks.workunit.client.0.smithi107.stdout:6/509: dread db/f3a [0,4194304] 0 2024-04-22T23:35:07.167 INFO:tasks.workunit.client.0.smithi107.stdout:6/510: symlink db/d20/d55/l97 0 2024-04-22T23:35:07.168 INFO:tasks.workunit.client.1.smithi157.stdout:8/500: unlink d2/d6/d66/d67/f79 0 2024-04-22T23:35:07.169 INFO:tasks.workunit.client.0.smithi107.stdout:0/476: dread d1/d27/d20/d2c/d2e/f39 [0,4194304] 0 2024-04-22T23:35:07.169 INFO:tasks.workunit.client.0.smithi107.stdout:0/477: fsync d1/d27/d20/f60 0 2024-04-22T23:35:07.169 INFO:tasks.workunit.client.0.smithi107.stdout:0/478: write d1/d27/d20/f60 [1631154,59229] 0 2024-04-22T23:35:07.170 INFO:tasks.workunit.client.1.smithi157.stdout:1/739: creat d4/db/d71/f102 x:0 0 0 2024-04-22T23:35:07.172 INFO:tasks.workunit.client.1.smithi157.stdout:5/755: dread - d1/d27/d29/f5f zero size 2024-04-22T23:35:07.172 INFO:tasks.workunit.client.1.smithi157.stdout:5/756: chown d1/d5/d7/d2f/d45/fd6 175 1 2024-04-22T23:35:07.172 INFO:tasks.workunit.client.1.smithi157.stdout:5/757: creat d1/db0/dff/d102/f124 x:0 0 0 2024-04-22T23:35:07.172 INFO:tasks.workunit.client.1.smithi157.stdout:5/758: readlink d1/d5/dc/d43/d5e/lb3 0 2024-04-22T23:35:07.173 INFO:tasks.workunit.client.1.smithi157.stdout:9/707: dread d4/df/d57/fb0 [0,4194304] 0 2024-04-22T23:35:07.173 INFO:tasks.workunit.client.1.smithi157.stdout:7/590: dwrite d0/d9/d42/d55/f8f [4194304,4194304] 0 2024-04-22T23:35:07.173 INFO:tasks.workunit.client.1.smithi157.stdout:7/591: chown d0/d9/d29/d5e/f6a 115 1 2024-04-22T23:35:07.176 INFO:tasks.workunit.client.1.smithi157.stdout:9/708: dread d4/df/f8b [0,4194304] 0 2024-04-22T23:35:07.177 INFO:tasks.workunit.client.0.smithi107.stdout:0/479: dread d1/d27/d20/f60 [0,4194304] 0 2024-04-22T23:35:07.177 INFO:tasks.workunit.client.0.smithi107.stdout:0/480: truncate d1/d9/d14/f8c 41972 0 2024-04-22T23:35:07.177 INFO:tasks.workunit.client.0.smithi107.stdout:6/511: write db/d20/d2a/d38/d4b/f54 [2441454,77797] 0 2024-04-22T23:35:07.179 INFO:tasks.workunit.client.1.smithi157.stdout:8/501: truncate d2/d4/f5b 298696 0 2024-04-22T23:35:07.180 INFO:tasks.workunit.client.1.smithi157.stdout:1/740: mknod d4/d17/d34/d75/d8a/c103 0 2024-04-22T23:35:07.180 INFO:tasks.workunit.client.1.smithi157.stdout:1/741: write d4/fc [1011748,88095] 0 2024-04-22T23:35:07.180 INFO:tasks.workunit.client.1.smithi157.stdout:1/742: truncate d4/db/d3d/ff9 497267 0 2024-04-22T23:35:07.182 INFO:tasks.workunit.client.1.smithi157.stdout:5/759: dread d1/d5/d7/d2f/d9c/fd5 [0,4194304] 0 2024-04-22T23:35:07.182 INFO:tasks.workunit.client.1.smithi157.stdout:5/760: truncate d1/d5/d7/d2f/d9c/faa 285438 0 2024-04-22T23:35:07.182 INFO:tasks.workunit.client.1.smithi157.stdout:5/761: truncate d1/d5/d7/d2f/d45/d54/d59/f70 4338246 0 2024-04-22T23:35:07.182 INFO:tasks.workunit.client.1.smithi157.stdout:5/762: chown d1/d5/d7/d2f/d45/d54/l9a 487 1 2024-04-22T23:35:07.183 INFO:tasks.workunit.client.1.smithi157.stdout:7/592: creat d0/d9/d42/d55/fb2 x:0 0 0 2024-04-22T23:35:07.183 INFO:tasks.workunit.client.0.smithi107.stdout:3/534: rename db/d21/d27/d54/d25/d6e to db/d21/d27/d15/db8 0 2024-04-22T23:35:07.185 INFO:tasks.workunit.client.1.smithi157.stdout:1/743: creat d4/f104 x:0 0 0 2024-04-22T23:35:07.186 INFO:tasks.workunit.client.1.smithi157.stdout:7/593: creat d0/d9/d29/d5e/d9e/fb3 x:0 0 0 2024-04-22T23:35:07.187 INFO:tasks.workunit.client.0.smithi107.stdout:0/481: symlink d1/d27/d20/d2c/la7 0 2024-04-22T23:35:07.188 INFO:tasks.workunit.client.0.smithi107.stdout:6/512: symlink db/d20/d2a/d38/d61/l98 0 2024-04-22T23:35:07.189 INFO:tasks.workunit.client.1.smithi157.stdout:8/502: symlink d2/d6/laf 0 2024-04-22T23:35:07.189 INFO:tasks.workunit.client.1.smithi157.stdout:9/709: dread d4/df/d61/f66 [0,4194304] 0 2024-04-22T23:35:07.190 INFO:tasks.workunit.client.1.smithi157.stdout:5/763: dread d1/d5/d7/f67 [0,4194304] 0 2024-04-22T23:35:07.190 INFO:tasks.workunit.client.1.smithi157.stdout:5/764: chown d1/la7 61 1 2024-04-22T23:35:07.190 INFO:tasks.workunit.client.1.smithi157.stdout:5/765: write d1/d5/d7/d2f/d45/f53 [250334,107064] 0 2024-04-22T23:35:07.190 INFO:tasks.workunit.client.1.smithi157.stdout:5/766: fsync d1/d5/d7/d2f/d45/d54/f82 0 2024-04-22T23:35:07.190 INFO:tasks.workunit.client.1.smithi157.stdout:5/767: chown d1/d5/l9e 3 1 2024-04-22T23:35:07.191 INFO:tasks.workunit.client.1.smithi157.stdout:5/768: fdatasync d1/d27/d29/fbf 0 2024-04-22T23:35:07.191 INFO:tasks.workunit.client.1.smithi157.stdout:5/769: creat d1/d5/d7/d2f/d45/d54/f125 x:0 0 0 2024-04-22T23:35:07.191 INFO:tasks.workunit.client.1.smithi157.stdout:1/744: rename d4/d17/d34/d3b/ceb to d4/db/d3d/dc1/c105 0 2024-04-22T23:35:07.191 INFO:tasks.workunit.client.1.smithi157.stdout:1/745: fdatasync d4/db/fe 0 2024-04-22T23:35:07.191 INFO:tasks.workunit.client.0.smithi107.stdout:3/535: truncate db/d21/d27/d54/d25/f34 4478433 0 2024-04-22T23:35:07.192 INFO:tasks.workunit.client.1.smithi157.stdout:8/503: chown d2/d6/f3c 259864 1 2024-04-22T23:35:07.193 INFO:tasks.workunit.client.0.smithi107.stdout:0/482: mknod d1/d9/ca8 0 2024-04-22T23:35:07.194 INFO:tasks.workunit.client.0.smithi107.stdout:6/513: rmdir db 39 2024-04-22T23:35:07.194 INFO:tasks.workunit.client.0.smithi107.stdout:6/514: fsync db/d20/d2a/f42 0 2024-04-22T23:35:07.195 INFO:tasks.workunit.client.1.smithi157.stdout:5/770: symlink d1/d5/dc/de0/l126 0 2024-04-22T23:35:07.196 INFO:tasks.workunit.client.0.smithi107.stdout:7/426: rename d7/cc to d7/d21/d48/d5c/c8c 0 2024-04-22T23:35:07.197 INFO:tasks.workunit.client.1.smithi157.stdout:8/504: symlink d2/lb0 0 2024-04-22T23:35:07.197 INFO:tasks.workunit.client.0.smithi107.stdout:3/536: symlink db/d21/d27/d15/d29/d2a/d45/lb9 0 2024-04-22T23:35:07.198 INFO:tasks.workunit.client.1.smithi157.stdout:5/771: mknod d1/d5/dc/d5d/c127 0 2024-04-22T23:35:07.198 INFO:tasks.workunit.client.1.smithi157.stdout:9/710: write d4/d15/d23/f69 [2310989,47606] 0 2024-04-22T23:35:07.199 INFO:tasks.workunit.client.0.smithi107.stdout:6/515: mknod db/c99 0 2024-04-22T23:35:07.199 INFO:tasks.workunit.client.0.smithi107.stdout:6/516: readlink l5 0 2024-04-22T23:35:07.199 INFO:tasks.workunit.client.0.smithi107.stdout:6/517: stat db/f29 0 2024-04-22T23:35:07.199 INFO:tasks.workunit.client.0.smithi107.stdout:6/518: readlink db/d20/d2c/d3b/l90 0 2024-04-22T23:35:07.199 INFO:tasks.workunit.client.0.smithi107.stdout:6/519: write db/f8a [722233,39871] 0 2024-04-22T23:35:07.200 INFO:tasks.workunit.client.1.smithi157.stdout:5/772: mknod d1/d1f/d22/d118/c128 0 2024-04-22T23:35:07.202 INFO:tasks.workunit.client.1.smithi157.stdout:5/773: chown d1/d5/d7/f34 2788 1 2024-04-22T23:35:07.202 INFO:tasks.workunit.client.1.smithi157.stdout:5/774: readlink d1/d5/d7/d10/l52 0 2024-04-22T23:35:07.202 INFO:tasks.workunit.client.1.smithi157.stdout:5/775: creat d1/d5/d7/d2f/d45/d54/d59/f129 x:0 0 0 2024-04-22T23:35:07.202 INFO:tasks.workunit.client.1.smithi157.stdout:5/776: creat d1/db0/df3/f12a x:0 0 0 2024-04-22T23:35:07.202 INFO:tasks.workunit.client.1.smithi157.stdout:5/777: write d1/d5/f4e [881360,69770] 0 2024-04-22T23:35:07.202 INFO:tasks.workunit.client.1.smithi157.stdout:5/778: write d1/f5a [2479755,78749] 0 2024-04-22T23:35:07.202 INFO:tasks.workunit.client.0.smithi107.stdout:7/427: mkdir d7/d2d/d8d 0 2024-04-22T23:35:07.208 INFO:tasks.workunit.client.0.smithi107.stdout:6/520: mknod db/d20/d2a/d5d/c9a 0 2024-04-22T23:35:07.208 INFO:tasks.workunit.client.0.smithi107.stdout:3/537: truncate f0 1192088 0 2024-04-22T23:35:07.208 INFO:tasks.workunit.client.0.smithi107.stdout:3/538: truncate db/d21/d27/d15/d29/d2a/d99/fa8 1007003 0 2024-04-22T23:35:07.208 INFO:tasks.workunit.client.0.smithi107.stdout:2/536: rmdir d1/d12/d35/d48 39 2024-04-22T23:35:07.208 INFO:tasks.workunit.client.0.smithi107.stdout:2/537: stat d1/d12/d35/d48/d13/d4f/da5 0 2024-04-22T23:35:07.208 INFO:tasks.workunit.client.1.smithi157.stdout:8/505: write d2/d6/d66/d97/fa1 [2419135,15939] 0 2024-04-22T23:35:07.210 INFO:tasks.workunit.client.0.smithi107.stdout:2/538: dread d1/d12/f46 [0,4194304] 0 2024-04-22T23:35:07.210 INFO:tasks.workunit.client.1.smithi157.stdout:1/746: dwrite d4/d17/fd6 [0,4194304] 0 2024-04-22T23:35:07.211 INFO:tasks.workunit.client.1.smithi157.stdout:1/747: readlink d4/d17/d34/d84/lf7 0 2024-04-22T23:35:07.211 INFO:tasks.workunit.client.1.smithi157.stdout:1/748: dread - d4/d17/d34/d3b/f76 zero size 2024-04-22T23:35:07.213 INFO:tasks.workunit.client.1.smithi157.stdout:7/594: dwrite d0/d9/d10/f30 [0,4194304] 0 2024-04-22T23:35:07.214 INFO:tasks.workunit.client.1.smithi157.stdout:1/749: mknod d4/d17/d34/d3b/d50/c106 0 2024-04-22T23:35:07.214 INFO:tasks.workunit.client.1.smithi157.stdout:1/750: write d4/d77/fcd [806931,11437] 0 2024-04-22T23:35:07.215 INFO:tasks.workunit.client.1.smithi157.stdout:8/506: rename d2/d34/d6a/d77/d42 to d2/db1 0 2024-04-22T23:35:07.215 INFO:tasks.workunit.client.1.smithi157.stdout:8/507: getdents d2/d4/dad 0 2024-04-22T23:35:07.215 INFO:tasks.workunit.client.1.smithi157.stdout:7/595: truncate d0/d9/f3c 8495344 0 2024-04-22T23:35:07.216 INFO:tasks.workunit.client.0.smithi107.stdout:3/539: unlink db/d21/d27/d15/d24/d66/da5/d8f/fa0 0 2024-04-22T23:35:07.216 INFO:tasks.workunit.client.0.smithi107.stdout:6/521: write db/f3f [567630,34773] 0 2024-04-22T23:35:07.216 INFO:tasks.workunit.client.1.smithi157.stdout:9/711: dwrite d4/d15/d23/d67/f95 [0,4194304] 0 2024-04-22T23:35:07.217 INFO:tasks.workunit.client.1.smithi157.stdout:9/712: write d4/d15/d23/f69 [3057945,17372] 0 2024-04-22T23:35:07.217 INFO:tasks.workunit.client.1.smithi157.stdout:9/713: fsync d4/d15/d23/d27/fb1 0 2024-04-22T23:35:07.217 INFO:tasks.workunit.client.1.smithi157.stdout:1/751: creat d4/db/d3d/dc1/d100/f107 x:0 0 0 2024-04-22T23:35:07.217 INFO:tasks.workunit.client.0.smithi107.stdout:2/539: link d1/d2b/d59/f90 d1/d63/fbd 0 2024-04-22T23:35:07.218 INFO:tasks.workunit.client.0.smithi107.stdout:2/540: creat d1/d2b/d49/d7e/fbe x:0 0 0 2024-04-22T23:35:07.218 INFO:tasks.workunit.client.1.smithi157.stdout:8/508: mknod d2/d4/d30/cb2 0 2024-04-22T23:35:07.218 INFO:tasks.workunit.client.1.smithi157.stdout:8/509: write d2/db1/f5c [61736,5313] 0 2024-04-22T23:35:07.219 INFO:tasks.workunit.client.1.smithi157.stdout:8/510: creat d2/d6/d66/fb3 x:0 0 0 2024-04-22T23:35:07.221 INFO:tasks.workunit.client.0.smithi107.stdout:4/551: rename d1/d8/d67 to d1/d8/daf/dc1 0 2024-04-22T23:35:07.222 INFO:tasks.workunit.client.0.smithi107.stdout:6/522: dread db/d20/d2a/d38/d59/f5a [0,4194304] 0 2024-04-22T23:35:07.224 INFO:tasks.workunit.client.0.smithi107.stdout:4/552: dread d1/d1b/f64 [0,4194304] 0 2024-04-22T23:35:07.225 INFO:tasks.workunit.client.0.smithi107.stdout:3/540: link db/d21/d27/d36/fb0 db/d21/d27/d36/d9c/fba 0 2024-04-22T23:35:07.227 INFO:tasks.workunit.client.1.smithi157.stdout:9/714: dread d4/d15/d55/d8d/fbb [0,4194304] 0 2024-04-22T23:35:07.227 INFO:tasks.workunit.client.1.smithi157.stdout:9/715: write d4/d15/fc5 [768406,118038] 0 2024-04-22T23:35:07.227 INFO:tasks.workunit.client.1.smithi157.stdout:7/596: dread d0/d9/d42/d55/f8f [4194304,4194304] 0 2024-04-22T23:35:07.227 INFO:tasks.workunit.client.1.smithi157.stdout:7/597: chown d0/d9/db/d35/la5 8496990 1 2024-04-22T23:35:07.229 INFO:tasks.workunit.client.1.smithi157.stdout:8/511: dread d2/db1/d58/fa5 [0,4194304] 0 2024-04-22T23:35:07.229 INFO:tasks.workunit.client.1.smithi157.stdout:8/512: creat d2/d6/d66/fb4 x:0 0 0 2024-04-22T23:35:07.229 INFO:tasks.workunit.client.1.smithi157.stdout:8/513: creat d2/d4/d73/fb5 x:0 0 0 2024-04-22T23:35:07.233 INFO:tasks.workunit.client.0.smithi107.stdout:8/490: rename d0/d3a/l93 to d0/d7/d8/d41/d53/d76/la4 0 2024-04-22T23:35:07.234 INFO:tasks.workunit.client.1.smithi157.stdout:1/752: link d4/d17/d34/l83 d4/d77/l108 0 2024-04-22T23:35:07.234 INFO:tasks.workunit.client.1.smithi157.stdout:1/753: chown d4/d17/d34/d84/lc3 34 1 2024-04-22T23:35:07.234 INFO:tasks.workunit.client.1.smithi157.stdout:1/754: stat d4/d17/d34/d75 0 2024-04-22T23:35:07.234 INFO:tasks.workunit.client.1.smithi157.stdout:1/755: chown d4/d77/fdf 765993587 1 2024-04-22T23:35:07.234 INFO:tasks.workunit.client.1.smithi157.stdout:1/756: fdatasync d4/d17/d34/d3b/d52/d65/dbc/faa 0 2024-04-22T23:35:07.235 INFO:tasks.workunit.client.0.smithi107.stdout:6/523: read db/d20/d2a/f42 [4191283,48398] 0 2024-04-22T23:35:07.235 INFO:tasks.workunit.client.0.smithi107.stdout:6/524: write db/d20/d2a/d38/f3c [2741719,114113] 0 2024-04-22T23:35:07.235 INFO:tasks.workunit.client.0.smithi107.stdout:4/553: rmdir d1/d1b/db1 0 2024-04-22T23:35:07.236 INFO:tasks.workunit.client.1.smithi157.stdout:9/716: rmdir d4/df/d57/dbd/dc6 39 2024-04-22T23:35:07.236 INFO:tasks.workunit.client.0.smithi107.stdout:3/541: mkdir db/d21/db1/dbb 0 2024-04-22T23:35:07.238 INFO:tasks.workunit.client.0.smithi107.stdout:9/555: rename d5/d29/dbb/fbf to d5/d14/d28/fc1 0 2024-04-22T23:35:07.238 INFO:tasks.workunit.client.1.smithi157.stdout:7/598: rename d0/d9/d29/d49/d4a/f67 to d0/d9/db/df/d68/fb4 0 2024-04-22T23:35:07.239 INFO:tasks.workunit.client.1.smithi157.stdout:7/599: write d0/d9/d10/f30 [4590404,75957] 0 2024-04-22T23:35:07.241 INFO:tasks.workunit.client.0.smithi107.stdout:6/525: mkdir db/d20/d2a/d38/d61/d6f/d9b 0 2024-04-22T23:35:07.242 INFO:tasks.workunit.client.0.smithi107.stdout:4/554: symlink d1/d7/d53/lc2 0 2024-04-22T23:35:07.242 INFO:tasks.workunit.client.0.smithi107.stdout:4/555: dread - d1/d7/d63/f72 zero size 2024-04-22T23:35:07.242 INFO:tasks.workunit.client.0.smithi107.stdout:4/556: getdents d1/d7/d63/d78/da4/dab 0 2024-04-22T23:35:07.242 INFO:tasks.workunit.client.0.smithi107.stdout:4/557: fsync d1/d21/d29/d2e/fac 0 2024-04-22T23:35:07.242 INFO:tasks.workunit.client.0.smithi107.stdout:4/558: chown d1/d8/l4a 24529 1 2024-04-22T23:35:07.244 INFO:tasks.workunit.client.0.smithi107.stdout:3/542: creat db/d21/d27/d36/fbc x:0 0 0 2024-04-22T23:35:07.244 INFO:tasks.workunit.client.1.smithi157.stdout:7/600: link d0/d9/d29/d33/d7c/f9c d0/d9/d29/d49/d64/fb5 0 2024-04-22T23:35:07.245 INFO:tasks.workunit.client.1.smithi157.stdout:9/717: dread d4/d15/d23/d67/f95 [0,4194304] 0 2024-04-22T23:35:07.245 INFO:tasks.workunit.client.0.smithi107.stdout:2/541: dwrite d1/d67/f7b [0,4194304] 0 2024-04-22T23:35:07.245 INFO:tasks.workunit.client.0.smithi107.stdout:2/542: stat d1/d12/d35/d48/d13/d4f/f53 0 2024-04-22T23:35:07.247 INFO:tasks.workunit.client.0.smithi107.stdout:8/491: dread d0/d7/df/d3b/f3d [0,4194304] 0 2024-04-22T23:35:07.247 INFO:tasks.workunit.client.0.smithi107.stdout:1/472: rename d7/d19/d20/d39/d53/f29 to d7/d19/d20/d39/d53/d43/f97 0 2024-04-22T23:35:07.247 INFO:tasks.workunit.client.0.smithi107.stdout:8/492: write d0/d7/d20/d2d/f30 [91024,108768] 0 2024-04-22T23:35:07.248 INFO:tasks.workunit.client.1.smithi157.stdout:9/718: mkdir d4/d15/d23/d3f/dde 0 2024-04-22T23:35:07.249 INFO:tasks.workunit.client.0.smithi107.stdout:6/526: creat db/d20/d55/f9c x:0 0 0 2024-04-22T23:35:07.249 INFO:tasks.workunit.client.0.smithi107.stdout:6/527: chown db/d20/d2c/c5c 1695227248 1 2024-04-22T23:35:07.251 INFO:tasks.workunit.client.0.smithi107.stdout:4/559: mknod d1/d1b/d44/cc3 0 2024-04-22T23:35:07.251 INFO:tasks.workunit.client.0.smithi107.stdout:4/560: write d1/d7/d5b/dbf/da8/fb8 [745244,94177] 0 2024-04-22T23:35:07.251 INFO:tasks.workunit.client.0.smithi107.stdout:4/561: chown d1/d7/d63/d78/da4 3392 1 2024-04-22T23:35:07.251 INFO:tasks.workunit.client.0.smithi107.stdout:4/562: chown d1/d21/d29/d2e/d88/d99/fb0 459473 1 2024-04-22T23:35:07.252 INFO:tasks.workunit.client.1.smithi157.stdout:1/757: dread d4/db/f14 [0,4194304] 0 2024-04-22T23:35:07.255 INFO:tasks.workunit.client.1.smithi157.stdout:1/758: creat d4/d17/d6b/f109 x:0 0 0 2024-04-22T23:35:07.258 INFO:tasks.workunit.client.1.smithi157.stdout:1/759: mknod d4/d5f/d7e/db8/c10a 0 2024-04-22T23:35:07.258 INFO:tasks.workunit.client.1.smithi157.stdout:1/760: dread - d4/db/d3d/fa1 zero size 2024-04-22T23:35:07.262 INFO:tasks.workunit.client.1.smithi157.stdout:1/761: mkdir d4/d17/d34/d3b/d52/d65/d10b 0 2024-04-22T23:35:07.265 INFO:tasks.workunit.client.0.smithi107.stdout:0/483: rename d1/d9/d14/l19 to d1/d9/d14/d4f/da6/la9 0 2024-04-22T23:35:07.265 INFO:tasks.workunit.client.0.smithi107.stdout:0/484: fsync d1/d27/d20/f60 0 2024-04-22T23:35:07.265 INFO:tasks.workunit.client.1.smithi157.stdout:1/762: getdents d4/d17/d34/dc8 0 2024-04-22T23:35:07.271 INFO:tasks.workunit.client.1.smithi157.stdout:1/763: unlink d4/d17/f2d 0 2024-04-22T23:35:07.271 INFO:tasks.workunit.client.0.smithi107.stdout:3/543: getdents db/d21/d27/d15/d29/d2a/d45 0 2024-04-22T23:35:07.271 INFO:tasks.workunit.client.0.smithi107.stdout:3/544: chown db/d21/d27/d54/d25/d53/f7b 921 1 2024-04-22T23:35:07.277 INFO:tasks.workunit.client.0.smithi107.stdout:1/473: creat d7/d19/d20/d39/d53/f98 x:0 0 0 2024-04-22T23:35:07.283 INFO:tasks.workunit.client.0.smithi107.stdout:7/428: rename d7/d21/d54/c56 to d7/d1b/d51/c8e 0 2024-04-22T23:35:07.283 INFO:tasks.workunit.client.0.smithi107.stdout:9/556: dwrite d5/d29/d7b/d91/f95 [0,4194304] 0 2024-04-22T23:35:07.285 INFO:tasks.workunit.client.1.smithi157.stdout:0/615: sync 2024-04-22T23:35:07.285 INFO:tasks.workunit.client.0.smithi107.stdout:0/485: creat d1/d9/d14/d4f/da6/faa x:0 0 0 2024-04-22T23:35:07.285 INFO:tasks.workunit.client.0.smithi107.stdout:0/486: chown d1/d9/l4e 969 1 2024-04-22T23:35:07.286 INFO:tasks.workunit.client.0.smithi107.stdout:3/545: dread db/d21/d27/d15/d29/d2a/d40/f6c [0,4194304] 0 2024-04-22T23:35:07.288 INFO:tasks.workunit.client.1.smithi157.stdout:0/616: rename d17/d2b/f5c to d17/d2b/d52/dd1/fd4 0 2024-04-22T23:35:07.290 INFO:tasks.workunit.client.1.smithi157.stdout:0/617: unlink d17/d2b/d4f/d5a/da1/da8/c6f 0 2024-04-22T23:35:07.293 INFO:tasks.workunit.client.1.smithi157.stdout:9/719: dwrite d4/d15/d23/d27/f53 [0,4194304] 0 2024-04-22T23:35:07.293 INFO:tasks.workunit.client.1.smithi157.stdout:7/601: dwrite d0/d9/d42/f7a [4194304,4194304] 0 2024-04-22T23:35:07.294 INFO:tasks.workunit.client.0.smithi107.stdout:4/563: getdents d1/d7/d63/d78/d96 0 2024-04-22T23:35:07.295 INFO:tasks.workunit.client.0.smithi107.stdout:0/487: dread d1/d27/f57 [0,4194304] 0 2024-04-22T23:35:07.295 INFO:tasks.workunit.client.0.smithi107.stdout:0/488: creat d1/d67/fab x:0 0 0 2024-04-22T23:35:07.296 INFO:tasks.workunit.client.1.smithi157.stdout:8/514: dwrite d2/d6/f37 [0,4194304] 0 2024-04-22T23:35:07.296 INFO:tasks.workunit.client.1.smithi157.stdout:8/515: write d2/d6/d21/f54 [1172636,49366] 0 2024-04-22T23:35:07.298 INFO:tasks.workunit.client.0.smithi107.stdout:2/543: dwrite d1/d12/d35/d48/d13/fb0 [0,4194304] 0 2024-04-22T23:35:07.298 INFO:tasks.workunit.client.0.smithi107.stdout:2/544: stat d1/d12/d35/d48/f10 0 2024-04-22T23:35:07.299 INFO:tasks.workunit.client.0.smithi107.stdout:6/528: dwrite f6 [4194304,4194304] 0 2024-04-22T23:35:07.300 INFO:tasks.workunit.client.0.smithi107.stdout:6/529: write db/d20/d2c/f91 [377170,95039] 0 2024-04-22T23:35:07.300 INFO:tasks.workunit.client.1.smithi157.stdout:7/602: mkdir d0/d9/d42/d55/d56/d91/db6 0 2024-04-22T23:35:07.300 INFO:tasks.workunit.client.1.smithi157.stdout:7/603: readlink d0/d9/d29/d5e/d85/l8d 0 2024-04-22T23:35:07.301 INFO:tasks.workunit.client.1.smithi157.stdout:8/516: creat d2/d6/d66/d97/fb6 x:0 0 0 2024-04-22T23:35:07.301 INFO:tasks.workunit.client.1.smithi157.stdout:1/764: dwrite d4/d17/fd6 [0,4194304] 0 2024-04-22T23:35:07.301 INFO:tasks.workunit.client.1.smithi157.stdout:1/765: write d4/db/fe [1458288,64151] 0 2024-04-22T23:35:07.301 INFO:tasks.workunit.client.1.smithi157.stdout:1/766: write d4/db/d3d/dc1/d100/f107 [968647,114621] 0 2024-04-22T23:35:07.301 INFO:tasks.workunit.client.0.smithi107.stdout:1/474: mkdir d7/d19/d20/d39/d53/d43/d46/d99 0 2024-04-22T23:35:07.302 INFO:tasks.workunit.client.1.smithi157.stdout:1/767: write d4/d17/d34/d75/dcc/fd7 [449415,76635] 0 2024-04-22T23:35:07.304 INFO:tasks.workunit.client.0.smithi107.stdout:8/493: rename d0/d7/df/d3b/d7b/l98 to d0/d7/la5 0 2024-04-22T23:35:07.307 INFO:tasks.workunit.client.0.smithi107.stdout:7/429: rmdir d7/d2d/d8d 0 2024-04-22T23:35:07.309 INFO:tasks.workunit.client.1.smithi157.stdout:7/604: dread d0/d9/d29/d33/d37/f4c [0,4194304] 0 2024-04-22T23:35:07.309 INFO:tasks.workunit.client.1.smithi157.stdout:7/605: dread - d0/d9/db/df/dac/fad zero size 2024-04-22T23:35:07.309 INFO:tasks.workunit.client.1.smithi157.stdout:1/768: mknod d4/d17/d1d/db2/c10c 0 2024-04-22T23:35:07.311 INFO:tasks.workunit.client.1.smithi157.stdout:0/618: dwrite d17/f2d [0,4194304] 0 2024-04-22T23:35:07.312 INFO:tasks.workunit.client.1.smithi157.stdout:8/517: rmdir d2/d6/d21/d5f/d17 39 2024-04-22T23:35:07.314 INFO:tasks.workunit.client.1.smithi157.stdout:7/606: symlink d0/d9/d29/d5e/d85/lb7 0 2024-04-22T23:35:07.314 INFO:tasks.workunit.client.1.smithi157.stdout:7/607: readlink d0/l5 0 2024-04-22T23:35:07.315 INFO:tasks.workunit.client.1.smithi157.stdout:1/769: mknod d4/d17/d1d/c10d 0 2024-04-22T23:35:07.318 INFO:tasks.workunit.client.0.smithi107.stdout:8/494: write d0/d7/df/d3b/f2b [1712572,39521] 0 2024-04-22T23:35:07.318 INFO:tasks.workunit.client.0.smithi107.stdout:8/495: fsync d0/d3a/d46/f70 0 2024-04-22T23:35:07.318 INFO:tasks.workunit.client.0.smithi107.stdout:8/496: truncate d0/d7/df/d17/f48 848194 0 2024-04-22T23:35:07.318 INFO:tasks.workunit.client.0.smithi107.stdout:8/497: chown d0/d7/df/d3b/f2b 8 1 2024-04-22T23:35:07.319 INFO:tasks.workunit.client.0.smithi107.stdout:4/564: link d1/l4 d1/d8/daf/dc1/lc4 0 2024-04-22T23:35:07.319 INFO:tasks.workunit.client.0.smithi107.stdout:4/565: chown d1/d7/d53/lc2 1816735 1 2024-04-22T23:35:07.320 INFO:tasks.workunit.client.1.smithi157.stdout:8/518: rename d2/d6/d21/d5f/d17/l36 to d2/d6/d21/d5f/d17/d7f/lb7 0 2024-04-22T23:35:07.320 INFO:tasks.workunit.client.1.smithi157.stdout:8/519: write d2/d6/d21/f54 [1435585,52571] 0 2024-04-22T23:35:07.320 INFO:tasks.workunit.client.1.smithi157.stdout:8/520: stat d2/d6/d21/d5f/d17 0 2024-04-22T23:35:07.321 INFO:tasks.workunit.client.0.smithi107.stdout:2/545: mknod d1/cbf 0 2024-04-22T23:35:07.324 INFO:tasks.workunit.client.0.smithi107.stdout:1/475: dwrite d7/d19/d42/f6e [0,4194304] 0 2024-04-22T23:35:07.325 INFO:tasks.workunit.client.0.smithi107.stdout:1/476: write d7/d19/d20/f38 [1490358,64438] 0 2024-04-22T23:35:07.325 INFO:tasks.workunit.client.1.smithi157.stdout:9/720: dwrite d4/df/f26 [0,4194304] 0 2024-04-22T23:35:07.327 INFO:tasks.workunit.client.1.smithi157.stdout:7/608: rmdir d0/d9/d29 39 2024-04-22T23:35:07.329 INFO:tasks.workunit.client.1.smithi157.stdout:8/521: mkdir d2/d4/d30/db8 0 2024-04-22T23:35:07.330 INFO:tasks.workunit.client.1.smithi157.stdout:9/721: mknod d4/df/d57/d6b/cdf 0 2024-04-22T23:35:07.330 INFO:tasks.workunit.client.1.smithi157.stdout:9/722: creat d4/df/d80/fe0 x:0 0 0 2024-04-22T23:35:07.330 INFO:tasks.workunit.client.1.smithi157.stdout:9/723: write d4/df/d57/f44 [2064675,56191] 0 2024-04-22T23:35:07.331 INFO:tasks.workunit.client.0.smithi107.stdout:6/530: mkdir db/d20/d2a/d38/d4b/d9d 0 2024-04-22T23:35:07.331 INFO:tasks.workunit.client.1.smithi157.stdout:7/609: symlink d0/d9/d29/d98/lb8 0 2024-04-22T23:35:07.332 INFO:tasks.workunit.client.1.smithi157.stdout:1/770: getdents d4/d17/db9 0 2024-04-22T23:35:07.333 INFO:tasks.workunit.client.1.smithi157.stdout:1/771: stat d4/d17/da6/fa9 0 2024-04-22T23:35:07.333 INFO:tasks.workunit.client.1.smithi157.stdout:1/772: fdatasync d4/d17/d34/d3b/d50/f56 0 2024-04-22T23:35:07.335 INFO:tasks.workunit.client.0.smithi107.stdout:7/430: creat d7/d2d/d81/f8f x:0 0 0 2024-04-22T23:35:07.337 INFO:tasks.workunit.client.1.smithi157.stdout:7/610: creat d0/d9/de/fb9 x:0 0 0 2024-04-22T23:35:07.338 INFO:tasks.workunit.client.1.smithi157.stdout:7/611: dread d0/d9/db/f57 [0,4194304] 0 2024-04-22T23:35:07.339 INFO:tasks.workunit.client.1.smithi157.stdout:1/773: write d4/d17/d6b/f86 [4965528,28482] 0 2024-04-22T23:35:07.339 INFO:tasks.workunit.client.0.smithi107.stdout:9/557: rmdir d5/d14/d2e/d4f 39 2024-04-22T23:35:07.341 INFO:tasks.workunit.client.0.smithi107.stdout:4/566: unlink d1/fbd 0 2024-04-22T23:35:07.341 INFO:tasks.workunit.client.0.smithi107.stdout:4/567: write d1/d7/f32 [4332723,58553] 0 2024-04-22T23:35:07.343 INFO:tasks.workunit.client.0.smithi107.stdout:2/546: link d1/d12/d35/d48/d13/l76 d1/d12/d35/d48/lc0 0 2024-04-22T23:35:07.343 INFO:tasks.workunit.client.0.smithi107.stdout:2/547: dread - d1/d12/d35/d48/f9d zero size 2024-04-22T23:35:07.343 INFO:tasks.workunit.client.0.smithi107.stdout:2/548: fdatasync d1/d12/f31 0 2024-04-22T23:35:07.343 INFO:tasks.workunit.client.0.smithi107.stdout:2/549: chown d1/d2b/d3d/d57/c9b 356919 1 2024-04-22T23:35:07.343 INFO:tasks.workunit.client.0.smithi107.stdout:2/550: truncate d1/d12/f46 1711699 0 2024-04-22T23:35:07.344 INFO:tasks.workunit.client.0.smithi107.stdout:2/551: dread - d1/d67/f9f zero size 2024-04-22T23:35:07.344 INFO:tasks.workunit.client.1.smithi157.stdout:0/619: dwrite d17/d2b/d4f/d5a/da1/fcf [0,4194304] 0 2024-04-22T23:35:07.347 INFO:tasks.workunit.client.0.smithi107.stdout:1/477: mknod d7/d19/d20/d39/d53/d43/d46/d99/c9a 0 2024-04-22T23:35:07.348 INFO:tasks.workunit.client.0.smithi107.stdout:4/568: write d1/d7/d63/d78/d80/f82 [711288,13075] 0 2024-04-22T23:35:07.350 INFO:tasks.workunit.client.1.smithi157.stdout:7/612: write d0/d9/db/df/f2b [386261,102365] 0 2024-04-22T23:35:07.353 INFO:tasks.workunit.client.1.smithi157.stdout:0/620: symlink d17/d23/d34/d67/d7f/ld5 0 2024-04-22T23:35:07.353 INFO:tasks.workunit.client.0.smithi107.stdout:2/552: unlink d1/d12/d35/l77 0 2024-04-22T23:35:07.353 INFO:tasks.workunit.client.0.smithi107.stdout:2/553: dread - d1/d63/fb6 zero size 2024-04-22T23:35:07.353 INFO:tasks.workunit.client.0.smithi107.stdout:2/554: dread - d1/d12/f95 zero size 2024-04-22T23:35:07.353 INFO:tasks.workunit.client.0.smithi107.stdout:2/555: chown d1/d12/d35/f3f 7 1 2024-04-22T23:35:07.354 INFO:tasks.workunit.client.1.smithi157.stdout:7/613: truncate d0/d9/db/df/d68/f8e 618814 0 2024-04-22T23:35:07.354 INFO:tasks.workunit.client.0.smithi107.stdout:1/478: truncate d7/d19/d20/d39/d53/d43/f7a 3160342 0 2024-04-22T23:35:07.355 INFO:tasks.workunit.client.1.smithi157.stdout:7/614: creat d0/d9/db/d35/fba x:0 0 0 2024-04-22T23:35:07.355 INFO:tasks.workunit.client.1.smithi157.stdout:7/615: chown d0/d9/d29/d49 3 1 2024-04-22T23:35:07.355 INFO:tasks.workunit.client.0.smithi107.stdout:2/556: symlink d1/d12/d35/d48/d13/d4f/lc1 0 2024-04-22T23:35:07.356 INFO:tasks.workunit.client.0.smithi107.stdout:8/498: dwrite d0/d7/d8/f6c [0,4194304] 0 2024-04-22T23:35:07.357 INFO:tasks.workunit.client.0.smithi107.stdout:7/431: dwrite d7/f3b [4194304,4194304] 0 2024-04-22T23:35:07.357 INFO:tasks.workunit.client.0.smithi107.stdout:7/432: creat d7/d21/d54/f90 x:0 0 0 2024-04-22T23:35:07.358 INFO:tasks.workunit.client.1.smithi157.stdout:0/621: mknod d17/d2b/d52/cd6 0 2024-04-22T23:35:07.358 INFO:tasks.workunit.client.1.smithi157.stdout:1/774: dwrite d4/d17/d34/f49 [0,4194304] 0 2024-04-22T23:35:07.358 INFO:tasks.workunit.client.1.smithi157.stdout:1/775: write d4/d5f/fce [907486,89527] 0 2024-04-22T23:35:07.358 INFO:tasks.workunit.client.1.smithi157.stdout:1/776: creat d4/d17/d34/d75/f10e x:0 0 0 2024-04-22T23:35:07.358 INFO:tasks.workunit.client.0.smithi107.stdout:1/479: creat d7/d19/d2d/f9b x:0 0 0 2024-04-22T23:35:07.360 INFO:tasks.workunit.client.1.smithi157.stdout:7/616: mkdir d0/d9/d36/dbb 0 2024-04-22T23:35:07.360 INFO:tasks.workunit.client.0.smithi107.stdout:2/557: symlink d1/d12/d35/d48/lc2 0 2024-04-22T23:35:07.361 INFO:tasks.workunit.client.0.smithi107.stdout:2/558: write d1/d67/f5d [245832,15022] 0 2024-04-22T23:35:07.362 INFO:tasks.workunit.client.0.smithi107.stdout:4/569: dread d1/d21/f36 [0,4194304] 0 2024-04-22T23:35:07.365 INFO:tasks.workunit.client.0.smithi107.stdout:8/499: mknod d0/d85/d51/ca6 0 2024-04-22T23:35:07.366 INFO:tasks.workunit.client.1.smithi157.stdout:0/622: dread d17/f2d [0,4194304] 0 2024-04-22T23:35:07.367 INFO:tasks.workunit.client.1.smithi157.stdout:0/623: creat d17/d2b/fd7 x:0 0 0 2024-04-22T23:35:07.368 INFO:tasks.workunit.client.1.smithi157.stdout:3/717: sync 2024-04-22T23:35:07.368 INFO:tasks.workunit.client.1.smithi157.stdout:4/715: sync 2024-04-22T23:35:07.368 INFO:tasks.workunit.client.1.smithi157.stdout:3/718: creat d0/d3/d34/db9/fe9 x:0 0 0 2024-04-22T23:35:07.368 INFO:tasks.workunit.client.1.smithi157.stdout:2/786: sync 2024-04-22T23:35:07.368 INFO:tasks.workunit.client.1.smithi157.stdout:7/617: unlink d0/la1 0 2024-04-22T23:35:07.368 INFO:tasks.workunit.client.1.smithi157.stdout:7/618: chown d0/d9/d42/d55/d80 6773 1 2024-04-22T23:35:07.369 INFO:tasks.workunit.client.1.smithi157.stdout:2/787: truncate d7/d1f/d3e/f5e 1015890 0 2024-04-22T23:35:07.370 INFO:tasks.workunit.client.0.smithi107.stdout:7/433: creat d7/d2d/d66/f91 x:0 0 0 2024-04-22T23:35:07.370 INFO:tasks.workunit.client.0.smithi107.stdout:7/434: write d7/d21/d48/d5c/f71 [10661,90278] 0 2024-04-22T23:35:07.371 INFO:tasks.workunit.client.1.smithi157.stdout:7/619: mknod d0/d9/d29/d5e/da9/cbc 0 2024-04-22T23:35:07.372 INFO:tasks.workunit.client.1.smithi157.stdout:2/788: mkdir d7/d8/d30/d23/d27/d57/d108 0 2024-04-22T23:35:07.374 INFO:tasks.workunit.client.1.smithi157.stdout:0/624: dread fe [0,4194304] 0 2024-04-22T23:35:07.374 INFO:tasks.workunit.client.1.smithi157.stdout:0/625: write d17/d23/f31 [1227521,116806] 0 2024-04-22T23:35:07.375 INFO:tasks.workunit.client.0.smithi107.stdout:2/559: symlink d1/d12/d35/lc3 0 2024-04-22T23:35:07.375 INFO:tasks.workunit.client.0.smithi107.stdout:2/560: chown d1/d12/d35/d54/laf 4016853 1 2024-04-22T23:35:07.375 INFO:tasks.workunit.client.0.smithi107.stdout:2/561: dread - d1/d2b/d59/fac zero size 2024-04-22T23:35:07.375 INFO:tasks.workunit.client.0.smithi107.stdout:2/562: write d1/d63/f65 [1117103,25753] 0 2024-04-22T23:35:07.375 INFO:tasks.workunit.client.0.smithi107.stdout:4/570: symlink d1/d7/d63/d78/d96/da0/lc5 0 2024-04-22T23:35:07.377 INFO:tasks.workunit.client.0.smithi107.stdout:9/558: dwrite d5/d29/d7b/d91/f95 [0,4194304] 0 2024-04-22T23:35:07.379 INFO:tasks.workunit.client.0.smithi107.stdout:4/571: read d1/d7/d53/d66/f79 [285963,120613] 0 2024-04-22T23:35:07.380 INFO:tasks.workunit.client.1.smithi157.stdout:3/719: dread d0/f41 [0,4194304] 0 2024-04-22T23:35:07.382 INFO:tasks.workunit.client.1.smithi157.stdout:4/716: dread d3/d4b/f45 [0,4194304] 0 2024-04-22T23:35:07.383 INFO:tasks.workunit.client.0.smithi107.stdout:8/500: symlink d0/d85/la7 0 2024-04-22T23:35:07.384 INFO:tasks.workunit.client.0.smithi107.stdout:7/435: mkdir d7/d21/d48/d5c/d92 0 2024-04-22T23:35:07.385 INFO:tasks.workunit.client.0.smithi107.stdout:1/480: dwrite d7/d19/d2d/f31 [0,4194304] 0 2024-04-22T23:35:07.386 INFO:tasks.workunit.client.1.smithi157.stdout:4/717: rename d3/d96/d81 to d3/d96/dee 0 2024-04-22T23:35:07.386 INFO:tasks.workunit.client.1.smithi157.stdout:1/777: dwrite d4/d17/d34/d75/d8a/fd3 [0,4194304] 0 2024-04-22T23:35:07.391 INFO:tasks.workunit.client.0.smithi107.stdout:2/563: rename d1/d2b/f39 to d1/d2b/d59/fc4 0 2024-04-22T23:35:07.391 INFO:tasks.workunit.client.0.smithi107.stdout:5/532: sync 2024-04-22T23:35:07.391 INFO:tasks.workunit.client.0.smithi107.stdout:9/559: creat d5/d29/d7b/d91/dc0/fc2 x:0 0 0 2024-04-22T23:35:07.392 INFO:tasks.workunit.client.0.smithi107.stdout:9/560: readlink d5/d14/d2e/d4f/d73/l7c 0 2024-04-22T23:35:07.392 INFO:tasks.workunit.client.0.smithi107.stdout:9/561: chown d5/d24/f7d 1442203 1 2024-04-22T23:35:07.392 INFO:tasks.workunit.client.1.smithi157.stdout:3/720: dread d0/d3/d34/d3d/d57/d60/f89 [0,4194304] 0 2024-04-22T23:35:07.392 INFO:tasks.workunit.client.1.smithi157.stdout:3/721: fsync d0/d3/d34/d39/f49 0 2024-04-22T23:35:07.392 INFO:tasks.workunit.client.1.smithi157.stdout:3/722: fdatasync d0/d3/d34/d47/fb2 0 2024-04-22T23:35:07.392 INFO:tasks.workunit.client.1.smithi157.stdout:3/723: chown d0/d3/d34/d62/d9a/d9e/dac/cd6 11654463 1 2024-04-22T23:35:07.392 INFO:tasks.workunit.client.1.smithi157.stdout:4/718: creat d3/d11/d1d/d73/d80/dac/fef x:0 0 0 2024-04-22T23:35:07.393 INFO:tasks.workunit.client.0.smithi107.stdout:9/562: write d5/d7/d27/d46/d4b/f56 [228985,26117] 0 2024-04-22T23:35:07.393 INFO:tasks.workunit.client.0.smithi107.stdout:9/563: chown d5/d7/d17/f4e 2452 1 2024-04-22T23:35:07.393 INFO:tasks.workunit.client.0.smithi107.stdout:9/564: stat d5/d14/d28/f81 0 2024-04-22T23:35:07.394 INFO:tasks.workunit.client.1.smithi157.stdout:1/778: creat d4/d17/d34/d3b/d52/d58/f10f x:0 0 0 2024-04-22T23:35:07.394 INFO:tasks.workunit.client.1.smithi157.stdout:3/724: link d0/d3/l12 d0/d3/d34/d62/d9a/d9e/dac/lea 0 2024-04-22T23:35:07.395 INFO:tasks.workunit.client.1.smithi157.stdout:1/779: creat d4/d5f/d7e/f110 x:0 0 0 2024-04-22T23:35:07.397 INFO:tasks.workunit.client.1.smithi157.stdout:1/780: symlink d4/d17/d34/d3b/d52/d65/df1/l111 0 2024-04-22T23:35:07.397 INFO:tasks.workunit.client.1.smithi157.stdout:1/781: creat d4/d17/d34/d75/d8a/f112 x:0 0 0 2024-04-22T23:35:07.398 INFO:tasks.workunit.client.1.smithi157.stdout:2/789: dread d7/d8/d30/fa6 [0,4194304] 0 2024-04-22T23:35:07.399 INFO:tasks.workunit.client.0.smithi107.stdout:8/501: mknod d0/d7/df/d3b/d7b/ca8 0 2024-04-22T23:35:07.399 INFO:tasks.workunit.client.0.smithi107.stdout:7/436: read d7/d1b/f4d [594064,79233] 0 2024-04-22T23:35:07.399 INFO:tasks.workunit.client.1.smithi157.stdout:1/782: mknod d4/d17/d1d/d33/d41/d54/c113 0 2024-04-22T23:35:07.399 INFO:tasks.workunit.client.1.smithi157.stdout:1/783: readlink d4/d5f/d7e/db8/le7 0 2024-04-22T23:35:07.404 INFO:tasks.workunit.client.1.smithi157.stdout:2/790: creat d7/d9f/dc7/f109 x:0 0 0 2024-04-22T23:35:07.406 INFO:tasks.workunit.client.0.smithi107.stdout:5/533: symlink d9/db/d42/lbb 0 2024-04-22T23:35:07.406 INFO:tasks.workunit.client.0.smithi107.stdout:5/534: read f3 [617208,42157] 0 2024-04-22T23:35:07.409 INFO:tasks.workunit.client.0.smithi107.stdout:9/565: truncate d5/d24/f77 1234936 0 2024-04-22T23:35:07.410 INFO:tasks.workunit.client.1.smithi157.stdout:7/620: dwrite d0/d9/d72/f44 [0,4194304] 0 2024-04-22T23:35:07.410 INFO:tasks.workunit.client.1.smithi157.stdout:7/621: chown d0/d9/db/c1b 55003664 1 2024-04-22T23:35:07.410 INFO:tasks.workunit.client.1.smithi157.stdout:7/622: dread - d0/d9/db/df/f9d zero size 2024-04-22T23:35:07.411 INFO:tasks.workunit.client.1.smithi157.stdout:7/623: stat d0/d9/db/da3 0 2024-04-22T23:35:07.411 INFO:tasks.workunit.client.0.smithi107.stdout:4/572: rmdir d1/d21/d29/d2e/d88/d99 39 2024-04-22T23:35:07.411 INFO:tasks.workunit.client.0.smithi107.stdout:4/573: readlink d1/d21/d29/l2b 0 2024-04-22T23:35:07.417 INFO:tasks.workunit.client.0.smithi107.stdout:8/502: creat d0/d7/d20/da2/fa9 x:0 0 0 2024-04-22T23:35:07.418 INFO:tasks.workunit.client.0.smithi107.stdout:7/437: rename d7/l35 to d7/d4c/l93 0 2024-04-22T23:35:07.418 INFO:tasks.workunit.client.0.smithi107.stdout:7/438: chown d7/d2d/l46 22629158 1 2024-04-22T23:35:07.420 INFO:tasks.workunit.client.1.smithi157.stdout:0/626: dwrite f11 [0,4194304] 0 2024-04-22T23:35:07.422 INFO:tasks.workunit.client.1.smithi157.stdout:4/719: dwrite d3/d4b/d7a/fe4 [0,4194304] 0 2024-04-22T23:35:07.422 INFO:tasks.workunit.client.1.smithi157.stdout:4/720: write d3/fda [1297781,87047] 0 2024-04-22T23:35:07.423 INFO:tasks.workunit.client.1.smithi157.stdout:2/791: write d7/d8/d30/d23/f7f [354869,64250] 0 2024-04-22T23:35:07.423 INFO:tasks.workunit.client.1.smithi157.stdout:2/792: getdents d7/d8/d95/dd2 0 2024-04-22T23:35:07.423 INFO:tasks.workunit.client.1.smithi157.stdout:2/793: readlink d7/d8/l1d 0 2024-04-22T23:35:07.424 INFO:tasks.workunit.client.1.smithi157.stdout:0/627: unlink d17/l26 0 2024-04-22T23:35:07.424 INFO:tasks.workunit.client.1.smithi157.stdout:0/628: write d17/d38/f73 [2954291,91481] 0 2024-04-22T23:35:07.424 INFO:tasks.workunit.client.1.smithi157.stdout:1/784: dwrite d4/d77/fdf [0,4194304] 0 2024-04-22T23:35:07.424 INFO:tasks.workunit.client.1.smithi157.stdout:3/725: dwrite d0/d3/f1f [0,4194304] 0 2024-04-22T23:35:07.425 INFO:tasks.workunit.client.0.smithi107.stdout:2/564: rmdir d1/d2b/d49 39 2024-04-22T23:35:07.425 INFO:tasks.workunit.client.0.smithi107.stdout:1/481: dwrite d7/d19/d54/f70 [4194304,4194304] 0 2024-04-22T23:35:07.425 INFO:tasks.workunit.client.0.smithi107.stdout:2/565: chown d1/d12/d35/d48/f10 34386334 1 2024-04-22T23:35:07.425 INFO:tasks.workunit.client.0.smithi107.stdout:2/566: dread - d1/d12/d35/d48/fb3 zero size 2024-04-22T23:35:07.425 INFO:tasks.workunit.client.0.smithi107.stdout:1/482: write d7/d19/d20/d39/d53/f5f [4022727,15645] 0 2024-04-22T23:35:07.425 INFO:tasks.workunit.client.0.smithi107.stdout:1/483: fdatasync d7/d19/d54/f70 0 2024-04-22T23:35:07.425 INFO:tasks.workunit.client.0.smithi107.stdout:1/484: chown d7/d19/d20/d39/d53/d43/d46/d5c/f8a 17 1 2024-04-22T23:35:07.425 INFO:tasks.workunit.client.0.smithi107.stdout:1/485: write d7/d19/d20/d39/d53/f98 [944647,24559] 0 2024-04-22T23:35:07.426 INFO:tasks.workunit.client.1.smithi157.stdout:4/721: link l2 d3/d5c/dd3/lf0 0 2024-04-22T23:35:07.426 INFO:tasks.workunit.client.1.smithi157.stdout:1/785: write d4/d17/d1d/d33/d41/f69 [256409,19949] 0 2024-04-22T23:35:07.426 INFO:tasks.workunit.client.1.smithi157.stdout:1/786: readlink d4/d17/d34/d3b/d52/lac 0 2024-04-22T23:35:07.427 INFO:tasks.workunit.client.0.smithi107.stdout:5/535: creat d9/db/d42/d75/fbc x:0 0 0 2024-04-22T23:35:07.428 INFO:tasks.workunit.client.0.smithi107.stdout:5/536: write d9/db/f8c [364549,115465] 0 2024-04-22T23:35:07.428 INFO:tasks.workunit.client.0.smithi107.stdout:5/537: truncate d9/db/d42/d75/f86 841177 0 2024-04-22T23:35:07.428 INFO:tasks.workunit.client.1.smithi157.stdout:2/794: link d7/d8/dd/d4d/lb6 d7/d8/d25/d55/l10a 0 2024-04-22T23:35:07.428 INFO:tasks.workunit.client.1.smithi157.stdout:2/795: chown d7/d8/d25/d55 2300 1 2024-04-22T23:35:07.430 INFO:tasks.workunit.client.1.smithi157.stdout:3/726: mknod d0/d3/d34/d3d/d5b/d72/ceb 0 2024-04-22T23:35:07.431 INFO:tasks.workunit.client.0.smithi107.stdout:9/566: creat d5/d6c/fc3 x:0 0 0 2024-04-22T23:35:07.432 INFO:tasks.workunit.client.1.smithi157.stdout:1/787: link d4/db/d3d/fa1 d4/db/d71/f114 0 2024-04-22T23:35:07.433 INFO:tasks.workunit.client.0.smithi107.stdout:4/574: creat d1/d21/d29/d2e/d7e/fc6 x:0 0 0 2024-04-22T23:35:07.435 INFO:tasks.workunit.client.1.smithi157.stdout:3/727: symlink d0/d83/lec 0 2024-04-22T23:35:07.435 INFO:tasks.workunit.client.1.smithi157.stdout:3/728: chown d0/d7/f4f 2510122 1 2024-04-22T23:35:07.435 INFO:tasks.workunit.client.1.smithi157.stdout:3/729: truncate d0/d3/d34/d62/fae 521430 0 2024-04-22T23:35:07.436 INFO:tasks.workunit.client.0.smithi107.stdout:8/503: write d0/d7/d8/d41/d53/f67 [3090835,9903] 0 2024-04-22T23:35:07.437 INFO:tasks.workunit.client.1.smithi157.stdout:1/788: rename d4/db/l40 to d4/d17/d1d/d33/l115 0 2024-04-22T23:35:07.439 INFO:tasks.workunit.client.1.smithi157.stdout:2/796: creat d7/d8/d25/d102/ddb/f10b x:0 0 0 2024-04-22T23:35:07.440 INFO:tasks.workunit.client.1.smithi157.stdout:7/624: write d0/d9/f7d [1409635,38581] 0 2024-04-22T23:35:07.440 INFO:tasks.workunit.client.1.smithi157.stdout:3/730: mkdir d0/d3/d34/d47/ded 0 2024-04-22T23:35:07.443 INFO:tasks.workunit.client.1.smithi157.stdout:1/789: symlink d4/d17/d34/d75/d8a/dfc/l116 0 2024-04-22T23:35:07.443 INFO:tasks.workunit.client.0.smithi107.stdout:2/567: read d1/d2b/d49/f5a [1907711,37114] 0 2024-04-22T23:35:07.443 INFO:tasks.workunit.client.0.smithi107.stdout:2/568: chown d1/d12/d35/d48/fb3 56 1 2024-04-22T23:35:07.443 INFO:tasks.workunit.client.0.smithi107.stdout:2/569: stat d1/d2b/d88/fab 0 2024-04-22T23:35:07.444 INFO:tasks.workunit.client.0.smithi107.stdout:2/570: write d1/d2b/d88/fab [120412,32218] 0 2024-04-22T23:35:07.444 INFO:tasks.workunit.client.1.smithi157.stdout:2/797: unlink d7/d1f/led 0 2024-04-22T23:35:07.445 INFO:tasks.workunit.client.1.smithi157.stdout:2/798: fdatasync d7/d8/d25/d55/f106 0 2024-04-22T23:35:07.445 INFO:tasks.workunit.client.0.smithi107.stdout:2/571: dread d1/d2b/d3d/f6c [0,4194304] 0 2024-04-22T23:35:07.445 INFO:tasks.workunit.client.0.smithi107.stdout:2/572: dread - d1/d2b/d3d/f5e zero size 2024-04-22T23:35:07.445 INFO:tasks.workunit.client.0.smithi107.stdout:2/573: truncate d1/f7c 183820 0 2024-04-22T23:35:07.445 INFO:tasks.workunit.client.0.smithi107.stdout:2/574: read - d1/d2b/f80 zero size 2024-04-22T23:35:07.445 INFO:tasks.workunit.client.0.smithi107.stdout:2/575: dread - d1/fad zero size 2024-04-22T23:35:07.446 INFO:tasks.workunit.client.0.smithi107.stdout:7/439: mkdir d7/d21/d7e/d94 0 2024-04-22T23:35:07.450 INFO:tasks.workunit.client.1.smithi157.stdout:1/790: chown d4/d17/d34/d3b/d52/d65/dae/f99 15 1 2024-04-22T23:35:07.450 INFO:tasks.workunit.client.1.smithi157.stdout:1/791: chown d4/d77/f93 600485 1 2024-04-22T23:35:07.451 INFO:tasks.workunit.client.0.smithi107.stdout:5/538: dwrite d9/db/d42/d75/db6/fb9 [0,4194304] 0 2024-04-22T23:35:07.451 INFO:tasks.workunit.client.1.smithi157.stdout:7/625: write d0/f7 [957382,102351] 0 2024-04-22T23:35:07.451 INFO:tasks.workunit.client.1.smithi157.stdout:7/626: dread - d0/d9/d29/d49/f84 zero size 2024-04-22T23:35:07.452 INFO:tasks.workunit.client.1.smithi157.stdout:2/799: rename d7/d8/dd/d4d/c84 to d7/d9f/dc7/c10c 0 2024-04-22T23:35:07.452 INFO:tasks.workunit.client.1.smithi157.stdout:2/800: write d7/d8/d95/dd2/dda/f8b [1667089,63749] 0 2024-04-22T23:35:07.453 INFO:tasks.workunit.client.1.smithi157.stdout:2/801: write d7/d8/d25/f68 [863457,99918] 0 2024-04-22T23:35:07.457 INFO:tasks.workunit.client.0.smithi107.stdout:4/575: getdents d1/d7/d5b/dbf/da8 0 2024-04-22T23:35:07.457 INFO:tasks.workunit.client.0.smithi107.stdout:4/576: fsync d1/d8/daf/dc1/f8f 0 2024-04-22T23:35:07.457 INFO:tasks.workunit.client.1.smithi157.stdout:7/627: mkdir d0/d9/d29/dbd 0 2024-04-22T23:35:07.458 INFO:tasks.workunit.client.1.smithi157.stdout:7/628: chown d0/d9/d29/l79 1775 1 2024-04-22T23:35:07.458 INFO:tasks.workunit.client.1.smithi157.stdout:7/629: chown d0/f46 159168 1 2024-04-22T23:35:07.458 INFO:tasks.workunit.client.0.smithi107.stdout:1/486: dwrite d7/d19/d42/f6e [0,4194304] 0 2024-04-22T23:35:07.461 INFO:tasks.workunit.client.0.smithi107.stdout:8/504: creat d0/d7/df/d33/faa x:0 0 0 2024-04-22T23:35:07.462 INFO:tasks.workunit.client.1.smithi157.stdout:0/629: dwrite d17/d2b/d4f/d5a/f63 [0,4194304] 0 2024-04-22T23:35:07.464 INFO:tasks.workunit.client.0.smithi107.stdout:2/576: creat d1/d2b/d49/fc5 x:0 0 0 2024-04-22T23:35:07.465 INFO:tasks.workunit.client.0.smithi107.stdout:7/440: mkdir d7/d21/d48/d95 0 2024-04-22T23:35:07.465 INFO:tasks.workunit.client.0.smithi107.stdout:7/441: creat d7/d21/f96 x:0 0 0 2024-04-22T23:35:07.465 INFO:tasks.workunit.client.0.smithi107.stdout:7/442: dread - d7/d21/d54/f90 zero size 2024-04-22T23:35:07.465 INFO:tasks.workunit.client.0.smithi107.stdout:7/443: fsync d7/d21/d48/f6b 0 2024-04-22T23:35:07.465 INFO:tasks.workunit.client.0.smithi107.stdout:7/444: dread - d7/d21/d48/f5b zero size 2024-04-22T23:35:07.465 INFO:tasks.workunit.client.0.smithi107.stdout:7/445: chown d7/d21 0 1 2024-04-22T23:35:07.472 INFO:tasks.workunit.client.1.smithi157.stdout:0/630: symlink d17/d23/d34/d67/d7f/d69/dce/ld8 0 2024-04-22T23:35:07.472 INFO:tasks.workunit.client.0.smithi107.stdout:9/567: rmdir d5/d7/d17/d1b/d1f/d36 39 2024-04-22T23:35:07.477 INFO:tasks.workunit.client.1.smithi157.stdout:2/802: dread d7/d8/d30/f9c [0,4194304] 0 2024-04-22T23:35:07.477 INFO:tasks.workunit.client.0.smithi107.stdout:8/505: symlink d0/d7/d20/d2d/lab 0 2024-04-22T23:35:07.479 INFO:tasks.workunit.client.0.smithi107.stdout:2/577: mkdir d1/d2b/d49/d7e/dc6 0 2024-04-22T23:35:07.479 INFO:tasks.workunit.client.0.smithi107.stdout:2/578: chown d1/f2d 32134509 1 2024-04-22T23:35:07.479 INFO:tasks.workunit.client.1.smithi157.stdout:2/803: rename d7/d8/d25/c40 to d7/d8/d8e/d80/c10d 0 2024-04-22T23:35:07.479 INFO:tasks.workunit.client.1.smithi157.stdout:2/804: fdatasync d7/d8/d25/f50 0 2024-04-22T23:35:07.481 INFO:tasks.workunit.client.1.smithi157.stdout:3/731: dwrite d0/d3/d34/d39/f56 [0,4194304] 0 2024-04-22T23:35:07.483 INFO:tasks.workunit.client.1.smithi157.stdout:1/792: dwrite d4/d17/d6b/f97 [0,4194304] 0 2024-04-22T23:35:07.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:35:07 smithi107 ceph-mon[26355]: pgmap v260: 65 pgs: 65 active+clean; 1.6 GiB data, 6.1 GiB used, 530 GiB / 536 GiB avail; 181 MiB/s rd, 210 MiB/s wr, 753 op/s 2024-04-22T23:35:07.484 INFO:tasks.workunit.client.1.smithi157.stdout:2/805: rename d7/d8/d30/d23/fbe to d7/d8/dca/f10e 0 2024-04-22T23:35:07.484 INFO:tasks.workunit.client.1.smithi157.stdout:2/806: write d7/d8/d8e/f78 [1333177,47745] 0 2024-04-22T23:35:07.484 INFO:tasks.workunit.client.0.smithi107.stdout:7/446: rename d7/d21/f89 to d7/d2d/d81/f97 0 2024-04-22T23:35:07.485 INFO:tasks.workunit.client.0.smithi107.stdout:7/447: write d7/d1b/f53 [97022,20293] 0 2024-04-22T23:35:07.485 INFO:tasks.workunit.client.0.smithi107.stdout:7/448: creat d7/d21/d54/f98 x:0 0 0 2024-04-22T23:35:07.485 INFO:tasks.workunit.client.1.smithi157.stdout:0/631: dread d17/f19 [4194304,4194304] 0 2024-04-22T23:35:07.487 INFO:tasks.workunit.client.1.smithi157.stdout:3/732: dread d0/f41 [0,4194304] 0 2024-04-22T23:35:07.487 INFO:tasks.workunit.client.0.smithi107.stdout:4/577: rmdir d1/d1b 39 2024-04-22T23:35:07.488 INFO:tasks.workunit.client.0.smithi107.stdout:4/578: creat d1/d21/d29/d2e/d7e/fc7 x:0 0 0 2024-04-22T23:35:07.489 INFO:tasks.workunit.client.1.smithi157.stdout:1/793: link d4/d17/d1d/db2/lc5 d4/d17/d34/d3b/d52/d65/d10b/l117 0 2024-04-22T23:35:07.489 INFO:tasks.workunit.client.1.smithi157.stdout:1/794: dread - d4/d17/d34/d75/d8a/f112 zero size 2024-04-22T23:35:07.492 INFO:tasks.workunit.client.0.smithi107.stdout:2/579: write d1/d67/f7b [3854954,87473] 0 2024-04-22T23:35:07.494 INFO:tasks.workunit.client.1.smithi157.stdout:3/733: unlink d0/d83/l9d 0 2024-04-22T23:35:07.496 INFO:tasks.workunit.client.0.smithi107.stdout:7/449: mknod d7/d1b/d51/d6e/c99 0 2024-04-22T23:35:07.496 INFO:tasks.workunit.client.0.smithi107.stdout:7/450: readlink d7/d4c/l8a 0 2024-04-22T23:35:07.496 INFO:tasks.workunit.client.0.smithi107.stdout:7/451: dread - d7/d21/d49/f6a zero size 2024-04-22T23:35:07.497 INFO:tasks.workunit.client.0.smithi107.stdout:7/452: stat d7/d1b/l77 0 2024-04-22T23:35:07.497 INFO:tasks.workunit.client.0.smithi107.stdout:7/453: fdatasync d7/d21/f22 0 2024-04-22T23:35:07.497 INFO:tasks.workunit.client.1.smithi157.stdout:0/632: dread d17/f20 [0,4194304] 0 2024-04-22T23:35:07.499 INFO:tasks.workunit.client.1.smithi157.stdout:3/734: rename d0/d3/d34/d62/d9a to d0/d3/d34/d3d/d5b/dee 0 2024-04-22T23:35:07.500 INFO:tasks.workunit.client.0.smithi107.stdout:4/579: getdents d1/d7 0 2024-04-22T23:35:07.500 INFO:tasks.workunit.client.0.smithi107.stdout:4/580: write d1/d7/d63/d78/d96/fbc [975128,8994] 0 2024-04-22T23:35:07.502 INFO:tasks.workunit.client.0.smithi107.stdout:8/506: getdents d0/d73 0 2024-04-22T23:35:07.502 INFO:tasks.workunit.client.0.smithi107.stdout:9/568: dwrite d5/d29/d7b/d91/f95 [0,4194304] 0 2024-04-22T23:35:07.505 INFO:tasks.workunit.client.1.smithi157.stdout:3/735: rmdir d0/d3/d34/d47/dcd 39 2024-04-22T23:35:07.506 INFO:tasks.workunit.client.0.smithi107.stdout:9/569: write d5/d24/f7d [384816,120718] 0 2024-04-22T23:35:07.506 INFO:tasks.workunit.client.0.smithi107.stdout:2/580: mknod d1/d2b/d3d/d57/cc7 0 2024-04-22T23:35:07.508 INFO:tasks.workunit.client.1.smithi157.stdout:5/779: sync 2024-04-22T23:35:07.510 INFO:tasks.workunit.client.0.smithi107.stdout:7/454: symlink d7/d21/l9a 0 2024-04-22T23:35:07.510 INFO:tasks.workunit.client.0.smithi107.stdout:7/455: dread - d7/d21/d49/f6a zero size 2024-04-22T23:35:07.510 INFO:tasks.workunit.client.0.smithi107.stdout:7/456: read d7/ff [5888676,100777] 0 2024-04-22T23:35:07.510 INFO:tasks.workunit.client.1.smithi157.stdout:5/780: rmdir d1/d5/dc 39 2024-04-22T23:35:07.514 INFO:tasks.workunit.client.0.smithi107.stdout:4/581: creat d1/d8/fc8 x:0 0 0 2024-04-22T23:35:07.514 INFO:tasks.workunit.client.0.smithi107.stdout:7/457: dread d7/f2f [0,4194304] 0 2024-04-22T23:35:07.517 INFO:tasks.workunit.client.0.smithi107.stdout:5/539: rmdir d9/db/d42/d75 39 2024-04-22T23:35:07.524 INFO:tasks.workunit.client.0.smithi107.stdout:9/570: write d5/d29/f61 [2335356,51814] 0 2024-04-22T23:35:07.525 INFO:tasks.workunit.client.0.smithi107.stdout:9/571: stat d5/d7/d17/d1b/d1f 0 2024-04-22T23:35:07.527 INFO:tasks.workunit.client.0.smithi107.stdout:2/581: creat d1/d2b/d49/d7e/fc8 x:0 0 0 2024-04-22T23:35:07.527 INFO:tasks.workunit.client.0.smithi107.stdout:2/582: chown d1/d12/d35/d54/f5f 1593706027 1 2024-04-22T23:35:07.527 INFO:tasks.workunit.client.0.smithi107.stdout:7/458: dread d7/f3b [4194304,4194304] 0 2024-04-22T23:35:07.527 INFO:tasks.workunit.client.0.smithi107.stdout:7/459: stat d7/d4c/l4e 0 2024-04-22T23:35:07.529 INFO:tasks.workunit.client.0.smithi107.stdout:5/540: dread d9/f43 [0,4194304] 0 2024-04-22T23:35:07.530 INFO:tasks.workunit.client.0.smithi107.stdout:5/541: write d9/db/f8f [96059,61490] 0 2024-04-22T23:35:07.531 INFO:tasks.workunit.client.0.smithi107.stdout:9/572: rename d5/d7/d27/d69/l93 to d5/d7/d27/d69/lc4 0 2024-04-22T23:35:07.533 INFO:tasks.workunit.client.0.smithi107.stdout:2/583: mkdir d1/d2b/d88/dc9 0 2024-04-22T23:35:07.533 INFO:tasks.workunit.client.0.smithi107.stdout:2/584: chown d1/d12/d35/l41 3 1 2024-04-22T23:35:07.534 INFO:tasks.workunit.client.0.smithi107.stdout:2/585: write d1/d63/fa1 [281628,112969] 0 2024-04-22T23:35:07.534 INFO:tasks.workunit.client.0.smithi107.stdout:7/460: unlink d7/d21/d48/c6f 0 2024-04-22T23:35:07.534 INFO:tasks.workunit.client.0.smithi107.stdout:7/461: fsync d7/d21/d48/d5c/f7a 0 2024-04-22T23:35:07.534 INFO:tasks.workunit.client.0.smithi107.stdout:7/462: readlink d7/d1b/d51/l65 0 2024-04-22T23:35:07.536 INFO:tasks.workunit.client.1.smithi157.stdout:0/633: dwrite d17/d2b/d71/fc4 [0,4194304] 0 2024-04-22T23:35:07.537 INFO:tasks.workunit.client.1.smithi157.stdout:0/634: write d17/d23/d34/f9e [578981,102958] 0 2024-04-22T23:35:07.537 INFO:tasks.workunit.client.1.smithi157.stdout:0/635: truncate d17/d23/f31 1641556 0 2024-04-22T23:35:07.537 INFO:tasks.workunit.client.0.smithi107.stdout:5/542: truncate d9/db/f5f 1498468 0 2024-04-22T23:35:07.539 INFO:tasks.workunit.client.1.smithi157.stdout:1/795: dwrite d4/d5f/d7e/db8/fdd [0,4194304] 0 2024-04-22T23:35:07.539 INFO:tasks.workunit.client.0.smithi107.stdout:9/573: creat d5/d7/d17/d1b/fc5 x:0 0 0 2024-04-22T23:35:07.539 INFO:tasks.workunit.client.0.smithi107.stdout:9/574: creat d5/d7/d27/d46/fc6 x:0 0 0 2024-04-22T23:35:07.540 INFO:tasks.workunit.client.1.smithi157.stdout:1/796: write d4/db/f7d [95812,67248] 0 2024-04-22T23:35:07.540 INFO:tasks.workunit.client.0.smithi107.stdout:5/543: mknod d9/cbd 0 2024-04-22T23:35:07.544 INFO:tasks.workunit.client.0.smithi107.stdout:9/575: mkdir d5/d7/d27/d46/dc7 0 2024-04-22T23:35:07.544 INFO:tasks.workunit.client.0.smithi107.stdout:9/576: creat d5/d14/d2e/d43/fc8 x:0 0 0 2024-04-22T23:35:07.546 INFO:tasks.workunit.client.1.smithi157.stdout:2/807: dwrite d7/d8/d30/d5b/f73 [0,4194304] 0 2024-04-22T23:35:07.546 INFO:tasks.workunit.client.1.smithi157.stdout:2/808: stat d7/d8/d95/dd2/dda/f8c 0 2024-04-22T23:35:07.548 INFO:tasks.workunit.client.1.smithi157.stdout:5/781: dwrite d1/d5/d7/d2f/d45/d54/d59/f70 [4194304,4194304] 0 2024-04-22T23:35:07.548 INFO:tasks.workunit.client.1.smithi157.stdout:5/782: creat d1/d5/d7/d2f/d45/d54/f12b x:0 0 0 2024-04-22T23:35:07.548 INFO:tasks.workunit.client.0.smithi107.stdout:5/544: unlink d9/d26/f5b 0 2024-04-22T23:35:07.549 INFO:tasks.workunit.client.1.smithi157.stdout:3/736: dwrite d0/d7/d6c/fc7 [0,4194304] 0 2024-04-22T23:35:07.550 INFO:tasks.workunit.client.0.smithi107.stdout:4/582: dwrite d1/d21/d29/d2e/d3b/f90 [0,4194304] 0 2024-04-22T23:35:07.551 INFO:tasks.workunit.client.1.smithi157.stdout:2/809: mkdir d7/d8/d95/dea/d10f 0 2024-04-22T23:35:07.551 INFO:tasks.workunit.client.1.smithi157.stdout:2/810: chown d7/d8/d25/f4f 4288381 1 2024-04-22T23:35:07.552 INFO:tasks.workunit.client.0.smithi107.stdout:5/545: dread d9/db/f8f [0,4194304] 0 2024-04-22T23:35:07.554 INFO:tasks.workunit.client.1.smithi157.stdout:1/797: write d4/d17/d1d/d33/d41/d54/f98 [3535106,81243] 0 2024-04-22T23:35:07.554 INFO:tasks.workunit.client.1.smithi157.stdout:5/783: creat d1/f12c x:0 0 0 2024-04-22T23:35:07.554 INFO:tasks.workunit.client.1.smithi157.stdout:1/798: creat d4/d17/d34/d3b/d52/d65/dae/f118 x:0 0 0 2024-04-22T23:35:07.554 INFO:tasks.workunit.client.1.smithi157.stdout:3/737: symlink d0/d3/d34/lef 0 2024-04-22T23:35:07.555 INFO:tasks.workunit.client.1.smithi157.stdout:1/799: stat d4/db/lb5 0 2024-04-22T23:35:07.556 INFO:tasks.workunit.client.0.smithi107.stdout:5/546: mkdir d9/db/de/dbe 0 2024-04-22T23:35:07.557 INFO:tasks.workunit.client.1.smithi157.stdout:5/784: rename d1/db0/dc1/lce to d1/d1f/l12d 0 2024-04-22T23:35:07.560 INFO:tasks.workunit.client.1.smithi157.stdout:1/800: creat d4/d17/d34/d3b/d52/d65/df1/f119 x:0 0 0 2024-04-22T23:35:07.565 INFO:tasks.workunit.client.0.smithi107.stdout:7/463: dwrite d7/d1b/f53 [0,4194304] 0 2024-04-22T23:35:07.566 INFO:tasks.workunit.client.1.smithi157.stdout:0/636: dwrite d17/d2b/d4f/d5a/f63 [0,4194304] 0 2024-04-22T23:35:07.568 INFO:tasks.workunit.client.1.smithi157.stdout:2/811: dread d7/d8/d95/dd2/dda/f89 [0,4194304] 0 2024-04-22T23:35:07.568 INFO:tasks.workunit.client.1.smithi157.stdout:2/812: chown d7/d8/dd/c1b 1 1 2024-04-22T23:35:07.570 INFO:tasks.workunit.client.1.smithi157.stdout:0/637: truncate d17/d2b/d52/d7e/fd3 143992 0 2024-04-22T23:35:07.570 INFO:tasks.workunit.client.1.smithi157.stdout:0/638: chown d17/d2b/d4f/d5a/d7c/f98 71 1 2024-04-22T23:35:07.572 INFO:tasks.workunit.client.1.smithi157.stdout:2/813: creat d7/d8/d30/d23/f110 x:0 0 0 2024-04-22T23:35:07.572 INFO:tasks.workunit.client.1.smithi157.stdout:1/801: write d4/d77/fdf [3852815,76114] 0 2024-04-22T23:35:07.572 INFO:tasks.workunit.client.1.smithi157.stdout:0/639: symlink d17/d23/ld9 0 2024-04-22T23:35:07.572 INFO:tasks.workunit.client.1.smithi157.stdout:0/640: creat d17/d23/fda x:0 0 0 2024-04-22T23:35:07.580 INFO:tasks.workunit.client.1.smithi157.stdout:1/802: mkdir d4/d17/d34/d3b/d11a 0 2024-04-22T23:35:07.580 INFO:tasks.workunit.client.1.smithi157.stdout:1/803: write d4/d17/f59 [44973,18449] 0 2024-04-22T23:35:07.580 INFO:tasks.workunit.client.1.smithi157.stdout:0/641: mkdir d17/ddb 0 2024-04-22T23:35:07.582 INFO:tasks.workunit.client.0.smithi107.stdout:7/464: symlink d7/l9b 0 2024-04-22T23:35:07.583 INFO:tasks.workunit.client.0.smithi107.stdout:7/465: fdatasync d7/f26 0 2024-04-22T23:35:07.583 INFO:tasks.workunit.client.0.smithi107.stdout:7/466: dread - d7/d21/d54/f58 zero size 2024-04-22T23:35:07.583 INFO:tasks.workunit.client.1.smithi157.stdout:0/642: getdents d17/d23/d9c 0 2024-04-22T23:35:07.583 INFO:tasks.workunit.client.1.smithi157.stdout:0/643: getdents d17/d23/d34/d67/d78/dc7 0 2024-04-22T23:35:07.583 INFO:tasks.workunit.client.1.smithi157.stdout:0/644: stat d17/d2b/fb4 0 2024-04-22T23:35:07.583 INFO:tasks.workunit.client.1.smithi157.stdout:0/645: chown d17/d2b/d52/cd6 5385765 1 2024-04-22T23:35:07.585 INFO:tasks.workunit.client.1.smithi157.stdout:0/646: unlink d17/d2b/l79 0 2024-04-22T23:35:07.586 INFO:tasks.workunit.client.0.smithi107.stdout:6/531: sync 2024-04-22T23:35:07.586 INFO:tasks.workunit.client.0.smithi107.stdout:3/546: sync 2024-04-22T23:35:07.586 INFO:tasks.workunit.client.0.smithi107.stdout:6/532: readlink db/l19 0 2024-04-22T23:35:07.586 INFO:tasks.workunit.client.0.smithi107.stdout:6/533: chown db/d20/d2a/d35/d69/d79/c87 986 1 2024-04-22T23:35:07.586 INFO:tasks.workunit.client.0.smithi107.stdout:0/489: sync 2024-04-22T23:35:07.587 INFO:tasks.workunit.client.0.smithi107.stdout:0/490: dread d1/d9/d14/f8c [0,4194304] 0 2024-04-22T23:35:07.588 INFO:tasks.workunit.client.1.smithi157.stdout:5/785: dwrite d1/d5/d7/d2f/d9c/faa [0,4194304] 0 2024-04-22T23:35:07.588 INFO:tasks.workunit.client.0.smithi107.stdout:3/547: creat db/d21/d27/d15/d24/d52/fbd x:0 0 0 2024-04-22T23:35:07.589 INFO:tasks.workunit.client.1.smithi157.stdout:0/647: mkdir d17/d2b/d52/ddc 0 2024-04-22T23:35:07.589 INFO:tasks.workunit.client.1.smithi157.stdout:0/648: write d17/d23/d34/fb6 [407711,69906] 0 2024-04-22T23:35:07.590 INFO:tasks.workunit.client.0.smithi107.stdout:6/534: mkdir db/d20/d2c/d3b/d40/d9e 0 2024-04-22T23:35:07.592 INFO:tasks.workunit.client.0.smithi107.stdout:0/491: creat d1/d27/d20/d2c/d41/fac x:0 0 0 2024-04-22T23:35:07.594 INFO:tasks.workunit.client.0.smithi107.stdout:3/548: symlink db/d21/d27/d15/d29/d4f/db2/lbe 0 2024-04-22T23:35:07.597 INFO:tasks.workunit.client.0.smithi107.stdout:3/549: rename db/d21/d27/d54/da7/lb5 to db/d21/d27/d15/d24/lbf 0 2024-04-22T23:35:07.597 INFO:tasks.workunit.client.0.smithi107.stdout:3/550: write f0 [1984096,63038] 0 2024-04-22T23:35:07.598 INFO:tasks.workunit.client.1.smithi157.stdout:2/814: dwrite d7/d1f/d26/d4a/f60 [0,4194304] 0 2024-04-22T23:35:07.598 INFO:tasks.workunit.client.1.smithi157.stdout:2/815: chown d7/d8/d30/d23/c32 30 1 2024-04-22T23:35:07.599 INFO:tasks.workunit.client.1.smithi157.stdout:2/816: dread - d7/d8/d95/fcb zero size 2024-04-22T23:35:07.599 INFO:tasks.workunit.client.1.smithi157.stdout:0/649: rmdir d17/d2b/d52/dd1 39 2024-04-22T23:35:07.599 INFO:tasks.workunit.client.0.smithi107.stdout:5/547: dwrite d9/fa5 [0,4194304] 0 2024-04-22T23:35:07.599 INFO:tasks.workunit.client.0.smithi107.stdout:7/467: dread d7/d2d/f41 [0,4194304] 0 2024-04-22T23:35:07.600 INFO:tasks.workunit.client.0.smithi107.stdout:7/468: dread - d7/d2d/d81/f97 zero size 2024-04-22T23:35:07.602 INFO:tasks.workunit.client.0.smithi107.stdout:3/551: dread - db/d21/d27/d36/fbc zero size 2024-04-22T23:35:07.602 INFO:tasks.workunit.client.0.smithi107.stdout:3/552: fsync db/d21/d27/d15/db8/f94 0 2024-04-22T23:35:07.602 INFO:tasks.workunit.client.0.smithi107.stdout:3/553: chown db/d21/d27/d15/d24 2064523 1 2024-04-22T23:35:07.603 INFO:tasks.workunit.client.1.smithi157.stdout:5/786: write d1/d5/d7/f67 [45702,111438] 0 2024-04-22T23:35:07.603 INFO:tasks.workunit.client.1.smithi157.stdout:5/787: write d1/d5/d7/d2f/d45/da4/f113 [168932,25371] 0 2024-04-22T23:35:07.604 INFO:tasks.workunit.client.1.smithi157.stdout:2/817: creat d7/d8/d30/d23/d27/d57/d108/f111 x:0 0 0 2024-04-22T23:35:07.604 INFO:tasks.workunit.client.0.smithi107.stdout:5/548: creat d9/d76/da0/fbf x:0 0 0 2024-04-22T23:35:07.605 INFO:tasks.workunit.client.0.smithi107.stdout:5/549: write d9/db/de/d5e/f90 [328000,122485] 0 2024-04-22T23:35:07.606 INFO:tasks.workunit.client.0.smithi107.stdout:5/550: symlink d9/db/d42/lc0 0 2024-04-22T23:35:07.611 INFO:tasks.workunit.client.1.smithi157.stdout:5/788: mknod d1/db0/c12e 0 2024-04-22T23:35:07.612 INFO:tasks.workunit.client.1.smithi157.stdout:3/738: dwrite d0/d3/d13/f2b [0,4194304] 0 2024-04-22T23:35:07.614 INFO:tasks.workunit.client.1.smithi157.stdout:2/818: unlink d7/d8/d30/l5d 0 2024-04-22T23:35:07.622 INFO:tasks.workunit.client.1.smithi157.stdout:1/804: dwrite d4/f25 [4194304,4194304] 0 2024-04-22T23:35:07.625 INFO:tasks.workunit.client.1.smithi157.stdout:1/805: rmdir d4/d17/d34/d3b/d52/d65/dae 39 2024-04-22T23:35:07.625 INFO:tasks.workunit.client.1.smithi157.stdout:1/806: dread - d4/d17/d1d/d33/d41/ffb zero size 2024-04-22T23:35:07.626 INFO:tasks.workunit.client.0.smithi107.stdout:6/535: dwrite db/d20/d2a/d38/d4b/f6b [0,4194304] 0 2024-04-22T23:35:07.628 INFO:tasks.workunit.client.1.smithi157.stdout:1/807: getdents d4/d17/d34/d75/d8a 0 2024-04-22T23:35:07.630 INFO:tasks.workunit.client.0.smithi107.stdout:6/536: creat db/d20/d2a/d38/f9f x:0 0 0 2024-04-22T23:35:07.631 INFO:tasks.workunit.client.1.smithi157.stdout:1/808: link d4/d5f/d7e/db8/lf4 d4/d17/d34/d3b/d11a/l11b 0 2024-04-22T23:35:07.632 INFO:tasks.workunit.client.1.smithi157.stdout:1/809: readlink d4/d17/d34/d3b/d50/dad/lde 0 2024-04-22T23:35:07.632 INFO:tasks.workunit.client.1.smithi157.stdout:1/810: chown d4/d17/fc4 1319243077 1 2024-04-22T23:35:07.632 INFO:tasks.workunit.client.1.smithi157.stdout:1/811: dread - d4/d17/d34/d3b/d50/ff8 zero size 2024-04-22T23:35:07.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:35:07 smithi157 ceph-mon[33877]: pgmap v260: 65 pgs: 65 active+clean; 1.6 GiB data, 6.1 GiB used, 530 GiB / 536 GiB avail; 181 MiB/s rd, 210 MiB/s wr, 753 op/s 2024-04-22T23:35:07.632 INFO:tasks.workunit.client.1.smithi157.stdout:0/650: dwrite d17/d2b/d71/db3/fc8 [4194304,4194304] 0 2024-04-22T23:35:07.635 INFO:tasks.workunit.client.1.smithi157.stdout:1/812: rename d4/d17/d34/d3b/d52/d65/dbc/l79 to d4/db/l11c 0 2024-04-22T23:35:07.637 INFO:tasks.workunit.client.1.smithi157.stdout:0/651: creat d17/d23/d9c/fdd x:0 0 0 2024-04-22T23:35:07.637 INFO:tasks.workunit.client.1.smithi157.stdout:0/652: dread - d17/d38/fb9 zero size 2024-04-22T23:35:07.640 INFO:tasks.workunit.client.1.smithi157.stdout:0/653: link d17/c2f d17/d23/d34/d67/d7f/d69/cde 0 2024-04-22T23:35:07.640 INFO:tasks.workunit.client.1.smithi157.stdout:0/654: write d17/d23/f32 [575075,108075] 0 2024-04-22T23:35:07.640 INFO:tasks.workunit.client.1.smithi157.stdout:0/655: chown d17/c36 0 1 2024-04-22T23:35:07.640 INFO:tasks.workunit.client.1.smithi157.stdout:0/656: dread - d17/d23/d34/d67/f8d zero size 2024-04-22T23:35:07.642 INFO:tasks.workunit.client.1.smithi157.stdout:0/657: symlink d17/d2b/d4f/d9b/ldf 0 2024-04-22T23:35:07.643 INFO:tasks.workunit.client.0.smithi107.stdout:0/492: dwrite d1/d9/d14/d4f/f82 [0,4194304] 0 2024-04-22T23:35:07.645 INFO:tasks.workunit.client.1.smithi157.stdout:0/658: getdents d17/d2b 0 2024-04-22T23:35:07.646 INFO:tasks.workunit.client.0.smithi107.stdout:0/493: rmdir d1/d27/d20/d4a 39 2024-04-22T23:35:07.646 INFO:tasks.workunit.client.0.smithi107.stdout:0/494: fsync d1/f8f 0 2024-04-22T23:35:07.648 INFO:tasks.workunit.client.0.smithi107.stdout:0/495: rename d1/d67 to d1/d2a/dad 0 2024-04-22T23:35:07.648 INFO:tasks.workunit.client.0.smithi107.stdout:0/496: read - d1/d27/d8a/d8b/fa4 zero size 2024-04-22T23:35:07.650 INFO:tasks.workunit.client.0.smithi107.stdout:0/497: rename d1/d27/d20/d2c/d2e/l31 to d1/d9/d14/d4f/d59/lae 0 2024-04-22T23:35:07.650 INFO:tasks.workunit.client.0.smithi107.stdout:0/498: fdatasync d1/d27/f57 0 2024-04-22T23:35:07.652 INFO:tasks.workunit.client.0.smithi107.stdout:5/551: dwrite d9/db/d42/f5a [4194304,4194304] 0 2024-04-22T23:35:07.652 INFO:tasks.workunit.client.0.smithi107.stdout:5/552: write d9/f36 [2054167,45477] 0 2024-04-22T23:35:07.660 INFO:tasks.workunit.client.1.smithi157.stdout:0/659: dread d17/d2b/f47 [0,4194304] 0 2024-04-22T23:35:07.661 INFO:tasks.workunit.client.1.smithi157.stdout:0/660: symlink d17/d2b/d4f/d5a/da1/da8/le0 0 2024-04-22T23:35:07.662 INFO:tasks.workunit.client.1.smithi157.stdout:0/661: unlink d17/d23/f7a 0 2024-04-22T23:35:07.662 INFO:tasks.workunit.client.1.smithi157.stdout:0/662: getdents d17/d2b/d52/ddc 0 2024-04-22T23:35:07.662 INFO:tasks.workunit.client.1.smithi157.stdout:3/739: dwrite d0/d3/d34/d3d/d5b/d72/f79 [0,4194304] 0 2024-04-22T23:35:07.663 INFO:tasks.workunit.client.1.smithi157.stdout:3/740: chown d0/d3/d34/d3d/d57/d60/l86 456648951 1 2024-04-22T23:35:07.663 INFO:tasks.workunit.client.1.smithi157.stdout:2/819: dwrite d7/d8/d30/d23/d27/d57/dcf/ffd [0,4194304] 0 2024-04-22T23:35:07.663 INFO:tasks.workunit.client.1.smithi157.stdout:2/820: read - d7/d8/d8e/ff6 zero size 2024-04-22T23:35:07.664 INFO:tasks.workunit.client.0.smithi107.stdout:6/537: dwrite db/d20/d2c/f91 [0,4194304] 0 2024-04-22T23:35:07.664 INFO:tasks.workunit.client.0.smithi107.stdout:6/538: chown ca 0 1 2024-04-22T23:35:07.664 INFO:tasks.workunit.client.0.smithi107.stdout:7/469: dwrite d7/d2d/f41 [0,4194304] 0 2024-04-22T23:35:07.664 INFO:tasks.workunit.client.0.smithi107.stdout:7/470: stat d7/d2d/d81 0 2024-04-22T23:35:07.664 INFO:tasks.workunit.client.0.smithi107.stdout:7/471: chown d7/l52 1612973 1 2024-04-22T23:35:07.665 INFO:tasks.workunit.client.1.smithi157.stdout:1/813: dwrite d4/db/f36 [0,4194304] 0 2024-04-22T23:35:07.665 INFO:tasks.workunit.client.1.smithi157.stdout:1/814: chown d4/d17/d34/d3b/d50/ff3 31983118 1 2024-04-22T23:35:07.665 INFO:tasks.workunit.client.1.smithi157.stdout:1/815: write d4/db/f14 [120711,23762] 0 2024-04-22T23:35:07.665 INFO:tasks.workunit.client.1.smithi157.stdout:1/816: creat d4/db/d3d/dc1/d100/f11d x:0 0 0 2024-04-22T23:35:07.665 INFO:tasks.workunit.client.1.smithi157.stdout:1/817: creat d4/d17/d1d/d33/d41/d54/db3/f11e x:0 0 0 2024-04-22T23:35:07.665 INFO:tasks.workunit.client.1.smithi157.stdout:0/663: link d17/d2b/d71/lb2 d17/d2b/d4f/d5a/le1 0 2024-04-22T23:35:07.666 INFO:tasks.workunit.client.1.smithi157.stdout:0/664: creat d17/d2b/d71/fe2 x:0 0 0 2024-04-22T23:35:07.666 INFO:tasks.workunit.client.1.smithi157.stdout:0/665: dread - d17/d23/fc9 zero size 2024-04-22T23:35:07.667 INFO:tasks.workunit.client.0.smithi107.stdout:6/539: dread db/d20/d2a/d38/f53 [0,4194304] 0 2024-04-22T23:35:07.667 INFO:tasks.workunit.client.0.smithi107.stdout:6/540: readlink db/d20/d2a/d35/d69/d79/l89 0 2024-04-22T23:35:07.667 INFO:tasks.workunit.client.0.smithi107.stdout:6/541: creat db/d20/d2c/d3b/d40/fa0 x:0 0 0 2024-04-22T23:35:07.669 INFO:tasks.workunit.client.1.smithi157.stdout:3/741: rename d0/d7/d95/lc6 to d0/d3/d34/d3d/d5b/dee/d9e/dac/lf0 0 2024-04-22T23:35:07.669 INFO:tasks.workunit.client.0.smithi107.stdout:6/542: rename db/d20/d2c/d3b/l77 to db/d20/d2a/d38/d61/d6f/d9b/la1 0 2024-04-22T23:35:07.670 INFO:tasks.workunit.client.0.smithi107.stdout:3/554: dwrite db/d21/d27/f20 [0,4194304] 0 2024-04-22T23:35:07.670 INFO:tasks.workunit.client.0.smithi107.stdout:3/555: write db/d21/d27/d15/d29/d2a/d45/f8a [821796,62884] 0 2024-04-22T23:35:07.670 INFO:tasks.workunit.client.0.smithi107.stdout:3/556: chown db/d21/d27/d15/d29/d4f/f77 10125375 1 2024-04-22T23:35:07.671 INFO:tasks.workunit.client.1.smithi157.stdout:5/789: dwrite d1/d5/dc/d43/d5e/fe6 [0,4194304] 0 2024-04-22T23:35:07.671 INFO:tasks.workunit.client.1.smithi157.stdout:2/821: dread d7/d8/d30/fa6 [0,4194304] 0 2024-04-22T23:35:07.672 INFO:tasks.workunit.client.1.smithi157.stdout:5/790: readlink d1/d5/d7/d2f/d45/lfa 0 2024-04-22T23:35:07.672 INFO:tasks.workunit.client.1.smithi157.stdout:5/791: readlink d1/d5/d7/d2f/d45/l7c 0 2024-04-22T23:35:07.672 INFO:tasks.workunit.client.1.smithi157.stdout:2/822: read d7/d1f/d26/d8f/fab [3467480,31463] 0 2024-04-22T23:35:07.672 INFO:tasks.workunit.client.0.smithi107.stdout:7/472: write d7/d1b/d51/f62 [1768729,5133] 0 2024-04-22T23:35:07.673 INFO:tasks.workunit.client.0.smithi107.stdout:3/557: mknod db/d21/d27/d36/cc0 0 2024-04-22T23:35:07.674 INFO:tasks.workunit.client.0.smithi107.stdout:0/499: dwrite d1/d27/d20/d2c/d2e/f71 [0,4194304] 0 2024-04-22T23:35:07.674 INFO:tasks.workunit.client.0.smithi107.stdout:0/500: chown d1/d27 39299 1 2024-04-22T23:35:07.674 INFO:tasks.workunit.client.0.smithi107.stdout:0/501: chown d1/d27/d20/d4a/l63 4994 1 2024-04-22T23:35:07.674 INFO:tasks.workunit.client.0.smithi107.stdout:7/473: write d7/f2f [1048044,35541] 0 2024-04-22T23:35:07.676 INFO:tasks.workunit.client.1.smithi157.stdout:1/818: unlink d4/d17/d34/f38 0 2024-04-22T23:35:07.677 INFO:tasks.workunit.client.0.smithi107.stdout:3/558: symlink db/d21/d27/d54/lc1 0 2024-04-22T23:35:07.677 INFO:tasks.workunit.client.1.smithi157.stdout:0/666: truncate d17/d2b/f86 2363589 0 2024-04-22T23:35:07.679 INFO:tasks.workunit.client.1.smithi157.stdout:3/742: link d0/d3/d34/d3d/d5b/d72/ceb d0/d3/d34/d3d/d5b/dee/cf1 0 2024-04-22T23:35:07.679 INFO:tasks.workunit.client.1.smithi157.stdout:5/792: rmdir d1/d1f/d22/d118 39 2024-04-22T23:35:07.679 INFO:tasks.workunit.client.1.smithi157.stdout:5/793: chown d1/c18 6 1 2024-04-22T23:35:07.680 INFO:tasks.workunit.client.0.smithi107.stdout:0/502: creat d1/d27/d20/d2c/d2e/faf x:0 0 0 2024-04-22T23:35:07.681 INFO:tasks.workunit.client.0.smithi107.stdout:0/503: stat d1/d9/d14/d4f/c96 0 2024-04-22T23:35:07.684 INFO:tasks.workunit.client.1.smithi157.stdout:1/819: unlink d4/d17/d1d/d33/f9a 0 2024-04-22T23:35:07.685 INFO:tasks.workunit.client.1.smithi157.stdout:0/667: mkdir d17/d23/d34/d67/d7f/d69/dce/de3 0 2024-04-22T23:35:07.686 INFO:tasks.workunit.client.0.smithi107.stdout:3/559: rename db/d21/d27/d15/d29/d2a/d99/fa8 to db/d21/d27/d91/fc2 0 2024-04-22T23:35:07.686 INFO:tasks.workunit.client.0.smithi107.stdout:6/543: dread db/f12 [0,4194304] 0 2024-04-22T23:35:07.687 INFO:tasks.workunit.client.1.smithi157.stdout:3/743: creat d0/d3/d34/d3d/d5b/d72/ff2 x:0 0 0 2024-04-22T23:35:07.687 INFO:tasks.workunit.client.1.smithi157.stdout:0/668: mkdir d17/d2b/d4f/d5a/dd2/de4 0 2024-04-22T23:35:07.687 INFO:tasks.workunit.client.0.smithi107.stdout:5/553: dwrite d9/f36 [0,4194304] 0 2024-04-22T23:35:07.688 INFO:tasks.workunit.client.1.smithi157.stdout:1/820: rename d4/d17/d6b/f8d to d4/d17/de5/f11f 0 2024-04-22T23:35:07.691 INFO:tasks.workunit.client.0.smithi107.stdout:0/504: truncate d1/d27/d20/d2c/d41/f58 796662 0 2024-04-22T23:35:07.691 INFO:tasks.workunit.client.0.smithi107.stdout:0/505: fsync d1/d2a/dad/fa3 0 2024-04-22T23:35:07.691 INFO:tasks.workunit.client.0.smithi107.stdout:0/506: read - d1/d2a/dad/fab zero size 2024-04-22T23:35:07.692 INFO:tasks.workunit.client.1.smithi157.stdout:1/821: rename d4/d5f/ld5 to d4/d17/l120 0 2024-04-22T23:35:07.692 INFO:tasks.workunit.client.1.smithi157.stdout:1/822: chown d4/d17/d1d/d33/d41/d54/db3/fbf 205207 1 2024-04-22T23:35:07.692 INFO:tasks.workunit.client.0.smithi107.stdout:5/554: mkdir d9/db/dc1 0 2024-04-22T23:35:07.695 INFO:tasks.workunit.client.0.smithi107.stdout:0/507: creat d1/d27/fb0 x:0 0 0 2024-04-22T23:35:07.696 INFO:tasks.workunit.client.0.smithi107.stdout:6/544: rename db/d20/d55 to db/d20/d2a/d38/d61/d6f/da2 0 2024-04-22T23:35:07.698 INFO:tasks.workunit.client.0.smithi107.stdout:0/508: mknod d1/d27/d20/d2c/cb1 0 2024-04-22T23:35:07.699 INFO:tasks.workunit.client.1.smithi157.stdout:8/522: sync 2024-04-22T23:35:07.699 INFO:tasks.workunit.client.1.smithi157.stdout:4/722: sync 2024-04-22T23:35:07.699 INFO:tasks.workunit.client.1.smithi157.stdout:7/630: sync 2024-04-22T23:35:07.699 INFO:tasks.workunit.client.1.smithi157.stdout:3/744: write d0/d3/d34/fe4 [2416299,103081] 0 2024-04-22T23:35:07.699 INFO:tasks.workunit.client.1.smithi157.stdout:7/631: chown d0/d9/d29/d5e/d9e/fb3 27328878 1 2024-04-22T23:35:07.699 INFO:tasks.workunit.client.1.smithi157.stdout:9/724: sync 2024-04-22T23:35:07.700 INFO:tasks.workunit.client.0.smithi107.stdout:6/545: symlink db/la3 0 2024-04-22T23:35:07.700 INFO:tasks.workunit.client.0.smithi107.stdout:6/546: fsync db/f12 0 2024-04-22T23:35:07.701 INFO:tasks.workunit.client.1.smithi157.stdout:4/723: mknod d3/d96/dee/cf1 0 2024-04-22T23:35:07.701 INFO:tasks.workunit.client.0.smithi107.stdout:0/509: rename d1/d27/d20/l33 to d1/lb2 0 2024-04-22T23:35:07.702 INFO:tasks.workunit.client.1.smithi157.stdout:7/632: truncate d0/d9/d42/f7a 5450950 0 2024-04-22T23:35:07.702 INFO:tasks.workunit.client.1.smithi157.stdout:9/725: symlink d4/df/d57/le1 0 2024-04-22T23:35:07.703 INFO:tasks.workunit.client.1.smithi157.stdout:3/745: getdents d0/d83 0 2024-04-22T23:35:07.704 INFO:tasks.workunit.client.1.smithi157.stdout:3/746: dread d0/d3/d34/f48 [0,4194304] 0 2024-04-22T23:35:07.704 INFO:tasks.workunit.client.1.smithi157.stdout:4/724: rename d3/d11/d1d/d73/d80/dac/db5/f6d to d3/d11/d34/da6/ff2 0 2024-04-22T23:35:07.705 INFO:tasks.workunit.client.1.smithi157.stdout:2/823: dwrite d7/d8/d25/f3a [0,4194304] 0 2024-04-22T23:35:07.705 INFO:tasks.workunit.client.1.smithi157.stdout:2/824: chown d7/d8/d25/d102/f103 730 1 2024-04-22T23:35:07.705 INFO:tasks.workunit.client.1.smithi157.stdout:5/794: dwrite d1/d1f/ff1 [4194304,4194304] 0 2024-04-22T23:35:07.705 INFO:tasks.workunit.client.1.smithi157.stdout:5/795: stat d1/d5/d7/d10/f31 0 2024-04-22T23:35:07.705 INFO:tasks.workunit.client.1.smithi157.stdout:5/796: write d1/d5/dc/f38 [940897,39255] 0 2024-04-22T23:35:07.705 INFO:tasks.workunit.client.0.smithi107.stdout:0/510: unlink d1/d27/d20/d2c/d41/f90 0 2024-04-22T23:35:07.707 INFO:tasks.workunit.client.1.smithi157.stdout:3/747: symlink d0/d3/d34/d3d/d5b/dee/d9e/dac/lf3 0 2024-04-22T23:35:07.707 INFO:tasks.workunit.client.1.smithi157.stdout:3/748: write d0/d3/d34/d3d/f69 [553626,38583] 0 2024-04-22T23:35:07.707 INFO:tasks.workunit.client.1.smithi157.stdout:3/749: truncate d0/d7/fe0 708316 0 2024-04-22T23:35:07.707 INFO:tasks.workunit.client.1.smithi157.stdout:2/825: creat d7/d105/f112 x:0 0 0 2024-04-22T23:35:07.709 INFO:tasks.workunit.client.1.smithi157.stdout:2/826: write d7/d1f/d26/fb0 [53243,59691] 0 2024-04-22T23:35:07.713 INFO:tasks.workunit.client.1.smithi157.stdout:9/726: rename d4/d15/d23/d27/d47/db3/ddd to d4/d15/d23/d27/db5/de2 0 2024-04-22T23:35:07.715 INFO:tasks.workunit.client.0.smithi107.stdout:7/474: dwrite d7/d21/f22 [0,4194304] 0 2024-04-22T23:35:07.715 INFO:tasks.workunit.client.0.smithi107.stdout:7/475: dread - d7/d21/d48/d5c/f75 zero size 2024-04-22T23:35:07.716 INFO:tasks.workunit.client.1.smithi157.stdout:5/797: dread d1/d5/dc/f1b [0,4194304] 0 2024-04-22T23:35:07.716 INFO:tasks.workunit.client.1.smithi157.stdout:5/798: creat d1/d5/d7/d2f/d45/d54/f12f x:0 0 0 2024-04-22T23:35:07.716 INFO:tasks.workunit.client.1.smithi157.stdout:7/633: write d0/d9/f66 [1714633,48367] 0 2024-04-22T23:35:07.717 INFO:tasks.workunit.client.1.smithi157.stdout:2/827: rename d7/d8/dd/f71 to d7/d8/d30/d23/d27/d57/dcf/df2/f113 0 2024-04-22T23:35:07.717 INFO:tasks.workunit.client.1.smithi157.stdout:2/828: write d7/d8/f101 [666314,107383] 0 2024-04-22T23:35:07.718 INFO:tasks.workunit.client.1.smithi157.stdout:5/799: dread d1/d5/d7/dbd/ff4 [0,4194304] 0 2024-04-22T23:35:07.720 INFO:tasks.workunit.client.0.smithi107.stdout:7/476: creat d7/d21/d48/f9c x:0 0 0 2024-04-22T23:35:07.722 INFO:tasks.workunit.client.1.smithi157.stdout:7/634: dread d0/d9/f1d [4194304,4194304] 0 2024-04-22T23:35:07.722 INFO:tasks.workunit.client.1.smithi157.stdout:7/635: chown d0/d9/d72 0 1 2024-04-22T23:35:07.722 INFO:tasks.workunit.client.1.smithi157.stdout:2/829: mkdir d7/d1f/d26/d114 0 2024-04-22T23:35:07.723 INFO:tasks.workunit.client.0.smithi107.stdout:7/477: getdents d7/d1b/d51/d6e 0 2024-04-22T23:35:07.723 INFO:tasks.workunit.client.1.smithi157.stdout:5/800: symlink d1/db0/dff/d11b/l130 0 2024-04-22T23:35:07.727 INFO:tasks.workunit.client.1.smithi157.stdout:5/801: symlink d1/d5/dc/d43/d4f/l131 0 2024-04-22T23:35:07.727 INFO:tasks.workunit.client.1.smithi157.stdout:5/802: truncate d1/d5/d7/d2f/d45/d54/f12f 847736 0 2024-04-22T23:35:07.730 INFO:tasks.workunit.client.1.smithi157.stdout:9/727: dread d4/d15/d23/d27/f4f [4194304,4194304] 0 2024-04-22T23:35:07.730 INFO:tasks.workunit.client.0.smithi107.stdout:7/478: rename d7/d1b/d51 to d7/d21/d48/d5c/d9d 0 2024-04-22T23:35:07.732 INFO:tasks.workunit.client.1.smithi157.stdout:9/728: mknod d4/d15/d23/d27/d47/ce3 0 2024-04-22T23:35:07.732 INFO:tasks.workunit.client.1.smithi157.stdout:9/729: creat d4/df/d57/d6b/dc4/fe4 x:0 0 0 2024-04-22T23:35:07.735 INFO:tasks.workunit.client.0.smithi107.stdout:7/479: read d7/f3b [8119263,110513] 0 2024-04-22T23:35:07.736 INFO:tasks.workunit.client.1.smithi157.stdout:9/730: unlink d4/df/d61/d9c/fb2 0 2024-04-22T23:35:07.736 INFO:tasks.workunit.client.1.smithi157.stdout:9/731: fsync d4/d15/d55/f9b 0 2024-04-22T23:35:07.736 INFO:tasks.workunit.client.1.smithi157.stdout:9/732: fsync d4/d15/d23/f88 0 2024-04-22T23:35:07.736 INFO:tasks.workunit.client.1.smithi157.stdout:9/733: write d4/d15/d55/f76 [1009967,72078] 0 2024-04-22T23:35:07.736 INFO:tasks.workunit.client.0.smithi107.stdout:6/547: dwrite db/d20/d2c/d3b/d40/f71 [0,4194304] 0 2024-04-22T23:35:07.737 INFO:tasks.workunit.client.0.smithi107.stdout:7/480: link d7/l4f d7/d21/d48/l9e 0 2024-04-22T23:35:07.739 INFO:tasks.workunit.client.1.smithi157.stdout:5/803: dread d1/d5/d7/d10/f31 [0,4194304] 0 2024-04-22T23:35:07.740 INFO:tasks.workunit.client.1.smithi157.stdout:2/830: read d7/d8/d25/f50 [149499,75938] 0 2024-04-22T23:35:07.740 INFO:tasks.workunit.client.1.smithi157.stdout:2/831: write d7/d8/dca/f10e [4433707,123464] 0 2024-04-22T23:35:07.740 INFO:tasks.workunit.client.0.smithi107.stdout:6/548: symlink db/d20/d49/la4 0 2024-04-22T23:35:07.740 INFO:tasks.workunit.client.1.smithi157.stdout:5/804: readlink d1/d27/d29/le1 0 2024-04-22T23:35:07.741 INFO:tasks.workunit.client.0.smithi107.stdout:7/481: symlink d7/d21/d48/l9f 0 2024-04-22T23:35:07.741 INFO:tasks.workunit.client.1.smithi157.stdout:4/725: dwrite d3/d5c/d7b/fa8 [0,4194304] 0 2024-04-22T23:35:07.741 INFO:tasks.workunit.client.1.smithi157.stdout:2/832: mknod d7/d8/d25/c115 0 2024-04-22T23:35:07.742 INFO:tasks.workunit.client.0.smithi107.stdout:6/549: mknod db/d20/d2a/d38/d59/d70/ca5 0 2024-04-22T23:35:07.743 INFO:tasks.workunit.client.0.smithi107.stdout:1/487: sync 2024-04-22T23:35:07.743 INFO:tasks.workunit.client.0.smithi107.stdout:1/488: chown d7/d19/d20/d39/d53/f4e 4900 1 2024-04-22T23:35:07.743 INFO:tasks.workunit.client.1.smithi157.stdout:8/523: dwrite d2/d6/d21/d5f/d11/f4a [4194304,4194304] 0 2024-04-22T23:35:07.743 INFO:tasks.workunit.client.1.smithi157.stdout:5/805: truncate d1/d5/d7/f67 2223840 0 2024-04-22T23:35:07.744 INFO:tasks.workunit.client.1.smithi157.stdout:3/750: dwrite d0/d3/f35 [0,4194304] 0 2024-04-22T23:35:07.744 INFO:tasks.workunit.client.0.smithi107.stdout:7/482: truncate d7/d21/d48/f55 878434 0 2024-04-22T23:35:07.744 INFO:tasks.workunit.client.0.smithi107.stdout:7/483: fdatasync d7/d21/d48/d5c/f75 0 2024-04-22T23:35:07.745 INFO:tasks.workunit.client.1.smithi157.stdout:4/726: symlink d3/d11/d1d/d73/lf3 0 2024-04-22T23:35:07.747 INFO:tasks.workunit.client.1.smithi157.stdout:2/833: mkdir d7/d8/dd/d4d/d116 0 2024-04-22T23:35:07.747 INFO:tasks.workunit.client.1.smithi157.stdout:2/834: write d7/d1f/d26/fe1 [136300,118723] 0 2024-04-22T23:35:07.748 INFO:tasks.workunit.client.0.smithi107.stdout:1/489: rename d7/d19/l4a to d7/d19/d20/d39/d53/d43/l9c 0 2024-04-22T23:35:07.749 INFO:tasks.workunit.client.1.smithi157.stdout:8/524: mknod d2/d6/d66/d67/cb9 0 2024-04-22T23:35:07.754 INFO:tasks.workunit.client.0.smithi107.stdout:1/490: symlink d7/d19/d2d/d89/l9d 0 2024-04-22T23:35:07.756 INFO:tasks.workunit.client.0.smithi107.stdout:6/550: dread db/f3f [0,4194304] 0 2024-04-22T23:35:07.757 INFO:tasks.workunit.client.0.smithi107.stdout:1/491: creat d7/d19/d20/d39/d93/f9e x:0 0 0 2024-04-22T23:35:07.758 INFO:tasks.workunit.client.1.smithi157.stdout:4/727: unlink l2 0 2024-04-22T23:35:07.761 INFO:tasks.workunit.client.0.smithi107.stdout:0/511: dwrite d1/d2a/dad/fab [0,4194304] 0 2024-04-22T23:35:07.761 INFO:tasks.workunit.client.0.smithi107.stdout:0/512: chown d1/d9/d14/d4f/c96 3647437 1 2024-04-22T23:35:07.761 INFO:tasks.workunit.client.0.smithi107.stdout:0/513: chown d1/d9/d14/d4f/fa2 51910022 1 2024-04-22T23:35:07.762 INFO:tasks.workunit.client.1.smithi157.stdout:4/728: mknod d3/d11/d30/cf4 0 2024-04-22T23:35:07.764 INFO:tasks.workunit.client.1.smithi157.stdout:7/636: dwrite d0/d9/db/f57 [0,4194304] 0 2024-04-22T23:35:07.764 INFO:tasks.workunit.client.1.smithi157.stdout:7/637: chown d0/d9/db/df/d60/l7b 3 1 2024-04-22T23:35:07.764 INFO:tasks.workunit.client.1.smithi157.stdout:7/638: fdatasync d0/d9/d29/d5e/f65 0 2024-04-22T23:35:07.773 INFO:tasks.workunit.client.1.smithi157.stdout:2/835: dwrite d7/d8/d30/db8/ffe [0,4194304] 0 2024-04-22T23:35:07.773 INFO:tasks.workunit.client.1.smithi157.stdout:4/729: mkdir d3/d11/d1d/d73/da3/d31/df5 0 2024-04-22T23:35:07.773 INFO:tasks.workunit.client.1.smithi157.stdout:4/730: fdatasync d3/d11/d1d/d73/d80/f8c 0 2024-04-22T23:35:07.773 INFO:tasks.workunit.client.1.smithi157.stdout:4/731: read d3/d4b/f16 [2732216,72016] 0 2024-04-22T23:35:07.775 INFO:tasks.workunit.client.1.smithi157.stdout:7/639: link d0/d9/db/df/d68/f8e d0/d9/d36/dbb/fbe 0 2024-04-22T23:35:07.775 INFO:tasks.workunit.client.0.smithi107.stdout:7/484: dwrite d7/d21/d48/f61 [8388608,4194304] 0 2024-04-22T23:35:07.775 INFO:tasks.workunit.client.0.smithi107.stdout:6/551: read db/d20/d2c/d3b/f45 [2088292,126711] 0 2024-04-22T23:35:07.776 INFO:tasks.workunit.client.1.smithi157.stdout:9/734: write d4/f29 [176499,99946] 0 2024-04-22T23:35:07.776 INFO:tasks.workunit.client.1.smithi157.stdout:5/806: dwrite d1/d27/d29/f2a [0,4194304] 0 2024-04-22T23:35:07.776 INFO:tasks.workunit.client.1.smithi157.stdout:9/735: write d4/df/d52/fac [2010319,2701] 0 2024-04-22T23:35:07.776 INFO:tasks.workunit.client.1.smithi157.stdout:9/736: readlink d4/df/l17 0 2024-04-22T23:35:07.777 INFO:tasks.workunit.client.0.smithi107.stdout:6/552: link db/d20/l36 db/d20/d49/la6 0 2024-04-22T23:35:07.778 INFO:tasks.workunit.client.1.smithi157.stdout:8/525: rmdir d2/d6/d66 39 2024-04-22T23:35:07.778 INFO:tasks.workunit.client.1.smithi157.stdout:8/526: write d2/fa9 [1032543,130523] 0 2024-04-22T23:35:07.778 INFO:tasks.workunit.client.1.smithi157.stdout:8/527: fdatasync d2/d4/f52 0 2024-04-22T23:35:07.778 INFO:tasks.workunit.client.1.smithi157.stdout:8/528: write d2/d6/d21/d5f/d11/f5a [371865,89876] 0 2024-04-22T23:35:07.782 INFO:tasks.workunit.client.0.smithi107.stdout:6/553: creat db/d20/d2c/d3b/fa7 x:0 0 0 2024-04-22T23:35:07.782 INFO:tasks.workunit.client.0.smithi107.stdout:6/554: creat db/d20/d2c/d3b/fa8 x:0 0 0 2024-04-22T23:35:07.784 INFO:tasks.workunit.client.1.smithi157.stdout:2/836: mknod d7/d8/d8e/c117 0 2024-04-22T23:35:07.788 INFO:tasks.workunit.client.1.smithi157.stdout:7/640: rename d0/c25 to d0/d9/d42/d55/d56/cbf 0 2024-04-22T23:35:07.790 INFO:tasks.workunit.client.1.smithi157.stdout:9/737: symlink d4/d15/d23/d27/d47/le5 0 2024-04-22T23:35:07.792 INFO:tasks.workunit.client.1.smithi157.stdout:2/837: unlink d7/d1f/d26/f74 0 2024-04-22T23:35:07.792 INFO:tasks.workunit.client.0.smithi107.stdout:0/514: dwrite d1/d9/d14/d4f/f82 [0,4194304] 0 2024-04-22T23:35:07.793 INFO:tasks.workunit.client.0.smithi107.stdout:0/515: write d1/d27/d20/f6b [2664650,17227] 0 2024-04-22T23:35:07.797 INFO:tasks.workunit.client.1.smithi157.stdout:4/732: write d3/d11/d1d/f52 [2382387,84649] 0 2024-04-22T23:35:07.797 INFO:tasks.workunit.client.1.smithi157.stdout:4/733: creat d3/d11/d34/d9d/ff6 x:0 0 0 2024-04-22T23:35:07.798 INFO:tasks.workunit.client.0.smithi107.stdout:0/516: unlink d1/d84/d99/l9e 0 2024-04-22T23:35:07.798 INFO:tasks.workunit.client.1.smithi157.stdout:5/807: getdents d1/d27/d29 0 2024-04-22T23:35:07.798 INFO:tasks.workunit.client.1.smithi157.stdout:5/808: chown d1/d5/dc/d43/d5e/d111 461705885 1 2024-04-22T23:35:07.798 INFO:tasks.workunit.client.0.smithi107.stdout:1/492: dwrite d7/d19/d42/f6d [0,4194304] 0 2024-04-22T23:35:07.799 INFO:tasks.workunit.client.1.smithi157.stdout:9/738: symlink d4/df/d57/db7/le6 0 2024-04-22T23:35:07.799 INFO:tasks.workunit.client.1.smithi157.stdout:9/739: read - d4/df/d57/fd9 zero size 2024-04-22T23:35:07.799 INFO:tasks.workunit.client.1.smithi157.stdout:9/740: chown d4/d7e 0 1 2024-04-22T23:35:07.799 INFO:tasks.workunit.client.1.smithi157.stdout:9/741: dread d4/d15/d23/d27/fb1 [0,4194304] 0 2024-04-22T23:35:07.800 INFO:tasks.workunit.client.1.smithi157.stdout:9/742: fsync d4/d15/d23/d27/fb1 0 2024-04-22T23:35:07.800 INFO:tasks.workunit.client.0.smithi107.stdout:7/485: dwrite d7/d21/f96 [0,4194304] 0 2024-04-22T23:35:07.800 INFO:tasks.workunit.client.0.smithi107.stdout:7/486: creat d7/d2d/d66/fa0 x:0 0 0 2024-04-22T23:35:07.800 INFO:tasks.workunit.client.0.smithi107.stdout:7/487: chown d7/d21/d48/l9e 7 1 2024-04-22T23:35:07.800 INFO:tasks.workunit.client.1.smithi157.stdout:8/529: write d2/d6/f37 [1559108,22507] 0 2024-04-22T23:35:07.802 INFO:tasks.workunit.client.1.smithi157.stdout:2/838: dread d7/d8/dd/d4d/f99 [0,4194304] 0 2024-04-22T23:35:07.804 INFO:tasks.workunit.client.0.smithi107.stdout:1/493: mknod d7/d19/d42/c9f 0 2024-04-22T23:35:07.808 INFO:tasks.workunit.client.0.smithi107.stdout:6/555: dwrite db/f21 [0,4194304] 0 2024-04-22T23:35:07.808 INFO:tasks.workunit.client.0.smithi107.stdout:6/556: truncate db/d20/d2c/d3b/fa8 845143 0 2024-04-22T23:35:07.808 INFO:tasks.workunit.client.0.smithi107.stdout:6/557: fdatasync db/d20/d2a/d38/d4b/f54 0 2024-04-22T23:35:07.808 INFO:tasks.workunit.client.0.smithi107.stdout:6/558: fsync db/f5e 0 2024-04-22T23:35:07.808 INFO:tasks.workunit.client.1.smithi157.stdout:9/743: mknod d4/d7e/ce7 0 2024-04-22T23:35:07.810 INFO:tasks.workunit.client.0.smithi107.stdout:7/488: dread d7/f11 [0,4194304] 0 2024-04-22T23:35:07.812 INFO:tasks.workunit.client.0.smithi107.stdout:6/559: creat db/d20/d2a/d5d/fa9 x:0 0 0 2024-04-22T23:35:07.812 INFO:tasks.workunit.client.0.smithi107.stdout:6/560: mknod db/d20/d2a/d5d/caa 0 2024-04-22T23:35:07.814 INFO:tasks.workunit.client.1.smithi157.stdout:4/734: read d3/d4b/f4f [1208930,124935] 0 2024-04-22T23:35:07.814 INFO:tasks.workunit.client.0.smithi107.stdout:6/561: truncate db/d20/d2a/d38/f3c 2422604 0 2024-04-22T23:35:07.816 INFO:tasks.workunit.client.0.smithi107.stdout:1/494: write d7/d19/f2c [4026767,34841] 0 2024-04-22T23:35:07.820 INFO:tasks.workunit.client.1.smithi157.stdout:2/839: stat d7/d8/dd/d4d/lb6 0 2024-04-22T23:35:07.821 INFO:tasks.workunit.client.1.smithi157.stdout:4/735: mkdir d3/d11/d1d/d73/d80/dac/db5/df7 0 2024-04-22T23:35:07.821 INFO:tasks.workunit.client.1.smithi157.stdout:4/736: dread - d3/d11/fde zero size 2024-04-22T23:35:07.823 INFO:tasks.workunit.client.1.smithi157.stdout:2/840: truncate d7/d8/d25/d102/fe5 5101554 0 2024-04-22T23:35:07.824 INFO:tasks.workunit.client.1.smithi157.stdout:7/641: dwrite d0/d9/d29/d33/d37/f87 [0,4194304] 0 2024-04-22T23:35:07.825 INFO:tasks.workunit.client.1.smithi157.stdout:4/737: symlink d3/d11/d1d/d73/d80/dac/db5/dc8/lf8 0 2024-04-22T23:35:07.826 INFO:tasks.workunit.client.1.smithi157.stdout:8/530: dread d2/d6/d21/d5f/d17/f4f [0,4194304] 0 2024-04-22T23:35:07.826 INFO:tasks.workunit.client.1.smithi157.stdout:8/531: truncate d2/d34/d6a/d77/f82 78429 0 2024-04-22T23:35:07.826 INFO:tasks.workunit.client.1.smithi157.stdout:8/532: stat d2/f5 0 2024-04-22T23:35:07.827 INFO:tasks.workunit.client.1.smithi157.stdout:7/642: truncate d0/f2 752697 0 2024-04-22T23:35:07.828 INFO:tasks.workunit.client.1.smithi157.stdout:4/738: creat d3/d11/d1d/d73/da7/ff9 x:0 0 0 2024-04-22T23:35:07.829 INFO:tasks.workunit.client.1.smithi157.stdout:4/739: symlink d3/d11/d1d/d73/da3/d31/d91/lfa 0 2024-04-22T23:35:07.830 INFO:tasks.workunit.client.1.smithi157.stdout:4/740: unlink d3/d4b/l5b 0 2024-04-22T23:35:07.837 INFO:tasks.workunit.client.1.smithi157.stdout:9/744: dwrite d4/df/f12 [0,4194304] 0 2024-04-22T23:35:07.837 INFO:tasks.workunit.client.1.smithi157.stdout:5/809: dwrite d1/db0/d7a/f11e [0,4194304] 0 2024-04-22T23:35:07.838 INFO:tasks.workunit.client.0.smithi107.stdout:0/517: dwrite d1/d27/d20/d2c/d2e/f39 [0,4194304] 0 2024-04-22T23:35:07.838 INFO:tasks.workunit.client.1.smithi157.stdout:9/745: creat d4/df/d57/d6b/fe8 x:0 0 0 2024-04-22T23:35:07.838 INFO:tasks.workunit.client.1.smithi157.stdout:9/746: chown d4/d15/d23 217366861 1 2024-04-22T23:35:07.840 INFO:tasks.workunit.client.0.smithi107.stdout:6/562: read db/f11 [102236,66651] 0 2024-04-22T23:35:07.840 INFO:tasks.workunit.client.0.smithi107.stdout:6/563: chown db/d20/d49/d7b/c80 92210878 1 2024-04-22T23:35:07.841 INFO:tasks.workunit.client.1.smithi157.stdout:5/810: truncate d1/d27/d29/d4b/f4d 1902608 0 2024-04-22T23:35:07.841 INFO:tasks.workunit.client.1.smithi157.stdout:7/643: write d0/d9/d29/d33/d37/f87 [2327269,100787] 0 2024-04-22T23:35:07.841 INFO:tasks.workunit.client.1.smithi157.stdout:7/644: dread - d0/d9/de/fb9 zero size 2024-04-22T23:35:07.842 INFO:tasks.workunit.client.0.smithi107.stdout:7/489: dwrite d7/d1b/f4d [4194304,4194304] 0 2024-04-22T23:35:07.842 INFO:tasks.workunit.client.0.smithi107.stdout:7/490: truncate d7/d1b/f38 1172017 0 2024-04-22T23:35:07.842 INFO:tasks.workunit.client.0.smithi107.stdout:7/491: dread - d7/d2d/d81/f8f zero size 2024-04-22T23:35:07.843 INFO:tasks.workunit.client.1.smithi157.stdout:5/811: write d1/d5/fd0 [286901,89473] 0 2024-04-22T23:35:07.843 INFO:tasks.workunit.client.0.smithi107.stdout:6/564: getdents db/d20/d2a/d38/d4b 0 2024-04-22T23:35:07.845 INFO:tasks.workunit.client.1.smithi157.stdout:7/645: creat d0/d9/d72/fc0 x:0 0 0 2024-04-22T23:35:07.845 INFO:tasks.workunit.client.1.smithi157.stdout:7/646: chown d0/d9/db/l90 21 1 2024-04-22T23:35:07.846 INFO:tasks.workunit.client.0.smithi107.stdout:7/492: rmdir d7/d21/d48/d5c/d92 0 2024-04-22T23:35:07.851 INFO:tasks.workunit.client.1.smithi157.stdout:2/841: rmdir d7/d8/d95/dea 39 2024-04-22T23:35:07.851 INFO:tasks.workunit.client.1.smithi157.stdout:2/842: write d7/d1f/d26/f44 [2432832,127736] 0 2024-04-22T23:35:07.852 INFO:tasks.workunit.client.1.smithi157.stdout:2/843: getdents d7/d8/d25/d102 0 2024-04-22T23:35:07.862 INFO:tasks.workunit.client.1.smithi157.stdout:7/647: dread d0/f4 [0,4194304] 0 2024-04-22T23:35:07.862 INFO:tasks.workunit.client.1.smithi157.stdout:2/844: read d7/d8/d25/f3a [722170,73857] 0 2024-04-22T23:35:07.862 INFO:tasks.workunit.client.1.smithi157.stdout:7/648: readlink d0/d9/d29/d49/d4a/l53 0 2024-04-22T23:35:07.864 INFO:tasks.workunit.client.1.smithi157.stdout:2/845: creat d7/d8/df7/f118 x:0 0 0 2024-04-22T23:35:07.864 INFO:tasks.workunit.client.1.smithi157.stdout:7/649: link d0/d9/db/d35/la5 d0/d9/d29/d5e/lc1 0 2024-04-22T23:35:07.865 INFO:tasks.workunit.client.1.smithi157.stdout:7/650: chown d0/d9/d29/d49/d64/c70 1374335 1 2024-04-22T23:35:07.865 INFO:tasks.workunit.client.1.smithi157.stdout:7/651: stat d0/d9/d29/d49/d4a/l6f 0 2024-04-22T23:35:07.865 INFO:tasks.workunit.client.1.smithi157.stdout:7/652: creat d0/d9/d42/d55/d56/d91/fc2 x:0 0 0 2024-04-22T23:35:07.866 INFO:tasks.workunit.client.1.smithi157.stdout:8/533: dwrite d2/d34/d6a/d77/f82 [0,4194304] 0 2024-04-22T23:35:07.866 INFO:tasks.workunit.client.1.smithi157.stdout:2/846: mkdir d7/d8/d95/dea/d119 0 2024-04-22T23:35:07.871 INFO:tasks.workunit.client.1.smithi157.stdout:8/534: rename d2/d34/fae to d2/d6/fba 0 2024-04-22T23:35:07.877 INFO:tasks.workunit.client.1.smithi157.stdout:7/653: dread d0/d9/db/f61 [0,4194304] 0 2024-04-22T23:35:07.880 INFO:tasks.workunit.client.0.smithi107.stdout:0/518: dwrite d1/d27/f21 [0,4194304] 0 2024-04-22T23:35:07.880 INFO:tasks.workunit.client.0.smithi107.stdout:0/519: chown d1/d27/d20/f6b 62405 1 2024-04-22T23:35:07.880 INFO:tasks.workunit.client.0.smithi107.stdout:6/565: dwrite db/d20/d2a/d38/f3e [0,4194304] 0 2024-04-22T23:35:07.880 INFO:tasks.workunit.client.0.smithi107.stdout:6/566: read - db/d20/d2a/d38/d59/f93 zero size 2024-04-22T23:35:07.881 INFO:tasks.workunit.client.0.smithi107.stdout:7/493: dwrite d7/d21/f87 [0,4194304] 0 2024-04-22T23:35:07.881 INFO:tasks.workunit.client.1.smithi157.stdout:7/654: unlink d0/l5 0 2024-04-22T23:35:07.881 INFO:tasks.workunit.client.1.smithi157.stdout:2/847: write d7/d8/d95/dd2/dda/f89 [725526,124437] 0 2024-04-22T23:35:07.881 INFO:tasks.workunit.client.1.smithi157.stdout:4/741: dwrite d3/d11/d1d/f24 [4194304,4194304] 0 2024-04-22T23:35:07.881 INFO:tasks.workunit.client.1.smithi157.stdout:2/848: readlink d7/d8/d30/d23/l29 0 2024-04-22T23:35:07.881 INFO:tasks.workunit.client.1.smithi157.stdout:2/849: chown d7/d1f/d26/lbc 0 1 2024-04-22T23:35:07.884 INFO:tasks.workunit.client.0.smithi107.stdout:6/567: rename db/c33 to db/d20/d2a/d5f/cab 0 2024-04-22T23:35:07.885 INFO:tasks.workunit.client.1.smithi157.stdout:2/850: mkdir d7/d9f/d11a 0 2024-04-22T23:35:07.886 INFO:tasks.workunit.client.1.smithi157.stdout:2/851: creat d7/d1f/d26/d4a/f11b x:0 0 0 2024-04-22T23:35:07.891 INFO:tasks.workunit.client.0.smithi107.stdout:6/568: creat db/d20/d2a/d38/d61/d6f/da2/fac x:0 0 0 2024-04-22T23:35:07.891 INFO:tasks.workunit.client.0.smithi107.stdout:7/494: mkdir d7/d4c/da1 0 2024-04-22T23:35:07.891 INFO:tasks.workunit.client.0.smithi107.stdout:7/495: write d7/f76 [1042083,8497] 0 2024-04-22T23:35:07.892 INFO:tasks.workunit.client.0.smithi107.stdout:7/496: fdatasync d7/d1b/f24 0 2024-04-22T23:35:07.896 INFO:tasks.workunit.client.0.smithi107.stdout:0/520: dread d1/d27/d20/d40/fa0 [0,4194304] 0 2024-04-22T23:35:07.906 INFO:tasks.workunit.client.0.smithi107.stdout:6/569: dread db/d20/d2c/d3b/f60 [0,4194304] 0 2024-04-22T23:35:07.907 INFO:tasks.workunit.client.0.smithi107.stdout:6/570: rename db/d20/c22 to db/d20/d2a/d38/d4b/d9d/cad 0 2024-04-22T23:35:07.908 INFO:tasks.workunit.client.0.smithi107.stdout:6/571: write db/d20/d2a/f42 [3295874,2445] 0 2024-04-22T23:35:07.908 INFO:tasks.workunit.client.0.smithi107.stdout:7/497: dread d7/d21/f87 [0,4194304] 0 2024-04-22T23:35:07.908 INFO:tasks.workunit.client.0.smithi107.stdout:7/498: creat d7/d1b/fa2 x:0 0 0 2024-04-22T23:35:07.917 INFO:tasks.workunit.client.0.smithi107.stdout:6/572: write f6 [5725290,77628] 0 2024-04-22T23:35:07.920 INFO:tasks.workunit.client.0.smithi107.stdout:6/573: unlink db/d20/d2c/f91 0 2024-04-22T23:35:07.920 INFO:tasks.workunit.client.0.smithi107.stdout:6/574: write db/d20/d2c/d3b/d40/f8b [6624,21200] 0 2024-04-22T23:35:07.931 INFO:tasks.workunit.client.1.smithi157.stdout:4/742: dwrite d3/d11/d34/f51 [0,4194304] 0 2024-04-22T23:35:07.931 INFO:tasks.workunit.client.1.smithi157.stdout:7/655: dwrite d0/d9/d36/dbb/fbe [0,4194304] 0 2024-04-22T23:35:07.931 INFO:tasks.workunit.client.1.smithi157.stdout:2/852: dwrite d7/d8/dd/f2b [0,4194304] 0 2024-04-22T23:35:07.931 INFO:tasks.workunit.client.1.smithi157.stdout:2/853: write d7/d1f/d26/f44 [402396,119511] 0 2024-04-22T23:35:07.932 INFO:tasks.workunit.client.0.smithi107.stdout:7/499: dwrite d7/d2d/d66/f91 [0,4194304] 0 2024-04-22T23:35:07.932 INFO:tasks.workunit.client.0.smithi107.stdout:7/500: chown d7/d21/l9a 30 1 2024-04-22T23:35:07.932 INFO:tasks.workunit.client.0.smithi107.stdout:0/521: dwrite d1/d27/d20/f36 [0,4194304] 0 2024-04-22T23:35:07.932 INFO:tasks.workunit.client.0.smithi107.stdout:7/501: write d7/d1b/f68 [545413,46460] 0 2024-04-22T23:35:07.933 INFO:tasks.workunit.client.0.smithi107.stdout:0/522: chown d1/d2a/dad/l97 1730638 1 2024-04-22T23:35:07.933 INFO:tasks.workunit.client.0.smithi107.stdout:7/502: dread - d7/d2d/d81/f97 zero size 2024-04-22T23:35:07.933 INFO:tasks.workunit.client.0.smithi107.stdout:0/523: creat d1/d27/d20/d2c/d2e/fb3 x:0 0 0 2024-04-22T23:35:07.933 INFO:tasks.workunit.client.0.smithi107.stdout:0/524: write d1/d27/d20/d2c/d2e/fb3 [399005,110072] 0 2024-04-22T23:35:07.935 INFO:tasks.workunit.client.0.smithi107.stdout:2/586: sync 2024-04-22T23:35:07.935 INFO:tasks.workunit.client.0.smithi107.stdout:2/587: read - d1/d63/fb6 zero size 2024-04-22T23:35:07.935 INFO:tasks.workunit.client.0.smithi107.stdout:2/588: creat d1/d2b/d3d/fca x:0 0 0 2024-04-22T23:35:07.935 INFO:tasks.workunit.client.0.smithi107.stdout:2/589: read - d1/d2b/d3d/f5e zero size 2024-04-22T23:35:07.935 INFO:tasks.workunit.client.0.smithi107.stdout:4/583: sync 2024-04-22T23:35:07.935 INFO:tasks.workunit.client.0.smithi107.stdout:9/577: sync 2024-04-22T23:35:07.936 INFO:tasks.workunit.client.1.smithi157.stdout:7/656: mkdir d0/d9/db/dc3 0 2024-04-22T23:35:07.936 INFO:tasks.workunit.client.1.smithi157.stdout:2/854: mkdir d7/d1f/d26/d11c 0 2024-04-22T23:35:07.936 INFO:tasks.workunit.client.0.smithi107.stdout:3/560: sync 2024-04-22T23:35:07.937 INFO:tasks.workunit.client.0.smithi107.stdout:8/507: sync 2024-04-22T23:35:07.937 INFO:tasks.workunit.client.0.smithi107.stdout:3/561: chown db/d21/d27/d15/d24/d52/l90 1523915 1 2024-04-22T23:35:07.937 INFO:tasks.workunit.client.0.smithi107.stdout:3/562: truncate db/d21/d27/d15/d24/d52/d60/f92 1176745 0 2024-04-22T23:35:07.937 INFO:tasks.workunit.client.0.smithi107.stdout:0/525: truncate d1/d84/f92 330747 0 2024-04-22T23:35:07.940 INFO:tasks.workunit.client.0.smithi107.stdout:6/575: dwrite db/d20/d2c/f44 [4194304,4194304] 0 2024-04-22T23:35:07.940 INFO:tasks.workunit.client.1.smithi157.stdout:7/657: dread d0/f4 [0,4194304] 0 2024-04-22T23:35:07.941 INFO:tasks.workunit.client.0.smithi107.stdout:4/584: mknod d1/d7/cc9 0 2024-04-22T23:35:07.942 INFO:tasks.workunit.client.1.smithi157.stdout:7/658: truncate d0/d9/d29/d33/d37/f50 115492 0 2024-04-22T23:35:07.942 INFO:tasks.workunit.client.1.smithi157.stdout:7/659: fdatasync d0/d9/d72/f44 0 2024-04-22T23:35:07.942 INFO:tasks.workunit.client.1.smithi157.stdout:7/660: creat d0/d9/d29/d98/fc4 x:0 0 0 2024-04-22T23:35:07.943 INFO:tasks.workunit.client.0.smithi107.stdout:8/508: truncate d0/d85/f50 158429 0 2024-04-22T23:35:07.943 INFO:tasks.workunit.client.0.smithi107.stdout:5/555: sync 2024-04-22T23:35:07.944 INFO:tasks.workunit.client.0.smithi107.stdout:1/495: sync 2024-04-22T23:35:07.944 INFO:tasks.workunit.client.0.smithi107.stdout:5/556: chown d9/db/de/f6b 33838 1 2024-04-22T23:35:07.945 INFO:tasks.workunit.client.1.smithi157.stdout:7/661: getdents d0/d9/db/df/d60 0 2024-04-22T23:35:07.946 INFO:tasks.workunit.client.0.smithi107.stdout:6/576: mknod db/d20/d2a/d5f/cae 0 2024-04-22T23:35:07.946 INFO:tasks.workunit.client.0.smithi107.stdout:6/577: readlink db/d20/d2c/d3b/l85 0 2024-04-22T23:35:07.946 INFO:tasks.workunit.client.0.smithi107.stdout:6/578: write db/d20/d58/f65 [998334,20148] 0 2024-04-22T23:35:07.947 INFO:tasks.workunit.client.0.smithi107.stdout:4/585: creat d1/d1b/fca x:0 0 0 2024-04-22T23:35:07.948 INFO:tasks.workunit.client.0.smithi107.stdout:9/578: read d5/d29/d7b/d91/f95 [1353297,101558] 0 2024-04-22T23:35:07.948 INFO:tasks.workunit.client.0.smithi107.stdout:9/579: fdatasync d5/f13 0 2024-04-22T23:35:07.948 INFO:tasks.workunit.client.0.smithi107.stdout:9/580: dread - d5/d7/d17/d1b/d1f/d36/fb8 zero size 2024-04-22T23:35:07.950 INFO:tasks.workunit.client.0.smithi107.stdout:8/509: rename d0/d7/df/d3b/c61 to d0/d7/cac 0 2024-04-22T23:35:07.951 INFO:tasks.workunit.client.0.smithi107.stdout:8/510: creat d0/d7/d8/d41/fad x:0 0 0 2024-04-22T23:35:07.951 INFO:tasks.workunit.client.0.smithi107.stdout:8/511: write d0/f83 [883129,7660] 0 2024-04-22T23:35:07.951 INFO:tasks.workunit.client.0.smithi107.stdout:2/590: read d1/d12/d35/d48/d13/d4f/f5b [1594294,100713] 0 2024-04-22T23:35:07.952 INFO:tasks.workunit.client.0.smithi107.stdout:5/557: mkdir d9/db/d92/dc2 0 2024-04-22T23:35:07.952 INFO:tasks.workunit.client.0.smithi107.stdout:5/558: readlink d9/db/l83 0 2024-04-22T23:35:07.953 INFO:tasks.workunit.client.0.smithi107.stdout:5/559: read - d9/db/de/fa6 zero size 2024-04-22T23:35:07.954 INFO:tasks.workunit.client.0.smithi107.stdout:0/526: dread d1/d27/d20/f36 [0,4194304] 0 2024-04-22T23:35:07.954 INFO:tasks.workunit.client.1.smithi157.stdout:2/855: write d7/d8/d30/d23/f48 [1360731,121915] 0 2024-04-22T23:35:07.955 INFO:tasks.workunit.client.0.smithi107.stdout:7/503: write d7/d21/f22 [382916,1235] 0 2024-04-22T23:35:07.955 INFO:tasks.workunit.client.0.smithi107.stdout:7/504: chown d7/d21/f7f 488355 1 2024-04-22T23:35:07.958 INFO:tasks.workunit.client.0.smithi107.stdout:4/586: symlink d1/d7/d63/d78/d80/lcb 0 2024-04-22T23:35:07.958 INFO:tasks.workunit.client.0.smithi107.stdout:4/587: write d1/d8/f7c [4511912,42715] 0 2024-04-22T23:35:07.958 INFO:tasks.workunit.client.0.smithi107.stdout:4/588: truncate d1/d7/f32 5862514 0 2024-04-22T23:35:07.960 INFO:tasks.workunit.client.0.smithi107.stdout:5/560: link d9/db/d94/f99 d9/d26/d79/fc3 0 2024-04-22T23:35:07.961 INFO:tasks.workunit.client.0.smithi107.stdout:0/527: mkdir d1/d27/d20/d40/db4 0 2024-04-22T23:35:07.961 INFO:tasks.workunit.client.0.smithi107.stdout:0/528: dread - d1/d27/d20/d2c/d41/fac zero size 2024-04-22T23:35:07.964 INFO:tasks.workunit.client.0.smithi107.stdout:8/512: write d0/d7/d8/f6c [835552,28488] 0 2024-04-22T23:35:07.968 INFO:tasks.workunit.client.0.smithi107.stdout:4/589: symlink d1/d21/d29/d2e/d7e/lcc 0 2024-04-22T23:35:07.968 INFO:tasks.workunit.client.0.smithi107.stdout:4/590: truncate d1/d21/d29/d2e/d88/f8b 1011227 0 2024-04-22T23:35:07.968 INFO:tasks.workunit.client.0.smithi107.stdout:4/591: readlink d1/d7/d5b/dbf/la5 0 2024-04-22T23:35:07.974 INFO:tasks.workunit.client.1.smithi157.stdout:4/743: dwrite d3/d11/d1d/d73/da7/fc5 [0,4194304] 0 2024-04-22T23:35:07.974 INFO:tasks.workunit.client.0.smithi107.stdout:0/529: mkdir d1/d27/d8a/db5 0 2024-04-22T23:35:07.974 INFO:tasks.workunit.client.0.smithi107.stdout:0/530: read - d1/d9/d14/f9c zero size 2024-04-22T23:35:07.976 INFO:tasks.workunit.client.0.smithi107.stdout:6/579: read db/f1d [2430161,78590] 0 2024-04-22T23:35:07.976 INFO:tasks.workunit.client.0.smithi107.stdout:6/580: read db/fd [301336,67437] 0 2024-04-22T23:35:07.976 INFO:tasks.workunit.client.0.smithi107.stdout:3/563: dwrite db/d21/d27/d15/db8/f94 [0,4194304] 0 2024-04-22T23:35:07.977 INFO:tasks.workunit.client.1.smithi157.stdout:7/662: dwrite d0/f2 [0,4194304] 0 2024-04-22T23:35:07.977 INFO:tasks.workunit.client.1.smithi157.stdout:7/663: write d0/d9/db/df/f9d [309431,63254] 0 2024-04-22T23:35:07.977 INFO:tasks.workunit.client.1.smithi157.stdout:7/664: write d0/d9/de/f23 [1150718,61263] 0 2024-04-22T23:35:07.978 INFO:tasks.workunit.client.0.smithi107.stdout:9/581: rename d5/d7/d27/d6f to d5/d14/d2e/d43/dc9 0 2024-04-22T23:35:07.978 INFO:tasks.workunit.client.0.smithi107.stdout:8/513: stat d0/d7/df/d33/c4d 0 2024-04-22T23:35:07.978 INFO:tasks.workunit.client.0.smithi107.stdout:8/514: chown d0/d7/d20/l4e 109926 1 2024-04-22T23:35:07.978 INFO:tasks.workunit.client.0.smithi107.stdout:7/505: read d7/d1b/f24 [6461836,11328] 0 2024-04-22T23:35:07.980 INFO:tasks.workunit.client.0.smithi107.stdout:4/592: symlink d1/d8/lcd 0 2024-04-22T23:35:07.982 INFO:tasks.workunit.client.0.smithi107.stdout:0/531: mkdir d1/d27/d20/d2c/db6 0 2024-04-22T23:35:07.982 INFO:tasks.workunit.client.0.smithi107.stdout:0/532: chown d1 2842 1 2024-04-22T23:35:07.984 INFO:tasks.workunit.client.1.smithi157.stdout:4/744: dread d3/d11/d1d/d73/da7/fad [0,4194304] 0 2024-04-22T23:35:07.984 INFO:tasks.workunit.client.1.smithi157.stdout:4/745: write d3/d11/d1d/f49 [887136,122177] 0 2024-04-22T23:35:07.984 INFO:tasks.workunit.client.1.smithi157.stdout:2/856: dwrite d7/d1f/d26/d4a/f11b [0,4194304] 0 2024-04-22T23:35:07.984 INFO:tasks.workunit.client.1.smithi157.stdout:2/857: fdatasync d7/d1f/d26/f44 0 2024-04-22T23:35:07.984 INFO:tasks.workunit.client.0.smithi107.stdout:6/581: creat db/d20/d2a/d38/d61/d6f/da2/d84/faf x:0 0 0 2024-04-22T23:35:07.986 INFO:tasks.workunit.client.1.smithi157.stdout:4/746: truncate d3/fda 574499 0 2024-04-22T23:35:07.990 INFO:tasks.workunit.client.0.smithi107.stdout:6/582: dread db/d20/d49/f6e [0,4194304] 0 2024-04-22T23:35:07.990 INFO:tasks.workunit.client.0.smithi107.stdout:5/561: dwrite d9/d76/da0/fbf [0,4194304] 0 2024-04-22T23:35:07.990 INFO:tasks.workunit.client.0.smithi107.stdout:5/562: chown d9/db/d42/lc0 118 1 2024-04-22T23:35:07.990 INFO:tasks.workunit.client.0.smithi107.stdout:5/563: stat d9/d26/c44 0 2024-04-22T23:35:07.991 INFO:tasks.workunit.client.1.smithi157.stdout:2/858: rename d7/d8/d30/d23/d27/d57/dcf/df2 to d7/d8/dd/d4d/d11d 0 2024-04-22T23:35:07.991 INFO:tasks.workunit.client.0.smithi107.stdout:3/564: truncate db/d21/d27/f20 1388104 0 2024-04-22T23:35:07.993 INFO:tasks.workunit.client.1.smithi157.stdout:2/859: creat d7/d9f/f11e x:0 0 0 2024-04-22T23:35:07.993 INFO:tasks.workunit.client.1.smithi157.stdout:2/860: write d7/d8/f101 [788900,39203] 0 2024-04-22T23:35:07.993 INFO:tasks.workunit.client.1.smithi157.stdout:2/861: stat d7/d8/d25/f3a 0 2024-04-22T23:35:07.994 INFO:tasks.workunit.client.0.smithi107.stdout:2/591: rename d1/d2b/f94 to d1/fcb 0 2024-04-22T23:35:07.994 INFO:tasks.workunit.client.0.smithi107.stdout:8/515: dread d0/d7/df/d33/d37/f80 [0,4194304] 0 2024-04-22T23:35:07.994 INFO:tasks.workunit.client.0.smithi107.stdout:1/496: dwrite d7/d19/d71/f78 [0,4194304] 0 2024-04-22T23:35:07.994 INFO:tasks.workunit.client.0.smithi107.stdout:9/582: creat d5/d14/fca x:0 0 0 2024-04-22T23:35:07.995 INFO:tasks.workunit.client.0.smithi107.stdout:4/593: symlink d1/d21/d29/lce 0 2024-04-22T23:35:07.995 INFO:tasks.workunit.client.0.smithi107.stdout:4/594: chown d1/d8/lcd 348 1 2024-04-22T23:35:07.995 INFO:tasks.workunit.client.0.smithi107.stdout:4/595: dread - d1/d7/d5b/dbf/da8/db9/fbb zero size 2024-04-22T23:35:07.995 INFO:tasks.workunit.client.0.smithi107.stdout:4/596: readlink d1/d21/d52/la7 0 2024-04-22T23:35:07.995 INFO:tasks.workunit.client.1.smithi157.stdout:2/862: unlink d7/d8/d30/d23/d27/d57/dcf/cd3 0 2024-04-22T23:35:07.997 INFO:tasks.workunit.client.0.smithi107.stdout:0/533: unlink d1/d9/c10 0 2024-04-22T23:35:07.997 INFO:tasks.workunit.client.0.smithi107.stdout:5/564: symlink d9/db/d94/lc4 0 2024-04-22T23:35:07.998 INFO:tasks.workunit.client.1.smithi157.stdout:2/863: mkdir d7/d8/d30/d23/d27/d57/d108/d11f 0 2024-04-22T23:35:07.998 INFO:tasks.workunit.client.1.smithi157.stdout:2/864: dread - d7/d8/d30/d23/f110 zero size 2024-04-22T23:35:07.998 INFO:tasks.workunit.client.1.smithi157.stdout:0/669: sync 2024-04-22T23:35:08.001 INFO:tasks.workunit.client.0.smithi107.stdout:8/516: creat d0/d7/df/d33/d9b/fae x:0 0 0 2024-04-22T23:35:08.001 INFO:tasks.workunit.client.0.smithi107.stdout:0/534: stat d1/d27/d20/d4a/c6a 0 2024-04-22T23:35:08.001 INFO:tasks.workunit.client.1.smithi157.stdout:2/865: mkdir d7/d8/d30/d23/d27/d57/d120 0 2024-04-22T23:35:08.001 INFO:tasks.workunit.client.1.smithi157.stdout:1/823: sync 2024-04-22T23:35:08.001 INFO:tasks.workunit.client.1.smithi157.stdout:3/751: sync 2024-04-22T23:35:08.001 INFO:tasks.workunit.client.1.smithi157.stdout:1/824: getdents d4/d17/d34/dc8 0 2024-04-22T23:35:08.002 INFO:tasks.workunit.client.0.smithi107.stdout:2/592: rmdir d1/d63 39 2024-04-22T23:35:08.002 INFO:tasks.workunit.client.0.smithi107.stdout:2/593: creat d1/d2b/d49/d7e/fcc x:0 0 0 2024-04-22T23:35:08.003 INFO:tasks.workunit.client.1.smithi157.stdout:0/670: creat d17/d23/d34/d67/d7f/d69/dce/fe5 x:0 0 0 2024-04-22T23:35:08.003 INFO:tasks.workunit.client.1.smithi157.stdout:0/671: chown d17/d23/c4e 435078 1 2024-04-22T23:35:08.003 INFO:tasks.workunit.client.1.smithi157.stdout:0/672: chown d17/d23/d34/d67 5402561 1 2024-04-22T23:35:08.005 INFO:tasks.workunit.client.1.smithi157.stdout:4/747: dread d3/f58 [0,4194304] 0 2024-04-22T23:35:08.006 INFO:tasks.workunit.client.0.smithi107.stdout:1/497: dread d7/d19/d20/d39/f7e [0,4194304] 0 2024-04-22T23:35:08.007 INFO:tasks.workunit.client.0.smithi107.stdout:2/594: read d1/d12/d35/f3f [77925,71116] 0 2024-04-22T23:35:08.007 INFO:tasks.workunit.client.0.smithi107.stdout:2/595: getdents d1/d2b/d49/d7e/dc6 0 2024-04-22T23:35:08.007 INFO:tasks.workunit.client.0.smithi107.stdout:2/596: dread - d1/d12/d35/d54/f73 zero size 2024-04-22T23:35:08.007 INFO:tasks.workunit.client.0.smithi107.stdout:2/597: fsync d1/d12/d35/d48/f2a 0 2024-04-22T23:35:08.007 INFO:tasks.workunit.client.0.smithi107.stdout:2/598: dread - d1/d12/fa4 zero size 2024-04-22T23:35:08.007 INFO:tasks.workunit.client.1.smithi157.stdout:1/825: unlink d4/d17/d1d/d33/d41/f5e 0 2024-04-22T23:35:08.008 INFO:tasks.workunit.client.0.smithi107.stdout:8/517: rmdir d0/d7/df/d17 39 2024-04-22T23:35:08.008 INFO:tasks.workunit.client.0.smithi107.stdout:8/518: creat d0/d7/df/d3b/faf x:0 0 0 2024-04-22T23:35:08.008 INFO:tasks.workunit.client.0.smithi107.stdout:8/519: write d0/f5f [682584,23535] 0 2024-04-22T23:35:08.008 INFO:tasks.workunit.client.0.smithi107.stdout:8/520: chown d0/d3a/d46 1052300107 1 2024-04-22T23:35:08.009 INFO:tasks.workunit.client.1.smithi157.stdout:4/748: mkdir d3/d11/d1d/d73/d80/dac/db5/dfb 0 2024-04-22T23:35:08.009 INFO:tasks.workunit.client.1.smithi157.stdout:4/749: stat d3/d96/d40/fab 0 2024-04-22T23:35:08.009 INFO:tasks.workunit.client.0.smithi107.stdout:2/599: getdents d1/d12/d35/d48 0 2024-04-22T23:35:08.009 INFO:tasks.workunit.client.0.smithi107.stdout:2/600: chown d1/d67/c7d 5 1 2024-04-22T23:35:08.009 INFO:tasks.workunit.client.0.smithi107.stdout:6/583: dread db/d20/d2a/d38/f3e [0,4194304] 0 2024-04-22T23:35:08.010 INFO:tasks.workunit.client.0.smithi107.stdout:6/584: creat db/d20/d2c/d3b/fb0 x:0 0 0 2024-04-22T23:35:08.010 INFO:tasks.workunit.client.1.smithi157.stdout:4/750: link d3/d11/d1d/fe3 d3/d11/d1d/ffc 0 2024-04-22T23:35:08.011 INFO:tasks.workunit.client.0.smithi107.stdout:9/583: write d5/d29/f2c [8325288,94841] 0 2024-04-22T23:35:08.013 INFO:tasks.workunit.client.1.smithi157.stdout:4/751: truncate d3/d11/d1d/d73/da7/fad 2347957 0 2024-04-22T23:35:08.016 INFO:tasks.workunit.client.1.smithi157.stdout:7/665: dwrite d0/d9/d42/f8c [0,4194304] 0 2024-04-22T23:35:08.016 INFO:tasks.workunit.client.1.smithi157.stdout:7/666: chown d0/d9/d10/f18 74949 1 2024-04-22T23:35:08.018 INFO:tasks.workunit.client.1.smithi157.stdout:0/673: dread d17/d23/f51 [0,4194304] 0 2024-04-22T23:35:08.019 INFO:tasks.workunit.client.1.smithi157.stdout:7/667: truncate d0/f4 686946 0 2024-04-22T23:35:08.019 INFO:tasks.workunit.client.0.smithi107.stdout:8/521: mkdir d0/d73/db0 0 2024-04-22T23:35:08.019 INFO:tasks.workunit.client.0.smithi107.stdout:8/522: write d0/d7/f31 [214856,108548] 0 2024-04-22T23:35:08.020 INFO:tasks.workunit.client.1.smithi157.stdout:0/674: mknod d17/ce6 0 2024-04-22T23:35:08.020 INFO:tasks.workunit.client.1.smithi157.stdout:0/675: readlink d17/d2b/d71/lb0 0 2024-04-22T23:35:08.024 INFO:tasks.workunit.client.0.smithi107.stdout:1/498: link d7/d19/f80 d7/fa0 0 2024-04-22T23:35:08.024 INFO:tasks.workunit.client.0.smithi107.stdout:3/565: dwrite db/d21/d27/d15/d29/d2a/f5b [4194304,4194304] 0 2024-04-22T23:35:08.031 INFO:tasks.workunit.client.0.smithi107.stdout:7/506: dwrite d7/ff [8388608,4194304] 0 2024-04-22T23:35:08.032 INFO:tasks.workunit.client.0.smithi107.stdout:7/507: fdatasync d7/d21/d49/f6a 0 2024-04-22T23:35:08.032 INFO:tasks.workunit.client.0.smithi107.stdout:7/508: write d7/d21/d48/f9c [754787,36326] 0 2024-04-22T23:35:08.032 INFO:tasks.workunit.client.0.smithi107.stdout:2/601: mkdir d1/dcd 0 2024-04-22T23:35:08.036 INFO:tasks.workunit.client.1.smithi157.stdout:7/668: dread d0/d9/d29/d49/d64/fb5 [0,4194304] 0 2024-04-22T23:35:08.036 INFO:tasks.workunit.client.1.smithi157.stdout:7/669: chown d0/d9/d29/d33/d7c/f9c 6073 1 2024-04-22T23:35:08.036 INFO:tasks.workunit.client.1.smithi157.stdout:7/670: read d0/d9/d42/d55/f8f [709258,112547] 0 2024-04-22T23:35:08.036 INFO:tasks.workunit.client.1.smithi157.stdout:7/671: dread - d0/d9/d29/d33/d37/f9b zero size 2024-04-22T23:35:08.036 INFO:tasks.workunit.client.1.smithi157.stdout:7/672: creat d0/d9/d29/d5e/d9e/fc5 x:0 0 0 2024-04-22T23:35:08.042 INFO:tasks.workunit.client.0.smithi107.stdout:8/523: mknod d0/d7/d8/cb1 0 2024-04-22T23:35:08.042 INFO:tasks.workunit.client.0.smithi107.stdout:8/524: write d0/f83 [909817,116976] 0 2024-04-22T23:35:08.044 INFO:tasks.workunit.client.1.smithi157.stdout:7/673: dread d0/f46 [0,4194304] 0 2024-04-22T23:35:08.045 INFO:tasks.workunit.client.1.smithi157.stdout:0/676: write d17/d2b/d52/f3f [1498549,108411] 0 2024-04-22T23:35:08.045 INFO:tasks.workunit.client.0.smithi107.stdout:3/566: mknod db/d21/d27/d54/da7/cc3 0 2024-04-22T23:35:08.046 INFO:tasks.workunit.client.1.smithi157.stdout:7/674: mkdir d0/d9/db/df/d68/dc6 0 2024-04-22T23:35:08.047 INFO:tasks.workunit.client.0.smithi107.stdout:7/509: symlink d7/d21/d7e/la3 0 2024-04-22T23:35:08.048 INFO:tasks.workunit.client.1.smithi157.stdout:2/866: dwrite d7/d1f/ff4 [0,4194304] 0 2024-04-22T23:35:08.048 INFO:tasks.workunit.client.1.smithi157.stdout:0/677: creat d17/d2b/d4f/d5a/da1/da8/fe7 x:0 0 0 2024-04-22T23:35:08.049 INFO:tasks.workunit.client.1.smithi157.stdout:7/675: symlink d0/d9/d29/lc7 0 2024-04-22T23:35:08.050 INFO:tasks.workunit.client.1.smithi157.stdout:1/826: dwrite d4/db/d71/f114 [0,4194304] 0 2024-04-22T23:35:08.051 INFO:tasks.workunit.client.0.smithi107.stdout:2/602: dread d1/d12/d35/d48/f29 [0,4194304] 0 2024-04-22T23:35:08.051 INFO:tasks.workunit.client.1.smithi157.stdout:4/752: dwrite d3/d11/d1d/d73/da3/d31/f6f [0,4194304] 0 2024-04-22T23:35:08.052 INFO:tasks.workunit.client.0.smithi107.stdout:8/525: symlink d0/d7/df/lb2 0 2024-04-22T23:35:08.055 INFO:tasks.workunit.client.1.smithi157.stdout:2/867: creat d7/d8/d95/dd2/dda/f121 x:0 0 0 2024-04-22T23:35:08.055 INFO:tasks.workunit.client.1.smithi157.stdout:0/678: symlink d17/d2b/d4f/d9b/le8 0 2024-04-22T23:35:08.057 INFO:tasks.workunit.client.0.smithi107.stdout:8/526: symlink d0/d85/d51/lb3 0 2024-04-22T23:35:08.057 INFO:tasks.workunit.client.0.smithi107.stdout:8/527: readlink d0/d7/d8/d41/d53/d76/l7a 0 2024-04-22T23:35:08.061 INFO:tasks.workunit.client.0.smithi107.stdout:7/510: rename d7/d2d to d7/d21/da4 0 2024-04-22T23:35:08.061 INFO:tasks.workunit.client.1.smithi157.stdout:7/676: mknod d0/d9/d36/cc8 0 2024-04-22T23:35:08.062 INFO:tasks.workunit.client.1.smithi157.stdout:1/827: creat d4/d17/d34/d75/d8a/f121 x:0 0 0 2024-04-22T23:35:08.066 INFO:tasks.workunit.client.1.smithi157.stdout:3/752: dwrite d0/d3/d34/d3d/d5b/d72/fb1 [0,4194304] 0 2024-04-22T23:35:08.066 INFO:tasks.workunit.client.0.smithi107.stdout:3/567: write db/d21/d27/d54/d25/d53/f7b [3997399,117611] 0 2024-04-22T23:35:08.070 INFO:tasks.workunit.client.1.smithi157.stdout:4/753: read d3/d5c/d7b/fa8 [534829,15486] 0 2024-04-22T23:35:08.070 INFO:tasks.workunit.client.1.smithi157.stdout:0/679: dread d17/d38/f56 [0,4194304] 0 2024-04-22T23:35:08.071 INFO:tasks.workunit.client.0.smithi107.stdout:0/535: dwrite d1/d9/f68 [0,4194304] 0 2024-04-22T23:35:08.071 INFO:tasks.workunit.client.0.smithi107.stdout:0/536: readlink d1/d2a/dad/l97 0 2024-04-22T23:35:08.072 INFO:tasks.workunit.client.1.smithi157.stdout:4/754: mkdir d3/d11/d30/dfd 0 2024-04-22T23:35:08.074 INFO:tasks.workunit.client.0.smithi107.stdout:2/603: dwrite d1/d2b/d59/fac [0,4194304] 0 2024-04-22T23:35:08.074 INFO:tasks.workunit.client.0.smithi107.stdout:8/528: rename d0/d7/d8/c22 to d0/d3a/cb4 0 2024-04-22T23:35:08.074 INFO:tasks.workunit.client.0.smithi107.stdout:5/565: dwrite d9/f29 [0,4194304] 0 2024-04-22T23:35:08.074 INFO:tasks.workunit.client.0.smithi107.stdout:5/566: stat d9/db/l83 0 2024-04-22T23:35:08.075 INFO:tasks.workunit.client.1.smithi157.stdout:0/680: dread d17/d23/d34/d67/d7f/f61 [0,4194304] 0 2024-04-22T23:35:08.075 INFO:tasks.workunit.client.1.smithi157.stdout:0/681: truncate d17/d2b/f3d 4889664 0 2024-04-22T23:35:08.075 INFO:tasks.workunit.client.0.smithi107.stdout:3/568: symlink db/d21/d27/d15/d29/d4f/db2/lc4 0 2024-04-22T23:35:08.076 INFO:tasks.workunit.client.0.smithi107.stdout:3/569: stat db/d21/d27/d54 0 2024-04-22T23:35:08.077 INFO:tasks.workunit.client.1.smithi157.stdout:2/868: write d7/f13 [7491145,3076] 0 2024-04-22T23:35:08.078 INFO:tasks.workunit.client.0.smithi107.stdout:0/537: unlink d1/d2a/dad/fab 0 2024-04-22T23:35:08.080 INFO:tasks.workunit.client.0.smithi107.stdout:9/584: dwrite d5/d14/f20 [0,4194304] 0 2024-04-22T23:35:08.080 INFO:tasks.workunit.client.0.smithi107.stdout:9/585: write d5/d29/f2c [4718857,125684] 0 2024-04-22T23:35:08.080 INFO:tasks.workunit.client.0.smithi107.stdout:9/586: truncate d5/d14/d28/f81 711943 0 2024-04-22T23:35:08.080 INFO:tasks.workunit.client.1.smithi157.stdout:2/869: link d7/d8/d30/f83 d7/d8/dca/f122 0 2024-04-22T23:35:08.080 INFO:tasks.workunit.client.1.smithi157.stdout:2/870: readlink d7/d8/d30/d5b/d8a/la9 0 2024-04-22T23:35:08.082 INFO:tasks.workunit.client.1.smithi157.stdout:2/871: rmdir d7/d8/d30/d23/d27/d57/d120 0 2024-04-22T23:35:08.082 INFO:tasks.workunit.client.0.smithi107.stdout:6/585: dwrite db/d20/d49/f6e [0,4194304] 0 2024-04-22T23:35:08.083 INFO:tasks.workunit.client.0.smithi107.stdout:1/499: dwrite d7/d19/d20/d39/f5a [0,4194304] 0 2024-04-22T23:35:08.083 INFO:tasks.workunit.client.0.smithi107.stdout:1/500: write d7/d19/d2d/f3d [4810094,100895] 0 2024-04-22T23:35:08.085 INFO:tasks.workunit.client.1.smithi157.stdout:2/872: dread d7/d8/d30/f83 [0,4194304] 0 2024-04-22T23:35:08.085 INFO:tasks.workunit.client.1.smithi157.stdout:2/873: chown d7/d8/d30/fe4 4367 1 2024-04-22T23:35:08.086 INFO:tasks.workunit.client.0.smithi107.stdout:7/511: mkdir d7/da5 0 2024-04-22T23:35:08.086 INFO:tasks.workunit.client.0.smithi107.stdout:7/512: fdatasync d7/f2f 0 2024-04-22T23:35:08.086 INFO:tasks.workunit.client.0.smithi107.stdout:5/567: unlink d9/db/de/f53 0 2024-04-22T23:35:08.088 INFO:tasks.workunit.client.1.smithi157.stdout:5/812: sync 2024-04-22T23:35:08.088 INFO:tasks.workunit.client.1.smithi157.stdout:5/813: dread - d1/d5/d7/d2f/d45/d54/f125 zero size 2024-04-22T23:35:08.088 INFO:tasks.workunit.client.1.smithi157.stdout:5/814: chown d1/d1f/c73 50716918 1 2024-04-22T23:35:08.088 INFO:tasks.workunit.client.1.smithi157.stdout:5/815: fsync d1/d27/d29/f5f 0 2024-04-22T23:35:08.088 INFO:tasks.workunit.client.1.smithi157.stdout:8/535: sync 2024-04-22T23:35:08.088 INFO:tasks.workunit.client.1.smithi157.stdout:2/874: mknod d7/d8/dd/c123 0 2024-04-22T23:35:08.089 INFO:tasks.workunit.client.1.smithi157.stdout:9/747: sync 2024-04-22T23:35:08.089 INFO:tasks.workunit.client.1.smithi157.stdout:9/748: chown d4/d15 156 1 2024-04-22T23:35:08.090 INFO:tasks.workunit.client.0.smithi107.stdout:3/570: mkdir db/d21/d27/d15/d29/d4f/db2/dc5 0 2024-04-22T23:35:08.090 INFO:tasks.workunit.client.0.smithi107.stdout:0/538: mknod d1/d9/d14/d4f/d59/cb7 0 2024-04-22T23:35:08.094 INFO:tasks.workunit.client.1.smithi157.stdout:0/682: write d17/d2b/f66 [59908,67296] 0 2024-04-22T23:35:08.097 INFO:tasks.workunit.client.1.smithi157.stdout:4/755: dread d3/d11/d1d/d73/da3/d31/f69 [0,4194304] 0 2024-04-22T23:35:08.097 INFO:tasks.workunit.client.1.smithi157.stdout:4/756: readlink d3/d96/dee/l93 0 2024-04-22T23:35:08.098 INFO:tasks.workunit.client.0.smithi107.stdout:9/587: truncate d5/f47 52507 0 2024-04-22T23:35:08.099 INFO:tasks.workunit.client.1.smithi157.stdout:4/757: creat d3/d11/d1d/d73/da3/d31/d91/d9f/ffe x:0 0 0 2024-04-22T23:35:08.099 INFO:tasks.workunit.client.1.smithi157.stdout:4/758: chown d3/d11/d30 4823271 1 2024-04-22T23:35:08.101 INFO:tasks.workunit.client.1.smithi157.stdout:4/759: rmdir d3/d11/d1d/d73/da7 39 2024-04-22T23:35:08.101 INFO:tasks.workunit.client.1.smithi157.stdout:4/760: chown d3/f1b 10 1 2024-04-22T23:35:08.106 INFO:tasks.workunit.client.0.smithi107.stdout:1/501: rename d7/d19/d20/d39/c77 to d7/d19/d2d/ca1 0 2024-04-22T23:35:08.107 INFO:tasks.workunit.client.1.smithi157.stdout:9/749: dread d4/df/d61/f84 [0,4194304] 0 2024-04-22T23:35:08.108 INFO:tasks.workunit.client.0.smithi107.stdout:7/513: symlink d7/d1b/la6 0 2024-04-22T23:35:08.108 INFO:tasks.workunit.client.1.smithi157.stdout:8/536: read d2/d6/fa2 [3405423,79060] 0 2024-04-22T23:35:08.109 INFO:tasks.workunit.client.1.smithi157.stdout:9/750: rename d4/d2d/f8e to d4/fe9 0 2024-04-22T23:35:08.109 INFO:tasks.workunit.client.1.smithi157.stdout:9/751: read d4/d15/d23/d27/f38 [288938,51727] 0 2024-04-22T23:35:08.110 INFO:tasks.workunit.client.0.smithi107.stdout:5/568: symlink d9/db/de/d72/d7c/lc5 0 2024-04-22T23:35:08.110 INFO:tasks.workunit.client.1.smithi157.stdout:8/537: link d2/db1/d58/l96 d2/d6/d6b/lbb 0 2024-04-22T23:35:08.110 INFO:tasks.workunit.client.1.smithi157.stdout:8/538: readlink d2/d6/d21/l39 0 2024-04-22T23:35:08.111 INFO:tasks.workunit.client.1.smithi157.stdout:9/752: getdents d4/d15/d23/dad 0 2024-04-22T23:35:08.114 INFO:tasks.workunit.client.1.smithi157.stdout:8/539: rename d2/d34/d6a/la4 to d2/d6/d6b/lbc 0 2024-04-22T23:35:08.114 INFO:tasks.workunit.client.1.smithi157.stdout:5/816: dread d1/d5/dc/d5d/fd3 [0,4194304] 0 2024-04-22T23:35:08.116 INFO:tasks.workunit.client.1.smithi157.stdout:7/677: dwrite d0/d9/d42/d55/fb2 [0,4194304] 0 2024-04-22T23:35:08.117 INFO:tasks.workunit.client.1.smithi157.stdout:8/540: rename d2/d6/d66 to d2/d4/dbd 0 2024-04-22T23:35:08.118 INFO:tasks.workunit.client.1.smithi157.stdout:8/541: write d2/d6/f3c [2885607,49797] 0 2024-04-22T23:35:08.118 INFO:tasks.workunit.client.1.smithi157.stdout:5/817: symlink d1/db0/df3/l132 0 2024-04-22T23:35:08.119 INFO:tasks.workunit.client.0.smithi107.stdout:6/586: dwrite db/f8c [0,4194304] 0 2024-04-22T23:35:08.121 INFO:tasks.workunit.client.1.smithi157.stdout:7/678: unlink d0/d9/db/f74 0 2024-04-22T23:35:08.121 INFO:tasks.workunit.client.1.smithi157.stdout:8/542: symlink d2/d6/d21/d5f/d17/d7f/lbe 0 2024-04-22T23:35:08.123 INFO:tasks.workunit.client.1.smithi157.stdout:8/543: write d2/d34/d6a/f76 [244965,95597] 0 2024-04-22T23:35:08.124 INFO:tasks.workunit.client.1.smithi157.stdout:1/828: dwrite d4/d17/d34/dc8/ff2 [0,4194304] 0 2024-04-22T23:35:08.124 INFO:tasks.workunit.client.1.smithi157.stdout:2/875: dread d7/d8/d30/d23/d27/f7b [0,4194304] 0 2024-04-22T23:35:08.125 INFO:tasks.workunit.client.1.smithi157.stdout:2/876: fsync d7/d8/d8e/f78 0 2024-04-22T23:35:08.125 INFO:tasks.workunit.client.1.smithi157.stdout:2/877: truncate d7/d8/d30/d23/f48 4996191 0 2024-04-22T23:35:08.125 INFO:tasks.workunit.client.1.smithi157.stdout:3/753: dwrite d0/d3/d34/d3d/d5b/fbb [0,4194304] 0 2024-04-22T23:35:08.125 INFO:tasks.workunit.client.1.smithi157.stdout:3/754: write d0/d3/d34/db9/fd4 [876667,98971] 0 2024-04-22T23:35:08.126 INFO:tasks.workunit.client.1.smithi157.stdout:8/544: rename d2/d4/ca6 to d2/cbf 0 2024-04-22T23:35:08.126 INFO:tasks.workunit.client.1.smithi157.stdout:8/545: write d2/d34/d6a/d77/f9f [418242,85941] 0 2024-04-22T23:35:08.126 INFO:tasks.workunit.client.1.smithi157.stdout:8/546: chown d2/d6/d6b/l90 44042923 1 2024-04-22T23:35:08.126 INFO:tasks.workunit.client.1.smithi157.stdout:1/829: creat d4/d17/db9/f122 x:0 0 0 2024-04-22T23:35:08.129 INFO:tasks.workunit.client.1.smithi157.stdout:8/547: mknod d2/d6/d6b/cc0 0 2024-04-22T23:35:08.129 INFO:tasks.workunit.client.1.smithi157.stdout:8/548: chown d2/db1/c8c 153254 1 2024-04-22T23:35:08.129 INFO:tasks.workunit.client.1.smithi157.stdout:1/830: mknod d4/d17/c123 0 2024-04-22T23:35:08.131 INFO:tasks.workunit.client.1.smithi157.stdout:8/549: unlink d2/d34/d6a/f6d 0 2024-04-22T23:35:08.132 INFO:tasks.workunit.client.1.smithi157.stdout:1/831: mkdir d4/d17/d1d/db2/d124 0 2024-04-22T23:35:08.133 INFO:tasks.workunit.client.0.smithi107.stdout:3/571: dwrite db/d21/d27/d91/fc2 [0,4194304] 0 2024-04-22T23:35:08.133 INFO:tasks.workunit.client.0.smithi107.stdout:3/572: chown db/d21/d27/d54/d25/c70 1320170 1 2024-04-22T23:35:08.133 INFO:tasks.workunit.client.1.smithi157.stdout:8/550: read d2/d34/d6a/d77/f9f [233798,94697] 0 2024-04-22T23:35:08.134 INFO:tasks.workunit.client.1.smithi157.stdout:5/818: dread d1/d5/f10b [0,4194304] 0 2024-04-22T23:35:08.134 INFO:tasks.workunit.client.1.smithi157.stdout:5/819: dread - d1/d5/d7/d2f/d45/d54/f125 zero size 2024-04-22T23:35:08.135 INFO:tasks.workunit.client.0.smithi107.stdout:8/529: dwrite d0/d7/d8/d41/d53/f67 [0,4194304] 0 2024-04-22T23:35:08.136 INFO:tasks.workunit.client.1.smithi157.stdout:1/832: unlink d4/d77/fcd 0 2024-04-22T23:35:08.137 INFO:tasks.workunit.client.0.smithi107.stdout:0/539: rmdir d1/d7b 39 2024-04-22T23:35:08.138 INFO:tasks.workunit.client.0.smithi107.stdout:1/502: creat d7/d19/d20/d39/d53/d43/d46/d68/fa2 x:0 0 0 2024-04-22T23:35:08.140 INFO:tasks.workunit.client.0.smithi107.stdout:7/514: symlink d7/d21/d7e/la7 0 2024-04-22T23:35:08.140 INFO:tasks.workunit.client.0.smithi107.stdout:7/515: truncate d7/d1b/fa2 389491 0 2024-04-22T23:35:08.140 INFO:tasks.workunit.client.0.smithi107.stdout:7/516: stat d7/c45 0 2024-04-22T23:35:08.141 INFO:tasks.workunit.client.0.smithi107.stdout:6/587: rmdir db/d20/d2a/d38/d61/d6f/d9b 39 2024-04-22T23:35:08.141 INFO:tasks.workunit.client.1.smithi157.stdout:8/551: rename d2/d6/d21/c91 to d2/d6/cc1 0 2024-04-22T23:35:08.141 INFO:tasks.workunit.client.1.smithi157.stdout:8/552: dread - d2/d4/dbd/f9d zero size 2024-04-22T23:35:08.143 INFO:tasks.workunit.client.0.smithi107.stdout:9/588: getdents d5/d7/d27/d46 0 2024-04-22T23:35:08.145 INFO:tasks.workunit.client.0.smithi107.stdout:3/573: write db/d21/d27/d15/db8/f94 [522901,63087] 0 2024-04-22T23:35:08.145 INFO:tasks.workunit.client.1.smithi157.stdout:8/553: unlink d2/d6/d21/d5f/c88 0 2024-04-22T23:35:08.145 INFO:tasks.workunit.client.0.smithi107.stdout:0/540: symlink d1/d2a/lb8 0 2024-04-22T23:35:08.148 INFO:tasks.workunit.client.1.smithi157.stdout:4/761: dwrite d3/d11/d1d/f41 [4194304,4194304] 0 2024-04-22T23:35:08.148 INFO:tasks.workunit.client.0.smithi107.stdout:1/503: symlink d7/d19/d20/d39/d53/d43/d46/d5c/la3 0 2024-04-22T23:35:08.149 INFO:tasks.workunit.client.1.smithi157.stdout:4/762: dread d3/d11/d30/f71 [0,4194304] 0 2024-04-22T23:35:08.149 INFO:tasks.workunit.client.0.smithi107.stdout:9/589: mkdir d5/d7/d17/d1b/dcb 0 2024-04-22T23:35:08.150 INFO:tasks.workunit.client.1.smithi157.stdout:5/820: rename d1/d1f/d22 to d1/db0/d7a/d133 0 2024-04-22T23:35:08.151 INFO:tasks.workunit.client.1.smithi157.stdout:3/755: dwrite d0/d3/d34/d39/f66 [4194304,4194304] 0 2024-04-22T23:35:08.151 INFO:tasks.workunit.client.0.smithi107.stdout:3/574: unlink db/d21/d27/d54/l31 0 2024-04-22T23:35:08.152 INFO:tasks.workunit.client.0.smithi107.stdout:5/569: dwrite d9/f60 [0,4194304] 0 2024-04-22T23:35:08.152 INFO:tasks.workunit.client.1.smithi157.stdout:0/683: dwrite d17/d2b/fd7 [0,4194304] 0 2024-04-22T23:35:08.154 INFO:tasks.workunit.client.0.smithi107.stdout:3/575: rename db/d21/d27/d15/f86 to db/d21/d27/d15/d24/d66/fc6 0 2024-04-22T23:35:08.155 INFO:tasks.workunit.client.1.smithi157.stdout:1/833: rename d4/d17/d1d/c10d to d4/d17/c125 0 2024-04-22T23:35:08.155 INFO:tasks.workunit.client.1.smithi157.stdout:7/679: dwrite d0/d9/db/df/d68/fb4 [0,4194304] 0 2024-04-22T23:35:08.157 INFO:tasks.workunit.client.0.smithi107.stdout:6/588: dread db/f12 [0,4194304] 0 2024-04-22T23:35:08.157 INFO:tasks.workunit.client.0.smithi107.stdout:6/589: dread - db/d20/d2a/d38/d61/f6d zero size 2024-04-22T23:35:08.157 INFO:tasks.workunit.client.1.smithi157.stdout:3/756: rmdir d0/d3/db7 0 2024-04-22T23:35:08.158 INFO:tasks.workunit.client.1.smithi157.stdout:3/757: chown d0/d3/d34/d3d/d5b/fda 433145 1 2024-04-22T23:35:08.158 INFO:tasks.workunit.client.0.smithi107.stdout:5/570: unlink d9/db/de/fa6 0 2024-04-22T23:35:08.158 INFO:tasks.workunit.client.0.smithi107.stdout:5/571: getdents d9/db/d92/dc2 0 2024-04-22T23:35:08.158 INFO:tasks.workunit.client.0.smithi107.stdout:5/572: write d9/d26/f38 [9414789,100028] 0 2024-04-22T23:35:08.159 INFO:tasks.workunit.client.0.smithi107.stdout:5/573: write d9/db/f8f [1144772,122312] 0 2024-04-22T23:35:08.159 INFO:tasks.workunit.client.0.smithi107.stdout:5/574: chown d9/d26/d79/l89 2 1 2024-04-22T23:35:08.159 INFO:tasks.workunit.client.1.smithi157.stdout:4/763: dread d3/d11/f1f [4194304,4194304] 0 2024-04-22T23:35:08.159 INFO:tasks.workunit.client.0.smithi107.stdout:3/576: rename db/d21/d27/d15/d29/d4f/db2/lc4 to db/d21/d27/d15/d24/d52/d60/lc7 0 2024-04-22T23:35:08.160 INFO:tasks.workunit.client.1.smithi157.stdout:2/878: dwrite d7/d8/dd/f21 [0,4194304] 0 2024-04-22T23:35:08.161 INFO:tasks.workunit.client.0.smithi107.stdout:7/517: dread d7/f18 [0,4194304] 0 2024-04-22T23:35:08.163 INFO:tasks.workunit.client.0.smithi107.stdout:3/577: creat db/d21/d27/d15/d29/d4f/fc8 x:0 0 0 2024-04-22T23:35:08.163 INFO:tasks.workunit.client.1.smithi157.stdout:0/684: write d17/d2b/fd7 [1470687,124995] 0 2024-04-22T23:35:08.164 INFO:tasks.workunit.client.1.smithi157.stdout:5/821: mknod d1/db0/d7a/c134 0 2024-04-22T23:35:08.165 INFO:tasks.workunit.client.0.smithi107.stdout:3/578: mkdir db/d21/d27/d15/d24/d66/dc9 0 2024-04-22T23:35:08.168 INFO:tasks.workunit.client.1.smithi157.stdout:1/834: unlink d4/db/fe 0 2024-04-22T23:35:08.174 INFO:tasks.workunit.client.1.smithi157.stdout:7/680: symlink d0/d9/d29/d5e/lc9 0 2024-04-22T23:35:08.174 INFO:tasks.workunit.client.1.smithi157.stdout:7/681: write d0/d9/d42/d55/d56/d91/fab [1154621,101400] 0 2024-04-22T23:35:08.174 INFO:tasks.workunit.client.1.smithi157.stdout:7/682: truncate d0/d9/d29/d33/d37/f9b 25151 0 2024-04-22T23:35:08.177 INFO:tasks.workunit.client.0.smithi107.stdout:0/541: dwrite d1/d9/d14/d4f/da6/faa [0,4194304] 0 2024-04-22T23:35:08.179 INFO:tasks.workunit.client.0.smithi107.stdout:8/530: dwrite d0/d3a/d46/f8a [0,4194304] 0 2024-04-22T23:35:08.184 INFO:tasks.workunit.client.1.smithi157.stdout:2/879: rename d7/d8/d25/d102/ld5 to d7/d1f/d26/d114/l124 0 2024-04-22T23:35:08.184 INFO:tasks.workunit.client.1.smithi157.stdout:3/758: write d0/d3/d34/d39/f96 [2689063,22054] 0 2024-04-22T23:35:08.184 INFO:tasks.workunit.client.1.smithi157.stdout:3/759: creat d0/d3/d34/d39/ff4 x:0 0 0 2024-04-22T23:35:08.186 INFO:tasks.workunit.client.1.smithi157.stdout:0/685: unlink d17/d38/f73 0 2024-04-22T23:35:08.187 INFO:tasks.workunit.client.1.smithi157.stdout:5/822: creat d1/d5/dc/d43/d4f/d10e/f135 x:0 0 0 2024-04-22T23:35:08.187 INFO:tasks.workunit.client.1.smithi157.stdout:7/683: creat d0/d9/db/df/d68/fca x:0 0 0 2024-04-22T23:35:08.188 INFO:tasks.workunit.client.0.smithi107.stdout:0/542: link d1/lb2 d1/d84/lb9 0 2024-04-22T23:35:08.189 INFO:tasks.workunit.client.1.smithi157.stdout:2/880: mknod d7/c125 0 2024-04-22T23:35:08.194 INFO:tasks.workunit.client.1.smithi157.stdout:1/835: dread d4/f25 [0,4194304] 0 2024-04-22T23:35:08.194 INFO:tasks.workunit.client.1.smithi157.stdout:8/554: dwrite d2/d4/d30/f62 [4194304,4194304] 0 2024-04-22T23:35:08.196 INFO:tasks.workunit.client.1.smithi157.stdout:7/684: rename d0/d9/db/c47 to d0/d9/d10/ccb 0 2024-04-22T23:35:08.197 INFO:tasks.workunit.client.1.smithi157.stdout:0/686: dread d17/d2b/d52/dd1/fd4 [0,4194304] 0 2024-04-22T23:35:08.198 INFO:tasks.workunit.client.0.smithi107.stdout:8/531: symlink d0/d7/d8/d41/lb5 0 2024-04-22T23:35:08.199 INFO:tasks.workunit.client.1.smithi157.stdout:2/881: dread d7/d8/d25/d102/fc4 [0,4194304] 0 2024-04-22T23:35:08.199 INFO:tasks.workunit.client.1.smithi157.stdout:2/882: chown d7/ff8 30691520 1 2024-04-22T23:35:08.199 INFO:tasks.workunit.client.1.smithi157.stdout:2/883: fdatasync d7/d1f/d3e/f96 0 2024-04-22T23:35:08.200 INFO:tasks.workunit.client.0.smithi107.stdout:7/518: dwrite d7/d21/d48/d5c/f75 [0,4194304] 0 2024-04-22T23:35:08.201 INFO:tasks.workunit.client.0.smithi107.stdout:7/519: creat d7/d21/d54/fa8 x:0 0 0 2024-04-22T23:35:08.201 INFO:tasks.workunit.client.0.smithi107.stdout:6/590: dwrite db/d20/d2a/f6a [0,4194304] 0 2024-04-22T23:35:08.202 INFO:tasks.workunit.client.0.smithi107.stdout:0/543: rmdir d1/d2a/dad/d86 0 2024-04-22T23:35:08.206 INFO:tasks.workunit.client.0.smithi107.stdout:9/590: dwrite d5/d14/d2e/d4f/d73/f9b [0,4194304] 0 2024-04-22T23:35:08.206 INFO:tasks.workunit.client.0.smithi107.stdout:5/575: dwrite d9/db/f27 [0,4194304] 0 2024-04-22T23:35:08.208 INFO:tasks.workunit.client.1.smithi157.stdout:8/555: rmdir d2/d4/dbd/d97 39 2024-04-22T23:35:08.209 INFO:tasks.workunit.client.1.smithi157.stdout:7/685: dread d0/d9/d72/f44 [4194304,4194304] 0 2024-04-22T23:35:08.210 INFO:tasks.workunit.client.1.smithi157.stdout:7/686: chown d0/f7 18709674 1 2024-04-22T23:35:08.210 INFO:tasks.workunit.client.1.smithi157.stdout:0/687: unlink ca 0 2024-04-22T23:35:08.212 INFO:tasks.workunit.client.0.smithi107.stdout:0/544: dread d1/d84/f6c [0,4194304] 0 2024-04-22T23:35:08.213 INFO:tasks.workunit.client.1.smithi157.stdout:2/884: unlink d7/d8/d8e/f88 0 2024-04-22T23:35:08.213 INFO:tasks.workunit.client.1.smithi157.stdout:2/885: fdatasync d7/d8/dd/d4d/f6e 0 2024-04-22T23:35:08.213 INFO:tasks.workunit.client.1.smithi157.stdout:2/886: dread - d7/ff8 zero size 2024-04-22T23:35:08.213 INFO:tasks.workunit.client.1.smithi157.stdout:1/836: rmdir d4/d5f/d7e 39 2024-04-22T23:35:08.215 INFO:tasks.workunit.client.1.smithi157.stdout:7/687: link d0/d9/f1d d0/d9/d42/fcc 0 2024-04-22T23:35:08.216 INFO:tasks.workunit.client.1.smithi157.stdout:0/688: mkdir d17/d2b/d4f/dc5/dcc/de9 0 2024-04-22T23:35:08.216 INFO:tasks.workunit.client.1.smithi157.stdout:0/689: write d17/d2b/fd7 [520085,89149] 0 2024-04-22T23:35:08.217 INFO:tasks.workunit.client.1.smithi157.stdout:0/690: chown f11 23 1 2024-04-22T23:35:08.217 INFO:tasks.workunit.client.1.smithi157.stdout:0/691: creat d17/d2b/d4f/d5a/da1/da8/fea x:0 0 0 2024-04-22T23:35:08.217 INFO:tasks.workunit.client.1.smithi157.stdout:0/692: fdatasync d17/d23/d34/d67/f8d 0 2024-04-22T23:35:08.217 INFO:tasks.workunit.client.0.smithi107.stdout:9/591: rename d5/d14/d28/f81 to d5/d7/d27/fcc 0 2024-04-22T23:35:08.220 INFO:tasks.workunit.client.1.smithi157.stdout:7/688: creat d0/d9/d29/d33/d37/fcd x:0 0 0 2024-04-22T23:35:08.220 INFO:tasks.workunit.client.1.smithi157.stdout:3/760: dwrite d0/d3/d34/d47/fad [0,4194304] 0 2024-04-22T23:35:08.220 INFO:tasks.workunit.client.1.smithi157.stdout:5/823: dwrite d1/d5/d7/d2f/d45/d54/f7b [0,4194304] 0 2024-04-22T23:35:08.220 INFO:tasks.workunit.client.0.smithi107.stdout:5/576: symlink d9/db/d42/lc6 0 2024-04-22T23:35:08.222 INFO:tasks.workunit.client.0.smithi107.stdout:6/591: rmdir db/d20/d2a/d38/d59/d70 39 2024-04-22T23:35:08.222 INFO:tasks.workunit.client.1.smithi157.stdout:1/837: getdents d4/db/d3d/dc1/d100 0 2024-04-22T23:35:08.223 INFO:tasks.workunit.client.0.smithi107.stdout:6/592: write db/d20/d2a/d38/f53 [695915,76338] 0 2024-04-22T23:35:08.224 INFO:tasks.workunit.client.0.smithi107.stdout:0/545: mknod d1/d27/d8a/d8b/cba 0 2024-04-22T23:35:08.224 INFO:tasks.workunit.client.1.smithi157.stdout:7/689: unlink d0/d9/de/laf 0 2024-04-22T23:35:08.226 INFO:tasks.workunit.client.0.smithi107.stdout:9/592: symlink d5/d24/lcd 0 2024-04-22T23:35:08.227 INFO:tasks.workunit.client.1.smithi157.stdout:8/556: dread d2/d6/d21/d5f/d11/f4b [0,4194304] 0 2024-04-22T23:35:08.227 INFO:tasks.workunit.client.1.smithi157.stdout:8/557: chown d2/d6/d21/d5f/f1b 37 1 2024-04-22T23:35:08.229 INFO:tasks.workunit.client.0.smithi107.stdout:5/577: unlink d9/db/de/c18 0 2024-04-22T23:35:08.233 INFO:tasks.workunit.client.0.smithi107.stdout:0/546: creat d1/d27/d20/d2c/fbb x:0 0 0 2024-04-22T23:35:08.234 INFO:tasks.workunit.client.1.smithi157.stdout:3/761: truncate d0/d3/d34/d62/f98 304137 0 2024-04-22T23:35:08.235 INFO:tasks.workunit.client.0.smithi107.stdout:5/578: mknod d9/db/d42/d75/db6/cc7 0 2024-04-22T23:35:08.235 INFO:tasks.workunit.client.0.smithi107.stdout:5/579: readlink d9/db/de/l62 0 2024-04-22T23:35:08.236 INFO:tasks.workunit.client.0.smithi107.stdout:0/547: rmdir d1/d7b 39 2024-04-22T23:35:08.236 INFO:tasks.workunit.client.0.smithi107.stdout:7/520: dwrite d7/d21/d48/d5c/d9d/f5a [0,4194304] 0 2024-04-22T23:35:08.236 INFO:tasks.workunit.client.0.smithi107.stdout:6/593: dread db/d20/d2c/f32 [4194304,4194304] 0 2024-04-22T23:35:08.237 INFO:tasks.workunit.client.1.smithi157.stdout:1/838: mkdir d4/d17/d1d/d33/d41/d54/db3/d126 0 2024-04-22T23:35:08.237 INFO:tasks.workunit.client.1.smithi157.stdout:1/839: chown d4/d17/d1d/d33/d41/l6a 352605954 1 2024-04-22T23:35:08.237 INFO:tasks.workunit.client.1.smithi157.stdout:1/840: chown d4/d17/d1d/d33/d41/l6a 25 1 2024-04-22T23:35:08.238 INFO:tasks.workunit.client.0.smithi107.stdout:5/580: link d9/db/f7b d9/d76/fc8 0 2024-04-22T23:35:08.239 INFO:tasks.workunit.client.1.smithi157.stdout:7/690: mkdir d0/d9/d42/d55/d80/d95/dce 0 2024-04-22T23:35:08.239 INFO:tasks.workunit.client.1.smithi157.stdout:7/691: fdatasync d0/d9/f7d 0 2024-04-22T23:35:08.239 INFO:tasks.workunit.client.1.smithi157.stdout:7/692: write d0/d9/f1d [4888107,33864] 0 2024-04-22T23:35:08.240 INFO:tasks.workunit.client.0.smithi107.stdout:7/521: mknod d7/d21/d48/d5c/ca9 0 2024-04-22T23:35:08.243 INFO:tasks.workunit.client.0.smithi107.stdout:5/581: creat d9/db/dc1/fc9 x:0 0 0 2024-04-22T23:35:08.243 INFO:tasks.workunit.client.0.smithi107.stdout:5/582: write d9/db/f5f [1046116,81885] 0 2024-04-22T23:35:08.244 INFO:tasks.workunit.client.1.smithi157.stdout:1/841: rename d4/d17/d6b to d4/db/d71/d127 0 2024-04-22T23:35:08.244 INFO:tasks.workunit.client.0.smithi107.stdout:0/548: rmdir d1/d9/d14/d4f 39 2024-04-22T23:35:08.244 INFO:tasks.workunit.client.1.smithi157.stdout:7/693: creat d0/d9/d29/d5e/d85/fcf x:0 0 0 2024-04-22T23:35:08.249 INFO:tasks.workunit.client.0.smithi107.stdout:6/594: rmdir db/d20/d2a 39 2024-04-22T23:35:08.250 INFO:tasks.workunit.client.0.smithi107.stdout:5/583: mknod d9/db/d92/dc2/cca 0 2024-04-22T23:35:08.252 INFO:tasks.workunit.client.0.smithi107.stdout:6/595: mkdir db/d20/db1 0 2024-04-22T23:35:08.253 INFO:tasks.workunit.client.1.smithi157.stdout:1/842: write d4/db/f36 [2667032,112777] 0 2024-04-22T23:35:08.253 INFO:tasks.workunit.client.0.smithi107.stdout:6/596: mknod db/d20/d2a/d38/d61/d6f/da2/cb2 0 2024-04-22T23:35:08.254 INFO:tasks.workunit.client.1.smithi157.stdout:1/843: read d4/d77/f93 [46414,62359] 0 2024-04-22T23:35:08.256 INFO:tasks.workunit.client.0.smithi107.stdout:6/597: mkdir db/d20/d2a/d35/d69/db3 0 2024-04-22T23:35:08.258 INFO:tasks.workunit.client.0.smithi107.stdout:6/598: creat db/d20/d49/d7b/fb4 x:0 0 0 2024-04-22T23:35:08.258 INFO:tasks.workunit.client.0.smithi107.stdout:6/599: stat db/d20/d2a/d38/d4b/l67 0 2024-04-22T23:35:08.260 INFO:tasks.workunit.client.1.smithi157.stdout:5/824: dread d1/d5/f9 [0,4194304] 0 2024-04-22T23:35:08.261 INFO:tasks.workunit.client.0.smithi107.stdout:6/600: unlink db/d20/d2a/d5d/caa 0 2024-04-22T23:35:08.263 INFO:tasks.workunit.client.0.smithi107.stdout:6/601: mkdir db/d20/d2c/db5 0 2024-04-22T23:35:08.267 INFO:tasks.workunit.client.1.smithi157.stdout:7/694: unlink d0/d9/d29/d5e/lc9 0 2024-04-22T23:35:08.267 INFO:tasks.workunit.client.1.smithi157.stdout:7/695: fdatasync d0/d9/d42/f8c 0 2024-04-22T23:35:08.268 INFO:tasks.workunit.client.1.smithi157.stdout:7/696: chown d0/d9/d10/l3f 4971 1 2024-04-22T23:35:08.268 INFO:tasks.workunit.client.1.smithi157.stdout:7/697: getdents d0/d9/d42/d55/d80/d95/dce 0 2024-04-22T23:35:08.269 INFO:tasks.workunit.client.1.smithi157.stdout:7/698: unlink d0/d9/db/f5a 0 2024-04-22T23:35:08.270 INFO:tasks.workunit.client.1.smithi157.stdout:7/699: mknod d0/d9/d29/d49/cd0 0 2024-04-22T23:35:08.270 INFO:tasks.workunit.client.1.smithi157.stdout:7/700: write d0/d9/db/df/d68/f8e [5137480,28257] 0 2024-04-22T23:35:08.272 INFO:tasks.workunit.client.1.smithi157.stdout:7/701: creat d0/d9/db/df/fd1 x:0 0 0 2024-04-22T23:35:08.274 INFO:tasks.workunit.client.1.smithi157.stdout:7/702: creat d0/d9/d29/d33/fd2 x:0 0 0 2024-04-22T23:35:08.276 INFO:tasks.workunit.client.1.smithi157.stdout:7/703: getdents d0/d9 0 2024-04-22T23:35:08.278 INFO:tasks.workunit.client.1.smithi157.stdout:7/704: symlink d0/d9/d29/d5e/d85/ld3 0 2024-04-22T23:35:08.278 INFO:tasks.workunit.client.1.smithi157.stdout:7/705: creat d0/d9/d29/d5e/d9e/fd4 x:0 0 0 2024-04-22T23:35:08.283 INFO:tasks.workunit.client.1.smithi157.stdout:5/825: write d1/d5/d7/d2f/d45/d54/f7b [3262867,89364] 0 2024-04-22T23:35:08.284 INFO:tasks.workunit.client.1.smithi157.stdout:5/826: mkdir d1/d5/dc/d43/d5e/d111/d136 0 2024-04-22T23:35:08.285 INFO:tasks.workunit.client.1.smithi157.stdout:5/827: symlink d1/d5/dc/l137 0 2024-04-22T23:35:08.290 INFO:tasks.workunit.client.1.smithi157.stdout:0/693: dwrite d17/d23/d34/d67/d7f/f61 [0,4194304] 0 2024-04-22T23:35:08.290 INFO:tasks.workunit.client.1.smithi157.stdout:0/694: write f3 [1925464,26686] 0 2024-04-22T23:35:08.292 INFO:tasks.workunit.client.1.smithi157.stdout:4/764: dwrite d3/d11/d1d/d73/da3/f2b [0,4194304] 0 2024-04-22T23:35:08.292 INFO:tasks.workunit.client.1.smithi157.stdout:4/765: truncate d3/d11/d34/da4/fbc 290044 0 2024-04-22T23:35:08.295 INFO:tasks.workunit.client.1.smithi157.stdout:4/766: mknod d3/d96/cff 0 2024-04-22T23:35:08.296 INFO:tasks.workunit.client.1.smithi157.stdout:4/767: fdatasync d3/d4b/d7a/f9c 0 2024-04-22T23:35:08.296 INFO:tasks.workunit.client.1.smithi157.stdout:4/768: fdatasync d3/d11/d30/f57 0 2024-04-22T23:35:08.296 INFO:tasks.workunit.client.1.smithi157.stdout:4/769: mknod d3/d11/d34/da4/c100 0 2024-04-22T23:35:08.296 INFO:tasks.workunit.client.1.smithi157.stdout:4/770: stat d3/d11/d1d/feb 0 2024-04-22T23:35:08.297 INFO:tasks.workunit.client.1.smithi157.stdout:3/762: rmdir d0/d3 39 2024-04-22T23:35:08.299 INFO:tasks.workunit.client.1.smithi157.stdout:8/558: dwrite d2/d6/d6b/f8b [0,4194304] 0 2024-04-22T23:35:08.299 INFO:tasks.workunit.client.1.smithi157.stdout:3/763: creat d0/d3/d34/d3d/ff5 x:0 0 0 2024-04-22T23:35:08.299 INFO:tasks.workunit.client.1.smithi157.stdout:1/844: dwrite d4/d17/d34/faf [0,4194304] 0 2024-04-22T23:35:08.299 INFO:tasks.workunit.client.1.smithi157.stdout:7/706: dread d0/d9/d10/f1f [0,4194304] 0 2024-04-22T23:35:08.299 INFO:tasks.workunit.client.1.smithi157.stdout:7/707: write d0/d9/db/df/dac/fad [495688,125698] 0 2024-04-22T23:35:08.299 INFO:tasks.workunit.client.1.smithi157.stdout:7/708: write d0/d9/d29/d33/d37/fcd [945251,111452] 0 2024-04-22T23:35:08.300 INFO:tasks.workunit.client.0.smithi107.stdout:8/532: dwrite d0/d73/f94 [0,4194304] 0 2024-04-22T23:35:08.300 INFO:tasks.workunit.client.0.smithi107.stdout:0/549: dwrite d1/d9/f5e [0,4194304] 0 2024-04-22T23:35:08.300 INFO:tasks.workunit.client.0.smithi107.stdout:5/584: dwrite d9/db/d42/d75/fb1 [0,4194304] 0 2024-04-22T23:35:08.300 INFO:tasks.workunit.client.0.smithi107.stdout:5/585: readlink d9/db/la7 0 2024-04-22T23:35:08.300 INFO:tasks.workunit.client.0.smithi107.stdout:5/586: write d9/db/f8c [1338049,75232] 0 2024-04-22T23:35:08.301 INFO:tasks.workunit.client.1.smithi157.stdout:2/887: dwrite d7/d8/d25/f33 [0,4194304] 0 2024-04-22T23:35:08.301 INFO:tasks.workunit.client.0.smithi107.stdout:6/602: dwrite db/d20/d2a/d35/d69/d79/f95 [0,4194304] 0 2024-04-22T23:35:08.301 INFO:tasks.workunit.client.0.smithi107.stdout:6/603: chown db/d20/d2c/d3b/f60 23358699 1 2024-04-22T23:35:08.301 INFO:tasks.workunit.client.0.smithi107.stdout:7/522: dwrite d7/f13 [0,4194304] 0 2024-04-22T23:35:08.302 INFO:tasks.workunit.client.0.smithi107.stdout:7/523: readlink d7/d21/d48/d5c/d9d/l65 0 2024-04-22T23:35:08.303 INFO:tasks.workunit.client.0.smithi107.stdout:8/533: mknod d0/d7/d20/d2d/cb6 0 2024-04-22T23:35:08.303 INFO:tasks.workunit.client.0.smithi107.stdout:8/534: read d0/d7/df/d3b/f2b [3069005,81363] 0 2024-04-22T23:35:08.304 INFO:tasks.workunit.client.0.smithi107.stdout:9/593: rmdir d5/d24 39 2024-04-22T23:35:08.306 INFO:tasks.workunit.client.0.smithi107.stdout:5/587: getdents d9/db/de/d72/d7c 0 2024-04-22T23:35:08.306 INFO:tasks.workunit.client.0.smithi107.stdout:8/535: dread d0/f83 [0,4194304] 0 2024-04-22T23:35:08.306 INFO:tasks.workunit.client.0.smithi107.stdout:8/536: write d0/d7/fa [257714,39975] 0 2024-04-22T23:35:08.312 INFO:tasks.workunit.client.0.smithi107.stdout:6/604: symlink db/d20/d2a/d38/d59/lb6 0 2024-04-22T23:35:08.313 INFO:tasks.workunit.client.1.smithi157.stdout:8/559: mkdir d2/d6/d21/d5f/d17/dc2 0 2024-04-22T23:35:08.313 INFO:tasks.workunit.client.1.smithi157.stdout:8/560: fsync d2/d6/f3c 0 2024-04-22T23:35:08.314 INFO:tasks.workunit.client.0.smithi107.stdout:7/524: mkdir d7/d4c/da1/daa 0 2024-04-22T23:35:08.314 INFO:tasks.workunit.client.0.smithi107.stdout:7/525: getdents d7/d21/d48/d95 0 2024-04-22T23:35:08.314 INFO:tasks.workunit.client.1.smithi157.stdout:3/764: rename d0/d3/d34/d3d/d5b to d0/d3/d34/d3d/df6 0 2024-04-22T23:35:08.317 INFO:tasks.workunit.client.0.smithi107.stdout:9/594: mkdir d5/d7/d17/d1b/d1f/d36/d96/dce 0 2024-04-22T23:35:08.318 INFO:tasks.workunit.client.0.smithi107.stdout:5/588: mkdir d9/db/dc1/dcb 0 2024-04-22T23:35:08.318 INFO:tasks.workunit.client.0.smithi107.stdout:5/589: chown d9/db/d92/l98 0 1 2024-04-22T23:35:08.318 INFO:tasks.workunit.client.1.smithi157.stdout:1/845: rename d4/d17/d34/d75/f89 to d4/d17/d1d/df5/f128 0 2024-04-22T23:35:08.319 INFO:tasks.workunit.client.0.smithi107.stdout:5/590: dread d9/db/d42/f4b [0,4194304] 0 2024-04-22T23:35:08.319 INFO:tasks.workunit.client.0.smithi107.stdout:5/591: fsync d9/d26/d6a/f6f 0 2024-04-22T23:35:08.320 INFO:tasks.workunit.client.0.smithi107.stdout:8/537: symlink d0/d73/lb7 0 2024-04-22T23:35:08.320 INFO:tasks.workunit.client.0.smithi107.stdout:8/538: chown d0/d3a/d46/l47 13499 1 2024-04-22T23:35:08.322 INFO:tasks.workunit.client.0.smithi107.stdout:6/605: truncate db/d20/d2a/d38/d4b/f54 1667957 0 2024-04-22T23:35:08.323 INFO:tasks.workunit.client.1.smithi157.stdout:2/888: mkdir d7/d8/d30/d23/d27/d57/d108/d11f/d126 0 2024-04-22T23:35:08.323 INFO:tasks.workunit.client.0.smithi107.stdout:0/550: dwrite d1/d9/d14/d4f/f82 [0,4194304] 0 2024-04-22T23:35:08.323 INFO:tasks.workunit.client.0.smithi107.stdout:0/551: chown d1/d27/d20/d2c/d41/c88 20 1 2024-04-22T23:35:08.324 INFO:tasks.workunit.client.0.smithi107.stdout:0/552: chown d1/d9/d14/d4f/l9b 291551 1 2024-04-22T23:35:08.324 INFO:tasks.workunit.client.0.smithi107.stdout:7/526: creat d7/d21/da4/fab x:0 0 0 2024-04-22T23:35:08.324 INFO:tasks.workunit.client.0.smithi107.stdout:7/527: stat d7/d21/da4/d66/l79 0 2024-04-22T23:35:08.324 INFO:tasks.workunit.client.1.smithi157.stdout:7/709: dwrite d0/f4 [0,4194304] 0 2024-04-22T23:35:08.324 INFO:tasks.workunit.client.1.smithi157.stdout:8/561: symlink d2/d6/d21/d5f/d17/lc3 0 2024-04-22T23:35:08.325 INFO:tasks.workunit.client.0.smithi107.stdout:9/595: mknod d5/d14/d2e/d4f/d73/ccf 0 2024-04-22T23:35:08.326 INFO:tasks.workunit.client.0.smithi107.stdout:4/597: sync 2024-04-22T23:35:08.326 INFO:tasks.workunit.client.0.smithi107.stdout:4/598: truncate d1/d21/d29/d2e/d7e/fc6 99989 0 2024-04-22T23:35:08.326 INFO:tasks.workunit.client.0.smithi107.stdout:4/599: creat d1/d8/fcf x:0 0 0 2024-04-22T23:35:08.326 INFO:tasks.workunit.client.0.smithi107.stdout:4/600: truncate d1/d21/d85/f9c 4672056 0 2024-04-22T23:35:08.326 INFO:tasks.workunit.client.0.smithi107.stdout:4/601: chown d1/f13 258390099 1 2024-04-22T23:35:08.326 INFO:tasks.workunit.client.0.smithi107.stdout:0/553: creat d1/d2a/dad/fbc x:0 0 0 2024-04-22T23:35:08.327 INFO:tasks.workunit.client.0.smithi107.stdout:0/554: chown d1/d9/d14/d4f/c65 12579 1 2024-04-22T23:35:08.327 INFO:tasks.workunit.client.1.smithi157.stdout:3/765: creat d0/d3/d34/d3d/d57/d60/d6b/ff7 x:0 0 0 2024-04-22T23:35:08.328 INFO:tasks.workunit.client.0.smithi107.stdout:7/528: mknod d7/d21/d7e/cac 0 2024-04-22T23:35:08.329 INFO:tasks.workunit.client.0.smithi107.stdout:5/592: dread d9/d26/f2c [0,4194304] 0 2024-04-22T23:35:08.329 INFO:tasks.workunit.client.1.smithi157.stdout:1/846: creat d4/d17/d34/f129 x:0 0 0 2024-04-22T23:35:08.332 INFO:tasks.workunit.client.1.smithi157.stdout:2/889: mknod d7/d8/d8e/d80/c127 0 2024-04-22T23:35:08.332 INFO:tasks.workunit.client.1.smithi157.stdout:2/890: read - d7/d8/dd/d4d/d11d/f113 zero size 2024-04-22T23:35:08.333 INFO:tasks.workunit.client.1.smithi157.stdout:8/562: symlink d2/d6/d21/d5f/d17/lc4 0 2024-04-22T23:35:08.333 INFO:tasks.workunit.client.1.smithi157.stdout:8/563: chown d2/d4/dbd/d97/fb6 31716222 1 2024-04-22T23:35:08.333 INFO:tasks.workunit.client.1.smithi157.stdout:8/564: truncate d2/d34/f60 553578 0 2024-04-22T23:35:08.336 INFO:tasks.workunit.client.0.smithi107.stdout:4/602: creat d1/d7/d63/d78/da4/fd0 x:0 0 0 2024-04-22T23:35:08.338 INFO:tasks.workunit.client.1.smithi157.stdout:3/766: link d0/d3/d34/d3d/df6/d72/f7f d0/d3/d13/ff8 0 2024-04-22T23:35:08.338 INFO:tasks.workunit.client.1.smithi157.stdout:3/767: fdatasync d0/d3/f6 0 2024-04-22T23:35:08.338 INFO:tasks.workunit.client.0.smithi107.stdout:0/555: mkdir d1/d27/d20/d40/dbd 0 2024-04-22T23:35:08.339 INFO:tasks.workunit.client.1.smithi157.stdout:2/891: truncate d7/d8/d30/fa6 1474933 0 2024-04-22T23:35:08.340 INFO:tasks.workunit.client.0.smithi107.stdout:7/529: rename d7/d21/d54/fa8 to d7/d4c/da1/fad 0 2024-04-22T23:35:08.341 INFO:tasks.workunit.client.1.smithi157.stdout:3/768: unlink d0/d3/l61 0 2024-04-22T23:35:08.341 INFO:tasks.workunit.client.1.smithi157.stdout:3/769: fdatasync d0/fa9 0 2024-04-22T23:35:08.341 INFO:tasks.workunit.client.1.smithi157.stdout:2/892: rmdir d7/d1f/d26/d114 39 2024-04-22T23:35:08.342 INFO:tasks.workunit.client.1.smithi157.stdout:9/753: sync 2024-04-22T23:35:08.343 INFO:tasks.workunit.client.1.smithi157.stdout:3/770: rmdir d0/d3/d34/db9 39 2024-04-22T23:35:08.343 INFO:tasks.workunit.client.0.smithi107.stdout:5/593: truncate d9/fd 257273 0 2024-04-22T23:35:08.345 INFO:tasks.workunit.client.0.smithi107.stdout:1/504: sync 2024-04-22T23:35:08.345 INFO:tasks.workunit.client.0.smithi107.stdout:2/604: sync 2024-04-22T23:35:08.345 INFO:tasks.workunit.client.0.smithi107.stdout:4/603: symlink d1/d8/ld1 0 2024-04-22T23:35:08.346 INFO:tasks.workunit.client.0.smithi107.stdout:0/556: getdents d1/d9/d14/d4f/da6 0 2024-04-22T23:35:08.346 INFO:tasks.workunit.client.0.smithi107.stdout:0/557: fdatasync d1/d27/fb0 0 2024-04-22T23:35:08.346 INFO:tasks.workunit.client.0.smithi107.stdout:0/558: dread - d1/d27/d20/d2c/d41/fac zero size 2024-04-22T23:35:08.346 INFO:tasks.workunit.client.0.smithi107.stdout:0/559: truncate d1/d9/d14/f8c 208581 0 2024-04-22T23:35:08.346 INFO:tasks.workunit.client.1.smithi157.stdout:9/754: rmdir d4/df/d57/dbd/dc6 39 2024-04-22T23:35:08.347 INFO:tasks.workunit.client.0.smithi107.stdout:3/579: sync 2024-04-22T23:35:08.347 INFO:tasks.workunit.client.1.smithi157.stdout:3/771: mkdir d0/d83/df9 0 2024-04-22T23:35:08.347 INFO:tasks.workunit.client.1.smithi157.stdout:3/772: write d0/d3/d34/db9/d9f/fdb [427363,129068] 0 2024-04-22T23:35:08.348 INFO:tasks.workunit.client.0.smithi107.stdout:2/605: mknod d1/d2b/d49/d7e/cce 0 2024-04-22T23:35:08.349 INFO:tasks.workunit.client.0.smithi107.stdout:1/505: mkdir d7/d19/d2d/da4 0 2024-04-22T23:35:08.350 INFO:tasks.workunit.client.1.smithi157.stdout:7/710: dwrite d0/d9/d42/d55/fb2 [0,4194304] 0 2024-04-22T23:35:08.351 INFO:tasks.workunit.client.1.smithi157.stdout:3/773: unlink d0/d3/d34/d3d/d57/d5c/fdd 0 2024-04-22T23:35:08.354 INFO:tasks.workunit.client.0.smithi107.stdout:3/580: rmdir db/d21/d27/d15/d24/d66/da5 39 2024-04-22T23:35:08.355 INFO:tasks.workunit.client.1.smithi157.stdout:3/774: rmdir d0/d3/d34/d3d/d57/d60/d6b/d97 39 2024-04-22T23:35:08.356 INFO:tasks.workunit.client.1.smithi157.stdout:3/775: write d0/d3/d34/db9/d9f/fdb [498276,17151] 0 2024-04-22T23:35:08.356 INFO:tasks.workunit.client.1.smithi157.stdout:3/776: read - d0/d3/d34/d4a/fdc zero size 2024-04-22T23:35:08.356 INFO:tasks.workunit.client.0.smithi107.stdout:7/530: dread d7/d21/d48/d5c/f75 [0,4194304] 0 2024-04-22T23:35:08.356 INFO:tasks.workunit.client.0.smithi107.stdout:9/596: dwrite d5/d7/d17/f4e [0,4194304] 0 2024-04-22T23:35:08.358 INFO:tasks.workunit.client.1.smithi157.stdout:3/777: link d0/d3/d34/d39/f49 d0/d3/d34/d3d/d57/d60/d9c/ffa 0 2024-04-22T23:35:08.359 INFO:tasks.workunit.client.0.smithi107.stdout:3/581: mkdir db/d21/d27/d15/d24/d52/d60/dca 0 2024-04-22T23:35:08.359 INFO:tasks.workunit.client.0.smithi107.stdout:3/582: readlink db/d21/d27/d15/l9e 0 2024-04-22T23:35:08.359 INFO:tasks.workunit.client.0.smithi107.stdout:3/583: chown db/d21/d27/d15/d24/d87/faf 18 1 2024-04-22T23:35:08.360 INFO:tasks.workunit.client.0.smithi107.stdout:7/531: rmdir d7/d4c/da1/daa 0 2024-04-22T23:35:08.360 INFO:tasks.workunit.client.0.smithi107.stdout:6/606: dwrite db/d20/d2a/d38/f56 [0,4194304] 0 2024-04-22T23:35:08.362 INFO:tasks.workunit.client.0.smithi107.stdout:1/506: truncate d7/f15 1199587 0 2024-04-22T23:35:08.364 INFO:tasks.workunit.client.0.smithi107.stdout:3/584: creat db/d21/d27/fcb x:0 0 0 2024-04-22T23:35:08.365 INFO:tasks.workunit.client.1.smithi157.stdout:2/893: dread d7/d8/d30/d23/f7f [0,4194304] 0 2024-04-22T23:35:08.365 INFO:tasks.workunit.client.0.smithi107.stdout:6/607: creat db/d20/d2a/d38/d59/d70/fb7 x:0 0 0 2024-04-22T23:35:08.366 INFO:tasks.workunit.client.0.smithi107.stdout:1/507: fdatasync d7/f67 0 2024-04-22T23:35:08.368 INFO:tasks.workunit.client.1.smithi157.stdout:9/755: write d4/df/d61/f71 [1797611,15019] 0 2024-04-22T23:35:08.368 INFO:tasks.workunit.client.1.smithi157.stdout:9/756: write d4/df/d57/fd9 [1015662,76934] 0 2024-04-22T23:35:08.369 INFO:tasks.workunit.client.1.smithi157.stdout:9/757: readlink d4/df/d80/la0 0 2024-04-22T23:35:08.373 INFO:tasks.workunit.client.1.smithi157.stdout:8/565: dwrite d2/d6/d21/d5f/d11/d13/f6c [0,4194304] 0 2024-04-22T23:35:08.373 INFO:tasks.workunit.client.1.smithi157.stdout:9/758: creat d4/d15/d23/d27/d75/fea x:0 0 0 2024-04-22T23:35:08.373 INFO:tasks.workunit.client.1.smithi157.stdout:9/759: stat d4/df/d57/l7b 0 2024-04-22T23:35:08.375 INFO:tasks.workunit.client.0.smithi107.stdout:7/532: dread d7/d21/f22 [0,4194304] 0 2024-04-22T23:35:08.376 INFO:tasks.workunit.client.1.smithi157.stdout:8/566: mknod d2/d6/d21/d5f/cc5 0 2024-04-22T23:35:08.377 INFO:tasks.workunit.client.1.smithi157.stdout:9/760: unlink d4/d15/d23/d27/d75/c77 0 2024-04-22T23:35:08.379 INFO:tasks.workunit.client.1.smithi157.stdout:8/567: creat d2/d4/d30/db8/fc6 x:0 0 0 2024-04-22T23:35:08.380 INFO:tasks.workunit.client.0.smithi107.stdout:7/533: link d7/d21/d49/f6a d7/d4c/da1/fae 0 2024-04-22T23:35:08.382 INFO:tasks.workunit.client.0.smithi107.stdout:1/508: dread d7/d19/d20/d39/d53/d43/f49 [0,4194304] 0 2024-04-22T23:35:08.388 INFO:tasks.workunit.client.0.smithi107.stdout:1/509: link d7/d19/d20/d39/f3c d7/d19/d71/fa5 0 2024-04-22T23:35:08.391 INFO:tasks.workunit.client.1.smithi157.stdout:1/847: dwrite d4/d17/d1d/d33/d41/d54/f98 [0,4194304] 0 2024-04-22T23:35:08.392 INFO:tasks.workunit.client.0.smithi107.stdout:1/510: getdents d7/d19/d20/d39/d53/d43/d63 0 2024-04-22T23:35:08.393 INFO:tasks.workunit.client.1.smithi157.stdout:1/848: unlink d4/db/f4a 0 2024-04-22T23:35:08.396 INFO:tasks.workunit.client.0.smithi107.stdout:1/511: rmdir d7/d19/d20/d39/d53 39 2024-04-22T23:35:08.396 INFO:tasks.workunit.client.0.smithi107.stdout:1/512: readlink d7/d19/d20/l7c 0 2024-04-22T23:35:08.396 INFO:tasks.workunit.client.0.smithi107.stdout:1/513: fdatasync d7/d19/d20/d39/f3c 0 2024-04-22T23:35:08.397 INFO:tasks.workunit.client.1.smithi157.stdout:1/849: dread d4/d17/d34/d3b/d52/fd9 [0,4194304] 0 2024-04-22T23:35:08.405 INFO:tasks.workunit.client.0.smithi107.stdout:2/606: dwrite d1/d2b/d49/d7e/fbe [0,4194304] 0 2024-04-22T23:35:08.405 INFO:tasks.workunit.client.1.smithi157.stdout:7/711: dwrite d0/d9/f1d [4194304,4194304] 0 2024-04-22T23:35:08.406 INFO:tasks.workunit.client.0.smithi107.stdout:1/514: dread d7/d19/d20/d39/d53/d43/d46/f47 [0,4194304] 0 2024-04-22T23:35:08.409 INFO:tasks.workunit.client.1.smithi157.stdout:7/712: mkdir d0/d9/db/da3/dd5 0 2024-04-22T23:35:08.409 INFO:tasks.workunit.client.1.smithi157.stdout:7/713: read d0/d9/db/f61 [840300,50378] 0 2024-04-22T23:35:08.410 INFO:tasks.workunit.client.1.smithi157.stdout:7/714: symlink d0/d9/db/da3/ld6 0 2024-04-22T23:35:08.411 INFO:tasks.workunit.client.1.smithi157.stdout:7/715: write d0/d9/d29/d5e/d9e/fb3 [875777,13893] 0 2024-04-22T23:35:08.411 INFO:tasks.workunit.client.0.smithi107.stdout:5/594: dwrite d9/d26/d6a/f6f [0,4194304] 0 2024-04-22T23:35:08.411 INFO:tasks.workunit.client.0.smithi107.stdout:5/595: fsync f7 0 2024-04-22T23:35:08.411 INFO:tasks.workunit.client.0.smithi107.stdout:1/515: symlink d7/d19/d20/d39/d93/la6 0 2024-04-22T23:35:08.413 INFO:tasks.workunit.client.0.smithi107.stdout:4/604: rename d1/d8 to d1/d21/d29/d2e/d88/d99/dd2 0 2024-04-22T23:35:08.413 INFO:tasks.workunit.client.0.smithi107.stdout:4/605: chown d1/d21/d29/d2e/f46 41441877 1 2024-04-22T23:35:08.414 INFO:tasks.workunit.client.0.smithi107.stdout:1/516: creat d7/d19/d20/fa7 x:0 0 0 2024-04-22T23:35:08.414 INFO:tasks.workunit.client.0.smithi107.stdout:1/517: fdatasync d7/d19/d71/fa5 0 2024-04-22T23:35:08.415 INFO:tasks.workunit.client.0.smithi107.stdout:4/606: creat d1/d7/d63/d78/d96/da0/fd3 x:0 0 0 2024-04-22T23:35:08.415 INFO:tasks.workunit.client.0.smithi107.stdout:4/607: chown d1/d7/d63/d78/l94 0 1 2024-04-22T23:35:08.415 INFO:tasks.workunit.client.0.smithi107.stdout:4/608: fsync d1/f28 0 2024-04-22T23:35:08.415 INFO:tasks.workunit.client.0.smithi107.stdout:4/609: write d1/d21/d52/f97 [1014046,65488] 0 2024-04-22T23:35:08.416 INFO:tasks.workunit.client.0.smithi107.stdout:1/518: link d7/d19/d20/d39/f5a d7/d19/d20/d39/d53/d43/fa8 0 2024-04-22T23:35:08.416 INFO:tasks.workunit.client.0.smithi107.stdout:1/519: stat d7/d19/d42/c50 0 2024-04-22T23:35:08.417 INFO:tasks.workunit.client.0.smithi107.stdout:1/520: fdatasync d7/d19/d42/f6d 0 2024-04-22T23:35:08.417 INFO:tasks.workunit.client.0.smithi107.stdout:4/610: truncate d1/d21/d29/f71 186498 0 2024-04-22T23:35:08.417 INFO:tasks.workunit.client.1.smithi157.stdout:7/716: read d0/d9/db/f39 [1784397,48334] 0 2024-04-22T23:35:08.418 INFO:tasks.workunit.client.0.smithi107.stdout:1/521: link d7/d19/d20/d39/d53/d43/d46/d5c/la3 d7/d19/d20/d39/d53/d43/d46/d68/la9 0 2024-04-22T23:35:08.420 INFO:tasks.workunit.client.0.smithi107.stdout:5/596: dread d9/d26/f38 [4194304,4194304] 0 2024-04-22T23:35:08.425 INFO:tasks.workunit.client.0.smithi107.stdout:3/585: dwrite db/d21/d27/d15/f8c [0,4194304] 0 2024-04-22T23:35:08.429 INFO:tasks.workunit.client.1.smithi157.stdout:3/778: dwrite d0/d3/d34/d47/f70 [0,4194304] 0 2024-04-22T23:35:08.429 INFO:tasks.workunit.client.1.smithi157.stdout:2/894: dwrite d7/d8/d30/d5b/f73 [4194304,4194304] 0 2024-04-22T23:35:08.429 INFO:tasks.workunit.client.1.smithi157.stdout:2/895: fdatasync d7/d1f/d26/d4a/f53 0 2024-04-22T23:35:08.430 INFO:tasks.workunit.client.0.smithi107.stdout:6/608: dwrite db/d20/d58/f65 [0,4194304] 0 2024-04-22T23:35:08.430 INFO:tasks.workunit.client.0.smithi107.stdout:3/586: mkdir db/d21/d27/d15/d24/d52/d60/dca/dcc 0 2024-04-22T23:35:08.433 INFO:tasks.workunit.client.1.smithi157.stdout:3/779: rename d0/d3/d34/db9/d9f/fdb to d0/d3/ffb 0 2024-04-22T23:35:08.435 INFO:tasks.workunit.client.0.smithi107.stdout:6/609: rename db/d20/d2a/f6a to db/d20/d2c/d3b/d40/d9e/fb8 0 2024-04-22T23:35:08.435 INFO:tasks.workunit.client.0.smithi107.stdout:6/610: chown db/d20/d2a/d5d 7322 1 2024-04-22T23:35:08.435 INFO:tasks.workunit.client.0.smithi107.stdout:6/611: write db/d20/d2c/d3b/fb0 [236823,35189] 0 2024-04-22T23:35:08.437 INFO:tasks.workunit.client.0.smithi107.stdout:3/587: mkdir db/d21/db1/dcd 0 2024-04-22T23:35:08.438 INFO:tasks.workunit.client.0.smithi107.stdout:3/588: getdents db/d21/d27/d15/d29/d4f/db2/dc5 0 2024-04-22T23:35:08.438 INFO:tasks.workunit.client.0.smithi107.stdout:3/589: write db/d21/d27/d15/d24/d52/fbd [457844,82310] 0 2024-04-22T23:35:08.439 INFO:tasks.workunit.client.1.smithi157.stdout:9/761: rmdir d4/d15 39 2024-04-22T23:35:08.439 INFO:tasks.workunit.client.1.smithi157.stdout:9/762: fsync d4/d15/d23/d27/d47/fd2 0 2024-04-22T23:35:08.441 INFO:tasks.workunit.client.1.smithi157.stdout:9/763: read d4/df/f8b [225797,21920] 0 2024-04-22T23:35:08.441 INFO:tasks.workunit.client.1.smithi157.stdout:9/764: write d4/d15/d23/d67/f95 [5208528,126265] 0 2024-04-22T23:35:08.442 INFO:tasks.workunit.client.1.smithi157.stdout:9/765: symlink d4/df/d61/leb 0 2024-04-22T23:35:08.442 INFO:tasks.workunit.client.0.smithi107.stdout:6/612: creat db/d20/d58/fb9 x:0 0 0 2024-04-22T23:35:08.442 INFO:tasks.workunit.client.0.smithi107.stdout:6/613: dread - db/d20/d2a/d38/d59/d70/fb7 zero size 2024-04-22T23:35:08.443 INFO:tasks.workunit.client.0.smithi107.stdout:3/590: mknod db/d21/db1/dcd/cce 0 2024-04-22T23:35:08.443 INFO:tasks.workunit.client.1.smithi157.stdout:8/568: dwrite d2/f5 [8388608,4194304] 0 2024-04-22T23:35:08.444 INFO:tasks.workunit.client.1.smithi157.stdout:1/850: dwrite d4/d17/d34/d3b/d52/fc9 [0,4194304] 0 2024-04-22T23:35:08.444 INFO:tasks.workunit.client.1.smithi157.stdout:1/851: creat d4/d17/d34/d75/f12a x:0 0 0 2024-04-22T23:35:08.445 INFO:tasks.workunit.client.0.smithi107.stdout:7/534: dwrite d7/d21/d54/f90 [0,4194304] 0 2024-04-22T23:35:08.445 INFO:tasks.workunit.client.1.smithi157.stdout:8/569: dread d2/d4/f80 [0,4194304] 0 2024-04-22T23:35:08.445 INFO:tasks.workunit.client.1.smithi157.stdout:8/570: write d2/d4/f80 [1351149,43022] 0 2024-04-22T23:35:08.445 INFO:tasks.workunit.client.1.smithi157.stdout:8/571: dread - d2/d4/dbd/f9d zero size 2024-04-22T23:35:08.446 INFO:tasks.workunit.client.0.smithi107.stdout:2/607: dwrite d1/d2b/d49/fc5 [0,4194304] 0 2024-04-22T23:35:08.448 INFO:tasks.workunit.client.0.smithi107.stdout:3/591: rename db/d21/d27/d15/d29/d4f/fc8 to db/fcf 0 2024-04-22T23:35:08.454 INFO:tasks.workunit.client.0.smithi107.stdout:5/597: dwrite d9/f4e [4194304,4194304] 0 2024-04-22T23:35:08.455 INFO:tasks.workunit.client.0.smithi107.stdout:5/598: fdatasync d9/db/f8f 0 2024-04-22T23:35:08.455 INFO:tasks.workunit.client.0.smithi107.stdout:5/599: write d9/db/dc1/fc9 [493901,79378] 0 2024-04-22T23:35:08.455 INFO:tasks.workunit.client.0.smithi107.stdout:1/522: dwrite d7/d19/d71/f78 [0,4194304] 0 2024-04-22T23:35:08.455 INFO:tasks.workunit.client.0.smithi107.stdout:1/523: read - d7/d19/d2d/f9b zero size 2024-04-22T23:35:08.455 INFO:tasks.workunit.client.1.smithi157.stdout:7/717: dwrite d0/d9/d29/d33/d37/f4c [0,4194304] 0 2024-04-22T23:35:08.457 INFO:tasks.workunit.client.1.smithi157.stdout:1/852: write d4/db/f24 [1903418,127959] 0 2024-04-22T23:35:08.457 INFO:tasks.workunit.client.1.smithi157.stdout:3/780: dwrite d0/d3/d34/d3d/d57/d60/d9c/ffa [4194304,4194304] 0 2024-04-22T23:35:08.457 INFO:tasks.workunit.client.1.smithi157.stdout:7/718: creat d0/d9/d29/d33/d7c/fd7 x:0 0 0 2024-04-22T23:35:08.457 INFO:tasks.workunit.client.1.smithi157.stdout:2/896: dwrite d7/d8/d30/d23/d27/f49 [0,4194304] 0 2024-04-22T23:35:08.457 INFO:tasks.workunit.client.0.smithi107.stdout:2/608: rename d1/d12/d35/d48/f29 to d1/d2b/d49/d87/fcf 0 2024-04-22T23:35:08.460 INFO:tasks.workunit.client.0.smithi107.stdout:3/592: unlink db/f1c 0 2024-04-22T23:35:08.461 INFO:tasks.workunit.client.0.smithi107.stdout:7/535: getdents d7/d21/d48 0 2024-04-22T23:35:08.463 INFO:tasks.workunit.client.1.smithi157.stdout:2/897: mkdir d7/d1f/d26/d4a/d128 0 2024-04-22T23:35:08.466 INFO:tasks.workunit.client.1.smithi157.stdout:2/898: link d7/d8/d25/d102/ddb/cf3 d7/d1f/d26/d4a/d128/c129 0 2024-04-22T23:35:08.466 INFO:tasks.workunit.client.1.smithi157.stdout:2/899: creat d7/d9f/f12a x:0 0 0 2024-04-22T23:35:08.466 INFO:tasks.workunit.client.1.smithi157.stdout:2/900: write d7/d1f/d26/fe1 [254542,40597] 0 2024-04-22T23:35:08.468 INFO:tasks.workunit.client.0.smithi107.stdout:5/600: creat d9/db/d42/fcc x:0 0 0 2024-04-22T23:35:08.468 INFO:tasks.workunit.client.1.smithi157.stdout:7/719: dread d0/d9/db/df/d68/fb4 [0,4194304] 0 2024-04-22T23:35:08.468 INFO:tasks.workunit.client.1.smithi157.stdout:7/720: dread - d0/d9/d29/d49/f84 zero size 2024-04-22T23:35:08.469 INFO:tasks.workunit.client.0.smithi107.stdout:1/524: mknod d7/d19/d54/caa 0 2024-04-22T23:35:08.470 INFO:tasks.workunit.client.1.smithi157.stdout:5/828: sync 2024-04-22T23:35:08.470 INFO:tasks.workunit.client.1.smithi157.stdout:5/829: truncate d1/d27/fd7 686556 0 2024-04-22T23:35:08.470 INFO:tasks.workunit.client.1.smithi157.stdout:5/830: fsync d1/d27/fdf 0 2024-04-22T23:35:08.470 INFO:tasks.workunit.client.1.smithi157.stdout:8/572: dwrite d2/d6/d21/d5f/d11/f27 [0,4194304] 0 2024-04-22T23:35:08.471 INFO:tasks.workunit.client.1.smithi157.stdout:0/695: sync 2024-04-22T23:35:08.471 INFO:tasks.workunit.client.1.smithi157.stdout:5/831: creat d1/d5/dc/d5d/f138 x:0 0 0 2024-04-22T23:35:08.471 INFO:tasks.workunit.client.1.smithi157.stdout:8/573: creat d2/d4/d73/fc7 x:0 0 0 2024-04-22T23:35:08.471 INFO:tasks.workunit.client.1.smithi157.stdout:0/696: chown d17/d2b/d4f/d5a/d7c/l8a 1069223296 1 2024-04-22T23:35:08.471 INFO:tasks.workunit.client.1.smithi157.stdout:0/697: chown d17/d38/f6d 1 1 2024-04-22T23:35:08.471 INFO:tasks.workunit.client.1.smithi157.stdout:8/574: fdatasync d2/d6/d6b/f8b 0 2024-04-22T23:35:08.471 INFO:tasks.workunit.client.1.smithi157.stdout:4/771: sync 2024-04-22T23:35:08.474 INFO:tasks.workunit.client.1.smithi157.stdout:5/832: symlink d1/d5/d7/d2f/d45/d54/d105/l139 0 2024-04-22T23:35:08.474 INFO:tasks.workunit.client.1.smithi157.stdout:7/721: read d0/d9/d29/d49/d64/fb5 [1594261,85777] 0 2024-04-22T23:35:08.475 INFO:tasks.workunit.client.0.smithi107.stdout:2/609: symlink d1/d67/ld0 0 2024-04-22T23:35:08.475 INFO:tasks.workunit.client.0.smithi107.stdout:4/611: write d1/d21/d29/f71 [1190132,88912] 0 2024-04-22T23:35:08.475 INFO:tasks.workunit.client.0.smithi107.stdout:4/612: fsync d1/f13 0 2024-04-22T23:35:08.476 INFO:tasks.workunit.client.1.smithi157.stdout:0/698: rename d17/d23/d9c/c9a to d17/d2b/d71/ceb 0 2024-04-22T23:35:08.476 INFO:tasks.workunit.client.1.smithi157.stdout:0/699: fdatasync d17/d38/fb9 0 2024-04-22T23:35:08.476 INFO:tasks.workunit.client.1.smithi157.stdout:0/700: chown d17/d2b/l39 17735900 1 2024-04-22T23:35:08.477 INFO:tasks.workunit.client.1.smithi157.stdout:5/833: mknod d1/d5/dc/d43/c13a 0 2024-04-22T23:35:08.477 INFO:tasks.workunit.client.1.smithi157.stdout:5/834: write d1/d5/d7/d2f/d45/da4/fdc [1974525,22437] 0 2024-04-22T23:35:08.478 INFO:tasks.workunit.client.0.smithi107.stdout:3/593: creat db/d21/d27/d36/fd0 x:0 0 0 2024-04-22T23:35:08.478 INFO:tasks.workunit.client.0.smithi107.stdout:7/536: mkdir d7/d21/d48/daf 0 2024-04-22T23:35:08.478 INFO:tasks.workunit.client.1.smithi157.stdout:4/772: symlink d3/d5c/dd3/l101 0 2024-04-22T23:35:08.479 INFO:tasks.workunit.client.1.smithi157.stdout:0/701: rmdir d17/d2b/d4f/d5a/dd2 39 2024-04-22T23:35:08.480 INFO:tasks.workunit.client.1.smithi157.stdout:0/702: truncate d17/d2b/d4f/d9b/fa4 608574 0 2024-04-22T23:35:08.480 INFO:tasks.workunit.client.0.smithi107.stdout:5/601: symlink d9/db/d42/lcd 0 2024-04-22T23:35:08.480 INFO:tasks.workunit.client.0.smithi107.stdout:5/602: chown d9/db/de/c6e 43 1 2024-04-22T23:35:08.481 INFO:tasks.workunit.client.0.smithi107.stdout:1/525: creat d7/d19/d20/d39/d53/d43/fab x:0 0 0 2024-04-22T23:35:08.484 INFO:tasks.workunit.client.1.smithi157.stdout:3/781: dwrite d0/f11 [0,4194304] 0 2024-04-22T23:35:08.484 INFO:tasks.workunit.client.1.smithi157.stdout:3/782: readlink d0/l68 0 2024-04-22T23:35:08.484 INFO:tasks.workunit.client.0.smithi107.stdout:3/594: rmdir db/d21 39 2024-04-22T23:35:08.485 INFO:tasks.workunit.client.0.smithi107.stdout:3/595: fdatasync db/d21/d27/d15/d24/d52/d60/f6b 0 2024-04-22T23:35:08.485 INFO:tasks.workunit.client.1.smithi157.stdout:4/773: mknod d3/d11/d1d/d73/da3/d31/d91/dcb/dcd/c102 0 2024-04-22T23:35:08.486 INFO:tasks.workunit.client.1.smithi157.stdout:3/783: creat d0/d3/d34/d3d/df6/dee/ffc x:0 0 0 2024-04-22T23:35:08.486 INFO:tasks.workunit.client.1.smithi157.stdout:3/784: dread - d0/d3/d34/d3d/df6/dee/d9e/fe2 zero size 2024-04-22T23:35:08.488 INFO:tasks.workunit.client.1.smithi157.stdout:4/774: rmdir d3/d11/d34 39 2024-04-22T23:35:08.488 INFO:tasks.workunit.client.1.smithi157.stdout:4/775: truncate d3/d11/d1d/d73/da7/fe8 401199 0 2024-04-22T23:35:08.489 INFO:tasks.workunit.client.1.smithi157.stdout:3/785: creat d0/d3/d34/d3d/d57/d5c/ffd x:0 0 0 2024-04-22T23:35:08.490 INFO:tasks.workunit.client.0.smithi107.stdout:5/603: creat d9/db/d92/fce x:0 0 0 2024-04-22T23:35:08.491 INFO:tasks.workunit.client.1.smithi157.stdout:8/575: write d2/d6/f46 [3022481,18899] 0 2024-04-22T23:35:08.491 INFO:tasks.workunit.client.1.smithi157.stdout:0/703: dread d17/d2b/f2c [0,4194304] 0 2024-04-22T23:35:08.492 INFO:tasks.workunit.client.0.smithi107.stdout:1/526: mkdir d7/d19/d20/d39/d53/d43/d46/d5c/dac 0 2024-04-22T23:35:08.492 INFO:tasks.workunit.client.1.smithi157.stdout:2/901: dwrite d7/d8/d30/d23/d27/f7b [0,4194304] 0 2024-04-22T23:35:08.492 INFO:tasks.workunit.client.1.smithi157.stdout:3/786: mknod d0/d3/d34/d3d/df6/dee/cfe 0 2024-04-22T23:35:08.492 INFO:tasks.workunit.client.1.smithi157.stdout:2/902: readlink d7/d8/l1d 0 2024-04-22T23:35:08.492 INFO:tasks.workunit.client.1.smithi157.stdout:2/903: write d7/d8/d25/d102/f103 [988122,94954] 0 2024-04-22T23:35:08.493 INFO:tasks.workunit.client.1.smithi157.stdout:7/722: write d0/d9/d42/f8c [3567233,38235] 0 2024-04-22T23:35:08.495 INFO:tasks.workunit.client.0.smithi107.stdout:7/537: link d7/d21/d48/d5c/d9d/c60 d7/d21/d7e/cb0 0 2024-04-22T23:35:08.497 INFO:tasks.workunit.client.1.smithi157.stdout:3/787: creat d0/d3/d34/d3d/d57/d60/d6b/d97/dcb/fff x:0 0 0 2024-04-22T23:35:08.497 INFO:tasks.workunit.client.1.smithi157.stdout:0/704: dread d17/d2b/d52/d7e/fd3 [0,4194304] 0 2024-04-22T23:35:08.498 INFO:tasks.workunit.client.0.smithi107.stdout:5/604: symlink d9/db/de/d5e/lcf 0 2024-04-22T23:35:08.498 INFO:tasks.workunit.client.0.smithi107.stdout:6/614: fsync db/d20/d58/fb9 0 2024-04-22T23:35:08.498 INFO:tasks.workunit.client.0.smithi107.stdout:6/615: creat db/d20/d58/fba x:0 0 0 2024-04-22T23:35:08.498 INFO:tasks.workunit.client.0.smithi107.stdout:6/616: read - db/d20/d2a/d38/d59/d70/f8e zero size 2024-04-22T23:35:08.498 INFO:tasks.workunit.client.0.smithi107.stdout:6/617: fdatasync db/d20/d2a/d5d/fa9 0 2024-04-22T23:35:08.499 INFO:tasks.workunit.client.0.smithi107.stdout:7/538: read d7/d21/da4/f43 [452331,107177] 0 2024-04-22T23:35:08.499 INFO:tasks.workunit.client.1.smithi157.stdout:7/723: creat d0/d9/d10/fd8 x:0 0 0 2024-04-22T23:35:08.500 INFO:tasks.workunit.client.0.smithi107.stdout:1/527: symlink d7/d19/d20/d39/d53/d43/lad 0 2024-04-22T23:35:08.502 INFO:tasks.workunit.client.1.smithi157.stdout:3/788: rename d0/d3/d34/d3d/df6/d72/f79 to d0/d3/d34/d3d/df6/dee/d9e/f100 0 2024-04-22T23:35:08.502 INFO:tasks.workunit.client.1.smithi157.stdout:0/705: getdents d17/d2b/d52/dd1 0 2024-04-22T23:35:08.504 INFO:tasks.workunit.client.1.smithi157.stdout:2/904: mkdir d7/d8/d95/dde/d12b 0 2024-04-22T23:35:08.505 INFO:tasks.workunit.client.0.smithi107.stdout:6/618: mknod db/d20/d2a/d38/d4b/cbb 0 2024-04-22T23:35:08.506 INFO:tasks.workunit.client.0.smithi107.stdout:1/528: symlink d7/d19/d2d/d89/lae 0 2024-04-22T23:35:08.506 INFO:tasks.workunit.client.0.smithi107.stdout:1/529: truncate d7/d19/d20/d39/d53/d43/d46/f47 5033023 0 2024-04-22T23:35:08.507 INFO:tasks.workunit.client.1.smithi157.stdout:7/724: creat d0/d9/d29/d5e/fd9 x:0 0 0 2024-04-22T23:35:08.507 INFO:tasks.workunit.client.1.smithi157.stdout:7/725: read - d0/d9/d29/d98/fc4 zero size 2024-04-22T23:35:08.508 INFO:tasks.workunit.client.1.smithi157.stdout:7/726: fdatasync d0/d9/d42/d55/fb2 0 2024-04-22T23:35:08.508 INFO:tasks.workunit.client.1.smithi157.stdout:7/727: write d0/d9/d29/d5e/d9e/fd4 [8269,41013] 0 2024-04-22T23:35:08.508 INFO:tasks.workunit.client.0.smithi107.stdout:7/539: dread d7/d21/f22 [0,4194304] 0 2024-04-22T23:35:08.509 INFO:tasks.workunit.client.0.smithi107.stdout:5/605: write d9/f60 [3727534,44052] 0 2024-04-22T23:35:08.509 INFO:tasks.workunit.client.0.smithi107.stdout:5/606: creat d9/d76/da0/fd0 x:0 0 0 2024-04-22T23:35:08.510 INFO:tasks.workunit.client.1.smithi157.stdout:8/576: dread d2/d6/d21/d5f/f1b [0,4194304] 0 2024-04-22T23:35:08.513 INFO:tasks.workunit.client.0.smithi107.stdout:4/613: dwrite d1/d21/d29/d2e/d3b/f93 [0,4194304] 0 2024-04-22T23:35:08.513 INFO:tasks.workunit.client.0.smithi107.stdout:3/596: dwrite db/d21/d27/d15/d29/d2a/f5b [4194304,4194304] 0 2024-04-22T23:35:08.513 INFO:tasks.workunit.client.1.smithi157.stdout:8/577: mkdir d2/d6/d21/d5f/d17/dc8 0 2024-04-22T23:35:08.513 INFO:tasks.workunit.client.1.smithi157.stdout:8/578: truncate d2/d6/d21/d5f/d17/f1e 5072053 0 2024-04-22T23:35:08.513 INFO:tasks.workunit.client.0.smithi107.stdout:6/619: dread db/f8a [0,4194304] 0 2024-04-22T23:35:08.517 INFO:tasks.workunit.client.0.smithi107.stdout:2/610: dwrite d1/d2b/d59/f6b [0,4194304] 0 2024-04-22T23:35:08.517 INFO:tasks.workunit.client.0.smithi107.stdout:2/611: write d1/d12/d35/d48/fb3 [595028,36292] 0 2024-04-22T23:35:08.517 INFO:tasks.workunit.client.0.smithi107.stdout:2/612: chown d1/d67/c7d 1324880 1 2024-04-22T23:35:08.517 INFO:tasks.workunit.client.0.smithi107.stdout:1/530: link d7/f1b d7/d19/d20/d39/faf 0 2024-04-22T23:35:08.517 INFO:tasks.workunit.client.1.smithi157.stdout:5/835: dwrite d1/d5/d7/d2f/d9c/fd5 [0,4194304] 0 2024-04-22T23:35:08.517 INFO:tasks.workunit.client.1.smithi157.stdout:4/776: dwrite d3/d4b/d7a/fe4 [4194304,4194304] 0 2024-04-22T23:35:08.517 INFO:tasks.workunit.client.1.smithi157.stdout:4/777: chown d3/d11/d34/da6/le9 13702325 1 2024-04-22T23:35:08.518 INFO:tasks.workunit.client.1.smithi157.stdout:4/778: write d3/d4b/f5 [695514,64014] 0 2024-04-22T23:35:08.518 INFO:tasks.workunit.client.1.smithi157.stdout:4/779: fsync d3/d11/d1d/f52 0 2024-04-22T23:35:08.518 INFO:tasks.workunit.client.1.smithi157.stdout:8/579: symlink d2/d6/lc9 0 2024-04-22T23:35:08.518 INFO:tasks.workunit.client.1.smithi157.stdout:8/580: chown d2/d6/d21/d5f/d11/d13 29468821 1 2024-04-22T23:35:08.518 INFO:tasks.workunit.client.1.smithi157.stdout:2/905: dread d7/d1f/d26/fe1 [0,4194304] 0 2024-04-22T23:35:08.520 INFO:tasks.workunit.client.0.smithi107.stdout:4/614: unlink d1/d21/d29/d2e/d3b/l43 0 2024-04-22T23:35:08.521 INFO:tasks.workunit.client.0.smithi107.stdout:4/615: stat d1/d21/d29/d2e/d88/d99/fb0 0 2024-04-22T23:35:08.521 INFO:tasks.workunit.client.0.smithi107.stdout:4/616: fdatasync d1/d1b/fa9 0 2024-04-22T23:35:08.523 INFO:tasks.workunit.client.0.smithi107.stdout:3/597: creat db/d21/d27/d15/d29/d4f/db2/fd1 x:0 0 0 2024-04-22T23:35:08.524 INFO:tasks.workunit.client.0.smithi107.stdout:7/540: dread d7/d21/f22 [0,4194304] 0 2024-04-22T23:35:08.524 INFO:tasks.workunit.client.1.smithi157.stdout:8/581: link d2/d34/d6a/l9e d2/d6/d21/d5f/d17/dc2/lca 0 2024-04-22T23:35:08.524 INFO:tasks.workunit.client.1.smithi157.stdout:8/582: chown d2/d6/d21/d8d/laa 175 1 2024-04-22T23:35:08.524 INFO:tasks.workunit.client.0.smithi107.stdout:6/620: mkdir db/d20/d2a/d35/d69/db3/dbc 0 2024-04-22T23:35:08.526 INFO:tasks.workunit.client.1.smithi157.stdout:2/906: truncate d7/d1f/d26/d8f/fab 3626051 0 2024-04-22T23:35:08.526 INFO:tasks.workunit.client.1.smithi157.stdout:2/907: stat d7/d1f/f52 0 2024-04-22T23:35:08.526 INFO:tasks.workunit.client.1.smithi157.stdout:2/908: chown d7/d1f/lb9 1162 1 2024-04-22T23:35:08.530 INFO:tasks.workunit.client.1.smithi157.stdout:4/780: symlink d3/d5c/dd3/l103 0 2024-04-22T23:35:08.530 INFO:tasks.workunit.client.1.smithi157.stdout:8/583: symlink d2/d6/d21/d8d/lcb 0 2024-04-22T23:35:08.532 INFO:tasks.workunit.client.0.smithi107.stdout:2/613: creat d1/d2b/fd1 x:0 0 0 2024-04-22T23:35:08.532 INFO:tasks.workunit.client.0.smithi107.stdout:2/614: chown d1/d67/c82 4200 1 2024-04-22T23:35:08.532 INFO:tasks.workunit.client.0.smithi107.stdout:2/615: creat d1/d2b/d59/fd2 x:0 0 0 2024-04-22T23:35:08.533 INFO:tasks.workunit.client.1.smithi157.stdout:4/781: creat d3/d11/d1d/d73/d80/dac/db5/dc8/f104 x:0 0 0 2024-04-22T23:35:08.533 INFO:tasks.workunit.client.1.smithi157.stdout:4/782: truncate d3/d11/d1d/d73/da3/d31/d91/f9b 543606 0 2024-04-22T23:35:08.533 INFO:tasks.workunit.client.1.smithi157.stdout:4/783: write d3/d96/d40/dc3/fdb [139919,24227] 0 2024-04-22T23:35:08.535 INFO:tasks.workunit.client.1.smithi157.stdout:8/584: rename d2/d4/dbd/d97/cac to d2/d6/d6b/ccc 0 2024-04-22T23:35:08.535 INFO:tasks.workunit.client.1.smithi157.stdout:8/585: write d2/db1/f59 [293990,63879] 0 2024-04-22T23:35:08.537 INFO:tasks.workunit.client.0.smithi107.stdout:4/617: dread d1/d21/d52/f73 [0,4194304] 0 2024-04-22T23:35:08.539 INFO:tasks.workunit.client.0.smithi107.stdout:1/531: truncate d7/f1b 521859 0 2024-04-22T23:35:08.539 INFO:tasks.workunit.client.0.smithi107.stdout:1/532: write d7/d19/d20/d39/d53/d43/d46/f4c [560234,97103] 0 2024-04-22T23:35:08.539 INFO:tasks.workunit.client.0.smithi107.stdout:1/533: readlink d7/d19/d20/d39/d53/d43/lad 0 2024-04-22T23:35:08.542 INFO:tasks.workunit.client.1.smithi157.stdout:0/706: dwrite d17/d23/f51 [4194304,4194304] 0 2024-04-22T23:35:08.543 INFO:tasks.workunit.client.0.smithi107.stdout:7/541: mkdir d7/d21/d48/d5c/d9d/db1 0 2024-04-22T23:35:08.543 INFO:tasks.workunit.client.0.smithi107.stdout:7/542: truncate d7/d21/d54/f58 160992 0 2024-04-22T23:35:08.543 INFO:tasks.workunit.client.0.smithi107.stdout:7/543: write d7/d4c/da1/fae [258883,16886] 0 2024-04-22T23:35:08.543 INFO:tasks.workunit.client.0.smithi107.stdout:7/544: write d7/fd [844110,66557] 0 2024-04-22T23:35:08.544 INFO:tasks.workunit.client.0.smithi107.stdout:2/616: symlink d1/d2b/d3d/d57/ld3 0 2024-04-22T23:35:08.544 INFO:tasks.workunit.client.0.smithi107.stdout:2/617: fsync d1/d2b/d49/d7e/fcc 0 2024-04-22T23:35:08.544 INFO:tasks.workunit.client.0.smithi107.stdout:2/618: write d1/d63/fa1 [580958,54566] 0 2024-04-22T23:35:08.544 INFO:tasks.workunit.client.0.smithi107.stdout:2/619: stat d1/d12/d35/d48/lc2 0 2024-04-22T23:35:08.545 INFO:tasks.workunit.client.0.smithi107.stdout:4/618: symlink d1/d7/d63/d78/d96/da0/ld4 0 2024-04-22T23:35:08.545 INFO:tasks.workunit.client.0.smithi107.stdout:4/619: truncate d1/d21/d29/d2e/d3b/f8a 1013326 0 2024-04-22T23:35:08.546 INFO:tasks.workunit.client.0.smithi107.stdout:1/534: creat d7/d19/d2d/da4/fb0 x:0 0 0 2024-04-22T23:35:08.548 INFO:tasks.workunit.client.0.smithi107.stdout:2/620: link d1/f78 d1/d2b/d88/fd4 0 2024-04-22T23:35:08.548 INFO:tasks.workunit.client.0.smithi107.stdout:2/621: chown d1/d12/d35/d54 50 1 2024-04-22T23:35:08.549 INFO:tasks.workunit.client.1.smithi157.stdout:4/784: write d3/d96/f67 [2745843,69657] 0 2024-04-22T23:35:08.549 INFO:tasks.workunit.client.1.smithi157.stdout:4/785: creat d3/d5c/d7b/f105 x:0 0 0 2024-04-22T23:35:08.551 INFO:tasks.workunit.client.0.smithi107.stdout:4/620: link d1/d21/d29/d2e/d88/d99/dd2/daf/dc1/lc4 d1/d7/d53/d66/ld5 0 2024-04-22T23:35:08.552 INFO:tasks.workunit.client.1.smithi157.stdout:4/786: dread d3/d4b/f45 [0,4194304] 0 2024-04-22T23:35:08.553 INFO:tasks.workunit.client.0.smithi107.stdout:7/545: read d7/d21/d48/d5c/f5d [508526,94276] 0 2024-04-22T23:35:08.553 INFO:tasks.workunit.client.0.smithi107.stdout:4/621: creat d1/d7/d53/fd6 x:0 0 0 2024-04-22T23:35:08.554 INFO:tasks.workunit.client.0.smithi107.stdout:7/546: fdatasync d7/d21/d48/f9c 0 2024-04-22T23:35:08.557 INFO:tasks.workunit.client.0.smithi107.stdout:7/547: rename d7/da5 to d7/d21/d48/d5c/d9d/d6e/db2 0 2024-04-22T23:35:08.559 INFO:tasks.workunit.client.0.smithi107.stdout:7/548: truncate d7/f3f 313559 0 2024-04-22T23:35:08.559 INFO:tasks.workunit.client.0.smithi107.stdout:7/549: chown d7/l44 123 1 2024-04-22T23:35:08.560 INFO:tasks.workunit.client.1.smithi157.stdout:3/789: dwrite d0/d3/d34/d3d/f42 [0,4194304] 0 2024-04-22T23:35:08.561 INFO:tasks.workunit.client.0.smithi107.stdout:3/598: dread db/d21/d27/f23 [4194304,4194304] 0 2024-04-22T23:35:08.561 INFO:tasks.workunit.client.1.smithi157.stdout:3/790: creat d0/d3/d34/d3d/f101 x:0 0 0 2024-04-22T23:35:08.562 INFO:tasks.workunit.client.0.smithi107.stdout:3/599: symlink db/ld2 0 2024-04-22T23:35:08.563 INFO:tasks.workunit.client.0.smithi107.stdout:4/622: dread d1/d21/d29/d2e/d3b/f7d [0,4194304] 0 2024-04-22T23:35:08.563 INFO:tasks.workunit.client.0.smithi107.stdout:4/623: creat d1/d21/d29/d2e/d88/fd7 x:0 0 0 2024-04-22T23:35:08.564 INFO:tasks.workunit.client.1.smithi157.stdout:3/791: symlink d0/d3/d34/db9/d9f/l102 0 2024-04-22T23:35:08.565 INFO:tasks.workunit.client.1.smithi157.stdout:3/792: mknod d0/d3/d34/d3d/d57/d60/d6b/c103 0 2024-04-22T23:35:08.566 INFO:tasks.workunit.client.1.smithi157.stdout:3/793: rename d0/d7/l80 to d0/d3/d34/d3d/df6/l104 0 2024-04-22T23:35:08.566 INFO:tasks.workunit.client.1.smithi157.stdout:3/794: fsync d0/f11 0 2024-04-22T23:35:08.567 INFO:tasks.workunit.client.0.smithi107.stdout:3/600: stat db/d21/d27/d54/c4b 0 2024-04-22T23:35:08.569 INFO:tasks.workunit.client.0.smithi107.stdout:4/624: rename d1/l4 to d1/d21/d29/d2e/d88/d99/dd2/ld8 0 2024-04-22T23:35:08.573 INFO:tasks.workunit.client.0.smithi107.stdout:4/625: creat d1/d7/d63/d78/fd9 x:0 0 0 2024-04-22T23:35:08.576 INFO:tasks.workunit.client.1.smithi157.stdout:7/728: dwrite d0/d9/d29/d5e/fd9 [0,4194304] 0 2024-04-22T23:35:08.576 INFO:tasks.workunit.client.0.smithi107.stdout:4/626: unlink d1/d7/d5b/dbf/d27/f30 0 2024-04-22T23:35:08.576 INFO:tasks.workunit.client.0.smithi107.stdout:4/627: chown d1/d1b/d44/cc3 1392 1 2024-04-22T23:35:08.578 INFO:tasks.workunit.client.0.smithi107.stdout:4/628: mknod d1/d7/d5b/dbf/d27/cda 0 2024-04-22T23:35:08.579 INFO:tasks.workunit.client.0.smithi107.stdout:4/629: mkdir d1/d7/d53/d66/d6b/ddb 0 2024-04-22T23:35:08.581 INFO:tasks.workunit.client.1.smithi157.stdout:5/836: dwrite d1/d5/dc/d5d/f138 [0,4194304] 0 2024-04-22T23:35:08.581 INFO:tasks.workunit.client.0.smithi107.stdout:5/607: dwrite d9/f29 [0,4194304] 0 2024-04-22T23:35:08.581 INFO:tasks.workunit.client.0.smithi107.stdout:5/608: write d9/f9d [691616,48762] 0 2024-04-22T23:35:08.582 INFO:tasks.workunit.client.0.smithi107.stdout:5/609: getdents d9/db/dc1/dcb 0 2024-04-22T23:35:08.582 INFO:tasks.workunit.client.1.smithi157.stdout:7/729: mknod d0/d9/d29/d98/cda 0 2024-04-22T23:35:08.582 INFO:tasks.workunit.client.0.smithi107.stdout:4/630: creat d1/d7/d5b/dbf/fdc x:0 0 0 2024-04-22T23:35:08.582 INFO:tasks.workunit.client.0.smithi107.stdout:4/631: write d1/f28 [3863973,46412] 0 2024-04-22T23:35:08.583 INFO:tasks.workunit.client.0.smithi107.stdout:5/610: mknod d9/db/d92/cd1 0 2024-04-22T23:35:08.584 INFO:tasks.workunit.client.0.smithi107.stdout:4/632: creat d1/d7/d63/fdd x:0 0 0 2024-04-22T23:35:08.584 INFO:tasks.workunit.client.1.smithi157.stdout:2/909: dwrite d7/d1f/d26/d4a/fc9 [0,4194304] 0 2024-04-22T23:35:08.585 INFO:tasks.workunit.client.1.smithi157.stdout:2/910: chown d7/d8/d30/d23/d27/d57/d108 816 1 2024-04-22T23:35:08.585 INFO:tasks.workunit.client.1.smithi157.stdout:2/911: dread - d7/d8/d95/dd2/dda/f121 zero size 2024-04-22T23:35:08.585 INFO:tasks.workunit.client.1.smithi157.stdout:2/912: write d7/d8/d30/d23/f110 [963183,82597] 0 2024-04-22T23:35:08.586 INFO:tasks.workunit.client.0.smithi107.stdout:5/611: link d9/db/de/l34 d9/db/de/d72/d7c/ld2 0 2024-04-22T23:35:08.586 INFO:tasks.workunit.client.0.smithi107.stdout:5/612: readlink d9/d26/d79/l9b 0 2024-04-22T23:35:08.586 INFO:tasks.workunit.client.0.smithi107.stdout:5/613: creat d9/db/d94/fd3 x:0 0 0 2024-04-22T23:35:08.586 INFO:tasks.workunit.client.0.smithi107.stdout:5/614: fsync d9/d26/f68 0 2024-04-22T23:35:08.586 INFO:tasks.workunit.client.0.smithi107.stdout:5/615: write f4 [253651,25678] 0 2024-04-22T23:35:08.587 INFO:tasks.workunit.client.1.smithi157.stdout:5/837: symlink d1/db0/dff/d102/l13b 0 2024-04-22T23:35:08.589 INFO:tasks.workunit.client.0.smithi107.stdout:2/622: dwrite d1/d2b/d49/d7e/f8e [0,4194304] 0 2024-04-22T23:35:08.589 INFO:tasks.workunit.client.0.smithi107.stdout:1/535: dwrite d7/d19/d20/d39/d53/f98 [0,4194304] 0 2024-04-22T23:35:08.589 INFO:tasks.workunit.client.1.smithi157.stdout:7/730: rename d0/d9/d42/f7a to d0/d9/db/df/dac/fdb 0 2024-04-22T23:35:08.590 INFO:tasks.workunit.client.1.smithi157.stdout:3/795: dwrite d0/d3/d34/d3d/df6/dee/d9e/fe2 [0,4194304] 0 2024-04-22T23:35:08.592 INFO:tasks.workunit.client.0.smithi107.stdout:5/616: symlink d9/db/de/dbe/ld4 0 2024-04-22T23:35:08.592 INFO:tasks.workunit.client.0.smithi107.stdout:5/617: truncate d9/f12 2402141 0 2024-04-22T23:35:08.593 INFO:tasks.workunit.client.1.smithi157.stdout:7/731: mknod d0/d9/d42/d55/d56/cdc 0 2024-04-22T23:35:08.593 INFO:tasks.workunit.client.1.smithi157.stdout:7/732: write d0/d9/d29/d98/fc4 [722918,34808] 0 2024-04-22T23:35:08.597 INFO:tasks.workunit.client.1.smithi157.stdout:3/796: rename d0/d3/d34/d4a/le3 to d0/d7/d6c/l105 0 2024-04-22T23:35:08.600 INFO:tasks.workunit.client.0.smithi107.stdout:0/560: sync 2024-04-22T23:35:08.600 INFO:tasks.workunit.client.0.smithi107.stdout:9/597: sync 2024-04-22T23:35:08.600 INFO:tasks.workunit.client.0.smithi107.stdout:8/539: sync 2024-04-22T23:35:08.600 INFO:tasks.workunit.client.0.smithi107.stdout:8/540: truncate d0/f72 428350 0 2024-04-22T23:35:08.600 INFO:tasks.workunit.client.0.smithi107.stdout:8/541: creat d0/d7/df/d33/d9b/fb8 x:0 0 0 2024-04-22T23:35:08.600 INFO:tasks.workunit.client.0.smithi107.stdout:8/542: write d0/d7/df/d3b/f3d [3389014,13541] 0 2024-04-22T23:35:08.601 INFO:tasks.workunit.client.1.smithi157.stdout:3/797: symlink d0/d3/d34/d3d/d57/l106 0 2024-04-22T23:35:08.604 INFO:tasks.workunit.client.0.smithi107.stdout:2/623: dread d1/d2b/d49/d7e/fbe [0,4194304] 0 2024-04-22T23:35:08.604 INFO:tasks.workunit.client.0.smithi107.stdout:2/624: stat d1/d67/la7 0 2024-04-22T23:35:08.606 INFO:tasks.workunit.client.0.smithi107.stdout:3/601: dwrite db/d21/d27/d36/fd0 [0,4194304] 0 2024-04-22T23:35:08.607 INFO:tasks.workunit.client.1.smithi157.stdout:3/798: creat d0/d3/d34/d47/f107 x:0 0 0 2024-04-22T23:35:08.608 INFO:tasks.workunit.client.0.smithi107.stdout:1/536: rename d7/d19/d20/d39/d53/d43/d63/f92 to d7/d19/d2d/d89/fb1 0 2024-04-22T23:35:08.611 INFO:tasks.workunit.client.1.smithi157.stdout:3/799: symlink d0/d3/d34/d3d/df6/dee/l108 0 2024-04-22T23:35:08.614 INFO:tasks.workunit.client.1.smithi157.stdout:3/800: mknod d0/d3/d34/d3d/d57/d60/c109 0 2024-04-22T23:35:08.617 INFO:tasks.workunit.client.1.smithi157.stdout:3/801: chown d0/d3/f35 0 1 2024-04-22T23:35:08.618 INFO:tasks.workunit.client.0.smithi107.stdout:7/550: read d7/f3f [59963,12931] 0 2024-04-22T23:35:08.618 INFO:tasks.workunit.client.1.smithi157.stdout:3/802: creat d0/d3/d34/d3d/df6/dee/d9e/dac/f10a x:0 0 0 2024-04-22T23:35:08.618 INFO:tasks.workunit.client.1.smithi157.stdout:3/803: chown d0/d3/d34/db9/fe9 96 1 2024-04-22T23:35:08.618 INFO:tasks.workunit.client.1.smithi157.stdout:3/804: write d0/d3/d34/db9/f3f [2902337,123508] 0 2024-04-22T23:35:08.619 INFO:tasks.workunit.client.0.smithi107.stdout:0/561: mkdir d1/d27/d20/d2c/d41/dbe 0 2024-04-22T23:35:08.621 INFO:tasks.workunit.client.0.smithi107.stdout:8/543: mknod d0/d7/d87/cb9 0 2024-04-22T23:35:08.621 INFO:tasks.workunit.client.1.smithi157.stdout:2/913: dwrite d7/d8/d25/d102/ddb/f10b [0,4194304] 0 2024-04-22T23:35:08.622 INFO:tasks.workunit.client.1.smithi157.stdout:2/914: write d7/d8/d30/fc1 [217497,37261] 0 2024-04-22T23:35:08.622 INFO:tasks.workunit.client.1.smithi157.stdout:2/915: truncate d7/d1f/d26/d4a/fff 885659 0 2024-04-22T23:35:08.624 INFO:tasks.workunit.client.0.smithi107.stdout:2/625: unlink d1/d12/d35/d48/d13/fb0 0 2024-04-22T23:35:08.630 INFO:tasks.workunit.client.0.smithi107.stdout:1/537: creat d7/d19/d54/fb2 x:0 0 0 2024-04-22T23:35:08.631 INFO:tasks.workunit.client.0.smithi107.stdout:1/538: chown d7/d19/d20/f38 2474 1 2024-04-22T23:35:08.631 INFO:tasks.workunit.client.0.smithi107.stdout:7/551: dread d7/fd [0,4194304] 0 2024-04-22T23:35:08.634 INFO:tasks.workunit.client.0.smithi107.stdout:0/562: creat d1/d7b/fbf x:0 0 0 2024-04-22T23:35:08.634 INFO:tasks.workunit.client.0.smithi107.stdout:8/544: mknod d0/d7/d8/d41/d53/d76/cba 0 2024-04-22T23:35:08.636 INFO:tasks.workunit.client.0.smithi107.stdout:7/552: unlink d7/f18 0 2024-04-22T23:35:08.638 INFO:tasks.workunit.client.0.smithi107.stdout:0/563: symlink d1/d9/d14/d4f/da6/lc0 0 2024-04-22T23:35:08.640 INFO:tasks.workunit.client.0.smithi107.stdout:7/553: rename d7/d21/d48/d5c/f5d to d7/d21/d7e/fb3 0 2024-04-22T23:35:08.641 INFO:tasks.workunit.client.0.smithi107.stdout:0/564: mknod d1/d27/d20/d40/dbd/cc1 0 2024-04-22T23:35:08.644 INFO:tasks.workunit.client.0.smithi107.stdout:0/565: creat d1/d84/fc2 x:0 0 0 2024-04-22T23:35:08.646 INFO:tasks.workunit.client.0.smithi107.stdout:7/554: mkdir d7/db4 0 2024-04-22T23:35:08.646 INFO:tasks.workunit.client.0.smithi107.stdout:7/555: truncate d7/d21/d48/d5c/f7a 481806 0 2024-04-22T23:35:08.647 INFO:tasks.workunit.client.0.smithi107.stdout:7/556: read d7/d4c/da1/fae [206274,73196] 0 2024-04-22T23:35:08.647 INFO:tasks.workunit.client.0.smithi107.stdout:0/566: mknod d1/d7b/cc3 0 2024-04-22T23:35:08.647 INFO:tasks.workunit.client.0.smithi107.stdout:0/567: fdatasync d1/d27/d20/f6b 0 2024-04-22T23:35:08.664 INFO:tasks.workunit.client.1.smithi157.stdout:5/838: dwrite d1/d5/f10f [0,4194304] 0 2024-04-22T23:35:08.667 INFO:tasks.workunit.client.1.smithi157.stdout:5/839: unlink d1/d27/d29/l33 0 2024-04-22T23:35:08.673 INFO:tasks.workunit.client.1.smithi157.stdout:5/840: dread d1/d5/d7/d2f/d45/d54/f65 [0,4194304] 0 2024-04-22T23:35:08.674 INFO:tasks.workunit.client.1.smithi157.stdout:5/841: mknod d1/d5/c13c 0 2024-04-22T23:35:08.674 INFO:tasks.workunit.client.1.smithi157.stdout:5/842: fsync d1/d27/d29/d4b/f63 0 2024-04-22T23:35:08.703 INFO:tasks.workunit.client.0.smithi107.stdout:9/598: dwrite d5/d29/d7b/d91/dc0/fc2 [0,4194304] 0 2024-04-22T23:35:08.704 INFO:tasks.workunit.client.0.smithi107.stdout:4/633: dwrite d1/d21/d29/d2e/d88/fd7 [0,4194304] 0 2024-04-22T23:35:08.704 INFO:tasks.workunit.client.0.smithi107.stdout:4/634: fsync d1/d21/f36 0 2024-04-22T23:35:08.705 INFO:tasks.workunit.client.0.smithi107.stdout:9/599: getdents d5/d7/d27/d69/db6 0 2024-04-22T23:35:08.705 INFO:tasks.workunit.client.0.smithi107.stdout:5/618: dwrite f3 [4194304,4194304] 0 2024-04-22T23:35:08.705 INFO:tasks.workunit.client.1.smithi157.stdout:3/805: dwrite d0/d3/d34/d4a/fd7 [0,4194304] 0 2024-04-22T23:35:08.706 INFO:tasks.workunit.client.0.smithi107.stdout:1/539: dwrite d7/d19/d20/f41 [4194304,4194304] 0 2024-04-22T23:35:08.706 INFO:tasks.workunit.client.0.smithi107.stdout:3/602: dwrite db/d21/d27/d15/d24/d66/fc6 [0,4194304] 0 2024-04-22T23:35:08.707 INFO:tasks.workunit.client.1.smithi157.stdout:2/916: dwrite d7/d8/dd/d4d/d11d/f104 [0,4194304] 0 2024-04-22T23:35:08.707 INFO:tasks.workunit.client.1.smithi157.stdout:7/733: dwrite d0/d9/d72/fc0 [0,4194304] 0 2024-04-22T23:35:08.707 INFO:tasks.workunit.client.1.smithi157.stdout:7/734: chown d0/d9/d10/l83 754354149 1 2024-04-22T23:35:08.707 INFO:tasks.workunit.client.0.smithi107.stdout:8/545: dwrite d0/d7/fa [0,4194304] 0 2024-04-22T23:35:08.707 INFO:tasks.workunit.client.0.smithi107.stdout:8/546: getdents d0/d7/d20/d89 0 2024-04-22T23:35:08.707 INFO:tasks.workunit.client.0.smithi107.stdout:8/547: truncate d0/d3a/d46/f70 843146 0 2024-04-22T23:35:08.708 INFO:tasks.workunit.client.0.smithi107.stdout:7/557: dwrite d7/d21/d48/d5c/d9d/f62 [0,4194304] 0 2024-04-22T23:35:08.708 INFO:tasks.workunit.client.0.smithi107.stdout:2/626: dwrite d1/f1a [4194304,4194304] 0 2024-04-22T23:35:08.708 INFO:tasks.workunit.client.0.smithi107.stdout:4/635: mknod d1/d7/d5b/dbf/d27/cde 0 2024-04-22T23:35:08.708 INFO:tasks.workunit.client.0.smithi107.stdout:4/636: dread - d1/d7/d5b/dbf/fdc zero size 2024-04-22T23:35:08.709 INFO:tasks.workunit.client.0.smithi107.stdout:2/627: write d1/d63/f65 [1149407,77700] 0 2024-04-22T23:35:08.710 INFO:tasks.workunit.client.0.smithi107.stdout:9/600: creat d5/d14/d2e/d43/dc9/daf/fd0 x:0 0 0 2024-04-22T23:35:08.710 INFO:tasks.workunit.client.0.smithi107.stdout:9/601: creat d5/d29/d7b/d91/dc0/fd1 x:0 0 0 2024-04-22T23:35:08.712 INFO:tasks.workunit.client.0.smithi107.stdout:5/619: getdents d9/db/d42/d75/db6 0 2024-04-22T23:35:08.718 INFO:tasks.workunit.client.0.smithi107.stdout:8/548: creat d0/d7/d20/da2/da3/fbb x:0 0 0 2024-04-22T23:35:08.720 INFO:tasks.workunit.client.0.smithi107.stdout:3/603: dread db/d21/d27/d15/d24/d87/f89 [0,4194304] 0 2024-04-22T23:35:08.727 INFO:tasks.workunit.client.1.smithi157.stdout:2/917: truncate d7/d1f/d3e/f96 1133168 0 2024-04-22T23:35:08.727 INFO:tasks.workunit.client.1.smithi157.stdout:2/918: write d7/d9f/f12a [804410,72381] 0 2024-04-22T23:35:08.727 INFO:tasks.workunit.client.1.smithi157.stdout:3/806: dwrite d0/d3/d34/d39/ff4 [0,4194304] 0 2024-04-22T23:35:08.728 INFO:tasks.workunit.client.0.smithi107.stdout:1/540: dwrite d7/d19/d20/d39/d93/f9e [0,4194304] 0 2024-04-22T23:35:08.728 INFO:tasks.workunit.client.0.smithi107.stdout:1/541: truncate d7/d19/d20/d39/d53/d43/d46/d5c/f8a 831737 0 2024-04-22T23:35:08.728 INFO:tasks.workunit.client.0.smithi107.stdout:1/542: stat d7/d19/d20/f38 0 2024-04-22T23:35:08.728 INFO:tasks.workunit.client.1.smithi157.stdout:7/735: symlink d0/d9/d42/d55/d56/d91/ldd 0 2024-04-22T23:35:08.728 INFO:tasks.workunit.client.1.smithi157.stdout:2/919: mknod d7/d1f/d3e/c12c 0 2024-04-22T23:35:08.728 INFO:tasks.workunit.client.1.smithi157.stdout:2/920: write d7/d8/d25/d102/fc4 [536758,88693] 0 2024-04-22T23:35:08.730 INFO:tasks.workunit.client.0.smithi107.stdout:7/558: truncate d7/d21/f6d 9621280 0 2024-04-22T23:35:08.730 INFO:tasks.workunit.client.0.smithi107.stdout:7/559: chown d7/d21/d48/d5c/d9d/l65 164 1 2024-04-22T23:35:08.730 INFO:tasks.workunit.client.0.smithi107.stdout:7/560: truncate d7/d21/d49/f86 993356 0 2024-04-22T23:35:08.732 INFO:tasks.workunit.client.0.smithi107.stdout:4/637: rename d1/f75 to d1/d7/d53/d66/d6b/fdf 0 2024-04-22T23:35:08.733 INFO:tasks.workunit.client.1.smithi157.stdout:7/736: rmdir d0/d9/d10 39 2024-04-22T23:35:08.734 INFO:tasks.workunit.client.1.smithi157.stdout:3/807: dread d0/d3/d34/d39/f96 [0,4194304] 0 2024-04-22T23:35:08.735 INFO:tasks.workunit.client.1.smithi157.stdout:2/921: mkdir d7/d8/d30/d12d 0 2024-04-22T23:35:08.735 INFO:tasks.workunit.client.1.smithi157.stdout:2/922: write d7/d1f/d3e/f5e [1536939,90897] 0 2024-04-22T23:35:08.735 INFO:tasks.workunit.client.0.smithi107.stdout:2/628: creat d1/d12/d35/d54/fd5 x:0 0 0 2024-04-22T23:35:08.735 INFO:tasks.workunit.client.0.smithi107.stdout:2/629: chown d1/d2b/d3d/d57/fa0 36748 1 2024-04-22T23:35:08.736 INFO:tasks.workunit.client.1.smithi157.stdout:7/737: mkdir d0/d9/d29/dde 0 2024-04-22T23:35:08.736 INFO:tasks.workunit.client.1.smithi157.stdout:3/808: truncate d0/d3/d34/fe4 4010510 0 2024-04-22T23:35:08.737 INFO:tasks.workunit.client.0.smithi107.stdout:5/620: mknod d9/d76/cd5 0 2024-04-22T23:35:08.737 INFO:tasks.workunit.client.0.smithi107.stdout:5/621: stat d9/db/d42/f6c 0 2024-04-22T23:35:08.737 INFO:tasks.workunit.client.0.smithi107.stdout:5/622: write d9/d76/da0/fa9 [177897,102528] 0 2024-04-22T23:35:08.738 INFO:tasks.workunit.client.1.smithi157.stdout:7/738: symlink d0/d9/db/da3/ldf 0 2024-04-22T23:35:08.738 INFO:tasks.workunit.client.1.smithi157.stdout:7/739: chown d0/d9/d29/d5e/d85/l8d 2530 1 2024-04-22T23:35:08.739 INFO:tasks.workunit.client.0.smithi107.stdout:8/549: creat d0/d73/db0/fbc x:0 0 0 2024-04-22T23:35:08.739 INFO:tasks.workunit.client.0.smithi107.stdout:8/550: truncate d0/f96 234523 0 2024-04-22T23:35:08.739 INFO:tasks.workunit.client.0.smithi107.stdout:3/604: stat db/d21/d27/l2c 0 2024-04-22T23:35:08.739 INFO:tasks.workunit.client.0.smithi107.stdout:3/605: fsync db/d21/d27/d15/d24/d52/d60/f93 0 2024-04-22T23:35:08.740 INFO:tasks.workunit.client.1.smithi157.stdout:3/809: creat d0/d3/d34/d3d/df6/d72/f10b x:0 0 0 2024-04-22T23:35:08.740 INFO:tasks.workunit.client.1.smithi157.stdout:9/766: sync 2024-04-22T23:35:08.740 INFO:tasks.workunit.client.1.smithi157.stdout:9/767: stat d4/d15/d23/f97 0 2024-04-22T23:35:08.741 INFO:tasks.workunit.client.1.smithi157.stdout:3/810: creat d0/d83/f10c x:0 0 0 2024-04-22T23:35:08.741 INFO:tasks.workunit.client.1.smithi157.stdout:1/853: sync 2024-04-22T23:35:08.741 INFO:tasks.workunit.client.1.smithi157.stdout:1/854: creat d4/d17/d34/d75/f12b x:0 0 0 2024-04-22T23:35:08.741 INFO:tasks.workunit.client.1.smithi157.stdout:1/855: creat d4/db/d71/f12c x:0 0 0 2024-04-22T23:35:08.742 INFO:tasks.workunit.client.0.smithi107.stdout:0/568: getdents d1/d7b 0 2024-04-22T23:35:08.744 INFO:tasks.workunit.client.0.smithi107.stdout:1/543: creat d7/d19/d20/d39/d53/d43/d46/d5c/fb3 x:0 0 0 2024-04-22T23:35:08.744 INFO:tasks.workunit.client.1.smithi157.stdout:3/811: mknod d0/d3/c10d 0 2024-04-22T23:35:08.746 INFO:tasks.workunit.client.1.smithi157.stdout:2/923: dread d7/d1f/d26/d4a/f60 [4194304,4194304] 0 2024-04-22T23:35:08.746 INFO:tasks.workunit.client.0.smithi107.stdout:7/561: mknod d7/d21/cb5 0 2024-04-22T23:35:08.747 INFO:tasks.workunit.client.0.smithi107.stdout:7/562: stat d7/d1b/f4d 0 2024-04-22T23:35:08.747 INFO:tasks.workunit.client.0.smithi107.stdout:7/563: fdatasync d7/d21/d54/f83 0 2024-04-22T23:35:08.751 INFO:tasks.workunit.client.1.smithi157.stdout:1/856: read d4/db/d3d/fa1 [277744,105340] 0 2024-04-22T23:35:08.754 INFO:tasks.workunit.client.0.smithi107.stdout:2/630: symlink d1/d2b/d88/ld6 0 2024-04-22T23:35:08.754 INFO:tasks.workunit.client.0.smithi107.stdout:2/631: truncate d1/fad 205306 0 2024-04-22T23:35:08.754 INFO:tasks.workunit.client.0.smithi107.stdout:2/632: write d1/f1a [3472575,43298] 0 2024-04-22T23:35:08.756 INFO:tasks.workunit.client.1.smithi157.stdout:3/812: write d0/d3/d34/d47/fad [765549,48595] 0 2024-04-22T23:35:08.757 INFO:tasks.workunit.client.0.smithi107.stdout:8/551: creat d0/d7/d20/fbd x:0 0 0 2024-04-22T23:35:08.757 INFO:tasks.workunit.client.0.smithi107.stdout:8/552: fsync d0/d7/df/d33/d37/f49 0 2024-04-22T23:35:08.757 INFO:tasks.workunit.client.0.smithi107.stdout:0/569: mknod d1/d27/d20/d40/db4/cc4 0 2024-04-22T23:35:08.759 INFO:tasks.workunit.client.1.smithi157.stdout:3/813: creat d0/d83/dbc/f10e x:0 0 0 2024-04-22T23:35:08.759 INFO:tasks.workunit.client.1.smithi157.stdout:3/814: stat d0/d3/d34/d3d/f42 0 2024-04-22T23:35:08.759 INFO:tasks.workunit.client.0.smithi107.stdout:4/638: getdents d1/d1b 0 2024-04-22T23:35:08.760 INFO:tasks.workunit.client.1.smithi157.stdout:3/815: rmdir d0/d3/d34/d3d/d57/d60 39 2024-04-22T23:35:08.763 INFO:tasks.workunit.client.0.smithi107.stdout:2/633: truncate d1/d2b/d3d/d57/f97 59942 0 2024-04-22T23:35:08.763 INFO:tasks.workunit.client.0.smithi107.stdout:2/634: readlink d1/d12/d35/d54/l72 0 2024-04-22T23:35:08.765 INFO:tasks.workunit.client.1.smithi157.stdout:9/768: dwrite d4/d15/d23/f88 [0,4194304] 0 2024-04-22T23:35:08.766 INFO:tasks.workunit.client.1.smithi157.stdout:2/924: write d7/d8/d30/d23/d27/f49 [2542757,17783] 0 2024-04-22T23:35:08.768 INFO:tasks.workunit.client.1.smithi157.stdout:9/769: link d4/f8 d4/d15/d23/d27/d47/db3/fec 0 2024-04-22T23:35:08.768 INFO:tasks.workunit.client.1.smithi157.stdout:9/770: chown d4/d15/d23/d27/c5f 4166 1 2024-04-22T23:35:08.768 INFO:tasks.workunit.client.1.smithi157.stdout:9/771: chown d4/d2d/fca 837801 1 2024-04-22T23:35:08.769 INFO:tasks.workunit.client.0.smithi107.stdout:8/553: unlink d0/d7/df/d33/l4f 0 2024-04-22T23:35:08.769 INFO:tasks.workunit.client.0.smithi107.stdout:8/554: write d0/d7/df/d17/f5b [680968,32153] 0 2024-04-22T23:35:08.769 INFO:tasks.workunit.client.0.smithi107.stdout:8/555: chown d0/d7/d20/d89 4719743 1 2024-04-22T23:35:08.770 INFO:tasks.workunit.client.0.smithi107.stdout:0/570: mknod d1/d84/d99/cc5 0 2024-04-22T23:35:08.770 INFO:tasks.workunit.client.1.smithi157.stdout:7/740: dwrite d0/d9/d42/fcc [4194304,4194304] 0 2024-04-22T23:35:08.770 INFO:tasks.workunit.client.1.smithi157.stdout:2/925: creat d7/d8/d30/d23/d27/d57/dcf/dfa/f12e x:0 0 0 2024-04-22T23:35:08.770 INFO:tasks.workunit.client.1.smithi157.stdout:9/772: rmdir d4/df/d72 39 2024-04-22T23:35:08.771 INFO:tasks.workunit.client.0.smithi107.stdout:8/556: symlink d0/d3a/lbe 0 2024-04-22T23:35:08.771 INFO:tasks.workunit.client.0.smithi107.stdout:8/557: write d0/d7/df/d3b/f3d [986720,6267] 0 2024-04-22T23:35:08.772 INFO:tasks.workunit.client.0.smithi107.stdout:8/558: write d0/d7/df/d17/f5b [351316,84896] 0 2024-04-22T23:35:08.772 INFO:tasks.workunit.client.0.smithi107.stdout:8/559: write d0/d73/db0/fbc [436078,88068] 0 2024-04-22T23:35:08.772 INFO:tasks.workunit.client.0.smithi107.stdout:8/560: truncate d0/d7/df/d33/d9b/fb8 706570 0 2024-04-22T23:35:08.772 INFO:tasks.workunit.client.0.smithi107.stdout:0/571: rmdir d1/d9/d14/d4f/d59 39 2024-04-22T23:35:08.772 INFO:tasks.workunit.client.0.smithi107.stdout:0/572: fsync d1/d27/f38 0 2024-04-22T23:35:08.772 INFO:tasks.workunit.client.0.smithi107.stdout:0/573: dread - d1/d2a/dad/fbc zero size 2024-04-22T23:35:08.774 INFO:tasks.workunit.client.1.smithi157.stdout:9/773: truncate d4/df/f6a 1901012 0 2024-04-22T23:35:08.775 INFO:tasks.workunit.client.1.smithi157.stdout:2/926: getdents d7/d8/d25/d102/ddb 0 2024-04-22T23:35:08.775 INFO:tasks.workunit.client.1.smithi157.stdout:2/927: creat d7/d8/d30/d23/d27/d57/d108/f12f x:0 0 0 2024-04-22T23:35:08.775 INFO:tasks.workunit.client.1.smithi157.stdout:2/928: write d7/d1f/d26/d4a/f11b [5141123,28966] 0 2024-04-22T23:35:08.776 INFO:tasks.workunit.client.1.smithi157.stdout:7/741: truncate d0/d9/f66 2250379 0 2024-04-22T23:35:08.776 INFO:tasks.workunit.client.1.smithi157.stdout:7/742: stat d0/d9/db/dc3 0 2024-04-22T23:35:08.777 INFO:tasks.workunit.client.0.smithi107.stdout:3/606: dwrite db/ff [0,4194304] 0 2024-04-22T23:35:08.777 INFO:tasks.workunit.client.0.smithi107.stdout:8/561: truncate d0/d3a/d46/f8a 1981536 0 2024-04-22T23:35:08.779 INFO:tasks.workunit.client.1.smithi157.stdout:9/774: creat d4/df/d57/dd0/fed x:0 0 0 2024-04-22T23:35:08.779 INFO:tasks.workunit.client.1.smithi157.stdout:9/775: chown d4/d15/d23/dad 185310739 1 2024-04-22T23:35:08.782 INFO:tasks.workunit.client.1.smithi157.stdout:7/743: creat d0/d9/d42/d55/d80/d82/fe0 x:0 0 0 2024-04-22T23:35:08.785 INFO:tasks.workunit.client.1.smithi157.stdout:7/744: write d0/d9/d29/f3b [954866,57605] 0 2024-04-22T23:35:08.786 INFO:tasks.workunit.client.0.smithi107.stdout:6/621: sync 2024-04-22T23:35:08.786 INFO:tasks.workunit.client.0.smithi107.stdout:6/622: chown db/d20/d2a/d38/d59/d70/f8f 31850 1 2024-04-22T23:35:08.788 INFO:tasks.workunit.client.0.smithi107.stdout:3/607: truncate db/d21/d27/d15/d24/d87/f89 3929967 0 2024-04-22T23:35:08.788 INFO:tasks.workunit.client.1.smithi157.stdout:3/816: dwrite d0/d3/d13/ff8 [0,4194304] 0 2024-04-22T23:35:08.788 INFO:tasks.workunit.client.1.smithi157.stdout:3/817: write d0/d3/d34/d39/f76 [294583,91364] 0 2024-04-22T23:35:08.791 INFO:tasks.workunit.client.0.smithi107.stdout:8/562: mknod d0/d7/d8/d41/d53/cbf 0 2024-04-22T23:35:08.791 INFO:tasks.workunit.client.0.smithi107.stdout:8/563: chown d0/d3a/d46/d59/f68 19694102 1 2024-04-22T23:35:08.791 INFO:tasks.workunit.client.0.smithi107.stdout:8/564: write d0/d7/d8/d41/d53/f77 [203781,36970] 0 2024-04-22T23:35:08.793 INFO:tasks.workunit.client.0.smithi107.stdout:5/623: rmdir d9/d76 39 2024-04-22T23:35:08.793 INFO:tasks.workunit.client.0.smithi107.stdout:5/624: write d9/fd [629651,429] 0 2024-04-22T23:35:08.793 INFO:tasks.workunit.client.0.smithi107.stdout:5/625: readlink d9/db/l77 0 2024-04-22T23:35:08.793 INFO:tasks.workunit.client.1.smithi157.stdout:3/818: rename d0/d3/d34/d3d/d57/d60/d6b/d97/dcb/ld9 to d0/d3/d34/d3d/d57/d60/d6b/l10f 0 2024-04-22T23:35:08.795 INFO:tasks.workunit.client.0.smithi107.stdout:8/565: creat d0/d3a/d46/d59/d78/fc0 x:0 0 0 2024-04-22T23:35:08.795 INFO:tasks.workunit.client.0.smithi107.stdout:8/566: write d0/d7/d8/d41/f9a [52937,65431] 0 2024-04-22T23:35:08.796 INFO:tasks.workunit.client.1.smithi157.stdout:3/819: creat d0/d3/d34/d47/dcd/f110 x:0 0 0 2024-04-22T23:35:08.796 INFO:tasks.workunit.client.1.smithi157.stdout:3/820: chown d0/d3/d34/d3d/d57/d60/c75 1 1 2024-04-22T23:35:08.799 INFO:tasks.workunit.client.0.smithi107.stdout:1/544: dwrite d7/d19/d20/d39/d53/f33 [0,4194304] 0 2024-04-22T23:35:08.799 INFO:tasks.workunit.client.0.smithi107.stdout:1/545: fsync d7/d19/d2d/f84 0 2024-04-22T23:35:08.800 INFO:tasks.workunit.client.0.smithi107.stdout:6/623: dread db/f15 [0,4194304] 0 2024-04-22T23:35:08.800 INFO:tasks.workunit.client.0.smithi107.stdout:3/608: dread db/d21/d27/d15/d24/d52/fbd [0,4194304] 0 2024-04-22T23:35:08.800 INFO:tasks.workunit.client.0.smithi107.stdout:3/609: chown db/d21/d27/d54/d25/f34 82485 1 2024-04-22T23:35:08.800 INFO:tasks.workunit.client.1.smithi157.stdout:4/787: sync 2024-04-22T23:35:08.800 INFO:tasks.workunit.client.1.smithi157.stdout:4/788: chown d3/d96/d4c/fd4 182 1 2024-04-22T23:35:08.800 INFO:tasks.workunit.client.1.smithi157.stdout:0/707: sync 2024-04-22T23:35:08.800 INFO:tasks.workunit.client.1.smithi157.stdout:8/586: sync 2024-04-22T23:35:08.801 INFO:tasks.workunit.client.1.smithi157.stdout:5/843: sync 2024-04-22T23:35:08.801 INFO:tasks.workunit.client.1.smithi157.stdout:5/844: write d1/d5/d7/d2f/d45/d54/d59/f129 [647168,10207] 0 2024-04-22T23:35:08.801 INFO:tasks.workunit.client.0.smithi107.stdout:8/567: creat d0/d73/fc1 x:0 0 0 2024-04-22T23:35:08.801 INFO:tasks.workunit.client.0.smithi107.stdout:8/568: read - d0/d7/d20/fbd zero size 2024-04-22T23:35:08.801 INFO:tasks.workunit.client.0.smithi107.stdout:8/569: fsync d0/d7/df/d33/d37/f49 0 2024-04-22T23:35:08.801 INFO:tasks.workunit.client.0.smithi107.stdout:8/570: truncate d0/d7/d8/f6c 4260078 0 2024-04-22T23:35:08.804 INFO:tasks.workunit.client.1.smithi157.stdout:0/708: symlink d17/d2b/d4f/d5a/dd2/de4/lec 0 2024-04-22T23:35:08.805 INFO:tasks.workunit.client.1.smithi157.stdout:0/709: symlink d17/led 0 2024-04-22T23:35:08.805 INFO:tasks.workunit.client.1.smithi157.stdout:0/710: getdents d17/d2b 0 2024-04-22T23:35:08.805 INFO:tasks.workunit.client.1.smithi157.stdout:0/711: dread - d17/d23/d9c/fdd zero size 2024-04-22T23:35:08.805 INFO:tasks.workunit.client.1.smithi157.stdout:0/712: getdents d17/d23/d34/d67/d7f/d69/dce/de3 0 2024-04-22T23:35:08.807 INFO:tasks.workunit.client.0.smithi107.stdout:6/624: mknod db/cbd 0 2024-04-22T23:35:08.808 INFO:tasks.workunit.client.0.smithi107.stdout:6/625: fdatasync f0 0 2024-04-22T23:35:08.809 INFO:tasks.workunit.client.1.smithi157.stdout:5/845: dread d1/d5/d7/dbd/ff4 [0,4194304] 0 2024-04-22T23:35:08.811 INFO:tasks.workunit.client.0.smithi107.stdout:1/546: mknod d7/cb4 0 2024-04-22T23:35:08.811 INFO:tasks.workunit.client.0.smithi107.stdout:1/547: creat d7/d19/d2d/d89/fb5 x:0 0 0 2024-04-22T23:35:08.811 INFO:tasks.workunit.client.0.smithi107.stdout:1/548: dread - d7/d19/d20/d39/d53/d43/fab zero size 2024-04-22T23:35:08.812 INFO:tasks.workunit.client.1.smithi157.stdout:5/846: dread d1/d5/d7/d2f/d45/d54/f74 [0,4194304] 0 2024-04-22T23:35:08.812 INFO:tasks.workunit.client.1.smithi157.stdout:2/929: rmdir d7/d8/d30/d23 39 2024-04-22T23:35:08.813 INFO:tasks.workunit.client.0.smithi107.stdout:7/564: dwrite d7/d21/f22 [0,4194304] 0 2024-04-22T23:35:08.813 INFO:tasks.workunit.client.0.smithi107.stdout:6/626: creat db/d20/db1/fbe x:0 0 0 2024-04-22T23:35:08.814 INFO:tasks.workunit.client.0.smithi107.stdout:4/639: dwrite d1/d1b/f81 [0,4194304] 0 2024-04-22T23:35:08.814 INFO:tasks.workunit.client.0.smithi107.stdout:4/640: truncate d1/d7/d63/d78/d96/da0/fd3 622511 0 2024-04-22T23:35:08.814 INFO:tasks.workunit.client.0.smithi107.stdout:5/626: write d9/f29 [780809,78043] 0 2024-04-22T23:35:08.814 INFO:tasks.workunit.client.0.smithi107.stdout:3/610: write db/ff [134554,124339] 0 2024-04-22T23:35:08.815 INFO:tasks.workunit.client.0.smithi107.stdout:3/611: truncate db/d21/d27/d15/d29/d2a/f4c 767193 0 2024-04-22T23:35:08.815 INFO:tasks.workunit.client.1.smithi157.stdout:2/930: write d7/d8/fa4 [458740,43647] 0 2024-04-22T23:35:08.817 INFO:tasks.workunit.client.0.smithi107.stdout:1/549: rename d7/d19/d54/caa to d7/d19/d2d/da4/cb6 0 2024-04-22T23:35:08.818 INFO:tasks.workunit.client.1.smithi157.stdout:2/931: rmdir d7/d8/d95/dde/d12b 0 2024-04-22T23:35:08.819 INFO:tasks.workunit.client.1.smithi157.stdout:9/776: dwrite d4/d15/d23/f8f [0,4194304] 0 2024-04-22T23:35:08.822 INFO:tasks.workunit.client.0.smithi107.stdout:7/565: creat d7/d21/d48/fb6 x:0 0 0 2024-04-22T23:35:08.822 INFO:tasks.workunit.client.1.smithi157.stdout:5/847: write d1/d5/d7/d2f/d9c/faa [1973192,38613] 0 2024-04-22T23:35:08.822 INFO:tasks.workunit.client.1.smithi157.stdout:5/848: dread - d1/db0/d7a/fa3 zero size 2024-04-22T23:35:08.823 INFO:tasks.workunit.client.1.smithi157.stdout:7/745: dwrite d0/d9/d29/f4d [4194304,4194304] 0 2024-04-22T23:35:08.826 INFO:tasks.workunit.client.0.smithi107.stdout:7/566: dread d7/d1b/f68 [0,4194304] 0 2024-04-22T23:35:08.826 INFO:tasks.workunit.client.0.smithi107.stdout:6/627: link db/d20/d2a/d5f/c83 db/d20/d58/cbf 0 2024-04-22T23:35:08.831 INFO:tasks.workunit.client.1.smithi157.stdout:5/849: write d1/d27/d29/f2a [2414295,49492] 0 2024-04-22T23:35:08.831 INFO:tasks.workunit.client.0.smithi107.stdout:5/627: creat d9/d76/fd6 x:0 0 0 2024-04-22T23:35:08.832 INFO:tasks.workunit.client.1.smithi157.stdout:4/789: dwrite d3/d11/d1d/d73/d80/dac/db5/f94 [0,4194304] 0 2024-04-22T23:35:08.832 INFO:tasks.workunit.client.0.smithi107.stdout:0/574: dwrite d1/d84/fc2 [0,4194304] 0 2024-04-22T23:35:08.834 INFO:tasks.workunit.client.0.smithi107.stdout:1/550: mkdir d7/d19/d20/d39/d53/d43/d46/d5c/db7 0 2024-04-22T23:35:08.834 INFO:tasks.workunit.client.0.smithi107.stdout:1/551: write d7/d19/d20/d39/d53/d43/d46/d5c/f8a [973999,26053] 0 2024-04-22T23:35:08.834 INFO:tasks.workunit.client.1.smithi157.stdout:5/850: rename d1/d5/d7/d2f/d9c/lca to d1/db0/dff/l13d 0 2024-04-22T23:35:08.835 INFO:tasks.workunit.client.1.smithi157.stdout:8/587: dwrite d2/d4/dbd/fb4 [0,4194304] 0 2024-04-22T23:35:08.837 INFO:tasks.workunit.client.0.smithi107.stdout:7/567: mknod d7/d4c/da1/cb7 0 2024-04-22T23:35:08.839 INFO:tasks.workunit.client.0.smithi107.stdout:6/628: truncate f6 2798274 0 2024-04-22T23:35:08.841 INFO:tasks.workunit.client.1.smithi157.stdout:8/588: mkdir d2/d6/d21/d5f/dcd 0 2024-04-22T23:35:08.846 INFO:tasks.workunit.client.1.smithi157.stdout:4/790: write d3/d11/d1d/d73/d80/f8c [850811,66021] 0 2024-04-22T23:35:08.847 INFO:tasks.workunit.client.0.smithi107.stdout:0/575: mknod d1/d84/cc6 0 2024-04-22T23:35:08.848 INFO:tasks.workunit.client.1.smithi157.stdout:4/791: creat d3/d11/d1d/d73/da7/f106 x:0 0 0 2024-04-22T23:35:08.850 INFO:tasks.workunit.client.0.smithi107.stdout:1/552: mkdir d7/d19/d71/db8 0 2024-04-22T23:35:08.850 INFO:tasks.workunit.client.0.smithi107.stdout:1/553: creat d7/d19/d20/d39/d53/d43/d46/d5c/fb9 x:0 0 0 2024-04-22T23:35:08.850 INFO:tasks.workunit.client.0.smithi107.stdout:1/554: truncate d7/d19/d54/f3a 701361 0 2024-04-22T23:35:08.852 INFO:tasks.workunit.client.0.smithi107.stdout:7/568: creat d7/d21/d7e/fb8 x:0 0 0 2024-04-22T23:35:08.853 INFO:tasks.workunit.client.0.smithi107.stdout:5/628: getdents d9/db/de/d72 0 2024-04-22T23:35:08.855 INFO:tasks.workunit.client.0.smithi107.stdout:0/576: creat d1/d9/fc7 x:0 0 0 2024-04-22T23:35:08.855 INFO:tasks.workunit.client.0.smithi107.stdout:0/577: creat d1/d84/fc8 x:0 0 0 2024-04-22T23:35:08.855 INFO:tasks.workunit.client.0.smithi107.stdout:0/578: readlink d1/d9/d14/d4f/da6/lc0 0 2024-04-22T23:35:08.858 INFO:tasks.workunit.client.0.smithi107.stdout:0/579: creat d1/d27/d8a/db5/fc9 x:0 0 0 2024-04-22T23:35:08.858 INFO:tasks.workunit.client.0.smithi107.stdout:0/580: rename d1 to d1/d27/dca 22 2024-04-22T23:35:08.859 INFO:tasks.workunit.client.0.smithi107.stdout:1/555: rmdir d7/d19/d2d/d89 39 2024-04-22T23:35:08.863 INFO:tasks.workunit.client.0.smithi107.stdout:0/581: rename d1/d9/fc7 to d1/d27/d8a/fcb 0 2024-04-22T23:35:08.864 INFO:tasks.workunit.client.0.smithi107.stdout:6/629: read db/d20/d58/f65 [1423024,385] 0 2024-04-22T23:35:08.864 INFO:tasks.workunit.client.0.smithi107.stdout:6/630: write db/d20/d2c/d3b/fa7 [191909,62667] 0 2024-04-22T23:35:08.864 INFO:tasks.workunit.client.0.smithi107.stdout:6/631: read db/f11 [812997,54273] 0 2024-04-22T23:35:08.864 INFO:tasks.workunit.client.0.smithi107.stdout:6/632: chown db/d20/d2a/d38/d4b/f6b 1 1 2024-04-22T23:35:08.865 INFO:tasks.workunit.client.0.smithi107.stdout:6/633: getdents db/d20/db1 0 2024-04-22T23:35:08.866 INFO:tasks.workunit.client.0.smithi107.stdout:6/634: dread - db/d20/d49/d7b/fb4 zero size 2024-04-22T23:35:08.868 INFO:tasks.workunit.client.0.smithi107.stdout:1/556: dread d7/f67 [0,4194304] 0 2024-04-22T23:35:08.869 INFO:tasks.workunit.client.0.smithi107.stdout:6/635: creat db/d20/d2a/d35/d69/db3/dbc/fc0 x:0 0 0 2024-04-22T23:35:08.869 INFO:tasks.workunit.client.0.smithi107.stdout:6/636: chown db/d20/d2a/d38/d61/d6f/da2/d84 1618 1 2024-04-22T23:35:08.878 INFO:tasks.workunit.client.0.smithi107.stdout:0/582: write d1/d27/d20/d2c/d2e/f71 [3420776,14163] 0 2024-04-22T23:35:08.878 INFO:tasks.workunit.client.0.smithi107.stdout:0/583: creat d1/d27/d8a/d8b/fcc x:0 0 0 2024-04-22T23:35:08.880 INFO:tasks.workunit.client.0.smithi107.stdout:6/637: read f0 [2029907,1653] 0 2024-04-22T23:35:08.881 INFO:tasks.workunit.client.0.smithi107.stdout:6/638: mknod db/cc1 0 2024-04-22T23:35:08.881 INFO:tasks.workunit.client.0.smithi107.stdout:6/639: getdents db/d20/d2a/d38 0 2024-04-22T23:35:08.882 INFO:tasks.workunit.client.0.smithi107.stdout:6/640: stat db/d20/d49/la6 0 2024-04-22T23:35:08.883 INFO:tasks.workunit.client.0.smithi107.stdout:6/641: truncate db/d20/d49/f6e 748895 0 2024-04-22T23:35:08.889 INFO:tasks.workunit.client.1.smithi157.stdout:2/932: dwrite d7/d8/d30/d5b/f72 [0,4194304] 0 2024-04-22T23:35:08.890 INFO:tasks.workunit.client.1.smithi157.stdout:2/933: creat d7/d8/d30/d12d/f130 x:0 0 0 2024-04-22T23:35:08.899 INFO:tasks.workunit.client.1.smithi157.stdout:9/777: dwrite d4/d15/d23/d27/d47/f74 [0,4194304] 0 2024-04-22T23:35:08.900 INFO:tasks.workunit.client.1.smithi157.stdout:9/778: creat d4/d2d/fee x:0 0 0 2024-04-22T23:35:08.901 INFO:tasks.workunit.client.1.smithi157.stdout:9/779: mknod d4/df/d80/cef 0 2024-04-22T23:35:08.901 INFO:tasks.workunit.client.1.smithi157.stdout:9/780: truncate d4/d15/d55/f76 539338 0 2024-04-22T23:35:08.903 INFO:tasks.workunit.client.1.smithi157.stdout:9/781: creat d4/d15/d23/d27/d75/ff0 x:0 0 0 2024-04-22T23:35:08.904 INFO:tasks.workunit.client.1.smithi157.stdout:9/782: creat d4/d15/d23/d27/ff1 x:0 0 0 2024-04-22T23:35:08.904 INFO:tasks.workunit.client.1.smithi157.stdout:9/783: fdatasync d4/d15/d23/d27/f8a 0 2024-04-22T23:35:08.907 INFO:tasks.workunit.client.1.smithi157.stdout:7/746: dwrite d0/d9/d42/f51 [0,4194304] 0 2024-04-22T23:35:08.910 INFO:tasks.workunit.client.0.smithi107.stdout:8/571: dwrite d0/d7/d8/d41/d53/f77 [0,4194304] 0 2024-04-22T23:35:08.910 INFO:tasks.workunit.client.0.smithi107.stdout:8/572: readlink d0/d85/d51/l58 0 2024-04-22T23:35:08.911 INFO:tasks.workunit.client.1.smithi157.stdout:4/792: dwrite d3/fda [0,4194304] 0 2024-04-22T23:35:08.911 INFO:tasks.workunit.client.0.smithi107.stdout:7/569: dwrite d7/d21/d48/d5c/d9d/f5a [0,4194304] 0 2024-04-22T23:35:08.911 INFO:tasks.workunit.client.0.smithi107.stdout:7/570: fsync d7/d1b/f53 0 2024-04-22T23:35:08.913 INFO:tasks.workunit.client.1.smithi157.stdout:4/793: creat d3/d4b/d5d/f107 x:0 0 0 2024-04-22T23:35:08.913 INFO:tasks.workunit.client.1.smithi157.stdout:4/794: chown d3/c64 7039 1 2024-04-22T23:35:08.915 INFO:tasks.workunit.client.1.smithi157.stdout:4/795: symlink d3/d11/d1d/l108 0 2024-04-22T23:35:08.915 INFO:tasks.workunit.client.1.smithi157.stdout:9/784: dread d4/df/d57/fb0 [0,4194304] 0 2024-04-22T23:35:08.916 INFO:tasks.workunit.client.0.smithi107.stdout:8/573: rename d0/d7/d20/d2d/c55 to d0/d73/cc2 0 2024-04-22T23:35:08.918 INFO:tasks.workunit.client.1.smithi157.stdout:8/589: dwrite d2/d6/d21/d5f/d11/f4a [0,4194304] 0 2024-04-22T23:35:08.918 INFO:tasks.workunit.client.0.smithi107.stdout:1/557: dwrite d7/d19/d2d/d89/fb1 [0,4194304] 0 2024-04-22T23:35:08.919 INFO:tasks.workunit.client.0.smithi107.stdout:1/558: write d7/d19/d54/f66 [336856,107956] 0 2024-04-22T23:35:08.919 INFO:tasks.workunit.client.0.smithi107.stdout:1/559: chown d7/d19/d2d/da4/cb6 52 1 2024-04-22T23:35:08.919 INFO:tasks.workunit.client.1.smithi157.stdout:4/796: rename d3/f43 to d3/d4b/d7a/db7/f109 0 2024-04-22T23:35:08.921 INFO:tasks.workunit.client.1.smithi157.stdout:9/785: mkdir d4/d15/d23/dad/df2 0 2024-04-22T23:35:08.923 INFO:tasks.workunit.client.0.smithi107.stdout:5/629: dwrite d9/d76/f8a [0,4194304] 0 2024-04-22T23:35:08.923 INFO:tasks.workunit.client.0.smithi107.stdout:5/630: write d9/db/f8f [2144294,91088] 0 2024-04-22T23:35:08.924 INFO:tasks.workunit.client.0.smithi107.stdout:5/631: truncate d9/db/f7b 799258 0 2024-04-22T23:35:08.924 INFO:tasks.workunit.client.0.smithi107.stdout:6/642: dwrite db/d20/d2a/f42 [0,4194304] 0 2024-04-22T23:35:08.928 INFO:tasks.workunit.client.1.smithi157.stdout:8/590: dread d2/d6/f3c [0,4194304] 0 2024-04-22T23:35:08.928 INFO:tasks.workunit.client.1.smithi157.stdout:8/591: readlink d2/d6/d21/d5f/d11/d13/l72 0 2024-04-22T23:35:08.929 INFO:tasks.workunit.client.0.smithi107.stdout:6/643: mknod db/d20/d2a/d38/d4b/d9d/cc2 0 2024-04-22T23:35:08.929 INFO:tasks.workunit.client.0.smithi107.stdout:6/644: fdatasync db/d20/d58/fb9 0 2024-04-22T23:35:08.930 INFO:tasks.workunit.client.1.smithi157.stdout:2/934: dwrite d7/f13 [0,4194304] 0 2024-04-22T23:35:08.930 INFO:tasks.workunit.client.1.smithi157.stdout:4/797: symlink d3/d11/d34/da6/l10a 0 2024-04-22T23:35:08.930 INFO:tasks.workunit.client.1.smithi157.stdout:9/786: mknod d4/d15/d23/dad/cf3 0 2024-04-22T23:35:08.932 INFO:tasks.workunit.client.1.smithi157.stdout:7/747: dwrite d0/d9/db/df/fd1 [0,4194304] 0 2024-04-22T23:35:08.933 INFO:tasks.workunit.client.0.smithi107.stdout:0/584: truncate d1/d27/d20/d2c/d2e/f71 227244 0 2024-04-22T23:35:08.933 INFO:tasks.workunit.client.0.smithi107.stdout:0/585: creat d1/d27/d20/d2c/d41/fcd x:0 0 0 2024-04-22T23:35:08.935 INFO:tasks.workunit.client.1.smithi157.stdout:4/798: truncate d3/d4b/d7a/f9c 1176234 0 2024-04-22T23:35:08.935 INFO:tasks.workunit.client.1.smithi157.stdout:5/851: dwrite d1/d5/d7/d2f/d9c/faa [0,4194304] 0 2024-04-22T23:35:08.935 INFO:tasks.workunit.client.1.smithi157.stdout:5/852: chown d1/d5/dc/f89 26 1 2024-04-22T23:35:08.935 INFO:tasks.workunit.client.1.smithi157.stdout:5/853: dread - d1/db0/dff/d102/f124 zero size 2024-04-22T23:35:08.935 INFO:tasks.workunit.client.1.smithi157.stdout:5/854: write d1/d27/d29/d4b/f4d [2920100,103404] 0 2024-04-22T23:35:08.937 INFO:tasks.workunit.client.1.smithi157.stdout:8/592: link d2/d4/c5e d2/d6/cce 0 2024-04-22T23:35:08.938 INFO:tasks.workunit.client.1.smithi157.stdout:8/593: fsync d2/d6/d21/d5f/d11/d13/f7d 0 2024-04-22T23:35:08.938 INFO:tasks.workunit.client.1.smithi157.stdout:8/594: read d2/d34/d6a/d77/f9f [177718,106142] 0 2024-04-22T23:35:08.938 INFO:tasks.workunit.client.1.smithi157.stdout:8/595: chown d2/d6/d21/c28 49 1 2024-04-22T23:35:08.940 INFO:tasks.workunit.client.0.smithi107.stdout:6/645: rename db/d20/d2c/d3b/d40/f71 to db/d20/d2a/d38/d61/d6f/da2/fc3 0 2024-04-22T23:35:08.941 INFO:tasks.workunit.client.0.smithi107.stdout:8/574: dread d0/d3a/d46/f8a [0,4194304] 0 2024-04-22T23:35:08.942 INFO:tasks.workunit.client.1.smithi157.stdout:9/787: dread d4/d15/d23/d27/f53 [0,4194304] 0 2024-04-22T23:35:08.943 INFO:tasks.workunit.client.1.smithi157.stdout:7/748: rename d0/d9/d42/d55/d56/d91/fc2 to d0/d9/d36/dbb/fe1 0 2024-04-22T23:35:08.943 INFO:tasks.workunit.client.0.smithi107.stdout:0/586: unlink d1/d27/d20/d4a/l63 0 2024-04-22T23:35:08.945 INFO:tasks.workunit.client.1.smithi157.stdout:7/749: dread d0/d9/d29/d33/d37/f50 [0,4194304] 0 2024-04-22T23:35:08.945 INFO:tasks.workunit.client.1.smithi157.stdout:7/750: stat d0/f4 0 2024-04-22T23:35:08.945 INFO:tasks.workunit.client.1.smithi157.stdout:7/751: chown d0/d9/d29/d49/d64/c70 26873545 1 2024-04-22T23:35:08.945 INFO:tasks.workunit.client.1.smithi157.stdout:7/752: dread - d0/d9/de/fb9 zero size 2024-04-22T23:35:08.945 INFO:tasks.workunit.client.1.smithi157.stdout:4/799: mkdir d3/d11/d1d/d10b 0 2024-04-22T23:35:08.946 INFO:tasks.workunit.client.1.smithi157.stdout:2/935: write d7/d8/dca/f10e [3124832,126959] 0 2024-04-22T23:35:08.946 INFO:tasks.workunit.client.0.smithi107.stdout:0/587: dread d1/d27/d20/d2c/d41/f64 [0,4194304] 0 2024-04-22T23:35:08.946 INFO:tasks.workunit.client.0.smithi107.stdout:0/588: fsync d1/d84/fc8 0 2024-04-22T23:35:08.946 INFO:tasks.workunit.client.0.smithi107.stdout:0/589: write d1/d9/d14/f8c [353584,56065] 0 2024-04-22T23:35:08.946 INFO:tasks.workunit.client.0.smithi107.stdout:0/590: dread - d1/d27/fb0 zero size 2024-04-22T23:35:08.946 INFO:tasks.workunit.client.0.smithi107.stdout:6/646: rename db/d20/d2c/d3b/c52 to db/d20/d2a/cc4 0 2024-04-22T23:35:08.947 INFO:tasks.workunit.client.0.smithi107.stdout:6/647: symlink db/d20/d2a/d38/d61/d6f/da2/d84/lc5 0 2024-04-22T23:35:08.948 INFO:tasks.workunit.client.1.smithi157.stdout:8/596: rmdir d2/d4/dbd/d67 39 2024-04-22T23:35:08.951 INFO:tasks.workunit.client.1.smithi157.stdout:7/753: mkdir d0/d9/db/df/dac/de2 0 2024-04-22T23:35:08.952 INFO:tasks.workunit.client.0.smithi107.stdout:6/648: dread db/f3a [0,4194304] 0 2024-04-22T23:35:08.952 INFO:tasks.workunit.client.0.smithi107.stdout:8/575: dread d0/d7/f31 [0,4194304] 0 2024-04-22T23:35:08.954 INFO:tasks.workunit.client.1.smithi157.stdout:4/800: creat d3/d11/d34/da4/f10c x:0 0 0 2024-04-22T23:35:08.955 INFO:tasks.workunit.client.0.smithi107.stdout:8/576: rmdir d0/d7/d8 39 2024-04-22T23:35:08.956 INFO:tasks.workunit.client.1.smithi157.stdout:2/936: getdents d7/d8/d30/d23/d27 0 2024-04-22T23:35:08.957 INFO:tasks.workunit.client.1.smithi157.stdout:2/937: stat d7/d8/d30/d23/d27/d57/dcf/lf1 0 2024-04-22T23:35:08.958 INFO:tasks.workunit.client.0.smithi107.stdout:8/577: mkdir d0/d7/d20/da2/dc3 0 2024-04-22T23:35:08.959 INFO:tasks.workunit.client.1.smithi157.stdout:5/855: dread d1/d5/dc/d43/d5e/fda [0,4194304] 0 2024-04-22T23:35:08.959 INFO:tasks.workunit.client.1.smithi157.stdout:5/856: chown d1/db0/d7a 1265 1 2024-04-22T23:35:08.960 INFO:tasks.workunit.client.0.smithi107.stdout:1/560: dwrite d7/d19/d2d/d89/f94 [0,4194304] 0 2024-04-22T23:35:08.963 INFO:tasks.workunit.client.0.smithi107.stdout:5/632: dwrite f7 [0,4194304] 0 2024-04-22T23:35:08.963 INFO:tasks.workunit.client.0.smithi107.stdout:5/633: chown d9/d26/c9f 0 1 2024-04-22T23:35:08.963 INFO:tasks.workunit.client.0.smithi107.stdout:1/561: mkdir d7/d19/d20/d39/d53/d43/d46/dba 0 2024-04-22T23:35:08.963 INFO:tasks.workunit.client.0.smithi107.stdout:1/562: creat d7/d19/d54/fbb x:0 0 0 2024-04-22T23:35:08.964 INFO:tasks.workunit.client.1.smithi157.stdout:7/754: rename d0/d9/l24 to d0/le3 0 2024-04-22T23:35:08.964 INFO:tasks.workunit.client.1.smithi157.stdout:7/755: stat d0/d9/db/df/lb0 0 2024-04-22T23:35:08.964 INFO:tasks.workunit.client.1.smithi157.stdout:7/756: creat d0/d9/d29/d33/fe4 x:0 0 0 2024-04-22T23:35:08.964 INFO:tasks.workunit.client.1.smithi157.stdout:1/857: sync 2024-04-22T23:35:08.964 INFO:tasks.workunit.client.1.smithi157.stdout:1/858: chown d4/fc 1318 1 2024-04-22T23:35:08.967 INFO:tasks.workunit.client.0.smithi107.stdout:1/563: rmdir d7/d19/d2d/d89 39 2024-04-22T23:35:08.967 INFO:tasks.workunit.client.0.smithi107.stdout:1/564: fsync d7/d19/d20/d39/d53/d43/d46/d5c/fb3 0 2024-04-22T23:35:08.967 INFO:tasks.workunit.client.0.smithi107.stdout:7/571: dwrite d7/d1b/f38 [0,4194304] 0 2024-04-22T23:35:08.967 INFO:tasks.workunit.client.1.smithi157.stdout:5/857: rename d1/d5/d7/d2f/d45/f53 to d1/d27/d29/f13e 0 2024-04-22T23:35:08.968 INFO:tasks.workunit.client.1.smithi157.stdout:4/801: symlink d3/d5c/dd3/l10d 0 2024-04-22T23:35:08.971 INFO:tasks.workunit.client.1.smithi157.stdout:7/757: rmdir d0/d9/db/df/dac/de2 0 2024-04-22T23:35:08.972 INFO:tasks.workunit.client.0.smithi107.stdout:8/578: write d0/d7/df/f23 [3785286,28386] 0 2024-04-22T23:35:08.972 INFO:tasks.workunit.client.0.smithi107.stdout:8/579: stat d0/d7/d8/d41/d53/cbf 0 2024-04-22T23:35:08.972 INFO:tasks.workunit.client.1.smithi157.stdout:4/802: creat d3/d11/d1d/d73/da3/d31/f10e x:0 0 0 2024-04-22T23:35:08.973 INFO:tasks.workunit.client.1.smithi157.stdout:4/803: write d3/d11/d1d/f49 [1300006,83011] 0 2024-04-22T23:35:08.973 INFO:tasks.workunit.client.1.smithi157.stdout:4/804: truncate d3/d11/d1d/d73/d80/f82 948889 0 2024-04-22T23:35:08.973 INFO:tasks.workunit.client.1.smithi157.stdout:7/758: creat d0/d9/d29/d49/d4a/fe5 x:0 0 0 2024-04-22T23:35:08.975 INFO:tasks.workunit.client.0.smithi107.stdout:8/580: rename d0/d3a/l8f to d0/d7/d8/d41/d53/lc4 0 2024-04-22T23:35:08.977 INFO:tasks.workunit.client.1.smithi157.stdout:4/805: symlink d3/d4b/d7a/l10f 0 2024-04-22T23:35:08.981 INFO:tasks.workunit.client.1.smithi157.stdout:4/806: creat d3/d11/d1d/d73/d80/dac/db5/f110 x:0 0 0 2024-04-22T23:35:08.981 INFO:tasks.workunit.client.1.smithi157.stdout:4/807: dread - d3/d11/d1d/d73/d80/dac/fcf zero size 2024-04-22T23:35:08.981 INFO:tasks.workunit.client.1.smithi157.stdout:4/808: chown d3/d11/fe6 1076 1 2024-04-22T23:35:08.982 INFO:tasks.workunit.client.0.smithi107.stdout:7/572: write d7/d21/d48/d5c/d9d/f5a [962138,51990] 0 2024-04-22T23:35:08.982 INFO:tasks.workunit.client.0.smithi107.stdout:7/573: symlink d7/d1b/lb9 0 2024-04-22T23:35:08.982 INFO:tasks.workunit.client.0.smithi107.stdout:7/574: truncate d7/d21/da4/d81/f97 702161 0 2024-04-22T23:35:08.982 INFO:tasks.workunit.client.1.smithi157.stdout:4/809: truncate d3/d11/d1d/f26 11365702 0 2024-04-22T23:35:08.983 INFO:tasks.workunit.client.0.smithi107.stdout:7/575: creat d7/d21/d7e/d94/fba x:0 0 0 2024-04-22T23:35:08.983 INFO:tasks.workunit.client.0.smithi107.stdout:7/576: write d7/d21/da4/f43 [1334661,70444] 0 2024-04-22T23:35:08.984 INFO:tasks.workunit.client.0.smithi107.stdout:8/581: dread d0/f4 [0,4194304] 0 2024-04-22T23:35:08.984 INFO:tasks.workunit.client.0.smithi107.stdout:8/582: write d0/d7/f11 [2453684,96338] 0 2024-04-22T23:35:08.984 INFO:tasks.workunit.client.0.smithi107.stdout:7/577: creat d7/db4/fbb x:0 0 0 2024-04-22T23:35:08.984 INFO:tasks.workunit.client.1.smithi157.stdout:4/810: mkdir d3/d11/d1d/d73/d80/dac/db5/df7/d111 0 2024-04-22T23:35:08.984 INFO:tasks.workunit.client.1.smithi157.stdout:4/811: fdatasync d3/d11/d1d/d73/d80/dac/fcf 0 2024-04-22T23:35:08.985 INFO:tasks.workunit.client.0.smithi107.stdout:8/583: rename d0/d7/f34 to d0/d7/d20/d89/fc5 0 2024-04-22T23:35:08.986 INFO:tasks.workunit.client.1.smithi157.stdout:4/812: link d3/d11/d1d/d73/da3/d31/d91/fa5 d3/d11/d1d/d73/d80/f112 0 2024-04-22T23:35:08.986 INFO:tasks.workunit.client.1.smithi157.stdout:4/813: fsync d3/d11/d1d/d73/da7/f106 0 2024-04-22T23:35:08.986 INFO:tasks.workunit.client.1.smithi157.stdout:4/814: write d3/d11/f1f [1025298,89226] 0 2024-04-22T23:35:08.986 INFO:tasks.workunit.client.1.smithi157.stdout:4/815: creat d3/d96/d4c/f113 x:0 0 0 2024-04-22T23:35:08.986 INFO:tasks.workunit.client.1.smithi157.stdout:4/816: fsync d3/d11/d1d/d73/da3/d31/f6f 0 2024-04-22T23:35:08.987 INFO:tasks.workunit.client.0.smithi107.stdout:7/578: truncate d7/d21/d48/f55 1289752 0 2024-04-22T23:35:08.987 INFO:tasks.workunit.client.1.smithi157.stdout:4/817: creat d3/d5c/f114 x:0 0 0 2024-04-22T23:35:08.988 INFO:tasks.workunit.client.1.smithi157.stdout:4/818: fsync d3/d11/d34/da6/fc6 0 2024-04-22T23:35:08.988 INFO:tasks.workunit.client.1.smithi157.stdout:4/819: truncate d3/d11/d1d/d73/d80/dac/fef 144238 0 2024-04-22T23:35:08.989 INFO:tasks.workunit.client.0.smithi107.stdout:8/584: mknod d0/d7/df/d33/d37/cc6 0 2024-04-22T23:35:08.989 INFO:tasks.workunit.client.0.smithi107.stdout:8/585: creat d0/d7/df/d3b/fc7 x:0 0 0 2024-04-22T23:35:08.990 INFO:tasks.workunit.client.1.smithi157.stdout:4/820: symlink d3/d11/d30/l115 0 2024-04-22T23:35:08.990 INFO:tasks.workunit.client.1.smithi157.stdout:4/821: truncate d3/d11/d1d/d73/da7/f106 933469 0 2024-04-22T23:35:08.990 INFO:tasks.workunit.client.1.smithi157.stdout:4/822: dread d3/d11/d34/da6/fc6 [0,4194304] 0 2024-04-22T23:35:08.990 INFO:tasks.workunit.client.1.smithi157.stdout:4/823: creat d3/d11/d1d/d10b/f116 x:0 0 0 2024-04-22T23:35:08.990 INFO:tasks.workunit.client.1.smithi157.stdout:4/824: chown d3/d11/f22 22576 1 2024-04-22T23:35:08.990 INFO:tasks.workunit.client.1.smithi157.stdout:4/825: stat d3/d11/d34/da6/fc6 0 2024-04-22T23:35:08.990 INFO:tasks.workunit.client.1.smithi157.stdout:4/826: chown d3/d11/d34/f48 477 1 2024-04-22T23:35:08.991 INFO:tasks.workunit.client.1.smithi157.stdout:4/827: creat d3/d11/d1d/d73/d80/dac/db5/dfb/f117 x:0 0 0 2024-04-22T23:35:08.991 INFO:tasks.workunit.client.1.smithi157.stdout:4/828: mknod d3/d11/d34/d9d/c118 0 2024-04-22T23:35:08.991 INFO:tasks.workunit.client.1.smithi157.stdout:4/829: getdents d3/d96/d4c/de7 0 2024-04-22T23:35:08.992 INFO:tasks.workunit.client.0.smithi107.stdout:8/586: mkdir d0/d7/d87/dc8 0 2024-04-22T23:35:08.992 INFO:tasks.workunit.client.0.smithi107.stdout:7/579: mknod d7/d21/d48/d95/cbc 0 2024-04-22T23:35:08.992 INFO:tasks.workunit.client.0.smithi107.stdout:7/580: write d7/d21/f96 [4272871,105178] 0 2024-04-22T23:35:08.993 INFO:tasks.workunit.client.1.smithi157.stdout:4/830: unlink d3/d11/d1d/d73/da7/ff9 0 2024-04-22T23:35:08.993 INFO:tasks.workunit.client.1.smithi157.stdout:4/831: symlink d3/d11/d1d/d73/d80/dac/db5/dfb/l119 0 2024-04-22T23:35:08.994 INFO:tasks.workunit.client.1.smithi157.stdout:4/832: mknod d3/d11/d1d/c11a 0 2024-04-22T23:35:08.994 INFO:tasks.workunit.client.0.smithi107.stdout:8/587: link d0/d3a/cb4 d0/cc9 0 2024-04-22T23:35:08.996 INFO:tasks.workunit.client.0.smithi107.stdout:7/581: truncate d7/d21/f6d 7883735 0 2024-04-22T23:35:08.997 INFO:tasks.workunit.client.0.smithi107.stdout:8/588: creat d0/d7/df/fca x:0 0 0 2024-04-22T23:35:08.998 INFO:tasks.workunit.client.0.smithi107.stdout:6/649: dwrite db/d20/d2a/d38/f9f [0,4194304] 0 2024-04-22T23:35:08.998 INFO:tasks.workunit.client.0.smithi107.stdout:6/650: stat db/f3a 0 2024-04-22T23:35:08.998 INFO:tasks.workunit.client.0.smithi107.stdout:6/651: readlink db/d20/d2a/l34 0 2024-04-22T23:35:08.998 INFO:tasks.workunit.client.0.smithi107.stdout:6/652: stat db/d20/d2a/d38/f9f 0 2024-04-22T23:35:08.998 INFO:tasks.workunit.client.0.smithi107.stdout:6/653: write db/fd [2519382,113679] 0 2024-04-22T23:35:08.999 INFO:tasks.workunit.client.0.smithi107.stdout:7/582: mkdir d7/d21/d48/d95/dbd 0 2024-04-22T23:35:09.000 INFO:tasks.workunit.client.0.smithi107.stdout:6/654: stat db/d20/d2c/d3b/c88 0 2024-04-22T23:35:09.000 INFO:tasks.workunit.client.0.smithi107.stdout:0/591: dwrite d1/d27/d8a/db5/fc9 [0,4194304] 0 2024-04-22T23:35:09.000 INFO:tasks.workunit.client.0.smithi107.stdout:0/592: chown d1/d27/d20/d4a 27663542 1 2024-04-22T23:35:09.001 INFO:tasks.workunit.client.0.smithi107.stdout:5/634: dwrite f4 [0,4194304] 0 2024-04-22T23:35:09.001 INFO:tasks.workunit.client.0.smithi107.stdout:6/655: mknod db/d20/d2a/d35/d69/d79/cc6 0 2024-04-22T23:35:09.001 INFO:tasks.workunit.client.1.smithi157.stdout:8/597: dwrite d2/d6/d21/d5f/f55 [0,4194304] 0 2024-04-22T23:35:09.002 INFO:tasks.workunit.client.0.smithi107.stdout:5/635: dread d9/d76/fc8 [0,4194304] 0 2024-04-22T23:35:09.004 INFO:tasks.workunit.client.1.smithi157.stdout:8/598: creat d2/d6/d21/d5f/d17/fcf x:0 0 0 2024-04-22T23:35:09.005 INFO:tasks.workunit.client.1.smithi157.stdout:8/599: symlink d2/d4/dbd/d97/ld0 0 2024-04-22T23:35:09.007 INFO:tasks.workunit.client.0.smithi107.stdout:0/593: creat d1/d27/d20/d40/dbd/fce x:0 0 0 2024-04-22T23:35:09.007 INFO:tasks.workunit.client.0.smithi107.stdout:0/594: chown d1/lb2 11082971 1 2024-04-22T23:35:09.008 INFO:tasks.workunit.client.0.smithi107.stdout:7/583: write d7/f26 [1508818,10820] 0 2024-04-22T23:35:09.008 INFO:tasks.workunit.client.0.smithi107.stdout:0/595: mknod d1/d27/d20/d2c/d2e/ccf 0 2024-04-22T23:35:09.009 INFO:tasks.workunit.client.0.smithi107.stdout:5/636: rmdir d9/db/d92 39 2024-04-22T23:35:09.010 INFO:tasks.workunit.client.0.smithi107.stdout:0/596: creat d1/d27/d20/d40/db4/fd0 x:0 0 0 2024-04-22T23:35:09.011 INFO:tasks.workunit.client.0.smithi107.stdout:5/637: link d9/d76/f7d d9/db/d42/d75/fd7 0 2024-04-22T23:35:09.012 INFO:tasks.workunit.client.0.smithi107.stdout:7/584: mknod d7/d21/d48/cbe 0 2024-04-22T23:35:09.012 INFO:tasks.workunit.client.0.smithi107.stdout:7/585: dread - d7/d21/f7f zero size 2024-04-22T23:35:09.013 INFO:tasks.workunit.client.0.smithi107.stdout:7/586: truncate d7/d4c/da1/fae 393261 0 2024-04-22T23:35:09.013 INFO:tasks.workunit.client.0.smithi107.stdout:7/587: chown d7/d4c/f6c 0 1 2024-04-22T23:35:09.017 INFO:tasks.workunit.client.1.smithi157.stdout:8/600: dread d2/d34/d6a/d77/f9f [0,4194304] 0 2024-04-22T23:35:09.020 INFO:tasks.workunit.client.1.smithi157.stdout:5/858: dwrite d1/d5/d7/d2f/d45/d54/f12b [0,4194304] 0 2024-04-22T23:35:09.021 INFO:tasks.workunit.client.0.smithi107.stdout:8/589: getdents d0/d3a 0 2024-04-22T23:35:09.021 INFO:tasks.workunit.client.0.smithi107.stdout:8/590: stat d0/d7/df/d3b/c99 0 2024-04-22T23:35:09.023 INFO:tasks.workunit.client.0.smithi107.stdout:8/591: creat d0/d7/df/d3b/d7b/fcb x:0 0 0 2024-04-22T23:35:09.023 INFO:tasks.workunit.client.0.smithi107.stdout:8/592: chown d0/d7/d8/d41 795408 1 2024-04-22T23:35:09.024 INFO:tasks.workunit.client.0.smithi107.stdout:5/638: write d9/f36 [2566096,118507] 0 2024-04-22T23:35:09.026 INFO:tasks.workunit.client.0.smithi107.stdout:8/593: creat d0/d7/df/d33/d37/fcc x:0 0 0 2024-04-22T23:35:09.027 INFO:tasks.workunit.client.1.smithi157.stdout:5/859: mknod d1/db0/d7a/d133/d118/c13f 0 2024-04-22T23:35:09.027 INFO:tasks.workunit.client.1.smithi157.stdout:8/601: write d2/d4/f52 [3599718,62366] 0 2024-04-22T23:35:09.027 INFO:tasks.workunit.client.0.smithi107.stdout:0/597: read d1/d9/d14/f4d [1385206,82357] 0 2024-04-22T23:35:09.028 INFO:tasks.workunit.client.0.smithi107.stdout:8/594: mknod d0/d3a/d46/d59/d78/ccd 0 2024-04-22T23:35:09.030 INFO:tasks.workunit.client.1.smithi157.stdout:8/602: dread d2/db1/f5c [0,4194304] 0 2024-04-22T23:35:09.030 INFO:tasks.workunit.client.0.smithi107.stdout:8/595: rename d0/d7/d20/d2d/c42 to d0/d7/df/d33/cce 0 2024-04-22T23:35:09.031 INFO:tasks.workunit.client.0.smithi107.stdout:6/656: dwrite db/f21 [0,4194304] 0 2024-04-22T23:35:09.032 INFO:tasks.workunit.client.1.smithi157.stdout:4/833: dwrite d3/d96/d4c/fd4 [0,4194304] 0 2024-04-22T23:35:09.033 INFO:tasks.workunit.client.1.smithi157.stdout:5/860: mkdir d1/d5/d104/d140 0 2024-04-22T23:35:09.034 INFO:tasks.workunit.client.1.smithi157.stdout:5/861: chown d1/d5/dc/d43/d4f/d7f/d99 23299 1 2024-04-22T23:35:09.034 INFO:tasks.workunit.client.0.smithi107.stdout:6/657: write db/f8a [718844,128790] 0 2024-04-22T23:35:09.036 INFO:tasks.workunit.client.0.smithi107.stdout:6/658: truncate db/d20/d2c/d3b/fa7 244691 0 2024-04-22T23:35:09.038 INFO:tasks.workunit.client.1.smithi157.stdout:8/603: mknod d2/d4/dbd/d97/cd1 0 2024-04-22T23:35:09.040 INFO:tasks.workunit.client.0.smithi107.stdout:6/659: rename db/d20/d2c/d3b/c88 to db/cc7 0 2024-04-22T23:35:09.042 INFO:tasks.workunit.client.1.smithi157.stdout:4/834: dread d3/d96/d40/fab [0,4194304] 0 2024-04-22T23:35:09.044 INFO:tasks.workunit.client.0.smithi107.stdout:6/660: mknod db/d20/d2a/d38/d4b/cc8 0 2024-04-22T23:35:09.044 INFO:tasks.workunit.client.1.smithi157.stdout:5/862: unlink d1/d27/d29/d4b/fae 0 2024-04-22T23:35:09.047 INFO:tasks.workunit.client.0.smithi107.stdout:0/598: dwrite d1/d9/d14/f8c [0,4194304] 0 2024-04-22T23:35:09.047 INFO:tasks.workunit.client.0.smithi107.stdout:0/599: chown d1/d2a/dad/l97 23523 1 2024-04-22T23:35:09.047 INFO:tasks.workunit.client.0.smithi107.stdout:9/602: sync 2024-04-22T23:35:09.047 INFO:tasks.workunit.client.0.smithi107.stdout:9/603: fdatasync d5/d14/d2e/fb1 0 2024-04-22T23:35:09.047 INFO:tasks.workunit.client.0.smithi107.stdout:9/604: write d5/f13 [1327994,30475] 0 2024-04-22T23:35:09.050 INFO:tasks.workunit.client.1.smithi157.stdout:4/835: creat d3/d11/d1d/d73/da3/d31/d91/dcb/dcd/f11b x:0 0 0 2024-04-22T23:35:09.050 INFO:tasks.workunit.client.1.smithi157.stdout:4/836: readlink d3/d4b/lb 0 2024-04-22T23:35:09.050 INFO:tasks.workunit.client.1.smithi157.stdout:4/837: chown d3/d4b/lb 49118101 1 2024-04-22T23:35:09.052 INFO:tasks.workunit.client.0.smithi107.stdout:9/605: mknod d5/d14/d2e/d43/dc9/cd2 0 2024-04-22T23:35:09.052 INFO:tasks.workunit.client.0.smithi107.stdout:9/606: chown d5/d7/d17/d1b/d1f 335421 1 2024-04-22T23:35:09.054 INFO:tasks.workunit.client.1.smithi157.stdout:4/838: read d3/d11/d1d/d73/da3/d31/d91/f9b [135882,126060] 0 2024-04-22T23:35:09.054 INFO:tasks.workunit.client.1.smithi157.stdout:4/839: chown d3/d11/d1d/d10b 0 1 2024-04-22T23:35:09.056 INFO:tasks.workunit.client.1.smithi157.stdout:4/840: rmdir d3/d96/d40/dc3 39 2024-04-22T23:35:09.058 INFO:tasks.workunit.client.1.smithi157.stdout:4/841: creat d3/d11/d30/dfd/f11c x:0 0 0 2024-04-22T23:35:09.058 INFO:tasks.workunit.client.1.smithi157.stdout:4/842: fdatasync d3/d11/d30/f71 0 2024-04-22T23:35:09.059 INFO:tasks.workunit.client.1.smithi157.stdout:4/843: link d3/d4b/cb6 d3/d11/d30/dfd/c11d 0 2024-04-22T23:35:09.060 INFO:tasks.workunit.client.1.smithi157.stdout:4/844: fdatasync d3/d11/d1d/d73/da3/d31/f10e 0 2024-04-22T23:35:09.060 INFO:tasks.workunit.client.1.smithi157.stdout:4/845: readlink d3/d11/d1d/d73/l9e 0 2024-04-22T23:35:09.060 INFO:tasks.workunit.client.1.smithi157.stdout:4/846: fsync d3/d11/d1d/feb 0 2024-04-22T23:35:09.061 INFO:tasks.workunit.client.0.smithi107.stdout:9/607: rename d5/d29/d7b/d91/dc0 to d5/d7/d27/d46/dd3 0 2024-04-22T23:35:09.064 INFO:tasks.workunit.client.1.smithi157.stdout:8/604: dread d2/d4/f1c [0,4194304] 0 2024-04-22T23:35:09.064 INFO:tasks.workunit.client.1.smithi157.stdout:8/605: write d2/d34/f60 [760917,117985] 0 2024-04-22T23:35:09.065 INFO:tasks.workunit.client.0.smithi107.stdout:9/608: link d5/d29/f4d d5/d14/d2e/d4f/d83/fd4 0 2024-04-22T23:35:09.067 INFO:tasks.workunit.client.0.smithi107.stdout:9/609: unlink d5/d7/d27/d69/db6/fab 0 2024-04-22T23:35:09.071 INFO:tasks.workunit.client.0.smithi107.stdout:9/610: dread d5/d24/f98 [0,4194304] 0 2024-04-22T23:35:09.071 INFO:tasks.workunit.client.0.smithi107.stdout:8/596: dwrite d0/d7/df/d3b/f2b [0,4194304] 0 2024-04-22T23:35:09.072 INFO:tasks.workunit.client.1.smithi157.stdout:5/863: rmdir d1/d27/d29/d4b 39 2024-04-22T23:35:09.073 INFO:tasks.workunit.client.1.smithi157.stdout:5/864: chown d1/d27/d29/l10d 68798 1 2024-04-22T23:35:09.073 INFO:tasks.workunit.client.1.smithi157.stdout:5/865: dread - d1/d5/d7/d2f/d45/d54/f125 zero size 2024-04-22T23:35:09.073 INFO:tasks.workunit.client.0.smithi107.stdout:9/611: link d5/d7/f70 d5/d7/d27/d46/fd5 0 2024-04-22T23:35:09.073 INFO:tasks.workunit.client.0.smithi107.stdout:9/612: chown d5/d7/l85 1 1 2024-04-22T23:35:09.074 INFO:tasks.workunit.client.1.smithi157.stdout:5/866: link d1/la7 d1/d5/dc/d43/d4f/l141 0 2024-04-22T23:35:09.075 INFO:tasks.workunit.client.1.smithi157.stdout:5/867: truncate d1/d27/fdf 655868 0 2024-04-22T23:35:09.075 INFO:tasks.workunit.client.0.smithi107.stdout:8/597: rename d0/f19 to d0/d7/d20/da2/dc3/fcf 0 2024-04-22T23:35:09.076 INFO:tasks.workunit.client.0.smithi107.stdout:9/613: truncate d5/d24/f3c 1289342 0 2024-04-22T23:35:09.076 INFO:tasks.workunit.client.0.smithi107.stdout:5/639: dwrite d9/f58 [4194304,4194304] 0 2024-04-22T23:35:09.076 INFO:tasks.workunit.client.0.smithi107.stdout:6/661: dwrite db/d20/d2a/d38/f3e [4194304,4194304] 0 2024-04-22T23:35:09.082 INFO:tasks.workunit.client.0.smithi107.stdout:0/600: dwrite d1/d2a/f77 [0,4194304] 0 2024-04-22T23:35:09.082 INFO:tasks.workunit.client.0.smithi107.stdout:0/601: write d1/d84/f92 [773040,46002] 0 2024-04-22T23:35:09.083 INFO:tasks.workunit.client.0.smithi107.stdout:5/640: rmdir d9/db/de/d72 39 2024-04-22T23:35:09.085 INFO:tasks.workunit.client.0.smithi107.stdout:8/598: getdents d0/d7/df/d3b/d7b 0 2024-04-22T23:35:09.085 INFO:tasks.workunit.client.0.smithi107.stdout:8/599: chown d0/d85/d51/ca6 9390 1 2024-04-22T23:35:09.087 INFO:tasks.workunit.client.0.smithi107.stdout:9/614: write d5/d7/d17/d1b/d1f/d36/d53/f5b [3262220,78051] 0 2024-04-22T23:35:09.095 INFO:tasks.workunit.client.1.smithi157.stdout:4/847: dwrite d3/d11/d34/da6/ff2 [0,4194304] 0 2024-04-22T23:35:09.095 INFO:tasks.workunit.client.1.smithi157.stdout:4/848: creat d3/d11/d1d/f11e x:0 0 0 2024-04-22T23:35:09.097 INFO:tasks.workunit.client.1.smithi157.stdout:3/821: sync 2024-04-22T23:35:09.098 INFO:tasks.workunit.client.0.smithi107.stdout:9/615: dread d5/d6c/f87 [0,4194304] 0 2024-04-22T23:35:09.098 INFO:tasks.workunit.client.0.smithi107.stdout:9/616: read - d5/d14/d2e/d4f/fa4 zero size 2024-04-22T23:35:09.099 INFO:tasks.workunit.client.1.smithi157.stdout:4/849: mknod d3/d5c/c11f 0 2024-04-22T23:35:09.099 INFO:tasks.workunit.client.0.smithi107.stdout:0/602: dread d1/d27/d20/d2c/d2e/f39 [0,4194304] 0 2024-04-22T23:35:09.100 INFO:tasks.workunit.client.1.smithi157.stdout:3/822: mkdir d0/d3/d34/d3d/df6/d111 0 2024-04-22T23:35:09.101 INFO:tasks.workunit.client.1.smithi157.stdout:3/823: dread d0/d3/d34/d62/fae [0,4194304] 0 2024-04-22T23:35:09.101 INFO:tasks.workunit.client.1.smithi157.stdout:4/850: creat d3/d11/d1d/d73/da3/d31/df5/f120 x:0 0 0 2024-04-22T23:35:09.103 INFO:tasks.workunit.client.0.smithi107.stdout:0/603: link d1/d2a/lb8 d1/d9/ld1 0 2024-04-22T23:35:09.103 INFO:tasks.workunit.client.0.smithi107.stdout:0/604: readlink d1/l56 0 2024-04-22T23:35:09.104 INFO:tasks.workunit.client.1.smithi157.stdout:4/851: rmdir d3/d11/d34/da4/db1 39 2024-04-22T23:35:09.105 INFO:tasks.workunit.client.0.smithi107.stdout:6/662: dwrite db/d20/d2a/d38/d4b/f54 [0,4194304] 0 2024-04-22T23:35:09.105 INFO:tasks.workunit.client.0.smithi107.stdout:6/663: write db/d20/d2a/d38/f53 [445417,80926] 0 2024-04-22T23:35:09.107 INFO:tasks.workunit.client.0.smithi107.stdout:0/605: link d1/lb2 d1/d27/d20/d2c/d41/dbe/ld2 0 2024-04-22T23:35:09.107 INFO:tasks.workunit.client.0.smithi107.stdout:0/606: stat d1/d27/d20/d2c/d41/dbe 0 2024-04-22T23:35:09.107 INFO:tasks.workunit.client.0.smithi107.stdout:0/607: stat d1/d9/cd 0 2024-04-22T23:35:09.109 INFO:tasks.workunit.client.0.smithi107.stdout:8/600: dwrite d0/d85/f8b [0,4194304] 0 2024-04-22T23:35:09.109 INFO:tasks.workunit.client.0.smithi107.stdout:8/601: truncate d0/d7/d20/da2/fa9 225490 0 2024-04-22T23:35:09.109 INFO:tasks.workunit.client.0.smithi107.stdout:8/602: chown d0/d3a/d46/c4c 6258 1 2024-04-22T23:35:09.110 INFO:tasks.workunit.client.1.smithi157.stdout:3/824: dread d0/d3/d34/d47/f70 [0,4194304] 0 2024-04-22T23:35:09.110 INFO:tasks.workunit.client.1.smithi157.stdout:8/606: dwrite d2/d4/dbd/fb4 [4194304,4194304] 0 2024-04-22T23:35:09.113 INFO:tasks.workunit.client.1.smithi157.stdout:3/825: symlink d0/d3/d34/d3d/df6/l112 0 2024-04-22T23:35:09.113 INFO:tasks.workunit.client.1.smithi157.stdout:3/826: symlink d0/d3/d34/d47/dcd/l113 0 2024-04-22T23:35:09.114 INFO:tasks.workunit.client.0.smithi107.stdout:9/617: rmdir d5/d7/d27 39 2024-04-22T23:35:09.121 INFO:tasks.workunit.client.0.smithi107.stdout:9/618: truncate d5/d29/d7b/d91/f95 3796392 0 2024-04-22T23:35:09.122 INFO:tasks.workunit.client.0.smithi107.stdout:9/619: chown d5/d14/f20 9855 1 2024-04-22T23:35:09.125 INFO:tasks.workunit.client.0.smithi107.stdout:9/620: rename d5/d24/f98 to d5/d29/d7b/fd6 0 2024-04-22T23:35:09.126 INFO:tasks.workunit.client.1.smithi157.stdout:0/713: sync 2024-04-22T23:35:09.129 INFO:tasks.workunit.client.0.smithi107.stdout:5/641: dwrite d9/db/d42/f6c [0,4194304] 0 2024-04-22T23:35:09.129 INFO:tasks.workunit.client.0.smithi107.stdout:5/642: fdatasync d9/d76/f8a 0 2024-04-22T23:35:09.129 INFO:tasks.workunit.client.0.smithi107.stdout:9/621: mknod d5/d7/d27/d46/dc7/cd7 0 2024-04-22T23:35:09.129 INFO:tasks.workunit.client.0.smithi107.stdout:9/622: chown d5/d7/d27/d46/f66 1696 1 2024-04-22T23:35:09.130 INFO:tasks.workunit.client.0.smithi107.stdout:5/643: dread d9/db/f7b [0,4194304] 0 2024-04-22T23:35:09.134 INFO:tasks.workunit.client.0.smithi107.stdout:8/603: dwrite d0/d7/df/d3b/fc7 [0,4194304] 0 2024-04-22T23:35:09.135 INFO:tasks.workunit.client.0.smithi107.stdout:9/623: symlink d5/d7/d17/d1b/dcb/ld8 0 2024-04-22T23:35:09.136 INFO:tasks.workunit.client.0.smithi107.stdout:5/644: link d9/db/f50 d9/d26/fd8 0 2024-04-22T23:35:09.138 INFO:tasks.workunit.client.1.smithi157.stdout:0/714: write d17/f1b [6605499,87596] 0 2024-04-22T23:35:09.138 INFO:tasks.workunit.client.1.smithi157.stdout:0/715: write d17/d23/d34/d67/d7f/f74 [1318635,70915] 0 2024-04-22T23:35:09.138 INFO:tasks.workunit.client.0.smithi107.stdout:8/604: mkdir d0/d7/d20/da2/da3/dd0 0 2024-04-22T23:35:09.139 INFO:tasks.workunit.client.0.smithi107.stdout:2/635: sync 2024-04-22T23:35:09.139 INFO:tasks.workunit.client.0.smithi107.stdout:4/641: sync 2024-04-22T23:35:09.139 INFO:tasks.workunit.client.0.smithi107.stdout:9/624: mkdir d5/dd9 0 2024-04-22T23:35:09.139 INFO:tasks.workunit.client.0.smithi107.stdout:9/625: fsync d5/d6c/fc3 0 2024-04-22T23:35:09.139 INFO:tasks.workunit.client.0.smithi107.stdout:2/636: creat d1/d2b/d3d/fd7 x:0 0 0 2024-04-22T23:35:09.139 INFO:tasks.workunit.client.0.smithi107.stdout:2/637: dread - d1/d2b/d49/faa zero size 2024-04-22T23:35:09.140 INFO:tasks.workunit.client.0.smithi107.stdout:8/605: rename d0/d7/d8/d41/d53/d76/la4 to d0/d7/df/d33/d37/d9f/ld1 0 2024-04-22T23:35:09.140 INFO:tasks.workunit.client.0.smithi107.stdout:8/606: write d0/d7/df/d3b/faf [63501,40450] 0 2024-04-22T23:35:09.141 INFO:tasks.workunit.client.0.smithi107.stdout:4/642: link d1/d7/d63/d78/d80/lcb d1/d21/le0 0 2024-04-22T23:35:09.141 INFO:tasks.workunit.client.0.smithi107.stdout:4/643: read - d1/d7/d5b/dbf/fdc zero size 2024-04-22T23:35:09.141 INFO:tasks.workunit.client.1.smithi157.stdout:0/716: rename d17/d2b/d4f/d5a/da1 to d17/d23/d34/dee 0 2024-04-22T23:35:09.142 INFO:tasks.workunit.client.1.smithi157.stdout:0/717: creat d17/d2b/d4f/dc5/dca/fef x:0 0 0 2024-04-22T23:35:09.144 INFO:tasks.workunit.client.0.smithi107.stdout:9/626: unlink d5/d7/d17/d1b/d1f/d36/d96/faa 0 2024-04-22T23:35:09.144 INFO:tasks.workunit.client.0.smithi107.stdout:9/627: dread - d5/d14/fca zero size 2024-04-22T23:35:09.146 INFO:tasks.workunit.client.0.smithi107.stdout:8/607: getdents d0/d7/d20/da2/dc3 0 2024-04-22T23:35:09.146 INFO:tasks.workunit.client.0.smithi107.stdout:8/608: write d0/d7/d8/d41/fad [634798,13429] 0 2024-04-22T23:35:09.146 INFO:tasks.workunit.client.0.smithi107.stdout:8/609: write d0/d7/d20/da2/fa9 [643893,28228] 0 2024-04-22T23:35:09.147 INFO:tasks.workunit.client.0.smithi107.stdout:4/644: dread - d1/d21/d29/d2e/d88/d99/dd2/daf/dc1/f9e zero size 2024-04-22T23:35:09.148 INFO:tasks.workunit.client.1.smithi157.stdout:8/607: dwrite d2/d6/d21/d5f/d17/fcf [0,4194304] 0 2024-04-22T23:35:09.148 INFO:tasks.workunit.client.1.smithi157.stdout:4/852: dwrite d3/d11/d1d/d73/da3/d31/f69 [0,4194304] 0 2024-04-22T23:35:09.149 INFO:tasks.workunit.client.0.smithi107.stdout:4/645: dread d1/d21/d29/d2e/d3b/f8a [0,4194304] 0 2024-04-22T23:35:09.149 INFO:tasks.workunit.client.0.smithi107.stdout:4/646: truncate d1/d21/d29/d2e/d88/d99/dd2/fcf 780045 0 2024-04-22T23:35:09.149 INFO:tasks.workunit.client.0.smithi107.stdout:4/647: creat d1/d7/d63/fe1 x:0 0 0 2024-04-22T23:35:09.149 INFO:tasks.workunit.client.0.smithi107.stdout:9/628: mkdir d5/d7/d17/d1b/d1f/dda 0 2024-04-22T23:35:09.149 INFO:tasks.workunit.client.0.smithi107.stdout:2/638: getdents d1/d67 0 2024-04-22T23:35:09.149 INFO:tasks.workunit.client.0.smithi107.stdout:6/664: dwrite db/d20/d2a/d38/d59/f93 [0,4194304] 0 2024-04-22T23:35:09.150 INFO:tasks.workunit.client.0.smithi107.stdout:0/608: dwrite d1/d27/f21 [0,4194304] 0 2024-04-22T23:35:09.152 INFO:tasks.workunit.client.0.smithi107.stdout:8/610: link d0/d7/d20/d2d/lab d0/d7/d20/da2/ld2 0 2024-04-22T23:35:09.152 INFO:tasks.workunit.client.0.smithi107.stdout:5/645: write d9/f12 [1330510,125453] 0 2024-04-22T23:35:09.152 INFO:tasks.workunit.client.0.smithi107.stdout:4/648: rmdir d1/d1b/d89 39 2024-04-22T23:35:09.152 INFO:tasks.workunit.client.0.smithi107.stdout:4/649: write d1/d21/d29/d2e/d88/d99/dd2/f25 [874951,3686] 0 2024-04-22T23:35:09.152 INFO:tasks.workunit.client.0.smithi107.stdout:4/650: creat d1/d7/d63/d78/d80/fe2 x:0 0 0 2024-04-22T23:35:09.152 INFO:tasks.workunit.client.0.smithi107.stdout:9/629: rename d5/d14/d2e/d43/dc9/cd2 to d5/d7/d27/d46/dd3/cdb 0 2024-04-22T23:35:09.152 INFO:tasks.workunit.client.0.smithi107.stdout:9/630: write d5/f13 [4978884,105808] 0 2024-04-22T23:35:09.153 INFO:tasks.workunit.client.1.smithi157.stdout:8/608: unlink d2/d6/d21/d5f/f55 0 2024-04-22T23:35:09.153 INFO:tasks.workunit.client.1.smithi157.stdout:8/609: stat d2/d6/fba 0 2024-04-22T23:35:09.153 INFO:tasks.workunit.client.1.smithi157.stdout:4/853: mknod d3/d11/d34/da4/c121 0 2024-04-22T23:35:09.155 INFO:tasks.workunit.client.1.smithi157.stdout:8/610: creat d2/d6/d21/d5f/d11/d13/fd2 x:0 0 0 2024-04-22T23:35:09.155 INFO:tasks.workunit.client.1.smithi157.stdout:3/827: dwrite d0/d3/d34/d3d/df6/fbb [4194304,4194304] 0 2024-04-22T23:35:09.160 INFO:tasks.workunit.client.1.smithi157.stdout:3/828: mkdir d0/d3/d34/d3d/d57/d114 0 2024-04-22T23:35:09.163 INFO:tasks.workunit.client.1.smithi157.stdout:3/829: link d0/d3/d13/le7 d0/d3/d34/d3d/df6/dee/d9e/l115 0 2024-04-22T23:35:09.163 INFO:tasks.workunit.client.1.smithi157.stdout:4/854: dread d3/d11/d1d/f49 [0,4194304] 0 2024-04-22T23:35:09.163 INFO:tasks.workunit.client.1.smithi157.stdout:4/855: readlink d3/d11/d30/l2c 0 2024-04-22T23:35:09.163 INFO:tasks.workunit.client.1.smithi157.stdout:4/856: readlink d3/d4b/l85 0 2024-04-22T23:35:09.164 INFO:tasks.workunit.client.1.smithi157.stdout:4/857: chown d3/d11/d34/da4/db1/fba 328 1 2024-04-22T23:35:09.164 INFO:tasks.workunit.client.0.smithi107.stdout:2/639: dread d1/d67/f7b [0,4194304] 0 2024-04-22T23:35:09.164 INFO:tasks.workunit.client.0.smithi107.stdout:2/640: truncate d1/d63/fbd 4592897 0 2024-04-22T23:35:09.164 INFO:tasks.workunit.client.0.smithi107.stdout:2/641: write d1/d2b/f80 [610772,103692] 0 2024-04-22T23:35:09.165 INFO:tasks.workunit.client.0.smithi107.stdout:2/642: truncate d1/d12/d35/d54/fd5 184767 0 2024-04-22T23:35:09.165 INFO:tasks.workunit.client.1.smithi157.stdout:3/830: symlink d0/d3/d13/l116 0 2024-04-22T23:35:09.166 INFO:tasks.workunit.client.1.smithi157.stdout:4/858: mkdir d3/d96/d122 0 2024-04-22T23:35:09.166 INFO:tasks.workunit.client.1.smithi157.stdout:4/859: stat d3/d11/d34/da4/c121 0 2024-04-22T23:35:09.166 INFO:tasks.workunit.client.1.smithi157.stdout:0/718: dwrite d17/d2b/d52/d7e/f95 [0,4194304] 0 2024-04-22T23:35:09.167 INFO:tasks.workunit.client.1.smithi157.stdout:0/719: write d17/d2b/d4f/dc5/dca/fef [240638,77308] 0 2024-04-22T23:35:09.172 INFO:tasks.workunit.client.1.smithi157.stdout:0/720: creat d17/d2b/d52/dd1/ff0 x:0 0 0 2024-04-22T23:35:09.175 INFO:tasks.workunit.client.0.smithi107.stdout:5/646: unlink d9/c20 0 2024-04-22T23:35:09.175 INFO:tasks.workunit.client.0.smithi107.stdout:8/611: symlink d0/d73/ld3 0 2024-04-22T23:35:09.176 INFO:tasks.workunit.client.0.smithi107.stdout:6/665: dwrite db/f3a [0,4194304] 0 2024-04-22T23:35:09.178 INFO:tasks.workunit.client.0.smithi107.stdout:4/651: mkdir d1/d7/d5b/de3 0 2024-04-22T23:35:09.178 INFO:tasks.workunit.client.0.smithi107.stdout:6/666: dread db/d20/d2c/d3b/fa8 [0,4194304] 0 2024-04-22T23:35:09.178 INFO:tasks.workunit.client.0.smithi107.stdout:4/652: chown d1/d7/d63/f72 23 1 2024-04-22T23:35:09.178 INFO:tasks.workunit.client.0.smithi107.stdout:4/653: readlink d1/d21/d29/d2e/d7e/l83 0 2024-04-22T23:35:09.178 INFO:tasks.workunit.client.0.smithi107.stdout:6/667: read db/f3f [1953781,31865] 0 2024-04-22T23:35:09.179 INFO:tasks.workunit.client.0.smithi107.stdout:6/668: fsync db/d20/d2c/d3b/fb0 0 2024-04-22T23:35:09.180 INFO:tasks.workunit.client.0.smithi107.stdout:6/669: dread db/d20/d2c/d3b/fa8 [0,4194304] 0 2024-04-22T23:35:09.180 INFO:tasks.workunit.client.0.smithi107.stdout:6/670: write f6 [2421557,59552] 0 2024-04-22T23:35:09.180 INFO:tasks.workunit.client.0.smithi107.stdout:2/643: dread d1/d12/d35/d48/f3a [0,4194304] 0 2024-04-22T23:35:09.181 INFO:tasks.workunit.client.0.smithi107.stdout:2/644: chown d1/d12/d35/d48/f9d 1384 1 2024-04-22T23:35:09.181 INFO:tasks.workunit.client.0.smithi107.stdout:2/645: dread - d1/d12/fa4 zero size 2024-04-22T23:35:09.181 INFO:tasks.workunit.client.0.smithi107.stdout:2/646: write d1/d67/f5d [355873,130198] 0 2024-04-22T23:35:09.181 INFO:tasks.workunit.client.1.smithi157.stdout:8/611: dwrite d2/db1/f59 [0,4194304] 0 2024-04-22T23:35:09.181 INFO:tasks.workunit.client.1.smithi157.stdout:8/612: fdatasync d2/db1/d58/fa5 0 2024-04-22T23:35:09.182 INFO:tasks.workunit.client.0.smithi107.stdout:9/631: unlink d5/d14/d2e/d4f/d83/l97 0 2024-04-22T23:35:09.183 INFO:tasks.workunit.client.0.smithi107.stdout:0/609: getdents d1/d27/d20/d40 0 2024-04-22T23:35:09.184 INFO:tasks.workunit.client.0.smithi107.stdout:2/647: dread d1/d12/d35/d48/fb3 [0,4194304] 0 2024-04-22T23:35:09.186 INFO:tasks.workunit.client.0.smithi107.stdout:8/612: mkdir d0/d3a/dd4 0 2024-04-22T23:35:09.187 INFO:tasks.workunit.client.1.smithi157.stdout:8/613: link d2/d4/dbd/fb3 d2/d6/fd3 0 2024-04-22T23:35:09.187 INFO:tasks.workunit.client.1.smithi157.stdout:8/614: creat d2/d34/fd4 x:0 0 0 2024-04-22T23:35:09.188 INFO:tasks.workunit.client.1.smithi157.stdout:8/615: rename d2/db1/d58 to d2/db1/d58/dd5 22 2024-04-22T23:35:09.188 INFO:tasks.workunit.client.1.smithi157.stdout:8/616: write d2/d4/d30/db8/fc6 [237659,112057] 0 2024-04-22T23:35:09.189 INFO:tasks.workunit.client.0.smithi107.stdout:2/648: write d1/d12/f34 [78392,103723] 0 2024-04-22T23:35:09.189 INFO:tasks.workunit.client.0.smithi107.stdout:4/654: dread d1/d21/d29/d2e/f46 [0,4194304] 0 2024-04-22T23:35:09.197 INFO:tasks.workunit.client.0.smithi107.stdout:6/671: creat db/d20/d2a/d38/fc9 x:0 0 0 2024-04-22T23:35:09.197 INFO:tasks.workunit.client.0.smithi107.stdout:6/672: write db/f29 [6182195,116618] 0 2024-04-22T23:35:09.197 INFO:tasks.workunit.client.0.smithi107.stdout:6/673: fsync db/d20/d2a/d38/d61/d6f/da2/fac 0 2024-04-22T23:35:09.198 INFO:tasks.workunit.client.0.smithi107.stdout:8/613: dread d0/d7/d8/f1e [0,4194304] 0 2024-04-22T23:35:09.203 INFO:tasks.workunit.client.0.smithi107.stdout:2/649: link d1/d12/d35/d48/f2c d1/d2b/d88/dc9/fd8 0 2024-04-22T23:35:09.203 INFO:tasks.workunit.client.0.smithi107.stdout:2/650: chown d1/d67/c82 994386969 1 2024-04-22T23:35:09.204 INFO:tasks.workunit.client.0.smithi107.stdout:2/651: dread d1/d12/d35/d48/f3e [0,4194304] 0 2024-04-22T23:35:09.204 INFO:tasks.workunit.client.0.smithi107.stdout:2/652: stat d1/d12/d35/d48/d13/d4f/l9c 0 2024-04-22T23:35:09.205 INFO:tasks.workunit.client.0.smithi107.stdout:4/655: rename d1/d21/d29/f41 to d1/d21/d29/d2e/d88/fe4 0 2024-04-22T23:35:09.205 INFO:tasks.workunit.client.0.smithi107.stdout:4/656: dread - d1/d21/d29/d2e/d88/d99/dd2/fc8 zero size 2024-04-22T23:35:09.205 INFO:tasks.workunit.client.0.smithi107.stdout:4/657: fsync d1/d7/d63/d78/d96/da0/fa2 0 2024-04-22T23:35:09.206 INFO:tasks.workunit.client.1.smithi157.stdout:3/831: dwrite d0/d3/d34/d4a/fd7 [0,4194304] 0 2024-04-22T23:35:09.206 INFO:tasks.workunit.client.0.smithi107.stdout:6/674: unlink db/d20/d2a/d38/d59/d70/f8e 0 2024-04-22T23:35:09.206 INFO:tasks.workunit.client.0.smithi107.stdout:6/675: chown db/d20/d2c/d3b/f43 326 1 2024-04-22T23:35:09.206 INFO:tasks.workunit.client.0.smithi107.stdout:0/610: rmdir d1/d9/d14 39 2024-04-22T23:35:09.208 INFO:tasks.workunit.client.1.smithi157.stdout:3/832: rename d0/d3/d34/db9/f8c to d0/d3/d34/d3d/df6/d111/f117 0 2024-04-22T23:35:09.208 INFO:tasks.workunit.client.1.smithi157.stdout:3/833: chown d0/d3/d34/d3d/df6 28091 1 2024-04-22T23:35:09.209 INFO:tasks.workunit.client.0.smithi107.stdout:6/676: mkdir db/d20/d2a/dca 0 2024-04-22T23:35:09.210 INFO:tasks.workunit.client.1.smithi157.stdout:3/834: symlink d0/d3/d34/d39/l118 0 2024-04-22T23:35:09.210 INFO:tasks.workunit.client.0.smithi107.stdout:6/677: creat db/fcb x:0 0 0 2024-04-22T23:35:09.212 INFO:tasks.workunit.client.0.smithi107.stdout:6/678: symlink db/d20/d2a/d38/d4b/lcc 0 2024-04-22T23:35:09.212 INFO:tasks.workunit.client.1.smithi157.stdout:3/835: mkdir d0/d83/df9/d119 0 2024-04-22T23:35:09.212 INFO:tasks.workunit.client.1.smithi157.stdout:3/836: creat d0/d3/d34/d3d/df6/dee/d9e/dac/f11a x:0 0 0 2024-04-22T23:35:09.214 INFO:tasks.workunit.client.1.smithi157.stdout:3/837: creat d0/d3/d34/d3d/d57/d114/f11b x:0 0 0 2024-04-22T23:35:09.216 INFO:tasks.workunit.client.1.smithi157.stdout:3/838: mkdir d0/d3/d34/d3d/d57/d60/d6b/d11c 0 2024-04-22T23:35:09.216 INFO:tasks.workunit.client.1.smithi157.stdout:3/839: chown d0/d3/d34/d47/dcd 1565350 1 2024-04-22T23:35:09.216 INFO:tasks.workunit.client.0.smithi107.stdout:3/612: sync 2024-04-22T23:35:09.218 INFO:tasks.workunit.client.0.smithi107.stdout:5/647: dwrite d9/f4a [0,4194304] 0 2024-04-22T23:35:09.218 INFO:tasks.workunit.client.0.smithi107.stdout:5/648: creat d9/db/de/fd9 x:0 0 0 2024-04-22T23:35:09.218 INFO:tasks.workunit.client.1.smithi157.stdout:4/860: dwrite d3/d11/d1d/f11e [0,4194304] 0 2024-04-22T23:35:09.219 INFO:tasks.workunit.client.1.smithi157.stdout:4/861: write d3/d96/d40/dc3/fdb [767893,122651] 0 2024-04-22T23:35:09.219 INFO:tasks.workunit.client.1.smithi157.stdout:3/840: creat d0/d3/d34/d3d/d57/d60/d6b/d97/f11d x:0 0 0 2024-04-22T23:35:09.221 INFO:tasks.workunit.client.0.smithi107.stdout:3/613: creat db/d21/d27/d15/db8/fd3 x:0 0 0 2024-04-22T23:35:09.221 INFO:tasks.workunit.client.1.smithi157.stdout:0/721: dwrite d17/f25 [0,4194304] 0 2024-04-22T23:35:09.221 INFO:tasks.workunit.client.0.smithi107.stdout:9/632: dwrite d5/d14/d2e/d4f/d73/f9b [0,4194304] 0 2024-04-22T23:35:09.222 INFO:tasks.workunit.client.1.smithi157.stdout:4/862: mknod d3/d11/d34/c123 0 2024-04-22T23:35:09.224 INFO:tasks.workunit.client.0.smithi107.stdout:1/565: sync 2024-04-22T23:35:09.224 INFO:tasks.workunit.client.0.smithi107.stdout:1/566: stat d7/d19/d20/d39/d53/f5f 0 2024-04-22T23:35:09.227 INFO:tasks.workunit.client.0.smithi107.stdout:3/614: mknod db/d21/cd4 0 2024-04-22T23:35:09.227 INFO:tasks.workunit.client.0.smithi107.stdout:9/633: link d5/d7/d17/d1b/d1f/d36/d53/d55/l86 d5/d29/ldc 0 2024-04-22T23:35:09.227 INFO:tasks.workunit.client.0.smithi107.stdout:3/615: getdents db/d21/d27/d15/d29/d2a/d45 0 2024-04-22T23:35:09.228 INFO:tasks.workunit.client.0.smithi107.stdout:3/616: fdatasync db/d21/d27/d15/d29/d2a/d45/f8a 0 2024-04-22T23:35:09.229 INFO:tasks.workunit.client.0.smithi107.stdout:6/679: dread db/d20/d2a/d38/f3e [4194304,4194304] 0 2024-04-22T23:35:09.234 INFO:tasks.workunit.client.0.smithi107.stdout:3/617: symlink db/d21/d27/d15/ld5 0 2024-04-22T23:35:09.234 INFO:tasks.workunit.client.0.smithi107.stdout:3/618: write db/d21/d27/d15/fb4 [15925,59892] 0 2024-04-22T23:35:09.234 INFO:tasks.workunit.client.0.smithi107.stdout:6/680: mkdir db/d20/d2a/d38/d61/d6f/da2/dcd 0 2024-04-22T23:35:09.234 INFO:tasks.workunit.client.0.smithi107.stdout:6/681: creat db/d20/d2c/d3b/fce x:0 0 0 2024-04-22T23:35:09.234 INFO:tasks.workunit.client.0.smithi107.stdout:7/588: sync 2024-04-22T23:35:09.234 INFO:tasks.workunit.client.0.smithi107.stdout:3/619: creat db/d21/d27/d15/d29/d4f/db2/dc5/fd6 x:0 0 0 2024-04-22T23:35:09.235 INFO:tasks.workunit.client.1.smithi157.stdout:0/722: rename d17/d2b/d5b/c5e to d17/d23/d34/d67/d7f/d69/dce/cf1 0 2024-04-22T23:35:09.236 INFO:tasks.workunit.client.1.smithi157.stdout:4/863: rename d3/d11/d1d/d73/d80/dac/db5/cd5 to d3/d11/d1d/d73/da3/d31/d91/dcb/c124 0 2024-04-22T23:35:09.236 INFO:tasks.workunit.client.1.smithi157.stdout:4/864: write d3/d11/d1d/d73/d80/f8c [2119419,125195] 0 2024-04-22T23:35:09.239 INFO:tasks.workunit.client.0.smithi107.stdout:3/620: rmdir db/d21/d27/d15/d24/d66/dc9 0 2024-04-22T23:35:09.239 INFO:tasks.workunit.client.0.smithi107.stdout:3/621: dread - db/d21/d27/d15/d29/d2a/fae zero size 2024-04-22T23:35:09.242 INFO:tasks.workunit.client.0.smithi107.stdout:7/589: rmdir d7/d21/d49 39 2024-04-22T23:35:09.243 INFO:tasks.workunit.client.0.smithi107.stdout:7/590: chown d7/d21/d48/f5b 1062581 1 2024-04-22T23:35:09.243 INFO:tasks.workunit.client.0.smithi107.stdout:7/591: chown d7/d21/da4/d81 15059 1 2024-04-22T23:35:09.244 INFO:tasks.workunit.client.0.smithi107.stdout:9/634: dread d5/d24/f3c [0,4194304] 0 2024-04-22T23:35:09.245 INFO:tasks.workunit.client.0.smithi107.stdout:9/635: chown f4 51 1 2024-04-22T23:35:09.245 INFO:tasks.workunit.client.0.smithi107.stdout:7/592: mkdir d7/d21/da4/dbf 0 2024-04-22T23:35:09.247 INFO:tasks.workunit.client.0.smithi107.stdout:2/653: dwrite d1/d2b/d3d/d57/f83 [4194304,4194304] 0 2024-04-22T23:35:09.247 INFO:tasks.workunit.client.0.smithi107.stdout:2/654: chown d1/d2b/d3d/d57/c9b 9 1 2024-04-22T23:35:09.247 INFO:tasks.workunit.client.0.smithi107.stdout:2/655: write d1/d12/d35/d48/d13/f68 [568443,52544] 0 2024-04-22T23:35:09.247 INFO:tasks.workunit.client.0.smithi107.stdout:2/656: write d1/d12/d35/d54/f5f [115740,41107] 0 2024-04-22T23:35:09.247 INFO:tasks.workunit.client.0.smithi107.stdout:2/657: fdatasync d1/d2b/d3d/d57/f97 0 2024-04-22T23:35:09.247 INFO:tasks.workunit.client.0.smithi107.stdout:7/593: unlink d7/d1b/la6 0 2024-04-22T23:35:09.249 INFO:tasks.workunit.client.0.smithi107.stdout:2/658: mknod d1/d12/d35/d48/d13/d4f/cd9 0 2024-04-22T23:35:09.249 INFO:tasks.workunit.client.0.smithi107.stdout:2/659: dread - d1/d12/d35/d54/f73 zero size 2024-04-22T23:35:09.257 INFO:tasks.workunit.client.0.smithi107.stdout:5/649: dwrite d9/db/d94/fd3 [0,4194304] 0 2024-04-22T23:35:09.257 INFO:tasks.workunit.client.0.smithi107.stdout:5/650: write f7 [4590254,118734] 0 2024-04-22T23:35:09.266 INFO:tasks.workunit.client.0.smithi107.stdout:5/651: dread d9/d26/f41 [0,4194304] 0 2024-04-22T23:35:09.267 INFO:tasks.workunit.client.0.smithi107.stdout:5/652: stat d9/db/d42/f4b 0 2024-04-22T23:35:09.268 INFO:tasks.workunit.client.0.smithi107.stdout:5/653: rename d9/db/l57 to d9/d76/lda 0 2024-04-22T23:35:09.271 INFO:tasks.workunit.client.0.smithi107.stdout:2/660: dread d1/d2b/d49/d7e/fbe [0,4194304] 0 2024-04-22T23:35:09.272 INFO:tasks.workunit.client.0.smithi107.stdout:2/661: rmdir d1/d12/d35/d48 39 2024-04-22T23:35:09.282 INFO:tasks.workunit.client.0.smithi107.stdout:7/594: dread d7/d21/f96 [0,4194304] 0 2024-04-22T23:35:09.283 INFO:tasks.workunit.client.0.smithi107.stdout:7/595: mknod d7/d21/d49/cc0 0 2024-04-22T23:35:09.284 INFO:tasks.workunit.client.0.smithi107.stdout:7/596: creat d7/d21/da4/d66/fc1 x:0 0 0 2024-04-22T23:35:09.284 INFO:tasks.workunit.client.0.smithi107.stdout:7/597: chown d7/d1b/fa2 1080882 1 2024-04-22T23:35:09.290 INFO:tasks.workunit.client.1.smithi157.stdout:4/865: dread d3/d11/d1d/f52 [0,4194304] 0 2024-04-22T23:35:09.291 INFO:tasks.workunit.client.1.smithi157.stdout:4/866: mkdir d3/d96/d4c/de7/d125 0 2024-04-22T23:35:09.293 INFO:tasks.workunit.client.1.smithi157.stdout:4/867: rmdir d3/d11/d1d/d73/da7 39 2024-04-22T23:35:09.301 INFO:tasks.workunit.client.0.smithi107.stdout:2/662: write d1/fcb [1185827,108686] 0 2024-04-22T23:35:09.305 INFO:tasks.workunit.client.0.smithi107.stdout:2/663: dread d1/d67/f5d [0,4194304] 0 2024-04-22T23:35:09.306 INFO:tasks.workunit.client.0.smithi107.stdout:2/664: dread d1/d67/f6f [0,4194304] 0 2024-04-22T23:35:09.308 INFO:tasks.workunit.client.0.smithi107.stdout:2/665: mknod d1/d2b/d49/d7e/da8/cda 0 2024-04-22T23:35:09.308 INFO:tasks.workunit.client.0.smithi107.stdout:8/614: dwrite d0/d7/df/d3b/faf [0,4194304] 0 2024-04-22T23:35:09.310 INFO:tasks.workunit.client.1.smithi157.stdout:4/868: write d3/d11/d1d/d73/da7/fc5 [1229007,37131] 0 2024-04-22T23:35:09.310 INFO:tasks.workunit.client.0.smithi107.stdout:2/666: unlink d1/d12/d35/d48/f3a 0 2024-04-22T23:35:09.311 INFO:tasks.workunit.client.0.smithi107.stdout:7/598: write d7/d1b/f4d [7707500,13112] 0 2024-04-22T23:35:09.311 INFO:tasks.workunit.client.1.smithi157.stdout:4/869: creat d3/d11/d1d/d73/d80/dac/db5/df7/d111/f126 x:0 0 0 2024-04-22T23:35:09.311 INFO:tasks.workunit.client.1.smithi157.stdout:4/870: write d3/d11/d1d/d10b/f116 [233980,104402] 0 2024-04-22T23:35:09.312 INFO:tasks.workunit.client.0.smithi107.stdout:0/611: dwrite d1/d27/d20/f36 [0,4194304] 0 2024-04-22T23:35:09.312 INFO:tasks.workunit.client.0.smithi107.stdout:3/622: dwrite db/d21/d27/d15/d29/d4f/db2/fd1 [0,4194304] 0 2024-04-22T23:35:09.313 INFO:tasks.workunit.client.1.smithi157.stdout:0/723: dwrite d17/d2b/d4f/d5a/f81 [0,4194304] 0 2024-04-22T23:35:09.313 INFO:tasks.workunit.client.0.smithi107.stdout:4/658: dwrite d1/d7/f11 [0,4194304] 0 2024-04-22T23:35:09.313 INFO:tasks.workunit.client.0.smithi107.stdout:1/567: dwrite d7/f67 [0,4194304] 0 2024-04-22T23:35:09.316 INFO:tasks.workunit.client.0.smithi107.stdout:8/615: creat d0/d85/d51/fd5 x:0 0 0 2024-04-22T23:35:09.316 INFO:tasks.workunit.client.0.smithi107.stdout:8/616: fsync d0/d7/df/fca 0 2024-04-22T23:35:09.316 INFO:tasks.workunit.client.0.smithi107.stdout:9/636: dwrite d5/d7/d27/d46/f88 [0,4194304] 0 2024-04-22T23:35:09.317 INFO:tasks.workunit.client.0.smithi107.stdout:7/599: unlink d7/d1b/l7d 0 2024-04-22T23:35:09.317 INFO:tasks.workunit.client.1.smithi157.stdout:3/841: dwrite d0/d3/d34/d39/f18 [0,4194304] 0 2024-04-22T23:35:09.317 INFO:tasks.workunit.client.1.smithi157.stdout:0/724: symlink d17/d38/lf2 0 2024-04-22T23:35:09.317 INFO:tasks.workunit.client.1.smithi157.stdout:8/617: dwrite d2/d4/f1c [0,4194304] 0 2024-04-22T23:35:09.319 INFO:tasks.workunit.client.1.smithi157.stdout:3/842: unlink d0/d3/d34/d39/f66 0 2024-04-22T23:35:09.319 INFO:tasks.workunit.client.1.smithi157.stdout:3/843: chown d0/d3/d34/d3d/d57/d60/d9c/fc8 4245 1 2024-04-22T23:35:09.321 INFO:tasks.workunit.client.1.smithi157.stdout:8/618: creat d2/db1/d58/fd6 x:0 0 0 2024-04-22T23:35:09.322 INFO:tasks.workunit.client.1.smithi157.stdout:3/844: mkdir d0/d83/dbc/d11e 0 2024-04-22T23:35:09.323 INFO:tasks.workunit.client.0.smithi107.stdout:2/667: dread d1/d2b/d49/fc5 [0,4194304] 0 2024-04-22T23:35:09.324 INFO:tasks.workunit.client.1.smithi157.stdout:8/619: mknod d2/d6/d21/d5f/dcd/cd7 0 2024-04-22T23:35:09.326 INFO:tasks.workunit.client.0.smithi107.stdout:3/623: write db/d21/d27/d15/d29/d2a/f43 [139763,130851] 0 2024-04-22T23:35:09.327 INFO:tasks.workunit.client.1.smithi157.stdout:3/845: mknod d0/d3/d34/d3d/d57/d60/c11f 0 2024-04-22T23:35:09.327 INFO:tasks.workunit.client.1.smithi157.stdout:0/725: dread d17/d23/d34/f7b [0,4194304] 0 2024-04-22T23:35:09.328 INFO:tasks.workunit.client.1.smithi157.stdout:0/726: write d17/d2b/fb4 [1416618,82886] 0 2024-04-22T23:35:09.328 INFO:tasks.workunit.client.1.smithi157.stdout:0/727: fsync d17/d23/d34/d67/d7f/d69/f89 0 2024-04-22T23:35:09.328 INFO:tasks.workunit.client.1.smithi157.stdout:0/728: chown d17/d2b 1464699 1 2024-04-22T23:35:09.328 INFO:tasks.workunit.client.1.smithi157.stdout:0/729: dread - d17/d23/fbf zero size 2024-04-22T23:35:09.328 INFO:tasks.workunit.client.0.smithi107.stdout:4/659: symlink d1/d21/d85/le5 0 2024-04-22T23:35:09.329 INFO:tasks.workunit.client.0.smithi107.stdout:4/660: chown d1/d7/d63/d78/d96/da0/fa2 100 1 2024-04-22T23:35:09.329 INFO:tasks.workunit.client.0.smithi107.stdout:4/661: stat d1/d21/d29/d2e/d7e/lcc 0 2024-04-22T23:35:09.329 INFO:tasks.workunit.client.0.smithi107.stdout:4/662: chown d1/d1b/c7b 88 1 2024-04-22T23:35:09.329 INFO:tasks.workunit.client.0.smithi107.stdout:4/663: readlink d1/d7/d63/d78/l94 0 2024-04-22T23:35:09.329 INFO:tasks.workunit.client.0.smithi107.stdout:8/617: dread d0/d7/fa [0,4194304] 0 2024-04-22T23:35:09.330 INFO:tasks.workunit.client.1.smithi157.stdout:0/730: mknod d17/d2b/d71/da5/cf3 0 2024-04-22T23:35:09.332 INFO:tasks.workunit.client.1.smithi157.stdout:0/731: creat d17/d2b/d5b/ff4 x:0 0 0 2024-04-22T23:35:09.333 INFO:tasks.workunit.client.0.smithi107.stdout:4/664: dread d1/d7/d63/d78/d80/f82 [0,4194304] 0 2024-04-22T23:35:09.333 INFO:tasks.workunit.client.0.smithi107.stdout:4/665: chown d1/d1b/fca 149286 1 2024-04-22T23:35:09.333 INFO:tasks.workunit.client.0.smithi107.stdout:4/666: stat d1/d7/d53/d66/d6b/ddb 0 2024-04-22T23:35:09.333 INFO:tasks.workunit.client.1.smithi157.stdout:9/788: sync 2024-04-22T23:35:09.334 INFO:tasks.workunit.client.1.smithi157.stdout:7/759: sync 2024-04-22T23:35:09.334 INFO:tasks.workunit.client.1.smithi157.stdout:1/859: sync 2024-04-22T23:35:09.334 INFO:tasks.workunit.client.1.smithi157.stdout:5/868: sync 2024-04-22T23:35:09.334 INFO:tasks.workunit.client.0.smithi107.stdout:1/568: dwrite d7/d19/d20/d39/d53/d43/f7a [0,4194304] 0 2024-04-22T23:35:09.335 INFO:tasks.workunit.client.1.smithi157.stdout:7/760: mkdir d0/d9/d42/d55/d56/d91/de6 0 2024-04-22T23:35:09.336 INFO:tasks.workunit.client.1.smithi157.stdout:2/938: sync 2024-04-22T23:35:09.337 INFO:tasks.workunit.client.1.smithi157.stdout:9/789: mknod d4/df/d57/dbd/cf4 0 2024-04-22T23:35:09.337 INFO:tasks.workunit.client.1.smithi157.stdout:9/790: readlink d4/d15/d23/ld5 0 2024-04-22T23:35:09.337 INFO:tasks.workunit.client.1.smithi157.stdout:9/791: fsync d4/d2d/fb4 0 2024-04-22T23:35:09.337 INFO:tasks.workunit.client.1.smithi157.stdout:9/792: fdatasync d4/d15/d23/f88 0 2024-04-22T23:35:09.337 INFO:tasks.workunit.client.1.smithi157.stdout:9/793: read d4/d2d/f78 [1693679,57831] 0 2024-04-22T23:35:09.337 INFO:tasks.workunit.client.1.smithi157.stdout:9/794: truncate d4/f29 1545369 0 2024-04-22T23:35:09.342 INFO:tasks.workunit.client.1.smithi157.stdout:3/846: getdents d0/d3/d34/d39 0 2024-04-22T23:35:09.344 INFO:tasks.workunit.client.1.smithi157.stdout:9/795: dread d4/df/d57/fb0 [0,4194304] 0 2024-04-22T23:35:09.347 INFO:tasks.workunit.client.1.smithi157.stdout:7/761: dread d0/d9/db/f61 [0,4194304] 0 2024-04-22T23:35:09.347 INFO:tasks.workunit.client.1.smithi157.stdout:3/847: getdents d0/d3/d34/d47 0 2024-04-22T23:35:09.348 INFO:tasks.workunit.client.1.smithi157.stdout:9/796: mkdir d4/df/d57/df5 0 2024-04-22T23:35:09.349 INFO:tasks.workunit.client.1.smithi157.stdout:7/762: symlink d0/d9/d42/d55/d80/le7 0 2024-04-22T23:35:09.351 INFO:tasks.workunit.client.1.smithi157.stdout:5/869: dread d1/f90 [0,4194304] 0 2024-04-22T23:35:09.351 INFO:tasks.workunit.client.1.smithi157.stdout:5/870: write d1/d5/dc/d43/d4f/d7f/d99/f123 [593069,82727] 0 2024-04-22T23:35:09.351 INFO:tasks.workunit.client.1.smithi157.stdout:5/871: readlink d1/d5/d7/d2f/d45/l4c 0 2024-04-22T23:35:09.352 INFO:tasks.workunit.client.0.smithi107.stdout:9/637: symlink d5/d7/d17/d1b/ldd 0 2024-04-22T23:35:09.352 INFO:tasks.workunit.client.1.smithi157.stdout:3/848: rename d0/d3/d34/d3d/f101 to d0/d83/df9/f120 0 2024-04-22T23:35:09.352 INFO:tasks.workunit.client.1.smithi157.stdout:3/849: dread - d0/d3/d34/d3d/df6/d72/ff2 zero size 2024-04-22T23:35:09.353 INFO:tasks.workunit.client.1.smithi157.stdout:8/620: dwrite d2/d6/d21/d5f/fd [0,4194304] 0 2024-04-22T23:35:09.353 INFO:tasks.workunit.client.1.smithi157.stdout:7/763: mkdir d0/d9/d42/d55/d80/de8 0 2024-04-22T23:35:09.353 INFO:tasks.workunit.client.1.smithi157.stdout:7/764: readlink d0/l3 0 2024-04-22T23:35:09.354 INFO:tasks.workunit.client.0.smithi107.stdout:7/600: creat d7/d21/d48/fc2 x:0 0 0 2024-04-22T23:35:09.355 INFO:tasks.workunit.client.1.smithi157.stdout:5/872: unlink d1/d5/d7/d2f/d45/d54/ldb 0 2024-04-22T23:35:09.355 INFO:tasks.workunit.client.1.smithi157.stdout:5/873: fsync d1/d27/d29/f5f 0 2024-04-22T23:35:09.356 INFO:tasks.workunit.client.1.smithi157.stdout:3/850: rename d0/d3/d13/f2b to d0/d3/d34/d3d/df6/dee/d9e/f121 0 2024-04-22T23:35:09.357 INFO:tasks.workunit.client.0.smithi107.stdout:2/668: link d1/d12/d35/d54/c71 d1/d12/cdb 0 2024-04-22T23:35:09.357 INFO:tasks.workunit.client.0.smithi107.stdout:2/669: stat d1/d12/d35/d48/d13/l8b 0 2024-04-22T23:35:09.357 INFO:tasks.workunit.client.0.smithi107.stdout:3/624: write db/d21/d27/d36/f7e [5090768,28264] 0 2024-04-22T23:35:09.358 INFO:tasks.workunit.client.0.smithi107.stdout:8/618: creat d0/d7/d20/d2d/fd6 x:0 0 0 2024-04-22T23:35:09.362 INFO:tasks.workunit.client.0.smithi107.stdout:1/569: symlink d7/d19/d2d/lbc 0 2024-04-22T23:35:09.366 INFO:tasks.workunit.client.1.smithi157.stdout:8/621: mkdir d2/d4/dbd/dd8 0 2024-04-22T23:35:09.366 INFO:tasks.workunit.client.1.smithi157.stdout:5/874: creat d1/d5/d7/f142 x:0 0 0 2024-04-22T23:35:09.366 INFO:tasks.workunit.client.1.smithi157.stdout:5/875: truncate d1/fe 4420561 0 2024-04-22T23:35:09.366 INFO:tasks.workunit.client.1.smithi157.stdout:5/876: chown d1/d1f 500 1 2024-04-22T23:35:09.367 INFO:tasks.workunit.client.1.smithi157.stdout:7/765: creat d0/d9/d42/d55/d80/d95/fe9 x:0 0 0 2024-04-22T23:35:09.367 INFO:tasks.workunit.client.1.smithi157.stdout:3/851: symlink d0/d3/d34/d3d/d57/l122 0 2024-04-22T23:35:09.367 INFO:tasks.workunit.client.1.smithi157.stdout:5/877: symlink d1/db0/d7a/d133/d118/l143 0 2024-04-22T23:35:09.368 INFO:tasks.workunit.client.1.smithi157.stdout:5/878: mknod d1/d1f/dc6/c144 0 2024-04-22T23:35:09.369 INFO:tasks.workunit.client.0.smithi107.stdout:0/612: rmdir d1/d7c 0 2024-04-22T23:35:09.372 INFO:tasks.workunit.client.0.smithi107.stdout:2/670: mkdir d1/d2b/d49/d7e/da8/ddc 0 2024-04-22T23:35:09.373 INFO:tasks.workunit.client.1.smithi157.stdout:5/879: rename d1/d5/dc/d43/d4f/d7f/d99 to d1/d5/dc/d43/d5e/d145 0 2024-04-22T23:35:09.374 INFO:tasks.workunit.client.0.smithi107.stdout:8/619: creat d0/d3a/dd4/fd7 x:0 0 0 2024-04-22T23:35:09.374 INFO:tasks.workunit.client.0.smithi107.stdout:8/620: write d0/d7/d8/d41/f9a [322835,117525] 0 2024-04-22T23:35:09.376 INFO:tasks.workunit.client.1.smithi157.stdout:5/880: link d1/d5/dc/f38 d1/db0/dff/f146 0 2024-04-22T23:35:09.380 INFO:tasks.workunit.client.0.smithi107.stdout:7/601: symlink d7/d21/d48/lc3 0 2024-04-22T23:35:09.380 INFO:tasks.workunit.client.0.smithi107.stdout:7/602: write d7/f26 [2947715,128034] 0 2024-04-22T23:35:09.381 INFO:tasks.workunit.client.0.smithi107.stdout:2/671: mkdir d1/d12/d35/d48/d13/d4f/da5/ddd 0 2024-04-22T23:35:09.382 INFO:tasks.workunit.client.0.smithi107.stdout:8/621: mknod d0/d3a/d46/cd8 0 2024-04-22T23:35:09.383 INFO:tasks.workunit.client.1.smithi157.stdout:1/860: dwrite d4/d17/d34/d3b/d52/fbe [0,4194304] 0 2024-04-22T23:35:09.383 INFO:tasks.workunit.client.0.smithi107.stdout:1/570: getdents d7/d19/d20/d39 0 2024-04-22T23:35:09.384 INFO:tasks.workunit.client.1.smithi157.stdout:7/766: dread d0/d9/de/f5f [0,4194304] 0 2024-04-22T23:35:09.385 INFO:tasks.workunit.client.0.smithi107.stdout:9/638: rename d5/d14/d2e/d43 to d5/d29/d7b/d91/dde 0 2024-04-22T23:35:09.386 INFO:tasks.workunit.client.0.smithi107.stdout:7/603: readlink d7/d4c/l8a 0 2024-04-22T23:35:09.387 INFO:tasks.workunit.client.1.smithi157.stdout:0/732: dwrite d17/d23/d34/dee/da8/fea [0,4194304] 0 2024-04-22T23:35:09.387 INFO:tasks.workunit.client.1.smithi157.stdout:0/733: dread - d17/d23/d34/d67/d78/fbb zero size 2024-04-22T23:35:09.387 INFO:tasks.workunit.client.1.smithi157.stdout:1/861: symlink d4/d17/d34/d3b/d11a/l12d 0 2024-04-22T23:35:09.388 INFO:tasks.workunit.client.0.smithi107.stdout:8/622: mkdir d0/d7/df/d3b/dd9 0 2024-04-22T23:35:09.388 INFO:tasks.workunit.client.0.smithi107.stdout:8/623: write d0/d7/df/d17/f5b [742406,76545] 0 2024-04-22T23:35:09.390 INFO:tasks.workunit.client.1.smithi157.stdout:0/734: creat d17/d2b/d4f/dc5/dca/ff5 x:0 0 0 2024-04-22T23:35:09.390 INFO:tasks.workunit.client.1.smithi157.stdout:0/735: truncate d17/d38/f65 1699398 0 2024-04-22T23:35:09.390 INFO:tasks.workunit.client.1.smithi157.stdout:0/736: write f11 [4418850,5834] 0 2024-04-22T23:35:09.391 INFO:tasks.workunit.client.0.smithi107.stdout:7/604: dread d7/d21/d48/d5c/f71 [0,4194304] 0 2024-04-22T23:35:09.394 INFO:tasks.workunit.client.0.smithi107.stdout:7/605: getdents d7/d21/d48/d95 0 2024-04-22T23:35:09.394 INFO:tasks.workunit.client.0.smithi107.stdout:7/606: chown d7/d21/d54/f90 0 1 2024-04-22T23:35:09.394 INFO:tasks.workunit.client.0.smithi107.stdout:7/607: truncate d7/f63 4967696 0 2024-04-22T23:35:09.394 INFO:tasks.workunit.client.0.smithi107.stdout:7/608: creat d7/d21/d7e/d94/fc4 x:0 0 0 2024-04-22T23:35:09.394 INFO:tasks.workunit.client.1.smithi157.stdout:0/737: rename d17/d2b/d52/d7e to d17/d2b/d4f/dc5/dcc/de9/df6 0 2024-04-22T23:35:09.395 INFO:tasks.workunit.client.1.smithi157.stdout:2/939: dwrite d7/d8/d30/f83 [0,4194304] 0 2024-04-22T23:35:09.396 INFO:tasks.workunit.client.1.smithi157.stdout:0/738: rename d17/d38/l48 to d17/lf7 0 2024-04-22T23:35:09.397 INFO:tasks.workunit.client.0.smithi107.stdout:7/609: creat d7/d21/da4/d81/fc5 x:0 0 0 2024-04-22T23:35:09.397 INFO:tasks.workunit.client.0.smithi107.stdout:1/571: write d7/d19/d2d/d89/f94 [2425749,24328] 0 2024-04-22T23:35:09.397 INFO:tasks.workunit.client.0.smithi107.stdout:1/572: creat d7/d19/d20/fbd x:0 0 0 2024-04-22T23:35:09.399 INFO:tasks.workunit.client.1.smithi157.stdout:2/940: mkdir d7/d1f/d26/d8f/d131 0 2024-04-22T23:35:09.400 INFO:tasks.workunit.client.1.smithi157.stdout:0/739: symlink d17/d2b/d71/lf8 0 2024-04-22T23:35:09.400 INFO:tasks.workunit.client.1.smithi157.stdout:0/740: chown d17/d2b/f2c 31 1 2024-04-22T23:35:09.400 INFO:tasks.workunit.client.0.smithi107.stdout:7/610: dread d7/d21/d48/d5c/d9d/d6e/f70 [0,4194304] 0 2024-04-22T23:35:09.401 INFO:tasks.workunit.client.1.smithi157.stdout:2/941: mkdir d7/d1f/d26/d8f/d132 0 2024-04-22T23:35:09.403 INFO:tasks.workunit.client.1.smithi157.stdout:0/741: symlink d17/d2b/d71/db3/lf9 0 2024-04-22T23:35:09.403 INFO:tasks.workunit.client.0.smithi107.stdout:7/611: unlink d7/d21/d54/f58 0 2024-04-22T23:35:09.403 INFO:tasks.workunit.client.0.smithi107.stdout:8/624: read d0/d7/df/d33/d37/f49 [3305038,67859] 0 2024-04-22T23:35:09.403 INFO:tasks.workunit.client.0.smithi107.stdout:9/639: write d5/d7/f70 [1806660,128875] 0 2024-04-22T23:35:09.404 INFO:tasks.workunit.client.1.smithi157.stdout:0/742: mknod d17/d2b/d5b/cfa 0 2024-04-22T23:35:09.405 INFO:tasks.workunit.client.0.smithi107.stdout:7/612: link d7/d21/d48/lc3 d7/d21/d7e/d94/lc6 0 2024-04-22T23:35:09.405 INFO:tasks.workunit.client.1.smithi157.stdout:0/743: symlink d17/d2b/d4f/d5a/dd2/lfb 0 2024-04-22T23:35:09.405 INFO:tasks.workunit.client.1.smithi157.stdout:0/744: read - d17/d23/d34/d67/d7f/d69/f89 zero size 2024-04-22T23:35:09.406 INFO:tasks.workunit.client.1.smithi157.stdout:0/745: write d17/d23/d34/d67/d7f/f74 [1685055,85215] 0 2024-04-22T23:35:09.406 INFO:tasks.workunit.client.1.smithi157.stdout:2/942: dread d7/d8/dd/d4d/f6e [0,4194304] 0 2024-04-22T23:35:09.406 INFO:tasks.workunit.client.0.smithi107.stdout:8/625: unlink d0/d7/df/d33/d9b/ca0 0 2024-04-22T23:35:09.407 INFO:tasks.workunit.client.0.smithi107.stdout:9/640: symlink d5/d14/d2e/d4f/ldf 0 2024-04-22T23:35:09.407 INFO:tasks.workunit.client.0.smithi107.stdout:9/641: write d5/d7/d27/d46/f66 [1082405,17348] 0 2024-04-22T23:35:09.407 INFO:tasks.workunit.client.0.smithi107.stdout:9/642: readlink d5/d7/d27/l42 0 2024-04-22T23:35:09.407 INFO:tasks.workunit.client.0.smithi107.stdout:8/626: mkdir d0/d7/df/d3b/dd9/dda 0 2024-04-22T23:35:09.408 INFO:tasks.workunit.client.1.smithi157.stdout:9/797: dwrite d4/d15/d23/d27/d47/f74 [0,4194304] 0 2024-04-22T23:35:09.408 INFO:tasks.workunit.client.0.smithi107.stdout:4/667: dwrite d1/d7/d5b/dbf/fdc [0,4194304] 0 2024-04-22T23:35:09.409 INFO:tasks.workunit.client.1.smithi157.stdout:2/943: mkdir d7/d8/d30/db8/d133 0 2024-04-22T23:35:09.409 INFO:tasks.workunit.client.1.smithi157.stdout:9/798: dread d4/d15/d23/d27/fb1 [0,4194304] 0 2024-04-22T23:35:09.409 INFO:tasks.workunit.client.0.smithi107.stdout:9/643: unlink d5/d7/d17/c94 0 2024-04-22T23:35:09.410 INFO:tasks.workunit.client.1.smithi157.stdout:0/746: rename d17/d2b/d52/lbc to d17/d2b/d4f/dc5/dcc/de9/df6/lfc 0 2024-04-22T23:35:09.410 INFO:tasks.workunit.client.0.smithi107.stdout:8/627: symlink d0/d7/d20/d2d/ldb 0 2024-04-22T23:35:09.412 INFO:tasks.workunit.client.1.smithi157.stdout:2/944: creat d7/d8/d8e/d80/f134 x:0 0 0 2024-04-22T23:35:09.412 INFO:tasks.workunit.client.1.smithi157.stdout:2/945: read d7/d8/d95/dd2/dda/f8c [613970,124587] 0 2024-04-22T23:35:09.413 INFO:tasks.workunit.client.0.smithi107.stdout:2/672: dread d1/fcb [0,4194304] 0 2024-04-22T23:35:09.414 INFO:tasks.workunit.client.0.smithi107.stdout:2/673: creat d1/d2b/d3d/fde x:0 0 0 2024-04-22T23:35:09.414 INFO:tasks.workunit.client.0.smithi107.stdout:2/674: readlink d1/d12/d35/d48/d13/l76 0 2024-04-22T23:35:09.419 INFO:tasks.workunit.client.0.smithi107.stdout:4/668: getdents d1/d21 0 2024-04-22T23:35:09.421 INFO:tasks.workunit.client.0.smithi107.stdout:9/644: rename d5/d7/d17/d1b/d1f/d36/d53/d55 to d5/d7/d17/de0 0 2024-04-22T23:35:09.422 INFO:tasks.workunit.client.0.smithi107.stdout:0/613: dwrite d1/d27/d20/f36 [4194304,4194304] 0 2024-04-22T23:35:09.422 INFO:tasks.workunit.client.0.smithi107.stdout:2/675: symlink d1/d12/d35/d48/d13/ldf 0 2024-04-22T23:35:09.422 INFO:tasks.workunit.client.0.smithi107.stdout:8/628: dread d0/f2f [0,4194304] 0 2024-04-22T23:35:09.422 INFO:tasks.workunit.client.1.smithi157.stdout:7/767: dwrite d0/d9/d29/d33/d37/f4c [0,4194304] 0 2024-04-22T23:35:09.422 INFO:tasks.workunit.client.1.smithi157.stdout:7/768: read - d0/d9/db/df/d68/fca zero size 2024-04-22T23:35:09.423 INFO:tasks.workunit.client.1.smithi157.stdout:8/622: dwrite d2/d6/d21/d5f/d11/d13/f26 [0,4194304] 0 2024-04-22T23:35:09.423 INFO:tasks.workunit.client.1.smithi157.stdout:1/862: dwrite d4/d17/d34/d3b/d52/f7f [0,4194304] 0 2024-04-22T23:35:09.423 INFO:tasks.workunit.client.0.smithi107.stdout:4/669: creat d1/fe6 x:0 0 0 2024-04-22T23:35:09.424 INFO:tasks.workunit.client.0.smithi107.stdout:9/645: rename d5/d7/d17/d1b/l8f to d5/d29/dbb/le1 0 2024-04-22T23:35:09.430 INFO:tasks.workunit.client.0.smithi107.stdout:0/614: unlink d1/d84/c81 0 2024-04-22T23:35:09.430 INFO:tasks.workunit.client.0.smithi107.stdout:2/676: truncate d1/d63/f69 49604 0 2024-04-22T23:35:09.430 INFO:tasks.workunit.client.0.smithi107.stdout:2/677: chown d1/d63/c99 25 1 2024-04-22T23:35:09.431 INFO:tasks.workunit.client.1.smithi157.stdout:2/946: dread d7/d8/d95/fa8 [0,4194304] 0 2024-04-22T23:35:09.431 INFO:tasks.workunit.client.1.smithi157.stdout:3/852: dwrite d0/d3/d34/db9/fd4 [0,4194304] 0 2024-04-22T23:35:09.431 INFO:tasks.workunit.client.1.smithi157.stdout:2/947: read d7/d8/dca/f10e [1809077,127213] 0 2024-04-22T23:35:09.432 INFO:tasks.workunit.client.1.smithi157.stdout:7/769: dread d0/d9/d29/d33/d37/f87 [0,4194304] 0 2024-04-22T23:35:09.432 INFO:tasks.workunit.client.1.smithi157.stdout:8/623: mknod d2/d34/d6a/d77/cd9 0 2024-04-22T23:35:09.432 INFO:tasks.workunit.client.1.smithi157.stdout:8/624: chown d2/d6/d21/d5f/d17/dc2 88659 1 2024-04-22T23:35:09.438 INFO:tasks.workunit.client.0.smithi107.stdout:8/629: rename d0/d7/f35 to d0/d7/df/d3b/dd9/dda/fdc 0 2024-04-22T23:35:09.438 INFO:tasks.workunit.client.0.smithi107.stdout:8/630: fdatasync d0/d7/df/d3b/faf 0 2024-04-22T23:35:09.438 INFO:tasks.workunit.client.0.smithi107.stdout:9/646: mknod d5/d14/d2e/d4f/d83/ce2 0 2024-04-22T23:35:09.438 INFO:tasks.workunit.client.1.smithi157.stdout:1/863: symlink d4/d17/d34/d75/dc7/l12e 0 2024-04-22T23:35:09.438 INFO:tasks.workunit.client.1.smithi157.stdout:1/864: read - d4/d17/d1d/db2/ffa zero size 2024-04-22T23:35:09.438 INFO:tasks.workunit.client.1.smithi157.stdout:9/799: dwrite d4/d2d/fca [0,4194304] 0 2024-04-22T23:35:09.439 INFO:tasks.workunit.client.1.smithi157.stdout:9/800: chown d4/df/f6a 1301397 1 2024-04-22T23:35:09.439 INFO:tasks.workunit.client.1.smithi157.stdout:0/747: dwrite d17/d23/d34/dee/fcf [0,4194304] 0 2024-04-22T23:35:09.439 INFO:tasks.workunit.client.1.smithi157.stdout:8/625: mkdir d2/d4/dda 0 2024-04-22T23:35:09.439 INFO:tasks.workunit.client.1.smithi157.stdout:8/626: dread - d2/d6/fd3 zero size 2024-04-22T23:35:09.439 INFO:tasks.workunit.client.1.smithi157.stdout:8/627: creat d2/fdb x:0 0 0 2024-04-22T23:35:09.441 INFO:tasks.workunit.client.0.smithi107.stdout:0/615: truncate d1/d27/f21 1677520 0 2024-04-22T23:35:09.443 INFO:tasks.workunit.client.1.smithi157.stdout:1/865: mknod d4/db/d3d/dc1/d100/c12f 0 2024-04-22T23:35:09.447 INFO:tasks.workunit.client.1.smithi157.stdout:3/853: read d0/d3/d34/d39/f56 [1347376,87446] 0 2024-04-22T23:35:09.449 INFO:tasks.workunit.client.1.smithi157.stdout:8/628: creat d2/d6/d21/d5f/d11/fdc x:0 0 0 2024-04-22T23:35:09.449 INFO:tasks.workunit.client.1.smithi157.stdout:1/866: dread d4/d17/de5/f11f [0,4194304] 0 2024-04-22T23:35:09.451 INFO:tasks.workunit.client.0.smithi107.stdout:0/616: symlink d1/d27/d8a/ld3 0 2024-04-22T23:35:09.451 INFO:tasks.workunit.client.1.smithi157.stdout:3/854: symlink d0/d83/df9/d119/l123 0 2024-04-22T23:35:09.451 INFO:tasks.workunit.client.1.smithi157.stdout:8/629: unlink d2/d6/d21/f2e 0 2024-04-22T23:35:09.452 INFO:tasks.workunit.client.1.smithi157.stdout:8/630: readlink d2/d6/d21/d5f/d11/l68 0 2024-04-22T23:35:09.452 INFO:tasks.workunit.client.1.smithi157.stdout:0/748: dread d17/f25 [0,4194304] 0 2024-04-22T23:35:09.453 INFO:tasks.workunit.client.0.smithi107.stdout:0/617: mkdir d1/d9/dd4 0 2024-04-22T23:35:09.454 INFO:tasks.workunit.client.1.smithi157.stdout:0/749: mkdir d17/d2b/d4f/dc5/dcc/de9/df6/dfd 0 2024-04-22T23:35:09.455 INFO:tasks.workunit.client.0.smithi107.stdout:0/618: creat d1/d27/d20/d2c/d41/dbe/fd5 x:0 0 0 2024-04-22T23:35:09.455 INFO:tasks.workunit.client.0.smithi107.stdout:0/619: fdatasync d1/d27/d20/d2c/d2e/faf 0 2024-04-22T23:35:09.456 INFO:tasks.workunit.client.0.smithi107.stdout:9/647: getdents d5/d14/d2e 0 2024-04-22T23:35:09.458 INFO:tasks.workunit.client.0.smithi107.stdout:9/648: mkdir d5/d14/d28/de3 0 2024-04-22T23:35:09.461 INFO:tasks.workunit.client.1.smithi157.stdout:1/867: dread d4/f8e [0,4194304] 0 2024-04-22T23:35:09.461 INFO:tasks.workunit.client.1.smithi157.stdout:1/868: dread - d4/d17/d34/d3b/d50/ff3 zero size 2024-04-22T23:35:09.463 INFO:tasks.workunit.client.0.smithi107.stdout:4/670: dwrite d1/d21/d29/d2e/d88/fe4 [0,4194304] 0 2024-04-22T23:35:09.463 INFO:tasks.workunit.client.1.smithi157.stdout:0/750: dread d17/d2b/f86 [0,4194304] 0 2024-04-22T23:35:09.463 INFO:tasks.workunit.client.1.smithi157.stdout:0/751: fsync d17/d23/d9c/fdd 0 2024-04-22T23:35:09.463 INFO:tasks.workunit.client.1.smithi157.stdout:0/752: stat fc 0 2024-04-22T23:35:09.464 INFO:tasks.workunit.client.0.smithi107.stdout:2/678: dwrite d1/d67/f5d [0,4194304] 0 2024-04-22T23:35:09.464 INFO:tasks.workunit.client.1.smithi157.stdout:7/770: dwrite d0/d9/db/f15 [0,4194304] 0 2024-04-22T23:35:09.464 INFO:tasks.workunit.client.1.smithi157.stdout:9/801: dwrite d4/df/d61/fc1 [0,4194304] 0 2024-04-22T23:35:09.464 INFO:tasks.workunit.client.1.smithi157.stdout:9/802: write d4/d15/d23/d27/d47/db3/fb8 [858805,31745] 0 2024-04-22T23:35:09.464 INFO:tasks.workunit.client.1.smithi157.stdout:1/869: truncate d4/d17/d1d/d33/d41/f85 327286 0 2024-04-22T23:35:09.465 INFO:tasks.workunit.client.0.smithi107.stdout:9/649: link d5/d29/c41 d5/d7/d27/d46/d4b/ce4 0 2024-04-22T23:35:09.465 INFO:tasks.workunit.client.0.smithi107.stdout:9/650: write d5/d7/fb9 [484360,49838] 0 2024-04-22T23:35:09.467 INFO:tasks.workunit.client.1.smithi157.stdout:0/753: mknod d17/d38/cfe 0 2024-04-22T23:35:09.467 INFO:tasks.workunit.client.1.smithi157.stdout:0/754: chown d17/d2b/l94 93 1 2024-04-22T23:35:09.467 INFO:tasks.workunit.client.1.smithi157.stdout:0/755: readlink d17/d2b/d4f/d9b/ldf 0 2024-04-22T23:35:09.468 INFO:tasks.workunit.client.0.smithi107.stdout:4/671: mknod d1/d7/d63/d78/db4/ce7 0 2024-04-22T23:35:09.470 INFO:tasks.workunit.client.0.smithi107.stdout:0/620: write d1/f8f [2472823,65394] 0 2024-04-22T23:35:09.470 INFO:tasks.workunit.client.0.smithi107.stdout:0/621: dread - d1/d9/d14/d4f/d59/fa5 zero size 2024-04-22T23:35:09.470 INFO:tasks.workunit.client.0.smithi107.stdout:0/622: truncate d1/d27/d20/d2c/d2e/faf 888564 0 2024-04-22T23:35:09.470 INFO:tasks.workunit.client.0.smithi107.stdout:0/623: write d1/d27/d20/d2c/d41/fcd [462050,18042] 0 2024-04-22T23:35:09.472 INFO:tasks.workunit.client.0.smithi107.stdout:4/672: creat d1/d21/d29/d2e/d88/d99/dd2/daf/fe8 x:0 0 0 2024-04-22T23:35:09.472 INFO:tasks.workunit.client.0.smithi107.stdout:2/679: dread d1/d63/f65 [0,4194304] 0 2024-04-22T23:35:09.472 INFO:tasks.workunit.client.0.smithi107.stdout:2/680: fdatasync d1/d12/d35/d48/d13/d4f/f62 0 2024-04-22T23:35:09.473 INFO:tasks.workunit.client.0.smithi107.stdout:2/681: stat d1/d12/d35/d48/d13/l28 0 2024-04-22T23:35:09.473 INFO:tasks.workunit.client.1.smithi157.stdout:7/771: link d0/d9/d29/d5e/d85/lb7 d0/d9/d29/d33/d7c/lea 0 2024-04-22T23:35:09.473 INFO:tasks.workunit.client.1.smithi157.stdout:7/772: write d0/d9/db/d35/fba [343707,69472] 0 2024-04-22T23:35:09.474 INFO:tasks.workunit.client.1.smithi157.stdout:9/803: symlink d4/df/d57/dbd/lf6 0 2024-04-22T23:35:09.474 INFO:tasks.workunit.client.1.smithi157.stdout:9/804: write d4/d15/d23/fdb [82886,102715] 0 2024-04-22T23:35:09.474 INFO:tasks.workunit.client.1.smithi157.stdout:1/870: mknod d4/d17/d34/d3b/d52/c130 0 2024-04-22T23:35:09.475 INFO:tasks.workunit.client.0.smithi107.stdout:4/673: mkdir d1/d7/d63/d78/d96/da0/de9 0 2024-04-22T23:35:09.475 INFO:tasks.workunit.client.0.smithi107.stdout:4/674: write d1/d7/d63/fe1 [149395,14539] 0 2024-04-22T23:35:09.475 INFO:tasks.workunit.client.0.smithi107.stdout:4/675: creat d1/fea x:0 0 0 2024-04-22T23:35:09.475 INFO:tasks.workunit.client.1.smithi157.stdout:0/756: mknod d17/d2b/d5b/cff 0 2024-04-22T23:35:09.475 INFO:tasks.workunit.client.1.smithi157.stdout:0/757: chown d17/d38/fb9 6509 1 2024-04-22T23:35:09.475 INFO:tasks.workunit.client.1.smithi157.stdout:0/758: dread - d17/d38/fb9 zero size 2024-04-22T23:35:09.475 INFO:tasks.workunit.client.1.smithi157.stdout:0/759: write d17/d2b/d4f/dc5/dca/ff5 [709150,96908] 0 2024-04-22T23:35:09.476 INFO:tasks.workunit.client.0.smithi107.stdout:2/682: mknod d1/d67/ce0 0 2024-04-22T23:35:09.476 INFO:tasks.workunit.client.1.smithi157.stdout:7/773: unlink d0/f45 0 2024-04-22T23:35:09.476 INFO:tasks.workunit.client.1.smithi157.stdout:7/774: write d0/d9/d29/d33/fe4 [511930,3084] 0 2024-04-22T23:35:09.477 INFO:tasks.workunit.client.1.smithi157.stdout:9/805: symlink d4/df/d57/dbd/dc6/lf7 0 2024-04-22T23:35:09.477 INFO:tasks.workunit.client.1.smithi157.stdout:9/806: chown d4/d15/d23/ld5 594 1 2024-04-22T23:35:09.478 INFO:tasks.workunit.client.1.smithi157.stdout:0/760: mkdir d17/d23/d34/d67/d7f/d69/d100 0 2024-04-22T23:35:09.479 INFO:tasks.workunit.client.1.smithi157.stdout:9/807: mknod d4/df/d61/d9c/cf8 0 2024-04-22T23:35:09.479 INFO:tasks.workunit.client.0.smithi107.stdout:4/676: mkdir d1/d7/d5b/dbf/da8/deb 0 2024-04-22T23:35:09.480 INFO:tasks.workunit.client.0.smithi107.stdout:4/677: read d1/d21/d29/d2e/d7e/fc6 [63482,107074] 0 2024-04-22T23:35:09.480 INFO:tasks.workunit.client.0.smithi107.stdout:4/678: chown d1/d7/fc 19687 1 2024-04-22T23:35:09.480 INFO:tasks.workunit.client.0.smithi107.stdout:4/679: creat d1/d21/d52/fec x:0 0 0 2024-04-22T23:35:09.480 INFO:tasks.workunit.client.0.smithi107.stdout:2/683: dread d1/d2b/d88/dc9/fd8 [0,4194304] 0 2024-04-22T23:35:09.480 INFO:tasks.workunit.client.0.smithi107.stdout:2/684: creat d1/d2b/d88/fe1 x:0 0 0 2024-04-22T23:35:09.481 INFO:tasks.workunit.client.1.smithi157.stdout:3/855: dwrite d0/d3/d34/d3d/df6/fbb [4194304,4194304] 0 2024-04-22T23:35:09.481 INFO:tasks.workunit.client.0.smithi107.stdout:6/682: sync 2024-04-22T23:35:09.481 INFO:tasks.workunit.client.0.smithi107.stdout:5/654: sync 2024-04-22T23:35:09.482 INFO:tasks.workunit.client.1.smithi157.stdout:9/808: mkdir d4/df/d57/db7/df9 0 2024-04-22T23:35:09.482 INFO:tasks.workunit.client.1.smithi157.stdout:0/761: getdents d17/d2b/d52 0 2024-04-22T23:35:09.483 INFO:tasks.workunit.client.0.smithi107.stdout:9/651: dwrite d5/d24/f3c [0,4194304] 0 2024-04-22T23:35:09.483 INFO:tasks.workunit.client.0.smithi107.stdout:9/652: chown d5/d7/d17/d1b/d1f/d36/lac 61612 1 2024-04-22T23:35:09.484 INFO:tasks.workunit.client.0.smithi107.stdout:0/624: dread d1/d27/d20/d2c/d41/fcd [0,4194304] 0 2024-04-22T23:35:09.484 INFO:tasks.workunit.client.0.smithi107.stdout:4/680: mknod d1/d7/d5b/dbf/da8/ced 0 2024-04-22T23:35:09.484 INFO:tasks.workunit.client.0.smithi107.stdout:4/681: chown d1/d21/d29/d2e/d88/d99/dd2/lcd 5058754 1 2024-04-22T23:35:09.486 INFO:tasks.workunit.client.1.smithi157.stdout:3/856: creat d0/d3/d34/d4a/f124 x:0 0 0 2024-04-22T23:35:09.486 INFO:tasks.workunit.client.1.smithi157.stdout:3/857: dread - d0/d3/fb3 zero size 2024-04-22T23:35:09.487 INFO:tasks.workunit.client.0.smithi107.stdout:6/683: symlink db/d20/d2a/d5d/lcf 0 2024-04-22T23:35:09.488 INFO:tasks.workunit.client.0.smithi107.stdout:5/655: mkdir d9/db/d92/ddb 0 2024-04-22T23:35:09.489 INFO:tasks.workunit.client.1.smithi157.stdout:9/809: creat d4/df/d57/dbd/dc6/ffa x:0 0 0 2024-04-22T23:35:09.489 INFO:tasks.workunit.client.1.smithi157.stdout:9/810: write d4/df/f8b [2187198,10987] 0 2024-04-22T23:35:09.490 INFO:tasks.workunit.client.1.smithi157.stdout:0/762: mkdir d17/d2b/d101 0 2024-04-22T23:35:09.493 INFO:tasks.workunit.client.0.smithi107.stdout:9/653: rename d5/d14/d2e/d4f/d73/l7c to d5/d29/d7b/d91/le5 0 2024-04-22T23:35:09.493 INFO:tasks.workunit.client.0.smithi107.stdout:9/654: fdatasync d5/d7/d27/d46/f74 0 2024-04-22T23:35:09.498 INFO:tasks.workunit.client.0.smithi107.stdout:4/682: link d1/d21/d85/le5 d1/d7/d63/d78/d80/lee 0 2024-04-22T23:35:09.498 INFO:tasks.workunit.client.0.smithi107.stdout:4/683: write d1/d21/d52/f70 [3107813,32500] 0 2024-04-22T23:35:09.498 INFO:tasks.workunit.client.0.smithi107.stdout:4/684: fsync d1/d7/d5b/dbf/f92 0 2024-04-22T23:35:09.501 INFO:tasks.workunit.client.0.smithi107.stdout:5/656: write d9/db/d42/d75/fd7 [3020158,12097] 0 2024-04-22T23:35:09.503 INFO:tasks.workunit.client.0.smithi107.stdout:6/684: creat db/d20/d2c/d3b/d40/d9e/fd0 x:0 0 0 2024-04-22T23:35:09.504 INFO:tasks.workunit.client.1.smithi157.stdout:3/858: write d0/d3/d13/ff8 [3654946,99505] 0 2024-04-22T23:35:09.506 INFO:tasks.workunit.client.1.smithi157.stdout:3/859: mkdir d0/d83/df9/d119/d125 0 2024-04-22T23:35:09.508 INFO:tasks.workunit.client.1.smithi157.stdout:3/860: unlink d0/d3/d34/d39/ff4 0 2024-04-22T23:35:09.509 INFO:tasks.workunit.client.0.smithi107.stdout:4/685: link d1/d21/d29/d2e/d3b/f4c d1/d21/d29/d2e/d7e/fef 0 2024-04-22T23:35:09.509 INFO:tasks.workunit.client.0.smithi107.stdout:4/686: read - d1/d21/d29/d2e/d88/d99/dd2/daf/fe8 zero size 2024-04-22T23:35:09.513 INFO:tasks.workunit.client.0.smithi107.stdout:5/657: link d9/db/d94/lc4 d9/d26/d6a/ldc 0 2024-04-22T23:35:09.513 INFO:tasks.workunit.client.0.smithi107.stdout:5/658: stat d9/d26/lad 0 2024-04-22T23:35:09.514 INFO:tasks.workunit.client.0.smithi107.stdout:6/685: rmdir db/d20/d2a/d35/d69/db3 39 2024-04-22T23:35:09.514 INFO:tasks.workunit.client.1.smithi157.stdout:1/871: dwrite d4/db/f1e [0,4194304] 0 2024-04-22T23:35:09.517 INFO:tasks.workunit.client.1.smithi157.stdout:1/872: mkdir d4/d17/d34/d75/d8a/dfc/d131 0 2024-04-22T23:35:09.517 INFO:tasks.workunit.client.1.smithi157.stdout:1/873: chown d4/d17/d34/fcf 39 1 2024-04-22T23:35:09.517 INFO:tasks.workunit.client.1.smithi157.stdout:1/874: chown d4/d17/d34/d3b/d50/f56 408930269 1 2024-04-22T23:35:09.518 INFO:tasks.workunit.client.0.smithi107.stdout:4/687: unlink d1/d1b/d89/fb2 0 2024-04-22T23:35:09.519 INFO:tasks.workunit.client.1.smithi157.stdout:1/875: mknod d4/d17/d1d/d33/d41/d54/c132 0 2024-04-22T23:35:09.519 INFO:tasks.workunit.client.0.smithi107.stdout:5/659: mknod d9/db/d92/cdd 0 2024-04-22T23:35:09.519 INFO:tasks.workunit.client.1.smithi157.stdout:7/775: dwrite d0/d9/db/df/d68/f8e [4194304,4194304] 0 2024-04-22T23:35:09.520 INFO:tasks.workunit.client.0.smithi107.stdout:4/688: rename d1/d7/d63/d78/da4/dab to d1/d21/d29/d2e/d88/d99/dd2/df0 0 2024-04-22T23:35:09.521 INFO:tasks.workunit.client.1.smithi157.stdout:7/776: mknod d0/d9/d29/d5e/da6/ceb 0 2024-04-22T23:35:09.521 INFO:tasks.workunit.client.1.smithi157.stdout:7/777: creat d0/d9/db/df/dac/fec x:0 0 0 2024-04-22T23:35:09.522 INFO:tasks.workunit.client.0.smithi107.stdout:5/660: creat d9/db/d42/d75/fde x:0 0 0 2024-04-22T23:35:09.526 INFO:tasks.workunit.client.0.smithi107.stdout:0/625: truncate d1/d27/d20/d2c/d41/fcd 246040 0 2024-04-22T23:35:09.532 INFO:tasks.workunit.client.0.smithi107.stdout:0/626: symlink d1/d27/d20/d4a/ld6 0 2024-04-22T23:35:09.533 INFO:tasks.workunit.client.1.smithi157.stdout:7/778: write d0/d9/d29/f4d [6571728,98183] 0 2024-04-22T23:35:09.533 INFO:tasks.workunit.client.1.smithi157.stdout:7/779: dread - d0/d9/de/fb9 zero size 2024-04-22T23:35:09.533 INFO:tasks.workunit.client.1.smithi157.stdout:0/763: dwrite d17/d2b/d52/fa2 [0,4194304] 0 2024-04-22T23:35:09.536 INFO:tasks.workunit.client.1.smithi157.stdout:1/876: write d4/d17/da6/fa9 [1839133,71535] 0 2024-04-22T23:35:09.536 INFO:tasks.workunit.client.1.smithi157.stdout:1/877: readlink d4/d17/d34/d3b/l60 0 2024-04-22T23:35:09.537 INFO:tasks.workunit.client.1.smithi157.stdout:9/811: dwrite d4/d15/d23/f88 [4194304,4194304] 0 2024-04-22T23:35:09.537 INFO:tasks.workunit.client.0.smithi107.stdout:9/655: dwrite d5/d29/f6d [0,4194304] 0 2024-04-22T23:35:09.537 INFO:tasks.workunit.client.0.smithi107.stdout:4/689: write d1/d21/d29/d2e/f4f [1228089,114210] 0 2024-04-22T23:35:09.539 INFO:tasks.workunit.client.1.smithi157.stdout:1/878: mkdir d4/d17/db9/d133 0 2024-04-22T23:35:09.539 INFO:tasks.workunit.client.1.smithi157.stdout:1/879: dread - d4/f104 zero size 2024-04-22T23:35:09.539 INFO:tasks.workunit.client.1.smithi157.stdout:1/880: fdatasync d4/db/d3d/dc1/fd0 0 2024-04-22T23:35:09.541 INFO:tasks.workunit.client.1.smithi157.stdout:1/881: symlink d4/d17/d1d/df5/l134 0 2024-04-22T23:35:09.544 INFO:tasks.workunit.client.1.smithi157.stdout:1/882: creat d4/db/d3d/dc1/f135 x:0 0 0 2024-04-22T23:35:09.546 INFO:tasks.workunit.client.0.smithi107.stdout:9/656: readlink d5/d7/l64 0 2024-04-22T23:35:09.547 INFO:tasks.workunit.client.1.smithi157.stdout:1/883: creat d4/d17/d1d/d33/d41/d54/db3/d126/f136 x:0 0 0 2024-04-22T23:35:09.547 INFO:tasks.workunit.client.1.smithi157.stdout:3/861: dwrite d0/d3/d34/d4a/fd7 [0,4194304] 0 2024-04-22T23:35:09.547 INFO:tasks.workunit.client.0.smithi107.stdout:5/661: dwrite d9/db/d42/f6c [0,4194304] 0 2024-04-22T23:35:09.548 INFO:tasks.workunit.client.0.smithi107.stdout:5/662: readlink d9/db/d42/l8e 0 2024-04-22T23:35:09.552 INFO:tasks.workunit.client.0.smithi107.stdout:9/657: mknod d5/d14/d28/de3/ce6 0 2024-04-22T23:35:09.552 INFO:tasks.workunit.client.0.smithi107.stdout:9/658: write d5/d29/d7b/d91/dde/dc9/daf/fd0 [591545,112529] 0 2024-04-22T23:35:09.552 INFO:tasks.workunit.client.0.smithi107.stdout:9/659: readlink d5/d7/d17/l5d 0 2024-04-22T23:35:09.552 INFO:tasks.workunit.client.0.smithi107.stdout:9/660: rename d5/d7/d27/d69 to d5/d7/d27/d69/db6/da8/de7 22 2024-04-22T23:35:09.552 INFO:tasks.workunit.client.0.smithi107.stdout:5/663: mknod d9/d26/d79/cdf 0 2024-04-22T23:35:09.555 INFO:tasks.workunit.client.0.smithi107.stdout:9/661: mknod d5/d14/d2e/ce8 0 2024-04-22T23:35:09.555 INFO:tasks.workunit.client.1.smithi157.stdout:1/884: dread d4/db/d3d/dc1/d100/f107 [0,4194304] 0 2024-04-22T23:35:09.556 INFO:tasks.workunit.client.1.smithi157.stdout:7/780: write d0/d9/d29/d5e/fd9 [943520,26003] 0 2024-04-22T23:35:09.557 INFO:tasks.workunit.client.0.smithi107.stdout:9/662: mkdir d5/d7/de9 0 2024-04-22T23:35:09.560 INFO:tasks.workunit.client.0.smithi107.stdout:9/663: creat d5/d7/d27/d69/db6/fea x:0 0 0 2024-04-22T23:35:09.561 INFO:tasks.workunit.client.1.smithi157.stdout:0/764: dread d17/d2b/d4f/dc5/dca/ff5 [0,4194304] 0 2024-04-22T23:35:09.566 INFO:tasks.workunit.client.0.smithi107.stdout:5/664: dread d9/d76/f8a [0,4194304] 0 2024-04-22T23:35:09.567 INFO:tasks.workunit.client.1.smithi157.stdout:1/885: read d4/d17/d34/d3b/f66 [3269510,15915] 0 2024-04-22T23:35:09.569 INFO:tasks.workunit.client.0.smithi107.stdout:5/665: creat d9/db/de/d5e/fe0 x:0 0 0 2024-04-22T23:35:09.569 INFO:tasks.workunit.client.0.smithi107.stdout:0/627: dwrite d1/d84/f92 [0,4194304] 0 2024-04-22T23:35:09.569 INFO:tasks.workunit.client.1.smithi157.stdout:1/886: symlink d4/db/d3d/dc1/l137 0 2024-04-22T23:35:09.570 INFO:tasks.workunit.client.0.smithi107.stdout:5/666: truncate d9/d76/fc8 523376 0 2024-04-22T23:35:09.572 INFO:tasks.workunit.client.0.smithi107.stdout:6/686: dwrite db/d20/d2a/d38/d59/f93 [0,4194304] 0 2024-04-22T23:35:09.573 INFO:tasks.workunit.client.1.smithi157.stdout:1/887: rename d4/d5f/d7e/fab to d4/d17/da6/f138 0 2024-04-22T23:35:09.573 INFO:tasks.workunit.client.1.smithi157.stdout:1/888: chown d4/d77/fdf 438055232 1 2024-04-22T23:35:09.573 INFO:tasks.workunit.client.1.smithi157.stdout:1/889: write d4/d5f/d7e/db8/fdd [5104286,89208] 0 2024-04-22T23:35:09.573 INFO:tasks.workunit.client.1.smithi157.stdout:7/781: write d0/d9/d42/fcc [1210345,48551] 0 2024-04-22T23:35:09.574 INFO:tasks.workunit.client.1.smithi157.stdout:9/812: dwrite d4/d15/d23/d27/d75/fea [0,4194304] 0 2024-04-22T23:35:09.575 INFO:tasks.workunit.client.1.smithi157.stdout:7/782: mkdir d0/d9/db/df/dac/ded 0 2024-04-22T23:35:09.575 INFO:tasks.workunit.client.1.smithi157.stdout:7/783: chown d0/d9/d29/d98/lb8 7686538 1 2024-04-22T23:35:09.577 INFO:tasks.workunit.client.0.smithi107.stdout:5/667: symlink d9/d26/le1 0 2024-04-22T23:35:09.577 INFO:tasks.workunit.client.0.smithi107.stdout:5/668: readlink d9/db/d42/l65 0 2024-04-22T23:35:09.577 INFO:tasks.workunit.client.0.smithi107.stdout:5/669: creat d9/db/de/d5e/fe2 x:0 0 0 2024-04-22T23:35:09.577 INFO:tasks.workunit.client.0.smithi107.stdout:5/670: write d9/db/de/d72/d9e/fa3 [418362,42348] 0 2024-04-22T23:35:09.578 INFO:tasks.workunit.client.1.smithi157.stdout:1/890: dread d4/f26 [0,4194304] 0 2024-04-22T23:35:09.578 INFO:tasks.workunit.client.1.smithi157.stdout:1/891: chown d4/d17/d34/d3b/d50 1 1 2024-04-22T23:35:09.578 INFO:tasks.workunit.client.1.smithi157.stdout:1/892: chown d4/d17/d34/f63 2507 1 2024-04-22T23:35:09.579 INFO:tasks.workunit.client.1.smithi157.stdout:7/784: mknod d0/d9/d29/cee 0 2024-04-22T23:35:09.579 INFO:tasks.workunit.client.1.smithi157.stdout:7/785: creat d0/d9/d72/fef x:0 0 0 2024-04-22T23:35:09.579 INFO:tasks.workunit.client.1.smithi157.stdout:7/786: write d0/d9/de/fb9 [504947,115832] 0 2024-04-22T23:35:09.581 INFO:tasks.workunit.client.0.smithi107.stdout:4/690: dwrite d1/d7/d63/f72 [0,4194304] 0 2024-04-22T23:35:09.583 INFO:tasks.workunit.client.1.smithi157.stdout:3/862: dwrite d0/d3/d34/d3d/df6/dee/d9e/fd8 [0,4194304] 0 2024-04-22T23:35:09.583 INFO:tasks.workunit.client.1.smithi157.stdout:3/863: fsync d0/d3/d34/d3d/f63 0 2024-04-22T23:35:09.583 INFO:tasks.workunit.client.0.smithi107.stdout:9/664: dwrite d5/d7/d27/d46/f88 [0,4194304] 0 2024-04-22T23:35:09.584 INFO:tasks.workunit.client.0.smithi107.stdout:9/665: dread - d5/d7/d27/d46/fc6 zero size 2024-04-22T23:35:09.586 INFO:tasks.workunit.client.1.smithi157.stdout:1/893: link d4/db/d3d/ld8 d4/d17/d34/d3b/l139 0 2024-04-22T23:35:09.587 INFO:tasks.workunit.client.1.smithi157.stdout:1/894: write d4/d17/d34/d3b/d50/f56 [2342038,8783] 0 2024-04-22T23:35:09.587 INFO:tasks.workunit.client.1.smithi157.stdout:1/895: write d4/db/d3d/dc1/fd0 [4866173,2956] 0 2024-04-22T23:35:09.590 INFO:tasks.workunit.client.1.smithi157.stdout:9/813: write d4/d2d/fb4 [2572992,27838] 0 2024-04-22T23:35:09.591 INFO:tasks.workunit.client.0.smithi107.stdout:9/666: creat d5/d7/d17/d1b/d1f/feb x:0 0 0 2024-04-22T23:35:09.591 INFO:tasks.workunit.client.0.smithi107.stdout:9/667: write d5/d14/d2e/f3f [202949,64741] 0 2024-04-22T23:35:09.591 INFO:tasks.workunit.client.0.smithi107.stdout:9/668: write d5/d7/d27/d46/f74 [2682302,24326] 0 2024-04-22T23:35:09.591 INFO:tasks.workunit.client.0.smithi107.stdout:9/669: chown d5/d29/d7b/d91/c9e 6 1 2024-04-22T23:35:09.592 INFO:tasks.workunit.client.0.smithi107.stdout:9/670: dread - d5/d14/d2e/d4f/fa4 zero size 2024-04-22T23:35:09.592 INFO:tasks.workunit.client.0.smithi107.stdout:9/671: write d5/d14/d2e/d4f/fb7 [35201,516] 0 2024-04-22T23:35:09.593 INFO:tasks.workunit.client.0.smithi107.stdout:5/671: dread d9/db/de/f6b [0,4194304] 0 2024-04-22T23:35:09.593 INFO:tasks.workunit.client.0.smithi107.stdout:5/672: dread - d9/d26/d79/fc3 zero size 2024-04-22T23:35:09.594 INFO:tasks.workunit.client.1.smithi157.stdout:3/864: mkdir d0/d7/d6c/d126 0 2024-04-22T23:35:09.594 INFO:tasks.workunit.client.1.smithi157.stdout:3/865: chown d0/d3/d34/d4a/fd7 518537 1 2024-04-22T23:35:09.594 INFO:tasks.workunit.client.1.smithi157.stdout:3/866: chown d0/d3/d34/d47/f107 1470 1 2024-04-22T23:35:09.594 INFO:tasks.workunit.client.1.smithi157.stdout:7/787: dread d0/d9/d42/d55/f8f [4194304,4194304] 0 2024-04-22T23:35:09.594 INFO:tasks.workunit.client.1.smithi157.stdout:1/896: symlink d4/d77/db4/l13a 0 2024-04-22T23:35:09.595 INFO:tasks.workunit.client.1.smithi157.stdout:9/814: truncate d4/d15/d23/d27/d75/f82 2863136 0 2024-04-22T23:35:09.595 INFO:tasks.workunit.client.1.smithi157.stdout:9/815: write d4/df/d80/fe0 [178102,62404] 0 2024-04-22T23:35:09.596 INFO:tasks.workunit.client.0.smithi107.stdout:9/672: creat d5/d7/d17/fec x:0 0 0 2024-04-22T23:35:09.598 INFO:tasks.workunit.client.0.smithi107.stdout:5/673: truncate d9/f29 3140157 0 2024-04-22T23:35:09.601 INFO:tasks.workunit.client.0.smithi107.stdout:0/628: dwrite d1/f15 [0,4194304] 0 2024-04-22T23:35:09.604 INFO:tasks.workunit.client.0.smithi107.stdout:4/691: dwrite d1/d7/d63/d78/d80/f82 [0,4194304] 0 2024-04-22T23:35:09.604 INFO:tasks.workunit.client.0.smithi107.stdout:4/692: dread - d1/fe6 zero size 2024-04-22T23:35:09.604 INFO:tasks.workunit.client.0.smithi107.stdout:6/687: dwrite db/d20/d49/d7b/fb4 [0,4194304] 0 2024-04-22T23:35:09.604 INFO:tasks.workunit.client.0.smithi107.stdout:6/688: creat db/fd1 x:0 0 0 2024-04-22T23:35:09.604 INFO:tasks.workunit.client.0.smithi107.stdout:6/689: write db/d20/d2c/d3b/fa8 [1729071,87553] 0 2024-04-22T23:35:09.604 INFO:tasks.workunit.client.0.smithi107.stdout:0/629: dread d1/d27/d20/f60 [0,4194304] 0 2024-04-22T23:35:09.605 INFO:tasks.workunit.client.1.smithi157.stdout:1/897: write d4/d17/d34/d75/d8a/fd3 [1622575,6332] 0 2024-04-22T23:35:09.605 INFO:tasks.workunit.client.1.smithi157.stdout:1/898: truncate d4/db/f7d 1288586 0 2024-04-22T23:35:09.605 INFO:tasks.workunit.client.1.smithi157.stdout:1/899: dread - d4/d17/d34/d3b/d50/ff8 zero size 2024-04-22T23:35:09.606 INFO:tasks.workunit.client.0.smithi107.stdout:5/674: write d9/f37 [2122251,65784] 0 2024-04-22T23:35:09.611 INFO:tasks.workunit.client.0.smithi107.stdout:5/675: write d9/db/de/d72/d7c/da4/fb3 [7155101,84073] 0 2024-04-22T23:35:09.612 INFO:tasks.workunit.client.0.smithi107.stdout:6/690: mkdir db/d20/d2c/db5/dd2 0 2024-04-22T23:35:09.614 INFO:tasks.workunit.client.0.smithi107.stdout:0/630: rename d1/d27/d20/d4a/c6a to d1/d9/dd4/cd7 0 2024-04-22T23:35:09.614 INFO:tasks.workunit.client.0.smithi107.stdout:4/693: dread d1/d21/d29/d2e/f46 [0,4194304] 0 2024-04-22T23:35:09.614 INFO:tasks.workunit.client.0.smithi107.stdout:4/694: write d1/d21/d29/d2e/d3b/f8a [1641571,10838] 0 2024-04-22T23:35:09.617 INFO:tasks.workunit.client.1.smithi157.stdout:3/867: rename d0/d3/d34/d3d/df6/dee/d9e/dac to d0/d3/d127 0 2024-04-22T23:35:09.618 INFO:tasks.workunit.client.0.smithi107.stdout:6/691: mkdir db/d20/d2a/d35/d69/db3/dbc/dd3 0 2024-04-22T23:35:09.618 INFO:tasks.workunit.client.0.smithi107.stdout:6/692: dread - db/d20/d2a/d38/d61/d6f/da2/d84/faf zero size 2024-04-22T23:35:09.618 INFO:tasks.workunit.client.0.smithi107.stdout:6/693: stat db/d20/d2a/c41 0 2024-04-22T23:35:09.619 INFO:tasks.workunit.client.0.smithi107.stdout:6/694: chown db/d20/d2a/d5d/c7d 68 1 2024-04-22T23:35:09.619 INFO:tasks.workunit.client.1.smithi157.stdout:3/868: write d0/d3/ffb [204617,33638] 0 2024-04-22T23:35:09.619 INFO:tasks.workunit.client.1.smithi157.stdout:3/869: readlink d0/d3/d34/d3d/d57/l74 0 2024-04-22T23:35:09.620 INFO:tasks.workunit.client.0.smithi107.stdout:0/631: symlink d1/d9/d14/d4f/ld8 0 2024-04-22T23:35:09.620 INFO:tasks.workunit.client.0.smithi107.stdout:0/632: chown d1/d27/d20/d4a 5206 1 2024-04-22T23:35:09.620 INFO:tasks.workunit.client.0.smithi107.stdout:0/633: truncate d1/d27/d20/d2c/d41/f64 996739 0 2024-04-22T23:35:09.620 INFO:tasks.workunit.client.0.smithi107.stdout:4/695: creat d1/d7/ff1 x:0 0 0 2024-04-22T23:35:09.620 INFO:tasks.workunit.client.0.smithi107.stdout:4/696: creat d1/d21/d29/d2e/d7e/ff2 x:0 0 0 2024-04-22T23:35:09.621 INFO:tasks.workunit.client.1.smithi157.stdout:7/788: rename d0/d9/db/df/d68/fb4 to d0/d9/db/dc3/ff0 0 2024-04-22T23:35:09.622 INFO:tasks.workunit.client.1.smithi157.stdout:3/870: mknod d0/d7/d95/c128 0 2024-04-22T23:35:09.622 INFO:tasks.workunit.client.1.smithi157.stdout:7/789: symlink d0/d9/d72/lf1 0 2024-04-22T23:35:09.624 INFO:tasks.workunit.client.1.smithi157.stdout:9/816: rename d4/df/d52/f7d to d4/d15/d23/dad/ffb 0 2024-04-22T23:35:09.624 INFO:tasks.workunit.client.1.smithi157.stdout:9/817: write d4/d15/d23/d27/d47/db3/fec [837583,119397] 0 2024-04-22T23:35:09.624 INFO:tasks.workunit.client.1.smithi157.stdout:9/818: write d4/df/d57/f44 [5396757,17193] 0 2024-04-22T23:35:09.624 INFO:tasks.workunit.client.0.smithi107.stdout:0/634: mknod d1/d27/cd9 0 2024-04-22T23:35:09.627 INFO:tasks.workunit.client.1.smithi157.stdout:7/790: getdents d0/d9/d29/d5e/d85 0 2024-04-22T23:35:09.629 INFO:tasks.workunit.client.1.smithi157.stdout:1/900: dwrite d4/db/d71/d127/f86 [4194304,4194304] 0 2024-04-22T23:35:09.631 INFO:tasks.workunit.client.0.smithi107.stdout:4/697: unlink d1/d21/d29/c65 0 2024-04-22T23:35:09.631 INFO:tasks.workunit.client.0.smithi107.stdout:4/698: readlink d1/d7/d63/d78/l94 0 2024-04-22T23:35:09.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:35:09 smithi157 ceph-mon[33877]: pgmap v261: 65 pgs: 65 active+clean; 2.3 GiB data, 8.1 GiB used, 528 GiB / 536 GiB avail; 260 MiB/s rd, 307 MiB/s wr, 682 op/s 2024-04-22T23:35:09.632 INFO:tasks.workunit.client.1.smithi157.stdout:7/791: dread d0/d9/db/d35/fba [0,4194304] 0 2024-04-22T23:35:09.633 INFO:tasks.workunit.client.1.smithi157.stdout:1/901: getdents d4/d5f/d7e/db8 0 2024-04-22T23:35:09.636 INFO:tasks.workunit.client.0.smithi107.stdout:8/631: sync 2024-04-22T23:35:09.636 INFO:tasks.workunit.client.0.smithi107.stdout:3/625: sync 2024-04-22T23:35:09.636 INFO:tasks.workunit.client.0.smithi107.stdout:7/613: sync 2024-04-22T23:35:09.636 INFO:tasks.workunit.client.0.smithi107.stdout:3/626: fdatasync db/d21/d27/d15/d29/d4f/db2/dc5/fd6 0 2024-04-22T23:35:09.636 INFO:tasks.workunit.client.0.smithi107.stdout:3/627: dread - db/fcf zero size 2024-04-22T23:35:09.636 INFO:tasks.workunit.client.0.smithi107.stdout:3/628: read - db/d21/d27/d15/d29/d2a/fae zero size 2024-04-22T23:35:09.636 INFO:tasks.workunit.client.0.smithi107.stdout:1/573: sync 2024-04-22T23:35:09.636 INFO:tasks.workunit.client.0.smithi107.stdout:3/629: dread - db/fcf zero size 2024-04-22T23:35:09.636 INFO:tasks.workunit.client.0.smithi107.stdout:1/574: dread - d7/d19/d20/d39/d53/d43/d46/d5c/fb9 zero size 2024-04-22T23:35:09.637 INFO:tasks.workunit.client.1.smithi157.stdout:3/871: read d0/d3/d34/db9/fd4 [1674385,123150] 0 2024-04-22T23:35:09.637 INFO:tasks.workunit.client.1.smithi157.stdout:3/872: chown d0/d3/d34/d47/dcd/l113 17222 1 2024-04-22T23:35:09.637 INFO:tasks.workunit.client.1.smithi157.stdout:1/902: creat d4/d17/d34/dc8/f13b x:0 0 0 2024-04-22T23:35:09.637 INFO:tasks.workunit.client.1.smithi157.stdout:1/903: chown d4/d17/d1d/df5/l134 27937282 1 2024-04-22T23:35:09.638 INFO:tasks.workunit.client.0.smithi107.stdout:4/699: symlink d1/d1b/lf3 0 2024-04-22T23:35:09.639 INFO:tasks.workunit.client.0.smithi107.stdout:5/676: dwrite d9/d26/f3b [0,4194304] 0 2024-04-22T23:35:09.639 INFO:tasks.workunit.client.0.smithi107.stdout:5/677: fsync d9/db/d42/d75/f86 0 2024-04-22T23:35:09.639 INFO:tasks.workunit.client.0.smithi107.stdout:7/614: mknod d7/d21/da4/cc7 0 2024-04-22T23:35:09.640 INFO:tasks.workunit.client.0.smithi107.stdout:6/695: getdents db/d20/d2a/d35/d69/db3/dbc 0 2024-04-22T23:35:09.640 INFO:tasks.workunit.client.0.smithi107.stdout:6/696: truncate db/d20/d2c/d3b/fce 237367 0 2024-04-22T23:35:09.642 INFO:tasks.workunit.client.0.smithi107.stdout:8/632: mkdir d0/d3a/d46/d59/ddd 0 2024-04-22T23:35:09.642 INFO:tasks.workunit.client.0.smithi107.stdout:8/633: write d0/d7/df/fca [978130,101951] 0 2024-04-22T23:35:09.642 INFO:tasks.workunit.client.1.smithi157.stdout:3/873: mkdir d0/d3/d34/d3d/d57/d60/d6b/d129 0 2024-04-22T23:35:09.643 INFO:tasks.workunit.client.0.smithi107.stdout:3/630: link db/d21/d27/d36/fd0 db/d21/d27/d15/d29/d4f/db2/fd7 0 2024-04-22T23:35:09.643 INFO:tasks.workunit.client.0.smithi107.stdout:3/631: fsync db/d21/d27/d54/d25/d53/f7b 0 2024-04-22T23:35:09.644 INFO:tasks.workunit.client.0.smithi107.stdout:1/575: unlink d7/d19/d20/d39/f5a 0 2024-04-22T23:35:09.646 INFO:tasks.workunit.client.0.smithi107.stdout:4/700: symlink d1/d7/lf4 0 2024-04-22T23:35:09.646 INFO:tasks.workunit.client.0.smithi107.stdout:4/701: truncate d1/f13 7524934 0 2024-04-22T23:35:09.647 INFO:tasks.workunit.client.0.smithi107.stdout:5/678: rename d9/db/d92/dc2/cca to d9/db/d42/d75/ce3 0 2024-04-22T23:35:09.648 INFO:tasks.workunit.client.0.smithi107.stdout:7/615: mknod d7/d21/da4/cc8 0 2024-04-22T23:35:09.650 INFO:tasks.workunit.client.0.smithi107.stdout:6/697: creat db/d20/d2a/d35/d69/d79/fd4 x:0 0 0 2024-04-22T23:35:09.650 INFO:tasks.workunit.client.1.smithi157.stdout:9/819: dwrite d4/df/f26 [0,4194304] 0 2024-04-22T23:35:09.651 INFO:tasks.workunit.client.0.smithi107.stdout:1/576: unlink d7/d19/d54/f4f 0 2024-04-22T23:35:09.651 INFO:tasks.workunit.client.0.smithi107.stdout:1/577: getdents d7/d19/d20/d39/d53/d43/d46/d5c/dac 0 2024-04-22T23:35:09.653 INFO:tasks.workunit.client.0.smithi107.stdout:8/634: write d0/d7/df/d3b/f2b [985987,8141] 0 2024-04-22T23:35:09.653 INFO:tasks.workunit.client.0.smithi107.stdout:8/635: chown d0/d3a/d46/d59/d78 1794011 1 2024-04-22T23:35:09.653 INFO:tasks.workunit.client.0.smithi107.stdout:3/632: rename db/d21/d27/d15/db8/f8e to db/d21/d27/d15/d24/d52/d60/fd8 0 2024-04-22T23:35:09.654 INFO:tasks.workunit.client.0.smithi107.stdout:7/616: link d7/d4c/da1/cb7 d7/d21/d48/d5c/d9d/db1/cc9 0 2024-04-22T23:35:09.654 INFO:tasks.workunit.client.0.smithi107.stdout:7/617: dread - d7/d21/d54/f98 zero size 2024-04-22T23:35:09.655 INFO:tasks.workunit.client.0.smithi107.stdout:0/635: dwrite d1/d27/d20/d40/fa0 [0,4194304] 0 2024-04-22T23:35:09.655 INFO:tasks.workunit.client.0.smithi107.stdout:5/679: write d9/d76/f8a [2734003,26833] 0 2024-04-22T23:35:09.655 INFO:tasks.workunit.client.0.smithi107.stdout:0/636: chown d1/lb2 17815 1 2024-04-22T23:35:09.655 INFO:tasks.workunit.client.0.smithi107.stdout:5/680: fsync d9/f29 0 2024-04-22T23:35:09.655 INFO:tasks.workunit.client.0.smithi107.stdout:6/698: symlink db/d20/d2a/d38/d61/d6f/da2/d84/ld5 0 2024-04-22T23:35:09.658 INFO:tasks.workunit.client.0.smithi107.stdout:1/578: unlink d7/d19/d71/l85 0 2024-04-22T23:35:09.659 INFO:tasks.workunit.client.0.smithi107.stdout:8/636: unlink d0/l12 0 2024-04-22T23:35:09.661 INFO:tasks.workunit.client.0.smithi107.stdout:4/702: rename d1/d7/d63/d78/d96/da0/fa2 to d1/d21/d29/d2e/d88/d99/dd2/df0/ff5 0 2024-04-22T23:35:09.663 INFO:tasks.workunit.client.0.smithi107.stdout:7/618: mknod d7/d1b/cca 0 2024-04-22T23:35:09.663 INFO:tasks.workunit.client.0.smithi107.stdout:7/619: chown d7/d21/d7e/d94/fc4 110965 1 2024-04-22T23:35:09.664 INFO:tasks.workunit.client.0.smithi107.stdout:4/703: write d1/d21/d52/f97 [461301,122732] 0 2024-04-22T23:35:09.665 INFO:tasks.workunit.client.0.smithi107.stdout:0/637: write d1/d27/d20/f36 [1761424,125445] 0 2024-04-22T23:35:09.665 INFO:tasks.workunit.client.0.smithi107.stdout:1/579: creat d7/d19/d20/d39/d53/fbe x:0 0 0 2024-04-22T23:35:09.666 INFO:tasks.workunit.client.0.smithi107.stdout:8/637: symlink d0/d7/d20/d2d/lde 0 2024-04-22T23:35:09.669 INFO:tasks.workunit.client.0.smithi107.stdout:7/620: chown d7/d21/d48/l9e 16 1 2024-04-22T23:35:09.675 INFO:tasks.workunit.client.0.smithi107.stdout:4/704: creat d1/d7/d5b/dbf/da8/db9/ff6 x:0 0 0 2024-04-22T23:35:09.678 INFO:tasks.workunit.client.0.smithi107.stdout:0/638: creat d1/d27/d20/fda x:0 0 0 2024-04-22T23:35:09.680 INFO:tasks.workunit.client.0.smithi107.stdout:1/580: creat d7/d19/d20/d39/d53/d43/d46/d5c/dac/fbf x:0 0 0 2024-04-22T23:35:09.681 INFO:tasks.workunit.client.0.smithi107.stdout:8/638: getdents d0/d7/df/d33/d37/d9f 0 2024-04-22T23:35:09.681 INFO:tasks.workunit.client.0.smithi107.stdout:8/639: fsync d0/d3a/d46/d59/d78/fc0 0 2024-04-22T23:35:09.683 INFO:tasks.workunit.client.0.smithi107.stdout:7/621: mknod d7/d1b/ccb 0 2024-04-22T23:35:09.686 INFO:tasks.workunit.client.0.smithi107.stdout:0/639: read d1/d84/f92 [3469686,48419] 0 2024-04-22T23:35:09.686 INFO:tasks.workunit.client.0.smithi107.stdout:0/640: fsync d1/d27/d20/d2c/d41/f54 0 2024-04-22T23:35:09.686 INFO:tasks.workunit.client.0.smithi107.stdout:0/641: chown d1/d7b/fbf 79695780 1 2024-04-22T23:35:09.686 INFO:tasks.workunit.client.0.smithi107.stdout:0/642: write d1/d27/d20/d2c/d2e/fb3 [956640,41014] 0 2024-04-22T23:35:09.688 INFO:tasks.workunit.client.0.smithi107.stdout:4/705: symlink d1/d21/d29/d2e/lf7 0 2024-04-22T23:35:09.688 INFO:tasks.workunit.client.0.smithi107.stdout:4/706: truncate d1/d7/fc 5346557 0 2024-04-22T23:35:09.688 INFO:tasks.workunit.client.0.smithi107.stdout:4/707: creat d1/d21/d29/d2e/d7e/ff8 x:0 0 0 2024-04-22T23:35:09.688 INFO:tasks.workunit.client.0.smithi107.stdout:1/581: dread d7/d19/d20/d39/d93/f9e [0,4194304] 0 2024-04-22T23:35:09.690 INFO:tasks.workunit.client.0.smithi107.stdout:8/640: symlink d0/d7/d8/d41/d53/ldf 0 2024-04-22T23:35:09.692 INFO:tasks.workunit.client.0.smithi107.stdout:0/643: symlink d1/d84/d99/ldb 0 2024-04-22T23:35:09.693 INFO:tasks.workunit.client.0.smithi107.stdout:5/681: truncate d9/d76/f8a 2682236 0 2024-04-22T23:35:09.696 INFO:tasks.workunit.client.0.smithi107.stdout:4/708: symlink d1/d7/d5b/dbf/da8/db9/lf9 0 2024-04-22T23:35:09.698 INFO:tasks.workunit.client.0.smithi107.stdout:8/641: mkdir d0/d7/d8/de0 0 2024-04-22T23:35:09.699 INFO:tasks.workunit.client.0.smithi107.stdout:0/644: unlink d1/d9/d14/d4f/c65 0 2024-04-22T23:35:09.700 INFO:tasks.workunit.client.0.smithi107.stdout:0/645: fsync d1/d27/f38 0 2024-04-22T23:35:09.701 INFO:tasks.workunit.client.0.smithi107.stdout:5/682: mknod d9/db/dc1/ce4 0 2024-04-22T23:35:09.704 INFO:tasks.workunit.client.0.smithi107.stdout:1/582: symlink d7/d19/d20/d39/d53/d43/d63/d91/lc0 0 2024-04-22T23:35:09.705 INFO:tasks.workunit.client.0.smithi107.stdout:3/633: dwrite db/d21/d27/d15/d29/d2a/f4c [0,4194304] 0 2024-04-22T23:35:09.706 INFO:tasks.workunit.client.0.smithi107.stdout:0/646: mkdir d1/d27/d20/d40/dbd/ddc 0 2024-04-22T23:35:09.706 INFO:tasks.workunit.client.0.smithi107.stdout:5/683: rename d9/db/de/d5e/f90 to d9/db/de/fe5 0 2024-04-22T23:35:09.709 INFO:tasks.workunit.client.0.smithi107.stdout:7/622: dwrite d7/d21/d54/f90 [0,4194304] 0 2024-04-22T23:35:09.709 INFO:tasks.workunit.client.0.smithi107.stdout:8/642: dread d0/d3a/d46/d59/f6e [0,4194304] 0 2024-04-22T23:35:09.710 INFO:tasks.workunit.client.0.smithi107.stdout:4/709: getdents d1/d7/d5b/dbf/d27 0 2024-04-22T23:35:09.711 INFO:tasks.workunit.client.0.smithi107.stdout:7/623: dread d7/d4c/da1/fae [0,4194304] 0 2024-04-22T23:35:09.714 INFO:tasks.workunit.client.0.smithi107.stdout:3/634: write db/d21/d27/fcb [369854,83753] 0 2024-04-22T23:35:09.715 INFO:tasks.workunit.client.0.smithi107.stdout:0/647: stat d1/d9/d14/d4f/d59/l95 0 2024-04-22T23:35:09.716 INFO:tasks.workunit.client.0.smithi107.stdout:0/648: creat d1/d27/fdd x:0 0 0 2024-04-22T23:35:09.716 INFO:tasks.workunit.client.0.smithi107.stdout:0/649: dread - d1/d7b/fbf zero size 2024-04-22T23:35:09.716 INFO:tasks.workunit.client.0.smithi107.stdout:0/650: dread - d1/d9/d14/d4f/fa2 zero size 2024-04-22T23:35:09.716 INFO:tasks.workunit.client.0.smithi107.stdout:0/651: fdatasync d1/d27/d20/d2c/d41/fcd 0 2024-04-22T23:35:09.717 INFO:tasks.workunit.client.0.smithi107.stdout:8/643: unlink d0/d3a/d46/cd8 0 2024-04-22T23:35:09.719 INFO:tasks.workunit.client.0.smithi107.stdout:7/624: rename d7/d21/d48/d95/dbd to d7/d21/d54/dcc 0 2024-04-22T23:35:09.724 INFO:tasks.workunit.client.0.smithi107.stdout:3/635: unlink db/d21/d27/d15/d24/d52/l90 0 2024-04-22T23:35:09.724 INFO:tasks.workunit.client.0.smithi107.stdout:3/636: fdatasync db/d21/d27/d54/d25/fb6 0 2024-04-22T23:35:09.725 INFO:tasks.workunit.client.0.smithi107.stdout:1/583: dwrite d7/f67 [4194304,4194304] 0 2024-04-22T23:35:09.727 INFO:tasks.workunit.client.0.smithi107.stdout:0/652: mknod d1/d27/d20/d40/d75/cde 0 2024-04-22T23:35:09.728 INFO:tasks.workunit.client.0.smithi107.stdout:0/653: chown d1/d9/d14/c1d 15 1 2024-04-22T23:35:09.728 INFO:tasks.workunit.client.0.smithi107.stdout:0/654: creat d1/d27/d20/d2c/d41/dbe/fdf x:0 0 0 2024-04-22T23:35:09.729 INFO:tasks.workunit.client.0.smithi107.stdout:8/644: symlink d0/d7/d20/da2/le1 0 2024-04-22T23:35:09.730 INFO:tasks.workunit.client.0.smithi107.stdout:5/684: dwrite f3 [0,4194304] 0 2024-04-22T23:35:09.730 INFO:tasks.workunit.client.0.smithi107.stdout:5/685: write d9/db/de/fd9 [169313,107995] 0 2024-04-22T23:35:09.730 INFO:tasks.workunit.client.0.smithi107.stdout:5/686: write d9/f60 [3934837,39927] 0 2024-04-22T23:35:09.730 INFO:tasks.workunit.client.0.smithi107.stdout:5/687: read d9/f29 [2882132,9037] 0 2024-04-22T23:35:09.731 INFO:tasks.workunit.client.0.smithi107.stdout:4/710: dwrite d1/d21/d29/d2e/d88/fd7 [4194304,4194304] 0 2024-04-22T23:35:09.732 INFO:tasks.workunit.client.0.smithi107.stdout:4/711: write d1/d21/d52/fec [781497,100615] 0 2024-04-22T23:35:09.732 INFO:tasks.workunit.client.0.smithi107.stdout:4/712: write d1/d1b/d44/f95 [5159775,72865] 0 2024-04-22T23:35:09.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:35:09 smithi107 ceph-mon[26355]: pgmap v261: 65 pgs: 65 active+clean; 2.3 GiB data, 8.1 GiB used, 528 GiB / 536 GiB avail; 260 MiB/s rd, 307 MiB/s wr, 682 op/s 2024-04-22T23:35:09.738 INFO:tasks.workunit.client.0.smithi107.stdout:8/645: dread d0/d7/df/f5c [0,4194304] 0 2024-04-22T23:35:09.740 INFO:tasks.workunit.client.0.smithi107.stdout:1/584: dwrite d7/d19/d42/f6d [4194304,4194304] 0 2024-04-22T23:35:09.754 INFO:tasks.workunit.client.0.smithi107.stdout:8/646: dwrite d0/d3a/d46/d59/f68 [0,4194304] 0 2024-04-22T23:35:09.756 INFO:tasks.workunit.client.0.smithi107.stdout:8/647: dread d0/f4 [0,4194304] 0 2024-04-22T23:35:09.787 INFO:tasks.workunit.client.1.smithi157.stdout:4/871: sync 2024-04-22T23:35:09.787 INFO:tasks.workunit.client.1.smithi157.stdout:5/881: sync 2024-04-22T23:35:09.788 INFO:tasks.workunit.client.1.smithi157.stdout:4/872: rmdir d3/d96/d122 0 2024-04-22T23:35:09.789 INFO:tasks.workunit.client.1.smithi157.stdout:5/882: truncate d1/d5/f46 600476 0 2024-04-22T23:35:09.789 INFO:tasks.workunit.client.1.smithi157.stdout:5/883: write d1/db0/d7a/f8f [1694654,43404] 0 2024-04-22T23:35:09.790 INFO:tasks.workunit.client.1.smithi157.stdout:4/873: creat d3/d11/d30/f127 x:0 0 0 2024-04-22T23:35:09.790 INFO:tasks.workunit.client.1.smithi157.stdout:4/874: mknod d3/d11/d1d/d73/da3/d31/d91/d9f/c128 0 2024-04-22T23:35:09.790 INFO:tasks.workunit.client.1.smithi157.stdout:4/875: fdatasync d3/d4b/d5d/f97 0 2024-04-22T23:35:09.791 INFO:tasks.workunit.client.1.smithi157.stdout:4/876: stat d3/d11/d1d/d73/d80/dac/db5/df7/d111/f126 0 2024-04-22T23:35:09.791 INFO:tasks.workunit.client.1.smithi157.stdout:4/877: mkdir d3/d11/d1d/d73/da3/d31/d91/dcb/dcd/d129 0 2024-04-22T23:35:09.791 INFO:tasks.workunit.client.1.smithi157.stdout:4/878: readlink d3/d11/l56 0 2024-04-22T23:35:09.791 INFO:tasks.workunit.client.1.smithi157.stdout:4/879: readlink d3/d11/d1d/d73/d80/dac/db5/dfb/l119 0 2024-04-22T23:35:09.791 INFO:tasks.workunit.client.1.smithi157.stdout:4/880: write d3/d11/d1d/d73/da7/f106 [1398402,79952] 0 2024-04-22T23:35:09.792 INFO:tasks.workunit.client.1.smithi157.stdout:4/881: creat d3/d4b/d7a/db7/f12a x:0 0 0 2024-04-22T23:35:09.793 INFO:tasks.workunit.client.1.smithi157.stdout:4/882: unlink d3/d4b/d5d/f97 0 2024-04-22T23:35:09.793 INFO:tasks.workunit.client.1.smithi157.stdout:4/883: stat d3/d11/d1d/d73/d80/fd7 0 2024-04-22T23:35:09.794 INFO:tasks.workunit.client.1.smithi157.stdout:4/884: creat d3/d96/d40/dc3/f12b x:0 0 0 2024-04-22T23:35:09.794 INFO:tasks.workunit.client.1.smithi157.stdout:4/885: write d3/d11/d1d/d73/d80/dac/fef [1048743,74638] 0 2024-04-22T23:35:09.795 INFO:tasks.workunit.client.1.smithi157.stdout:5/884: read d1/d27/d29/d4b/f63 [2162518,54960] 0 2024-04-22T23:35:09.795 INFO:tasks.workunit.client.1.smithi157.stdout:5/885: write d1/d5/d7/f2e [1177666,45848] 0 2024-04-22T23:35:09.796 INFO:tasks.workunit.client.1.smithi157.stdout:5/886: truncate d1/d5/dc/d43/f88 313852 0 2024-04-22T23:35:09.796 INFO:tasks.workunit.client.1.smithi157.stdout:5/887: write d1/d5/d7/d2f/d45/d54/f74 [1722996,42910] 0 2024-04-22T23:35:09.798 INFO:tasks.workunit.client.0.smithi107.stdout:7/625: creat d7/d4c/da1/fcd x:0 0 0 2024-04-22T23:35:09.799 INFO:tasks.workunit.client.0.smithi107.stdout:3/637: creat db/d21/fd9 x:0 0 0 2024-04-22T23:35:09.801 INFO:tasks.workunit.client.1.smithi157.stdout:4/886: read d3/d11/d1d/d73/d80/f89 [3387915,42943] 0 2024-04-22T23:35:09.805 INFO:tasks.workunit.client.1.smithi157.stdout:5/888: dread d1/d5/d7/d2f/d45/d54/fc0 [0,4194304] 0 2024-04-22T23:35:09.807 INFO:tasks.workunit.client.1.smithi157.stdout:5/889: rmdir d1/d1f 39 2024-04-22T23:35:09.808 INFO:tasks.workunit.client.0.smithi107.stdout:7/626: dread d7/d21/da4/f88 [0,4194304] 0 2024-04-22T23:35:09.813 INFO:tasks.workunit.client.1.smithi157.stdout:4/887: dread d3/d11/d34/da6/ff2 [0,4194304] 0 2024-04-22T23:35:09.823 INFO:tasks.workunit.client.1.smithi157.stdout:5/890: dwrite d1/d1f/faf [4194304,4194304] 0 2024-04-22T23:35:09.823 INFO:tasks.workunit.client.1.smithi157.stdout:5/891: rmdir d1/d5/dc/d43/d5e/d145 39 2024-04-22T23:35:09.825 INFO:tasks.workunit.client.1.smithi157.stdout:5/892: mkdir d1/d1f/d147 0 2024-04-22T23:35:09.825 INFO:tasks.workunit.client.1.smithi157.stdout:5/893: write d1/d5/dc/d43/d5e/d145/f123 [1576848,69432] 0 2024-04-22T23:35:09.825 INFO:tasks.workunit.client.1.smithi157.stdout:4/888: dread d3/d11/fe6 [0,4194304] 0 2024-04-22T23:35:09.825 INFO:tasks.workunit.client.1.smithi157.stdout:4/889: write d3/d11/f1f [2929468,64540] 0 2024-04-22T23:35:09.826 INFO:tasks.workunit.client.1.smithi157.stdout:4/890: mknod d3/d11/d1d/d73/d80/dac/db5/dfb/c12c 0 2024-04-22T23:35:09.828 INFO:tasks.workunit.client.1.smithi157.stdout:4/891: unlink d3/d11/d1d/d73/da7/f106 0 2024-04-22T23:35:09.828 INFO:tasks.workunit.client.1.smithi157.stdout:4/892: write d3/d96/d40/fab [4313377,124708] 0 2024-04-22T23:35:09.831 INFO:tasks.workunit.client.1.smithi157.stdout:4/893: rename d3/d11/d1d/f2d to d3/d11/d30/dfd/f12d 0 2024-04-22T23:35:09.831 INFO:tasks.workunit.client.1.smithi157.stdout:4/894: dread - d3/d11/d1d/d73/da3/d31/d91/d9f/ffe zero size 2024-04-22T23:35:09.831 INFO:tasks.workunit.client.1.smithi157.stdout:4/895: truncate d3/d96/f9a 179259 0 2024-04-22T23:35:09.831 INFO:tasks.workunit.client.1.smithi157.stdout:4/896: chown d3/d11/d1d/d73/d80/dac 1689222161 1 2024-04-22T23:35:09.832 INFO:tasks.workunit.client.1.smithi157.stdout:4/897: link d3/d4b/d7a/db7/ca0 d3/d11/d34/da6/c12e 0 2024-04-22T23:35:09.833 INFO:tasks.workunit.client.0.smithi107.stdout:7/627: dwrite d7/d21/f22 [0,4194304] 0 2024-04-22T23:35:09.834 INFO:tasks.workunit.client.1.smithi157.stdout:4/898: rename d3/d11/d1d/d73/d80/dac/db5/dc8/cd6 to d3/d11/d30/dd9/c12f 0 2024-04-22T23:35:09.834 INFO:tasks.workunit.client.1.smithi157.stdout:4/899: chown d3/d5c/c11f 6 1 2024-04-22T23:35:09.836 INFO:tasks.workunit.client.1.smithi157.stdout:5/894: dread d1/d5/dc/d43/d5e/fda [0,4194304] 0 2024-04-22T23:35:09.836 INFO:tasks.workunit.client.1.smithi157.stdout:5/895: truncate d1/d5/d7/d2f/d45/fd6 436060 0 2024-04-22T23:35:09.836 INFO:tasks.workunit.client.1.smithi157.stdout:5/896: readlink d1/d5/d7/la5 0 2024-04-22T23:35:09.837 INFO:tasks.workunit.client.1.smithi157.stdout:5/897: mkdir d1/d1f/dc6/d148 0 2024-04-22T23:35:09.837 INFO:tasks.workunit.client.1.smithi157.stdout:4/900: dread d3/d11/fe6 [0,4194304] 0 2024-04-22T23:35:09.839 INFO:tasks.workunit.client.1.smithi157.stdout:5/898: dread d1/d5/f4e [0,4194304] 0 2024-04-22T23:35:09.847 INFO:tasks.workunit.client.1.smithi157.stdout:4/901: dread d3/d11/d1d/f52 [0,4194304] 0 2024-04-22T23:35:09.848 INFO:tasks.workunit.client.1.smithi157.stdout:4/902: mknod d3/d11/c130 0 2024-04-22T23:35:09.848 INFO:tasks.workunit.client.1.smithi157.stdout:4/903: readlink d3/d11/d1d/d73/l9e 0 2024-04-22T23:35:09.849 INFO:tasks.workunit.client.1.smithi157.stdout:4/904: link d3/d11/d1d/d73/da3/d31/f10e d3/d11/d34/da6/f131 0 2024-04-22T23:35:09.852 INFO:tasks.workunit.client.1.smithi157.stdout:5/899: dwrite d1/d27/d29/d4b/f4d [0,4194304] 0 2024-04-22T23:35:09.852 INFO:tasks.workunit.client.1.smithi157.stdout:5/900: dread - d1/d5/dc/d43/d4f/ff5 zero size 2024-04-22T23:35:09.854 INFO:tasks.workunit.client.1.smithi157.stdout:4/905: dread d3/d11/d1d/d73/d80/dac/fef [0,4194304] 0 2024-04-22T23:35:09.854 INFO:tasks.workunit.client.1.smithi157.stdout:4/906: truncate d3/d11/d34/d9d/ff6 267506 0 2024-04-22T23:35:09.856 INFO:tasks.workunit.client.1.smithi157.stdout:4/907: mknod d3/d11/c132 0 2024-04-22T23:35:09.857 INFO:tasks.workunit.client.1.smithi157.stdout:4/908: mkdir d3/d96/d4c/d133 0 2024-04-22T23:35:09.858 INFO:tasks.workunit.client.1.smithi157.stdout:4/909: mknod d3/d96/d40/c134 0 2024-04-22T23:35:09.859 INFO:tasks.workunit.client.1.smithi157.stdout:4/910: rmdir d3/d5c/dd3 39 2024-04-22T23:35:09.860 INFO:tasks.workunit.client.1.smithi157.stdout:4/911: chown d3/d96/d40/dc3/f12b 34029 1 2024-04-22T23:35:09.866 INFO:tasks.workunit.client.1.smithi157.stdout:5/901: dread d1/d5/dc/d43/f44 [0,4194304] 0 2024-04-22T23:35:09.866 INFO:tasks.workunit.client.1.smithi157.stdout:5/902: readlink d1/d5/dc/d43/d5e/lb3 0 2024-04-22T23:35:09.866 INFO:tasks.workunit.client.1.smithi157.stdout:5/903: creat d1/db0/d7a/f149 x:0 0 0 2024-04-22T23:35:09.873 INFO:tasks.workunit.client.1.smithi157.stdout:4/912: dwrite d3/d11/d1d/d73/da3/d31/d91/dcb/dcd/f11b [0,4194304] 0 2024-04-22T23:35:09.873 INFO:tasks.workunit.client.1.smithi157.stdout:4/913: fdatasync d3/d96/f67 0 2024-04-22T23:35:09.876 INFO:tasks.workunit.client.1.smithi157.stdout:5/904: write d1/d5/dc/f62 [1199650,87919] 0 2024-04-22T23:35:09.876 INFO:tasks.workunit.client.1.smithi157.stdout:5/905: chown d1/d27/d29/d4b/f4d 110543 1 2024-04-22T23:35:09.877 INFO:tasks.workunit.client.1.smithi157.stdout:5/906: dread - d1/db0/dff/f119 zero size 2024-04-22T23:35:09.877 INFO:tasks.workunit.client.1.smithi157.stdout:5/907: link d1/db0/dff/d102/f124 d1/d1f/f14a 0 2024-04-22T23:35:09.878 INFO:tasks.workunit.client.0.smithi107.stdout:4/713: mkdir d1/d21/d29/d2e/dfa 0 2024-04-22T23:35:09.879 INFO:tasks.workunit.client.1.smithi157.stdout:5/908: symlink d1/d5/d104/l14b 0 2024-04-22T23:35:09.879 INFO:tasks.workunit.client.1.smithi157.stdout:5/909: dread - d1/d27/fbc zero size 2024-04-22T23:35:09.882 INFO:tasks.workunit.client.0.smithi107.stdout:1/585: creat d7/fc1 x:0 0 0 2024-04-22T23:35:09.883 INFO:tasks.workunit.client.1.smithi157.stdout:4/914: write d3/d11/d1d/d73/d80/fd7 [4065809,109808] 0 2024-04-22T23:35:09.885 INFO:tasks.workunit.client.1.smithi157.stdout:4/915: creat d3/d11/d30/dd9/f135 x:0 0 0 2024-04-22T23:35:09.885 INFO:tasks.workunit.client.1.smithi157.stdout:4/916: dread - d3/d5c/f95 zero size 2024-04-22T23:35:09.888 INFO:tasks.workunit.client.1.smithi157.stdout:4/917: dread d3/d11/d1d/f49 [0,4194304] 0 2024-04-22T23:35:09.891 INFO:tasks.workunit.client.0.smithi107.stdout:8/648: rename d0/f5f to d0/d7/d20/d89/fe2 0 2024-04-22T23:35:09.895 INFO:tasks.workunit.client.1.smithi157.stdout:5/910: dwrite d1/f5a [0,4194304] 0 2024-04-22T23:35:09.895 INFO:tasks.workunit.client.0.smithi107.stdout:7/628: mkdir d7/d4c/da1/dce 0 2024-04-22T23:35:09.896 INFO:tasks.workunit.client.1.smithi157.stdout:5/911: rename d1/db0/dc1 to d1/d5/d7/d2f/d14c 0 2024-04-22T23:35:09.899 INFO:tasks.workunit.client.0.smithi107.stdout:5/688: rmdir d9/db/de 39 2024-04-22T23:35:09.900 INFO:tasks.workunit.client.0.smithi107.stdout:4/714: mknod d1/d7/d63/cfb 0 2024-04-22T23:35:09.901 INFO:tasks.workunit.client.0.smithi107.stdout:8/649: mknod d0/d7/df/d3b/d7b/ce3 0 2024-04-22T23:35:09.902 INFO:tasks.workunit.client.0.smithi107.stdout:9/673: sync 2024-04-22T23:35:09.902 INFO:tasks.workunit.client.0.smithi107.stdout:6/699: sync 2024-04-22T23:35:09.902 INFO:tasks.workunit.client.0.smithi107.stdout:2/685: sync 2024-04-22T23:35:09.902 INFO:tasks.workunit.client.0.smithi107.stdout:0/655: sync 2024-04-22T23:35:09.902 INFO:tasks.workunit.client.0.smithi107.stdout:0/656: chown d1/d27/fdd 1324 1 2024-04-22T23:35:09.903 INFO:tasks.workunit.client.0.smithi107.stdout:3/638: link db/d21/d27/f20 db/d21/fda 0 2024-04-22T23:35:09.903 INFO:tasks.workunit.client.0.smithi107.stdout:3/639: chown db/d21/d27/d15/d29/d2a/f5b 5 1 2024-04-22T23:35:09.903 INFO:tasks.workunit.client.0.smithi107.stdout:3/640: write db/d21/d27/fcb [422343,122827] 0 2024-04-22T23:35:09.903 INFO:tasks.workunit.client.0.smithi107.stdout:3/641: chown db/d21/d27/d54/d25/la4 1037585 1 2024-04-22T23:35:09.904 INFO:tasks.workunit.client.0.smithi107.stdout:5/689: mkdir d9/de6 0 2024-04-22T23:35:09.904 INFO:tasks.workunit.client.0.smithi107.stdout:5/690: fsync d9/d26/f64 0 2024-04-22T23:35:09.904 INFO:tasks.workunit.client.0.smithi107.stdout:5/691: truncate d9/db/d42/d75/db6/fb9 5048691 0 2024-04-22T23:35:09.905 INFO:tasks.workunit.client.0.smithi107.stdout:4/715: mkdir d1/d7/d63/d78/dfc 0 2024-04-22T23:35:09.905 INFO:tasks.workunit.client.0.smithi107.stdout:4/716: chown d1/d7/d53/d66/d6b/ddb 39 1 2024-04-22T23:35:09.905 INFO:tasks.workunit.client.0.smithi107.stdout:6/700: write db/d20/d2c/d3b/fa8 [424065,52062] 0 2024-04-22T23:35:09.907 INFO:tasks.workunit.client.0.smithi107.stdout:1/586: dread d7/d19/d20/d39/d53/d43/d46/f4c [0,4194304] 0 2024-04-22T23:35:09.909 INFO:tasks.workunit.client.1.smithi157.stdout:2/948: sync 2024-04-22T23:35:09.909 INFO:tasks.workunit.client.1.smithi157.stdout:8/631: sync 2024-04-22T23:35:09.909 INFO:tasks.workunit.client.0.smithi107.stdout:7/629: link d7/d21/d48/fc2 d7/d21/d48/fcf 0 2024-04-22T23:35:09.912 INFO:tasks.workunit.client.1.smithi157.stdout:4/918: dwrite d3/d11/d1d/d73/d80/f8c [0,4194304] 0 2024-04-22T23:35:09.913 INFO:tasks.workunit.client.1.smithi157.stdout:2/949: creat d7/d8/d30/db8/f135 x:0 0 0 2024-04-22T23:35:09.914 INFO:tasks.workunit.client.1.smithi157.stdout:8/632: truncate d2/d6/d21/d5f/f1b 431471 0 2024-04-22T23:35:09.914 INFO:tasks.workunit.client.1.smithi157.stdout:8/633: write d2/d4/d73/fb5 [265151,80504] 0 2024-04-22T23:35:09.914 INFO:tasks.workunit.client.1.smithi157.stdout:8/634: fdatasync d2/d6/d21/d5f/d11/d13/f7d 0 2024-04-22T23:35:09.916 INFO:tasks.workunit.client.0.smithi107.stdout:6/701: creat db/d20/d2a/d38/d61/d6f/da2/d84/fd6 x:0 0 0 2024-04-22T23:35:09.916 INFO:tasks.workunit.client.0.smithi107.stdout:3/642: dread db/d21/d27/d15/d29/d2a/d40/f6c [0,4194304] 0 2024-04-22T23:35:09.919 INFO:tasks.workunit.client.0.smithi107.stdout:5/692: read d9/db/de/d5e/f67 [4010248,92201] 0 2024-04-22T23:35:09.919 INFO:tasks.workunit.client.0.smithi107.stdout:0/657: getdents d1/d2a/dad 0 2024-04-22T23:35:09.920 INFO:tasks.workunit.client.1.smithi157.stdout:2/950: rmdir d7/d8/d25/d55 39 2024-04-22T23:35:09.920 INFO:tasks.workunit.client.1.smithi157.stdout:2/951: readlink d7/d1f/d3e/l7c 0 2024-04-22T23:35:09.920 INFO:tasks.workunit.client.1.smithi157.stdout:2/952: dread - d7/d8/dca/fe0 zero size 2024-04-22T23:35:09.920 INFO:tasks.workunit.client.1.smithi157.stdout:2/953: chown d7/d8/d30/d23/d27/c37 971 1 2024-04-22T23:35:09.921 INFO:tasks.workunit.client.0.smithi107.stdout:4/717: rmdir d1/d21/d29/d2e/d88/d99/dd2/daf 39 2024-04-22T23:35:09.921 INFO:tasks.workunit.client.1.smithi157.stdout:8/635: getdents d2/d6/d21/d5f/d11/d13 0 2024-04-22T23:35:09.922 INFO:tasks.workunit.client.1.smithi157.stdout:2/954: truncate d7/d8/fbf 256835 0 2024-04-22T23:35:09.923 INFO:tasks.workunit.client.1.smithi157.stdout:2/955: chown d7/d8/d30/d23/c32 1 1 2024-04-22T23:35:09.923 INFO:tasks.workunit.client.1.smithi157.stdout:8/636: mkdir d2/d4/dbd/d97/ddd 0 2024-04-22T23:35:09.923 INFO:tasks.workunit.client.1.smithi157.stdout:8/637: stat d2/db1/c35 0 2024-04-22T23:35:09.926 INFO:tasks.workunit.client.0.smithi107.stdout:3/643: link db/d21/d27/d15/d24/d52/l95 db/d21/d27/d15/d24/ldb 0 2024-04-22T23:35:09.926 INFO:tasks.workunit.client.0.smithi107.stdout:3/644: write db/d21/d27/d15/d24/d52/d60/f93 [483146,21010] 0 2024-04-22T23:35:09.926 INFO:tasks.workunit.client.1.smithi157.stdout:2/956: unlink d7/d8/dd/d4d/f6e 0 2024-04-22T23:35:09.928 INFO:tasks.workunit.client.0.smithi107.stdout:8/650: rename d0/d3a/dd4 to d0/d7/d20/de4 0 2024-04-22T23:35:09.931 INFO:tasks.workunit.client.1.smithi157.stdout:4/919: dread d3/d96/d4c/fd4 [0,4194304] 0 2024-04-22T23:35:09.931 INFO:tasks.workunit.client.0.smithi107.stdout:5/693: link d9/db/f8c d9/db/fe7 0 2024-04-22T23:35:09.932 INFO:tasks.workunit.client.0.smithi107.stdout:5/694: chown d9/db 3662707 1 2024-04-22T23:35:09.933 INFO:tasks.workunit.client.1.smithi157.stdout:4/920: mkdir d3/d11/d30/dfd/d136 0 2024-04-22T23:35:09.935 INFO:tasks.workunit.client.0.smithi107.stdout:0/658: write d1/d9/f68 [7555538,40239] 0 2024-04-22T23:35:09.935 INFO:tasks.workunit.client.0.smithi107.stdout:0/659: truncate d1/d27/d20/d2c/fbb 764439 0 2024-04-22T23:35:09.937 INFO:tasks.workunit.client.0.smithi107.stdout:6/702: rename db/d20/l7a to db/d20/d2a/d38/d61/ld7 0 2024-04-22T23:35:09.940 INFO:tasks.workunit.client.1.smithi157.stdout:2/957: dread d7/d8/d25/d102/f103 [0,4194304] 0 2024-04-22T23:35:09.941 INFO:tasks.workunit.client.1.smithi157.stdout:2/958: write d7/d8/d95/fa8 [4930888,130581] 0 2024-04-22T23:35:09.942 INFO:tasks.workunit.client.0.smithi107.stdout:8/651: creat d0/d7/df/d3b/dd9/fe5 x:0 0 0 2024-04-22T23:35:09.942 INFO:tasks.workunit.client.0.smithi107.stdout:5/695: mknod d9/db/dc1/ce8 0 2024-04-22T23:35:09.942 INFO:tasks.workunit.client.0.smithi107.stdout:0/660: truncate d1/d27/d20/d2c/d2e/fb3 561369 0 2024-04-22T23:35:09.943 INFO:tasks.workunit.client.0.smithi107.stdout:5/696: chown d9/db/f50 81 1 2024-04-22T23:35:09.943 INFO:tasks.workunit.client.0.smithi107.stdout:5/697: dread - d9/db/d42/d75/fde zero size 2024-04-22T23:35:09.943 INFO:tasks.workunit.client.0.smithi107.stdout:5/698: fdatasync d9/db/dc1/fc9 0 2024-04-22T23:35:09.943 INFO:tasks.workunit.client.0.smithi107.stdout:5/699: write d9/db/f5f [727633,126279] 0 2024-04-22T23:35:09.944 INFO:tasks.workunit.client.0.smithi107.stdout:2/686: dwrite d1/d12/d35/f3f [0,4194304] 0 2024-04-22T23:35:09.944 INFO:tasks.workunit.client.0.smithi107.stdout:2/687: dread - d1/d2b/d49/d7e/fcc zero size 2024-04-22T23:35:09.945 INFO:tasks.workunit.client.0.smithi107.stdout:0/661: mknod d1/d27/d20/d40/dbd/ddc/ce0 0 2024-04-22T23:35:09.946 INFO:tasks.workunit.client.0.smithi107.stdout:0/662: stat d1/d9/cd 0 2024-04-22T23:35:09.946 INFO:tasks.workunit.client.0.smithi107.stdout:0/663: write d1/d27/d20/d2c/fbb [1573796,56418] 0 2024-04-22T23:35:09.946 INFO:tasks.workunit.client.0.smithi107.stdout:0/664: chown d1/d9/d14/f4d 266561 1 2024-04-22T23:35:09.946 INFO:tasks.workunit.client.0.smithi107.stdout:0/665: truncate d1/d27/fb0 40953 0 2024-04-22T23:35:09.946 INFO:tasks.workunit.client.0.smithi107.stdout:0/666: stat d1/d27/f21 0 2024-04-22T23:35:09.947 INFO:tasks.workunit.client.0.smithi107.stdout:9/674: dwrite d5/f13 [0,4194304] 0 2024-04-22T23:35:09.947 INFO:tasks.workunit.client.0.smithi107.stdout:5/700: mkdir d9/db/dc1/dcb/de9 0 2024-04-22T23:35:09.948 INFO:tasks.workunit.client.0.smithi107.stdout:0/667: unlink d1/d27/d20/l5b 0 2024-04-22T23:35:09.948 INFO:tasks.workunit.client.0.smithi107.stdout:0/668: read - d1/d84/fc8 zero size 2024-04-22T23:35:09.951 INFO:tasks.workunit.client.0.smithi107.stdout:9/675: unlink d5/d6c/fc3 0 2024-04-22T23:35:09.951 INFO:tasks.workunit.client.0.smithi107.stdout:1/587: dwrite d7/d19/d20/d39/d53/f4e [4194304,4194304] 0 2024-04-22T23:35:09.951 INFO:tasks.workunit.client.0.smithi107.stdout:1/588: chown d7/d19/d2d/da4/cb6 529109136 1 2024-04-22T23:35:09.951 INFO:tasks.workunit.client.0.smithi107.stdout:1/589: dread - d7/d19/d20/fbd zero size 2024-04-22T23:35:09.951 INFO:tasks.workunit.client.0.smithi107.stdout:1/590: stat d7/d19/d71/f78 0 2024-04-22T23:35:09.951 INFO:tasks.workunit.client.0.smithi107.stdout:1/591: creat d7/d19/d71/fc2 x:0 0 0 2024-04-22T23:35:09.951 INFO:tasks.workunit.client.0.smithi107.stdout:1/592: creat d7/d19/d20/fc3 x:0 0 0 2024-04-22T23:35:09.951 INFO:tasks.workunit.client.0.smithi107.stdout:1/593: truncate d7/d19/d20/d39/d53/d43/d46/d5c/f6b 130681 0 2024-04-22T23:35:09.951 INFO:tasks.workunit.client.0.smithi107.stdout:2/688: getdents d1/d2b/d3d 0 2024-04-22T23:35:09.952 INFO:tasks.workunit.client.1.smithi157.stdout:5/912: dwrite d1/db0/d7a/f87 [0,4194304] 0 2024-04-22T23:35:09.952 INFO:tasks.workunit.client.1.smithi157.stdout:5/913: write d1/d27/d29/f5f [745565,38273] 0 2024-04-22T23:35:09.952 INFO:tasks.workunit.client.0.smithi107.stdout:5/701: creat d9/db/d92/fea x:0 0 0 2024-04-22T23:35:09.954 INFO:tasks.workunit.client.0.smithi107.stdout:6/703: write db/d20/d2a/d38/d59/f5a [3943002,28857] 0 2024-04-22T23:35:09.954 INFO:tasks.workunit.client.0.smithi107.stdout:6/704: chown db/d20/d58/fb9 200000 1 2024-04-22T23:35:09.955 INFO:tasks.workunit.client.0.smithi107.stdout:0/669: rename d1/d27/f57 to d1/d9/d14/d4f/fe1 0 2024-04-22T23:35:09.957 INFO:tasks.workunit.client.0.smithi107.stdout:7/630: dwrite d7/d21/d48/d5c/f75 [4194304,4194304] 0 2024-04-22T23:35:09.962 INFO:tasks.workunit.client.0.smithi107.stdout:9/676: mknod d5/d7/d17/d1b/d1f/d36/d96/ced 0 2024-04-22T23:35:09.967 INFO:tasks.workunit.client.0.smithi107.stdout:1/594: readlink d7/l86 0 2024-04-22T23:35:09.969 INFO:tasks.workunit.client.0.smithi107.stdout:5/702: creat d9/db/de/feb x:0 0 0 2024-04-22T23:35:09.969 INFO:tasks.workunit.client.0.smithi107.stdout:5/703: stat d9/db/d42/f6c 0 2024-04-22T23:35:09.970 INFO:tasks.workunit.client.0.smithi107.stdout:5/704: creat d9/db/d94/fec x:0 0 0 2024-04-22T23:35:09.971 INFO:tasks.workunit.client.0.smithi107.stdout:0/670: creat d1/d9/d14/d4f/d59/fe2 x:0 0 0 2024-04-22T23:35:09.972 INFO:tasks.workunit.client.1.smithi157.stdout:8/638: dwrite d2/d4/d30/db8/fc6 [0,4194304] 0 2024-04-22T23:35:09.972 INFO:tasks.workunit.client.1.smithi157.stdout:8/639: write d2/d34/d6a/d77/f9f [989074,130966] 0 2024-04-22T23:35:09.975 INFO:tasks.workunit.client.0.smithi107.stdout:7/631: rename d7/d21/d48/d5c/d9d/f5a to d7/d4c/da1/fd0 0 2024-04-22T23:35:09.977 INFO:tasks.workunit.client.0.smithi107.stdout:1/595: mkdir d7/d19/d20/d39/d53/d43/d46/dba/dc4 0 2024-04-22T23:35:09.977 INFO:tasks.workunit.client.0.smithi107.stdout:1/596: truncate d7/d19/d54/f66 1433054 0 2024-04-22T23:35:09.978 INFO:tasks.workunit.client.1.smithi157.stdout:8/640: dread d2/d6/d21/d5f/d11/f5a [0,4194304] 0 2024-04-22T23:35:09.979 INFO:tasks.workunit.client.0.smithi107.stdout:2/689: getdents d1/d12/d35/d54 0 2024-04-22T23:35:09.980 INFO:tasks.workunit.client.1.smithi157.stdout:8/641: creat d2/d4/dbd/d67/fde x:0 0 0 2024-04-22T23:35:09.981 INFO:tasks.workunit.client.1.smithi157.stdout:8/642: write d2/f5 [7052872,82949] 0 2024-04-22T23:35:09.982 INFO:tasks.workunit.client.0.smithi107.stdout:0/671: write d1/d27/f38 [1513366,72031] 0 2024-04-22T23:35:09.982 INFO:tasks.workunit.client.0.smithi107.stdout:5/705: symlink d9/db/de/d72/led 0 2024-04-22T23:35:09.982 INFO:tasks.workunit.client.0.smithi107.stdout:5/706: chown d9/db/d42/f4b 222827877 1 2024-04-22T23:35:09.982 INFO:tasks.workunit.client.0.smithi107.stdout:5/707: read d9/fd [327250,96697] 0 2024-04-22T23:35:09.983 INFO:tasks.workunit.client.0.smithi107.stdout:5/708: write d9/db/de/d72/d9e/fa3 [832836,108071] 0 2024-04-22T23:35:09.983 INFO:tasks.workunit.client.0.smithi107.stdout:1/597: truncate d7/d19/d2d/f31 4046602 0 2024-04-22T23:35:09.985 INFO:tasks.workunit.client.0.smithi107.stdout:2/690: rmdir d1/d2b/d49/d7e/da8/ddc 0 2024-04-22T23:35:09.986 INFO:tasks.workunit.client.0.smithi107.stdout:0/672: rename d1/d27/c98 to d1/d9/d14/ce3 0 2024-04-22T23:35:09.987 INFO:tasks.workunit.client.0.smithi107.stdout:5/709: mkdir d9/db/de/d72/d9e/dee 0 2024-04-22T23:35:09.988 INFO:tasks.workunit.client.0.smithi107.stdout:1/598: mknod d7/d19/d20/d39/d53/d43/d46/d5c/db7/cc5 0 2024-04-22T23:35:09.990 INFO:tasks.workunit.client.0.smithi107.stdout:2/691: link d1/d2b/d59/l98 d1/d2b/d49/d7e/dae/le2 0 2024-04-22T23:35:09.991 INFO:tasks.workunit.client.0.smithi107.stdout:0/673: rename d1/d9/d14/d4f/f82 to d1/d9/dd4/fe4 0 2024-04-22T23:35:09.993 INFO:tasks.workunit.client.0.smithi107.stdout:5/710: creat d9/db/d92/dc2/fef x:0 0 0 2024-04-22T23:35:09.993 INFO:tasks.workunit.client.1.smithi157.stdout:8/643: write d2/d6/d21/d5f/f3f [491752,48643] 0 2024-04-22T23:35:09.993 INFO:tasks.workunit.client.1.smithi157.stdout:8/644: write d2/d4/dbd/f9d [852926,37364] 0 2024-04-22T23:35:09.995 INFO:tasks.workunit.client.1.smithi157.stdout:8/645: unlink d2/d6/d21/d8d/lcb 0 2024-04-22T23:35:09.995 INFO:tasks.workunit.client.0.smithi107.stdout:4/718: dwrite d1/f87 [0,4194304] 0 2024-04-22T23:35:09.996 INFO:tasks.workunit.client.0.smithi107.stdout:6/705: dwrite db/d20/d2a/d38/f9f [0,4194304] 0 2024-04-22T23:35:09.997 INFO:tasks.workunit.client.0.smithi107.stdout:8/652: dwrite d0/d7/d20/d2d/f2c [0,4194304] 0 2024-04-22T23:35:09.997 INFO:tasks.workunit.client.0.smithi107.stdout:8/653: readlink d0/d7/l5d 0 2024-04-22T23:35:09.998 INFO:tasks.workunit.client.1.smithi157.stdout:4/921: dwrite d3/d11/fde [0,4194304] 0 2024-04-22T23:35:09.998 INFO:tasks.workunit.client.1.smithi157.stdout:5/914: dwrite d1/d5/d7/f67 [0,4194304] 0 2024-04-22T23:35:09.998 INFO:tasks.workunit.client.1.smithi157.stdout:8/646: rename d2/d6/l8a to d2/d6/d21/ldf 0 2024-04-22T23:35:09.998 INFO:tasks.workunit.client.1.smithi157.stdout:8/647: readlink d2/l4c 0 2024-04-22T23:35:09.999 INFO:tasks.workunit.client.0.smithi107.stdout:9/677: dwrite d5/d7/d27/d46/dd3/fd1 [0,4194304] 0 2024-04-22T23:35:09.999 INFO:tasks.workunit.client.0.smithi107.stdout:9/678: creat d5/d6c/fee x:0 0 0 2024-04-22T23:35:09.999 INFO:tasks.workunit.client.0.smithi107.stdout:2/692: unlink d1/f2d 0 2024-04-22T23:35:10.004 INFO:tasks.workunit.client.1.smithi157.stdout:4/922: symlink d3/d11/d34/d9d/l137 0 2024-04-22T23:35:10.005 INFO:tasks.workunit.client.0.smithi107.stdout:1/599: unlink d7/c11 0 2024-04-22T23:35:10.005 INFO:tasks.workunit.client.0.smithi107.stdout:1/600: fsync d7/d19/d2d/d89/f90 0 2024-04-22T23:35:10.008 INFO:tasks.workunit.client.0.smithi107.stdout:6/706: dread db/d20/d2a/f2f [0,4194304] 0 2024-04-22T23:35:10.008 INFO:tasks.workunit.client.0.smithi107.stdout:6/707: fdatasync db/d20/d2c/d3b/fa7 0 2024-04-22T23:35:10.008 INFO:tasks.workunit.client.1.smithi157.stdout:8/648: creat d2/d6/fe0 x:0 0 0 2024-04-22T23:35:10.009 INFO:tasks.workunit.client.0.smithi107.stdout:4/719: symlink d1/d7/d63/d78/d80/lfd 0 2024-04-22T23:35:10.009 INFO:tasks.workunit.client.0.smithi107.stdout:4/720: fdatasync d1/d21/d29/d2e/d7e/fef 0 2024-04-22T23:35:10.012 INFO:tasks.workunit.client.1.smithi157.stdout:4/923: rename d3/d11/d34/da4/db1/fe0 to d3/d11/d1d/d73/da3/d31/d91/dcb/f138 0 2024-04-22T23:35:10.013 INFO:tasks.workunit.client.1.smithi157.stdout:5/915: dwrite d1/d5/dc/d5d/fd3 [0,4194304] 0 2024-04-22T23:35:10.013 INFO:tasks.workunit.client.1.smithi157.stdout:5/916: fsync d1/db0/dff/d102/f124 0 2024-04-22T23:35:10.013 INFO:tasks.workunit.client.1.smithi157.stdout:5/917: fdatasync d1/d5/dc/d43/d4f/fcf 0 2024-04-22T23:35:10.015 INFO:tasks.workunit.client.0.smithi107.stdout:8/654: link d0/d73/db0/fbc d0/d7/df/d3b/d7b/fe6 0 2024-04-22T23:35:10.015 INFO:tasks.workunit.client.1.smithi157.stdout:5/918: truncate d1/d5/d7/d2f/d45/d54/f65 3936058 0 2024-04-22T23:35:10.015 INFO:tasks.workunit.client.1.smithi157.stdout:5/919: creat d1/d5/dc/d5d/f14d x:0 0 0 2024-04-22T23:35:10.016 INFO:tasks.workunit.client.1.smithi157.stdout:4/924: getdents d3/d11/d30 0 2024-04-22T23:35:10.018 INFO:tasks.workunit.client.0.smithi107.stdout:9/679: dwrite d5/d7/d17/d1b/d1f/d36/d96/fb2 [0,4194304] 0 2024-04-22T23:35:10.018 INFO:tasks.workunit.client.0.smithi107.stdout:9/680: write d5/d14/d2e/d4f/fa4 [444935,102680] 0 2024-04-22T23:35:10.022 INFO:tasks.workunit.client.1.smithi157.stdout:4/925: link d3/d11/d34/da6/l10a d3/d11/d1d/d73/d80/dac/db5/l139 0 2024-04-22T23:35:10.031 INFO:tasks.workunit.client.1.smithi157.stdout:4/926: dread d3/d11/fde [0,4194304] 0 2024-04-22T23:35:10.032 INFO:tasks.workunit.client.0.smithi107.stdout:2/693: mknod d1/d12/ce3 0 2024-04-22T23:35:10.033 INFO:tasks.workunit.client.0.smithi107.stdout:5/711: truncate d9/db/f5f 1076089 0 2024-04-22T23:35:10.034 INFO:tasks.workunit.client.0.smithi107.stdout:5/712: fdatasync d9/d26/f38 0 2024-04-22T23:35:10.035 INFO:tasks.workunit.client.0.smithi107.stdout:1/601: mknod d7/d19/d20/d39/d53/d43/d63/cc6 0 2024-04-22T23:35:10.035 INFO:tasks.workunit.client.0.smithi107.stdout:1/602: write d7/d19/d2d/f84 [160385,75370] 0 2024-04-22T23:35:10.037 INFO:tasks.workunit.client.0.smithi107.stdout:6/708: rename db/d20/d2a/d5d/c7d to db/d20/d2a/d5f/cd8 0 2024-04-22T23:35:10.039 INFO:tasks.workunit.client.0.smithi107.stdout:4/721: link d1/d7/d53/fd6 d1/d21/ffe 0 2024-04-22T23:35:10.040 INFO:tasks.workunit.client.0.smithi107.stdout:8/655: stat d0/d7/df/c1a 0 2024-04-22T23:35:10.040 INFO:tasks.workunit.client.0.smithi107.stdout:8/656: fdatasync d0/d7/d8/f1e 0 2024-04-22T23:35:10.040 INFO:tasks.workunit.client.0.smithi107.stdout:8/657: chown d0/d7/df/d33/d37/fcc 31053121 1 2024-04-22T23:35:10.042 INFO:tasks.workunit.client.0.smithi107.stdout:9/681: dwrite d5/d14/d2e/d4f/d73/f9b [0,4194304] 0 2024-04-22T23:35:10.042 INFO:tasks.workunit.client.0.smithi107.stdout:9/682: chown d5/d7/d27/d46/dd3/fd1 8938775 1 2024-04-22T23:35:10.042 INFO:tasks.workunit.client.1.smithi157.stdout:5/920: dwrite d1/d5/dc/d43/f88 [0,4194304] 0 2024-04-22T23:35:10.043 INFO:tasks.workunit.client.0.smithi107.stdout:2/694: unlink d1/d12/l8c 0 2024-04-22T23:35:10.043 INFO:tasks.workunit.client.0.smithi107.stdout:2/695: rename d1 to d1/d2b/d88/de4 22 2024-04-22T23:35:10.044 INFO:tasks.workunit.client.1.smithi157.stdout:7/792: sync 2024-04-22T23:35:10.044 INFO:tasks.workunit.client.1.smithi157.stdout:7/793: chown d0/d9/l40 10283877 1 2024-04-22T23:35:10.044 INFO:tasks.workunit.client.1.smithi157.stdout:0/765: sync 2024-04-22T23:35:10.044 INFO:tasks.workunit.client.1.smithi157.stdout:0/766: fdatasync d17/d2b/d4f/d5a/f63 0 2024-04-22T23:35:10.044 INFO:tasks.workunit.client.1.smithi157.stdout:7/794: write d0/d9/d29/d33/fd2 [604526,80442] 0 2024-04-22T23:35:10.045 INFO:tasks.workunit.client.0.smithi107.stdout:5/713: mknod d9/db/cf0 0 2024-04-22T23:35:10.045 INFO:tasks.workunit.client.1.smithi157.stdout:0/767: mknod d17/d23/d34/d67/d7f/d69/dce/de3/c102 0 2024-04-22T23:35:10.045 INFO:tasks.workunit.client.1.smithi157.stdout:0/768: write d17/d23/fda [737964,97333] 0 2024-04-22T23:35:10.046 INFO:tasks.workunit.client.0.smithi107.stdout:1/603: mknod d7/d19/d71/db8/cc7 0 2024-04-22T23:35:10.046 INFO:tasks.workunit.client.0.smithi107.stdout:1/604: readlink d7/d19/d20/d39/d53/d43/d46/d68/l79 0 2024-04-22T23:35:10.048 INFO:tasks.workunit.client.1.smithi157.stdout:4/927: write d3/fda [3674790,65535] 0 2024-04-22T23:35:10.049 INFO:tasks.workunit.client.1.smithi157.stdout:4/928: write d3/d11/d1d/d73/d80/dac/db5/f8b [165725,5796] 0 2024-04-22T23:35:10.049 INFO:tasks.workunit.client.1.smithi157.stdout:4/929: fsync d3/d11/d1d/d73/da3/d31/df5/f120 0 2024-04-22T23:35:10.049 INFO:tasks.workunit.client.1.smithi157.stdout:4/930: write d3/d11/d1d/feb [4839003,41587] 0 2024-04-22T23:35:10.049 INFO:tasks.workunit.client.1.smithi157.stdout:4/931: readlink d3/d11/l63 0 2024-04-22T23:35:10.049 INFO:tasks.workunit.client.1.smithi157.stdout:4/932: creat d3/d4b/d5d/f13a x:0 0 0 2024-04-22T23:35:10.054 INFO:tasks.workunit.client.0.smithi107.stdout:6/709: creat db/d20/d2a/d38/d61/d6f/da2/fd9 x:0 0 0 2024-04-22T23:35:10.055 INFO:tasks.workunit.client.1.smithi157.stdout:4/933: truncate d3/d4b/f16 2576379 0 2024-04-22T23:35:10.055 INFO:tasks.workunit.client.1.smithi157.stdout:5/921: write d1/d1f/faf [3851751,87358] 0 2024-04-22T23:35:10.055 INFO:tasks.workunit.client.1.smithi157.stdout:7/795: dread d0/d9/d42/d55/f8f [0,4194304] 0 2024-04-22T23:35:10.055 INFO:tasks.workunit.client.1.smithi157.stdout:7/796: fsync d0/d9/d10/f18 0 2024-04-22T23:35:10.055 INFO:tasks.workunit.client.1.smithi157.stdout:5/922: mknod d1/d1f/c14e 0 2024-04-22T23:35:10.055 INFO:tasks.workunit.client.1.smithi157.stdout:5/923: fdatasync d1/d5/d7/f34 0 2024-04-22T23:35:10.055 INFO:tasks.workunit.client.1.smithi157.stdout:5/924: stat d1/d5/d7/f34 0 2024-04-22T23:35:10.056 INFO:tasks.workunit.client.1.smithi157.stdout:4/934: dread d3/d96/d40/dc3/fdb [0,4194304] 0 2024-04-22T23:35:10.056 INFO:tasks.workunit.client.1.smithi157.stdout:5/925: dread d1/d27/fd7 [0,4194304] 0 2024-04-22T23:35:10.057 INFO:tasks.workunit.client.0.smithi107.stdout:9/683: getdents d5/d24 0 2024-04-22T23:35:10.058 INFO:tasks.workunit.client.1.smithi157.stdout:4/935: link d3/d96/lae d3/d4b/d7a/db7/l13b 0 2024-04-22T23:35:10.059 INFO:tasks.workunit.client.0.smithi107.stdout:2/696: unlink d1/d2b/d3d/d57/f83 0 2024-04-22T23:35:10.060 INFO:tasks.workunit.client.0.smithi107.stdout:2/697: dread - d1/d2b/d49/d7e/da8/fb8 zero size 2024-04-22T23:35:10.060 INFO:tasks.workunit.client.0.smithi107.stdout:2/698: fsync d1/d12/d35/d48/d13/f68 0 2024-04-22T23:35:10.060 INFO:tasks.workunit.client.0.smithi107.stdout:2/699: creat d1/d2b/d49/d7e/fe5 x:0 0 0 2024-04-22T23:35:10.060 INFO:tasks.workunit.client.0.smithi107.stdout:5/714: getdents d9/d26/d6a 0 2024-04-22T23:35:10.060 INFO:tasks.workunit.client.0.smithi107.stdout:5/715: truncate d9/db/d42/f6c 4206966 0 2024-04-22T23:35:10.061 INFO:tasks.workunit.client.0.smithi107.stdout:5/716: readlink d9/db/d94/lc4 0 2024-04-22T23:35:10.061 INFO:tasks.workunit.client.0.smithi107.stdout:5/717: creat d9/db/d42/d75/ff1 x:0 0 0 2024-04-22T23:35:10.063 INFO:tasks.workunit.client.0.smithi107.stdout:1/605: creat d7/d19/d20/d39/d53/d43/d46/dba/dc4/fc8 x:0 0 0 2024-04-22T23:35:10.063 INFO:tasks.workunit.client.0.smithi107.stdout:1/606: fdatasync d7/d19/d20/d39/d53/d43/d46/d5c/f8a 0 2024-04-22T23:35:10.064 INFO:tasks.workunit.client.0.smithi107.stdout:4/722: getdents d1/d7/d63/d78/db4 0 2024-04-22T23:35:10.065 INFO:tasks.workunit.client.0.smithi107.stdout:4/723: fdatasync d1/f5 0 2024-04-22T23:35:10.065 INFO:tasks.workunit.client.1.smithi157.stdout:5/926: rename d1/d1f to d1/d5/d7/d10/d14f 0 2024-04-22T23:35:10.066 INFO:tasks.workunit.client.1.smithi157.stdout:5/927: getdents d1/d5/d7/d2f/d45/da4 0 2024-04-22T23:35:10.066 INFO:tasks.workunit.client.1.smithi157.stdout:7/797: write d0/d9/d42/d55/f78 [672955,64501] 0 2024-04-22T23:35:10.066 INFO:tasks.workunit.client.1.smithi157.stdout:7/798: stat d0/d9/d42 0 2024-04-22T23:35:10.066 INFO:tasks.workunit.client.1.smithi157.stdout:7/799: write d0/d9/db/f61 [144941,59463] 0 2024-04-22T23:35:10.066 INFO:tasks.workunit.client.1.smithi157.stdout:7/800: creat d0/d9/d42/d55/d80/d82/ff2 x:0 0 0 2024-04-22T23:35:10.066 INFO:tasks.workunit.client.1.smithi157.stdout:7/801: write d0/d9/db/df/f2b [8421330,37813] 0 2024-04-22T23:35:10.067 INFO:tasks.workunit.client.1.smithi157.stdout:0/769: dwrite d17/d2b/d4f/dc5/dcc/de9/df6/fd3 [0,4194304] 0 2024-04-22T23:35:10.071 INFO:tasks.workunit.client.0.smithi107.stdout:6/710: creat db/d20/d2a/d38/d4b/fda x:0 0 0 2024-04-22T23:35:10.074 INFO:tasks.workunit.client.1.smithi157.stdout:7/802: rmdir d0/d9/d42 39 2024-04-22T23:35:10.074 INFO:tasks.workunit.client.1.smithi157.stdout:7/803: write d0/d9/d29/d49/d64/fb5 [5174265,18748] 0 2024-04-22T23:35:10.074 INFO:tasks.workunit.client.1.smithi157.stdout:7/804: truncate d0/d9/d29/d33/d37/fcd 1527868 0 2024-04-22T23:35:10.074 INFO:tasks.workunit.client.0.smithi107.stdout:8/658: dwrite d0/d7/d20/da2/fa9 [0,4194304] 0 2024-04-22T23:35:10.074 INFO:tasks.workunit.client.0.smithi107.stdout:2/700: dread d1/d67/f7b [0,4194304] 0 2024-04-22T23:35:10.075 INFO:tasks.workunit.client.0.smithi107.stdout:2/701: chown d1/f5 1 1 2024-04-22T23:35:10.075 INFO:tasks.workunit.client.0.smithi107.stdout:5/718: mknod d9/db/de/d72/cf2 0 2024-04-22T23:35:10.075 INFO:tasks.workunit.client.0.smithi107.stdout:5/719: readlink d9/db/l77 0 2024-04-22T23:35:10.076 INFO:tasks.workunit.client.0.smithi107.stdout:5/720: creat d9/db/d92/ff3 x:0 0 0 2024-04-22T23:35:10.076 INFO:tasks.workunit.client.0.smithi107.stdout:5/721: stat d9/db/de/d5e/lcf 0 2024-04-22T23:35:10.076 INFO:tasks.workunit.client.1.smithi157.stdout:7/805: dread d0/d9/d42/d55/d56/d91/fab [0,4194304] 0 2024-04-22T23:35:10.077 INFO:tasks.workunit.client.1.smithi157.stdout:0/770: mkdir d17/d23/d34/d67/d7f/d69/d103 0 2024-04-22T23:35:10.079 INFO:tasks.workunit.client.0.smithi107.stdout:1/607: creat d7/d19/d20/d39/d53/d43/d46/fc9 x:0 0 0 2024-04-22T23:35:10.081 INFO:tasks.workunit.client.1.smithi157.stdout:7/806: rename d0/d9/d42/d55/f78 to d0/d9/d42/d55/d80/d95/ff3 0 2024-04-22T23:35:10.083 INFO:tasks.workunit.client.0.smithi107.stdout:6/711: rename db/d20/d2a/d38/f3e to db/d20/d2a/d38/d61/d6f/da2/fdb 0 2024-04-22T23:35:10.092 INFO:tasks.workunit.client.0.smithi107.stdout:9/684: write d5/d7/f16 [115090,52147] 0 2024-04-22T23:35:10.093 INFO:tasks.workunit.client.0.smithi107.stdout:5/722: rename d9/db/de/d72/c80 to d9/db/de/d72/cf4 0 2024-04-22T23:35:10.093 INFO:tasks.workunit.client.0.smithi107.stdout:8/659: getdents d0/d73 0 2024-04-22T23:35:10.093 INFO:tasks.workunit.client.0.smithi107.stdout:8/660: chown d0/d85/l54 14875129 1 2024-04-22T23:35:10.093 INFO:tasks.workunit.client.0.smithi107.stdout:8/661: chown d0/d7/df/f5c 1577635 1 2024-04-22T23:35:10.095 INFO:tasks.workunit.client.0.smithi107.stdout:9/685: unlink d5/l6 0 2024-04-22T23:35:10.095 INFO:tasks.workunit.client.0.smithi107.stdout:9/686: write d5/d7/fd [478731,33990] 0 2024-04-22T23:35:10.097 INFO:tasks.workunit.client.0.smithi107.stdout:5/723: link d9/db/d42/lcd d9/d76/da0/lf5 0 2024-04-22T23:35:10.099 INFO:tasks.workunit.client.0.smithi107.stdout:8/662: link d0/d7/df/d33/d37/l66 d0/d7/df/le7 0 2024-04-22T23:35:10.100 INFO:tasks.workunit.client.0.smithi107.stdout:9/687: mkdir d5/d7/d27/d46/d4b/def 0 2024-04-22T23:35:10.100 INFO:tasks.workunit.client.0.smithi107.stdout:9/688: fsync d5/d7/d17/d1b/f30 0 2024-04-22T23:35:10.101 INFO:tasks.workunit.client.0.smithi107.stdout:5/724: creat d9/db/dc1/dcb/ff6 x:0 0 0 2024-04-22T23:35:10.102 INFO:tasks.workunit.client.0.smithi107.stdout:8/663: creat d0/d7/df/d3b/dd9/dda/fe8 x:0 0 0 2024-04-22T23:35:10.104 INFO:tasks.workunit.client.0.smithi107.stdout:8/664: write d0/d7/df/fca [5167,121182] 0 2024-04-22T23:35:10.105 INFO:tasks.workunit.client.0.smithi107.stdout:8/665: mkdir d0/d7/d20/da2/de9 0 2024-04-22T23:35:10.109 INFO:tasks.workunit.client.0.smithi107.stdout:7/632: sync 2024-04-22T23:35:10.109 INFO:tasks.workunit.client.0.smithi107.stdout:3/645: sync 2024-04-22T23:35:10.109 INFO:tasks.workunit.client.0.smithi107.stdout:0/674: sync 2024-04-22T23:35:10.109 INFO:tasks.workunit.client.0.smithi107.stdout:7/633: stat d7/d21/f6d 0 2024-04-22T23:35:10.109 INFO:tasks.workunit.client.0.smithi107.stdout:7/634: write d7/d21/da4/d81/f97 [1236919,82970] 0 2024-04-22T23:35:10.110 INFO:tasks.workunit.client.0.smithi107.stdout:8/666: dread d0/d7/df/f23 [0,4194304] 0 2024-04-22T23:35:10.110 INFO:tasks.workunit.client.0.smithi107.stdout:8/667: chown d0/d7/df/d17/f2a 346 1 2024-04-22T23:35:10.110 INFO:tasks.workunit.client.0.smithi107.stdout:0/675: mknod d1/d27/d20/d40/d75/ce5 0 2024-04-22T23:35:10.110 INFO:tasks.workunit.client.0.smithi107.stdout:0/676: chown d1/d27/d20/d2c/d41/dbe/fd5 26453 1 2024-04-22T23:35:10.110 INFO:tasks.workunit.client.0.smithi107.stdout:3/646: mknod db/d21/d27/d15/d29/d2a/d40/cdc 0 2024-04-22T23:35:10.112 INFO:tasks.workunit.client.0.smithi107.stdout:8/668: unlink d0/d7/d87/cb9 0 2024-04-22T23:35:10.112 INFO:tasks.workunit.client.0.smithi107.stdout:8/669: read - d0/d3a/d46/d59/d78/fc0 zero size 2024-04-22T23:35:10.113 INFO:tasks.workunit.client.0.smithi107.stdout:5/725: read d9/f43 [2443614,9377] 0 2024-04-22T23:35:10.113 INFO:tasks.workunit.client.0.smithi107.stdout:5/726: readlink d9/db/d94/lc4 0 2024-04-22T23:35:10.114 INFO:tasks.workunit.client.0.smithi107.stdout:3/647: unlink db/d21/d27/d15/d29/d2a/d45/lb9 0 2024-04-22T23:35:10.116 INFO:tasks.workunit.client.0.smithi107.stdout:8/670: rename d0/d85/d51/ca6 to d0/d7/df/d3b/dd9/cea 0 2024-04-22T23:35:10.116 INFO:tasks.workunit.client.1.smithi157.stdout:4/936: dwrite d3/d4b/f16 [0,4194304] 0 2024-04-22T23:35:10.118 INFO:tasks.workunit.client.0.smithi107.stdout:0/677: getdents d1/d27/d20 0 2024-04-22T23:35:10.121 INFO:tasks.workunit.client.0.smithi107.stdout:4/724: dwrite d1/d1b/fca [0,4194304] 0 2024-04-22T23:35:10.121 INFO:tasks.workunit.client.0.smithi107.stdout:4/725: fdatasync d1/d21/d29/d2e/d88/d99/dd2/daf/dc1/f9e 0 2024-04-22T23:35:10.123 INFO:tasks.workunit.client.1.smithi157.stdout:4/937: mkdir d3/d11/d1d/d73/d80/dac/d13c 0 2024-04-22T23:35:10.123 INFO:tasks.workunit.client.1.smithi157.stdout:4/938: readlink d3/d4b/d7a/l10f 0 2024-04-22T23:35:10.123 INFO:tasks.workunit.client.0.smithi107.stdout:1/608: dwrite d7/d19/d20/fbd [0,4194304] 0 2024-04-22T23:35:10.124 INFO:tasks.workunit.client.1.smithi157.stdout:5/928: dwrite d1/db0/d7a/f87 [4194304,4194304] 0 2024-04-22T23:35:10.125 INFO:tasks.workunit.client.1.smithi157.stdout:5/929: write d1/d5/d7/d10/d14f/dc6/fea [1027598,10828] 0 2024-04-22T23:35:10.125 INFO:tasks.workunit.client.0.smithi107.stdout:2/702: dwrite d1/d2b/d49/fc5 [0,4194304] 0 2024-04-22T23:35:10.125 INFO:tasks.workunit.client.0.smithi107.stdout:0/678: getdents d1/d27/d20/d2c/d2e 0 2024-04-22T23:35:10.126 INFO:tasks.workunit.client.1.smithi157.stdout:4/939: rmdir d3/d96/d4c/d133 0 2024-04-22T23:35:10.127 INFO:tasks.workunit.client.0.smithi107.stdout:3/648: rename db/d21/d27/d54 to db/d21/d27/d15/d29/d2a/d45/ddd 0 2024-04-22T23:35:10.127 INFO:tasks.workunit.client.1.smithi157.stdout:5/930: truncate d1/d5/d7/f67 2723749 0 2024-04-22T23:35:10.129 INFO:tasks.workunit.client.0.smithi107.stdout:1/609: symlink d7/d19/lca 0 2024-04-22T23:35:10.132 INFO:tasks.workunit.client.0.smithi107.stdout:6/712: dwrite f6 [0,4194304] 0 2024-04-22T23:35:10.132 INFO:tasks.workunit.client.0.smithi107.stdout:3/649: mknod db/d21/d27/d36/cde 0 2024-04-22T23:35:10.133 INFO:tasks.workunit.client.0.smithi107.stdout:3/650: write db/d21/d27/f39 [2899525,55761] 0 2024-04-22T23:35:10.135 INFO:tasks.workunit.client.0.smithi107.stdout:3/651: mknod db/d21/d27/d15/d24/d52/d60/dca/dcc/cdf 0 2024-04-22T23:35:10.135 INFO:tasks.workunit.client.0.smithi107.stdout:3/652: fdatasync db/d21/d27/d15/d24/d52/fbd 0 2024-04-22T23:35:10.136 INFO:tasks.workunit.client.1.smithi157.stdout:7/807: dread d0/d9/db/f61 [0,4194304] 0 2024-04-22T23:35:10.138 INFO:tasks.workunit.client.1.smithi157.stdout:7/808: rmdir d0/d9/db/df/d60 39 2024-04-22T23:35:10.138 INFO:tasks.workunit.client.1.smithi157.stdout:7/809: stat d0/d9/db 0 2024-04-22T23:35:10.140 INFO:tasks.workunit.client.0.smithi107.stdout:9/689: dwrite d5/d7/d27/d46/fd5 [0,4194304] 0 2024-04-22T23:35:10.140 INFO:tasks.workunit.client.0.smithi107.stdout:6/713: dread db/d20/d2a/d38/f3c [0,4194304] 0 2024-04-22T23:35:10.140 INFO:tasks.workunit.client.0.smithi107.stdout:5/727: dwrite d9/db/d42/f6c [0,4194304] 0 2024-04-22T23:35:10.140 INFO:tasks.workunit.client.0.smithi107.stdout:6/714: chown db/f1f 0 1 2024-04-22T23:35:10.141 INFO:tasks.workunit.client.0.smithi107.stdout:5/728: write d9/db/de/fd9 [737015,115254] 0 2024-04-22T23:35:10.141 INFO:tasks.workunit.client.0.smithi107.stdout:5/729: chown d9/db/d92/dc2/fef 6788506 1 2024-04-22T23:35:10.141 INFO:tasks.workunit.client.0.smithi107.stdout:5/730: chown d9/db/de/d72/d7c/da4/fb3 2889 1 2024-04-22T23:35:10.141 INFO:tasks.workunit.client.0.smithi107.stdout:5/731: write d9/db/d42/f5a [4937097,76954] 0 2024-04-22T23:35:10.141 INFO:tasks.workunit.client.0.smithi107.stdout:3/653: creat db/d21/d27/d15/d29/d2a/d45/ddd/d25/fe0 x:0 0 0 2024-04-22T23:35:10.143 INFO:tasks.workunit.client.0.smithi107.stdout:7/635: dwrite d7/d21/d48/f5b [0,4194304] 0 2024-04-22T23:35:10.143 INFO:tasks.workunit.client.0.smithi107.stdout:6/715: link db/d20/d58/cbf db/d20/d2a/d35/d69/db3/dbc/cdc 0 2024-04-22T23:35:10.145 INFO:tasks.workunit.client.1.smithi157.stdout:4/940: write d3/d11/d1d/d73/da3/d31/d91/f9b [1198279,49626] 0 2024-04-22T23:35:10.145 INFO:tasks.workunit.client.0.smithi107.stdout:5/732: symlink d9/db/d92/ddb/lf7 0 2024-04-22T23:35:10.146 INFO:tasks.workunit.client.0.smithi107.stdout:5/733: write d9/db/de/d5e/f71 [1747702,89701] 0 2024-04-22T23:35:10.148 INFO:tasks.workunit.client.0.smithi107.stdout:3/654: mkdir db/d21/d27/d15/d24/de1 0 2024-04-22T23:35:10.151 INFO:tasks.workunit.client.1.smithi157.stdout:7/810: dread d0/d9/d29/d33/d37/f4c [0,4194304] 0 2024-04-22T23:35:10.151 INFO:tasks.workunit.client.1.smithi157.stdout:7/811: creat d0/d9/db/df/d68/ff4 x:0 0 0 2024-04-22T23:35:10.154 INFO:tasks.workunit.client.0.smithi107.stdout:6/716: write db/d20/d2a/d38/d4b/f6b [2455402,113419] 0 2024-04-22T23:35:10.155 INFO:tasks.workunit.client.1.smithi157.stdout:7/812: unlink d0/d9/db/df/lb0 0 2024-04-22T23:35:10.155 INFO:tasks.workunit.client.1.smithi157.stdout:7/813: write d0/d9/d42/d55/d80/d95/ff3 [751114,35554] 0 2024-04-22T23:35:10.155 INFO:tasks.workunit.client.0.smithi107.stdout:2/703: dwrite d1/d12/d35/f45 [0,4194304] 0 2024-04-22T23:35:10.156 INFO:tasks.workunit.client.0.smithi107.stdout:2/704: fdatasync d1/d2b/d49/fa3 0 2024-04-22T23:35:10.156 INFO:tasks.workunit.client.0.smithi107.stdout:1/610: dwrite d7/fc1 [0,4194304] 0 2024-04-22T23:35:10.156 INFO:tasks.workunit.client.0.smithi107.stdout:1/611: dread - d7/d19/d71/f96 zero size 2024-04-22T23:35:10.156 INFO:tasks.workunit.client.1.smithi157.stdout:7/814: symlink d0/d9/d42/d55/d80/d95/dce/lf5 0 2024-04-22T23:35:10.156 INFO:tasks.workunit.client.1.smithi157.stdout:7/815: chown d0/le3 55 1 2024-04-22T23:35:10.156 INFO:tasks.workunit.client.1.smithi157.stdout:7/816: truncate d0/d9/d29/d5e/d85/fcf 267049 0 2024-04-22T23:35:10.157 INFO:tasks.workunit.client.1.smithi157.stdout:7/817: creat d0/d9/d29/d49/d4a/ff6 x:0 0 0 2024-04-22T23:35:10.157 INFO:tasks.workunit.client.0.smithi107.stdout:5/734: creat d9/db/de/d72/ff8 x:0 0 0 2024-04-22T23:35:10.159 INFO:tasks.workunit.client.0.smithi107.stdout:0/679: dwrite d1/d27/d20/d2c/d41/f64 [0,4194304] 0 2024-04-22T23:35:10.159 INFO:tasks.workunit.client.0.smithi107.stdout:3/655: creat db/d21/d27/d15/d29/d2a/d45/ddd/d25/fe2 x:0 0 0 2024-04-22T23:35:10.159 INFO:tasks.workunit.client.0.smithi107.stdout:3/656: chown db/c13 3 1 2024-04-22T23:35:10.160 INFO:tasks.workunit.client.1.smithi157.stdout:5/931: dwrite d1/db0/dff/f101 [0,4194304] 0 2024-04-22T23:35:10.162 INFO:tasks.workunit.client.0.smithi107.stdout:8/671: truncate d0/d7/df/fca 375399 0 2024-04-22T23:35:10.162 INFO:tasks.workunit.client.0.smithi107.stdout:8/672: fdatasync d0/d7/df/d3b/dd9/dda/fe8 0 2024-04-22T23:35:10.162 INFO:tasks.workunit.client.0.smithi107.stdout:8/673: truncate d0/d3a/d46/d59/d78/fc0 549296 0 2024-04-22T23:35:10.163 INFO:tasks.workunit.client.1.smithi157.stdout:4/941: dread d3/d11/d1d/d73/da3/fa9 [0,4194304] 0 2024-04-22T23:35:10.164 INFO:tasks.workunit.client.1.smithi157.stdout:5/932: symlink d1/d5/dc/d5d/d77/l150 0 2024-04-22T23:35:10.165 INFO:tasks.workunit.client.1.smithi157.stdout:5/933: chown d1/d5/f10f 4894636 1 2024-04-22T23:35:10.166 INFO:tasks.workunit.client.0.smithi107.stdout:6/717: write db/f5e [661103,50244] 0 2024-04-22T23:35:10.170 INFO:tasks.workunit.client.0.smithi107.stdout:1/612: symlink d7/d19/d2d/lcb 0 2024-04-22T23:35:10.171 INFO:tasks.workunit.client.0.smithi107.stdout:7/636: link d7/d21/d48/d5c/d9d/c60 d7/d21/d48/cd1 0 2024-04-22T23:35:10.171 INFO:tasks.workunit.client.1.smithi157.stdout:4/942: rename d3/d11/d1d/d73/l9e to d3/d11/d1d/daf/l13d 0 2024-04-22T23:35:10.179 INFO:tasks.workunit.client.1.smithi157.stdout:4/943: write d3/d96/d40/fab [699659,5683] 0 2024-04-22T23:35:10.180 INFO:tasks.workunit.client.0.smithi107.stdout:2/705: read d1/d2b/d3d/d57/f5c [288568,129684] 0 2024-04-22T23:35:10.180 INFO:tasks.workunit.client.0.smithi107.stdout:2/706: chown d1/d2b/d49/d7e/fc8 21849954 1 2024-04-22T23:35:10.181 INFO:tasks.workunit.client.0.smithi107.stdout:2/707: stat d1/d67/f9f 0 2024-04-22T23:35:10.181 INFO:tasks.workunit.client.0.smithi107.stdout:2/708: fdatasync d1/d2b/d49/d7e/da8/fb8 0 2024-04-22T23:35:10.190 INFO:tasks.workunit.client.0.smithi107.stdout:5/735: dwrite d9/db/d42/f4b [0,4194304] 0 2024-04-22T23:35:10.191 INFO:tasks.workunit.client.0.smithi107.stdout:5/736: creat d9/db/d92/dc2/ff9 x:0 0 0 2024-04-22T23:35:10.191 INFO:tasks.workunit.client.0.smithi107.stdout:6/718: dwrite db/d20/d2a/d38/d4b/f54 [4194304,4194304] 0 2024-04-22T23:35:10.191 INFO:tasks.workunit.client.0.smithi107.stdout:6/719: stat db/d20/d2c/d3b/l85 0 2024-04-22T23:35:10.192 INFO:tasks.workunit.client.0.smithi107.stdout:8/674: creat d0/d7/df/d33/d37/d9f/feb x:0 0 0 2024-04-22T23:35:10.194 INFO:tasks.workunit.client.0.smithi107.stdout:8/675: dread d0/f83 [0,4194304] 0 2024-04-22T23:35:10.195 INFO:tasks.workunit.client.0.smithi107.stdout:8/676: chown d0/f96 211 1 2024-04-22T23:35:10.195 INFO:tasks.workunit.client.1.smithi157.stdout:4/944: dread d3/d11/d1d/f11e [0,4194304] 0 2024-04-22T23:35:10.195 INFO:tasks.workunit.client.1.smithi157.stdout:4/945: readlink d3/d5c/dd3/l10d 0 2024-04-22T23:35:10.195 INFO:tasks.workunit.client.1.smithi157.stdout:4/946: mkdir d3/d11/d1d/d73/d80/dac/db5/df7/d13e 0 2024-04-22T23:35:10.196 INFO:tasks.workunit.client.1.smithi157.stdout:4/947: truncate d3/d5c/fca 4047970 0 2024-04-22T23:35:10.197 INFO:tasks.workunit.client.1.smithi157.stdout:4/948: unlink d3/fda 0 2024-04-22T23:35:10.197 INFO:tasks.workunit.client.0.smithi107.stdout:7/637: symlink d7/d21/d7e/d94/ld2 0 2024-04-22T23:35:10.197 INFO:tasks.workunit.client.0.smithi107.stdout:7/638: dread - d7/d21/d7e/d94/fba zero size 2024-04-22T23:35:10.198 INFO:tasks.workunit.client.0.smithi107.stdout:3/657: dread - db/d21/d27/d36/d9c/fa3 zero size 2024-04-22T23:35:10.199 INFO:tasks.workunit.client.0.smithi107.stdout:2/709: dwrite d1/d2b/d88/fd4 [0,4194304] 0 2024-04-22T23:35:10.199 INFO:tasks.workunit.client.0.smithi107.stdout:2/710: chown d1/d63/dbb 0 1 2024-04-22T23:35:10.202 INFO:tasks.workunit.client.0.smithi107.stdout:6/720: dread db/d20/d49/d7b/fb4 [0,4194304] 0 2024-04-22T23:35:10.202 INFO:tasks.workunit.client.0.smithi107.stdout:5/737: symlink d9/de6/lfa 0 2024-04-22T23:35:10.204 INFO:tasks.workunit.client.1.smithi157.stdout:4/949: dread d3/d96/f3d [0,4194304] 0 2024-04-22T23:35:10.204 INFO:tasks.workunit.client.1.smithi157.stdout:4/950: write d3/d11/d1d/d73/da3/d31/d91/fa5 [650141,7495] 0 2024-04-22T23:35:10.206 INFO:tasks.workunit.client.0.smithi107.stdout:8/677: rename d0/d7/df/d17 to d0/d7/d8/d41/dec 0 2024-04-22T23:35:10.208 INFO:tasks.workunit.client.0.smithi107.stdout:1/613: link d7/c1d d7/d19/ccc 0 2024-04-22T23:35:10.210 INFO:tasks.workunit.client.0.smithi107.stdout:7/639: rmdir d7/d21/d48/d5c/d9d 39 2024-04-22T23:35:10.210 INFO:tasks.workunit.client.0.smithi107.stdout:7/640: write d7/d21/da4/f41 [984515,35576] 0 2024-04-22T23:35:10.211 INFO:tasks.workunit.client.0.smithi107.stdout:3/658: getdents db/d21/d27/d15/d29/d2a/d99 0 2024-04-22T23:35:10.212 INFO:tasks.workunit.client.0.smithi107.stdout:8/678: dread d0/d3a/d46/d59/f6e [0,4194304] 0 2024-04-22T23:35:10.213 INFO:tasks.workunit.client.0.smithi107.stdout:5/738: dread d9/db/de/d72/d7c/da4/fb3 [0,4194304] 0 2024-04-22T23:35:10.213 INFO:tasks.workunit.client.0.smithi107.stdout:5/739: readlink d9/d26/le1 0 2024-04-22T23:35:10.213 INFO:tasks.workunit.client.0.smithi107.stdout:5/740: creat d9/db/de/d72/ffb x:0 0 0 2024-04-22T23:35:10.214 INFO:tasks.workunit.client.1.smithi157.stdout:4/951: dread d3/d4b/f45 [0,4194304] 0 2024-04-22T23:35:10.214 INFO:tasks.workunit.client.1.smithi157.stdout:4/952: stat d3/d96/f9a 0 2024-04-22T23:35:10.215 INFO:tasks.workunit.client.0.smithi107.stdout:2/711: mkdir d1/d63/de6 0 2024-04-22T23:35:10.216 INFO:tasks.workunit.client.0.smithi107.stdout:1/614: symlink d7/d19/d20/d39/d53/d43/d46/d5c/db7/lcd 0 2024-04-22T23:35:10.219 INFO:tasks.workunit.client.0.smithi107.stdout:7/641: mkdir d7/d21/d54/dd3 0 2024-04-22T23:35:10.219 INFO:tasks.workunit.client.0.smithi107.stdout:7/642: dread - d7/d21/d7e/d94/fc4 zero size 2024-04-22T23:35:10.219 INFO:tasks.workunit.client.0.smithi107.stdout:7/643: readlink d7/d21/d48/d5c/l67 0 2024-04-22T23:35:10.221 INFO:tasks.workunit.client.0.smithi107.stdout:6/721: dwrite db/d20/d2c/d3b/fa8 [0,4194304] 0 2024-04-22T23:35:10.222 INFO:tasks.workunit.client.0.smithi107.stdout:5/741: link d9/db/dc1/dcb/ff6 d9/db/de/d72/d7c/ffc 0 2024-04-22T23:35:10.223 INFO:tasks.workunit.client.0.smithi107.stdout:8/679: dread d0/d7/d8/d41/d53/f57 [0,4194304] 0 2024-04-22T23:35:10.223 INFO:tasks.workunit.client.0.smithi107.stdout:8/680: chown d0/d7/d8/d41/d53/lc4 327608 1 2024-04-22T23:35:10.224 INFO:tasks.workunit.client.0.smithi107.stdout:2/712: symlink d1/d63/dbb/le7 0 2024-04-22T23:35:10.226 INFO:tasks.workunit.client.0.smithi107.stdout:8/681: dread d0/d7/df/d33/d9b/fb8 [0,4194304] 0 2024-04-22T23:35:10.228 INFO:tasks.workunit.client.1.smithi157.stdout:3/874: sync 2024-04-22T23:35:10.228 INFO:tasks.workunit.client.1.smithi157.stdout:3/875: fsync d0/d7/d6c/fc7 0 2024-04-22T23:35:10.228 INFO:tasks.workunit.client.1.smithi157.stdout:3/876: fdatasync d0/d3/d34/d3d/d57/d60/d6b/d97/dcb/fff 0 2024-04-22T23:35:10.228 INFO:tasks.workunit.client.1.smithi157.stdout:1/904: sync 2024-04-22T23:35:10.229 INFO:tasks.workunit.client.0.smithi107.stdout:5/742: stat d9/d26/d6a/ca8 0 2024-04-22T23:35:10.229 INFO:tasks.workunit.client.0.smithi107.stdout:5/743: write d9/d76/fc8 [876981,118804] 0 2024-04-22T23:35:10.231 INFO:tasks.workunit.client.0.smithi107.stdout:2/713: mkdir d1/d2b/d49/d7e/dae/de8 0 2024-04-22T23:35:10.231 INFO:tasks.workunit.client.1.smithi157.stdout:3/877: unlink d0/d3/d34/d39/l2d 0 2024-04-22T23:35:10.231 INFO:tasks.workunit.client.1.smithi157.stdout:1/905: symlink d4/d17/d34/d3b/d52/d58/l13c 0 2024-04-22T23:35:10.231 INFO:tasks.workunit.client.1.smithi157.stdout:1/906: write d4/d17/d34/d3b/d52/d65/df1/f119 [177836,43547] 0 2024-04-22T23:35:10.233 INFO:tasks.workunit.client.1.smithi157.stdout:1/907: symlink d4/d17/d1d/df5/l13d 0 2024-04-22T23:35:10.233 INFO:tasks.workunit.client.0.smithi107.stdout:1/615: mkdir d7/dce 0 2024-04-22T23:35:10.233 INFO:tasks.workunit.client.0.smithi107.stdout:8/682: mkdir d0/d7/d8/ded 0 2024-04-22T23:35:10.234 INFO:tasks.workunit.client.1.smithi157.stdout:3/878: dread d0/d3/d34/d3d/f63 [0,4194304] 0 2024-04-22T23:35:10.234 INFO:tasks.workunit.client.1.smithi157.stdout:3/879: truncate d0/d3/d34/d39/f56 4684994 0 2024-04-22T23:35:10.238 INFO:tasks.workunit.client.0.smithi107.stdout:5/744: rename d9/d26/d6a/ldc to d9/db/d92/lfd 0 2024-04-22T23:35:10.239 INFO:tasks.workunit.client.1.smithi157.stdout:1/908: mkdir d4/d17/db9/d13e 0 2024-04-22T23:35:10.239 INFO:tasks.workunit.client.1.smithi157.stdout:1/909: write d4/d17/d1d/db2/fe1 [1001568,75585] 0 2024-04-22T23:35:10.239 INFO:tasks.workunit.client.1.smithi157.stdout:3/880: symlink d0/d3/d34/d3d/d57/d60/d6b/d129/l12a 0 2024-04-22T23:35:10.239 INFO:tasks.workunit.client.1.smithi157.stdout:3/881: chown d0/d3/d34/d3d/d57/d60/d9c/laf 41 1 2024-04-22T23:35:10.241 INFO:tasks.workunit.client.1.smithi157.stdout:1/910: creat d4/d17/d34/d75/d8a/dfc/f13f x:0 0 0 2024-04-22T23:35:10.241 INFO:tasks.workunit.client.0.smithi107.stdout:6/722: dread db/d20/d2c/d3b/f60 [4194304,4194304] 0 2024-04-22T23:35:10.241 INFO:tasks.workunit.client.0.smithi107.stdout:1/616: rename d7/d19/d20/d39/d53/d43/d46/d5c/f5e to d7/d19/d2d/fcf 0 2024-04-22T23:35:10.242 INFO:tasks.workunit.client.0.smithi107.stdout:5/745: symlink d9/db/d92/ddb/lfe 0 2024-04-22T23:35:10.244 INFO:tasks.workunit.client.1.smithi157.stdout:4/953: dwrite d3/d11/d30/f4d [0,4194304] 0 2024-04-22T23:35:10.245 INFO:tasks.workunit.client.0.smithi107.stdout:8/683: dread d0/d7/d8/d41/dec/f40 [0,4194304] 0 2024-04-22T23:35:10.246 INFO:tasks.workunit.client.0.smithi107.stdout:1/617: truncate d7/f10 3117423 0 2024-04-22T23:35:10.248 INFO:tasks.workunit.client.1.smithi157.stdout:4/954: symlink d3/d11/d1d/d73/d80/dac/db5/dc8/l13f 0 2024-04-22T23:35:10.249 INFO:tasks.workunit.client.0.smithi107.stdout:1/618: rename d7/d19/d20/d39/d53/d43/d63/f83 to d7/d19/d2d/d89/fd0 0 2024-04-22T23:35:10.249 INFO:tasks.workunit.client.0.smithi107.stdout:1/619: creat d7/d19/d20/d39/d53/d43/d46/fd1 x:0 0 0 2024-04-22T23:35:10.252 INFO:tasks.workunit.client.0.smithi107.stdout:1/620: rename d7/d19/d20/d39/d53/d43/d46/f47 to d7/d19/d20/d39/d53/d43/d46/d99/fd2 0 2024-04-22T23:35:10.253 INFO:tasks.workunit.client.0.smithi107.stdout:5/746: dread d9/fa5 [0,4194304] 0 2024-04-22T23:35:10.253 INFO:tasks.workunit.client.0.smithi107.stdout:2/714: dwrite d1/d63/f69 [0,4194304] 0 2024-04-22T23:35:10.253 INFO:tasks.workunit.client.0.smithi107.stdout:5/747: chown d9/db/de/dbe/ld4 948542012 1 2024-04-22T23:35:10.254 INFO:tasks.workunit.client.0.smithi107.stdout:7/644: dwrite d7/f63 [4194304,4194304] 0 2024-04-22T23:35:10.254 INFO:tasks.workunit.client.0.smithi107.stdout:7/645: chown d7/d21/d48/d5c/f75 1 1 2024-04-22T23:35:10.259 INFO:tasks.workunit.client.0.smithi107.stdout:1/621: dread d7/d19/d2d/fcf [0,4194304] 0 2024-04-22T23:35:10.260 INFO:tasks.workunit.client.1.smithi157.stdout:4/955: read d3/d4b/f5 [3322041,56159] 0 2024-04-22T23:35:10.260 INFO:tasks.workunit.client.1.smithi157.stdout:4/956: write d3/d11/d34/da6/f131 [96180,114607] 0 2024-04-22T23:35:10.260 INFO:tasks.workunit.client.1.smithi157.stdout:4/957: write d3/d4b/d7a/db7/f109 [251583,91782] 0 2024-04-22T23:35:10.260 INFO:tasks.workunit.client.1.smithi157.stdout:4/958: chown d3/d11/d1d/d73/da3/d31/d91/lfa 2235820 1 2024-04-22T23:35:10.261 INFO:tasks.workunit.client.0.smithi107.stdout:5/748: creat d9/de6/fff x:0 0 0 2024-04-22T23:35:10.261 INFO:tasks.workunit.client.0.smithi107.stdout:7/646: fsync d7/d21/d48/f6b 0 2024-04-22T23:35:10.261 INFO:tasks.workunit.client.0.smithi107.stdout:6/723: write db/d20/d2c/d3b/fa8 [1343180,18] 0 2024-04-22T23:35:10.261 INFO:tasks.workunit.client.0.smithi107.stdout:6/724: readlink db/d20/d2a/d38/d61/d6f/da2/d84/lc5 0 2024-04-22T23:35:10.261 INFO:tasks.workunit.client.0.smithi107.stdout:1/622: readlink d7/l8 0 2024-04-22T23:35:10.261 INFO:tasks.workunit.client.0.smithi107.stdout:7/647: creat d7/d21/d48/d95/fd4 x:0 0 0 2024-04-22T23:35:10.262 INFO:tasks.workunit.client.0.smithi107.stdout:8/684: dread d0/d7/f5e [0,4194304] 0 2024-04-22T23:35:10.264 INFO:tasks.workunit.client.0.smithi107.stdout:6/725: mknod db/d20/d2a/d38/d59/d70/cdd 0 2024-04-22T23:35:10.264 INFO:tasks.workunit.client.0.smithi107.stdout:6/726: chown db/d20/d2a/d38/d61/d6f 0 1 2024-04-22T23:35:10.265 INFO:tasks.workunit.client.1.smithi157.stdout:4/959: rename d3/d4b/d7a/db7/fdf to d3/d11/d1d/d73/da3/d31/d91/dcb/dcd/f140 0 2024-04-22T23:35:10.265 INFO:tasks.workunit.client.1.smithi157.stdout:4/960: write d3/d11/d34/d9d/ff6 [406944,99546] 0 2024-04-22T23:35:10.265 INFO:tasks.workunit.client.1.smithi157.stdout:4/961: creat d3/d4b/d5d/f141 x:0 0 0 2024-04-22T23:35:10.265 INFO:tasks.workunit.client.0.smithi107.stdout:2/715: write d1/d12/d35/f45 [1824768,114300] 0 2024-04-22T23:35:10.265 INFO:tasks.workunit.client.0.smithi107.stdout:1/623: truncate d7/d19/d71/f78 3921091 0 2024-04-22T23:35:10.267 INFO:tasks.workunit.client.0.smithi107.stdout:7/648: symlink d7/d4c/da1/ld5 0 2024-04-22T23:35:10.268 INFO:tasks.workunit.client.1.smithi157.stdout:4/962: dread - d3/d5c/d7b/f105 zero size 2024-04-22T23:35:10.268 INFO:tasks.workunit.client.1.smithi157.stdout:4/963: dread - d3/d11/d1d/d73/fbd zero size 2024-04-22T23:35:10.269 INFO:tasks.workunit.client.1.smithi157.stdout:4/964: mkdir d3/d11/d1d/daf/d142 0 2024-04-22T23:35:10.270 INFO:tasks.workunit.client.0.smithi107.stdout:8/685: creat d0/d7/d87/dc8/fee x:0 0 0 2024-04-22T23:35:10.270 INFO:tasks.workunit.client.0.smithi107.stdout:8/686: chown d0/d3a/cb4 1996174307 1 2024-04-22T23:35:10.274 INFO:tasks.workunit.client.0.smithi107.stdout:1/624: mknod d7/d19/d71/cd3 0 2024-04-22T23:35:10.277 INFO:tasks.workunit.client.0.smithi107.stdout:8/687: mkdir d0/d7/d8/d41/d53/d76/def 0 2024-04-22T23:35:10.278 INFO:tasks.workunit.client.0.smithi107.stdout:1/625: link d7/d19/d20/c8e d7/d19/d20/d39/d93/cd4 0 2024-04-22T23:35:10.280 INFO:tasks.workunit.client.1.smithi157.stdout:1/911: dwrite d4/d17/d34/d75/d8a/f121 [0,4194304] 0 2024-04-22T23:35:10.280 INFO:tasks.workunit.client.1.smithi157.stdout:1/912: chown d4/db/d3d/dc1/c105 6 1 2024-04-22T23:35:10.281 INFO:tasks.workunit.client.0.smithi107.stdout:8/688: creat d0/d7/d8/ff0 x:0 0 0 2024-04-22T23:35:10.281 INFO:tasks.workunit.client.1.smithi157.stdout:3/882: dwrite d0/d3/d34/d3d/df6/dee/d9e/fe2 [4194304,4194304] 0 2024-04-22T23:35:10.281 INFO:tasks.workunit.client.0.smithi107.stdout:1/626: creat d7/d19/d71/db8/fd5 x:0 0 0 2024-04-22T23:35:10.282 INFO:tasks.workunit.client.0.smithi107.stdout:6/727: dread db/d20/d2a/d38/f9f [0,4194304] 0 2024-04-22T23:35:10.282 INFO:tasks.workunit.client.0.smithi107.stdout:6/728: write db/d20/d2c/d3b/d40/d9e/fb8 [4848146,127538] 0 2024-04-22T23:35:10.284 INFO:tasks.workunit.client.0.smithi107.stdout:7/649: rmdir d7/d4c/da1 39 2024-04-22T23:35:10.285 INFO:tasks.workunit.client.0.smithi107.stdout:2/716: write d1/d12/f14 [2931933,108173] 0 2024-04-22T23:35:10.286 INFO:tasks.workunit.client.0.smithi107.stdout:7/650: dread d7/f5f [0,4194304] 0 2024-04-22T23:35:10.286 INFO:tasks.workunit.client.1.smithi157.stdout:3/883: creat d0/d3/d34/d47/ded/f12b x:0 0 0 2024-04-22T23:35:10.287 INFO:tasks.workunit.client.1.smithi157.stdout:3/884: truncate d0/d3/d127/f10a 1028258 0 2024-04-22T23:35:10.287 INFO:tasks.workunit.client.1.smithi157.stdout:3/885: fdatasync d0/d3/fbe 0 2024-04-22T23:35:10.287 INFO:tasks.workunit.client.1.smithi157.stdout:4/965: dwrite d3/d11/d1d/daf/fe2 [0,4194304] 0 2024-04-22T23:35:10.287 INFO:tasks.workunit.client.1.smithi157.stdout:4/966: fdatasync d3/d11/d34/d9d/fc9 0 2024-04-22T23:35:10.287 INFO:tasks.workunit.client.0.smithi107.stdout:5/749: dwrite d9/db/d42/d75/fb1 [0,4194304] 0 2024-04-22T23:35:10.288 INFO:tasks.workunit.client.0.smithi107.stdout:6/729: dread db/f5e [0,4194304] 0 2024-04-22T23:35:10.288 INFO:tasks.workunit.client.0.smithi107.stdout:2/717: creat d1/d63/dbb/fe9 x:0 0 0 2024-04-22T23:35:10.288 INFO:tasks.workunit.client.0.smithi107.stdout:6/730: write db/d20/db1/fbe [557611,90199] 0 2024-04-22T23:35:10.288 INFO:tasks.workunit.client.0.smithi107.stdout:2/718: write d1/f4c [476279,118757] 0 2024-04-22T23:35:10.291 INFO:tasks.workunit.client.0.smithi107.stdout:2/719: creat d1/d2b/d49/d7e/dae/fea x:0 0 0 2024-04-22T23:35:10.292 INFO:tasks.workunit.client.1.smithi157.stdout:4/967: mknod d3/d11/d1d/d73/d80/dac/db5/dc8/c143 0 2024-04-22T23:35:10.293 INFO:tasks.workunit.client.0.smithi107.stdout:5/750: dread d9/d76/f7d [4194304,4194304] 0 2024-04-22T23:35:10.293 INFO:tasks.workunit.client.0.smithi107.stdout:7/651: getdents d7/d21/da4/d66 0 2024-04-22T23:35:10.293 INFO:tasks.workunit.client.1.smithi157.stdout:4/968: write d3/d11/d30/f71 [478761,130418] 0 2024-04-22T23:35:10.295 INFO:tasks.workunit.client.0.smithi107.stdout:2/720: link d1/d12/d35/d48/f1f d1/d12/d35/d48/d13/d4f/feb 0 2024-04-22T23:35:10.295 INFO:tasks.workunit.client.0.smithi107.stdout:2/721: readlink d1/lf 0 2024-04-22T23:35:10.295 INFO:tasks.workunit.client.0.smithi107.stdout:2/722: creat d1/d2b/d3d/d57/fec x:0 0 0 2024-04-22T23:35:10.296 INFO:tasks.workunit.client.1.smithi157.stdout:3/886: link d0/d7/d6c/cdf d0/d3/d34/d3d/df6/dee/d9e/c12c 0 2024-04-22T23:35:10.298 INFO:tasks.workunit.client.0.smithi107.stdout:5/751: mkdir d9/d26/d6a/d100 0 2024-04-22T23:35:10.298 INFO:tasks.workunit.client.1.smithi157.stdout:4/969: write d3/d11/d1d/d73/d80/f112 [644538,110028] 0 2024-04-22T23:35:10.298 INFO:tasks.workunit.client.0.smithi107.stdout:1/627: write d7/d19/d20/d39/d53/f4e [137077,65541] 0 2024-04-22T23:35:10.299 INFO:tasks.workunit.client.0.smithi107.stdout:2/723: creat d1/d2b/d49/d7e/dc6/fed x:0 0 0 2024-04-22T23:35:10.300 INFO:tasks.workunit.client.1.smithi157.stdout:1/913: truncate d4/d17/d34/d75/d8a/f121 778118 0 2024-04-22T23:35:10.301 INFO:tasks.workunit.client.1.smithi157.stdout:3/887: mknod d0/d3/d34/d3d/d57/d60/d9c/dd5/c12d 0 2024-04-22T23:35:10.301 INFO:tasks.workunit.client.1.smithi157.stdout:3/888: readlink d0/d3/d34/d3d/d57/l122 0 2024-04-22T23:35:10.301 INFO:tasks.workunit.client.1.smithi157.stdout:3/889: write d0/d3/d34/d4a/fdc [689991,37381] 0 2024-04-22T23:35:10.301 INFO:tasks.workunit.client.1.smithi157.stdout:3/890: readlink d0/d3/d34/lef 0 2024-04-22T23:35:10.302 INFO:tasks.workunit.client.1.smithi157.stdout:3/891: chown d0/d3/d34/d3d/d57/d5c/ffd 6 1 2024-04-22T23:35:10.302 INFO:tasks.workunit.client.0.smithi107.stdout:1/628: rename d7/d19/d20/d39/d53/d43/d46/f4c to d7/d19/d20/fd6 0 2024-04-22T23:35:10.302 INFO:tasks.workunit.client.0.smithi107.stdout:2/724: symlink d1/d2b/d49/d7e/dae/de8/lee 0 2024-04-22T23:35:10.303 INFO:tasks.workunit.client.1.smithi157.stdout:1/914: truncate d4/db/f36 3806585 0 2024-04-22T23:35:10.303 INFO:tasks.workunit.client.1.smithi157.stdout:1/915: readlink d4/d17/d34/d3b/l46 0 2024-04-22T23:35:10.303 INFO:tasks.workunit.client.0.smithi107.stdout:1/629: symlink d7/d19/d54/ld7 0 2024-04-22T23:35:10.304 INFO:tasks.workunit.client.1.smithi157.stdout:3/892: rmdir d0/d3/d34/d3d/df6 39 2024-04-22T23:35:10.304 INFO:tasks.workunit.client.1.smithi157.stdout:3/893: chown d0/d3/d34/d3d/d57/d60/d6b/f85 353136 1 2024-04-22T23:35:10.304 INFO:tasks.workunit.client.1.smithi157.stdout:4/970: mknod d3/d96/d4c/de7/d125/c144 0 2024-04-22T23:35:10.304 INFO:tasks.workunit.client.1.smithi157.stdout:4/971: fdatasync d3/d96/d40/fab 0 2024-04-22T23:35:10.305 INFO:tasks.workunit.client.0.smithi107.stdout:2/725: getdents d1/d2b/d88 0 2024-04-22T23:35:10.305 INFO:tasks.workunit.client.0.smithi107.stdout:2/726: fdatasync d1/d63/f65 0 2024-04-22T23:35:10.306 INFO:tasks.workunit.client.1.smithi157.stdout:3/894: unlink d0/d3/d34/db9/fe9 0 2024-04-22T23:35:10.306 INFO:tasks.workunit.client.1.smithi157.stdout:3/895: dread - d0/d3/d34/d3d/d57/d60/d6b/d97/fb5 zero size 2024-04-22T23:35:10.307 INFO:tasks.workunit.client.0.smithi107.stdout:1/630: unlink d7/d19/d20/d39/d53/d43/l9c 0 2024-04-22T23:35:10.308 INFO:tasks.workunit.client.1.smithi157.stdout:4/972: symlink d3/d4b/d5d/l145 0 2024-04-22T23:35:10.309 INFO:tasks.workunit.client.0.smithi107.stdout:2/727: mkdir d1/d2b/d49/dba/def 0 2024-04-22T23:35:10.310 INFO:tasks.workunit.client.0.smithi107.stdout:1/631: mknod d7/d19/d20/d39/d53/cd8 0 2024-04-22T23:35:10.310 INFO:tasks.workunit.client.0.smithi107.stdout:1/632: chown d7/d19/l2b 57545074 1 2024-04-22T23:35:10.312 INFO:tasks.workunit.client.0.smithi107.stdout:8/689: dwrite d0/d7/f5e [0,4194304] 0 2024-04-22T23:35:10.312 INFO:tasks.workunit.client.0.smithi107.stdout:6/731: dwrite db/d20/d2a/d38/d61/f6d [0,4194304] 0 2024-04-22T23:35:10.312 INFO:tasks.workunit.client.0.smithi107.stdout:6/732: chown db/d20/d2a/d5f/cae 11236 1 2024-04-22T23:35:10.313 INFO:tasks.workunit.client.0.smithi107.stdout:2/728: mkdir d1/dcd/df0 0 2024-04-22T23:35:10.322 INFO:tasks.workunit.client.1.smithi157.stdout:1/916: dwrite d4/db/d3d/dc1/d100/f107 [0,4194304] 0 2024-04-22T23:35:10.324 INFO:tasks.workunit.client.0.smithi107.stdout:6/733: link db/d20/d2a/l74 db/d20/d2a/d38/d59/d70/lde 0 2024-04-22T23:35:10.326 INFO:tasks.workunit.client.1.smithi157.stdout:1/917: dread d4/db/f7d [0,4194304] 0 2024-04-22T23:35:10.327 INFO:tasks.workunit.client.0.smithi107.stdout:1/633: symlink d7/d19/d20/d39/d53/d43/ld9 0 2024-04-22T23:35:10.328 INFO:tasks.workunit.client.1.smithi157.stdout:1/918: rename d4/d17/d34/l53 to d4/d5f/d7e/l140 0 2024-04-22T23:35:10.329 INFO:tasks.workunit.client.0.smithi107.stdout:8/690: getdents d0/d7/df/d33/d37 0 2024-04-22T23:35:10.331 INFO:tasks.workunit.client.1.smithi157.stdout:3/896: dwrite d0/d3/fb3 [0,4194304] 0 2024-04-22T23:35:10.331 INFO:tasks.workunit.client.1.smithi157.stdout:4/973: dwrite d3/d11/d1d/daf/fe2 [0,4194304] 0 2024-04-22T23:35:10.331 INFO:tasks.workunit.client.1.smithi157.stdout:3/897: stat d0/d3/d34/d39/c8e 0 2024-04-22T23:35:10.331 INFO:tasks.workunit.client.0.smithi107.stdout:6/734: rename db/f1c to db/fdf 0 2024-04-22T23:35:10.334 INFO:tasks.workunit.client.0.smithi107.stdout:1/634: unlink d7/d19/d20/d39/d53/d43/d46/d99/c9a 0 2024-04-22T23:35:10.335 INFO:tasks.workunit.client.0.smithi107.stdout:8/691: creat d0/d7/d20/d89/ff1 x:0 0 0 2024-04-22T23:35:10.337 INFO:tasks.workunit.client.0.smithi107.stdout:6/735: symlink db/d20/d2c/d3b/le0 0 2024-04-22T23:35:10.337 INFO:tasks.workunit.client.0.smithi107.stdout:2/729: dwrite d1/d12/d35/d48/d13/f74 [0,4194304] 0 2024-04-22T23:35:10.339 INFO:tasks.workunit.client.0.smithi107.stdout:8/692: symlink d0/d7/d20/lf2 0 2024-04-22T23:35:10.339 INFO:tasks.workunit.client.0.smithi107.stdout:8/693: dread - d0/d7/d20/fbd zero size 2024-04-22T23:35:10.339 INFO:tasks.workunit.client.0.smithi107.stdout:1/635: getdents d7/d19/d2d/da4 0 2024-04-22T23:35:10.339 INFO:tasks.workunit.client.0.smithi107.stdout:1/636: truncate d7/d19/d2d/f31 4229345 0 2024-04-22T23:35:10.340 INFO:tasks.workunit.client.0.smithi107.stdout:1/637: chown d7/d19/d2d/f9b 318 1 2024-04-22T23:35:10.341 INFO:tasks.workunit.client.1.smithi157.stdout:4/974: creat d3/d11/d34/da4/db1/f146 x:0 0 0 2024-04-22T23:35:10.341 INFO:tasks.workunit.client.1.smithi157.stdout:4/975: dread - d3/d11/d30/dfd/f11c zero size 2024-04-22T23:35:10.341 INFO:tasks.workunit.client.1.smithi157.stdout:4/976: stat d3/d5c/d7b 0 2024-04-22T23:35:10.342 INFO:tasks.workunit.client.1.smithi157.stdout:3/898: getdents d0/d3/d13 0 2024-04-22T23:35:10.342 INFO:tasks.workunit.client.1.smithi157.stdout:1/919: dwrite d4/d17/d34/d3b/d52/fbe [4194304,4194304] 0 2024-04-22T23:35:10.342 INFO:tasks.workunit.client.1.smithi157.stdout:4/977: mknod d3/d11/d34/da4/c147 0 2024-04-22T23:35:10.343 INFO:tasks.workunit.client.1.smithi157.stdout:4/978: read d3/d11/d1d/d73/da3/d31/f10e [67021,70510] 0 2024-04-22T23:35:10.343 INFO:tasks.workunit.client.1.smithi157.stdout:4/979: read d3/d11/d1d/d73/da3/d31/f10e [59942,104048] 0 2024-04-22T23:35:10.343 INFO:tasks.workunit.client.1.smithi157.stdout:4/980: dread - d3/d11/d1d/d73/fbd zero size 2024-04-22T23:35:10.344 INFO:tasks.workunit.client.0.smithi107.stdout:0/680: sync 2024-04-22T23:35:10.344 INFO:tasks.workunit.client.0.smithi107.stdout:3/659: sync 2024-04-22T23:35:10.344 INFO:tasks.workunit.client.0.smithi107.stdout:9/690: sync 2024-04-22T23:35:10.344 INFO:tasks.workunit.client.0.smithi107.stdout:4/726: sync 2024-04-22T23:35:10.344 INFO:tasks.workunit.client.0.smithi107.stdout:9/691: chown d5/c80 520 1 2024-04-22T23:35:10.344 INFO:tasks.workunit.client.0.smithi107.stdout:4/727: chown d1/d21/d29/d2e/d88/d99/dd2/l4a 4 1 2024-04-22T23:35:10.344 INFO:tasks.workunit.client.0.smithi107.stdout:3/660: chown db/d21/d27/d15/d29/d2a/d45/ddd/d25/fe0 174242768 1 2024-04-22T23:35:10.344 INFO:tasks.workunit.client.0.smithi107.stdout:0/681: fdatasync d1/d27/d20/d40/dbd/fce 0 2024-04-22T23:35:10.345 INFO:tasks.workunit.client.1.smithi157.stdout:4/981: unlink d3/d11/d34/da4/cc7 0 2024-04-22T23:35:10.346 INFO:tasks.workunit.client.0.smithi107.stdout:1/638: rename d7/d19/d42/c9f to d7/d19/d20/d39/cda 0 2024-04-22T23:35:10.346 INFO:tasks.workunit.client.0.smithi107.stdout:1/639: fsync d7/d19/d20/d39/d53/f5f 0 2024-04-22T23:35:10.347 INFO:tasks.workunit.client.0.smithi107.stdout:8/694: rmdir d0/d7/df/d33/d37/d9f 39 2024-04-22T23:35:10.347 INFO:tasks.workunit.client.0.smithi107.stdout:8/695: write d0/d7/d20/d89/fc5 [3037964,130865] 0 2024-04-22T23:35:10.348 INFO:tasks.workunit.client.0.smithi107.stdout:2/730: dread d1/f11 [0,4194304] 0 2024-04-22T23:35:10.349 INFO:tasks.workunit.client.0.smithi107.stdout:9/692: creat d5/d7/d27/d46/ff0 x:0 0 0 2024-04-22T23:35:10.349 INFO:tasks.workunit.client.1.smithi157.stdout:4/982: mkdir d3/d11/d30/dfd/d148 0 2024-04-22T23:35:10.350 INFO:tasks.workunit.client.1.smithi157.stdout:1/920: dread d4/d17/d1d/db2/fe1 [0,4194304] 0 2024-04-22T23:35:10.350 INFO:tasks.workunit.client.1.smithi157.stdout:1/921: read - d4/fd1 zero size 2024-04-22T23:35:10.353 INFO:tasks.workunit.client.0.smithi107.stdout:3/661: symlink db/d21/d27/d15/d24/de1/le3 0 2024-04-22T23:35:10.353 INFO:tasks.workunit.client.0.smithi107.stdout:3/662: fdatasync db/d21/d27/d15/d29/d4f/db2/fd1 0 2024-04-22T23:35:10.353 INFO:tasks.workunit.client.0.smithi107.stdout:4/728: write d1/d21/d29/d2e/d88/d99/dd2/df0/ff5 [2182491,23901] 0 2024-04-22T23:35:10.353 INFO:tasks.workunit.client.0.smithi107.stdout:4/729: creat d1/d7/d63/d78/d80/fff x:0 0 0 2024-04-22T23:35:10.353 INFO:tasks.workunit.client.0.smithi107.stdout:4/730: chown d1/d21/d29/d2e/d7e/l83 0 1 2024-04-22T23:35:10.355 INFO:tasks.workunit.client.0.smithi107.stdout:8/696: mkdir d0/d7/df/d33/d9b/df3 0 2024-04-22T23:35:10.356 INFO:tasks.workunit.client.1.smithi157.stdout:2/959: sync 2024-04-22T23:35:10.356 INFO:tasks.workunit.client.1.smithi157.stdout:9/820: sync 2024-04-22T23:35:10.356 INFO:tasks.workunit.client.1.smithi157.stdout:8/649: sync 2024-04-22T23:35:10.356 INFO:tasks.workunit.client.1.smithi157.stdout:8/650: fdatasync d2/d34/fd4 0 2024-04-22T23:35:10.356 INFO:tasks.workunit.client.1.smithi157.stdout:9/821: chown d4/d15/d23/dad/ffb 64943593 1 2024-04-22T23:35:10.358 INFO:tasks.workunit.client.1.smithi157.stdout:1/922: write d4/d17/d1d/d33/d41/d54/f5b [1213450,128374] 0 2024-04-22T23:35:10.358 INFO:tasks.workunit.client.1.smithi157.stdout:1/923: read d4/d17/d34/d3b/f66 [190210,74913] 0 2024-04-22T23:35:10.359 INFO:tasks.workunit.client.0.smithi107.stdout:0/682: write d1/d84/fc2 [124488,10491] 0 2024-04-22T23:35:10.359 INFO:tasks.workunit.client.0.smithi107.stdout:0/683: chown d1/d2a/c6f 1948735 1 2024-04-22T23:35:10.359 INFO:tasks.workunit.client.0.smithi107.stdout:0/684: write d1/d7b/fbf [799096,44238] 0 2024-04-22T23:35:10.359 INFO:tasks.workunit.client.0.smithi107.stdout:3/663: creat db/d21/d27/d36/fe4 x:0 0 0 2024-04-22T23:35:10.362 INFO:tasks.workunit.client.0.smithi107.stdout:4/731: mknod d1/d7/d5b/dbf/da8/c100 0 2024-04-22T23:35:10.362 INFO:tasks.workunit.client.0.smithi107.stdout:4/732: chown d1/d21/d52/f9f 338 1 2024-04-22T23:35:10.363 INFO:tasks.workunit.client.1.smithi157.stdout:2/960: creat d7/d8/d8e/d94/f136 x:0 0 0 2024-04-22T23:35:10.364 INFO:tasks.workunit.client.0.smithi107.stdout:2/731: write d1/d2b/d88/fd4 [1205275,9027] 0 2024-04-22T23:35:10.364 INFO:tasks.workunit.client.0.smithi107.stdout:2/732: chown d1/d2b/d59/c7f 11969 1 2024-04-22T23:35:10.365 INFO:tasks.workunit.client.0.smithi107.stdout:1/640: link d7/d19/d20/d39/d53/d43/f7a d7/d19/d54/fdb 0 2024-04-22T23:35:10.365 INFO:tasks.workunit.client.0.smithi107.stdout:8/697: mkdir d0/d7/d20/da2/de9/df4 0 2024-04-22T23:35:10.366 INFO:tasks.workunit.client.0.smithi107.stdout:8/698: write d0/d85/f50 [412950,99082] 0 2024-04-22T23:35:10.366 INFO:tasks.workunit.client.0.smithi107.stdout:8/699: truncate d0/d7/df/d33/d37/fcc 257287 0 2024-04-22T23:35:10.366 INFO:tasks.workunit.client.0.smithi107.stdout:8/700: chown d0/d7/df/d3b/d7b/fcb 87160 1 2024-04-22T23:35:10.366 INFO:tasks.workunit.client.1.smithi157.stdout:3/899: rename d0/d3 to d0/d83/dbc/d12e 0 2024-04-22T23:35:10.366 INFO:tasks.workunit.client.1.smithi157.stdout:1/924: rename d4 to d4/d5f/d7e/db8/d141 22 2024-04-22T23:35:10.367 INFO:tasks.workunit.client.0.smithi107.stdout:9/693: rmdir d5/d7/d17/d1b/d1f/d36/d96/dce 0 2024-04-22T23:35:10.367 INFO:tasks.workunit.client.0.smithi107.stdout:9/694: chown d5/d29/f8c 1000 0 2024-04-22T23:35:10.368 INFO:tasks.workunit.client.1.smithi157.stdout:2/961: symlink d7/d8/d30/d23/d27/d57/dcf/l137 0 2024-04-22T23:35:10.370 INFO:tasks.workunit.client.1.smithi157.stdout:1/925: link d4/d17/d34/d3b/d52/c130 d4/d17/c142 0 2024-04-22T23:35:10.371 INFO:tasks.workunit.client.1.smithi157.stdout:1/926: creat d4/d17/db9/f143 x:0 0 0 2024-04-22T23:35:10.372 INFO:tasks.workunit.client.0.smithi107.stdout:0/685: unlink d1/l4c 0 2024-04-22T23:35:10.372 INFO:tasks.workunit.client.0.smithi107.stdout:0/686: readlink d1/d27/d20/d2c/d2e/l53 0 2024-04-22T23:35:10.372 INFO:tasks.workunit.client.0.smithi107.stdout:0/687: chown d1/d27/d8a/d8b/fa4 249777 1 2024-04-22T23:35:10.372 INFO:tasks.workunit.client.0.smithi107.stdout:3/664: creat db/fe5 x:0 0 0 2024-04-22T23:35:10.373 INFO:tasks.workunit.client.1.smithi157.stdout:3/900: creat d0/d83/dbc/d12e/d34/d47/f12f x:0 0 0 2024-04-22T23:35:10.374 INFO:tasks.workunit.client.1.smithi157.stdout:2/962: truncate d7/d1f/d26/d4a/f60 7490223 0 2024-04-22T23:35:10.374 INFO:tasks.workunit.client.1.smithi157.stdout:2/963: stat d7/d1f/d26/cfc 0 2024-04-22T23:35:10.375 INFO:tasks.workunit.client.0.smithi107.stdout:8/701: mknod d0/d7/d20/de4/cf5 0 2024-04-22T23:35:10.377 INFO:tasks.workunit.client.1.smithi157.stdout:1/927: mkdir d4/db/d144 0 2024-04-22T23:35:10.377 INFO:tasks.workunit.client.1.smithi157.stdout:1/928: truncate d4/d17/fc4 578890 0 2024-04-22T23:35:10.380 INFO:tasks.workunit.client.0.smithi107.stdout:8/702: symlink d0/d85/lf6 0 2024-04-22T23:35:10.381 INFO:tasks.workunit.client.0.smithi107.stdout:8/703: write d0/f83 [1478821,70009] 0 2024-04-22T23:35:10.381 INFO:tasks.workunit.client.0.smithi107.stdout:0/688: getdents d1/d27/d20/d40/dbd 0 2024-04-22T23:35:10.383 INFO:tasks.workunit.client.0.smithi107.stdout:8/704: symlink d0/d7/d8/d41/d53/lf7 0 2024-04-22T23:35:10.384 INFO:tasks.workunit.client.1.smithi157.stdout:4/983: dwrite d3/d11/d1d/f49 [0,4194304] 0 2024-04-22T23:35:10.384 INFO:tasks.workunit.client.1.smithi157.stdout:4/984: readlink d3/d11/d30/l115 0 2024-04-22T23:35:10.385 INFO:tasks.workunit.client.0.smithi107.stdout:0/689: creat d1/d84/d99/fe6 x:0 0 0 2024-04-22T23:35:10.385 INFO:tasks.workunit.client.0.smithi107.stdout:0/690: fsync d1/d84/d99/fa1 0 2024-04-22T23:35:10.388 INFO:tasks.workunit.client.0.smithi107.stdout:3/665: write db/d21/d27/d15/d24/d52/d60/f6b [6124062,54259] 0 2024-04-22T23:35:10.388 INFO:tasks.workunit.client.0.smithi107.stdout:0/691: unlink d1/d9/ld1 0 2024-04-22T23:35:10.388 INFO:tasks.workunit.client.0.smithi107.stdout:0/692: creat d1/d84/d99/fe7 x:0 0 0 2024-04-22T23:35:10.388 INFO:tasks.workunit.client.0.smithi107.stdout:0/693: truncate d1/d27/d20/d2c/d2e/fb3 644139 0 2024-04-22T23:35:10.390 INFO:tasks.workunit.client.1.smithi157.stdout:4/985: rmdir d3/d11/d1d/daf/d142 0 2024-04-22T23:35:10.391 INFO:tasks.workunit.client.1.smithi157.stdout:4/986: creat d3/d96/d4c/f149 x:0 0 0 2024-04-22T23:35:10.391 INFO:tasks.workunit.client.1.smithi157.stdout:2/964: write d7/d8/dd/f21 [4000945,124913] 0 2024-04-22T23:35:10.391 INFO:tasks.workunit.client.1.smithi157.stdout:2/965: fsync d7/d1f/d26/d4a/f11b 0 2024-04-22T23:35:10.393 INFO:tasks.workunit.client.1.smithi157.stdout:3/901: read d0/d83/dbc/d12e/d34/d3d/d57/d60/d9c/ffa [1034834,105899] 0 2024-04-22T23:35:10.393 INFO:tasks.workunit.client.1.smithi157.stdout:3/902: fdatasync d0/d83/dbc/d12e/d34/d3d/df6/fda 0 2024-04-22T23:35:10.393 INFO:tasks.workunit.client.0.smithi107.stdout:0/694: unlink d1/d84/d99/fa1 0 2024-04-22T23:35:10.393 INFO:tasks.workunit.client.0.smithi107.stdout:3/666: symlink db/d21/d27/d36/le6 0 2024-04-22T23:35:10.394 INFO:tasks.workunit.client.1.smithi157.stdout:3/903: mkdir d0/d7/d130 0 2024-04-22T23:35:10.396 INFO:tasks.workunit.client.1.smithi157.stdout:3/904: mknod d0/d83/dbc/d12e/d34/d3d/c131 0 2024-04-22T23:35:10.396 INFO:tasks.workunit.client.1.smithi157.stdout:3/905: fdatasync d0/d83/dbc/d12e/d34/d47/f12f 0 2024-04-22T23:35:10.398 INFO:tasks.workunit.client.1.smithi157.stdout:3/906: symlink d0/d83/dbc/d12e/d34/d3d/d57/d60/d9c/l132 0 2024-04-22T23:35:10.403 INFO:tasks.workunit.client.1.smithi157.stdout:8/651: dwrite d2/f32 [0,4194304] 0 2024-04-22T23:35:10.403 INFO:tasks.workunit.client.1.smithi157.stdout:8/652: chown d2/f5d 63 1 2024-04-22T23:35:10.403 INFO:tasks.workunit.client.1.smithi157.stdout:8/653: chown d2/lb0 25006 1 2024-04-22T23:35:10.404 INFO:tasks.workunit.client.1.smithi157.stdout:1/929: dwrite d4/d17/d34/d3b/d50/fb6 [0,4194304] 0 2024-04-22T23:35:10.404 INFO:tasks.workunit.client.0.smithi107.stdout:0/695: creat d1/d9/d14/d4f/fe8 x:0 0 0 2024-04-22T23:35:10.404 INFO:tasks.workunit.client.0.smithi107.stdout:0/696: readlink d1/d27/d20/l24 0 2024-04-22T23:35:10.405 INFO:tasks.workunit.client.0.smithi107.stdout:4/733: dwrite d1/d7/d53/d66/d6b/fdf [0,4194304] 0 2024-04-22T23:35:10.405 INFO:tasks.workunit.client.0.smithi107.stdout:0/697: write d1/d2a/dad/fa3 [896991,29334] 0 2024-04-22T23:35:10.405 INFO:tasks.workunit.client.0.smithi107.stdout:3/667: symlink db/d21/d27/d15/d29/d2a/d45/ddd/da7/le7 0 2024-04-22T23:35:10.405 INFO:tasks.workunit.client.0.smithi107.stdout:3/668: read f0 [2021306,91492] 0 2024-04-22T23:35:10.406 INFO:tasks.workunit.client.1.smithi157.stdout:8/654: symlink d2/db1/le1 0 2024-04-22T23:35:10.406 INFO:tasks.workunit.client.1.smithi157.stdout:8/655: readlink d2/d6/laf 0 2024-04-22T23:35:10.406 INFO:tasks.workunit.client.1.smithi157.stdout:8/656: dread - d2/d4/dbd/d67/fa8 zero size 2024-04-22T23:35:10.406 INFO:tasks.workunit.client.1.smithi157.stdout:8/657: truncate d2/db1/d58/fd6 386268 0 2024-04-22T23:35:10.406 INFO:tasks.workunit.client.0.smithi107.stdout:8/705: dread d0/d7/df/d3b/f2b [0,4194304] 0 2024-04-22T23:35:10.406 INFO:tasks.workunit.client.0.smithi107.stdout:1/641: dwrite d7/d19/d54/f66 [0,4194304] 0 2024-04-22T23:35:10.407 INFO:tasks.workunit.client.0.smithi107.stdout:1/642: creat d7/d19/d20/fdc x:0 0 0 2024-04-22T23:35:10.409 INFO:tasks.workunit.client.1.smithi157.stdout:3/907: write d0/d83/dbc/d12e/d34/d3d/df6/dee/d9e/f100 [3565746,95446] 0 2024-04-22T23:35:10.409 INFO:tasks.workunit.client.1.smithi157.stdout:1/930: mkdir d4/d17/db9/d133/d145 0 2024-04-22T23:35:10.409 INFO:tasks.workunit.client.1.smithi157.stdout:1/931: fdatasync d4/d17/d1d/d33/d41/f69 0 2024-04-22T23:35:10.410 INFO:tasks.workunit.client.1.smithi157.stdout:2/966: dwrite d7/d8/dd/d4d/f99 [0,4194304] 0 2024-04-22T23:35:10.410 INFO:tasks.workunit.client.1.smithi157.stdout:2/967: chown d7/l1e 559380 1 2024-04-22T23:35:10.411 INFO:tasks.workunit.client.0.smithi107.stdout:4/734: rename d1/d7/d63/d78/d96/da0/fd3 to d1/d21/d29/d2e/d3b/f101 0 2024-04-22T23:35:10.412 INFO:tasks.workunit.client.0.smithi107.stdout:0/698: rmdir d1/d27/d20/d40/d75 39 2024-04-22T23:35:10.415 INFO:tasks.workunit.client.0.smithi107.stdout:3/669: creat db/d21/db1/dbb/fe8 x:0 0 0 2024-04-22T23:35:10.415 INFO:tasks.workunit.client.0.smithi107.stdout:3/670: chown db 836005484 1 2024-04-22T23:35:10.417 INFO:tasks.workunit.client.1.smithi157.stdout:8/658: unlink d2/d6/d21/d5f/d17/lc4 0 2024-04-22T23:35:10.420 INFO:tasks.workunit.client.1.smithi157.stdout:1/932: mknod d4/d17/db9/d133/c146 0 2024-04-22T23:35:10.420 INFO:tasks.workunit.client.1.smithi157.stdout:8/659: write d2/d6/d21/d5f/f56 [48367,68527] 0 2024-04-22T23:35:10.420 INFO:tasks.workunit.client.1.smithi157.stdout:8/660: chown d2/d6/d21/d5f/d11/d13 452 1 2024-04-22T23:35:10.420 INFO:tasks.workunit.client.1.smithi157.stdout:2/968: chown d7/d8/d95/ff9 12 1 2024-04-22T23:35:10.421 INFO:tasks.workunit.client.1.smithi157.stdout:1/933: mknod d4/d17/d34/d75/d8a/c147 0 2024-04-22T23:35:10.421 INFO:tasks.workunit.client.1.smithi157.stdout:1/934: creat d4/d17/db9/f148 x:0 0 0 2024-04-22T23:35:10.421 INFO:tasks.workunit.client.1.smithi157.stdout:1/935: chown d4/db/d71/d127 105038 1 2024-04-22T23:35:10.422 INFO:tasks.workunit.client.1.smithi157.stdout:8/661: mknod d2/d6/d6b/ce2 0 2024-04-22T23:35:10.422 INFO:tasks.workunit.client.0.smithi107.stdout:2/733: dwrite d1/d2b/d59/fd2 [0,4194304] 0 2024-04-22T23:35:10.422 INFO:tasks.workunit.client.0.smithi107.stdout:2/734: dread - d1/d2b/d49/d7e/dc6/fed zero size 2024-04-22T23:35:10.422 INFO:tasks.workunit.client.0.smithi107.stdout:2/735: chown d1/d12/d35 29383 1 2024-04-22T23:35:10.423 INFO:tasks.workunit.client.1.smithi157.stdout:2/969: creat d7/d8/d25/d102/ddb/f138 x:0 0 0 2024-04-22T23:35:10.423 INFO:tasks.workunit.client.0.smithi107.stdout:8/706: unlink d0/d7/df/d33/d37/l66 0 2024-04-22T23:35:10.424 INFO:tasks.workunit.client.1.smithi157.stdout:2/970: chown d7/d8/d95/ffb 4123543 1 2024-04-22T23:35:10.425 INFO:tasks.workunit.client.1.smithi157.stdout:2/971: creat d7/d8/d30/d23/d27/d57/dcf/dfa/f139 x:0 0 0 2024-04-22T23:35:10.427 INFO:tasks.workunit.client.0.smithi107.stdout:0/699: creat d1/d27/d20/d4a/fe9 x:0 0 0 2024-04-22T23:35:10.427 INFO:tasks.workunit.client.1.smithi157.stdout:3/908: dwrite d0/d83/dbc/d12e/d34/d3d/d57/d60/d9c/ffa [4194304,4194304] 0 2024-04-22T23:35:10.428 INFO:tasks.workunit.client.0.smithi107.stdout:1/643: dwrite d7/d19/d2d/f84 [0,4194304] 0 2024-04-22T23:35:10.428 INFO:tasks.workunit.client.0.smithi107.stdout:1/644: dread - d7/d19/d2d/da4/fb0 zero size 2024-04-22T23:35:10.428 INFO:tasks.workunit.client.0.smithi107.stdout:3/671: stat db/d21/d27/d36/c71 0 2024-04-22T23:35:10.429 INFO:tasks.workunit.client.1.smithi157.stdout:2/972: rename d7/d9f/dc7/f109 to d7/d9f/f13a 0 2024-04-22T23:35:10.429 INFO:tasks.workunit.client.1.smithi157.stdout:1/936: dread d4/db/d3d/f82 [0,4194304] 0 2024-04-22T23:35:10.431 INFO:tasks.workunit.client.0.smithi107.stdout:8/707: dread d0/d7/d8/d41/d53/f57 [0,4194304] 0 2024-04-22T23:35:10.434 INFO:tasks.workunit.client.0.smithi107.stdout:8/708: dread d0/d7/d8/d41/fad [0,4194304] 0 2024-04-22T23:35:10.434 INFO:tasks.workunit.client.1.smithi157.stdout:2/973: mknod d7/d8/d30/d23/d27/d57/d108/d11f/c13b 0 2024-04-22T23:35:10.434 INFO:tasks.workunit.client.1.smithi157.stdout:2/974: chown d7/d8/d30/d23/d27/d57/dcf 3128 1 2024-04-22T23:35:10.437 INFO:tasks.workunit.client.1.smithi157.stdout:1/937: rename d4/d17/db9/d133/d145 to d4/d17/d34/d3b/d52/d65/dbc/d149 0 2024-04-22T23:35:10.438 INFO:tasks.workunit.client.1.smithi157.stdout:2/975: symlink d7/d8/d25/d102/l13c 0 2024-04-22T23:35:10.439 INFO:tasks.workunit.client.0.smithi107.stdout:2/736: mkdir d1/d2b/d49/dba/df1 0 2024-04-22T23:35:10.439 INFO:tasks.workunit.client.0.smithi107.stdout:4/735: getdents d1/d7/d63 0 2024-04-22T23:35:10.439 INFO:tasks.workunit.client.0.smithi107.stdout:2/737: read d1/d2b/d88/fd4 [1207980,11070] 0 2024-04-22T23:35:10.439 INFO:tasks.workunit.client.0.smithi107.stdout:2/738: truncate d1/d12/d35/f7a 430314 0 2024-04-22T23:35:10.439 INFO:tasks.workunit.client.0.smithi107.stdout:2/739: chown d1/d12/d35/d48/d13/c55 525175885 1 2024-04-22T23:35:10.439 INFO:tasks.workunit.client.0.smithi107.stdout:2/740: creat d1/d2b/d59/ff2 x:0 0 0 2024-04-22T23:35:10.441 INFO:tasks.workunit.client.1.smithi157.stdout:3/909: dread d0/f41 [0,4194304] 0 2024-04-22T23:35:10.441 INFO:tasks.workunit.client.1.smithi157.stdout:8/662: dwrite d2/d6/d21/d5f/d11/f27 [4194304,4194304] 0 2024-04-22T23:35:10.441 INFO:tasks.workunit.client.1.smithi157.stdout:3/910: stat d0/d7/d6c/ca0 0 2024-04-22T23:35:10.441 INFO:tasks.workunit.client.1.smithi157.stdout:8/663: creat d2/d6/d6b/fe3 x:0 0 0 2024-04-22T23:35:10.441 INFO:tasks.workunit.client.0.smithi107.stdout:3/672: write db/d21/f33 [1831335,79885] 0 2024-04-22T23:35:10.442 INFO:tasks.workunit.client.0.smithi107.stdout:3/673: fsync db/d21/d27/f20 0 2024-04-22T23:35:10.443 INFO:tasks.workunit.client.1.smithi157.stdout:2/976: mkdir d7/d8/d25/d102/d13d 0 2024-04-22T23:35:10.449 INFO:tasks.workunit.client.1.smithi157.stdout:3/911: rename d0/d83/dbc/d12e/d34/d47/cc1 to d0/d83/dbc/d12e/d34/d3d/d57/d5c/c133 0 2024-04-22T23:35:10.449 INFO:tasks.workunit.client.0.smithi107.stdout:2/741: dread d1/d2b/d88/fd4 [0,4194304] 0 2024-04-22T23:35:10.451 INFO:tasks.workunit.client.1.smithi157.stdout:3/912: symlink d0/d83/dbc/d12e/d34/d3d/d57/d5c/l134 0 2024-04-22T23:35:10.451 INFO:tasks.workunit.client.1.smithi157.stdout:3/913: chown d0/d83/df9/d119/l123 31 1 2024-04-22T23:35:10.453 INFO:tasks.workunit.client.1.smithi157.stdout:3/914: link d0/d7/f4f d0/d83/dbc/f135 0 2024-04-22T23:35:10.456 INFO:tasks.workunit.client.0.smithi107.stdout:4/736: write d1/d1b/fca [2746386,53182] 0 2024-04-22T23:35:10.456 INFO:tasks.workunit.client.1.smithi157.stdout:3/915: mkdir d0/d83/dbc/d12e/d34/d3d/d57/d114/d136 0 2024-04-22T23:35:10.457 INFO:tasks.workunit.client.0.smithi107.stdout:8/709: dwrite d0/d7/d8/f1e [0,4194304] 0 2024-04-22T23:35:10.458 INFO:tasks.workunit.client.1.smithi157.stdout:3/916: symlink d0/d83/dbc/d12e/d34/d47/dcd/l137 0 2024-04-22T23:35:10.458 INFO:tasks.workunit.client.0.smithi107.stdout:0/700: rmdir d1/d84/d99 39 2024-04-22T23:35:10.462 INFO:tasks.workunit.client.1.smithi157.stdout:1/938: dread d4/db/f47 [0,4194304] 0 2024-04-22T23:35:10.462 INFO:tasks.workunit.client.1.smithi157.stdout:1/939: stat d4/d17/cda 0 2024-04-22T23:35:10.462 INFO:tasks.workunit.client.1.smithi157.stdout:2/977: dread d7/d9f/f12a [0,4194304] 0 2024-04-22T23:35:10.462 INFO:tasks.workunit.client.1.smithi157.stdout:3/917: mknod d0/d83/dbc/d12e/d34/d47/ded/c138 0 2024-04-22T23:35:10.463 INFO:tasks.workunit.client.0.smithi107.stdout:2/742: rename d1/d12/d35/d48/d13/d4f/da5/ddd to d1/d12/d35/d48/df3 0 2024-04-22T23:35:10.463 INFO:tasks.workunit.client.0.smithi107.stdout:2/743: readlink d1/d12/l4a 0 2024-04-22T23:35:10.463 INFO:tasks.workunit.client.1.smithi157.stdout:2/978: dread d7/d8/d30/fc1 [0,4194304] 0 2024-04-22T23:35:10.463 INFO:tasks.workunit.client.1.smithi157.stdout:3/918: unlink d0/d83/dbc/d12e/d34/d62/cce 0 2024-04-22T23:35:10.465 INFO:tasks.workunit.client.1.smithi157.stdout:1/940: dread d4/d17/d34/d75/d8a/fd3 [0,4194304] 0 2024-04-22T23:35:10.465 INFO:tasks.workunit.client.1.smithi157.stdout:1/941: fdatasync d4/d77/f93 0 2024-04-22T23:35:10.467 INFO:tasks.workunit.client.0.smithi107.stdout:4/737: rename d1/d7/d5b/dbf/c9a to d1/d21/d29/d2e/d7e/c102 0 2024-04-22T23:35:10.468 INFO:tasks.workunit.client.1.smithi157.stdout:1/942: creat d4/db/d144/f14a x:0 0 0 2024-04-22T23:35:10.470 INFO:tasks.workunit.client.0.smithi107.stdout:0/701: write d1/d2a/f77 [2357484,105378] 0 2024-04-22T23:35:10.471 INFO:tasks.workunit.client.0.smithi107.stdout:2/744: creat d1/d12/d35/d48/d13/ff4 x:0 0 0 2024-04-22T23:35:10.472 INFO:tasks.workunit.client.0.smithi107.stdout:8/710: mkdir d0/d7/df/d33/d9b/df3/df8 0 2024-04-22T23:35:10.474 INFO:tasks.workunit.client.1.smithi157.stdout:2/979: dread d7/d8/d95/fa8 [0,4194304] 0 2024-04-22T23:35:10.475 INFO:tasks.workunit.client.0.smithi107.stdout:4/738: creat d1/d7/d5b/de3/f103 x:0 0 0 2024-04-22T23:35:10.475 INFO:tasks.workunit.client.0.smithi107.stdout:4/739: chown d1/d7/d5b/dbf/d27 6 1 2024-04-22T23:35:10.475 INFO:tasks.workunit.client.0.smithi107.stdout:4/740: fdatasync d1/d7/f11 0 2024-04-22T23:35:10.477 INFO:tasks.workunit.client.0.smithi107.stdout:3/674: dwrite db/d21/d27/d15/d29/d2a/d45/ddd/d25/fe2 [0,4194304] 0 2024-04-22T23:35:10.477 INFO:tasks.workunit.client.1.smithi157.stdout:8/664: dwrite d2/d6/d21/d5f/d11/d13/f94 [0,4194304] 0 2024-04-22T23:35:10.478 INFO:tasks.workunit.client.0.smithi107.stdout:0/702: mknod d1/d84/d99/cea 0 2024-04-22T23:35:10.479 INFO:tasks.workunit.client.0.smithi107.stdout:2/745: unlink d1/d2b/d49/faa 0 2024-04-22T23:35:10.479 INFO:tasks.workunit.client.0.smithi107.stdout:2/746: dread - d1/fb2 zero size 2024-04-22T23:35:10.481 INFO:tasks.workunit.client.0.smithi107.stdout:4/741: creat d1/d1b/f104 x:0 0 0 2024-04-22T23:35:10.481 INFO:tasks.workunit.client.0.smithi107.stdout:4/742: chown d1/d21/d29/d2e/d3b/c40 29 1 2024-04-22T23:35:10.482 INFO:tasks.workunit.client.1.smithi157.stdout:8/665: rename d2/db1/d58/fa5 to d2/d6/d21/d5f/d17/fe4 0 2024-04-22T23:35:10.482 INFO:tasks.workunit.client.1.smithi157.stdout:3/919: dread d0/d83/dbc/d12e/d34/d3d/df6/d72/f7f [0,4194304] 0 2024-04-22T23:35:10.484 INFO:tasks.workunit.client.1.smithi157.stdout:8/666: rename d2/d6/d21/d5f/d17/d7f/lbe to d2/d4/dad/le5 0 2024-04-22T23:35:10.484 INFO:tasks.workunit.client.1.smithi157.stdout:1/943: dread d4/d17/d34/d3b/d50/f56 [4194304,4194304] 0 2024-04-22T23:35:10.484 INFO:tasks.workunit.client.1.smithi157.stdout:1/944: truncate d4/db/d3d/ff9 1122279 0 2024-04-22T23:35:10.485 INFO:tasks.workunit.client.1.smithi157.stdout:3/920: creat d0/d83/dbc/d12e/d34/d3d/d57/d60/f139 x:0 0 0 2024-04-22T23:35:10.485 INFO:tasks.workunit.client.1.smithi157.stdout:8/667: mknod d2/d6/d21/d5f/dcd/ce6 0 2024-04-22T23:35:10.486 INFO:tasks.workunit.client.1.smithi157.stdout:3/921: symlink d0/d83/dbc/d12e/d34/l13a 0 2024-04-22T23:35:10.488 INFO:tasks.workunit.client.0.smithi107.stdout:3/675: unlink db/d21/d27/d15/d29/d2a/d45/ddd/d25/la4 0 2024-04-22T23:35:10.489 INFO:tasks.workunit.client.1.smithi157.stdout:8/668: rename d2/d6/d21/d5f/l63 to d2/d6/d21/d5f/d11/le7 0 2024-04-22T23:35:10.489 INFO:tasks.workunit.client.1.smithi157.stdout:8/669: getdents d2/d4/dbd/dd8 0 2024-04-22T23:35:10.490 INFO:tasks.workunit.client.0.smithi107.stdout:0/703: creat d1/d84/d99/feb x:0 0 0 2024-04-22T23:35:10.491 INFO:tasks.workunit.client.0.smithi107.stdout:2/747: rmdir d1/d63 39 2024-04-22T23:35:10.491 INFO:tasks.workunit.client.0.smithi107.stdout:4/743: symlink d1/d21/d29/d2e/l105 0 2024-04-22T23:35:10.491 INFO:tasks.workunit.client.1.smithi157.stdout:8/670: creat d2/d6/d21/d5f/d11/d13/fe8 x:0 0 0 2024-04-22T23:35:10.492 INFO:tasks.workunit.client.0.smithi107.stdout:3/676: mknod db/d21/ce9 0 2024-04-22T23:35:10.492 INFO:tasks.workunit.client.0.smithi107.stdout:0/704: getdents d1/d7b/d85 0 2024-04-22T23:35:10.495 INFO:tasks.workunit.client.0.smithi107.stdout:2/748: unlink d1/d2b/d49/f5a 0 2024-04-22T23:35:10.497 INFO:tasks.workunit.client.0.smithi107.stdout:4/744: mknod d1/d21/d29/d2e/d88/c106 0 2024-04-22T23:35:10.499 INFO:tasks.workunit.client.0.smithi107.stdout:3/677: mkdir db/d21/d27/d15/d24/d52/d60/dca/dcc/dea 0 2024-04-22T23:35:10.499 INFO:tasks.workunit.client.0.smithi107.stdout:3/678: write db/d21/d27/d15/fb4 [248275,87231] 0 2024-04-22T23:35:10.500 INFO:tasks.workunit.client.1.smithi157.stdout:3/922: read d0/f1c [2613358,106200] 0 2024-04-22T23:35:10.500 INFO:tasks.workunit.client.0.smithi107.stdout:0/705: mkdir d1/d27/d20/d2c/dec 0 2024-04-22T23:35:10.502 INFO:tasks.workunit.client.0.smithi107.stdout:0/706: mkdir d1/d7b/d85/ded 0 2024-04-22T23:35:10.505 INFO:tasks.workunit.client.0.smithi107.stdout:8/711: dread d0/d7/d20/da2/fa9 [0,4194304] 0 2024-04-22T23:35:10.505 INFO:tasks.workunit.client.0.smithi107.stdout:1/645: dwrite d7/f15 [0,4194304] 0 2024-04-22T23:35:10.505 INFO:tasks.workunit.client.0.smithi107.stdout:4/745: dread d1/d21/d29/d2e/d3b/f8a [0,4194304] 0 2024-04-22T23:35:10.505 INFO:tasks.workunit.client.0.smithi107.stdout:1/646: creat d7/d19/d20/d39/d53/d43/d46/dba/dc4/fdd x:0 0 0 2024-04-22T23:35:10.505 INFO:tasks.workunit.client.0.smithi107.stdout:1/647: chown d7/d19/d20/d39/d53 103 1 2024-04-22T23:35:10.505 INFO:tasks.workunit.client.0.smithi107.stdout:0/707: write d1/d27/d20/f60 [1301181,125080] 0 2024-04-22T23:35:10.505 INFO:tasks.workunit.client.0.smithi107.stdout:0/708: truncate d1/d27/d20/d2c/d2e/f71 805503 0 2024-04-22T23:35:10.505 INFO:tasks.workunit.client.0.smithi107.stdout:3/679: rename db/d21/d27/d15/d29/d4f/db2/dc5 to db/d21/d27/d15/d29/d2a/d45/deb 0 2024-04-22T23:35:10.506 INFO:tasks.workunit.client.0.smithi107.stdout:3/680: readlink db/d21/d27/d15/d24/d52/l95 0 2024-04-22T23:35:10.506 INFO:tasks.workunit.client.1.smithi157.stdout:2/980: dwrite d7/d1f/ff4 [4194304,4194304] 0 2024-04-22T23:35:10.507 INFO:tasks.workunit.client.0.smithi107.stdout:8/712: mkdir d0/d7/d20/df9 0 2024-04-22T23:35:10.507 INFO:tasks.workunit.client.0.smithi107.stdout:2/749: dread d1/d2b/d49/d7e/fbe [0,4194304] 0 2024-04-22T23:35:10.509 INFO:tasks.workunit.client.0.smithi107.stdout:4/746: truncate d1/d21/d29/d2e/f4f 2344238 0 2024-04-22T23:35:10.509 INFO:tasks.workunit.client.1.smithi157.stdout:3/923: truncate d0/d83/dbc/d12e/d34/d3d/f63 304980 0 2024-04-22T23:35:10.510 INFO:tasks.workunit.client.0.smithi107.stdout:1/648: unlink d7/d19/d20/d39/f69 0 2024-04-22T23:35:10.512 INFO:tasks.workunit.client.0.smithi107.stdout:0/709: creat d1/d2a/fee x:0 0 0 2024-04-22T23:35:10.513 INFO:tasks.workunit.client.1.smithi157.stdout:8/671: rmdir d2 39 2024-04-22T23:35:10.514 INFO:tasks.workunit.client.0.smithi107.stdout:1/649: dread d7/f1b [0,4194304] 0 2024-04-22T23:35:10.514 INFO:tasks.workunit.client.0.smithi107.stdout:1/650: read d7/d19/d20/d39/d53/f5f [3225514,99817] 0 2024-04-22T23:35:10.516 INFO:tasks.workunit.client.0.smithi107.stdout:8/713: mkdir d0/d7/dfa 0 2024-04-22T23:35:10.517 INFO:tasks.workunit.client.1.smithi157.stdout:2/981: rename d7/d1f/d26/leb to d7/d9f/dc7/l13e 0 2024-04-22T23:35:10.517 INFO:tasks.workunit.client.1.smithi157.stdout:2/982: readlink d7/d8/d30/d23/d27/d57/dcf/lf1 0 2024-04-22T23:35:10.517 INFO:tasks.workunit.client.1.smithi157.stdout:8/672: rmdir d2/d6/d21 39 2024-04-22T23:35:10.520 INFO:tasks.workunit.client.0.smithi107.stdout:7/652: sync 2024-04-22T23:35:10.520 INFO:tasks.workunit.client.0.smithi107.stdout:5/752: sync 2024-04-22T23:35:10.520 INFO:tasks.workunit.client.0.smithi107.stdout:7/653: truncate d7/d21/d54/f83 879996 0 2024-04-22T23:35:10.520 INFO:tasks.workunit.client.0.smithi107.stdout:2/750: creat d1/d2b/d49/dba/ff5 x:0 0 0 2024-04-22T23:35:10.521 INFO:tasks.workunit.client.1.smithi157.stdout:3/924: rename d0/d7 to d0/d83/dbc/d12e/d13/d13b 0 2024-04-22T23:35:10.521 INFO:tasks.workunit.client.1.smithi157.stdout:3/925: write d0/d83/dbc/d12e/d34/d3d/d57/d60/d9c/fc8 [1140899,128271] 0 2024-04-22T23:35:10.521 INFO:tasks.workunit.client.1.smithi157.stdout:2/983: truncate d7/d1f/d26/fe1 24872 0 2024-04-22T23:35:10.521 INFO:tasks.workunit.client.1.smithi157.stdout:2/984: write d7/d8/d30/fc5 [362579,15550] 0 2024-04-22T23:35:10.521 INFO:tasks.workunit.client.1.smithi157.stdout:2/985: chown d7/d8/dca/fe0 55406 1 2024-04-22T23:35:10.522 INFO:tasks.workunit.client.1.smithi157.stdout:8/673: getdents d2/db1 0 2024-04-22T23:35:10.522 INFO:tasks.workunit.client.0.smithi107.stdout:3/681: getdents db/d21/d27/d15/d29/d4f 0 2024-04-22T23:35:10.522 INFO:tasks.workunit.client.0.smithi107.stdout:3/682: write db/f6f [585563,19320] 0 2024-04-22T23:35:10.523 INFO:tasks.workunit.client.1.smithi157.stdout:3/926: mknod d0/d83/df9/c13c 0 2024-04-22T23:35:10.524 INFO:tasks.workunit.client.0.smithi107.stdout:1/651: mkdir d7/d19/d20/d39/d53/d43/d46/d68/dde 0 2024-04-22T23:35:10.524 INFO:tasks.workunit.client.0.smithi107.stdout:1/652: chown d7/d19/d2d/f31 126495742 1 2024-04-22T23:35:10.524 INFO:tasks.workunit.client.1.smithi157.stdout:3/927: creat d0/d83/df9/f13d x:0 0 0 2024-04-22T23:35:10.525 INFO:tasks.workunit.client.0.smithi107.stdout:4/747: dwrite d1/d1b/f104 [0,4194304] 0 2024-04-22T23:35:10.525 INFO:tasks.workunit.client.1.smithi157.stdout:2/986: rmdir d7/d1f/d26/d4a 39 2024-04-22T23:35:10.526 INFO:tasks.workunit.client.0.smithi107.stdout:5/753: link f7 d9/f101 0 2024-04-22T23:35:10.528 INFO:tasks.workunit.client.1.smithi157.stdout:3/928: creat d0/d83/dbc/d12e/d34/d47/f13e x:0 0 0 2024-04-22T23:35:10.528 INFO:tasks.workunit.client.1.smithi157.stdout:3/929: read d0/d83/dbc/d12e/d34/d62/f98 [150337,64938] 0 2024-04-22T23:35:10.529 INFO:tasks.workunit.client.0.smithi107.stdout:2/751: unlink d1/f1a 0 2024-04-22T23:35:10.529 INFO:tasks.workunit.client.0.smithi107.stdout:2/752: fsync d1/d67/f7b 0 2024-04-22T23:35:10.530 INFO:tasks.workunit.client.0.smithi107.stdout:3/683: mknod db/d21/d27/d15/db8/cec 0 2024-04-22T23:35:10.531 INFO:tasks.workunit.client.1.smithi157.stdout:0/771: sync 2024-04-22T23:35:10.531 INFO:tasks.workunit.client.1.smithi157.stdout:0/772: write d17/d2b/fb4 [107315,23034] 0 2024-04-22T23:35:10.533 INFO:tasks.workunit.client.1.smithi157.stdout:2/987: creat d7/d8/d95/dd2/f13f x:0 0 0 2024-04-22T23:35:10.535 INFO:tasks.workunit.client.0.smithi107.stdout:4/748: truncate d1/d21/d29/d2e/d88/d99/dd2/df0/ff5 1934772 0 2024-04-22T23:35:10.538 INFO:tasks.workunit.client.0.smithi107.stdout:5/754: write d9/db/d42/d75/fb1 [835496,92225] 0 2024-04-22T23:35:10.538 INFO:tasks.workunit.client.0.smithi107.stdout:5/755: chown d9/db/d42/c69 38576624 1 2024-04-22T23:35:10.538 INFO:tasks.workunit.client.0.smithi107.stdout:8/714: dread d0/d7/df/d3b/fc7 [0,4194304] 0 2024-04-22T23:35:10.538 INFO:tasks.workunit.client.0.smithi107.stdout:8/715: readlink d0/d85/lf6 0 2024-04-22T23:35:10.538 INFO:tasks.workunit.client.0.smithi107.stdout:8/716: chown d0/d7/d8/d41/dec/f2a 4180111 1 2024-04-22T23:35:10.540 INFO:tasks.workunit.client.1.smithi157.stdout:2/988: creat d7/f140 x:0 0 0 2024-04-22T23:35:10.545 INFO:tasks.workunit.client.1.smithi157.stdout:0/773: rmdir d17 39 2024-04-22T23:35:10.545 INFO:tasks.workunit.client.1.smithi157.stdout:0/774: truncate d17/d23/d34/d67/d7f/f70 5176337 0 2024-04-22T23:35:10.546 INFO:tasks.workunit.client.0.smithi107.stdout:1/653: dwrite d7/d19/f80 [0,4194304] 0 2024-04-22T23:35:10.546 INFO:tasks.workunit.client.0.smithi107.stdout:1/654: creat d7/d19/d71/fdf x:0 0 0 2024-04-22T23:35:10.547 INFO:tasks.workunit.client.0.smithi107.stdout:8/717: write d0/f5 [5709174,35477] 0 2024-04-22T23:35:10.547 INFO:tasks.workunit.client.0.smithi107.stdout:8/718: fsync d0/d7/d8/d41/dec/f5b 0 2024-04-22T23:35:10.547 INFO:tasks.workunit.client.0.smithi107.stdout:8/719: fsync d0/d7/df/d3b/d7b/fcb 0 2024-04-22T23:35:10.548 INFO:tasks.workunit.client.1.smithi157.stdout:3/930: dwrite d0/d83/dbc/d12e/d13/d13b/f4f [0,4194304] 0 2024-04-22T23:35:10.548 INFO:tasks.workunit.client.1.smithi157.stdout:3/931: chown d0/d83/dbc/d12e/d34/d39/f76 16169 1 2024-04-22T23:35:10.548 INFO:tasks.workunit.client.1.smithi157.stdout:2/989: write d7/d8/d95/fcb [307822,43529] 0 2024-04-22T23:35:10.550 INFO:tasks.workunit.client.1.smithi157.stdout:0/775: rename d17/d2b/d71/db3 to d17/d23/d34/d67/d104 0 2024-04-22T23:35:10.551 INFO:tasks.workunit.client.1.smithi157.stdout:3/932: mkdir d0/d83/dbc/d12e/d34/d3d/d57/d114/d136/d13f 0 2024-04-22T23:35:10.552 INFO:tasks.workunit.client.1.smithi157.stdout:2/990: symlink d7/d8/d30/d12d/l141 0 2024-04-22T23:35:10.553 INFO:tasks.workunit.client.1.smithi157.stdout:0/776: creat d17/d2b/d101/f105 x:0 0 0 2024-04-22T23:35:10.553 INFO:tasks.workunit.client.1.smithi157.stdout:0/777: fsync d17/d2b/f2c 0 2024-04-22T23:35:10.557 INFO:tasks.workunit.client.1.smithi157.stdout:3/933: mknod d0/d83/c140 0 2024-04-22T23:35:10.557 INFO:tasks.workunit.client.1.smithi157.stdout:2/991: creat d7/d8/d95/f142 x:0 0 0 2024-04-22T23:35:10.557 INFO:tasks.workunit.client.1.smithi157.stdout:3/934: chown d0/d83/dbc/d12e/lf 2 1 2024-04-22T23:35:10.557 INFO:tasks.workunit.client.1.smithi157.stdout:2/992: write d7/d8/d95/dd2/dda/f121 [776004,56315] 0 2024-04-22T23:35:10.557 INFO:tasks.workunit.client.1.smithi157.stdout:0/778: truncate d17/f2d 890992 0 2024-04-22T23:35:10.559 INFO:tasks.workunit.client.1.smithi157.stdout:2/993: dread d7/d8/d30/fc1 [0,4194304] 0 2024-04-22T23:35:10.559 INFO:tasks.workunit.client.1.smithi157.stdout:2/994: chown d7/d8/dd/d4d/f99 180 1 2024-04-22T23:35:10.560 INFO:tasks.workunit.client.1.smithi157.stdout:2/995: dread - d7/d8/d8e/d94/f136 zero size 2024-04-22T23:35:10.560 INFO:tasks.workunit.client.1.smithi157.stdout:3/935: mkdir d0/d83/dbc/d12e/d34/d3d/df6/dee/d9e/d141 0 2024-04-22T23:35:10.560 INFO:tasks.workunit.client.1.smithi157.stdout:0/779: symlink d17/d2b/d71/da5/l106 0 2024-04-22T23:35:10.560 INFO:tasks.workunit.client.1.smithi157.stdout:0/780: dread - d17/d2b/d52/dd1/ff0 zero size 2024-04-22T23:35:10.561 INFO:tasks.workunit.client.1.smithi157.stdout:0/781: readlink d17/d23/d34/d67/lcb 0 2024-04-22T23:35:10.562 INFO:tasks.workunit.client.1.smithi157.stdout:2/996: mkdir d7/d8/dd/d4d/d143 0 2024-04-22T23:35:10.562 INFO:tasks.workunit.client.1.smithi157.stdout:0/782: creat d17/ddb/f107 x:0 0 0 2024-04-22T23:35:10.562 INFO:tasks.workunit.client.1.smithi157.stdout:2/997: truncate d7/d8/d25/d102/f103 1589434 0 2024-04-22T23:35:10.577 INFO:tasks.workunit.client.1.smithi157.stdout:2/998: dread d7/d8/d30/d23/f48 [0,4194304] 0 2024-04-22T23:35:10.577 INFO:tasks.workunit.client.1.smithi157.stdout:2/999: write d7/d8/d25/d55/f106 [917305,53846] 0 2024-04-22T23:35:10.578 INFO:tasks.workunit.client.1.smithi157.stdout:0/783: dread d17/d2b/f3d [0,4194304] 0 2024-04-22T23:35:10.579 INFO:tasks.workunit.client.1.smithi157.stdout:0/784: symlink d17/d2b/d4f/dc5/dcc/de9/df6/l108 0 2024-04-22T23:35:10.584 INFO:tasks.workunit.client.1.smithi157.stdout:3/936: dwrite d0/f41 [4194304,4194304] 0 2024-04-22T23:35:10.587 INFO:tasks.workunit.client.0.smithi107.stdout:7/654: mknod d7/d21/d48/cd6 0 2024-04-22T23:35:10.588 INFO:tasks.workunit.client.0.smithi107.stdout:2/753: rmdir d1/d2b/d49/d7e/dae/de8 39 2024-04-22T23:35:10.589 INFO:tasks.workunit.client.0.smithi107.stdout:2/754: dread - d1/d2b/d3d/fde zero size 2024-04-22T23:35:10.589 INFO:tasks.workunit.client.0.smithi107.stdout:2/755: fsync d1/d63/fa1 0 2024-04-22T23:35:10.589 INFO:tasks.workunit.client.0.smithi107.stdout:6/736: sync 2024-04-22T23:35:10.589 INFO:tasks.workunit.client.0.smithi107.stdout:6/737: write db/d20/d2a/d38/d59/f5a [2855694,34253] 0 2024-04-22T23:35:10.591 INFO:tasks.workunit.client.1.smithi157.stdout:0/785: read d17/d2b/d71/fc4 [1080165,84352] 0 2024-04-22T23:35:10.592 INFO:tasks.workunit.client.0.smithi107.stdout:4/749: symlink d1/d21/d85/l107 0 2024-04-22T23:35:10.595 INFO:tasks.workunit.client.0.smithi107.stdout:5/756: unlink d9/d26/lad 0 2024-04-22T23:35:10.601 INFO:tasks.workunit.client.0.smithi107.stdout:1/655: mkdir d7/d19/d20/d39/de0 0 2024-04-22T23:35:10.601 INFO:tasks.workunit.client.0.smithi107.stdout:1/656: truncate d7/d19/d54/fb2 328488 0 2024-04-22T23:35:10.602 INFO:tasks.workunit.client.1.smithi157.stdout:0/786: dread d17/d23/d34/d67/d7f/f5f [4194304,4194304] 0 2024-04-22T23:35:10.605 INFO:tasks.workunit.client.0.smithi107.stdout:4/750: read d1/d21/d29/d2e/d3b/f90 [1959922,114490] 0 2024-04-22T23:35:10.607 INFO:tasks.workunit.client.0.smithi107.stdout:7/655: unlink d7/d21/d48/lc3 0 2024-04-22T23:35:10.607 INFO:tasks.workunit.client.0.smithi107.stdout:7/656: chown d7/d1b/f38 1389413 1 2024-04-22T23:35:10.607 INFO:tasks.workunit.client.0.smithi107.stdout:7/657: dread - d7/d21/d54/f98 zero size 2024-04-22T23:35:10.608 INFO:tasks.workunit.client.0.smithi107.stdout:2/756: creat d1/d12/d35/ff6 x:0 0 0 2024-04-22T23:35:10.608 INFO:tasks.workunit.client.0.smithi107.stdout:2/757: dread - d1/d2b/d3d/d57/fec zero size 2024-04-22T23:35:10.609 INFO:tasks.workunit.client.0.smithi107.stdout:3/684: mkdir db/d21/d27/d15/ded 0 2024-04-22T23:35:10.610 INFO:tasks.workunit.client.0.smithi107.stdout:3/685: stat db/d21/d27/d15/d29/d2a/d45/ddd/d25/fb6 0 2024-04-22T23:35:10.610 INFO:tasks.workunit.client.0.smithi107.stdout:3/686: readlink db/d21/d27/d15/d24/l26 0 2024-04-22T23:35:10.611 INFO:tasks.workunit.client.0.smithi107.stdout:5/757: symlink d9/db/de/d5e/l102 0 2024-04-22T23:35:10.611 INFO:tasks.workunit.client.0.smithi107.stdout:1/657: mknod d7/d19/d20/d39/d53/d43/d63/ce1 0 2024-04-22T23:35:10.612 INFO:tasks.workunit.client.0.smithi107.stdout:8/720: getdents d0/d3a/d46/d59 0 2024-04-22T23:35:10.613 INFO:tasks.workunit.client.0.smithi107.stdout:8/721: chown d0/d7/df/d3b/dd9/cea 8682901 1 2024-04-22T23:35:10.613 INFO:tasks.workunit.client.0.smithi107.stdout:8/722: truncate d0/d7/d87/dc8/fee 55952 0 2024-04-22T23:35:10.613 INFO:tasks.workunit.client.1.smithi157.stdout:0/787: dread f3 [0,4194304] 0 2024-04-22T23:35:10.614 INFO:tasks.workunit.client.0.smithi107.stdout:4/751: symlink d1/d7/d63/d78/l108 0 2024-04-22T23:35:10.614 INFO:tasks.workunit.client.0.smithi107.stdout:4/752: chown d1/d21/d29/d2e/d7e/l83 12948843 1 2024-04-22T23:35:10.614 INFO:tasks.workunit.client.0.smithi107.stdout:7/658: creat d7/d4c/da1/fd7 x:0 0 0 2024-04-22T23:35:10.615 INFO:tasks.workunit.client.0.smithi107.stdout:9/695: sync 2024-04-22T23:35:10.615 INFO:tasks.workunit.client.1.smithi157.stdout:3/937: dwrite d0/d83/dbc/d12e/d34/d3d/d57/d60/d6b/d97/fb5 [0,4194304] 0 2024-04-22T23:35:10.615 INFO:tasks.workunit.client.0.smithi107.stdout:5/758: symlink d9/d76/l103 0 2024-04-22T23:35:10.616 INFO:tasks.workunit.client.1.smithi157.stdout:0/788: creat d17/d23/d34/d67/d7f/d69/dce/de3/f109 x:0 0 0 2024-04-22T23:35:10.618 INFO:tasks.workunit.client.0.smithi107.stdout:8/723: mknod d0/d7/d20/da2/cfb 0 2024-04-22T23:35:10.618 INFO:tasks.workunit.client.0.smithi107.stdout:8/724: stat d0/d73/ld3 0 2024-04-22T23:35:10.618 INFO:tasks.workunit.client.0.smithi107.stdout:8/725: creat d0/d7/df/d3b/dd9/dda/ffc x:0 0 0 2024-04-22T23:35:10.619 INFO:tasks.workunit.client.0.smithi107.stdout:4/753: symlink d1/d7/d5b/l109 0 2024-04-22T23:35:10.619 INFO:tasks.workunit.client.0.smithi107.stdout:4/754: read - d1/d21/d29/d2e/d7e/ff8 zero size 2024-04-22T23:35:10.620 INFO:tasks.workunit.client.1.smithi157.stdout:0/789: mkdir d17/d2b/d4f/d5a/dd2/d10a 0 2024-04-22T23:35:10.620 INFO:tasks.workunit.client.1.smithi157.stdout:0/790: chown d17/f20 481351 1 2024-04-22T23:35:10.622 INFO:tasks.workunit.client.0.smithi107.stdout:3/687: write db/d21/d27/d15/d29/d2a/f5b [5025079,111513] 0 2024-04-22T23:35:10.623 INFO:tasks.workunit.client.0.smithi107.stdout:5/759: mknod d9/db/d42/d75/db6/c104 0 2024-04-22T23:35:10.625 INFO:tasks.workunit.client.0.smithi107.stdout:1/658: getdents d7 0 2024-04-22T23:35:10.625 INFO:tasks.workunit.client.0.smithi107.stdout:1/659: readlink d7/d19/d20/d39/d53/d43/d46/d68/l8d 0 2024-04-22T23:35:10.627 INFO:tasks.workunit.client.0.smithi107.stdout:8/726: mknod d0/d7/d20/cfd 0 2024-04-22T23:35:10.628 INFO:tasks.workunit.client.0.smithi107.stdout:8/727: truncate d0/f96 1275291 0 2024-04-22T23:35:10.628 INFO:tasks.workunit.client.0.smithi107.stdout:4/755: mknod d1/d7/d63/d78/d96/da0/de9/c10a 0 2024-04-22T23:35:10.629 INFO:tasks.workunit.client.0.smithi107.stdout:6/738: dwrite db/d20/d2a/d38/d4b/f54 [0,4194304] 0 2024-04-22T23:35:10.629 INFO:tasks.workunit.client.0.smithi107.stdout:6/739: read - db/d20/d2a/d38/fc9 zero size 2024-04-22T23:35:10.629 INFO:tasks.workunit.client.0.smithi107.stdout:6/740: truncate db/d20/db1/fbe 1650407 0 2024-04-22T23:35:10.633 INFO:tasks.workunit.client.1.smithi157.stdout:0/791: write d17/d23/d34/dee/da8/fea [1897235,120505] 0 2024-04-22T23:35:10.633 INFO:tasks.workunit.client.1.smithi157.stdout:0/792: write d17/d23/d34/d67/d78/fbb [955823,95577] 0 2024-04-22T23:35:10.633 INFO:tasks.workunit.client.1.smithi157.stdout:0/793: readlink d17/d2b/l4b 0 2024-04-22T23:35:10.634 INFO:tasks.workunit.client.0.smithi107.stdout:7/659: rmdir d7/d21 39 2024-04-22T23:35:10.635 INFO:tasks.workunit.client.0.smithi107.stdout:2/758: dwrite d1/d12/d35/f3f [4194304,4194304] 0 2024-04-22T23:35:10.636 INFO:tasks.workunit.client.1.smithi157.stdout:3/938: dwrite d0/d83/dbc/d12e/d34/d3d/f63 [0,4194304] 0 2024-04-22T23:35:10.636 INFO:tasks.workunit.client.1.smithi157.stdout:0/794: rename d17/d2b/d52/cba to d17/d2b/d4f/dc5/dcc/de9/df6/dfd/c10b 0 2024-04-22T23:35:10.637 INFO:tasks.workunit.client.0.smithi107.stdout:9/696: creat d5/d7/d17/d1b/d1f/d36/d53/ff1 x:0 0 0 2024-04-22T23:35:10.638 INFO:tasks.workunit.client.0.smithi107.stdout:7/660: dread d7/d21/d49/f6a [0,4194304] 0 2024-04-22T23:35:10.638 INFO:tasks.workunit.client.0.smithi107.stdout:1/660: chown d7/d19/d20/d39/d53/d43/f97 0 1 2024-04-22T23:35:10.639 INFO:tasks.workunit.client.0.smithi107.stdout:8/728: symlink d0/d7/d20/d89/lfe 0 2024-04-22T23:35:10.640 INFO:tasks.workunit.client.0.smithi107.stdout:4/756: creat d1/d21/d29/d2e/dfa/f10b x:0 0 0 2024-04-22T23:35:10.640 INFO:tasks.workunit.client.0.smithi107.stdout:4/757: fsync d1/d7/d5b/dbf/d27/fb3 0 2024-04-22T23:35:10.641 INFO:tasks.workunit.client.1.smithi157.stdout:3/939: symlink d0/d83/dbc/d12e/d13/d13b/d6c/l142 0 2024-04-22T23:35:10.641 INFO:tasks.workunit.client.1.smithi157.stdout:3/940: chown d0/d83/dbc/d12e/d34/d3d/d57/d60/d6b 24837305 1 2024-04-22T23:35:10.642 INFO:tasks.workunit.client.1.smithi157.stdout:0/795: symlink d17/d2b/d4f/d5a/dd2/de4/l10c 0 2024-04-22T23:35:10.642 INFO:tasks.workunit.client.0.smithi107.stdout:6/741: getdents db/d20/d2a/d35/d69/db3/dbc 0 2024-04-22T23:35:10.643 INFO:tasks.workunit.client.0.smithi107.stdout:6/742: chown db/d20/d2a/d38/c4c 8227 1 2024-04-22T23:35:10.645 INFO:tasks.workunit.client.0.smithi107.stdout:5/760: dwrite d9/db/f50 [0,4194304] 0 2024-04-22T23:35:10.648 INFO:tasks.workunit.client.1.smithi157.stdout:3/941: symlink d0/d83/dbc/d12e/d127/l143 0 2024-04-22T23:35:10.651 INFO:tasks.workunit.client.1.smithi157.stdout:0/796: rename d17/d23/d34/d67/d7f/f61 to d17/d23/d34/dee/da8/f10d 0 2024-04-22T23:35:10.651 INFO:tasks.workunit.client.1.smithi157.stdout:0/797: write d17/d23/d34/d67/d7f/d69/f89 [377474,8171] 0 2024-04-22T23:35:10.651 INFO:tasks.workunit.client.1.smithi157.stdout:0/798: truncate d17/d2b/d71/fc2 775596 0 2024-04-22T23:35:10.651 INFO:tasks.workunit.client.1.smithi157.stdout:3/942: mkdir d0/d83/dbc/d12e/d144 0 2024-04-22T23:35:10.652 INFO:tasks.workunit.client.1.smithi157.stdout:0/799: truncate d17/d2b/d4f/dc5/dcc/de9/df6/fa6 314207 0 2024-04-22T23:35:10.656 INFO:tasks.workunit.client.0.smithi107.stdout:3/688: rmdir db/d21/d27/d15/d24/d52 39 2024-04-22T23:35:10.656 INFO:tasks.workunit.client.0.smithi107.stdout:4/758: dread d1/d21/d85/f9c [0,4194304] 0 2024-04-22T23:35:10.656 INFO:tasks.workunit.client.0.smithi107.stdout:4/759: fsync d1/d7/d63/d78/d80/f82 0 2024-04-22T23:35:10.657 INFO:tasks.workunit.client.0.smithi107.stdout:4/760: dread - d1/d21/ffe zero size 2024-04-22T23:35:10.657 INFO:tasks.workunit.client.0.smithi107.stdout:4/761: chown d1/f87 72069 1 2024-04-22T23:35:10.657 INFO:tasks.workunit.client.0.smithi107.stdout:4/762: stat d1/d7/d5b/dbf/da8/db9/lf9 0 2024-04-22T23:35:10.657 INFO:tasks.workunit.client.0.smithi107.stdout:4/763: write d1/d7/d63/d78/d80/fe2 [32734,86648] 0 2024-04-22T23:35:10.658 INFO:tasks.workunit.client.1.smithi157.stdout:3/943: symlink d0/d83/dbc/d12e/d34/d3d/d57/d60/d6b/d11c/l145 0 2024-04-22T23:35:10.659 INFO:tasks.workunit.client.1.smithi157.stdout:0/800: mknod d17/d2b/d5b/c10e 0 2024-04-22T23:35:10.659 INFO:tasks.workunit.client.1.smithi157.stdout:0/801: write d17/f2d [789174,125814] 0 2024-04-22T23:35:10.659 INFO:tasks.workunit.client.0.smithi107.stdout:2/759: dwrite d1/d12/d35/d48/d13/f68 [0,4194304] 0 2024-04-22T23:35:10.659 INFO:tasks.workunit.client.0.smithi107.stdout:1/661: link d7/d19/d2d/f3d d7/d19/d20/d39/d53/fe2 0 2024-04-22T23:35:10.660 INFO:tasks.workunit.client.0.smithi107.stdout:8/729: rename d0/d7/d8/cb1 to d0/cff 0 2024-04-22T23:35:10.660 INFO:tasks.workunit.client.0.smithi107.stdout:8/730: truncate d0/d7/d8/d41/dec/f91 756538 0 2024-04-22T23:35:10.661 INFO:tasks.workunit.client.0.smithi107.stdout:7/661: dwrite d7/f76 [0,4194304] 0 2024-04-22T23:35:10.667 INFO:tasks.workunit.client.0.smithi107.stdout:5/761: symlink d9/d26/l105 0 2024-04-22T23:35:10.667 INFO:tasks.workunit.client.0.smithi107.stdout:9/697: rmdir d5/d14/d28 39 2024-04-22T23:35:10.667 INFO:tasks.workunit.client.0.smithi107.stdout:3/689: rmdir db/d21/d27/d15/ded 0 2024-04-22T23:35:10.667 INFO:tasks.workunit.client.0.smithi107.stdout:4/764: mknod d1/d21/d29/d2e/d88/d99/dd2/daf/dc1/c10c 0 2024-04-22T23:35:10.670 INFO:tasks.workunit.client.0.smithi107.stdout:2/760: symlink d1/d2b/d49/d7e/lf7 0 2024-04-22T23:35:10.674 INFO:tasks.workunit.client.0.smithi107.stdout:6/743: rename db/d20/d2a/c41 to db/d20/d2a/d35/ce1 0 2024-04-22T23:35:10.676 INFO:tasks.workunit.client.0.smithi107.stdout:3/690: write db/d21/d27/d15/d24/d52/d60/f92 [487103,94899] 0 2024-04-22T23:35:10.678 INFO:tasks.workunit.client.1.smithi157.stdout:7/818: sync 2024-04-22T23:35:10.678 INFO:tasks.workunit.client.1.smithi157.stdout:7/819: stat d0/d9/de/l54 0 2024-04-22T23:35:10.679 INFO:tasks.workunit.client.1.smithi157.stdout:7/820: mknod d0/d9/db/df/d68/cf7 0 2024-04-22T23:35:10.681 INFO:tasks.workunit.client.1.smithi157.stdout:7/821: mkdir d0/d9/d42/d55/d80/d95/df8 0 2024-04-22T23:35:10.682 INFO:tasks.workunit.client.1.smithi157.stdout:7/822: chown d0/d9/d42/d55/d80 15 1 2024-04-22T23:35:10.682 INFO:tasks.workunit.client.1.smithi157.stdout:7/823: dread - d0/d9/d29/d49/d4a/fe5 zero size 2024-04-22T23:35:10.683 INFO:tasks.workunit.client.0.smithi107.stdout:4/765: symlink d1/d7/d5b/de3/l10d 0 2024-04-22T23:35:10.683 INFO:tasks.workunit.client.0.smithi107.stdout:4/766: creat d1/d21/d29/d2e/d7e/f10e x:0 0 0 2024-04-22T23:35:10.687 INFO:tasks.workunit.client.1.smithi157.stdout:0/802: dwrite d17/d23/d34/d67/d104/fbd [0,4194304] 0 2024-04-22T23:35:10.687 INFO:tasks.workunit.client.1.smithi157.stdout:0/803: write d17/ddb/f107 [226174,118920] 0 2024-04-22T23:35:10.688 INFO:tasks.workunit.client.0.smithi107.stdout:8/731: rename d0/d7/df/d3b/f3d to d0/d7/d20/df9/f100 0 2024-04-22T23:35:10.688 INFO:tasks.workunit.client.0.smithi107.stdout:8/732: read - d0/d7/df/d3b/d7b/fcb zero size 2024-04-22T23:35:10.688 INFO:tasks.workunit.client.0.smithi107.stdout:7/662: dwrite d7/d21/da4/f43 [0,4194304] 0 2024-04-22T23:35:10.689 INFO:tasks.workunit.client.1.smithi157.stdout:0/804: mknod d17/d2b/d4f/dc5/dcc/de9/c10f 0 2024-04-22T23:35:10.689 INFO:tasks.workunit.client.1.smithi157.stdout:0/805: write d17/f2d [562479,124369] 0 2024-04-22T23:35:10.690 INFO:tasks.workunit.client.0.smithi107.stdout:2/761: dwrite d1/d12/fa4 [0,4194304] 0 2024-04-22T23:35:10.690 INFO:tasks.workunit.client.1.smithi157.stdout:0/806: mknod d17/d23/d34/d67/d7f/c110 0 2024-04-22T23:35:10.694 INFO:tasks.workunit.client.0.smithi107.stdout:5/762: rename d9/db/de/fd9 to d9/db/dc1/f106 0 2024-04-22T23:35:10.694 INFO:tasks.workunit.client.0.smithi107.stdout:5/763: dread - d9/db/de/d72/d7c/ffc zero size 2024-04-22T23:35:10.694 INFO:tasks.workunit.client.0.smithi107.stdout:5/764: chown d9/d26/d6a/f6f 26674 1 2024-04-22T23:35:10.697 INFO:tasks.workunit.client.0.smithi107.stdout:3/691: dread db/d21/d27/d36/fb0 [0,4194304] 0 2024-04-22T23:35:10.697 INFO:tasks.workunit.client.0.smithi107.stdout:3/692: chown db/d21/d27/d15/d29/d2a/f43 1832 1 2024-04-22T23:35:10.698 INFO:tasks.workunit.client.0.smithi107.stdout:7/663: symlink d7/d21/d48/d95/ld8 0 2024-04-22T23:35:10.701 INFO:tasks.workunit.client.0.smithi107.stdout:2/762: symlink d1/d63/lf8 0 2024-04-22T23:35:10.702 INFO:tasks.workunit.client.0.smithi107.stdout:9/698: rename d5/d7/d17/l8d to d5/d14/d28/de3/lf2 0 2024-04-22T23:35:10.703 INFO:tasks.workunit.client.0.smithi107.stdout:5/765: mknod d9/db/de/d5e/c107 0 2024-04-22T23:35:10.705 INFO:tasks.workunit.client.0.smithi107.stdout:3/693: symlink db/d21/d27/d15/d29/d2a/d45/deb/lee 0 2024-04-22T23:35:10.705 INFO:tasks.workunit.client.1.smithi157.stdout:0/807: dwrite d17/d23/d9c/fdd [0,4194304] 0 2024-04-22T23:35:10.706 INFO:tasks.workunit.client.0.smithi107.stdout:2/763: dread d1/d12/d35/d48/f2c [0,4194304] 0 2024-04-22T23:35:10.710 INFO:tasks.workunit.client.0.smithi107.stdout:7/664: link d7/d21/d7e/fb3 d7/d4c/da1/fd9 0 2024-04-22T23:35:10.713 INFO:tasks.workunit.client.0.smithi107.stdout:6/744: rename db/d20/d2a/d38/d4b/cbb to db/d20/d58/ce2 0 2024-04-22T23:35:10.719 INFO:tasks.workunit.client.0.smithi107.stdout:5/766: symlink d9/db/dc1/l108 0 2024-04-22T23:35:10.719 INFO:tasks.workunit.client.0.smithi107.stdout:5/767: read - d9/db/d42/d75/fde zero size 2024-04-22T23:35:10.720 INFO:tasks.workunit.client.0.smithi107.stdout:2/764: symlink d1/d2b/d59/lf9 0 2024-04-22T23:35:10.720 INFO:tasks.workunit.client.0.smithi107.stdout:2/765: write d1/d2b/d49/d7e/fe5 [802194,13860] 0 2024-04-22T23:35:10.721 INFO:tasks.workunit.client.0.smithi107.stdout:2/766: chown d1/d2b/d59/cb1 5518 1 2024-04-22T23:35:10.722 INFO:tasks.workunit.client.0.smithi107.stdout:7/665: link d7/d21/d7e/fb8 d7/d4c/da1/dce/fda 0 2024-04-22T23:35:10.722 INFO:tasks.workunit.client.1.smithi157.stdout:0/808: dwrite d17/d38/f6d [0,4194304] 0 2024-04-22T23:35:10.722 INFO:tasks.workunit.client.1.smithi157.stdout:0/809: rename d17/d23/d34/d67 to d17/d23/d34/d67/d7f/d69/d111 22 2024-04-22T23:35:10.723 INFO:tasks.workunit.client.1.smithi157.stdout:0/810: fdatasync d17/d23/d34/d67/f8d 0 2024-04-22T23:35:10.724 INFO:tasks.workunit.client.0.smithi107.stdout:4/767: rename d1/d7/d63/d78/d80/f82 to d1/d7/d63/d78/d96/da0/de9/f10f 0 2024-04-22T23:35:10.724 INFO:tasks.workunit.client.0.smithi107.stdout:4/768: write d1/d21/d29/d2e/f4f [2226627,108968] 0 2024-04-22T23:35:10.724 INFO:tasks.workunit.client.0.smithi107.stdout:4/769: stat d1/d7/d63 0 2024-04-22T23:35:10.724 INFO:tasks.workunit.client.0.smithi107.stdout:5/768: mkdir d9/db/de/d72/d7c/d109 0 2024-04-22T23:35:10.725 INFO:tasks.workunit.client.0.smithi107.stdout:2/767: symlink d1/dcd/lfa 0 2024-04-22T23:35:10.726 INFO:tasks.workunit.client.1.smithi157.stdout:0/811: mknod d17/d2b/d4f/d5a/dd2/d10a/c112 0 2024-04-22T23:35:10.726 INFO:tasks.workunit.client.1.smithi157.stdout:0/812: chown d17/d38/fb9 39442 1 2024-04-22T23:35:10.726 INFO:tasks.workunit.client.1.smithi157.stdout:0/813: chown d17/d2b/d4f/d5a/d7c 14 1 2024-04-22T23:35:10.726 INFO:tasks.workunit.client.1.smithi157.stdout:0/814: chown d17/d2b/d52/dd1/fd4 451 1 2024-04-22T23:35:10.728 INFO:tasks.workunit.client.0.smithi107.stdout:7/666: rename d7/f5e to d7/d21/d48/d5c/d9d/db1/fdb 0 2024-04-22T23:35:10.728 INFO:tasks.workunit.client.0.smithi107.stdout:7/667: truncate d7/d21/da4/d66/fc1 685466 0 2024-04-22T23:35:10.728 INFO:tasks.workunit.client.0.smithi107.stdout:2/768: dread d1/d12/d35/f7a [0,4194304] 0 2024-04-22T23:35:10.729 INFO:tasks.workunit.client.0.smithi107.stdout:4/770: truncate d1/d7/d63/f72 440645 0 2024-04-22T23:35:10.729 INFO:tasks.workunit.client.0.smithi107.stdout:4/771: read - d1/d7/ff1 zero size 2024-04-22T23:35:10.729 INFO:tasks.workunit.client.0.smithi107.stdout:4/772: write d1/d7/d63/d78/d96/fa6 [1738187,32708] 0 2024-04-22T23:35:10.729 INFO:tasks.workunit.client.1.smithi157.stdout:0/815: unlink d17/d2b/l4b 0 2024-04-22T23:35:10.730 INFO:tasks.workunit.client.0.smithi107.stdout:5/769: rename f4 to d9/db/de/d72/d9e/f10a 0 2024-04-22T23:35:10.731 INFO:tasks.workunit.client.0.smithi107.stdout:5/770: write d9/d26/d6a/f6f [5143210,61827] 0 2024-04-22T23:35:10.732 INFO:tasks.workunit.client.0.smithi107.stdout:7/668: creat d7/d21/d48/daf/fdc x:0 0 0 2024-04-22T23:35:10.733 INFO:tasks.workunit.client.0.smithi107.stdout:4/773: rmdir d1/d7/d63 39 2024-04-22T23:35:10.733 INFO:tasks.workunit.client.1.smithi157.stdout:0/816: dread d17/d2b/d71/fc4 [0,4194304] 0 2024-04-22T23:35:10.734 INFO:tasks.workunit.client.1.smithi157.stdout:0/817: write d17/f2d [233830,61230] 0 2024-04-22T23:35:10.736 INFO:tasks.workunit.client.0.smithi107.stdout:7/669: mkdir d7/d21/d48/d5c/d9d/ddd 0 2024-04-22T23:35:10.736 INFO:tasks.workunit.client.0.smithi107.stdout:7/670: chown d7/d21/d48/daf/fdc 7377649 1 2024-04-22T23:35:10.736 INFO:tasks.workunit.client.0.smithi107.stdout:5/771: getdents d9/de6 0 2024-04-22T23:35:10.740 INFO:tasks.workunit.client.0.smithi107.stdout:5/772: getdents d9/db/d92 0 2024-04-22T23:35:10.741 INFO:tasks.workunit.client.0.smithi107.stdout:5/773: truncate d9/db/de/d72/f81 44993 0 2024-04-22T23:35:10.745 INFO:tasks.workunit.client.0.smithi107.stdout:2/769: write d1/d2b/d59/fac [1710124,35678] 0 2024-04-22T23:35:10.746 INFO:tasks.workunit.client.0.smithi107.stdout:2/770: fdatasync d1/d2b/d88/fe1 0 2024-04-22T23:35:10.749 INFO:tasks.workunit.client.0.smithi107.stdout:3/694: dwrite db/d21/d27/d15/d29/f6d [0,4194304] 0 2024-04-22T23:35:10.749 INFO:tasks.workunit.client.0.smithi107.stdout:6/745: dwrite db/fdf [0,4194304] 0 2024-04-22T23:35:10.749 INFO:tasks.workunit.client.0.smithi107.stdout:6/746: dread - db/d20/d2a/d38/d61/d6f/da2/f9c zero size 2024-04-22T23:35:10.749 INFO:tasks.workunit.client.0.smithi107.stdout:6/747: write db/d20/d2c/d3b/d40/d9e/fb8 [5779770,100231] 0 2024-04-22T23:35:10.750 INFO:tasks.workunit.client.0.smithi107.stdout:2/771: link d1/d12/ce3 d1/d63/de6/cfb 0 2024-04-22T23:35:10.751 INFO:tasks.workunit.client.1.smithi157.stdout:7/824: dwrite d0/d9/db/df/f2b [0,4194304] 0 2024-04-22T23:35:10.752 INFO:tasks.workunit.client.0.smithi107.stdout:9/699: dwrite d5/fc [0,4194304] 0 2024-04-22T23:35:10.752 INFO:tasks.workunit.client.0.smithi107.stdout:7/671: dread d7/d21/d54/f90 [0,4194304] 0 2024-04-22T23:35:10.754 INFO:tasks.workunit.client.0.smithi107.stdout:2/772: mknod d1/d12/d35/d48/d13/cfc 0 2024-04-22T23:35:10.757 INFO:tasks.workunit.client.0.smithi107.stdout:9/700: rename d5/d7/d17/d1b/f48 to d5/d14/d2e/ff3 0 2024-04-22T23:35:10.757 INFO:tasks.workunit.client.0.smithi107.stdout:9/701: getdents d5/d7/d27/d46/d4b/def 0 2024-04-22T23:35:10.758 INFO:tasks.workunit.client.0.smithi107.stdout:7/672: mknod d7/d21/d48/daf/cde 0 2024-04-22T23:35:10.760 INFO:tasks.workunit.client.0.smithi107.stdout:6/748: getdents db/d20/d2a/d38/d61/d6f/da2/d84 0 2024-04-22T23:35:10.764 INFO:tasks.workunit.client.0.smithi107.stdout:9/702: creat d5/d7/de9/ff4 x:0 0 0 2024-04-22T23:35:10.764 INFO:tasks.workunit.client.0.smithi107.stdout:6/749: symlink db/d20/d2a/le3 0 2024-04-22T23:35:10.764 INFO:tasks.workunit.client.0.smithi107.stdout:6/750: chown db/c99 822257 1 2024-04-22T23:35:10.765 INFO:tasks.workunit.client.0.smithi107.stdout:7/673: truncate d7/d21/d48/f6b 1125167 0 2024-04-22T23:35:10.770 INFO:tasks.workunit.client.0.smithi107.stdout:9/703: symlink d5/d7/d17/d1b/d1f/d36/d53/lf5 0 2024-04-22T23:35:10.770 INFO:tasks.workunit.client.0.smithi107.stdout:9/704: truncate d5/d14/fa2 377118 0 2024-04-22T23:35:10.770 INFO:tasks.workunit.client.0.smithi107.stdout:9/705: readlink d5/d29/d7b/d91/dde/lbe 0 2024-04-22T23:35:10.771 INFO:tasks.workunit.client.1.smithi157.stdout:7/825: read d0/d9/db/df/d68/f8e [2712218,14705] 0 2024-04-22T23:35:10.771 INFO:tasks.workunit.client.0.smithi107.stdout:7/674: mkdir d7/d21/d7e/ddf 0 2024-04-22T23:35:10.772 INFO:tasks.workunit.client.1.smithi157.stdout:7/826: symlink d0/d9/db/df/dac/ded/lf9 0 2024-04-22T23:35:10.772 INFO:tasks.workunit.client.1.smithi157.stdout:7/827: chown d0/d9/db/d35 1818403 1 2024-04-22T23:35:10.773 INFO:tasks.workunit.client.0.smithi107.stdout:7/675: mkdir d7/d21/d48/d5c/d9d/d6e/db2/de0 0 2024-04-22T23:35:10.773 INFO:tasks.workunit.client.0.smithi107.stdout:7/676: creat d7/d21/d48/daf/fe1 x:0 0 0 2024-04-22T23:35:10.774 INFO:tasks.workunit.client.1.smithi157.stdout:7/828: mknod d0/d9/db/cfa 0 2024-04-22T23:35:10.774 INFO:tasks.workunit.client.1.smithi157.stdout:7/829: getdents d0/d9/d29/d5e/d85 0 2024-04-22T23:35:10.776 INFO:tasks.workunit.client.0.smithi107.stdout:7/677: creat d7/d21/d48/d5c/d9d/d6e/fe2 x:0 0 0 2024-04-22T23:35:10.777 INFO:tasks.workunit.client.0.smithi107.stdout:6/751: dread db/d20/d2a/d38/f56 [0,4194304] 0 2024-04-22T23:35:10.777 INFO:tasks.workunit.client.0.smithi107.stdout:6/752: chown db/d20/d2a/d38/d61/d6f/da2/fc3 36 1 2024-04-22T23:35:10.777 INFO:tasks.workunit.client.0.smithi107.stdout:7/678: link d7/d1b/f24 d7/d21/d48/d5c/d9d/ddd/fe3 0 2024-04-22T23:35:10.782 INFO:tasks.workunit.client.0.smithi107.stdout:9/706: dread d5/d29/f61 [0,4194304] 0 2024-04-22T23:35:10.783 INFO:tasks.workunit.client.0.smithi107.stdout:9/707: mkdir d5/d14/d2e/d4f/d73/df6 0 2024-04-22T23:35:10.783 INFO:tasks.workunit.client.0.smithi107.stdout:9/708: write d5/d14/d2e/f84 [1494236,106396] 0 2024-04-22T23:35:10.783 INFO:tasks.workunit.client.0.smithi107.stdout:9/709: chown d5/d29/d7b/d91/dde/dc9/c8a 9 1 2024-04-22T23:35:10.783 INFO:tasks.workunit.client.0.smithi107.stdout:9/710: truncate d5/d7/d27/d46/d4b/fa9 216104 0 2024-04-22T23:35:10.785 INFO:tasks.workunit.client.1.smithi157.stdout:7/830: write d0/f46 [2445831,59101] 0 2024-04-22T23:35:10.785 INFO:tasks.workunit.client.1.smithi157.stdout:7/831: getdents d0/d9/d29/d5e/d85 0 2024-04-22T23:35:10.786 INFO:tasks.workunit.client.0.smithi107.stdout:9/711: mkdir d5/d14/df7 0 2024-04-22T23:35:10.787 INFO:tasks.workunit.client.0.smithi107.stdout:9/712: creat d5/d14/d2e/d4f/d73/df6/ff8 x:0 0 0 2024-04-22T23:35:10.787 INFO:tasks.workunit.client.0.smithi107.stdout:9/713: write d5/d7/d17/d1b/d1f/d36/d53/ff1 [1277,122762] 0 2024-04-22T23:35:10.788 INFO:tasks.workunit.client.1.smithi157.stdout:7/832: creat d0/d9/d42/d55/d56/d91/de6/ffb x:0 0 0 2024-04-22T23:35:10.788 INFO:tasks.workunit.client.1.smithi157.stdout:7/833: stat d0/d9/d29/d98 0 2024-04-22T23:35:10.788 INFO:tasks.workunit.client.1.smithi157.stdout:7/834: creat d0/d9/db/d35/ffc x:0 0 0 2024-04-22T23:35:10.788 INFO:tasks.workunit.client.0.smithi107.stdout:7/679: write d7/d1b/f38 [1881015,95081] 0 2024-04-22T23:35:10.788 INFO:tasks.workunit.client.0.smithi107.stdout:3/695: dwrite db/d21/d27/d15/d29/d4f/db2/fd1 [0,4194304] 0 2024-04-22T23:35:10.788 INFO:tasks.workunit.client.0.smithi107.stdout:2/773: truncate d1/d2b/d49/d7e/fe5 737249 0 2024-04-22T23:35:10.789 INFO:tasks.workunit.client.1.smithi157.stdout:0/818: dwrite d17/d2b/d71/fc4 [0,4194304] 0 2024-04-22T23:35:10.790 INFO:tasks.workunit.client.0.smithi107.stdout:4/774: dwrite d1/d21/d29/d2e/d88/d99/dd2/daf/dc1/f9e [0,4194304] 0 2024-04-22T23:35:10.790 INFO:tasks.workunit.client.0.smithi107.stdout:9/714: creat d5/d7/d27/ff9 x:0 0 0 2024-04-22T23:35:10.790 INFO:tasks.workunit.client.0.smithi107.stdout:6/753: dread db/f2d [0,4194304] 0 2024-04-22T23:35:10.790 INFO:tasks.workunit.client.0.smithi107.stdout:5/774: dwrite d9/db/d92/dc2/ff9 [0,4194304] 0 2024-04-22T23:35:10.793 INFO:tasks.workunit.client.0.smithi107.stdout:3/696: mkdir db/d21/d27/d15/d24/d66/da5/def 0 2024-04-22T23:35:10.794 INFO:tasks.workunit.client.1.smithi157.stdout:0/819: write d17/f19 [686315,90352] 0 2024-04-22T23:35:10.794 INFO:tasks.workunit.client.1.smithi157.stdout:7/835: getdents d0/d9/d42/d55/d56/d91 0 2024-04-22T23:35:10.795 INFO:tasks.workunit.client.0.smithi107.stdout:2/774: rmdir d1/d12/d35/d54 39 2024-04-22T23:35:10.795 INFO:tasks.workunit.client.0.smithi107.stdout:2/775: fdatasync d1/d63/fb6 0 2024-04-22T23:35:10.796 INFO:tasks.workunit.client.1.smithi157.stdout:0/820: rename d17/d2b/d4f/d5a/dd2/de4/lec to d17/d23/d34/d67/l113 0 2024-04-22T23:35:10.797 INFO:tasks.workunit.client.1.smithi157.stdout:0/821: getdents d17/d23/d34/d67/d7f 0 2024-04-22T23:35:10.797 INFO:tasks.workunit.client.1.smithi157.stdout:0/822: chown d17/d2b/d4f/d5a/dd2/de4/l10c 731294306 1 2024-04-22T23:35:10.798 INFO:tasks.workunit.client.0.smithi107.stdout:4/775: mkdir d1/d21/d29/d110 0 2024-04-22T23:35:10.798 INFO:tasks.workunit.client.0.smithi107.stdout:4/776: chown d1/d21/d29/d2e/f4f 232 1 2024-04-22T23:35:10.798 INFO:tasks.workunit.client.0.smithi107.stdout:7/680: write d7/d21/f6d [6977821,76813] 0 2024-04-22T23:35:10.799 INFO:tasks.workunit.client.0.smithi107.stdout:9/715: creat d5/d14/d2e/ffa x:0 0 0 2024-04-22T23:35:10.799 INFO:tasks.workunit.client.0.smithi107.stdout:9/716: fdatasync d5/d29/f2c 0 2024-04-22T23:35:10.801 INFO:tasks.workunit.client.1.smithi157.stdout:0/823: rename d17/f1b to d17/d23/d34/dee/f114 0 2024-04-22T23:35:10.802 INFO:tasks.workunit.client.1.smithi157.stdout:7/836: write d0/f2 [2657495,17596] 0 2024-04-22T23:35:10.803 INFO:tasks.workunit.client.0.smithi107.stdout:6/754: link db/d20/d2a/d35/d69/d79/f95 db/d20/d2a/dca/fe4 0 2024-04-22T23:35:10.803 INFO:tasks.workunit.client.0.smithi107.stdout:6/755: creat db/d20/d2a/d38/d4b/fe5 x:0 0 0 2024-04-22T23:35:10.803 INFO:tasks.workunit.client.1.smithi157.stdout:8/674: sync 2024-04-22T23:35:10.804 INFO:tasks.workunit.client.1.smithi157.stdout:4/987: sync 2024-04-22T23:35:10.804 INFO:tasks.workunit.client.1.smithi157.stdout:4/988: stat d3/d96/d40 0 2024-04-22T23:35:10.804 INFO:tasks.workunit.client.1.smithi157.stdout:9/822: sync 2024-04-22T23:35:10.804 INFO:tasks.workunit.client.1.smithi157.stdout:9/823: fdatasync d4/d15/d23/fdb 0 2024-04-22T23:35:10.804 INFO:tasks.workunit.client.1.smithi157.stdout:1/945: sync 2024-04-22T23:35:10.804 INFO:tasks.workunit.client.1.smithi157.stdout:5/934: sync 2024-04-22T23:35:10.804 INFO:tasks.workunit.client.1.smithi157.stdout:3/944: sync 2024-04-22T23:35:10.804 INFO:tasks.workunit.client.1.smithi157.stdout:5/935: creat d1/d5/d7/d10/f151 x:0 0 0 2024-04-22T23:35:10.805 INFO:tasks.workunit.client.1.smithi157.stdout:8/675: chown d2/d6/d21/c93 13618 1 2024-04-22T23:35:10.806 INFO:tasks.workunit.client.1.smithi157.stdout:1/946: mknod d4/d17/d34/d75/dc7/dea/c14b 0 2024-04-22T23:35:10.807 INFO:tasks.workunit.client.0.smithi107.stdout:2/776: dread d1/d12/d35/d48/f2a [0,4194304] 0 2024-04-22T23:35:10.807 INFO:tasks.workunit.client.0.smithi107.stdout:5/775: creat d9/db/de/d72/d7c/f10b x:0 0 0 2024-04-22T23:35:10.816 INFO:tasks.workunit.client.0.smithi107.stdout:3/697: mknod db/d21/d27/d15/d29/d2a/d45/cf0 0 2024-04-22T23:35:10.822 INFO:tasks.workunit.client.0.smithi107.stdout:6/756: rename db/d20/d2a/d38/d61/d6f/da2/f9c to db/d20/d2a/d38/d61/d6f/da2/fe6 0 2024-04-22T23:35:10.822 INFO:tasks.workunit.client.0.smithi107.stdout:5/776: mknod d9/db/c10c 0 2024-04-22T23:35:10.823 INFO:tasks.workunit.client.0.smithi107.stdout:3/698: write db/d21/d27/f20 [953826,86039] 0 2024-04-22T23:35:10.823 INFO:tasks.workunit.client.0.smithi107.stdout:6/757: truncate db/d20/d2a/d38/d61/d6f/da2/fdb 498974 0 2024-04-22T23:35:10.823 INFO:tasks.workunit.client.1.smithi157.stdout:9/824: symlink d4/d15/d23/d27/db5/db9/dcd/lfc 0 2024-04-22T23:35:10.824 INFO:tasks.workunit.client.0.smithi107.stdout:5/777: truncate f8 7546429 0 2024-04-22T23:35:10.826 INFO:tasks.workunit.client.1.smithi157.stdout:8/676: creat d2/d4/d73/fe9 x:0 0 0 2024-04-22T23:35:10.826 INFO:tasks.workunit.client.1.smithi157.stdout:4/989: getdents d3/d11/d1d/d73/da3/d31/d91/dcb/dcd 0 2024-04-22T23:35:10.827 INFO:tasks.workunit.client.0.smithi107.stdout:3/699: truncate db/d21/d27/d15/d29/d2a/d45/ddd/d25/f34 1309744 0 2024-04-22T23:35:10.827 INFO:tasks.workunit.client.0.smithi107.stdout:3/700: fdatasync db/d21/d27/d15/d29/d4f/f77 0 2024-04-22T23:35:10.827 INFO:tasks.workunit.client.0.smithi107.stdout:3/701: fsync db/d21/d27/d36/fd0 0 2024-04-22T23:35:10.827 INFO:tasks.workunit.client.1.smithi157.stdout:1/947: creat d4/d17/d34/d84/f14c x:0 0 0 2024-04-22T23:35:10.828 INFO:tasks.workunit.client.0.smithi107.stdout:5/778: symlink d9/db/d42/d75/l10d 0 2024-04-22T23:35:10.829 INFO:tasks.workunit.client.1.smithi157.stdout:9/825: creat d4/d15/d23/d27/d47/ffd x:0 0 0 2024-04-22T23:35:10.829 INFO:tasks.workunit.client.1.smithi157.stdout:8/677: creat d2/d4/dbd/d97/fea x:0 0 0 2024-04-22T23:35:10.830 INFO:tasks.workunit.client.0.smithi107.stdout:6/758: getdents db/d20/d2c/d3b 0 2024-04-22T23:35:10.831 INFO:tasks.workunit.client.1.smithi157.stdout:4/990: mkdir d3/d11/d1d/d73/da3/d31/df5/d14a 0 2024-04-22T23:35:10.831 INFO:tasks.workunit.client.1.smithi157.stdout:1/948: creat d4/d17/d34/d84/d94/f14d x:0 0 0 2024-04-22T23:35:10.831 INFO:tasks.workunit.client.1.smithi157.stdout:8/678: symlink d2/d6/d21/d5f/d17/d7f/leb 0 2024-04-22T23:35:10.831 INFO:tasks.workunit.client.1.smithi157.stdout:8/679: dread - d2/d6/d21/d5f/d11/d13/fe8 zero size 2024-04-22T23:35:10.833 INFO:tasks.workunit.client.1.smithi157.stdout:1/949: mkdir d4/d14e 0 2024-04-22T23:35:10.833 INFO:tasks.workunit.client.1.smithi157.stdout:1/950: write d4/f26 [2762565,2543] 0 2024-04-22T23:35:10.835 INFO:tasks.workunit.client.1.smithi157.stdout:1/951: creat d4/d17/d34/d3b/d52/d65/dbc/d149/f14f x:0 0 0 2024-04-22T23:35:10.835 INFO:tasks.workunit.client.1.smithi157.stdout:7/837: dwrite d0/d9/d29/d49/f84 [0,4194304] 0 2024-04-22T23:35:10.836 INFO:tasks.workunit.client.0.smithi107.stdout:5/779: link d9/db/c35 d9/db/d42/c10e 0 2024-04-22T23:35:10.836 INFO:tasks.workunit.client.0.smithi107.stdout:5/780: truncate d9/db/dc1/fc9 1261644 0 2024-04-22T23:35:10.836 INFO:tasks.workunit.client.0.smithi107.stdout:7/681: dwrite d7/d21/d49/f86 [0,4194304] 0 2024-04-22T23:35:10.836 INFO:tasks.workunit.client.0.smithi107.stdout:7/682: truncate d7/d21/d48/d5c/d9d/d6e/f70 1403065 0 2024-04-22T23:35:10.837 INFO:tasks.workunit.client.1.smithi157.stdout:1/952: rename d4/d17/d1d/d33/d41/d54/db3/f11e to d4/d5f/f150 0 2024-04-22T23:35:10.838 INFO:tasks.workunit.client.0.smithi107.stdout:5/781: mknod d9/d26/d6a/d100/c10f 0 2024-04-22T23:35:10.839 INFO:tasks.workunit.client.1.smithi157.stdout:9/826: dread d4/df/d80/fe0 [0,4194304] 0 2024-04-22T23:35:10.839 INFO:tasks.workunit.client.1.smithi157.stdout:9/827: truncate d4/f8 4765310 0 2024-04-22T23:35:10.840 INFO:tasks.workunit.client.1.smithi157.stdout:0/824: dwrite d17/d2b/d101/f105 [0,4194304] 0 2024-04-22T23:35:10.840 INFO:tasks.workunit.client.1.smithi157.stdout:7/838: rename d0/d9/d42/d55/d80/d95/fe9 to d0/d9/d42/d55/d80/d95/ffd 0 2024-04-22T23:35:10.841 INFO:tasks.workunit.client.1.smithi157.stdout:1/953: mknod d4/d17/d34/d3b/d50/dad/c151 0 2024-04-22T23:35:10.841 INFO:tasks.workunit.client.1.smithi157.stdout:1/954: write d4/f25 [9388592,113220] 0 2024-04-22T23:35:10.841 INFO:tasks.workunit.client.1.smithi157.stdout:9/828: rmdir d4/d15/d23/d27/d47/db3 39 2024-04-22T23:35:10.843 INFO:tasks.workunit.client.0.smithi107.stdout:5/782: rename d9/db/f50 to d9/de6/f110 0 2024-04-22T23:35:10.843 INFO:tasks.workunit.client.0.smithi107.stdout:5/783: stat d9/db/f5f 0 2024-04-22T23:35:10.843 INFO:tasks.workunit.client.0.smithi107.stdout:5/784: fdatasync d9/f4e 0 2024-04-22T23:35:10.844 INFO:tasks.workunit.client.1.smithi157.stdout:5/936: dwrite d1/db0/df3/f12a [0,4194304] 0 2024-04-22T23:35:10.844 INFO:tasks.workunit.client.1.smithi157.stdout:3/945: dwrite d0/d83/dbc/d12e/d34/d3d/df6/dee/d9e/fe2 [4194304,4194304] 0 2024-04-22T23:35:10.846 INFO:tasks.workunit.client.1.smithi157.stdout:0/825: unlink d17/d2b/d71/l87 0 2024-04-22T23:35:10.846 INFO:tasks.workunit.client.1.smithi157.stdout:0/826: readlink d17/d38/l54 0 2024-04-22T23:35:10.847 INFO:tasks.workunit.client.1.smithi157.stdout:7/839: creat d0/d9/d29/d5e/da6/ffe x:0 0 0 2024-04-22T23:35:10.848 INFO:tasks.workunit.client.1.smithi157.stdout:8/680: read d2/d4/d73/fb5 [153544,122037] 0 2024-04-22T23:35:10.849 INFO:tasks.workunit.client.1.smithi157.stdout:1/955: rename d4/d17/c48 to d4/db/d144/c152 0 2024-04-22T23:35:10.850 INFO:tasks.workunit.client.0.smithi107.stdout:0/710: sync 2024-04-22T23:35:10.850 INFO:tasks.workunit.client.0.smithi107.stdout:1/662: sync 2024-04-22T23:35:10.851 INFO:tasks.workunit.client.0.smithi107.stdout:3/702: read db/d21/d27/d15/d29/f42 [414895,96368] 0 2024-04-22T23:35:10.852 INFO:tasks.workunit.client.1.smithi157.stdout:3/946: creat d0/d83/dbc/d12e/d13/f146 x:0 0 0 2024-04-22T23:35:10.852 INFO:tasks.workunit.client.1.smithi157.stdout:3/947: chown d0/d83/dbc/d12e/d34/d3d/f44 5 1 2024-04-22T23:35:10.852 INFO:tasks.workunit.client.1.smithi157.stdout:3/948: write d0/f1c [2338141,21923] 0 2024-04-22T23:35:10.852 INFO:tasks.workunit.client.1.smithi157.stdout:3/949: truncate d0/d83/f10c 891369 0 2024-04-22T23:35:10.853 INFO:tasks.workunit.client.1.smithi157.stdout:0/827: unlink d17/d2b/f2c 0 2024-04-22T23:35:10.853 INFO:tasks.workunit.client.1.smithi157.stdout:0/828: fsync d17/d2b/d52/f3f 0 2024-04-22T23:35:10.853 INFO:tasks.workunit.client.1.smithi157.stdout:0/829: readlink d17/d2b/d4f/dc5/dcc/de9/df6/lfc 0 2024-04-22T23:35:10.854 INFO:tasks.workunit.client.0.smithi107.stdout:0/711: mknod d1/d27/d8a/db5/cef 0 2024-04-22T23:35:10.854 INFO:tasks.workunit.client.0.smithi107.stdout:0/712: chown d1/d9/d14/d4f/d59/f5c 777134 1 2024-04-22T23:35:10.855 INFO:tasks.workunit.client.0.smithi107.stdout:3/703: creat db/d21/d27/d15/d24/d52/d60/d96/ff1 x:0 0 0 2024-04-22T23:35:10.856 INFO:tasks.workunit.client.0.smithi107.stdout:9/717: dwrite d5/d7/d17/d1b/d1f/d36/d96/fb2 [0,4194304] 0 2024-04-22T23:35:10.858 INFO:tasks.workunit.client.1.smithi157.stdout:8/681: unlink d2/d4/dbd/d97/fb6 0 2024-04-22T23:35:10.858 INFO:tasks.workunit.client.1.smithi157.stdout:9/829: getdents d4/d15/d23/d27 0 2024-04-22T23:35:10.858 INFO:tasks.workunit.client.0.smithi107.stdout:1/663: write d7/f2a [1446109,70135] 0 2024-04-22T23:35:10.863 INFO:tasks.workunit.client.0.smithi107.stdout:2/777: dwrite d1/d67/f7b [0,4194304] 0 2024-04-22T23:35:10.863 INFO:tasks.workunit.client.0.smithi107.stdout:6/759: dwrite db/f1d [0,4194304] 0 2024-04-22T23:35:10.864 INFO:tasks.workunit.client.0.smithi107.stdout:3/704: dread db/d21/d27/d15/d29/d2a/f5b [4194304,4194304] 0 2024-04-22T23:35:10.864 INFO:tasks.workunit.client.0.smithi107.stdout:3/705: fsync db/d21/d27/d15/d24/d52/d60/fd8 0 2024-04-22T23:35:10.864 INFO:tasks.workunit.client.0.smithi107.stdout:3/706: write db/d21/d27/d15/d29/d2a/d45/ddd/d25/fe2 [4676721,29541] 0 2024-04-22T23:35:10.864 INFO:tasks.workunit.client.0.smithi107.stdout:7/683: read d7/d21/d54/f90 [284666,57520] 0 2024-04-22T23:35:10.865 INFO:tasks.workunit.client.1.smithi157.stdout:3/950: mkdir d0/d83/dbc/d12e/d13/d147 0 2024-04-22T23:35:10.865 INFO:tasks.workunit.client.1.smithi157.stdout:3/951: chown d0/d83/dbc/d12e/d13/d13b 23 1 2024-04-22T23:35:10.865 INFO:tasks.workunit.client.0.smithi107.stdout:1/664: unlink d7/d19/d20/c51 0 2024-04-22T23:35:10.866 INFO:tasks.workunit.client.0.smithi107.stdout:1/665: stat d7/d19/d71/fc2 0 2024-04-22T23:35:10.866 INFO:tasks.workunit.client.0.smithi107.stdout:1/666: write d7/d19/d54/f70 [2525953,84118] 0 2024-04-22T23:35:10.867 INFO:tasks.workunit.client.1.smithi157.stdout:4/991: dwrite d3/d11/d1d/d73/da3/d31/d91/f9b [0,4194304] 0 2024-04-22T23:35:10.868 INFO:tasks.workunit.client.1.smithi157.stdout:0/830: mknod d17/ddb/c115 0 2024-04-22T23:35:10.868 INFO:tasks.workunit.client.0.smithi107.stdout:9/718: rmdir d5/d14/d2e 39 2024-04-22T23:35:10.868 INFO:tasks.workunit.client.1.smithi157.stdout:7/840: getdents d0/d9/d29/d98 0 2024-04-22T23:35:10.872 INFO:tasks.workunit.client.0.smithi107.stdout:2/778: link d1/d2b/d49/d7e/dae/fea d1/d2b/d3d/ffd 0 2024-04-22T23:35:10.872 INFO:tasks.workunit.client.0.smithi107.stdout:2/779: write d1/d2b/d3d/d57/fec [283309,123684] 0 2024-04-22T23:35:10.874 INFO:tasks.workunit.client.1.smithi157.stdout:8/682: creat d2/db1/d58/fec x:0 0 0 2024-04-22T23:35:10.874 INFO:tasks.workunit.client.1.smithi157.stdout:3/952: creat d0/d83/dbc/d12e/d34/d4a/f148 x:0 0 0 2024-04-22T23:35:10.875 INFO:tasks.workunit.client.0.smithi107.stdout:6/760: rmdir db/d20/d2a/d38/d61/d6f/da2/dcd 0 2024-04-22T23:35:10.877 INFO:tasks.workunit.client.0.smithi107.stdout:7/684: getdents d7/d21/d48/d5c/d9d/db1 0 2024-04-22T23:35:10.879 INFO:tasks.workunit.client.1.smithi157.stdout:4/992: creat d3/d11/d1d/d73/d80/f14b x:0 0 0 2024-04-22T23:35:10.879 INFO:tasks.workunit.client.1.smithi157.stdout:4/993: write d3/d96/d4c/f149 [713636,53092] 0 2024-04-22T23:35:10.879 INFO:tasks.workunit.client.1.smithi157.stdout:4/994: truncate d3/d96/d40/dc3/f12b 359438 0 2024-04-22T23:35:10.879 INFO:tasks.workunit.client.1.smithi157.stdout:4/995: fdatasync d3/d11/d34/da6/f131 0 2024-04-22T23:35:10.881 INFO:tasks.workunit.client.0.smithi107.stdout:1/667: link d7/l34 d7/d19/d71/db8/le3 0 2024-04-22T23:35:10.881 INFO:tasks.workunit.client.0.smithi107.stdout:1/668: fdatasync d7/d19/d20/d39/f3c 0 2024-04-22T23:35:10.883 INFO:tasks.workunit.client.0.smithi107.stdout:0/713: dwrite d1/d9/f3e [0,4194304] 0 2024-04-22T23:35:10.883 INFO:tasks.workunit.client.1.smithi157.stdout:0/831: link d17/d38/f65 d17/d2b/d4f/d9b/f116 0 2024-04-22T23:35:10.884 INFO:tasks.workunit.client.1.smithi157.stdout:0/832: write d17/d23/fbf [690223,113493] 0 2024-04-22T23:35:10.884 INFO:tasks.workunit.client.0.smithi107.stdout:5/785: dwrite d9/f36 [4194304,4194304] 0 2024-04-22T23:35:10.884 INFO:tasks.workunit.client.1.smithi157.stdout:1/956: dwrite d4/d17/d34/f49 [0,4194304] 0 2024-04-22T23:35:10.886 INFO:tasks.workunit.client.0.smithi107.stdout:0/714: dread d1/d27/d20/d2c/d2e/faf [0,4194304] 0 2024-04-22T23:35:10.886 INFO:tasks.workunit.client.0.smithi107.stdout:0/715: creat d1/d9/ff0 x:0 0 0 2024-04-22T23:35:10.886 INFO:tasks.workunit.client.0.smithi107.stdout:0/716: chown d1/c4 0 1 2024-04-22T23:35:10.886 INFO:tasks.workunit.client.0.smithi107.stdout:0/717: dread - d1/d9/ff0 zero size 2024-04-22T23:35:10.886 INFO:tasks.workunit.client.0.smithi107.stdout:0/718: chown d1/d27/d20/d2c/d41/dbe 3762 1 2024-04-22T23:35:10.887 INFO:tasks.workunit.client.0.smithi107.stdout:0/719: readlink d1/d27/d20/d2c/d41/dbe/ld2 0 2024-04-22T23:35:10.887 INFO:tasks.workunit.client.0.smithi107.stdout:6/761: mknod db/ce7 0 2024-04-22T23:35:10.887 INFO:tasks.workunit.client.0.smithi107.stdout:6/762: fdatasync db/d20/f4f 0 2024-04-22T23:35:10.887 INFO:tasks.workunit.client.1.smithi157.stdout:7/841: rename d0/d9/d29/d33/d37/c97 to d0/d9/db/df/d68/cff 0 2024-04-22T23:35:10.887 INFO:tasks.workunit.client.1.smithi157.stdout:7/842: truncate d0/d9/db/df/f9d 758571 0 2024-04-22T23:35:10.887 INFO:tasks.workunit.client.1.smithi157.stdout:7/843: fdatasync d0/d9/d29/f3b 0 2024-04-22T23:35:10.887 INFO:tasks.workunit.client.1.smithi157.stdout:7/844: write d0/d9/d72/fef [404542,26002] 0 2024-04-22T23:35:10.888 INFO:tasks.workunit.client.0.smithi107.stdout:3/707: rename db/d21/d27/d36/d9c/fba to db/d21/d27/d15/d29/d2a/d40/ff2 0 2024-04-22T23:35:10.888 INFO:tasks.workunit.client.1.smithi157.stdout:9/830: mkdir d4/d2d/dfe 0 2024-04-22T23:35:10.888 INFO:tasks.workunit.client.1.smithi157.stdout:8/683: mknod d2/d4/d73/ced 0 2024-04-22T23:35:10.889 INFO:tasks.workunit.client.0.smithi107.stdout:1/669: symlink d7/d19/d42/le4 0 2024-04-22T23:35:10.890 INFO:tasks.workunit.client.1.smithi157.stdout:4/996: symlink d3/d11/d34/da4/db1/l14c 0 2024-04-22T23:35:10.891 INFO:tasks.workunit.client.1.smithi157.stdout:0/833: rename d17/d38/ca3 to d17/d23/d34/d67/d7f/c117 0 2024-04-22T23:35:10.895 INFO:tasks.workunit.client.1.smithi157.stdout:9/831: mknod d4/df/d57/db7/df9/cff 0 2024-04-22T23:35:10.895 INFO:tasks.workunit.client.1.smithi157.stdout:7/845: link d0/d9/db/f57 d0/d9/db/df/d68/dc6/f100 0 2024-04-22T23:35:10.896 INFO:tasks.workunit.client.1.smithi157.stdout:4/997: mknod d3/d11/d1d/c14d 0 2024-04-22T23:35:10.898 INFO:tasks.workunit.client.1.smithi157.stdout:1/957: rename d4/d17/d34/d3b/d52/f7f to d4/d17/db9/d133/f153 0 2024-04-22T23:35:10.899 INFO:tasks.workunit.client.1.smithi157.stdout:3/953: dwrite d0/d83/dbc/d12e/d34/fe4 [0,4194304] 0 2024-04-22T23:35:10.900 INFO:tasks.workunit.client.0.smithi107.stdout:7/685: dwrite d7/d21/d48/fc2 [0,4194304] 0 2024-04-22T23:35:10.900 INFO:tasks.workunit.client.1.smithi157.stdout:9/832: chown d4/d15/d23/d3f/c7c 196694099 1 2024-04-22T23:35:10.900 INFO:tasks.workunit.client.1.smithi157.stdout:9/833: chown d4/d15/d23/d27/fba 14593159 1 2024-04-22T23:35:10.900 INFO:tasks.workunit.client.1.smithi157.stdout:9/834: write d4/f79 [4323074,15914] 0 2024-04-22T23:35:10.902 INFO:tasks.workunit.client.1.smithi157.stdout:3/954: creat d0/d83/df9/d119/f149 x:0 0 0 2024-04-22T23:35:10.903 INFO:tasks.workunit.client.0.smithi107.stdout:2/780: rmdir d1/d2b/d49 39 2024-04-22T23:35:10.905 INFO:tasks.workunit.client.0.smithi107.stdout:0/720: dread d1/d27/d20/f6b [0,4194304] 0 2024-04-22T23:35:10.905 INFO:tasks.workunit.client.1.smithi157.stdout:8/684: rename d2/db1/l3b to d2/d34/d6a/d77/lee 0 2024-04-22T23:35:10.907 INFO:tasks.workunit.client.1.smithi157.stdout:3/955: mkdir d0/d83/dbc/d12e/d34/d3d/d57/d60/d6b/d14a 0 2024-04-22T23:35:10.910 INFO:tasks.workunit.client.1.smithi157.stdout:4/998: rename d3/d11/d1d/d73/da7/fad to d3/d11/d30/dfd/d148/f14e 0 2024-04-22T23:35:10.911 INFO:tasks.workunit.client.1.smithi157.stdout:3/956: stat d0/d83/dbc/d12e/d34/db9/d9f/ld0 0 2024-04-22T23:35:10.911 INFO:tasks.workunit.client.1.smithi157.stdout:3/957: chown d0/d83/dbc/d12e/d34/d39/c8e 50327373 1 2024-04-22T23:35:10.911 INFO:tasks.workunit.client.1.smithi157.stdout:3/958: fdatasync d0/d83/dbc/d12e/d34/db9/fd4 0 2024-04-22T23:35:10.914 INFO:tasks.workunit.client.1.smithi157.stdout:3/959: creat d0/d83/dbc/d12e/d127/dcf/f14b x:0 0 0 2024-04-22T23:35:10.915 INFO:tasks.workunit.client.1.smithi157.stdout:4/999: getdents d3/d4b/d7a/db7 0 2024-04-22T23:35:10.915 INFO:tasks.workunit.client.0.smithi107.stdout:0/721: dread d1/d27/d20/f60 [0,4194304] 0 2024-04-22T23:35:10.918 INFO:tasks.workunit.client.0.smithi107.stdout:6/763: dwrite db/d20/d2a/d38/d4b/f81 [0,4194304] 0 2024-04-22T23:35:10.919 INFO:tasks.workunit.client.0.smithi107.stdout:1/670: truncate d7/d19/d20/d39/d53/d43/fab 320451 0 2024-04-22T23:35:10.921 INFO:tasks.workunit.client.1.smithi157.stdout:3/960: dread d0/d83/dbc/d12e/d34/d3d/d57/d60/f89 [0,4194304] 0 2024-04-22T23:35:10.921 INFO:tasks.workunit.client.1.smithi157.stdout:3/961: chown d0/d83/dbc/d12e/d34/d3d/df6/d111/f117 2716 1 2024-04-22T23:35:10.921 INFO:tasks.workunit.client.1.smithi157.stdout:3/962: write d0/d83/dbc/d12e/d34/d47/f12f [22045,91992] 0 2024-04-22T23:35:10.921 INFO:tasks.workunit.client.1.smithi157.stdout:1/958: dread d4/f23 [4194304,4194304] 0 2024-04-22T23:35:10.921 INFO:tasks.workunit.client.1.smithi157.stdout:1/959: readlink d4/d17/lb0 0 2024-04-22T23:35:10.922 INFO:tasks.workunit.client.0.smithi107.stdout:7/686: creat d7/d4c/fe4 x:0 0 0 2024-04-22T23:35:10.924 INFO:tasks.workunit.client.0.smithi107.stdout:2/781: link d1/d67/f6f d1/d12/d35/d54/ffe 0 2024-04-22T23:35:10.925 INFO:tasks.workunit.client.0.smithi107.stdout:0/722: stat d1/d9/d14/d4f/d59/lae 0 2024-04-22T23:35:10.925 INFO:tasks.workunit.client.1.smithi157.stdout:0/834: dwrite d17/d2b/d4f/d5a/f81 [4194304,4194304] 0 2024-04-22T23:35:10.926 INFO:tasks.workunit.client.0.smithi107.stdout:3/708: getdents db/d21/d27/d15/d24/d52/d60/d96 0 2024-04-22T23:35:10.928 INFO:tasks.workunit.client.1.smithi157.stdout:0/835: link d17/d23/d34/c72 d17/d2b/d52/dd1/c118 0 2024-04-22T23:35:10.931 INFO:tasks.workunit.client.0.smithi107.stdout:5/786: dwrite d9/db/de/d5e/f61 [0,4194304] 0 2024-04-22T23:35:10.931 INFO:tasks.workunit.client.0.smithi107.stdout:6/764: mkdir db/d20/d2c/de8 0 2024-04-22T23:35:10.932 INFO:tasks.workunit.client.1.smithi157.stdout:7/846: dwrite d0/d9/d42/fcc [8388608,4194304] 0 2024-04-22T23:35:10.934 INFO:tasks.workunit.client.0.smithi107.stdout:1/671: creat d7/d19/d20/d39/d53/d43/d46/dba/fe5 x:0 0 0 2024-04-22T23:35:10.934 INFO:tasks.workunit.client.0.smithi107.stdout:1/672: chown d7/d19/d20/d39/d53/d43/d46/dba 2733 1 2024-04-22T23:35:10.934 INFO:tasks.workunit.client.0.smithi107.stdout:1/673: chown d7/d19/d20/d39/d53/d43/d46/fc9 3586458 1 2024-04-22T23:35:10.935 INFO:tasks.workunit.client.1.smithi157.stdout:7/847: getdents d0/d9/db 0 2024-04-22T23:35:10.937 INFO:tasks.workunit.client.0.smithi107.stdout:7/687: truncate d7/d1b/f53 3479058 0 2024-04-22T23:35:10.938 INFO:tasks.workunit.client.1.smithi157.stdout:7/848: link d0/d9/d42/d55/d80/d82/lae d0/d9/d29/d5e/d9e/l101 0 2024-04-22T23:35:10.938 INFO:tasks.workunit.client.1.smithi157.stdout:1/960: dread d4/f18 [0,4194304] 0 2024-04-22T23:35:10.940 INFO:tasks.workunit.client.0.smithi107.stdout:0/723: link d1/d27/d20/d2c/cb1 d1/d9/d14/cf1 0 2024-04-22T23:35:10.941 INFO:tasks.workunit.client.1.smithi157.stdout:1/961: link d4/d17/fc4 d4/d17/d34/d3b/d50/f154 0 2024-04-22T23:35:10.941 INFO:tasks.workunit.client.1.smithi157.stdout:1/962: fsync d4/d17/d34/dc8/fe2 0 2024-04-22T23:35:10.942 INFO:tasks.workunit.client.1.smithi157.stdout:9/835: dwrite d4/d2d/f78 [0,4194304] 0 2024-04-22T23:35:10.943 INFO:tasks.workunit.client.0.smithi107.stdout:3/709: rmdir db/d21/d27/d15/d24/d52/d60/dca/dcc/dea 0 2024-04-22T23:35:10.943 INFO:tasks.workunit.client.0.smithi107.stdout:3/710: chown db/d21/d27/d15/d29/d2a/d40 1835 1 2024-04-22T23:35:10.943 INFO:tasks.workunit.client.0.smithi107.stdout:3/711: fdatasync db/d21/d27/d15/f8c 0 2024-04-22T23:35:10.943 INFO:tasks.workunit.client.0.smithi107.stdout:3/712: creat db/d21/d27/d15/d24/d87/ff3 x:0 0 0 2024-04-22T23:35:10.944 INFO:tasks.workunit.client.1.smithi157.stdout:1/963: mkdir d4/db/d3d/dc1/d100/d155 0 2024-04-22T23:35:10.944 INFO:tasks.workunit.client.1.smithi157.stdout:1/964: write d4/d17/d1d/d33/d41/d54/fe9 [1261505,14561] 0 2024-04-22T23:35:10.944 INFO:tasks.workunit.client.1.smithi157.stdout:1/965: dread - d4/d5f/f150 zero size 2024-04-22T23:35:10.945 INFO:tasks.workunit.client.1.smithi157.stdout:1/966: stat d4/d17/d34/faf 0 2024-04-22T23:35:10.945 INFO:tasks.workunit.client.1.smithi157.stdout:1/967: stat d4/d5f/f150 0 2024-04-22T23:35:10.945 INFO:tasks.workunit.client.1.smithi157.stdout:1/968: dread - d4/d17/d34/d84/d94/f14d zero size 2024-04-22T23:35:10.945 INFO:tasks.workunit.client.0.smithi107.stdout:5/787: unlink d9/db/d42/lb5 0 2024-04-22T23:35:10.945 INFO:tasks.workunit.client.0.smithi107.stdout:5/788: stat d9/db/de/d72/d7c/da4/fb3 0 2024-04-22T23:35:10.946 INFO:tasks.workunit.client.0.smithi107.stdout:7/688: creat d7/d4c/da1/dce/fe5 x:0 0 0 2024-04-22T23:35:10.946 INFO:tasks.workunit.client.1.smithi157.stdout:9/836: symlink d4/df/d57/dbd/dc6/l100 0 2024-04-22T23:35:10.946 INFO:tasks.workunit.client.1.smithi157.stdout:9/837: write d4/df/d57/fd9 [1157284,23618] 0 2024-04-22T23:35:10.947 INFO:tasks.workunit.client.0.smithi107.stdout:1/674: read d7/d19/d42/f6d [1232612,116223] 0 2024-04-22T23:35:10.947 INFO:tasks.workunit.client.0.smithi107.stdout:1/675: chown d7/cb4 90067200 1 2024-04-22T23:35:10.947 INFO:tasks.workunit.client.0.smithi107.stdout:1/676: truncate d7/d19/d2d/d89/fd0 925386 0 2024-04-22T23:35:10.948 INFO:tasks.workunit.client.1.smithi157.stdout:1/969: mknod d4/db/d144/c156 0 2024-04-22T23:35:10.948 INFO:tasks.workunit.client.0.smithi107.stdout:7/689: dread d7/d21/d48/d5c/f7a [0,4194304] 0 2024-04-22T23:35:10.950 INFO:tasks.workunit.client.0.smithi107.stdout:5/789: rename d9/db/f7b to d9/db/de/d72/f111 0 2024-04-22T23:35:10.950 INFO:tasks.workunit.client.0.smithi107.stdout:5/790: stat d9/db/d94/f99 0 2024-04-22T23:35:10.950 INFO:tasks.workunit.client.0.smithi107.stdout:5/791: truncate d9/db/de/d5e/fe2 869238 0 2024-04-22T23:35:10.950 INFO:tasks.workunit.client.0.smithi107.stdout:1/677: rename d7/d19/d20/d39/d53/d43/d46/dba to d7/d19/d20/d39/d53/d43/d46/dba/dc4/de6 22 2024-04-22T23:35:10.951 INFO:tasks.workunit.client.0.smithi107.stdout:1/678: chown d7/d19/f95 240724 1 2024-04-22T23:35:10.952 INFO:tasks.workunit.client.0.smithi107.stdout:7/690: dread d7/f2f [0,4194304] 0 2024-04-22T23:35:10.954 INFO:tasks.workunit.client.1.smithi157.stdout:7/849: dread d0/d9/d29/d5e/fd9 [0,4194304] 0 2024-04-22T23:35:10.957 INFO:tasks.workunit.client.1.smithi157.stdout:9/838: dread d4/d15/d23/d27/f64 [0,4194304] 0 2024-04-22T23:35:10.959 INFO:tasks.workunit.client.1.smithi157.stdout:3/963: dwrite d0/d83/dbc/d12e/d34/d3d/d57/d60/f89 [0,4194304] 0 2024-04-22T23:35:10.959 INFO:tasks.workunit.client.1.smithi157.stdout:3/964: chown d0/d83/dbc/d12e/d127/f10a 789275 1 2024-04-22T23:35:10.959 INFO:tasks.workunit.client.1.smithi157.stdout:3/965: chown d0/d83/dbc/d12e/d34/d3d/df6/d72/cbd 18753 1 2024-04-22T23:35:10.963 INFO:tasks.workunit.client.1.smithi157.stdout:9/839: read d4/d15/d23/f97 [194030,16971] 0 2024-04-22T23:35:10.968 INFO:tasks.workunit.client.0.smithi107.stdout:8/733: sync 2024-04-22T23:35:10.968 INFO:tasks.workunit.client.0.smithi107.stdout:8/734: creat d0/d85/d51/f101 x:0 0 0 2024-04-22T23:35:10.968 INFO:tasks.workunit.client.0.smithi107.stdout:8/735: chown d0/f5 13100 1 2024-04-22T23:35:10.969 INFO:tasks.workunit.client.0.smithi107.stdout:0/724: rmdir d1 39 2024-04-22T23:35:10.970 INFO:tasks.workunit.client.0.smithi107.stdout:5/792: creat d9/d26/d6a/d100/f112 x:0 0 0 2024-04-22T23:35:10.971 INFO:tasks.workunit.client.0.smithi107.stdout:0/725: mkdir d1/d27/d20/d40/dbd/df2 0 2024-04-22T23:35:10.975 INFO:tasks.workunit.client.0.smithi107.stdout:0/726: write d1/d27/d20/d2c/fbb [719605,108055] 0 2024-04-22T23:35:10.975 INFO:tasks.workunit.client.0.smithi107.stdout:0/727: truncate d1/d27/d20/fda 110127 0 2024-04-22T23:35:10.977 INFO:tasks.workunit.client.0.smithi107.stdout:0/728: unlink d1/d9/cd 0 2024-04-22T23:35:10.978 INFO:tasks.workunit.client.1.smithi157.stdout:9/840: rename d4/d15/d23/d67/ca2 to d4/d15/d23/dad/df2/c101 0 2024-04-22T23:35:10.979 INFO:tasks.workunit.client.0.smithi107.stdout:5/793: dread d9/f12 [0,4194304] 0 2024-04-22T23:35:10.980 INFO:tasks.workunit.client.1.smithi157.stdout:0/836: dwrite d17/d2b/d5b/ff4 [0,4194304] 0 2024-04-22T23:35:10.980 INFO:tasks.workunit.client.1.smithi157.stdout:0/837: write d17/d23/d34/fb6 [1356899,7483] 0 2024-04-22T23:35:10.983 INFO:tasks.workunit.client.1.smithi157.stdout:0/838: symlink d17/d23/d34/l119 0 2024-04-22T23:35:10.983 INFO:tasks.workunit.client.1.smithi157.stdout:0/839: dread - d17/d2b/d5b/fc6 zero size 2024-04-22T23:35:10.984 INFO:tasks.workunit.client.1.smithi157.stdout:0/840: rmdir d17/d23/d34/d67/d7f/d69/dce 39 2024-04-22T23:35:10.985 INFO:tasks.workunit.client.1.smithi157.stdout:0/841: fdatasync d17/d23/f28 0 2024-04-22T23:35:10.985 INFO:tasks.workunit.client.0.smithi107.stdout:2/782: dwrite d1/d12/d35/d48/d13/d4f/f62 [4194304,4194304] 0 2024-04-22T23:35:10.988 INFO:tasks.workunit.client.0.smithi107.stdout:6/765: dwrite db/d20/d2a/d38/d61/f6d [0,4194304] 0 2024-04-22T23:35:10.988 INFO:tasks.workunit.client.0.smithi107.stdout:2/783: creat d1/d2b/d49/fff x:0 0 0 2024-04-22T23:35:10.988 INFO:tasks.workunit.client.0.smithi107.stdout:6/766: read - db/fcb zero size 2024-04-22T23:35:10.990 INFO:tasks.workunit.client.0.smithi107.stdout:2/784: dread d1/d12/f46 [0,4194304] 0 2024-04-22T23:35:10.991 INFO:tasks.workunit.client.0.smithi107.stdout:2/785: creat d1/d63/f100 x:0 0 0 2024-04-22T23:35:10.992 INFO:tasks.workunit.client.0.smithi107.stdout:2/786: symlink d1/d67/l101 0 2024-04-22T23:35:10.998 INFO:tasks.workunit.client.0.smithi107.stdout:2/787: rename d1/d2b/d49 to d1/d12/d35/d102 0 2024-04-22T23:35:10.998 INFO:tasks.workunit.client.0.smithi107.stdout:2/788: fsync d1/d12/d35/d102/d7e/f8e 0 2024-04-22T23:35:10.998 INFO:tasks.workunit.client.0.smithi107.stdout:2/789: dread - d1/d2b/d59/ff2 zero size 2024-04-22T23:35:10.999 INFO:tasks.workunit.client.0.smithi107.stdout:2/790: symlink d1/d12/d35/d54/l103 0 2024-04-22T23:35:10.999 INFO:tasks.workunit.client.0.smithi107.stdout:2/791: chown d1/l4d 117055 1 2024-04-22T23:35:10.999 INFO:tasks.workunit.client.0.smithi107.stdout:2/792: stat d1/d12/d35/d54/f73 0 2024-04-22T23:35:10.999 INFO:tasks.workunit.client.0.smithi107.stdout:6/767: dread db/f5e [0,4194304] 0 2024-04-22T23:35:11.000 INFO:tasks.workunit.client.0.smithi107.stdout:6/768: truncate db/d20/d2c/f32 6348112 0 2024-04-22T23:35:11.001 INFO:tasks.workunit.client.0.smithi107.stdout:6/769: rename db/d20/d2c/c5c to db/d20/d2a/d35/d69/ce9 0 2024-04-22T23:35:11.010 INFO:tasks.workunit.client.0.smithi107.stdout:2/793: read d1/d63/fa1 [1744342,20539] 0 2024-04-22T23:35:11.012 INFO:tasks.workunit.client.0.smithi107.stdout:2/794: creat d1/d12/d35/d48/d13/d4f/f104 x:0 0 0 2024-04-22T23:35:11.014 INFO:tasks.workunit.client.1.smithi157.stdout:7/850: dwrite d0/d9/db/df/f2b [8388608,4194304] 0 2024-04-22T23:35:11.016 INFO:tasks.workunit.client.0.smithi107.stdout:3/713: dwrite db/d21/d27/d15/fb4 [0,4194304] 0 2024-04-22T23:35:11.016 INFO:tasks.workunit.client.0.smithi107.stdout:1/679: dwrite d7/f67 [4194304,4194304] 0 2024-04-22T23:35:11.017 INFO:tasks.workunit.client.0.smithi107.stdout:1/680: chown d7/d19/d2d/d89/fd0 15484282 1 2024-04-22T23:35:11.017 INFO:tasks.workunit.client.1.smithi157.stdout:9/841: dwrite d4/df/d61/f71 [0,4194304] 0 2024-04-22T23:35:11.018 INFO:tasks.workunit.client.0.smithi107.stdout:8/736: dwrite d0/d7/df/d33/faa [0,4194304] 0 2024-04-22T23:35:11.018 INFO:tasks.workunit.client.0.smithi107.stdout:8/737: chown d0/d7/d8/d41/dec/l60 6220 1 2024-04-22T23:35:11.019 INFO:tasks.workunit.client.0.smithi107.stdout:0/729: dwrite d1/d27/d20/d40/dbd/fce [0,4194304] 0 2024-04-22T23:35:11.020 INFO:tasks.workunit.client.0.smithi107.stdout:7/691: dwrite d7/d21/da4/f41 [4194304,4194304] 0 2024-04-22T23:35:11.020 INFO:tasks.workunit.client.0.smithi107.stdout:7/692: readlink d7/d21/l40 0 2024-04-22T23:35:11.020 INFO:tasks.workunit.client.0.smithi107.stdout:7/693: chown d7/d21/d48/d5c/d9d/db1/fdb 0 1 2024-04-22T23:35:11.020 INFO:tasks.workunit.client.1.smithi157.stdout:0/842: truncate d17/f25 76280 0 2024-04-22T23:35:11.021 INFO:tasks.workunit.client.0.smithi107.stdout:3/714: link db/d21/d27/d15/d24/d52/d60/lc7 db/d21/d27/d15/d24/lf4 0 2024-04-22T23:35:11.021 INFO:tasks.workunit.client.0.smithi107.stdout:3/715: truncate db/d21/fda 1875224 0 2024-04-22T23:35:11.025 INFO:tasks.workunit.client.0.smithi107.stdout:0/730: rename d1/d7b/d85/l9d to d1/d27/d8a/db5/lf3 0 2024-04-22T23:35:11.025 INFO:tasks.workunit.client.0.smithi107.stdout:0/731: fsync d1/d84/d99/feb 0 2024-04-22T23:35:11.030 INFO:tasks.workunit.client.0.smithi107.stdout:2/795: dwrite d1/d12/d35/d102/d87/fcf [0,4194304] 0 2024-04-22T23:35:11.030 INFO:tasks.workunit.client.0.smithi107.stdout:2/796: write d1/d12/d35/d102/d7e/da8/fb8 [851890,37566] 0 2024-04-22T23:35:11.031 INFO:tasks.workunit.client.0.smithi107.stdout:1/681: dread d7/d19/d20/d39/d53/f98 [0,4194304] 0 2024-04-22T23:35:11.031 INFO:tasks.workunit.client.0.smithi107.stdout:1/682: fsync d7/d19/d20/d39/d53/d43/d46/d5c/f8a 0 2024-04-22T23:35:11.032 INFO:tasks.workunit.client.0.smithi107.stdout:6/770: dwrite db/d20/d2a/d38/d61/d6f/da2/fac [0,4194304] 0 2024-04-22T23:35:11.038 INFO:tasks.workunit.client.0.smithi107.stdout:8/738: mknod d0/d7/df/c102 0 2024-04-22T23:35:11.038 INFO:tasks.workunit.client.0.smithi107.stdout:8/739: fdatasync d0/d7/d20/da2/fa9 0 2024-04-22T23:35:11.038 INFO:tasks.workunit.client.0.smithi107.stdout:8/740: dread - d0/d7/df/d3b/dd9/dda/ffc zero size 2024-04-22T23:35:11.040 INFO:tasks.workunit.client.0.smithi107.stdout:7/694: rename d7/d21/da4/c47 to d7/d21/d48/d5c/d9d/db1/ce6 0 2024-04-22T23:35:11.042 INFO:tasks.workunit.client.0.smithi107.stdout:3/716: dwrite db/d21/d27/d15/d24/d52/fbd [0,4194304] 0 2024-04-22T23:35:11.043 INFO:tasks.workunit.client.0.smithi107.stdout:2/797: dread d1/d12/d35/d48/f1f [0,4194304] 0 2024-04-22T23:35:11.044 INFO:tasks.workunit.client.0.smithi107.stdout:6/771: dread db/d20/d2c/d3b/fb0 [0,4194304] 0 2024-04-22T23:35:11.044 INFO:tasks.workunit.client.0.smithi107.stdout:6/772: chown db/d20/d2c/f44 28 1 2024-04-22T23:35:11.045 INFO:tasks.workunit.client.0.smithi107.stdout:7/695: creat d7/d4c/fe7 x:0 0 0 2024-04-22T23:35:11.045 INFO:tasks.workunit.client.0.smithi107.stdout:7/696: dread - d7/db4/fbb zero size 2024-04-22T23:35:11.045 INFO:tasks.workunit.client.0.smithi107.stdout:7/697: chown d7/d21/d7e/ddf 44 1 2024-04-22T23:35:11.048 INFO:tasks.workunit.client.0.smithi107.stdout:3/717: truncate db/d21/d27/d15/d29/d4f/db2/fd7 1509189 0 2024-04-22T23:35:11.048 INFO:tasks.workunit.client.0.smithi107.stdout:3/718: chown db/d21/d27/d15/d24/d52/d60/dca/dcc 750210947 1 2024-04-22T23:35:11.048 INFO:tasks.workunit.client.0.smithi107.stdout:8/741: write d0/d3a/d46/d59/f68 [3554082,50388] 0 2024-04-22T23:35:11.049 INFO:tasks.workunit.client.0.smithi107.stdout:2/798: symlink d1/d12/d35/d102/d87/l105 0 2024-04-22T23:35:11.049 INFO:tasks.workunit.client.0.smithi107.stdout:2/799: truncate d1/d2b/d59/ff2 198434 0 2024-04-22T23:35:11.049 INFO:tasks.workunit.client.0.smithi107.stdout:2/800: readlink d1/d12/d35/d54/la6 0 2024-04-22T23:35:11.049 INFO:tasks.workunit.client.0.smithi107.stdout:2/801: chown d1/d12/f26 220742 1 2024-04-22T23:35:11.049 INFO:tasks.workunit.client.0.smithi107.stdout:1/683: dwrite d7/f15 [0,4194304] 0 2024-04-22T23:35:11.051 INFO:tasks.workunit.client.0.smithi107.stdout:7/698: symlink d7/d21/d49/le8 0 2024-04-22T23:35:11.052 INFO:tasks.workunit.client.0.smithi107.stdout:7/699: write d7/d21/d54/f90 [1601783,32935] 0 2024-04-22T23:35:11.054 INFO:tasks.workunit.client.0.smithi107.stdout:2/802: mkdir d1/d12/d35/d102/d106 0 2024-04-22T23:35:11.056 INFO:tasks.workunit.client.0.smithi107.stdout:6/773: getdents db/d20/d2a/d5d 0 2024-04-22T23:35:11.058 INFO:tasks.workunit.client.0.smithi107.stdout:7/700: symlink d7/d1b/le9 0 2024-04-22T23:35:11.059 INFO:tasks.workunit.client.0.smithi107.stdout:1/684: write d7/f15 [669720,9322] 0 2024-04-22T23:35:11.059 INFO:tasks.workunit.client.0.smithi107.stdout:3/719: rmdir db/d21/d27/d15/db8 39 2024-04-22T23:35:11.059 INFO:tasks.workunit.client.0.smithi107.stdout:3/720: write db/d21/d27/d15/d29/d2a/d45/deb/fd6 [203240,47350] 0 2024-04-22T23:35:11.062 INFO:tasks.workunit.client.0.smithi107.stdout:2/803: symlink d1/d12/d35/d102/d87/l107 0 2024-04-22T23:35:11.064 INFO:tasks.workunit.client.0.smithi107.stdout:7/701: creat d7/d21/d54/fea x:0 0 0 2024-04-22T23:35:11.065 INFO:tasks.workunit.client.0.smithi107.stdout:7/702: fdatasync d7/d21/f87 0 2024-04-22T23:35:11.065 INFO:tasks.workunit.client.0.smithi107.stdout:8/742: dread d0/f72 [0,4194304] 0 2024-04-22T23:35:11.066 INFO:tasks.workunit.client.0.smithi107.stdout:2/804: creat d1/d12/d35/d102/dba/def/f108 x:0 0 0 2024-04-22T23:35:11.066 INFO:tasks.workunit.client.0.smithi107.stdout:7/703: mkdir d7/d21/d7e/deb 0 2024-04-22T23:35:11.067 INFO:tasks.workunit.client.0.smithi107.stdout:8/743: write d0/d7/df/f5c [5015984,104149] 0 2024-04-22T23:35:11.068 INFO:tasks.workunit.client.0.smithi107.stdout:8/744: chown d0/d7/d8/d41/fad 2473 1 2024-04-22T23:35:11.069 INFO:tasks.workunit.client.0.smithi107.stdout:2/805: creat d1/d12/d35/d102/d7e/dae/f109 x:0 0 0 2024-04-22T23:35:11.070 INFO:tasks.workunit.client.0.smithi107.stdout:7/704: creat d7/d21/d48/d5c/d9d/db1/fec x:0 0 0 2024-04-22T23:35:11.072 INFO:tasks.workunit.client.0.smithi107.stdout:8/745: creat d0/d7/f103 x:0 0 0 2024-04-22T23:35:11.074 INFO:tasks.workunit.client.0.smithi107.stdout:2/806: creat d1/dcd/f10a x:0 0 0 2024-04-22T23:35:11.083 INFO:tasks.workunit.client.0.smithi107.stdout:6/774: dwrite db/d20/d2a/f42 [4194304,4194304] 0 2024-04-22T23:35:11.084 INFO:tasks.workunit.client.0.smithi107.stdout:1/685: dwrite d7/fa0 [0,4194304] 0 2024-04-22T23:35:11.084 INFO:tasks.workunit.client.0.smithi107.stdout:1/686: readlink d7/d19/d20/d39/d53/d43/ld9 0 2024-04-22T23:35:11.091 INFO:tasks.workunit.client.0.smithi107.stdout:6/775: mkdir db/d20/d49/d7b/dea 0 2024-04-22T23:35:11.095 INFO:tasks.workunit.client.0.smithi107.stdout:8/746: dwrite d0/d7/df/d3b/dd9/dda/fe8 [0,4194304] 0 2024-04-22T23:35:11.095 INFO:tasks.workunit.client.0.smithi107.stdout:6/776: unlink db/d20/db1/fbe 0 2024-04-22T23:35:11.097 INFO:tasks.workunit.client.0.smithi107.stdout:7/705: dwrite d7/d21/d7e/d94/fc4 [0,4194304] 0 2024-04-22T23:35:11.098 INFO:tasks.workunit.client.0.smithi107.stdout:7/706: write d7/d21/d7e/fb8 [906439,38222] 0 2024-04-22T23:35:11.098 INFO:tasks.workunit.client.0.smithi107.stdout:7/707: truncate d7/d21/d49/f6a 922856 0 2024-04-22T23:35:11.098 INFO:tasks.workunit.client.0.smithi107.stdout:7/708: stat d7/d21/d48/f6b 0 2024-04-22T23:35:11.098 INFO:tasks.workunit.client.0.smithi107.stdout:8/747: rmdir d0/d7/df/d3b/d7b 39 2024-04-22T23:35:11.098 INFO:tasks.workunit.client.0.smithi107.stdout:8/748: creat d0/d7/d20/f104 x:0 0 0 2024-04-22T23:35:11.100 INFO:tasks.workunit.client.0.smithi107.stdout:6/777: unlink db/fdf 0 2024-04-22T23:35:11.104 INFO:tasks.workunit.client.0.smithi107.stdout:2/807: dwrite d1/d63/f100 [0,4194304] 0 2024-04-22T23:35:11.104 INFO:tasks.workunit.client.0.smithi107.stdout:2/808: chown d1/d2b/d59/f90 27829122 1 2024-04-22T23:35:11.105 INFO:tasks.workunit.client.0.smithi107.stdout:8/749: unlink d0/d7/df/d3b/dd9/dda/fe8 0 2024-04-22T23:35:11.106 INFO:tasks.workunit.client.0.smithi107.stdout:6/778: creat db/d20/d2c/d3b/feb x:0 0 0 2024-04-22T23:35:11.106 INFO:tasks.workunit.client.0.smithi107.stdout:6/779: chown db/d20/l28 37 1 2024-04-22T23:35:11.106 INFO:tasks.workunit.client.0.smithi107.stdout:6/780: write db/d20/d2a/d38/d4b/fe5 [391951,16561] 0 2024-04-22T23:35:11.109 INFO:tasks.workunit.client.0.smithi107.stdout:7/709: dread d7/d21/da4/f88 [0,4194304] 0 2024-04-22T23:35:11.110 INFO:tasks.workunit.client.0.smithi107.stdout:6/781: creat db/d20/d2a/d35/fec x:0 0 0 2024-04-22T23:35:11.111 INFO:tasks.workunit.client.0.smithi107.stdout:7/710: symlink d7/d21/d54/dd3/led 0 2024-04-22T23:35:11.111 INFO:tasks.workunit.client.0.smithi107.stdout:7/711: getdents d7/d21/d48/d5c/d9d/d6e/db2/de0 0 2024-04-22T23:35:11.112 INFO:tasks.workunit.client.0.smithi107.stdout:6/782: rmdir db/d20/d49 39 2024-04-22T23:35:11.112 INFO:tasks.workunit.client.0.smithi107.stdout:6/783: write db/d20/d2a/d38/f53 [1058417,119877] 0 2024-04-22T23:35:11.114 INFO:tasks.workunit.client.0.smithi107.stdout:7/712: truncate d7/f13 2327777 0 2024-04-22T23:35:11.117 INFO:tasks.workunit.client.0.smithi107.stdout:6/784: truncate db/d20/d2a/d38/f9f 2291095 0 2024-04-22T23:35:11.117 INFO:tasks.workunit.client.0.smithi107.stdout:6/785: truncate db/fd1 62864 0 2024-04-22T23:35:11.118 INFO:tasks.workunit.client.0.smithi107.stdout:7/713: rename d7/d21/d49/f86 to d7/db4/fee 0 2024-04-22T23:35:11.119 INFO:tasks.workunit.client.0.smithi107.stdout:8/750: dread d0/d3a/f56 [0,4194304] 0 2024-04-22T23:35:11.120 INFO:tasks.workunit.client.0.smithi107.stdout:6/786: mkdir db/d20/d2c/d3b/d40/d9e/ded 0 2024-04-22T23:35:11.120 INFO:tasks.workunit.client.0.smithi107.stdout:6/787: truncate db/d20/d2a/dca/fe4 1681071 0 2024-04-22T23:35:11.121 INFO:tasks.workunit.client.0.smithi107.stdout:8/751: truncate d0/d7/df/fca 1307533 0 2024-04-22T23:35:11.121 INFO:tasks.workunit.client.0.smithi107.stdout:8/752: dread - d0/d7/df/d3b/dd9/dda/ffc zero size 2024-04-22T23:35:11.121 INFO:tasks.workunit.client.0.smithi107.stdout:8/753: write d0/d7/f103 [283455,30763] 0 2024-04-22T23:35:11.121 INFO:tasks.workunit.client.0.smithi107.stdout:8/754: creat d0/d7/d20/da2/f105 x:0 0 0 2024-04-22T23:35:11.123 INFO:tasks.workunit.client.0.smithi107.stdout:2/809: dwrite d1/d67/f7b [0,4194304] 0 2024-04-22T23:35:11.123 INFO:tasks.workunit.client.0.smithi107.stdout:2/810: chown d1/d2b/d3d/d57/fa2 614705 1 2024-04-22T23:35:11.123 INFO:tasks.workunit.client.0.smithi107.stdout:2/811: truncate d1/d12/d35/d48/f10 1698119 0 2024-04-22T23:35:11.123 INFO:tasks.workunit.client.0.smithi107.stdout:2/812: chown d1/f78 0 1 2024-04-22T23:35:11.123 INFO:tasks.workunit.client.0.smithi107.stdout:6/788: rename db/d20/d2a/d5d/c7f to db/d20/d2a/d5d/cee 0 2024-04-22T23:35:11.126 INFO:tasks.workunit.client.0.smithi107.stdout:8/755: rename d0/d7/df/d33/d37/l8e to d0/d7/d87/l106 0 2024-04-22T23:35:11.129 INFO:tasks.workunit.client.0.smithi107.stdout:6/789: mknod db/d20/d2a/d5f/cef 0 2024-04-22T23:35:11.129 INFO:tasks.workunit.client.0.smithi107.stdout:6/790: write db/d20/d2c/d3b/fa8 [1018013,52613] 0 2024-04-22T23:35:11.131 INFO:tasks.workunit.client.0.smithi107.stdout:2/813: dread d1/d12/f14 [0,4194304] 0 2024-04-22T23:35:11.137 INFO:tasks.workunit.client.0.smithi107.stdout:7/714: dwrite d7/d21/d48/fb6 [0,4194304] 0 2024-04-22T23:35:11.142 INFO:tasks.workunit.client.0.smithi107.stdout:7/715: creat d7/d21/da4/d81/fef x:0 0 0 2024-04-22T23:35:11.144 INFO:tasks.workunit.client.0.smithi107.stdout:7/716: link d7/d21/d48/d5c/d9d/ddd/fe3 d7/d21/d7e/ddf/ff0 0 2024-04-22T23:35:11.145 INFO:tasks.workunit.client.0.smithi107.stdout:7/717: chown d7/d21/d48/daf/fe1 139 1 2024-04-22T23:35:11.146 INFO:tasks.workunit.client.0.smithi107.stdout:7/718: mknod d7/d21/d54/dcc/cf1 0 2024-04-22T23:35:11.146 INFO:tasks.workunit.client.0.smithi107.stdout:7/719: chown d7/d21/da4/d81 3 1 2024-04-22T23:35:11.147 INFO:tasks.workunit.client.0.smithi107.stdout:7/720: creat d7/d21/da4/dbf/ff2 x:0 0 0 2024-04-22T23:35:11.147 INFO:tasks.workunit.client.0.smithi107.stdout:7/721: write d7/d21/d48/f9c [965969,102830] 0 2024-04-22T23:35:11.149 INFO:tasks.workunit.client.0.smithi107.stdout:8/756: dwrite d0/d7/df/d33/d37/f80 [0,4194304] 0 2024-04-22T23:35:11.149 INFO:tasks.workunit.client.0.smithi107.stdout:7/722: dread d7/d21/da4/d66/fc1 [0,4194304] 0 2024-04-22T23:35:11.149 INFO:tasks.workunit.client.0.smithi107.stdout:7/723: dread - d7/d4c/da1/fcd zero size 2024-04-22T23:35:11.149 INFO:tasks.workunit.client.0.smithi107.stdout:7/724: dread - d7/d21/da4/d81/f8f zero size 2024-04-22T23:35:11.149 INFO:tasks.workunit.client.0.smithi107.stdout:7/725: stat d7/d21/d48/f6b 0 2024-04-22T23:35:11.151 INFO:tasks.workunit.client.0.smithi107.stdout:7/726: creat d7/d21/d7e/ff3 x:0 0 0 2024-04-22T23:35:11.153 INFO:tasks.workunit.client.0.smithi107.stdout:6/791: dwrite db/d20/d2a/d38/d61/d6f/da2/fdb [0,4194304] 0 2024-04-22T23:35:11.153 INFO:tasks.workunit.client.0.smithi107.stdout:2/814: dwrite d1/d63/dbb/fe9 [0,4194304] 0 2024-04-22T23:35:11.153 INFO:tasks.workunit.client.0.smithi107.stdout:2/815: creat d1/d12/d35/d102/d7e/dae/f10b x:0 0 0 2024-04-22T23:35:11.157 INFO:tasks.workunit.client.0.smithi107.stdout:8/757: rename d0/d7/df to d0/d73/db0/d107 0 2024-04-22T23:35:11.158 INFO:tasks.workunit.client.0.smithi107.stdout:7/727: mknod d7/d21/d54/cf4 0 2024-04-22T23:35:11.159 INFO:tasks.workunit.client.0.smithi107.stdout:6/792: creat db/d20/d2a/d35/d69/db3/ff0 x:0 0 0 2024-04-22T23:35:11.168 INFO:tasks.workunit.client.0.smithi107.stdout:6/793: read db/d20/d2a/d38/d61/d6f/da2/fdb [1289495,28533] 0 2024-04-22T23:35:11.171 INFO:tasks.workunit.client.0.smithi107.stdout:2/816: truncate d1/d12/d35/d102/f51 40358 0 2024-04-22T23:35:11.172 INFO:tasks.workunit.client.0.smithi107.stdout:8/758: mknod d0/d7/d8/d41/d53/d76/c108 0 2024-04-22T23:35:11.172 INFO:tasks.workunit.client.0.smithi107.stdout:8/759: creat d0/d7/d20/da2/da3/f109 x:0 0 0 2024-04-22T23:35:11.172 INFO:tasks.workunit.client.0.smithi107.stdout:8/760: truncate d0/d73/db0/d107/d3b/dd9/fe5 738565 0 2024-04-22T23:35:11.172 INFO:tasks.workunit.client.0.smithi107.stdout:8/761: chown d0/d7/d20/d2d/l90 17871 1 2024-04-22T23:35:11.172 INFO:tasks.workunit.client.0.smithi107.stdout:8/762: fsync d0/d85/d51/f101 0 2024-04-22T23:35:11.175 INFO:tasks.workunit.client.0.smithi107.stdout:2/817: symlink d1/d67/l10c 0 2024-04-22T23:35:11.178 INFO:tasks.workunit.client.0.smithi107.stdout:7/728: rmdir d7/d21/d48/d5c/d9d/d6e 39 2024-04-22T23:35:11.180 INFO:tasks.workunit.client.0.smithi107.stdout:2/818: symlink d1/d12/d35/d102/d87/l10d 0 2024-04-22T23:35:11.184 INFO:tasks.workunit.client.1.smithi157.stdout:7/851: creat d0/d9/d36/f102 x:0 0 0 2024-04-22T23:35:11.184 INFO:tasks.workunit.client.1.smithi157.stdout:7/852: getdents d0/d9/d42/d55/d80/de8 0 2024-04-22T23:35:11.184 INFO:tasks.workunit.client.1.smithi157.stdout:7/853: write d0/d9/d29/d33/d37/f9b [624739,53405] 0 2024-04-22T23:35:11.185 INFO:tasks.workunit.client.1.smithi157.stdout:7/854: creat d0/d9/db/d35/f103 x:0 0 0 2024-04-22T23:35:11.187 INFO:tasks.workunit.client.0.smithi107.stdout:6/794: dwrite db/f29 [8388608,4194304] 0 2024-04-22T23:35:11.187 INFO:tasks.workunit.client.0.smithi107.stdout:6/795: chown db/d20/c25 201 1 2024-04-22T23:35:11.189 INFO:tasks.workunit.client.0.smithi107.stdout:2/819: write d1/d12/d35/d102/fc5 [1807798,72776] 0 2024-04-22T23:35:11.190 INFO:tasks.workunit.client.0.smithi107.stdout:7/729: rmdir d7 39 2024-04-22T23:35:11.192 INFO:tasks.workunit.client.0.smithi107.stdout:2/820: mkdir d1/d12/d35/d102/d7e/da8/d10e 0 2024-04-22T23:35:11.193 INFO:tasks.workunit.client.0.smithi107.stdout:7/730: rename d7/d4c/da1/fd7 to d7/d21/d7e/ff5 0 2024-04-22T23:35:11.194 INFO:tasks.workunit.client.0.smithi107.stdout:7/731: write d7/d21/d54/f90 [2801022,47521] 0 2024-04-22T23:35:11.194 INFO:tasks.workunit.client.0.smithi107.stdout:6/796: truncate db/d20/d2a/f42 634816 0 2024-04-22T23:35:11.195 INFO:tasks.workunit.client.0.smithi107.stdout:2/821: creat d1/d2b/d88/f10f x:0 0 0 2024-04-22T23:35:11.197 INFO:tasks.workunit.client.0.smithi107.stdout:7/732: creat d7/d21/d48/d95/ff6 x:0 0 0 2024-04-22T23:35:11.197 INFO:tasks.workunit.client.0.smithi107.stdout:7/733: dread - d7/d21/d48/daf/fdc zero size 2024-04-22T23:35:11.198 INFO:tasks.workunit.client.1.smithi157.stdout:0/843: unlink d17/d2b/d4f/dc5/dcc/de9/df6/f95 0 2024-04-22T23:35:11.198 INFO:tasks.workunit.client.0.smithi107.stdout:7/734: write d7/d1b/fa2 [147543,25483] 0 2024-04-22T23:35:11.198 INFO:tasks.workunit.client.0.smithi107.stdout:7/735: chown d7/d1b/le9 113499853 1 2024-04-22T23:35:11.198 INFO:tasks.workunit.client.0.smithi107.stdout:4/777: sync 2024-04-22T23:35:11.198 INFO:tasks.workunit.client.0.smithi107.stdout:9/719: sync 2024-04-22T23:35:11.199 INFO:tasks.workunit.client.0.smithi107.stdout:6/797: creat db/d20/d2c/db5/ff1 x:0 0 0 2024-04-22T23:35:11.199 INFO:tasks.workunit.client.0.smithi107.stdout:6/798: readlink db/la3 0 2024-04-22T23:35:11.199 INFO:tasks.workunit.client.0.smithi107.stdout:6/799: readlink db/d20/d2a/d38/d61/d6f/da2/d84/ld5 0 2024-04-22T23:35:11.199 INFO:tasks.workunit.client.0.smithi107.stdout:6/800: chown db/d20/d2a/d35/d69/db3 14400550 1 2024-04-22T23:35:11.199 INFO:tasks.workunit.client.0.smithi107.stdout:6/801: creat db/d20/d2a/d38/d4b/ff2 x:0 0 0 2024-04-22T23:35:11.201 INFO:tasks.workunit.client.0.smithi107.stdout:2/822: mkdir d1/d12/d35/d102/d7e/d110 0 2024-04-22T23:35:11.201 INFO:tasks.workunit.client.0.smithi107.stdout:2/823: stat d1/d12/fa4 0 2024-04-22T23:35:11.202 INFO:tasks.workunit.client.0.smithi107.stdout:8/763: dwrite d0/d7/d8/d41/dec/f5b [0,4194304] 0 2024-04-22T23:35:11.205 INFO:tasks.workunit.client.0.smithi107.stdout:4/778: mkdir d1/d7/d63/d78/d111 0 2024-04-22T23:35:11.207 INFO:tasks.workunit.client.0.smithi107.stdout:9/720: symlink d5/d29/d7b/d91/dde/dc9/lfb 0 2024-04-22T23:35:11.207 INFO:tasks.workunit.client.0.smithi107.stdout:9/721: read d5/d14/d28/f59 [1348336,77386] 0 2024-04-22T23:35:11.211 INFO:tasks.workunit.client.0.smithi107.stdout:2/824: unlink d1/d2b/d3d/d57/c9b 0 2024-04-22T23:35:11.215 INFO:tasks.workunit.client.1.smithi157.stdout:0/844: dwrite d17/d2b/d52/dd1/fd4 [0,4194304] 0 2024-04-22T23:35:11.216 INFO:tasks.workunit.client.0.smithi107.stdout:7/736: mkdir d7/df7 0 2024-04-22T23:35:11.217 INFO:tasks.workunit.client.0.smithi107.stdout:8/764: mknod d0/d73/db0/d107/d3b/d7b/c10a 0 2024-04-22T23:35:11.218 INFO:tasks.workunit.client.0.smithi107.stdout:8/765: read d0/d73/db0/d107/f5c [3185457,92838] 0 2024-04-22T23:35:11.219 INFO:tasks.workunit.client.0.smithi107.stdout:9/722: rename d5/dd9 to d5/d7/d27/dfc 0 2024-04-22T23:35:11.219 INFO:tasks.workunit.client.1.smithi157.stdout:0/845: creat d17/d2b/d4f/d5a/dd2/de4/f11a x:0 0 0 2024-04-22T23:35:11.221 INFO:tasks.workunit.client.0.smithi107.stdout:6/802: dread db/d20/d2a/f42 [0,4194304] 0 2024-04-22T23:35:11.221 INFO:tasks.workunit.client.0.smithi107.stdout:2/825: unlink d1/d12/d35/d102/d7e/dae/fea 0 2024-04-22T23:35:11.221 INFO:tasks.workunit.client.1.smithi157.stdout:0/846: mknod d17/d23/d34/dee/c11b 0 2024-04-22T23:35:11.222 INFO:tasks.workunit.client.1.smithi157.stdout:0/847: write d17/d2b/d4f/d5a/dd2/de4/f11a [93746,2714] 0 2024-04-22T23:35:11.222 INFO:tasks.workunit.client.1.smithi157.stdout:0/848: dread - d17/d38/fb9 zero size 2024-04-22T23:35:11.224 INFO:tasks.workunit.client.0.smithi107.stdout:4/779: dwrite d1/d21/d52/f97 [0,4194304] 0 2024-04-22T23:35:11.224 INFO:tasks.workunit.client.0.smithi107.stdout:4/780: fdatasync d1/d7/d5b/dbf/da8/db9/ff6 0 2024-04-22T23:35:11.229 INFO:tasks.workunit.client.0.smithi107.stdout:7/737: mknod d7/d21/da4/d66/cf8 0 2024-04-22T23:35:11.229 INFO:tasks.workunit.client.0.smithi107.stdout:7/738: stat d7/l27 0 2024-04-22T23:35:11.229 INFO:tasks.workunit.client.0.smithi107.stdout:7/739: chown d7/d21/da4/cc8 52439 1 2024-04-22T23:35:11.229 INFO:tasks.workunit.client.0.smithi107.stdout:7/740: stat d7/d21/d48/daf/fdc 0 2024-04-22T23:35:11.231 INFO:tasks.workunit.client.0.smithi107.stdout:8/766: mknod d0/d7/d87/dc8/c10b 0 2024-04-22T23:35:11.231 INFO:tasks.workunit.client.0.smithi107.stdout:8/767: chown d0/d3a/d46/d59/d78/c97 196459244 1 2024-04-22T23:35:11.233 INFO:tasks.workunit.client.0.smithi107.stdout:9/723: mknod d5/d14/d2e/d4f/d83/cfd 0 2024-04-22T23:35:11.234 INFO:tasks.workunit.client.0.smithi107.stdout:2/826: dread d1/d67/f33 [0,4194304] 0 2024-04-22T23:35:11.234 INFO:tasks.workunit.client.0.smithi107.stdout:2/827: write d1/d12/d35/d48/d13/ff4 [816280,57936] 0 2024-04-22T23:35:11.235 INFO:tasks.workunit.client.0.smithi107.stdout:6/803: symlink db/d20/d2a/d38/d4b/lf3 0 2024-04-22T23:35:11.235 INFO:tasks.workunit.client.0.smithi107.stdout:6/804: write db/d20/d2a/d38/d61/d6f/da2/d84/fd6 [317321,129765] 0 2024-04-22T23:35:11.243 INFO:tasks.workunit.client.1.smithi157.stdout:0/849: dwrite d17/d2b/f47 [0,4194304] 0 2024-04-22T23:35:11.243 INFO:tasks.workunit.client.1.smithi157.stdout:0/850: fsync d17/d23/d34/d67/d7f/d69/dce/fe5 0 2024-04-22T23:35:11.243 INFO:tasks.workunit.client.0.smithi107.stdout:4/781: link d1/d21/d29/d2e/d88/d99/dd2/fcf d1/d21/d29/d2e/d88/d99/dd2/daf/f112 0 2024-04-22T23:35:11.244 INFO:tasks.workunit.client.1.smithi157.stdout:0/851: write d17/d2b/d4f/d9b/fa4 [501229,92618] 0 2024-04-22T23:35:11.245 INFO:tasks.workunit.client.0.smithi107.stdout:7/741: link d7/d21/da4/d81/f8f d7/d21/d54/ff9 0 2024-04-22T23:35:11.245 INFO:tasks.workunit.client.0.smithi107.stdout:7/742: chown d7/f13 1771598 1 2024-04-22T23:35:11.246 INFO:tasks.workunit.client.1.smithi157.stdout:0/852: mkdir d17/d23/d34/d67/d7f/d69/d11c 0 2024-04-22T23:35:11.247 INFO:tasks.workunit.client.0.smithi107.stdout:9/724: rename d5/d7/d17/fec to d5/d7/d17/d1b/dcb/ffe 0 2024-04-22T23:35:11.248 INFO:tasks.workunit.client.1.smithi157.stdout:0/853: creat d17/d38/f11d x:0 0 0 2024-04-22T23:35:11.248 INFO:tasks.workunit.client.0.smithi107.stdout:2/828: truncate d1/d12/d35/d54/ffe 652334 0 2024-04-22T23:35:11.249 INFO:tasks.workunit.client.0.smithi107.stdout:2/829: creat d1/d2b/f111 x:0 0 0 2024-04-22T23:35:11.251 INFO:tasks.workunit.client.0.smithi107.stdout:6/805: mknod db/d20/d2c/d3b/cf4 0 2024-04-22T23:35:11.253 INFO:tasks.workunit.client.0.smithi107.stdout:4/782: mkdir d1/d7/d63/d113 0 2024-04-22T23:35:11.256 INFO:tasks.workunit.client.0.smithi107.stdout:9/725: mkdir d5/d7/d27/d46/d4b/dff 0 2024-04-22T23:35:11.262 INFO:tasks.workunit.client.0.smithi107.stdout:2/830: mknod d1/d12/d35/d102/dba/df1/c112 0 2024-04-22T23:35:11.262 INFO:tasks.workunit.client.0.smithi107.stdout:8/768: dwrite d0/d7/f103 [0,4194304] 0 2024-04-22T23:35:11.262 INFO:tasks.workunit.client.0.smithi107.stdout:6/806: creat db/d20/d2c/db5/dd2/ff5 x:0 0 0 2024-04-22T23:35:11.262 INFO:tasks.workunit.client.0.smithi107.stdout:6/807: chown f6 905790 1 2024-04-22T23:35:11.262 INFO:tasks.workunit.client.0.smithi107.stdout:6/808: readlink db/d20/l73 0 2024-04-22T23:35:11.262 INFO:tasks.workunit.client.0.smithi107.stdout:6/809: dread - db/d20/d2c/d3b/feb zero size 2024-04-22T23:35:11.265 INFO:tasks.workunit.client.0.smithi107.stdout:4/783: creat d1/d7/d5b/dbf/d27/f114 x:0 0 0 2024-04-22T23:35:11.267 INFO:tasks.workunit.client.0.smithi107.stdout:7/743: link d7/d21/d7e/d94/lc6 d7/d21/lfa 0 2024-04-22T23:35:11.270 INFO:tasks.workunit.client.0.smithi107.stdout:2/831: chown d1/d2b/d3d/fb9 37799263 1 2024-04-22T23:35:11.274 INFO:tasks.workunit.client.1.smithi157.stdout:8/685: sync 2024-04-22T23:35:11.274 INFO:tasks.workunit.client.1.smithi157.stdout:8/686: fsync d2/d6/d21/d5f/f56 0 2024-04-22T23:35:11.275 INFO:tasks.workunit.client.1.smithi157.stdout:1/970: sync 2024-04-22T23:35:11.275 INFO:tasks.workunit.client.1.smithi157.stdout:1/971: chown d4/d17/c125 15415232 1 2024-04-22T23:35:11.276 INFO:tasks.workunit.client.1.smithi157.stdout:1/972: chown d4/d17/d34/d75/ca8 1456921 1 2024-04-22T23:35:11.276 INFO:tasks.workunit.client.1.smithi157.stdout:1/973: truncate d4/fd1 713144 0 2024-04-22T23:35:11.276 INFO:tasks.workunit.client.1.smithi157.stdout:1/974: fdatasync d4/d17/d34/d75/f12b 0 2024-04-22T23:35:11.276 INFO:tasks.workunit.client.1.smithi157.stdout:1/975: read d4/f23 [7897102,3608] 0 2024-04-22T23:35:11.276 INFO:tasks.workunit.client.1.smithi157.stdout:3/966: sync 2024-04-22T23:35:11.276 INFO:tasks.workunit.client.1.smithi157.stdout:5/937: sync 2024-04-22T23:35:11.277 INFO:tasks.workunit.client.0.smithi107.stdout:8/769: mknod d0/d73/db0/d107/d33/d37/c10c 0 2024-04-22T23:35:11.277 INFO:tasks.workunit.client.1.smithi157.stdout:1/976: rmdir d4/db/d3d 39 2024-04-22T23:35:11.277 INFO:tasks.workunit.client.1.smithi157.stdout:5/938: mknod d1/d5/d7/d10/d14f/dc6/d148/c152 0 2024-04-22T23:35:11.282 INFO:tasks.workunit.client.0.smithi107.stdout:4/784: link d1/d21/d85/f9c d1/d7/d5b/dbf/d27/f115 0 2024-04-22T23:35:11.283 INFO:tasks.workunit.client.0.smithi107.stdout:7/744: creat d7/d21/d48/d5c/d9d/d6e/db2/ffb x:0 0 0 2024-04-22T23:35:11.283 INFO:tasks.workunit.client.0.smithi107.stdout:7/745: readlink d7/d21/d48/d95/ld8 0 2024-04-22T23:35:11.283 INFO:tasks.workunit.client.0.smithi107.stdout:7/746: write d7/d1b/f53 [4517909,80446] 0 2024-04-22T23:35:11.285 INFO:tasks.workunit.client.1.smithi157.stdout:3/967: write d0/d83/dbc/d12e/d34/d3d/df6/dee/d9e/fe2 [473512,24905] 0 2024-04-22T23:35:11.286 INFO:tasks.workunit.client.0.smithi107.stdout:2/832: mkdir d1/d12/d35/d102/dba/df1/d113 0 2024-04-22T23:35:11.286 INFO:tasks.workunit.client.1.smithi157.stdout:9/842: rename d4/df/l1c to d4/l102 0 2024-04-22T23:35:11.286 INFO:tasks.workunit.client.1.smithi157.stdout:0/854: dwrite d17/d2b/d71/fac [0,4194304] 0 2024-04-22T23:35:11.289 INFO:tasks.workunit.client.0.smithi107.stdout:4/785: getdents d1/d21/d29/d2e/d88/d99/dd2/daf 0 2024-04-22T23:35:11.289 INFO:tasks.workunit.client.0.smithi107.stdout:4/786: dread - d1/d7/d63/d78/d80/fff zero size 2024-04-22T23:35:11.289 INFO:tasks.workunit.client.1.smithi157.stdout:3/968: creat d0/d83/dbc/d12e/d34/d3d/df6/dee/d9e/d141/f14c x:0 0 0 2024-04-22T23:35:11.290 INFO:tasks.workunit.client.1.smithi157.stdout:0/855: creat d17/d2b/d4f/dc5/dcc/de9/f11e x:0 0 0 2024-04-22T23:35:11.290 INFO:tasks.workunit.client.1.smithi157.stdout:0/856: fsync d17/d2b/d5b/fc6 0 2024-04-22T23:35:11.291 INFO:tasks.workunit.client.1.smithi157.stdout:7/855: rename d0/d9/d29/d33/d7c/fd7 to d0/d9/d29/d5e/f104 0 2024-04-22T23:35:11.292 INFO:tasks.workunit.client.0.smithi107.stdout:2/833: rename d1/d12/d35/lc3 to d1/d63/de6/l114 0 2024-04-22T23:35:11.294 INFO:tasks.workunit.client.1.smithi157.stdout:7/856: unlink d0/d9/d10/fd8 0 2024-04-22T23:35:11.295 INFO:tasks.workunit.client.1.smithi157.stdout:0/857: mknod d17/d23/d34/c11f 0 2024-04-22T23:35:11.295 INFO:tasks.workunit.client.1.smithi157.stdout:0/858: readlink d17/lb1 0 2024-04-22T23:35:11.296 INFO:tasks.workunit.client.0.smithi107.stdout:9/726: dwrite d5/d7/d27/d46/f88 [4194304,4194304] 0 2024-04-22T23:35:11.298 INFO:tasks.workunit.client.1.smithi157.stdout:7/857: unlink d0/d9/d29/d5e/d9e/fb3 0 2024-04-22T23:35:11.298 INFO:tasks.workunit.client.1.smithi157.stdout:7/858: write d0/f2 [3586543,85337] 0 2024-04-22T23:35:11.298 INFO:tasks.workunit.client.1.smithi157.stdout:7/859: write d0/d9/db/df/d68/fca [434219,31764] 0 2024-04-22T23:35:11.299 INFO:tasks.workunit.client.1.smithi157.stdout:0/859: dread d17/f25 [0,4194304] 0 2024-04-22T23:35:11.299 INFO:tasks.workunit.client.1.smithi157.stdout:0/860: creat d17/d38/f120 x:0 0 0 2024-04-22T23:35:11.300 INFO:tasks.workunit.client.0.smithi107.stdout:6/810: dwrite db/f21 [0,4194304] 0 2024-04-22T23:35:11.301 INFO:tasks.workunit.client.0.smithi107.stdout:6/811: fsync db/d20/d2a/d38/d61/d6f/da2/d84/faf 0 2024-04-22T23:35:11.301 INFO:tasks.workunit.client.0.smithi107.stdout:6/812: stat db/d20/d2a/l86 0 2024-04-22T23:35:11.301 INFO:tasks.workunit.client.0.smithi107.stdout:4/787: rename d1/d7/d5b/de3/f103 to d1/d21/d29/d2e/d88/d99/dd2/daf/dc1/f116 0 2024-04-22T23:35:11.302 INFO:tasks.workunit.client.1.smithi157.stdout:3/969: dread d0/d83/dbc/d12e/d34/d3d/d57/d60/d6b/d97/fb5 [0,4194304] 0 2024-04-22T23:35:11.303 INFO:tasks.workunit.client.0.smithi107.stdout:8/770: dwrite d0/d3a/d46/f70 [0,4194304] 0 2024-04-22T23:35:11.304 INFO:tasks.workunit.client.0.smithi107.stdout:8/771: dread - d0/d73/db0/d107/d33/d9b/fae zero size 2024-04-22T23:35:11.304 INFO:tasks.workunit.client.1.smithi157.stdout:8/687: dwrite d2/d4/dbd/f9d [0,4194304] 0 2024-04-22T23:35:11.305 INFO:tasks.workunit.client.1.smithi157.stdout:3/970: mkdir d0/d83/dbc/d12e/d34/d3d/d57/d114/d14d 0 2024-04-22T23:35:11.308 INFO:tasks.workunit.client.0.smithi107.stdout:4/788: symlink d1/d7/d5b/dbf/da8/db9/l117 0 2024-04-22T23:35:11.309 INFO:tasks.workunit.client.0.smithi107.stdout:9/727: rename d5/f13 to d5/d7/d17/f100 0 2024-04-22T23:35:11.309 INFO:tasks.workunit.client.0.smithi107.stdout:2/834: dread d1/d12/d35/d48/d13/f68 [0,4194304] 0 2024-04-22T23:35:11.310 INFO:tasks.workunit.client.0.smithi107.stdout:2/835: chown d1/d12/d35/d102/dba/df1/d113 0 1 2024-04-22T23:35:11.310 INFO:tasks.workunit.client.0.smithi107.stdout:2/836: truncate d1/d2b/d59/fc4 1116016 0 2024-04-22T23:35:11.310 INFO:tasks.workunit.client.1.smithi157.stdout:1/977: unlink d4/d17/d1d/d33/d41/la7 0 2024-04-22T23:35:11.312 INFO:tasks.workunit.client.1.smithi157.stdout:8/688: creat d2/d4/dda/fef x:0 0 0 2024-04-22T23:35:11.312 INFO:tasks.workunit.client.1.smithi157.stdout:9/843: dwrite d4/d15/d23/f69 [0,4194304] 0 2024-04-22T23:35:11.312 INFO:tasks.workunit.client.1.smithi157.stdout:9/844: chown d4/d15/d23/d3f/l48 30 1 2024-04-22T23:35:11.315 INFO:tasks.workunit.client.1.smithi157.stdout:3/971: rename d0/d83/dbc/d12e/d34/d47/ded/c138 to d0/d83/dbc/c14e 0 2024-04-22T23:35:11.315 INFO:tasks.workunit.client.1.smithi157.stdout:3/972: write d0/d83/dbc/d12e/d34/d39/f76 [649222,11055] 0 2024-04-22T23:35:11.316 INFO:tasks.workunit.client.0.smithi107.stdout:9/728: symlink d5/d7/d27/d46/d4b/dff/l101 0 2024-04-22T23:35:11.317 INFO:tasks.workunit.client.1.smithi157.stdout:0/861: dread d17/d2b/d52/fa2 [0,4194304] 0 2024-04-22T23:35:11.317 INFO:tasks.workunit.client.1.smithi157.stdout:0/862: chown fc 0 1 2024-04-22T23:35:11.317 INFO:tasks.workunit.client.0.smithi107.stdout:8/772: write d0/d7/d8/d41/dec/f5b [952257,52940] 0 2024-04-22T23:35:11.318 INFO:tasks.workunit.client.0.smithi107.stdout:8/773: chown d0/d73/db0/d107/d3b/dd9/cea 1 1 2024-04-22T23:35:11.319 INFO:tasks.workunit.client.0.smithi107.stdout:2/837: rename d1/d2b/d59/f6b to d1/d2b/d59/f115 0 2024-04-22T23:35:11.320 INFO:tasks.workunit.client.1.smithi157.stdout:3/973: creat d0/d83/dbc/d12e/d144/f14f x:0 0 0 2024-04-22T23:35:11.321 INFO:tasks.workunit.client.1.smithi157.stdout:3/974: symlink d0/d83/dbc/d12e/d34/d3d/d57/d60/d6b/d129/l150 0 2024-04-22T23:35:11.323 INFO:tasks.workunit.client.0.smithi107.stdout:6/813: write db/d20/d2a/d38/d61/d6f/da2/fc3 [3169861,23] 0 2024-04-22T23:35:11.323 INFO:tasks.workunit.client.0.smithi107.stdout:6/814: truncate db/f1f 5053773 0 2024-04-22T23:35:11.323 INFO:tasks.workunit.client.0.smithi107.stdout:6/815: chown db/d20/d2c/d3b/feb 445913792 1 2024-04-22T23:35:11.329 INFO:tasks.workunit.client.1.smithi157.stdout:7/860: write d0/d9/db/df/f2b [1065689,18074] 0 2024-04-22T23:35:11.330 INFO:tasks.workunit.client.0.smithi107.stdout:8/774: getdents d0/d73/db0/d107/d33/d37 0 2024-04-22T23:35:11.330 INFO:tasks.workunit.client.0.smithi107.stdout:7/747: dwrite d7/f2f [0,4194304] 0 2024-04-22T23:35:11.332 INFO:tasks.workunit.client.1.smithi157.stdout:9/845: dread d4/d15/d23/d27/f4f [0,4194304] 0 2024-04-22T23:35:11.332 INFO:tasks.workunit.client.1.smithi157.stdout:9/846: stat d4/df/fb 0 2024-04-22T23:35:11.333 INFO:tasks.workunit.client.0.smithi107.stdout:2/838: mknod d1/d12/d35/d102/dba/df1/d113/c116 0 2024-04-22T23:35:11.336 INFO:tasks.workunit.client.0.smithi107.stdout:6/816: mknod db/cf6 0 2024-04-22T23:35:11.337 INFO:tasks.workunit.client.1.smithi157.stdout:9/847: mkdir d4/d15/d55/d103 0 2024-04-22T23:35:11.338 INFO:tasks.workunit.client.0.smithi107.stdout:8/775: truncate d0/d73/db0/d107/d33/faa 1041380 0 2024-04-22T23:35:11.339 INFO:tasks.workunit.client.0.smithi107.stdout:8/776: stat d0/d73/db0/d107/d3b/f88 0 2024-04-22T23:35:11.341 INFO:tasks.workunit.client.1.smithi157.stdout:1/978: dwrite d4/d17/d34/d75/dcc/fd7 [0,4194304] 0 2024-04-22T23:35:11.345 INFO:tasks.workunit.client.1.smithi157.stdout:1/979: rename d4/d17/d34/d3b/d52/d65/d10b/l117 to d4/db/d3d/dc1/d100/d155/l157 0 2024-04-22T23:35:11.345 INFO:tasks.workunit.client.1.smithi157.stdout:1/980: fdatasync d4/d17/d34/d3b/d52/d65/dbc/d149/f14f 0 2024-04-22T23:35:11.346 INFO:tasks.workunit.client.0.smithi107.stdout:7/748: symlink d7/d21/d48/d95/lfc 0 2024-04-22T23:35:11.348 INFO:tasks.workunit.client.0.smithi107.stdout:5/794: sync 2024-04-22T23:35:11.348 INFO:tasks.workunit.client.0.smithi107.stdout:5/795: chown d9/db/dc1/ce8 61896 1 2024-04-22T23:35:11.349 INFO:tasks.workunit.client.0.smithi107.stdout:6/817: unlink db/cc7 0 2024-04-22T23:35:11.349 INFO:tasks.workunit.client.0.smithi107.stdout:6/818: write db/d20/d58/fba [758715,82338] 0 2024-04-22T23:35:11.349 INFO:tasks.workunit.client.0.smithi107.stdout:6/819: creat db/d20/d2c/d3b/d40/d9e/ff7 x:0 0 0 2024-04-22T23:35:11.350 INFO:tasks.workunit.client.0.smithi107.stdout:8/777: mknod d0/d7/d20/da2/dc3/c10d 0 2024-04-22T23:35:11.352 INFO:tasks.workunit.client.0.smithi107.stdout:7/749: mknod d7/d21/d7e/deb/cfd 0 2024-04-22T23:35:11.356 INFO:tasks.workunit.client.0.smithi107.stdout:6/820: creat db/d20/d2a/d38/d61/d6f/d9b/ff8 x:0 0 0 2024-04-22T23:35:11.357 INFO:tasks.workunit.client.0.smithi107.stdout:8/778: creat d0/d7/d87/f10e x:0 0 0 2024-04-22T23:35:11.357 INFO:tasks.workunit.client.0.smithi107.stdout:8/779: stat d0/d73/db0/d107/d3b/d7b/c10a 0 2024-04-22T23:35:11.358 INFO:tasks.workunit.client.0.smithi107.stdout:5/796: rmdir d9/d26 39 2024-04-22T23:35:11.361 INFO:tasks.workunit.client.0.smithi107.stdout:8/780: getdents d0/d7/d8/d41 0 2024-04-22T23:35:11.362 INFO:tasks.workunit.client.0.smithi107.stdout:8/781: fdatasync d0/d73/db0/d107/f5c 0 2024-04-22T23:35:11.364 INFO:tasks.workunit.client.0.smithi107.stdout:8/782: creat d0/d73/db0/d107/d33/f10f x:0 0 0 2024-04-22T23:35:11.364 INFO:tasks.workunit.client.1.smithi157.stdout:8/689: dwrite d2/d6/fe0 [0,4194304] 0 2024-04-22T23:35:11.365 INFO:tasks.workunit.client.1.smithi157.stdout:0/863: dwrite fe [0,4194304] 0 2024-04-22T23:35:11.365 INFO:tasks.workunit.client.1.smithi157.stdout:0/864: chown d17/d23/d34/d67/d7f/c117 32889658 1 2024-04-22T23:35:11.365 INFO:tasks.workunit.client.1.smithi157.stdout:0/865: chown d17/d2b/d5b/cfa 1793245369 1 2024-04-22T23:35:11.367 INFO:tasks.workunit.client.1.smithi157.stdout:9/848: dwrite d4/df/d61/fc1 [0,4194304] 0 2024-04-22T23:35:11.367 INFO:tasks.workunit.client.0.smithi107.stdout:2/839: write d1/d12/f26 [6634599,55421] 0 2024-04-22T23:35:11.367 INFO:tasks.workunit.client.0.smithi107.stdout:6/821: dread db/d20/d2a/d35/d69/d79/f95 [0,4194304] 0 2024-04-22T23:35:11.368 INFO:tasks.workunit.client.1.smithi157.stdout:8/690: mkdir d2/d34/d6a/d77/df0 0 2024-04-22T23:35:11.368 INFO:tasks.workunit.client.1.smithi157.stdout:7/861: dwrite d0/d9/d42/d55/d80/d82/fe0 [0,4194304] 0 2024-04-22T23:35:11.368 INFO:tasks.workunit.client.1.smithi157.stdout:7/862: write d0/d9/d29/d49/d4a/fe5 [402336,73089] 0 2024-04-22T23:35:11.370 INFO:tasks.workunit.client.1.smithi157.stdout:0/866: rmdir d17/d2b/d5b 39 2024-04-22T23:35:11.370 INFO:tasks.workunit.client.1.smithi157.stdout:0/867: dread - d17/d23/d34/dee/da8/fe7 zero size 2024-04-22T23:35:11.371 INFO:tasks.workunit.client.0.smithi107.stdout:6/822: link db/d20/d2a/cc4 db/d20/d2a/d38/cf9 0 2024-04-22T23:35:11.374 INFO:tasks.workunit.client.1.smithi157.stdout:9/849: dread d4/d15/d23/fdb [0,4194304] 0 2024-04-22T23:35:11.374 INFO:tasks.workunit.client.1.smithi157.stdout:9/850: chown d4/df/d57/db7/df9/cff 769955 1 2024-04-22T23:35:11.374 INFO:tasks.workunit.client.1.smithi157.stdout:9/851: stat d4/d15/d55/d103 0 2024-04-22T23:35:11.374 INFO:tasks.workunit.client.1.smithi157.stdout:3/975: dwrite d0/d83/dbc/d12e/d34/d3d/f42 [0,4194304] 0 2024-04-22T23:35:11.377 INFO:tasks.workunit.client.0.smithi107.stdout:9/729: dwrite d5/f47 [0,4194304] 0 2024-04-22T23:35:11.377 INFO:tasks.workunit.client.0.smithi107.stdout:7/750: dread d7/f76 [0,4194304] 0 2024-04-22T23:35:11.377 INFO:tasks.workunit.client.0.smithi107.stdout:3/721: sync 2024-04-22T23:35:11.377 INFO:tasks.workunit.client.0.smithi107.stdout:0/732: sync 2024-04-22T23:35:11.378 INFO:tasks.workunit.client.0.smithi107.stdout:1/687: sync 2024-04-22T23:35:11.378 INFO:tasks.workunit.client.0.smithi107.stdout:1/688: write d7/d19/d20/d39/d53/d43/f7a [4593798,16359] 0 2024-04-22T23:35:11.378 INFO:tasks.workunit.client.0.smithi107.stdout:4/789: dwrite d1/d1b/f49 [4194304,4194304] 0 2024-04-22T23:35:11.378 INFO:tasks.workunit.client.0.smithi107.stdout:4/790: truncate d1/d21/ffe 92295 0 2024-04-22T23:35:11.378 INFO:tasks.workunit.client.0.smithi107.stdout:4/791: stat d1/d21/d29/lce 0 2024-04-22T23:35:11.378 INFO:tasks.workunit.client.0.smithi107.stdout:4/792: write d1/d7/d63/d78/db4/fc0 [349059,5868] 0 2024-04-22T23:35:11.378 INFO:tasks.workunit.client.0.smithi107.stdout:4/793: write d1/d7/d63/d78/da4/fd0 [408986,65831] 0 2024-04-22T23:35:11.378 INFO:tasks.workunit.client.0.smithi107.stdout:6/823: getdents db/d20/d2a/d38/d59 0 2024-04-22T23:35:11.379 INFO:tasks.workunit.client.1.smithi157.stdout:3/976: rename d0/d83/dbc/d12e/d13/d13b/d6c/cb0 to d0/d83/dbc/d12e/d34/d3d/d57/d60/d6b/d97/c151 0 2024-04-22T23:35:11.379 INFO:tasks.workunit.client.1.smithi157.stdout:3/977: write d0/d83/dbc/d12e/d34/d47/ded/f12b [963883,79504] 0 2024-04-22T23:35:11.379 INFO:tasks.workunit.client.1.smithi157.stdout:3/978: write d0/d83/dbc/d12e/d34/d3d/d57/d5c/ffd [226293,95992] 0 2024-04-22T23:35:11.380 INFO:tasks.workunit.client.1.smithi157.stdout:3/979: creat d0/d83/dbc/d12e/d34/d47/ded/f152 x:0 0 0 2024-04-22T23:35:11.384 INFO:tasks.workunit.client.1.smithi157.stdout:1/981: dwrite d4/db/d144/f14a [0,4194304] 0 2024-04-22T23:35:11.385 INFO:tasks.workunit.client.1.smithi157.stdout:8/691: dread d2/d6/d21/d5f/d11/d13/f7d [0,4194304] 0 2024-04-22T23:35:11.385 INFO:tasks.workunit.client.1.smithi157.stdout:8/692: chown d2/d6/cce 3 1 2024-04-22T23:35:11.385 INFO:tasks.workunit.client.1.smithi157.stdout:8/693: write d2/d6/d21/d5f/d11/d13/f7d [3982092,53028] 0 2024-04-22T23:35:11.385 INFO:tasks.workunit.client.1.smithi157.stdout:8/694: readlink d2/d6/laf 0 2024-04-22T23:35:11.385 INFO:tasks.workunit.client.1.smithi157.stdout:0/868: dread d17/d2b/d71/fac [0,4194304] 0 2024-04-22T23:35:11.385 INFO:tasks.workunit.client.0.smithi107.stdout:5/797: write d9/db/d42/d75/fb1 [7081336,128252] 0 2024-04-22T23:35:11.386 INFO:tasks.workunit.client.0.smithi107.stdout:5/798: write d9/db/de/d5e/fe0 [730828,122981] 0 2024-04-22T23:35:11.386 INFO:tasks.workunit.client.0.smithi107.stdout:5/799: write d9/db/de/faf [348554,111949] 0 2024-04-22T23:35:11.386 INFO:tasks.workunit.client.0.smithi107.stdout:5/800: write d9/db/de/f9a [985999,116517] 0 2024-04-22T23:35:11.386 INFO:tasks.workunit.client.1.smithi157.stdout:3/980: creat d0/d83/dbc/d12e/d34/f153 x:0 0 0 2024-04-22T23:35:11.387 INFO:tasks.workunit.client.1.smithi157.stdout:8/695: read d2/d34/d6a/f76 [336893,118797] 0 2024-04-22T23:35:11.387 INFO:tasks.workunit.client.1.smithi157.stdout:8/696: fdatasync d2/d4/dbd/d97/fa1 0 2024-04-22T23:35:11.388 INFO:tasks.workunit.client.0.smithi107.stdout:3/722: unlink db/d21/d27/d15/d29/d2a/d45/ddd/d25/c70 0 2024-04-22T23:35:11.391 INFO:tasks.workunit.client.0.smithi107.stdout:7/751: creat d7/d21/d48/d5c/d9d/db1/ffe x:0 0 0 2024-04-22T23:35:11.392 INFO:tasks.workunit.client.0.smithi107.stdout:0/733: mknod d1/d27/d20/d4a/cf4 0 2024-04-22T23:35:11.393 INFO:tasks.workunit.client.1.smithi157.stdout:0/869: mkdir d17/d23/d34/d67/d104/d121 0 2024-04-22T23:35:11.394 INFO:tasks.workunit.client.1.smithi157.stdout:3/981: mknod d0/d83/df9/c154 0 2024-04-22T23:35:11.394 INFO:tasks.workunit.client.1.smithi157.stdout:3/982: write d0/d83/dbc/d12e/d127/f11a [883710,123746] 0 2024-04-22T23:35:11.396 INFO:tasks.workunit.client.0.smithi107.stdout:6/824: truncate db/f29 4112540 0 2024-04-22T23:35:11.396 INFO:tasks.workunit.client.0.smithi107.stdout:6/825: write db/d20/d2c/f44 [9277272,76129] 0 2024-04-22T23:35:11.398 INFO:tasks.workunit.client.1.smithi157.stdout:8/697: creat d2/d6/d21/d5f/d11/ff1 x:0 0 0 2024-04-22T23:35:11.399 INFO:tasks.workunit.client.0.smithi107.stdout:3/723: symlink db/d21/d27/d15/d24/d66/da5/lf5 0 2024-04-22T23:35:11.399 INFO:tasks.workunit.client.0.smithi107.stdout:3/724: read - db/d21/d27/d15/db8/f82 zero size 2024-04-22T23:35:11.402 INFO:tasks.workunit.client.1.smithi157.stdout:0/870: creat d17/d23/d34/d67/d7f/d69/d103/f122 x:0 0 0 2024-04-22T23:35:11.403 INFO:tasks.workunit.client.0.smithi107.stdout:7/752: symlink d7/d4c/da1/lff 0 2024-04-22T23:35:11.406 INFO:tasks.workunit.client.1.smithi157.stdout:1/982: write d4/d17/d34/d3b/d52/fc9 [1134538,77312] 0 2024-04-22T23:35:11.408 INFO:tasks.workunit.client.0.smithi107.stdout:0/734: symlink d1/d84/d99/lf5 0 2024-04-22T23:35:11.408 INFO:tasks.workunit.client.0.smithi107.stdout:1/689: truncate d7/d19/d20/d39/d53/d43/fa8 581431 0 2024-04-22T23:35:11.408 INFO:tasks.workunit.client.0.smithi107.stdout:0/735: fsync d1/d27/d8a/fcb 0 2024-04-22T23:35:11.409 INFO:tasks.workunit.client.0.smithi107.stdout:6/826: link db/d20/d2a/d38/c4c db/d20/d2a/d38/d4b/d9d/cfa 0 2024-04-22T23:35:11.409 INFO:tasks.workunit.client.0.smithi107.stdout:0/736: write d1/d9/d14/f9c [609670,45390] 0 2024-04-22T23:35:11.410 INFO:tasks.workunit.client.1.smithi157.stdout:1/983: creat d4/d17/d34/d3b/d52/d65/dae/f158 x:0 0 0 2024-04-22T23:35:11.412 INFO:tasks.workunit.client.0.smithi107.stdout:6/827: creat db/d20/d2a/d35/d69/db3/dbc/ffb x:0 0 0 2024-04-22T23:35:11.414 INFO:tasks.workunit.client.0.smithi107.stdout:2/840: dwrite d1/d12/d35/d102/dba/ff5 [0,4194304] 0 2024-04-22T23:35:11.417 INFO:tasks.workunit.client.0.smithi107.stdout:7/753: write d7/d21/d48/fcf [3205749,37708] 0 2024-04-22T23:35:11.419 INFO:tasks.workunit.client.0.smithi107.stdout:2/841: dread - d1/d2b/d3d/d57/fa2 zero size 2024-04-22T23:35:11.420 INFO:tasks.workunit.client.0.smithi107.stdout:2/842: mknod d1/d12/d35/d102/dba/df1/c117 0 2024-04-22T23:35:11.422 INFO:tasks.workunit.client.0.smithi107.stdout:2/843: truncate d1/d2b/d59/f90 3994962 0 2024-04-22T23:35:11.423 INFO:tasks.workunit.client.0.smithi107.stdout:2/844: chown d1/d12/d35/d48/d13/l76 290727298 1 2024-04-22T23:35:11.423 INFO:tasks.workunit.client.0.smithi107.stdout:6/828: dread db/f3f [0,4194304] 0 2024-04-22T23:35:11.425 INFO:tasks.workunit.client.0.smithi107.stdout:1/690: write d7/d19/d2d/f31 [2716456,48853] 0 2024-04-22T23:35:11.425 INFO:tasks.workunit.client.0.smithi107.stdout:2/845: mknod d1/d12/d35/d102/d7e/dae/de8/c118 0 2024-04-22T23:35:11.426 INFO:tasks.workunit.client.0.smithi107.stdout:0/737: read d1/d27/d20/d2c/d41/f54 [1777919,11060] 0 2024-04-22T23:35:11.426 INFO:tasks.workunit.client.0.smithi107.stdout:0/738: chown d1/d27/d20/c78 201 1 2024-04-22T23:35:11.426 INFO:tasks.workunit.client.0.smithi107.stdout:0/739: dread - d1/d84/fc8 zero size 2024-04-22T23:35:11.426 INFO:tasks.workunit.client.0.smithi107.stdout:9/730: dwrite d5/d7/d17/d1b/d1f/d36/d53/f5b [4194304,4194304] 0 2024-04-22T23:35:11.426 INFO:tasks.workunit.client.0.smithi107.stdout:9/731: write d5/d14/fca [1007224,94569] 0 2024-04-22T23:35:11.426 INFO:tasks.workunit.client.0.smithi107.stdout:9/732: write d5/d24/f7d [630142,98509] 0 2024-04-22T23:35:11.426 INFO:tasks.workunit.client.0.smithi107.stdout:9/733: chown d5/d7/d27/d46/d4b/ce4 7707640 1 2024-04-22T23:35:11.427 INFO:tasks.workunit.client.0.smithi107.stdout:1/691: write d7/d19/d20/d39/d53/d43/fa8 [1422075,109067] 0 2024-04-22T23:35:11.428 INFO:tasks.workunit.client.0.smithi107.stdout:2/846: rename d1/d2b/d59/fc4 to d1/d12/d35/d48/df3/f119 0 2024-04-22T23:35:11.429 INFO:tasks.workunit.client.0.smithi107.stdout:2/847: write d1/d12/d35/d54/f5f [431997,128715] 0 2024-04-22T23:35:11.430 INFO:tasks.workunit.client.0.smithi107.stdout:0/740: creat d1/d9/ff6 x:0 0 0 2024-04-22T23:35:11.431 INFO:tasks.workunit.client.0.smithi107.stdout:0/741: fdatasync d1/d27/fb0 0 2024-04-22T23:35:11.431 INFO:tasks.workunit.client.0.smithi107.stdout:0/742: chown d1/d9/d14/l79 1 1 2024-04-22T23:35:11.431 INFO:tasks.workunit.client.0.smithi107.stdout:3/725: write db/d21/d27/d15/d29/d2a/d45/ddd/d25/f34 [601121,118027] 0 2024-04-22T23:35:11.432 INFO:tasks.workunit.client.0.smithi107.stdout:3/726: write db/d21/d27/d15/db8/f94 [4392313,45431] 0 2024-04-22T23:35:11.435 INFO:tasks.workunit.client.0.smithi107.stdout:6/829: rmdir db/d20/d2a/d38/d59/d70 39 2024-04-22T23:35:11.437 INFO:tasks.workunit.client.0.smithi107.stdout:1/692: mkdir d7/dce/de7 0 2024-04-22T23:35:11.437 INFO:tasks.workunit.client.0.smithi107.stdout:1/693: readlink d7/d19/d2d/l55 0 2024-04-22T23:35:11.437 INFO:tasks.workunit.client.0.smithi107.stdout:3/727: dread db/d21/d27/f20 [0,4194304] 0 2024-04-22T23:35:11.437 INFO:tasks.workunit.client.0.smithi107.stdout:1/694: creat d7/d19/d71/db8/fe8 x:0 0 0 2024-04-22T23:35:11.437 INFO:tasks.workunit.client.0.smithi107.stdout:1/695: readlink d7/d19/d20/d39/d53/d43/d46/d5c/db7/lcd 0 2024-04-22T23:35:11.437 INFO:tasks.workunit.client.0.smithi107.stdout:2/848: rename d1/d12/d35/d48/l25 to d1/d12/d35/d102/dba/df1/l11a 0 2024-04-22T23:35:11.438 INFO:tasks.workunit.client.0.smithi107.stdout:2/849: fdatasync d1/d67/f7b 0 2024-04-22T23:35:11.438 INFO:tasks.workunit.client.0.smithi107.stdout:2/850: chown d1/d12/d35/d48/d13 92 1 2024-04-22T23:35:11.441 INFO:tasks.workunit.client.0.smithi107.stdout:6/830: creat db/d20/d2a/d5d/ffc x:0 0 0 2024-04-22T23:35:11.441 INFO:tasks.workunit.client.0.smithi107.stdout:6/831: stat db/d20/d2c/d3b/l90 0 2024-04-22T23:35:11.441 INFO:tasks.workunit.client.0.smithi107.stdout:6/832: creat db/d20/ffd x:0 0 0 2024-04-22T23:35:11.441 INFO:tasks.workunit.client.0.smithi107.stdout:6/833: fsync db/d20/d2a/d35/d69/d79/fd4 0 2024-04-22T23:35:11.444 INFO:tasks.workunit.client.0.smithi107.stdout:3/728: creat db/d21/d27/d15/db8/ff6 x:0 0 0 2024-04-22T23:35:11.445 INFO:tasks.workunit.client.0.smithi107.stdout:4/794: dwrite d1/d1b/d44/f95 [0,4194304] 0 2024-04-22T23:35:11.445 INFO:tasks.workunit.client.0.smithi107.stdout:8/783: dwrite d0/f2f [4194304,4194304] 0 2024-04-22T23:35:11.445 INFO:tasks.workunit.client.0.smithi107.stdout:8/784: fdatasync d0/d73/db0/d107/d3b/faf 0 2024-04-22T23:35:11.445 INFO:tasks.workunit.client.0.smithi107.stdout:8/785: chown d0/d85/d51 522135157 1 2024-04-22T23:35:11.445 INFO:tasks.workunit.client.0.smithi107.stdout:8/786: fsync d0/d7/d8/d41/d53/f57 0 2024-04-22T23:35:11.445 INFO:tasks.workunit.client.0.smithi107.stdout:1/696: mknod d7/d19/d20/ce9 0 2024-04-22T23:35:11.446 INFO:tasks.workunit.client.0.smithi107.stdout:0/743: rename d1/d27/d20/d4a/cf4 to d1/d9/d14/d4f/d59/cf7 0 2024-04-22T23:35:11.448 INFO:tasks.workunit.client.0.smithi107.stdout:2/851: unlink d1/d67/c82 0 2024-04-22T23:35:11.450 INFO:tasks.workunit.client.1.smithi157.stdout:7/863: dwrite d0/d9/f7d [4194304,4194304] 0 2024-04-22T23:35:11.451 INFO:tasks.workunit.client.0.smithi107.stdout:4/795: creat d1/d7/d53/d66/d6b/ddb/f118 x:0 0 0 2024-04-22T23:35:11.453 INFO:tasks.workunit.client.0.smithi107.stdout:8/787: unlink d0/d7/l5d 0 2024-04-22T23:35:11.454 INFO:tasks.workunit.client.1.smithi157.stdout:7/864: mkdir d0/d9/d36/d105 0 2024-04-22T23:35:11.456 INFO:tasks.workunit.client.0.smithi107.stdout:3/729: rename db/d21/d27/d15/d29/d2a/d40/f6c to db/d21/d27/d15/d24/d66/ff7 0 2024-04-22T23:35:11.459 INFO:tasks.workunit.client.1.smithi157.stdout:3/983: dwrite d0/d83/dbc/d12e/d34/d3d/df6/dee/ffc [0,4194304] 0 2024-04-22T23:35:11.459 INFO:tasks.workunit.client.0.smithi107.stdout:4/796: link d1/l14 d1/d7/l119 0 2024-04-22T23:35:11.460 INFO:tasks.workunit.client.0.smithi107.stdout:2/852: dread d1/d12/d35/d48/d13/d4f/feb [0,4194304] 0 2024-04-22T23:35:11.460 INFO:tasks.workunit.client.0.smithi107.stdout:8/788: getdents d0/d7/d8/d41/d53/d76 0 2024-04-22T23:35:11.463 INFO:tasks.workunit.client.0.smithi107.stdout:0/744: rename d1/d9/d14/cf1 to d1/d27/d8a/db5/cf8 0 2024-04-22T23:35:11.463 INFO:tasks.workunit.client.1.smithi157.stdout:3/984: unlink d0/d83/dbc/d12e/d34/d3d/d57/d60/d6b/ff7 0 2024-04-22T23:35:11.464 INFO:tasks.workunit.client.0.smithi107.stdout:7/754: dwrite d7/d4c/f6c [0,4194304] 0 2024-04-22T23:35:11.464 INFO:tasks.workunit.client.0.smithi107.stdout:4/797: symlink d1/l11a 0 2024-04-22T23:35:11.466 INFO:tasks.workunit.client.0.smithi107.stdout:2/853: mknod d1/d12/d35/d102/d7e/dae/c11b 0 2024-04-22T23:35:11.466 INFO:tasks.workunit.client.0.smithi107.stdout:5/801: dwrite d9/f101 [0,4194304] 0 2024-04-22T23:35:11.467 INFO:tasks.workunit.client.1.smithi157.stdout:8/698: dwrite d2/d34/f78 [0,4194304] 0 2024-04-22T23:35:11.467 INFO:tasks.workunit.client.0.smithi107.stdout:9/734: dwrite d5/d7/d27/d46/f9f [0,4194304] 0 2024-04-22T23:35:11.467 INFO:tasks.workunit.client.0.smithi107.stdout:8/789: creat d0/d73/db0/d107/f110 x:0 0 0 2024-04-22T23:35:11.470 INFO:tasks.workunit.client.0.smithi107.stdout:0/745: unlink d1/d84/d99/ldb 0 2024-04-22T23:35:11.470 INFO:tasks.workunit.client.0.smithi107.stdout:0/746: truncate d1/d2a/fee 224626 0 2024-04-22T23:35:11.470 INFO:tasks.workunit.client.0.smithi107.stdout:0/747: write d1/d9/dd4/fe4 [5060044,42640] 0 2024-04-22T23:35:11.470 INFO:tasks.workunit.client.0.smithi107.stdout:4/798: dread d1/d21/d29/d2e/d3b/f101 [0,4194304] 0 2024-04-22T23:35:11.471 INFO:tasks.workunit.client.0.smithi107.stdout:7/755: creat d7/d21/d48/d5c/d9d/d6e/db2/f100 x:0 0 0 2024-04-22T23:35:11.472 INFO:tasks.workunit.client.0.smithi107.stdout:2/854: dread - d1/d2b/d3d/fca zero size 2024-04-22T23:35:11.473 INFO:tasks.workunit.client.0.smithi107.stdout:1/697: dwrite d7/d19/d20/d39/d53/d43/d46/fc9 [0,4194304] 0 2024-04-22T23:35:11.474 INFO:tasks.workunit.client.0.smithi107.stdout:5/802: truncate d9/f31 3108109 0 2024-04-22T23:35:11.475 INFO:tasks.workunit.client.1.smithi157.stdout:3/985: link d0/d83/dbc/d12e/d34/d3d/d57/l122 d0/d83/dbc/d12e/d13/d13b/d6c/d126/l155 0 2024-04-22T23:35:11.475 INFO:tasks.workunit.client.1.smithi157.stdout:8/699: rmdir d2/d4/dbd/d67/dab 0 2024-04-22T23:35:11.476 INFO:tasks.workunit.client.1.smithi157.stdout:3/986: creat d0/d83/dbc/d12e/d34/d3d/d57/f156 x:0 0 0 2024-04-22T23:35:11.477 INFO:tasks.workunit.client.1.smithi157.stdout:3/987: write d0/d83/dbc/d12e/d34/d3d/df6/d72/ff2 [458438,126452] 0 2024-04-22T23:35:11.477 INFO:tasks.workunit.client.1.smithi157.stdout:3/988: fsync d0/d83/dbc/d12e/d13/fca 0 2024-04-22T23:35:11.477 INFO:tasks.workunit.client.0.smithi107.stdout:9/735: mknod d5/d29/dbb/c102 0 2024-04-22T23:35:11.477 INFO:tasks.workunit.client.0.smithi107.stdout:9/736: readlink d5/d7/l85 0 2024-04-22T23:35:11.477 INFO:tasks.workunit.client.1.smithi157.stdout:8/700: getdents d2/db1 0 2024-04-22T23:35:11.478 INFO:tasks.workunit.client.0.smithi107.stdout:8/790: link d0/d7/d8/ff0 d0/d7/d20/de4/f111 0 2024-04-22T23:35:11.478 INFO:tasks.workunit.client.0.smithi107.stdout:8/791: readlink d0/d7/d20/lf2 0 2024-04-22T23:35:11.479 INFO:tasks.workunit.client.1.smithi157.stdout:8/701: mkdir d2/d34/d6a/d77/df2 0 2024-04-22T23:35:11.480 INFO:tasks.workunit.client.1.smithi157.stdout:8/702: fsync d2/d4/dbd/d97/fa1 0 2024-04-22T23:35:11.481 INFO:tasks.workunit.client.0.smithi107.stdout:0/748: symlink d1/d27/d20/d2c/d41/dbe/lf9 0 2024-04-22T23:35:11.482 INFO:tasks.workunit.client.1.smithi157.stdout:8/703: creat d2/d4/dbd/ff3 x:0 0 0 2024-04-22T23:35:11.482 INFO:tasks.workunit.client.1.smithi157.stdout:8/704: read d2/d6/d21/d5f/d17/f4f [1077240,123151] 0 2024-04-22T23:35:11.486 INFO:tasks.workunit.client.0.smithi107.stdout:2/855: dread d1/d12/d35/d48/d13/d4f/feb [0,4194304] 0 2024-04-22T23:35:11.486 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:35:11 smithi107 ceph-mon[26355]: pgmap v262: 65 pgs: 65 active+clean; 2.7 GiB data, 8.9 GiB used, 528 GiB / 536 GiB avail; 283 MiB/s rd, 335 MiB/s wr, 453 op/s 2024-04-22T23:35:11.487 INFO:tasks.workunit.client.0.smithi107.stdout:4/799: mknod d1/c11b 0 2024-04-22T23:35:11.487 INFO:tasks.workunit.client.0.smithi107.stdout:4/800: creat d1/d7/d5b/de3/f11c x:0 0 0 2024-04-22T23:35:11.487 INFO:tasks.workunit.client.0.smithi107.stdout:4/801: readlink d1/d7/d63/d78/l94 0 2024-04-22T23:35:11.488 INFO:tasks.workunit.client.0.smithi107.stdout:7/756: creat d7/d21/d54/dcc/f101 x:0 0 0 2024-04-22T23:35:11.488 INFO:tasks.workunit.client.0.smithi107.stdout:7/757: chown d7/d21/d7e/ddf/ff0 4168031 1 2024-04-22T23:35:11.488 INFO:tasks.workunit.client.0.smithi107.stdout:7/758: fdatasync d7/d21/da4/f33 0 2024-04-22T23:35:11.492 INFO:tasks.workunit.client.0.smithi107.stdout:9/737: symlink d5/d7/d17/d1b/d1f/l103 0 2024-04-22T23:35:11.498 INFO:tasks.workunit.client.0.smithi107.stdout:2/856: mkdir d1/d12/d35/d102/d7e/da8/d10e/d11c 0 2024-04-22T23:35:11.500 INFO:tasks.workunit.client.0.smithi107.stdout:2/857: write d1/d12/d35/d48/d13/f32 [613599,20062] 0 2024-04-22T23:35:11.501 INFO:tasks.workunit.client.0.smithi107.stdout:7/759: creat d7/d21/d48/d5c/d9d/ddd/f102 x:0 0 0 2024-04-22T23:35:11.503 INFO:tasks.workunit.client.0.smithi107.stdout:5/803: rmdir d9/d76 39 2024-04-22T23:35:11.504 INFO:tasks.workunit.client.0.smithi107.stdout:9/738: unlink d5/d29/d7b/d91/dde/f68 0 2024-04-22T23:35:11.505 INFO:tasks.workunit.client.0.smithi107.stdout:8/792: rmdir d0/d7/d20/d2d 39 2024-04-22T23:35:11.506 INFO:tasks.workunit.client.0.smithi107.stdout:8/793: chown d0/d7/d87 1 1 2024-04-22T23:35:11.510 INFO:tasks.workunit.client.0.smithi107.stdout:9/739: mkdir d5/d7/d27/d69/db6/d104 0 2024-04-22T23:35:11.510 INFO:tasks.workunit.client.0.smithi107.stdout:9/740: write d5/d6c/fee [860750,91034] 0 2024-04-22T23:35:11.510 INFO:tasks.workunit.client.0.smithi107.stdout:3/730: dwrite db/d21/d27/f39 [0,4194304] 0 2024-04-22T23:35:11.510 INFO:tasks.workunit.client.0.smithi107.stdout:8/794: chown d0/d73/db0/d107/l9e 2 1 2024-04-22T23:35:11.511 INFO:tasks.workunit.client.1.smithi157.stdout:7/865: dwrite d0/f46 [0,4194304] 0 2024-04-22T23:35:11.511 INFO:tasks.workunit.client.0.smithi107.stdout:7/760: dread d7/f13 [0,4194304] 0 2024-04-22T23:35:11.512 INFO:tasks.workunit.client.0.smithi107.stdout:5/804: rmdir d9/db/d42/d75 39 2024-04-22T23:35:11.513 INFO:tasks.workunit.client.0.smithi107.stdout:9/741: unlink d5/f47 0 2024-04-22T23:35:11.516 INFO:tasks.workunit.client.0.smithi107.stdout:3/731: link db/d21/d27/d15/d24/d66/fc6 db/d21/db1/dcd/ff8 0 2024-04-22T23:35:11.516 INFO:tasks.workunit.client.0.smithi107.stdout:3/732: truncate db/d21/d27/d15/d24/d66/fc6 5008099 0 2024-04-22T23:35:11.516 INFO:tasks.workunit.client.0.smithi107.stdout:3/733: chown db/d21/d27/d15/d24/c38 4212 1 2024-04-22T23:35:11.516 INFO:tasks.workunit.client.0.smithi107.stdout:8/795: creat d0/d73/db0/d107/d33/d9b/df3/f112 x:0 0 0 2024-04-22T23:35:11.517 INFO:tasks.workunit.client.0.smithi107.stdout:2/858: dread d1/d67/f7b [0,4194304] 0 2024-04-22T23:35:11.517 INFO:tasks.workunit.client.0.smithi107.stdout:2/859: write d1/d2b/d3d/d57/fa0 [665886,13239] 0 2024-04-22T23:35:11.517 INFO:tasks.workunit.client.0.smithi107.stdout:7/761: creat d7/d1b/f103 x:0 0 0 2024-04-22T23:35:11.518 INFO:tasks.workunit.client.0.smithi107.stdout:9/742: symlink d5/d7/d27/d46/d4b/l105 0 2024-04-22T23:35:11.518 INFO:tasks.workunit.client.0.smithi107.stdout:6/834: dwrite db/d20/d2a/d38/d61/d6f/da2/fc3 [4194304,4194304] 0 2024-04-22T23:35:11.519 INFO:tasks.workunit.client.0.smithi107.stdout:8/796: mknod d0/d73/db0/d107/d33/c113 0 2024-04-22T23:35:11.520 INFO:tasks.workunit.client.1.smithi157.stdout:8/705: dwrite d2/f69 [0,4194304] 0 2024-04-22T23:35:11.520 INFO:tasks.workunit.client.1.smithi157.stdout:8/706: readlink d2/l4c 0 2024-04-22T23:35:11.520 INFO:tasks.workunit.client.1.smithi157.stdout:3/989: dwrite d0/d83/dbc/d12e/d34/f48 [0,4194304] 0 2024-04-22T23:35:11.520 INFO:tasks.workunit.client.1.smithi157.stdout:3/990: chown d0/f1c 3265784 1 2024-04-22T23:35:11.521 INFO:tasks.workunit.client.0.smithi107.stdout:2/860: unlink d1/d12/d35/d48/f2c 0 2024-04-22T23:35:11.521 INFO:tasks.workunit.client.0.smithi107.stdout:2/861: chown d1/d12/d35/d102/d7e/fb4 378 1 2024-04-22T23:35:11.521 INFO:tasks.workunit.client.0.smithi107.stdout:2/862: creat d1/d12/d35/f11d x:0 0 0 2024-04-22T23:35:11.522 INFO:tasks.workunit.client.0.smithi107.stdout:7/762: rmdir d7/d21/d54/dd3 39 2024-04-22T23:35:11.522 INFO:tasks.workunit.client.0.smithi107.stdout:1/698: dwrite d7/d19/d42/f6d [4194304,4194304] 0 2024-04-22T23:35:11.522 INFO:tasks.workunit.client.0.smithi107.stdout:1/699: chown d7/fa0 194 1 2024-04-22T23:35:11.523 INFO:tasks.workunit.client.0.smithi107.stdout:9/743: creat d5/d7/d27/d46/dc7/f106 x:0 0 0 2024-04-22T23:35:11.523 INFO:tasks.workunit.client.0.smithi107.stdout:7/763: dread d7/f3f [0,4194304] 0 2024-04-22T23:35:11.525 INFO:tasks.workunit.client.0.smithi107.stdout:6/835: link db/f12 db/d20/d2a/d38/d61/d6f/ffe 0 2024-04-22T23:35:11.525 INFO:tasks.workunit.client.0.smithi107.stdout:6/836: read - db/d20/d2a/d38/d61/d6f/d9b/ff8 zero size 2024-04-22T23:35:11.526 INFO:tasks.workunit.client.0.smithi107.stdout:1/700: read d7/d19/d20/d39/d53/d43/fab [233777,114942] 0 2024-04-22T23:35:11.526 INFO:tasks.workunit.client.0.smithi107.stdout:1/701: write d7/d19/d71/db8/fd5 [92017,10545] 0 2024-04-22T23:35:11.526 INFO:tasks.workunit.client.0.smithi107.stdout:1/702: creat d7/d19/d20/d39/d53/d43/d46/d5c/dac/fea x:0 0 0 2024-04-22T23:35:11.526 INFO:tasks.workunit.client.0.smithi107.stdout:1/703: stat d7/d19/d2d/d89/fd0 0 2024-04-22T23:35:11.527 INFO:tasks.workunit.client.0.smithi107.stdout:8/797: rename d0/d3a/f56 to d0/d7/d8/d41/d53/d76/f114 0 2024-04-22T23:35:11.529 INFO:tasks.workunit.client.0.smithi107.stdout:0/749: dwrite d1/d9/d14/d4f/d59/fa5 [0,4194304] 0 2024-04-22T23:35:11.529 INFO:tasks.workunit.client.1.smithi157.stdout:3/991: rename d0/d83/dbc/d12e/d34/d47/fad to d0/d83/dbc/d12e/d34/d3d/df6/dee/f157 0 2024-04-22T23:35:11.529 INFO:tasks.workunit.client.1.smithi157.stdout:7/866: dread d0/d9/f1d [8388608,4194304] 0 2024-04-22T23:35:11.531 INFO:tasks.workunit.client.0.smithi107.stdout:2/863: creat d1/d12/d35/d102/d87/f11e x:0 0 0 2024-04-22T23:35:11.531 INFO:tasks.workunit.client.0.smithi107.stdout:2/864: truncate d1/d12/d35/d102/f50 6008932 0 2024-04-22T23:35:11.533 INFO:tasks.workunit.client.1.smithi157.stdout:3/992: symlink d0/d83/dbc/d12e/d34/d3d/d57/d60/l158 0 2024-04-22T23:35:11.534 INFO:tasks.workunit.client.0.smithi107.stdout:4/802: dwrite d1/d7/d53/fd6 [0,4194304] 0 2024-04-22T23:35:11.535 INFO:tasks.workunit.client.0.smithi107.stdout:9/744: unlink d5/d7/d17/de0/l86 0 2024-04-22T23:35:11.535 INFO:tasks.workunit.client.0.smithi107.stdout:9/745: fsync d5/fc 0 2024-04-22T23:35:11.535 INFO:tasks.workunit.client.1.smithi157.stdout:7/867: rename d0/d9/la to d0/d9/d29/d5e/d9e/l106 0 2024-04-22T23:35:11.536 INFO:tasks.workunit.client.1.smithi157.stdout:7/868: dread - d0/d9/d29/d5e/d9e/fc5 zero size 2024-04-22T23:35:11.536 INFO:tasks.workunit.client.1.smithi157.stdout:3/993: mknod d0/d83/dbc/d12e/d13/d13b/d95/c159 0 2024-04-22T23:35:11.536 INFO:tasks.workunit.client.1.smithi157.stdout:3/994: fdatasync d0/d83/dbc/d12e/d34/d3d/df6/d72/f10b 0 2024-04-22T23:35:11.537 INFO:tasks.workunit.client.1.smithi157.stdout:3/995: truncate d0/d83/dbc/d12e/d34/d3d/df6/dee/d9e/f121 4982853 0 2024-04-22T23:35:11.537 INFO:tasks.workunit.client.0.smithi107.stdout:6/837: symlink db/d20/d2a/d35/lff 0 2024-04-22T23:35:11.537 INFO:tasks.workunit.client.0.smithi107.stdout:6/838: write db/d20/d2a/d38/d61/d6f/da2/fd9 [928365,86158] 0 2024-04-22T23:35:11.538 INFO:tasks.workunit.client.0.smithi107.stdout:6/839: chown ca 5205 1 2024-04-22T23:35:11.538 INFO:tasks.workunit.client.1.smithi157.stdout:3/996: creat d0/d83/dbc/d12e/d34/d62/f15a x:0 0 0 2024-04-22T23:35:11.540 INFO:tasks.workunit.client.0.smithi107.stdout:5/805: dwrite f7 [4194304,4194304] 0 2024-04-22T23:35:11.541 INFO:tasks.workunit.client.0.smithi107.stdout:1/704: creat d7/d19/d2d/da4/feb x:0 0 0 2024-04-22T23:35:11.545 INFO:tasks.workunit.client.0.smithi107.stdout:8/798: rename d0/d73/db0/d107/d33/d37/f49 to d0/d73/f115 0 2024-04-22T23:35:11.546 INFO:tasks.workunit.client.0.smithi107.stdout:5/806: dread d9/db/de/d72/f111 [0,4194304] 0 2024-04-22T23:35:11.546 INFO:tasks.workunit.client.0.smithi107.stdout:5/807: write d9/f43 [1437076,55488] 0 2024-04-22T23:35:11.547 INFO:tasks.workunit.client.0.smithi107.stdout:3/734: dwrite db/d21/d27/d15/db8/f82 [0,4194304] 0 2024-04-22T23:35:11.547 INFO:tasks.workunit.client.0.smithi107.stdout:3/735: chown db/d21/d27/d15/d24/d52/d60/f93 59751 1 2024-04-22T23:35:11.550 INFO:tasks.workunit.client.0.smithi107.stdout:0/750: rmdir d1/d7b 39 2024-04-22T23:35:11.550 INFO:tasks.workunit.client.0.smithi107.stdout:0/751: chown d1/d9/d14/d4f/d59/cf7 2 1 2024-04-22T23:35:11.551 INFO:tasks.workunit.client.0.smithi107.stdout:2/865: creat d1/d67/f11f x:0 0 0 2024-04-22T23:35:11.552 INFO:tasks.workunit.client.0.smithi107.stdout:9/746: getdents d5/d7/d27/d46/d4b/dff 0 2024-04-22T23:35:11.553 INFO:tasks.workunit.client.0.smithi107.stdout:6/840: mkdir db/d20/d2a/d35/d69/db3/d100 0 2024-04-22T23:35:11.555 INFO:tasks.workunit.client.1.smithi157.stdout:7/869: truncate d0/d9/db/f15 101504 0 2024-04-22T23:35:11.555 INFO:tasks.workunit.client.1.smithi157.stdout:7/870: chown d0/d9/db/df/d68/dc6 274 1 2024-04-22T23:35:11.556 INFO:tasks.workunit.client.0.smithi107.stdout:1/705: mknod d7/d19/d20/d39/d53/d43/d46/d68/dde/cec 0 2024-04-22T23:35:11.558 INFO:tasks.workunit.client.1.smithi157.stdout:8/707: dwrite d2/d4/d73/fc7 [0,4194304] 0 2024-04-22T23:35:11.560 INFO:tasks.workunit.client.0.smithi107.stdout:3/736: dread db/d21/d27/d15/d29/d2a/f43 [4194304,4194304] 0 2024-04-22T23:35:11.560 INFO:tasks.workunit.client.0.smithi107.stdout:6/841: dread db/d20/d2c/d3b/d40/f8b [0,4194304] 0 2024-04-22T23:35:11.560 INFO:tasks.workunit.client.0.smithi107.stdout:6/842: read - db/d20/d2a/d38/d61/d6f/d9b/ff8 zero size 2024-04-22T23:35:11.560 INFO:tasks.workunit.client.0.smithi107.stdout:6/843: write db/d20/d2c/db5/ff1 [512009,14914] 0 2024-04-22T23:35:11.560 INFO:tasks.workunit.client.0.smithi107.stdout:6/844: chown db/d20/d2a/d35/d69 501 1 2024-04-22T23:35:11.560 INFO:tasks.workunit.client.0.smithi107.stdout:8/799: mknod d0/d7/d87/dc8/c116 0 2024-04-22T23:35:11.560 INFO:tasks.workunit.client.0.smithi107.stdout:8/800: readlink d0/d7/d87/l106 0 2024-04-22T23:35:11.560 INFO:tasks.workunit.client.0.smithi107.stdout:8/801: creat d0/d7/d87/dc8/f117 x:0 0 0 2024-04-22T23:35:11.560 INFO:tasks.workunit.client.0.smithi107.stdout:8/802: readlink d0/d73/ld3 0 2024-04-22T23:35:11.560 INFO:tasks.workunit.client.0.smithi107.stdout:8/803: creat d0/d73/db0/d107/d33/f118 x:0 0 0 2024-04-22T23:35:11.561 INFO:tasks.workunit.client.0.smithi107.stdout:5/808: symlink d9/d76/da0/l113 0 2024-04-22T23:35:11.563 INFO:tasks.workunit.client.1.smithi157.stdout:8/708: rename d2/la3 to d2/d6/d21/d5f/d17/dc8/lf4 0 2024-04-22T23:35:11.564 INFO:tasks.workunit.client.1.smithi157.stdout:3/997: dwrite d0/d83/dbc/d12e/d34/d39/f58 [0,4194304] 0 2024-04-22T23:35:11.565 INFO:tasks.workunit.client.0.smithi107.stdout:4/803: dwrite d1/d7/d5b/dbf/d27/f115 [4194304,4194304] 0 2024-04-22T23:35:11.565 INFO:tasks.workunit.client.0.smithi107.stdout:4/804: truncate d1/d7/d63/d78/d96/fa6 1866464 0 2024-04-22T23:35:11.566 INFO:tasks.workunit.client.1.smithi157.stdout:8/709: creat d2/d6/d21/d5f/ff5 x:0 0 0 2024-04-22T23:35:11.568 INFO:tasks.workunit.client.0.smithi107.stdout:9/747: dread d5/d14/d2e/f3f [0,4194304] 0 2024-04-22T23:35:11.570 INFO:tasks.workunit.client.1.smithi157.stdout:3/998: symlink d0/d83/dbc/d12e/d13/d13b/d6c/d126/l15b 0 2024-04-22T23:35:11.575 INFO:tasks.workunit.client.1.smithi157.stdout:3/999: dread d0/d83/dbc/d12e/d34/d3d/d57/d5c/f78 [0,4194304] 0 2024-04-22T23:35:11.578 INFO:tasks.workunit.client.1.smithi157.stdout:7/871: dwrite d0/d9/f1d [8388608,4194304] 0 2024-04-22T23:35:11.579 INFO:tasks.workunit.client.1.smithi157.stdout:7/872: mknod d0/d9/db/da3/c107 0 2024-04-22T23:35:11.581 INFO:tasks.workunit.client.1.smithi157.stdout:8/710: write d2/d34/d6a/d77/f82 [3609144,73327] 0 2024-04-22T23:35:11.581 INFO:tasks.workunit.client.1.smithi157.stdout:8/711: write d2/d6/d21/d5f/d11/fdc [765562,109456] 0 2024-04-22T23:35:11.581 INFO:tasks.workunit.client.1.smithi157.stdout:8/712: chown d2/lb0 854727559 1 2024-04-22T23:35:11.585 INFO:tasks.workunit.client.0.smithi107.stdout:4/805: dwrite d1/d7/d63/d78/fd9 [0,4194304] 0 2024-04-22T23:35:11.587 INFO:tasks.workunit.client.0.smithi107.stdout:0/752: creat d1/d2a/ffa x:0 0 0 2024-04-22T23:35:11.587 INFO:tasks.workunit.client.0.smithi107.stdout:0/753: write d1/d27/fb0 [99636,68369] 0 2024-04-22T23:35:11.593 INFO:tasks.workunit.client.0.smithi107.stdout:2/866: symlink d1/d12/d35/d102/d87/l120 0 2024-04-22T23:35:11.594 INFO:tasks.workunit.client.0.smithi107.stdout:1/706: mkdir d7/d19/d42/ded 0 2024-04-22T23:35:11.594 INFO:tasks.workunit.client.0.smithi107.stdout:1/707: creat d7/d19/d2d/fee x:0 0 0 2024-04-22T23:35:11.594 INFO:tasks.workunit.client.0.smithi107.stdout:2/867: read d1/d12/d35/d102/d7e/da8/fb8 [603684,37441] 0 2024-04-22T23:35:11.595 INFO:tasks.workunit.client.0.smithi107.stdout:3/737: mknod db/d21/d27/d15/d29/d2a/d45/ddd/d25/d53/cf9 0 2024-04-22T23:35:11.596 INFO:tasks.workunit.client.0.smithi107.stdout:6/845: truncate db/d20/d2c/d3b/fce 133295 0 2024-04-22T23:35:11.596 INFO:tasks.workunit.client.0.smithi107.stdout:6/846: getdents db/d20/d2a/d38/d61/d6f/da2/d84 0 2024-04-22T23:35:11.600 INFO:tasks.workunit.client.0.smithi107.stdout:8/804: mknod d0/d7/d87/dc8/c119 0 2024-04-22T23:35:11.601 INFO:tasks.workunit.client.1.smithi157.stdout:7/873: dwrite d0/f2 [0,4194304] 0 2024-04-22T23:35:11.601 INFO:tasks.workunit.client.1.smithi157.stdout:8/713: dwrite d2/d6/fe0 [0,4194304] 0 2024-04-22T23:35:11.602 INFO:tasks.workunit.client.0.smithi107.stdout:5/809: creat d9/d26/d79/f114 x:0 0 0 2024-04-22T23:35:11.602 INFO:tasks.workunit.client.0.smithi107.stdout:5/810: chown d9/db/de/d72 11804 1 2024-04-22T23:35:11.602 INFO:tasks.workunit.client.1.smithi157.stdout:7/874: rename d0/d9/d29/d33/l5b to d0/d9/d36/l108 0 2024-04-22T23:35:11.606 INFO:tasks.workunit.client.1.smithi157.stdout:8/714: symlink d2/lf6 0 2024-04-22T23:35:11.607 INFO:tasks.workunit.client.1.smithi157.stdout:7/875: creat d0/d9/db/f109 x:0 0 0 2024-04-22T23:35:11.608 INFO:tasks.workunit.client.1.smithi157.stdout:8/715: mknod d2/d4/dda/cf7 0 2024-04-22T23:35:11.608 INFO:tasks.workunit.client.1.smithi157.stdout:8/716: creat d2/d6/d21/d5f/ff8 x:0 0 0 2024-04-22T23:35:11.608 INFO:tasks.workunit.client.1.smithi157.stdout:8/717: chown d2/db1/c51 12216432 1 2024-04-22T23:35:11.610 INFO:tasks.workunit.client.1.smithi157.stdout:8/718: write d2/d6/d21/d5f/d17/f1e [4244736,129946] 0 2024-04-22T23:35:11.610 INFO:tasks.workunit.client.1.smithi157.stdout:8/719: creat d2/d4/dda/ff9 x:0 0 0 2024-04-22T23:35:11.611 INFO:tasks.workunit.client.0.smithi107.stdout:0/754: creat d1/d9/d14/d4f/da6/ffb x:0 0 0 2024-04-22T23:35:11.611 INFO:tasks.workunit.client.0.smithi107.stdout:0/755: getdents d1/d7b 0 2024-04-22T23:35:11.615 INFO:tasks.workunit.client.0.smithi107.stdout:1/708: unlink d7/d19/d20/d39/d53/d43/d63/d91/lc0 0 2024-04-22T23:35:11.615 INFO:tasks.workunit.client.1.smithi157.stdout:8/720: link d2/d6/d21/d5f/d11/ff1 d2/d6/d21/d5f/d17/dc8/ffa 0 2024-04-22T23:35:11.617 INFO:tasks.workunit.client.0.smithi107.stdout:6/847: creat db/d20/d2a/d38/d4b/d9d/f101 x:0 0 0 2024-04-22T23:35:11.617 INFO:tasks.workunit.client.0.smithi107.stdout:0/756: dread d1/d2a/f77 [0,4194304] 0 2024-04-22T23:35:11.618 INFO:tasks.workunit.client.1.smithi157.stdout:8/721: unlink d2/d4/dad/le5 0 2024-04-22T23:35:11.618 INFO:tasks.workunit.client.1.smithi157.stdout:8/722: fdatasync d2/d6/d21/d8d/f9c 0 2024-04-22T23:35:11.618 INFO:tasks.workunit.client.1.smithi157.stdout:5/939: sync 2024-04-22T23:35:11.619 INFO:tasks.workunit.client.1.smithi157.stdout:8/723: mkdir d2/db1/dfb 0 2024-04-22T23:35:11.619 INFO:tasks.workunit.client.1.smithi157.stdout:9/852: sync 2024-04-22T23:35:11.619 INFO:tasks.workunit.client.1.smithi157.stdout:1/984: sync 2024-04-22T23:35:11.620 INFO:tasks.workunit.client.1.smithi157.stdout:1/985: fdatasync d4/db/d3d/dc1/fd0 0 2024-04-22T23:35:11.620 INFO:tasks.workunit.client.1.smithi157.stdout:0/871: sync 2024-04-22T23:35:11.620 INFO:tasks.workunit.client.0.smithi107.stdout:8/805: symlink d0/d7/d8/d41/l11a 0 2024-04-22T23:35:11.621 INFO:tasks.workunit.client.0.smithi107.stdout:0/757: dread d1/d27/d20/f60 [0,4194304] 0 2024-04-22T23:35:11.622 INFO:tasks.workunit.client.1.smithi157.stdout:5/940: symlink d1/d5/dc/d43/d4f/d7f/dab/l153 0 2024-04-22T23:35:11.623 INFO:tasks.workunit.client.1.smithi157.stdout:9/853: creat d4/d7e/f104 x:0 0 0 2024-04-22T23:35:11.624 INFO:tasks.workunit.client.1.smithi157.stdout:0/872: symlink d17/d2b/d4f/dc5/l123 0 2024-04-22T23:35:11.626 INFO:tasks.workunit.client.1.smithi157.stdout:1/986: dread d4/d5f/fce [0,4194304] 0 2024-04-22T23:35:11.626 INFO:tasks.workunit.client.1.smithi157.stdout:0/873: creat d17/d2b/d5b/f124 x:0 0 0 2024-04-22T23:35:11.626 INFO:tasks.workunit.client.1.smithi157.stdout:1/987: chown d4/d17/d34/d75/d8a/dfc 378 1 2024-04-22T23:35:11.628 INFO:tasks.workunit.client.1.smithi157.stdout:1/988: creat d4/d17/d34/d3b/d50/dad/f159 x:0 0 0 2024-04-22T23:35:11.628 INFO:tasks.workunit.client.1.smithi157.stdout:1/989: chown d4/d17/l120 6358 1 2024-04-22T23:35:11.628 INFO:tasks.workunit.client.1.smithi157.stdout:1/990: fsync d4/d17/d34/d75/d8a/fc0 0 2024-04-22T23:35:11.630 INFO:tasks.workunit.client.0.smithi107.stdout:9/748: dwrite d5/d29/f2c [4194304,4194304] 0 2024-04-22T23:35:11.631 INFO:tasks.workunit.client.1.smithi157.stdout:0/874: rename d17/d2b/d4f/dc5/dcc/de9/c10f to d17/d23/d34/d67/d7f/d69/dce/c125 0 2024-04-22T23:35:11.631 INFO:tasks.workunit.client.1.smithi157.stdout:0/875: fsync d17/d2b/d4f/d5a/dd2/de4/f11a 0 2024-04-22T23:35:11.631 INFO:tasks.workunit.client.1.smithi157.stdout:1/991: write d4/db/d3d/ff9 [419332,50016] 0 2024-04-22T23:35:11.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:35:11 smithi157 ceph-mon[33877]: pgmap v262: 65 pgs: 65 active+clean; 2.7 GiB data, 8.9 GiB used, 528 GiB / 536 GiB avail; 283 MiB/s rd, 335 MiB/s wr, 453 op/s 2024-04-22T23:35:11.634 INFO:tasks.workunit.client.1.smithi157.stdout:1/992: getdents d4/d17 0 2024-04-22T23:35:11.634 INFO:tasks.workunit.client.0.smithi107.stdout:5/811: rename d9/db/de/d72/cf4 to d9/db/d92/dc2/c115 0 2024-04-22T23:35:11.635 INFO:tasks.workunit.client.1.smithi157.stdout:1/993: creat d4/d17/d34/d3b/d52/d65/dbc/d149/f15a x:0 0 0 2024-04-22T23:35:11.636 INFO:tasks.workunit.client.1.smithi157.stdout:7/876: dwrite d0/d9/db/df/dac/fad [0,4194304] 0 2024-04-22T23:35:11.638 INFO:tasks.workunit.client.1.smithi157.stdout:1/994: getdents d4/d17/d34/d3b/d52/d65/dbc 0 2024-04-22T23:35:11.638 INFO:tasks.workunit.client.1.smithi157.stdout:1/995: fdatasync d4/d5f/fce 0 2024-04-22T23:35:11.638 INFO:tasks.workunit.client.1.smithi157.stdout:1/996: chown d4/db/d3d/dc1/d100/f11d 424 1 2024-04-22T23:35:11.641 INFO:tasks.workunit.client.0.smithi107.stdout:4/806: rmdir d1/d21/d29/d2e 39 2024-04-22T23:35:11.642 INFO:tasks.workunit.client.0.smithi107.stdout:1/709: mkdir d7/def 0 2024-04-22T23:35:11.644 INFO:tasks.workunit.client.1.smithi157.stdout:8/724: dwrite d2/d6/fba [0,4194304] 0 2024-04-22T23:35:11.645 INFO:tasks.workunit.client.0.smithi107.stdout:3/738: mknod db/d21/d27/d36/d9c/cfa 0 2024-04-22T23:35:11.645 INFO:tasks.workunit.client.0.smithi107.stdout:0/758: dread d1/d9/f3e [4194304,4194304] 0 2024-04-22T23:35:11.645 INFO:tasks.workunit.client.0.smithi107.stdout:9/749: chown d5/d29/f3a 10294026 1 2024-04-22T23:35:11.647 INFO:tasks.workunit.client.0.smithi107.stdout:6/848: rename db/d20/d2a/d38/d61/d6f/ffe to db/d20/d2a/d35/d69/db3/d100/f102 0 2024-04-22T23:35:11.647 INFO:tasks.workunit.client.0.smithi107.stdout:6/849: write db/d20/d2c/d3b/feb [215765,22403] 0 2024-04-22T23:35:11.649 INFO:tasks.workunit.client.1.smithi157.stdout:1/997: dread d4/d17/fd6 [0,4194304] 0 2024-04-22T23:35:11.649 INFO:tasks.workunit.client.0.smithi107.stdout:4/807: creat d1/d21/d29/d2e/dfa/f11d x:0 0 0 2024-04-22T23:35:11.651 INFO:tasks.workunit.client.0.smithi107.stdout:1/710: creat d7/d19/d20/d39/d53/d43/ff0 x:0 0 0 2024-04-22T23:35:11.651 INFO:tasks.workunit.client.0.smithi107.stdout:1/711: write d7/d19/f2c [3691740,99553] 0 2024-04-22T23:35:11.652 INFO:tasks.workunit.client.1.smithi157.stdout:9/854: dwrite d4/d15/d23/d27/db5/fc3 [0,4194304] 0 2024-04-22T23:35:11.652 INFO:tasks.workunit.client.1.smithi157.stdout:9/855: fdatasync d4/d15/d23/d67/f93 0 2024-04-22T23:35:11.653 INFO:tasks.workunit.client.0.smithi107.stdout:8/806: getdents d0/d7/d20/da2/da3 0 2024-04-22T23:35:11.653 INFO:tasks.workunit.client.0.smithi107.stdout:8/807: chown d0/d73/fc1 79060 1 2024-04-22T23:35:11.654 INFO:tasks.workunit.client.1.smithi157.stdout:9/856: creat d4/d15/d23/d27/db5/f105 x:0 0 0 2024-04-22T23:35:11.656 INFO:tasks.workunit.client.0.smithi107.stdout:3/739: unlink db/d21/d27/d15/d24/d52/l95 0 2024-04-22T23:35:11.658 INFO:tasks.workunit.client.0.smithi107.stdout:9/750: fsync d5/d29/f51 0 2024-04-22T23:35:11.658 INFO:tasks.workunit.client.0.smithi107.stdout:9/751: chown d5/d7/d17/d1b/ldd 528594018 1 2024-04-22T23:35:11.659 INFO:tasks.workunit.client.0.smithi107.stdout:0/759: link d1/d27/d20/c50 d1/d27/d20/d40/d75/cfc 0 2024-04-22T23:35:11.660 INFO:tasks.workunit.client.0.smithi107.stdout:6/850: symlink db/d20/d2a/d38/d61/l103 0 2024-04-22T23:35:11.665 INFO:tasks.workunit.client.1.smithi157.stdout:9/857: write d4/d15/d23/d27/f33 [3955343,21788] 0 2024-04-22T23:35:11.666 INFO:tasks.workunit.client.0.smithi107.stdout:1/712: symlink d7/d19/d20/d39/d53/d43/d46/d99/lf1 0 2024-04-22T23:35:11.666 INFO:tasks.workunit.client.0.smithi107.stdout:8/808: truncate d0/d3a/d46/f70 1220288 0 2024-04-22T23:35:11.667 INFO:tasks.workunit.client.1.smithi157.stdout:8/725: dread d2/db1/d58/f92 [0,4194304] 0 2024-04-22T23:35:11.668 INFO:tasks.workunit.client.1.smithi157.stdout:9/858: link d4/d15/d23/d27/f87 d4/d2d/f106 0 2024-04-22T23:35:11.668 INFO:tasks.workunit.client.1.smithi157.stdout:5/941: dwrite d1/d5/dc/d43/f88 [0,4194304] 0 2024-04-22T23:35:11.668 INFO:tasks.workunit.client.1.smithi157.stdout:5/942: fsync d1/d5/d7/d10/f151 0 2024-04-22T23:35:11.668 INFO:tasks.workunit.client.1.smithi157.stdout:8/726: mkdir d2/d34/d6a/d77/dfc 0 2024-04-22T23:35:11.669 INFO:tasks.workunit.client.1.smithi157.stdout:8/727: chown d2/d4/d73/ced 100026860 1 2024-04-22T23:35:11.669 INFO:tasks.workunit.client.1.smithi157.stdout:8/728: write d2/d6/d21/d5f/d11/d13/f7d [2137535,130939] 0 2024-04-22T23:35:11.669 INFO:tasks.workunit.client.1.smithi157.stdout:8/729: symlink d2/d4/dbd/d97/ddd/lfd 0 2024-04-22T23:35:11.670 INFO:tasks.workunit.client.0.smithi107.stdout:3/740: rename db/d21/d27/d15/d24/d66/ff7 to db/d21/d27/d15/d29/d2a/d45/ddd/ffb 0 2024-04-22T23:35:11.675 INFO:tasks.workunit.client.0.smithi107.stdout:6/851: rename db/d20/d2c/l5b to db/d20/db1/l104 0 2024-04-22T23:35:11.677 INFO:tasks.workunit.client.0.smithi107.stdout:3/741: mkdir db/d21/d27/d15/d24/d52/d60/dca/dfc 0 2024-04-22T23:35:11.677 INFO:tasks.workunit.client.0.smithi107.stdout:1/713: getdents d7/d19/d20/d39/d53/d43/d46/d5c 0 2024-04-22T23:35:11.677 INFO:tasks.workunit.client.0.smithi107.stdout:1/714: read - d7/d19/d20/fdc zero size 2024-04-22T23:35:11.679 INFO:tasks.workunit.client.0.smithi107.stdout:1/715: truncate d7/fc1 71545 0 2024-04-22T23:35:11.682 INFO:tasks.workunit.client.0.smithi107.stdout:5/812: dwrite d9/d26/d6a/d100/f112 [0,4194304] 0 2024-04-22T23:35:11.683 INFO:tasks.workunit.client.0.smithi107.stdout:5/813: readlink d9/db/de/d72/d7c/ld2 0 2024-04-22T23:35:11.683 INFO:tasks.workunit.client.0.smithi107.stdout:1/716: dread d7/d19/d42/f6f [0,4194304] 0 2024-04-22T23:35:11.689 INFO:tasks.workunit.client.0.smithi107.stdout:6/852: read db/f2d [3854509,89522] 0 2024-04-22T23:35:11.695 INFO:tasks.workunit.client.0.smithi107.stdout:1/717: symlink d7/d19/d20/d39/d53/d43/d63/d91/lf2 0 2024-04-22T23:35:11.695 INFO:tasks.workunit.client.0.smithi107.stdout:5/814: creat d9/db/de/dbe/f116 x:0 0 0 2024-04-22T23:35:11.696 INFO:tasks.workunit.client.0.smithi107.stdout:5/815: chown d9/d26/d79/l89 35 1 2024-04-22T23:35:11.696 INFO:tasks.workunit.client.1.smithi157.stdout:0/876: dwrite d17/d23/d34/d67/d7f/d69/dce/fe5 [0,4194304] 0 2024-04-22T23:35:11.696 INFO:tasks.workunit.client.1.smithi157.stdout:7/877: dwrite d0/d9/db/f15 [0,4194304] 0 2024-04-22T23:35:11.696 INFO:tasks.workunit.client.1.smithi157.stdout:7/878: readlink d0/l3 0 2024-04-22T23:35:11.696 INFO:tasks.workunit.client.1.smithi157.stdout:7/879: write d0/d9/d29/d33/d37/f50 [1040317,129617] 0 2024-04-22T23:35:11.696 INFO:tasks.workunit.client.1.smithi157.stdout:0/877: dread d17/ddb/f107 [0,4194304] 0 2024-04-22T23:35:11.697 INFO:tasks.workunit.client.0.smithi107.stdout:4/808: dwrite d1/d21/d29/d2e/d88/fe4 [0,4194304] 0 2024-04-22T23:35:11.697 INFO:tasks.workunit.client.0.smithi107.stdout:4/809: chown d1/d21/d85/le5 7 1 2024-04-22T23:35:11.698 INFO:tasks.workunit.client.0.smithi107.stdout:5/816: dread d9/db/de/d72/f81 [0,4194304] 0 2024-04-22T23:35:11.698 INFO:tasks.workunit.client.0.smithi107.stdout:5/817: write d9/db/de/fe5 [621001,116487] 0 2024-04-22T23:35:11.698 INFO:tasks.workunit.client.0.smithi107.stdout:1/718: mkdir d7/d19/d20/d39/de0/df3 0 2024-04-22T23:35:11.698 INFO:tasks.workunit.client.0.smithi107.stdout:1/719: truncate d7/d19/d2d/d89/f90 203286 0 2024-04-22T23:35:11.700 INFO:tasks.workunit.client.1.smithi157.stdout:1/998: dwrite d4/d17/d34/d84/f14c [0,4194304] 0 2024-04-22T23:35:11.701 INFO:tasks.workunit.client.1.smithi157.stdout:1/999: fsync d4/d17/d1d/d33/d41/d54/db3/d126/f136 0 2024-04-22T23:35:11.703 INFO:tasks.workunit.client.1.smithi157.stdout:0/878: unlink f11 0 2024-04-22T23:35:11.703 INFO:tasks.workunit.client.1.smithi157.stdout:0/879: write d17/f2d [59050,7112] 0 2024-04-22T23:35:11.703 INFO:tasks.workunit.client.1.smithi157.stdout:0/880: chown d17/d23/d34/d67/d104/d121 341 1 2024-04-22T23:35:11.708 INFO:tasks.workunit.client.1.smithi157.stdout:7/880: rmdir d0/d9/db/d35 39 2024-04-22T23:35:11.713 INFO:tasks.workunit.client.0.smithi107.stdout:5/818: write d9/d26/f3b [3372816,111805] 0 2024-04-22T23:35:11.714 INFO:tasks.workunit.client.0.smithi107.stdout:4/810: rename d1/d21/d29/d2e/d88/d99/dd2/f7c to d1/d21/d29/d2e/d88/d99/dd2/df0/f11e 0 2024-04-22T23:35:11.714 INFO:tasks.workunit.client.0.smithi107.stdout:4/811: write d1/d21/d29/d2e/d88/d99/dd2/f1f [1248658,43338] 0 2024-04-22T23:35:11.715 INFO:tasks.workunit.client.0.smithi107.stdout:5/819: link d9/db/d42/f84 d9/d76/f117 0 2024-04-22T23:35:11.717 INFO:tasks.workunit.client.0.smithi107.stdout:5/820: mknod d9/db/de/d72/c118 0 2024-04-22T23:35:11.721 INFO:tasks.workunit.client.0.smithi107.stdout:5/821: creat d9/d76/da0/f119 x:0 0 0 2024-04-22T23:35:11.722 INFO:tasks.workunit.client.0.smithi107.stdout:1/720: rename d7/d19/d71 to d7/d19/d20/d39/d93/df4 0 2024-04-22T23:35:11.723 INFO:tasks.workunit.client.0.smithi107.stdout:5/822: truncate d9/f4e 6290967 0 2024-04-22T23:35:11.723 INFO:tasks.workunit.client.1.smithi157.stdout:0/881: dread d17/d23/f28 [0,4194304] 0 2024-04-22T23:35:11.725 INFO:tasks.workunit.client.1.smithi157.stdout:0/882: rename d17/d23/d34/dee/caa to d17/d38/c126 0 2024-04-22T23:35:11.727 INFO:tasks.workunit.client.1.smithi157.stdout:0/883: creat d17/d23/d34/d67/d78/f127 x:0 0 0 2024-04-22T23:35:11.728 INFO:tasks.workunit.client.1.smithi157.stdout:7/881: dread d0/f46 [0,4194304] 0 2024-04-22T23:35:11.728 INFO:tasks.workunit.client.1.smithi157.stdout:5/943: dwrite d1/d5/d7/d2f/d45/d54/d59/f70 [0,4194304] 0 2024-04-22T23:35:11.728 INFO:tasks.workunit.client.1.smithi157.stdout:9/859: dwrite d4/d15/d23/d27/d47/db3/fec [0,4194304] 0 2024-04-22T23:35:11.729 INFO:tasks.workunit.client.0.smithi107.stdout:6/853: dwrite db/d20/d2a/d38/f53 [0,4194304] 0 2024-04-22T23:35:11.729 INFO:tasks.workunit.client.0.smithi107.stdout:9/752: dwrite d5/d29/d7b/d91/dde/dc9/fae [0,4194304] 0 2024-04-22T23:35:11.729 INFO:tasks.workunit.client.0.smithi107.stdout:3/742: dwrite db/d21/d27/d15/d29/d2a/f4c [4194304,4194304] 0 2024-04-22T23:35:11.729 INFO:tasks.workunit.client.0.smithi107.stdout:4/812: rename d1/d7/d5b/dbf/da8/db9 to d1/d21/d29/d2e/d88/d99/dd2/daf/dc1/d11f 0 2024-04-22T23:35:11.731 INFO:tasks.workunit.client.1.smithi157.stdout:9/860: rename d4/c40 to d4/df/d57/dbd/dc6/c107 0 2024-04-22T23:35:11.732 INFO:tasks.workunit.client.0.smithi107.stdout:6/854: mkdir db/d20/d2a/d38/d61/d6f/d105 0 2024-04-22T23:35:11.733 INFO:tasks.workunit.client.0.smithi107.stdout:6/855: chown db/d20/d2a/f42 56500 1 2024-04-22T23:35:11.733 INFO:tasks.workunit.client.1.smithi157.stdout:9/861: creat d4/d15/d23/d27/d47/db3/f108 x:0 0 0 2024-04-22T23:35:11.735 INFO:tasks.workunit.client.1.smithi157.stdout:9/862: symlink d4/d15/d55/d103/l109 0 2024-04-22T23:35:11.739 INFO:tasks.workunit.client.1.smithi157.stdout:9/863: rename d4/df/d57/f2b to d4/d15/d23/d27/db5/f10a 0 2024-04-22T23:35:11.743 INFO:tasks.workunit.client.1.smithi157.stdout:9/864: mknod d4/df/d52/c10b 0 2024-04-22T23:35:11.743 INFO:tasks.workunit.client.1.smithi157.stdout:9/865: write d4/d15/d23/d27/db5/f105 [891482,116827] 0 2024-04-22T23:35:11.745 INFO:tasks.workunit.client.1.smithi157.stdout:5/944: dread d1/d5/d7/d2f/d45/fd6 [0,4194304] 0 2024-04-22T23:35:11.745 INFO:tasks.workunit.client.1.smithi157.stdout:0/884: write d17/d23/d34/dee/da8/f10d [3139523,16261] 0 2024-04-22T23:35:11.747 INFO:tasks.workunit.client.1.smithi157.stdout:5/945: unlink d1/d5/d7/f2e 0 2024-04-22T23:35:11.747 INFO:tasks.workunit.client.1.smithi157.stdout:5/946: fsync d1/d27/d29/fdd 0 2024-04-22T23:35:11.748 INFO:tasks.workunit.client.0.smithi107.stdout:9/753: rename d5/d7/d27/d46/f66 to d5/d14/d28/de3/f107 0 2024-04-22T23:35:11.748 INFO:tasks.workunit.client.1.smithi157.stdout:7/882: read d0/f4 [3900961,3511] 0 2024-04-22T23:35:11.748 INFO:tasks.workunit.client.1.smithi157.stdout:5/947: mkdir d1/d27/d29/d154 0 2024-04-22T23:35:11.749 INFO:tasks.workunit.client.0.smithi107.stdout:4/813: symlink d1/d21/d29/d2e/dfa/l120 0 2024-04-22T23:35:11.749 INFO:tasks.workunit.client.0.smithi107.stdout:4/814: stat d1/d21/d52/fec 0 2024-04-22T23:35:11.749 INFO:tasks.workunit.client.1.smithi157.stdout:0/885: creat d17/d2b/d52/dd1/f128 x:0 0 0 2024-04-22T23:35:11.750 INFO:tasks.workunit.client.1.smithi157.stdout:5/948: truncate d1/d5/d7/d2f/d45/d54/f7b 2445828 0 2024-04-22T23:35:11.750 INFO:tasks.workunit.client.1.smithi157.stdout:5/949: readlink d1/d5/d7/d10/l52 0 2024-04-22T23:35:11.752 INFO:tasks.workunit.client.1.smithi157.stdout:9/866: dread d4/d15/d23/d67/f95 [0,4194304] 0 2024-04-22T23:35:11.752 INFO:tasks.workunit.client.0.smithi107.stdout:9/754: unlink d5/d7/d27/d46/d4b/cb4 0 2024-04-22T23:35:11.753 INFO:tasks.workunit.client.1.smithi157.stdout:9/867: mknod d4/df/d57/d6b/dc4/c10c 0 2024-04-22T23:35:11.753 INFO:tasks.workunit.client.1.smithi157.stdout:9/868: fdatasync d4/d15/d23/d67/f93 0 2024-04-22T23:35:11.756 INFO:tasks.workunit.client.0.smithi107.stdout:3/743: rename db/d21/d27/d15/d24/d66/da5/d8f to db/d21/d27/d15/d29/d2a/d45/ddd/da7/dfd 0 2024-04-22T23:35:11.757 INFO:tasks.workunit.client.0.smithi107.stdout:9/755: getdents d5/d7/d17/d1b 0 2024-04-22T23:35:11.759 INFO:tasks.workunit.client.0.smithi107.stdout:1/721: dwrite d7/d19/d20/fa7 [0,4194304] 0 2024-04-22T23:35:11.759 INFO:tasks.workunit.client.0.smithi107.stdout:6/856: dwrite db/d20/d2a/d38/f9f [0,4194304] 0 2024-04-22T23:35:11.761 INFO:tasks.workunit.client.0.smithi107.stdout:5/823: dwrite d9/db/de/f6b [0,4194304] 0 2024-04-22T23:35:11.762 INFO:tasks.workunit.client.1.smithi157.stdout:5/950: dread d1/db0/df3/f12a [0,4194304] 0 2024-04-22T23:35:11.764 INFO:tasks.workunit.client.0.smithi107.stdout:4/815: rename d1/d21/d29/d2e/d7e/f10e to d1/d7/d5b/dbf/da8/f121 0 2024-04-22T23:35:11.767 INFO:tasks.workunit.client.0.smithi107.stdout:3/744: link db/d21/d27/d15/d29/d2a/d45/ddd/lc1 db/d21/d27/d15/d24/d52/d60/dca/dcc/lfe 0 2024-04-22T23:35:11.769 INFO:tasks.workunit.client.0.smithi107.stdout:9/756: link d5/f11 d5/d6c/f108 0 2024-04-22T23:35:11.769 INFO:tasks.workunit.client.1.smithi157.stdout:0/886: rename d17/l7d to d17/d2b/d4f/dc5/dcc/de9/df6/dfd/l129 0 2024-04-22T23:35:11.772 INFO:tasks.workunit.client.0.smithi107.stdout:9/757: read d5/d14/fa2 [104178,94729] 0 2024-04-22T23:35:11.774 INFO:tasks.workunit.client.0.smithi107.stdout:3/745: dread db/d21/d27/d15/d24/d52/d60/fd8 [0,4194304] 0 2024-04-22T23:35:11.775 INFO:tasks.workunit.client.0.smithi107.stdout:1/722: getdents d7/d19/d2d/d89 0 2024-04-22T23:35:11.775 INFO:tasks.workunit.client.0.smithi107.stdout:1/723: readlink d7/d19/d20/l7c 0 2024-04-22T23:35:11.775 INFO:tasks.workunit.client.0.smithi107.stdout:1/724: dread - d7/d19/d20/fc3 zero size 2024-04-22T23:35:11.776 INFO:tasks.workunit.client.1.smithi157.stdout:0/887: mknod d17/d23/d34/d67/d7f/d69/d11c/c12a 0 2024-04-22T23:35:11.776 INFO:tasks.workunit.client.1.smithi157.stdout:0/888: readlink d17/d23/d34/d67/d7f/ld5 0 2024-04-22T23:35:11.776 INFO:tasks.workunit.client.0.smithi107.stdout:6/857: mkdir db/d20/d2a/d38/d59/d106 0 2024-04-22T23:35:11.777 INFO:tasks.workunit.client.0.smithi107.stdout:5/824: symlink d9/d26/d79/l11a 0 2024-04-22T23:35:11.778 INFO:tasks.workunit.client.0.smithi107.stdout:6/858: read db/d20/d2a/dca/fe4 [583602,68335] 0 2024-04-22T23:35:11.779 INFO:tasks.workunit.client.0.smithi107.stdout:6/859: read db/d20/d2c/db5/ff1 [214571,32840] 0 2024-04-22T23:35:11.779 INFO:tasks.workunit.client.0.smithi107.stdout:4/816: creat d1/d21/d29/d2e/d88/d99/dd2/f122 x:0 0 0 2024-04-22T23:35:11.780 INFO:tasks.workunit.client.0.smithi107.stdout:6/860: read db/d20/d2a/d38/d61/d6f/da2/d84/fd6 [88361,19840] 0 2024-04-22T23:35:11.780 INFO:tasks.workunit.client.0.smithi107.stdout:6/861: fsync db/d20/d2a/d38/d4b/f54 0 2024-04-22T23:35:11.782 INFO:tasks.workunit.client.0.smithi107.stdout:3/746: symlink db/d21/d27/d15/d29/d2a/d45/ddd/d25/lff 0 2024-04-22T23:35:11.783 INFO:tasks.workunit.client.0.smithi107.stdout:4/817: dread d1/d7/d53/d66/f79 [0,4194304] 0 2024-04-22T23:35:11.784 INFO:tasks.workunit.client.0.smithi107.stdout:1/725: mkdir d7/d19/d20/d39/d93/df5 0 2024-04-22T23:35:11.784 INFO:tasks.workunit.client.0.smithi107.stdout:4/818: write d1/d7/d5b/dbf/d27/f114 [632333,13529] 0 2024-04-22T23:35:11.785 INFO:tasks.workunit.client.0.smithi107.stdout:5/825: mkdir d9/db/dc1/dcb/de9/d11b 0 2024-04-22T23:35:11.785 INFO:tasks.workunit.client.0.smithi107.stdout:5/826: write d9/db/de/fe5 [1457524,94966] 0 2024-04-22T23:35:11.785 INFO:tasks.workunit.client.0.smithi107.stdout:5/827: fsync d9/db/de/d5e/fe2 0 2024-04-22T23:35:11.785 INFO:tasks.workunit.client.0.smithi107.stdout:5/828: write d9/d76/da0/fd0 [926781,72326] 0 2024-04-22T23:35:11.786 INFO:tasks.workunit.client.0.smithi107.stdout:9/758: truncate d5/d29/f6d 2869769 0 2024-04-22T23:35:11.787 INFO:tasks.workunit.client.0.smithi107.stdout:9/759: write d5/d7/f16 [4636255,18664] 0 2024-04-22T23:35:11.788 INFO:tasks.workunit.client.0.smithi107.stdout:3/747: creat db/d21/d27/d15/d29/d2a/f100 x:0 0 0 2024-04-22T23:35:11.789 INFO:tasks.workunit.client.1.smithi157.stdout:0/889: dread d17/d2b/d52/fa2 [0,4194304] 0 2024-04-22T23:35:11.791 INFO:tasks.workunit.client.1.smithi157.stdout:0/890: mkdir d17/d2b/d4f/dc5/dcc/de9/df6/d12b 0 2024-04-22T23:35:11.791 INFO:tasks.workunit.client.0.smithi107.stdout:4/819: mknod d1/d21/d29/d2e/d88/d99/dd2/c123 0 2024-04-22T23:35:11.792 INFO:tasks.workunit.client.1.smithi157.stdout:0/891: creat d17/d2b/d4f/dc5/dcc/de9/df6/d12b/f12c x:0 0 0 2024-04-22T23:35:11.792 INFO:tasks.workunit.client.1.smithi157.stdout:9/869: dwrite d4/d15/d23/d27/f53 [0,4194304] 0 2024-04-22T23:35:11.792 INFO:tasks.workunit.client.1.smithi157.stdout:7/883: dwrite d0/d9/d29/d33/d37/f50 [0,4194304] 0 2024-04-22T23:35:11.794 INFO:tasks.workunit.client.1.smithi157.stdout:7/884: mknod d0/d9/d42/d55/d80/c10a 0 2024-04-22T23:35:11.795 INFO:tasks.workunit.client.0.smithi107.stdout:5/829: truncate d9/db/f1d 5370996 0 2024-04-22T23:35:11.795 INFO:tasks.workunit.client.0.smithi107.stdout:5/830: write d9/d26/f68 [441889,94595] 0 2024-04-22T23:35:11.795 INFO:tasks.workunit.client.0.smithi107.stdout:5/831: fsync d9/db/f8c 0 2024-04-22T23:35:11.795 INFO:tasks.workunit.client.0.smithi107.stdout:5/832: chown d9/db/dc1/ce8 0 1 2024-04-22T23:35:11.795 INFO:tasks.workunit.client.0.smithi107.stdout:5/833: chown d9/f9d 56516137 1 2024-04-22T23:35:11.795 INFO:tasks.workunit.client.0.smithi107.stdout:4/820: mkdir d1/d7/d63/d78/d96/da0/de9/d124 0 2024-04-22T23:35:11.798 INFO:tasks.workunit.client.1.smithi157.stdout:5/951: dwrite d1/d5/dc/d43/d5e/fe6 [0,4194304] 0 2024-04-22T23:35:11.800 INFO:tasks.workunit.client.0.smithi107.stdout:4/821: rename d1/d7/d63/d78/fd9 to d1/d21/d29/d2e/d88/d99/dd2/daf/dc1/d11f/f125 0 2024-04-22T23:35:11.800 INFO:tasks.workunit.client.0.smithi107.stdout:4/822: fdatasync d1/d7/d53/d66/d6b/fdf 0 2024-04-22T23:35:11.800 INFO:tasks.workunit.client.0.smithi107.stdout:4/823: chown d1/d7/d53/c5a 5707 1 2024-04-22T23:35:11.802 INFO:tasks.workunit.client.0.smithi107.stdout:3/748: rmdir db/d21/d27/d15/d29/d2a/d45/ddd/da7/dfd 0 2024-04-22T23:35:11.802 INFO:tasks.workunit.client.0.smithi107.stdout:3/749: write db/d21/d27/d15/db8/f94 [4871959,106608] 0 2024-04-22T23:35:11.802 INFO:tasks.workunit.client.0.smithi107.stdout:3/750: chown db/d21/d27/d36/cc0 6 1 2024-04-22T23:35:11.803 INFO:tasks.workunit.client.0.smithi107.stdout:1/726: getdents d7/d19/d20/d39/d93 0 2024-04-22T23:35:11.803 INFO:tasks.workunit.client.0.smithi107.stdout:1/727: fsync d7/d19/d20/d39/d53/d43/d46/dba/dc4/fdd 0 2024-04-22T23:35:11.803 INFO:tasks.workunit.client.0.smithi107.stdout:1/728: chown d7/d19/d20/d39/d53/d43/d46/fc9 2 1 2024-04-22T23:35:11.804 INFO:tasks.workunit.client.1.smithi157.stdout:7/885: symlink d0/d9/db/df/d68/l10b 0 2024-04-22T23:35:11.806 INFO:tasks.workunit.client.0.smithi107.stdout:4/824: mknod d1/d21/d29/d2e/c126 0 2024-04-22T23:35:11.806 INFO:tasks.workunit.client.0.smithi107.stdout:5/834: dread d9/d26/f41 [0,4194304] 0 2024-04-22T23:35:11.806 INFO:tasks.workunit.client.1.smithi157.stdout:5/952: unlink d1/d5/d7/d2f/l8e 0 2024-04-22T23:35:11.806 INFO:tasks.workunit.client.1.smithi157.stdout:9/870: dread d4/df/f62 [0,4194304] 0 2024-04-22T23:35:11.807 INFO:tasks.workunit.client.1.smithi157.stdout:9/871: fsync d4/d15/d23/d67/f95 0 2024-04-22T23:35:11.807 INFO:tasks.workunit.client.1.smithi157.stdout:7/886: mkdir d0/d9/db/da3/d10c 0 2024-04-22T23:35:11.808 INFO:tasks.workunit.client.1.smithi157.stdout:5/953: truncate d1/d5/f4e 522577 0 2024-04-22T23:35:11.808 INFO:tasks.workunit.client.0.smithi107.stdout:5/835: creat d9/db/dc1/dcb/de9/f11c x:0 0 0 2024-04-22T23:35:11.810 INFO:tasks.workunit.client.1.smithi157.stdout:9/872: symlink d4/d15/d55/l10d 0 2024-04-22T23:35:11.810 INFO:tasks.workunit.client.1.smithi157.stdout:9/873: chown d4/d15/d23/d27/f33 218890 1 2024-04-22T23:35:11.813 INFO:tasks.workunit.client.0.smithi107.stdout:5/836: mknod d9/de6/c11d 0 2024-04-22T23:35:11.813 INFO:tasks.workunit.client.0.smithi107.stdout:5/837: creat d9/db/de/f11e x:0 0 0 2024-04-22T23:35:11.814 INFO:tasks.workunit.client.0.smithi107.stdout:6/862: dwrite db/d20/d2a/d38/d59/d70/f8f [0,4194304] 0 2024-04-22T23:35:11.815 INFO:tasks.workunit.client.1.smithi157.stdout:7/887: rmdir d0/d9/d42/d55/d56 39 2024-04-22T23:35:11.815 INFO:tasks.workunit.client.1.smithi157.stdout:5/954: creat d1/d5/d7/d10/d14f/dc6/f155 x:0 0 0 2024-04-22T23:35:11.815 INFO:tasks.workunit.client.1.smithi157.stdout:0/892: write d17/d23/d34/d67/d104/fbd [2262303,3191] 0 2024-04-22T23:35:11.815 INFO:tasks.workunit.client.1.smithi157.stdout:7/888: creat d0/d9/d29/d33/d37/f10d x:0 0 0 2024-04-22T23:35:11.816 INFO:tasks.workunit.client.0.smithi107.stdout:5/838: mknod d9/db/d94/c11f 0 2024-04-22T23:35:11.819 INFO:tasks.workunit.client.1.smithi157.stdout:9/874: dread d4/df/f12 [4194304,4194304] 0 2024-04-22T23:35:11.819 INFO:tasks.workunit.client.1.smithi157.stdout:9/875: readlink d4/df/d72/l83 0 2024-04-22T23:35:11.824 INFO:tasks.workunit.client.1.smithi157.stdout:7/889: creat d0/d9/d29/dbd/f10e x:0 0 0 2024-04-22T23:35:11.828 INFO:tasks.workunit.client.1.smithi157.stdout:7/890: getdents d0/d9/d72 0 2024-04-22T23:35:11.828 INFO:tasks.workunit.client.1.smithi157.stdout:7/891: write d0/d9/d29/d33/d37/f10d [429439,53232] 0 2024-04-22T23:35:11.830 INFO:tasks.workunit.client.1.smithi157.stdout:7/892: symlink d0/d9/d29/dbd/l10f 0 2024-04-22T23:35:11.835 INFO:tasks.workunit.client.0.smithi107.stdout:5/839: write d9/db/de/d72/f81 [327678,69355] 0 2024-04-22T23:35:11.837 INFO:tasks.workunit.client.0.smithi107.stdout:5/840: truncate f7 2200018 0 2024-04-22T23:35:11.837 INFO:tasks.workunit.client.0.smithi107.stdout:5/841: creat d9/db/d92/dc2/f120 x:0 0 0 2024-04-22T23:35:11.840 INFO:tasks.workunit.client.1.smithi157.stdout:0/893: write d17/d38/f80 [7298503,130634] 0 2024-04-22T23:35:11.841 INFO:tasks.workunit.client.0.smithi107.stdout:5/842: rename d9/db/d42/f5a to d9/db/d92/dc2/f121 0 2024-04-22T23:35:11.841 INFO:tasks.workunit.client.0.smithi107.stdout:5/843: fdatasync d9/db/d42/d75/fb1 0 2024-04-22T23:35:11.842 INFO:tasks.workunit.client.0.smithi107.stdout:5/844: mkdir d9/db/de/d5e/d122 0 2024-04-22T23:35:11.843 INFO:tasks.workunit.client.0.smithi107.stdout:5/845: write d9/db/de/d72/d9e/fa3 [94171,130971] 0 2024-04-22T23:35:11.843 INFO:tasks.workunit.client.0.smithi107.stdout:5/846: truncate d9/db/d42/fcc 392113 0 2024-04-22T23:35:11.846 INFO:tasks.workunit.client.0.smithi107.stdout:5/847: unlink d9/db/d94/lc4 0 2024-04-22T23:35:11.847 INFO:tasks.workunit.client.0.smithi107.stdout:5/848: mknod d9/db/de/c123 0 2024-04-22T23:35:11.848 INFO:tasks.workunit.client.0.smithi107.stdout:5/849: mkdir d9/db/de/d5e/d124 0 2024-04-22T23:35:11.849 INFO:tasks.workunit.client.0.smithi107.stdout:5/850: read d9/f4e [680884,5861] 0 2024-04-22T23:35:11.849 INFO:tasks.workunit.client.0.smithi107.stdout:5/851: creat d9/db/de/dbe/f125 x:0 0 0 2024-04-22T23:35:11.849 INFO:tasks.workunit.client.0.smithi107.stdout:5/852: chown d9/db/de/d72/d7c/laa 1 1 2024-04-22T23:35:11.849 INFO:tasks.workunit.client.0.smithi107.stdout:5/853: chown d9/f12 28 1 2024-04-22T23:35:11.850 INFO:tasks.workunit.client.0.smithi107.stdout:5/854: creat d9/db/de/d72/d9e/dee/f126 x:0 0 0 2024-04-22T23:35:11.861 INFO:tasks.workunit.client.1.smithi157.stdout:7/893: write d0/d9/d10/f30 [11110691,79859] 0 2024-04-22T23:35:11.861 INFO:tasks.workunit.client.1.smithi157.stdout:7/894: creat d0/d9/db/f110 x:0 0 0 2024-04-22T23:35:11.866 INFO:tasks.workunit.client.0.smithi107.stdout:8/809: sync 2024-04-22T23:35:11.866 INFO:tasks.workunit.client.0.smithi107.stdout:7/764: sync 2024-04-22T23:35:11.866 INFO:tasks.workunit.client.0.smithi107.stdout:0/760: sync 2024-04-22T23:35:11.866 INFO:tasks.workunit.client.0.smithi107.stdout:2/868: sync 2024-04-22T23:35:11.866 INFO:tasks.workunit.client.0.smithi107.stdout:2/869: stat d1/d12/d35/d54 0 2024-04-22T23:35:11.874 INFO:tasks.workunit.client.1.smithi157.stdout:7/895: dread d0/d9/d29/f4d [4194304,4194304] 0 2024-04-22T23:35:11.878 INFO:tasks.workunit.client.1.smithi157.stdout:7/896: rename d0/d9/d72/fc0 to d0/d9/d29/d49/d4a/f111 0 2024-04-22T23:35:11.879 INFO:tasks.workunit.client.1.smithi157.stdout:7/897: creat d0/d9/d42/d55/d80/d95/dce/f112 x:0 0 0 2024-04-22T23:35:11.879 INFO:tasks.workunit.client.1.smithi157.stdout:7/898: creat d0/d9/db/f113 x:0 0 0 2024-04-22T23:35:11.879 INFO:tasks.workunit.client.1.smithi157.stdout:7/899: write d0/d9/f1d [3470181,46130] 0 2024-04-22T23:35:11.881 INFO:tasks.workunit.client.1.smithi157.stdout:7/900: getdents d0/d9/db/df 0 2024-04-22T23:35:11.886 INFO:tasks.workunit.client.0.smithi107.stdout:1/729: dwrite d7/d19/d20/d39/d53/d43/d46/fc9 [0,4194304] 0 2024-04-22T23:35:11.887 INFO:tasks.workunit.client.0.smithi107.stdout:4/825: dwrite d1/d21/d29/d2e/d88/d99/fb0 [4194304,4194304] 0 2024-04-22T23:35:11.887 INFO:tasks.workunit.client.0.smithi107.stdout:1/730: readlink d7/d19/d20/d39/d53/d43/d63/d91/lf2 0 2024-04-22T23:35:11.887 INFO:tasks.workunit.client.0.smithi107.stdout:6/863: dwrite db/f2d [0,4194304] 0 2024-04-22T23:35:11.887 INFO:tasks.workunit.client.0.smithi107.stdout:6/864: chown db/d20/d2c/d3b/f43 10373678 1 2024-04-22T23:35:11.887 INFO:tasks.workunit.client.1.smithi157.stdout:9/876: dwrite d4/d15/d23/d27/d47/db3/fb8 [0,4194304] 0 2024-04-22T23:35:11.888 INFO:tasks.workunit.client.1.smithi157.stdout:5/955: dwrite d1/db0/d7a/f11e [0,4194304] 0 2024-04-22T23:35:11.888 INFO:tasks.workunit.client.1.smithi157.stdout:5/956: write d1/db0/dff/f119 [191820,90534] 0 2024-04-22T23:35:11.888 INFO:tasks.workunit.client.1.smithi157.stdout:9/877: creat d4/d7e/f10e x:0 0 0 2024-04-22T23:35:11.888 INFO:tasks.workunit.client.1.smithi157.stdout:9/878: chown d4/d15/d23/d27/f38 28339103 1 2024-04-22T23:35:11.888 INFO:tasks.workunit.client.1.smithi157.stdout:0/894: dwrite d17/d2b/d4f/dc5/dcc/de9/df6/fd3 [0,4194304] 0 2024-04-22T23:35:11.888 INFO:tasks.workunit.client.1.smithi157.stdout:0/895: fsync d17/d23/fbf 0 2024-04-22T23:35:11.889 INFO:tasks.workunit.client.0.smithi107.stdout:5/855: dwrite d9/f4a [4194304,4194304] 0 2024-04-22T23:35:11.889 INFO:tasks.workunit.client.0.smithi107.stdout:5/856: dread - d9/d76/f117 zero size 2024-04-22T23:35:11.894 INFO:tasks.workunit.client.1.smithi157.stdout:9/879: mknod d4/d15/d23/dad/c10f 0 2024-04-22T23:35:11.896 INFO:tasks.workunit.client.1.smithi157.stdout:5/957: dread d1/d27/d29/fbf [0,4194304] 0 2024-04-22T23:35:11.897 INFO:tasks.workunit.client.1.smithi157.stdout:5/958: chown d1/d5/d7/d2f/d45/cbe 2042 1 2024-04-22T23:35:11.897 INFO:tasks.workunit.client.1.smithi157.stdout:5/959: truncate d1/d27/fbc 616208 0 2024-04-22T23:35:11.900 INFO:tasks.workunit.client.1.smithi157.stdout:0/896: rmdir d17/d23/d34 39 2024-04-22T23:35:11.901 INFO:tasks.workunit.client.1.smithi157.stdout:7/901: dwrite d0/d9/d29/d33/d37/f4c [0,4194304] 0 2024-04-22T23:35:11.902 INFO:tasks.workunit.client.1.smithi157.stdout:7/902: dread - d0/d9/db/d35/f103 zero size 2024-04-22T23:35:11.902 INFO:tasks.workunit.client.1.smithi157.stdout:7/903: chown d0/d9/db/df/d68/dc6/f100 724169 1 2024-04-22T23:35:11.902 INFO:tasks.workunit.client.1.smithi157.stdout:7/904: chown d0/d9/db/d35/la5 41776430 1 2024-04-22T23:35:11.902 INFO:tasks.workunit.client.1.smithi157.stdout:7/905: write d0/d9/db/df/dac/fad [4650107,96905] 0 2024-04-22T23:35:11.902 INFO:tasks.workunit.client.1.smithi157.stdout:7/906: truncate d0/d9/d29/f4d 8714578 0 2024-04-22T23:35:11.902 INFO:tasks.workunit.client.1.smithi157.stdout:9/880: rmdir d4/df/d57/dbd 39 2024-04-22T23:35:11.902 INFO:tasks.workunit.client.1.smithi157.stdout:9/881: chown d4/d15/d23/d27/d47 7872 1 2024-04-22T23:35:11.902 INFO:tasks.workunit.client.1.smithi157.stdout:9/882: chown d4/d2d/cc9 469081 1 2024-04-22T23:35:11.902 INFO:tasks.workunit.client.1.smithi157.stdout:0/897: creat d17/d2b/d4f/dc5/dca/f12d x:0 0 0 2024-04-22T23:35:11.904 INFO:tasks.workunit.client.1.smithi157.stdout:7/907: symlink d0/d9/db/da3/dd5/l114 0 2024-04-22T23:35:11.907 INFO:tasks.workunit.client.0.smithi107.stdout:7/765: rename d7/d21/d48/d5c/f7a to d7/d21/da4/d81/f104 0 2024-04-22T23:35:11.908 INFO:tasks.workunit.client.1.smithi157.stdout:9/883: symlink d4/df/d57/l110 0 2024-04-22T23:35:11.908 INFO:tasks.workunit.client.1.smithi157.stdout:9/884: chown d4/d15/d23/d27/db5 125 1 2024-04-22T23:35:11.908 INFO:tasks.workunit.client.1.smithi157.stdout:9/885: chown d4/df/d57/dbd/dc6/c107 29430939 1 2024-04-22T23:35:11.909 INFO:tasks.workunit.client.1.smithi157.stdout:9/886: write d4/d15/d23/f69 [4672845,54665] 0 2024-04-22T23:35:11.909 INFO:tasks.workunit.client.1.smithi157.stdout:9/887: mknod d4/d15/d23/d27/db5/db9/dcd/c111 0 2024-04-22T23:35:11.912 INFO:tasks.workunit.client.1.smithi157.stdout:8/730: sync 2024-04-22T23:35:11.912 INFO:tasks.workunit.client.1.smithi157.stdout:8/731: truncate d2/d6/f46 5211498 0 2024-04-22T23:35:11.913 INFO:tasks.workunit.client.0.smithi107.stdout:8/810: mknod d0/d73/db0/c11b 0 2024-04-22T23:35:11.913 INFO:tasks.workunit.client.0.smithi107.stdout:0/761: symlink d1/d27/d20/d40/dbd/ddc/lfd 0 2024-04-22T23:35:11.915 INFO:tasks.workunit.client.1.smithi157.stdout:8/732: rename d2/d34/d6a/d77/dfc to d2/d6/d21/d5f/d17/d7f/dfe 0 2024-04-22T23:35:11.915 INFO:tasks.workunit.client.1.smithi157.stdout:8/733: dread - d2/fdb zero size 2024-04-22T23:35:11.918 INFO:tasks.workunit.client.0.smithi107.stdout:2/870: truncate d1/d12/d35/f3f 4358091 0 2024-04-22T23:35:11.918 INFO:tasks.workunit.client.1.smithi157.stdout:8/734: dread d2/d4/d73/fb5 [0,4194304] 0 2024-04-22T23:35:11.921 INFO:tasks.workunit.client.1.smithi157.stdout:8/735: truncate d2/d6/d21/f54 127767 0 2024-04-22T23:35:11.921 INFO:tasks.workunit.client.1.smithi157.stdout:8/736: truncate d2/db1/d58/f92 5137075 0 2024-04-22T23:35:11.922 INFO:tasks.workunit.client.0.smithi107.stdout:0/762: dread d1/d9/f3e [4194304,4194304] 0 2024-04-22T23:35:11.924 INFO:tasks.workunit.client.1.smithi157.stdout:0/898: read d17/f22 [2546163,115511] 0 2024-04-22T23:35:11.933 INFO:tasks.workunit.client.0.smithi107.stdout:6/865: dwrite db/d20/d2a/d38/d4b/f81 [0,4194304] 0 2024-04-22T23:35:11.933 INFO:tasks.workunit.client.0.smithi107.stdout:6/866: chown db/d20/d2a/d5d/ffc 113 1 2024-04-22T23:35:11.935 INFO:tasks.workunit.client.1.smithi157.stdout:5/960: dwrite d1/d5/dc/d43/f44 [0,4194304] 0 2024-04-22T23:35:11.936 INFO:tasks.workunit.client.1.smithi157.stdout:7/908: dwrite d0/d9/f7d [4194304,4194304] 0 2024-04-22T23:35:11.938 INFO:tasks.workunit.client.1.smithi157.stdout:8/737: dread d2/d6/d21/d5f/d11/d13/f6c [0,4194304] 0 2024-04-22T23:35:11.938 INFO:tasks.workunit.client.1.smithi157.stdout:8/738: readlink d2/d6/d21/d5f/d17/l24 0 2024-04-22T23:35:11.939 INFO:tasks.workunit.client.1.smithi157.stdout:9/888: dwrite d4/d15/d23/d27/db5/f10a [0,4194304] 0 2024-04-22T23:35:11.939 INFO:tasks.workunit.client.1.smithi157.stdout:9/889: chown d4/df/f6a 76725030 1 2024-04-22T23:35:11.940 INFO:tasks.workunit.client.1.smithi157.stdout:8/739: rename d2/d4/dbd/f9d to d2/d6/d21/d5f/d11/d13/fff 0 2024-04-22T23:35:11.941 INFO:tasks.workunit.client.1.smithi157.stdout:0/899: write d17/d2b/d4f/dc5/dcc/de9/df6/fd3 [3823907,27429] 0 2024-04-22T23:35:11.941 INFO:tasks.workunit.client.1.smithi157.stdout:0/900: write d17/d23/f27 [1302806,47427] 0 2024-04-22T23:35:11.941 INFO:tasks.workunit.client.1.smithi157.stdout:0/901: dread - d17/d2b/d4f/d5a/d7c/f98 zero size 2024-04-22T23:35:11.941 INFO:tasks.workunit.client.1.smithi157.stdout:0/902: dread - d17/d2b/d4f/dc5/dcc/de9/f11e zero size 2024-04-22T23:35:11.941 INFO:tasks.workunit.client.1.smithi157.stdout:7/909: rmdir d0/d9/d42/d55/d80/d82 39 2024-04-22T23:35:11.941 INFO:tasks.workunit.client.1.smithi157.stdout:7/910: fdatasync d0/d9/db/df/d68/dc6/f100 0 2024-04-22T23:35:11.944 INFO:tasks.workunit.client.1.smithi157.stdout:8/740: mknod d2/d6/d21/d5f/d17/d7f/c100 0 2024-04-22T23:35:11.946 INFO:tasks.workunit.client.1.smithi157.stdout:0/903: symlink d17/d2b/d4f/d5a/dd2/de4/l12e 0 2024-04-22T23:35:11.948 INFO:tasks.workunit.client.1.smithi157.stdout:8/741: creat d2/d4/dbd/dd8/f101 x:0 0 0 2024-04-22T23:35:11.948 INFO:tasks.workunit.client.1.smithi157.stdout:8/742: chown d2/d4/dda/fef 27 1 2024-04-22T23:35:11.950 INFO:tasks.workunit.client.1.smithi157.stdout:7/911: dread d0/d9/d29/d33/d37/f50 [0,4194304] 0 2024-04-22T23:35:11.951 INFO:tasks.workunit.client.1.smithi157.stdout:7/912: write d0/d9/db/f110 [1034385,77351] 0 2024-04-22T23:35:11.951 INFO:tasks.workunit.client.1.smithi157.stdout:7/913: chown d0/d9/d42/c8a 1895 1 2024-04-22T23:35:11.953 INFO:tasks.workunit.client.1.smithi157.stdout:0/904: mkdir d17/d23/d34/d67/d78/dc7/d12f 0 2024-04-22T23:35:11.954 INFO:tasks.workunit.client.1.smithi157.stdout:9/890: dread d4/df/d61/f66 [0,4194304] 0 2024-04-22T23:35:11.956 INFO:tasks.workunit.client.1.smithi157.stdout:9/891: mknod d4/c112 0 2024-04-22T23:35:11.956 INFO:tasks.workunit.client.1.smithi157.stdout:9/892: read - d4/df/d57/d6b/dc4/fe4 zero size 2024-04-22T23:35:11.956 INFO:tasks.workunit.client.1.smithi157.stdout:9/893: chown d4/df/d61/fc1 301348 1 2024-04-22T23:35:11.957 INFO:tasks.workunit.client.1.smithi157.stdout:9/894: symlink d4/df/d61/l113 0 2024-04-22T23:35:11.957 INFO:tasks.workunit.client.1.smithi157.stdout:9/895: creat d4/df/d57/dd0/f114 x:0 0 0 2024-04-22T23:35:11.959 INFO:tasks.workunit.client.1.smithi157.stdout:9/896: truncate d4/df/f6a 299307 0 2024-04-22T23:35:11.961 INFO:tasks.workunit.client.1.smithi157.stdout:9/897: mknod d4/d15/d55/d103/c115 0 2024-04-22T23:35:11.961 INFO:tasks.workunit.client.1.smithi157.stdout:9/898: fdatasync d4/df/f26 0 2024-04-22T23:35:11.961 INFO:tasks.workunit.client.1.smithi157.stdout:9/899: truncate d4/df/d57/dbd/dc6/fdc 1967329 0 2024-04-22T23:35:11.965 INFO:tasks.workunit.client.0.smithi107.stdout:1/731: truncate d7/d19/d20/d39/d53/f5f 2490317 0 2024-04-22T23:35:11.971 INFO:tasks.workunit.client.0.smithi107.stdout:1/732: dread d7/d19/d20/fd6 [0,4194304] 0 2024-04-22T23:35:11.971 INFO:tasks.workunit.client.0.smithi107.stdout:1/733: chown d7/d19/d20/d39/d53/d43/d63/d91 311777 1 2024-04-22T23:35:11.971 INFO:tasks.workunit.client.1.smithi157.stdout:7/914: write d0/d9/db/df/dac/fad [3580757,129011] 0 2024-04-22T23:35:11.972 INFO:tasks.workunit.client.1.smithi157.stdout:0/905: dread d17/d2b/d52/f42 [0,4194304] 0 2024-04-22T23:35:11.973 INFO:tasks.workunit.client.1.smithi157.stdout:7/915: symlink d0/d9/d29/d49/d4a/l115 0 2024-04-22T23:35:11.973 INFO:tasks.workunit.client.1.smithi157.stdout:7/916: chown d0/d9/d42/d55 0 1 2024-04-22T23:35:11.974 INFO:tasks.workunit.client.1.smithi157.stdout:0/906: mknod d17/d2b/d52/dd1/c130 0 2024-04-22T23:35:11.975 INFO:tasks.workunit.client.1.smithi157.stdout:7/917: mkdir d0/d9/d42/d55/d80/d95/df8/d116 0 2024-04-22T23:35:11.975 INFO:tasks.workunit.client.0.smithi107.stdout:5/857: symlink d9/d26/d6a/d100/l127 0 2024-04-22T23:35:11.976 INFO:tasks.workunit.client.1.smithi157.stdout:0/907: rename d17/d2b/d5b/c10e to d17/d2b/d4f/d5a/dd2/d10a/c131 0 2024-04-22T23:35:11.977 INFO:tasks.workunit.client.1.smithi157.stdout:7/918: mknod d0/d9/d29/c117 0 2024-04-22T23:35:11.977 INFO:tasks.workunit.client.1.smithi157.stdout:7/919: fsync d0/d9/d29/d49/d4a/f111 0 2024-04-22T23:35:11.977 INFO:tasks.workunit.client.1.smithi157.stdout:7/920: chown d0/d9/d29/dbd/l10f 83116551 1 2024-04-22T23:35:11.990 INFO:tasks.workunit.client.0.smithi107.stdout:7/766: symlink d7/d21/d48/d5c/d9d/l105 0 2024-04-22T23:35:11.990 INFO:tasks.workunit.client.0.smithi107.stdout:7/767: write d7/d1b/f4d [1345241,43364] 0 2024-04-22T23:35:11.995 INFO:tasks.workunit.client.0.smithi107.stdout:8/811: rename d0/d73/db0/d107/d3b/f9d to d0/d7/d20/df9/f11c 0 2024-04-22T23:35:11.998 INFO:tasks.workunit.client.0.smithi107.stdout:0/763: rmdir d1/d27/d20/d2c/dec 0 2024-04-22T23:35:12.005 INFO:tasks.workunit.client.0.smithi107.stdout:6/867: symlink db/d20/d2a/d5f/l107 0 2024-04-22T23:35:12.005 INFO:tasks.workunit.client.0.smithi107.stdout:6/868: write db/d20/d2a/d38/d61/d6f/da2/fac [4225248,39039] 0 2024-04-22T23:35:12.005 INFO:tasks.workunit.client.0.smithi107.stdout:6/869: fdatasync db/d20/d58/fba 0 2024-04-22T23:35:12.005 INFO:tasks.workunit.client.0.smithi107.stdout:6/870: write db/d20/d2a/d38/d61/d6f/da2/d84/faf [525689,91058] 0 2024-04-22T23:35:12.008 INFO:tasks.workunit.client.0.smithi107.stdout:7/768: mknod d7/d21/d48/d5c/d9d/d6e/db2/de0/c106 0 2024-04-22T23:35:12.011 INFO:tasks.workunit.client.0.smithi107.stdout:2/871: mkdir d1/d2b/d3d/d121 0 2024-04-22T23:35:12.012 INFO:tasks.workunit.client.0.smithi107.stdout:0/764: mknod d1/d27/d20/d40/db4/cfe 0 2024-04-22T23:35:12.013 INFO:tasks.workunit.client.0.smithi107.stdout:6/871: creat db/d20/d2a/dca/f108 x:0 0 0 2024-04-22T23:35:12.014 INFO:tasks.workunit.client.0.smithi107.stdout:2/872: mknod d1/dcd/c122 0 2024-04-22T23:35:12.015 INFO:tasks.workunit.client.0.smithi107.stdout:6/872: symlink db/d20/d2a/d38/d61/d6f/d9b/l109 0 2024-04-22T23:35:12.015 INFO:tasks.workunit.client.0.smithi107.stdout:6/873: chown db/d20/d2a/d38/fc9 236262993 1 2024-04-22T23:35:12.015 INFO:tasks.workunit.client.0.smithi107.stdout:6/874: truncate db/d20/d2a/d5d/ffc 382394 0 2024-04-22T23:35:12.016 INFO:tasks.workunit.client.0.smithi107.stdout:2/873: mkdir d1/d2b/d59/d123 0 2024-04-22T23:35:12.017 INFO:tasks.workunit.client.0.smithi107.stdout:2/874: truncate d1/fcb 728472 0 2024-04-22T23:35:12.018 INFO:tasks.workunit.client.0.smithi107.stdout:2/875: chown d1/d12/d35/d102/d7e/fc8 1664 1 2024-04-22T23:35:12.027 INFO:tasks.workunit.client.0.smithi107.stdout:2/876: rename d1/d12/d35/d102/fff to d1/d12/d35/d102/d7e/da8/f124 0 2024-04-22T23:35:12.027 INFO:tasks.workunit.client.0.smithi107.stdout:2/877: fdatasync d1/d12/d35/d102/f51 0 2024-04-22T23:35:12.029 INFO:tasks.workunit.client.1.smithi157.stdout:8/743: dwrite d2/d4/f10 [0,4194304] 0 2024-04-22T23:35:12.029 INFO:tasks.workunit.client.1.smithi157.stdout:8/744: stat d2/d34/d6a/d77 0 2024-04-22T23:35:12.029 INFO:tasks.workunit.client.1.smithi157.stdout:5/961: dwrite d1/d5/dc/d43/d4f/d7f/dab/ff0 [0,4194304] 0 2024-04-22T23:35:12.029 INFO:tasks.workunit.client.1.smithi157.stdout:5/962: creat d1/d5/dc/d43/d4f/d7f/dab/f156 x:0 0 0 2024-04-22T23:35:12.029 INFO:tasks.workunit.client.1.smithi157.stdout:5/963: readlink d1/db0/d7a/d133/lf8 0 2024-04-22T23:35:12.034 INFO:tasks.workunit.client.1.smithi157.stdout:5/964: mkdir d1/d27/d29/d154/d157 0 2024-04-22T23:35:12.042 INFO:tasks.workunit.client.0.smithi107.stdout:8/812: getdents d0/d7/d20/df9 0 2024-04-22T23:35:12.047 INFO:tasks.workunit.client.0.smithi107.stdout:0/765: dread d1/d9/d14/d4f/d59/fa5 [0,4194304] 0 2024-04-22T23:35:12.051 INFO:tasks.workunit.client.0.smithi107.stdout:8/813: write d0/d73/db0/d107/d3b/f10 [1846829,107267] 0 2024-04-22T23:35:12.051 INFO:tasks.workunit.client.0.smithi107.stdout:8/814: write d0/d73/db0/d107/d3b/f88 [1563490,114395] 0 2024-04-22T23:35:12.067 INFO:tasks.workunit.client.0.smithi107.stdout:0/766: dread d1/d84/f6c [0,4194304] 0 2024-04-22T23:35:12.067 INFO:tasks.workunit.client.0.smithi107.stdout:0/767: write d1/d27/fdd [682726,63237] 0 2024-04-22T23:35:12.068 INFO:tasks.workunit.client.0.smithi107.stdout:0/768: mknod d1/d27/d20/d2c/d41/dbe/cff 0 2024-04-22T23:35:12.068 INFO:tasks.workunit.client.0.smithi107.stdout:0/769: write d1/d27/d20/d2c/d41/dbe/fdf [825462,75782] 0 2024-04-22T23:35:12.069 INFO:tasks.workunit.client.0.smithi107.stdout:0/770: chown d1/d2a/l55 273 1 2024-04-22T23:35:12.069 INFO:tasks.workunit.client.0.smithi107.stdout:0/771: chown d1/d9/d14/f8c 6 1 2024-04-22T23:35:12.069 INFO:tasks.workunit.client.0.smithi107.stdout:0/772: readlink d1/d27/d20/d2c/d41/dbe/ld2 0 2024-04-22T23:35:12.069 INFO:tasks.workunit.client.0.smithi107.stdout:0/773: write d1/d9/f68 [8176239,46760] 0 2024-04-22T23:35:12.070 INFO:tasks.workunit.client.0.smithi107.stdout:0/774: link d1/d84/cc6 d1/d7b/d85/ded/c100 0 2024-04-22T23:35:12.071 INFO:tasks.workunit.client.0.smithi107.stdout:0/775: rmdir d1/d7b/d85 39 2024-04-22T23:35:12.083 INFO:tasks.workunit.client.1.smithi157.stdout:5/965: dread d1/d5/dc/d43/d4f/d7f/dab/ff0 [0,4194304] 0 2024-04-22T23:35:12.096 INFO:tasks.workunit.client.1.smithi157.stdout:9/900: dwrite d4/d15/d23/d27/d47/db3/fec [4194304,4194304] 0 2024-04-22T23:35:12.096 INFO:tasks.workunit.client.1.smithi157.stdout:9/901: truncate d4/d15/d23/d27/d47/f74 4762773 0 2024-04-22T23:35:12.096 INFO:tasks.workunit.client.0.smithi107.stdout:1/734: dwrite d7/d19/d2d/f84 [4194304,4194304] 0 2024-04-22T23:35:12.107 INFO:tasks.workunit.client.0.smithi107.stdout:9/760: sync 2024-04-22T23:35:12.107 INFO:tasks.workunit.client.0.smithi107.stdout:9/761: chown d5/d29/d7b/d91/dde/dc9/daf 0 1 2024-04-22T23:35:12.108 INFO:tasks.workunit.client.0.smithi107.stdout:9/762: getdents d5/d7/d17/d1b/d1f 0 2024-04-22T23:35:12.109 INFO:tasks.workunit.client.0.smithi107.stdout:9/763: fsync d5/d29/f4d 0 2024-04-22T23:35:12.110 INFO:tasks.workunit.client.1.smithi157.stdout:0/908: dwrite d17/d2b/d52/fa2 [0,4194304] 0 2024-04-22T23:35:12.110 INFO:tasks.workunit.client.1.smithi157.stdout:0/909: write d17/d38/f120 [317071,73183] 0 2024-04-22T23:35:12.111 INFO:tasks.workunit.client.0.smithi107.stdout:9/764: symlink d5/d29/d7b/l109 0 2024-04-22T23:35:12.111 INFO:tasks.workunit.client.0.smithi107.stdout:6/875: dwrite db/d20/d2a/d38/f53 [0,4194304] 0 2024-04-22T23:35:12.111 INFO:tasks.workunit.client.0.smithi107.stdout:6/876: fdatasync db/d20/d58/fba 0 2024-04-22T23:35:12.111 INFO:tasks.workunit.client.0.smithi107.stdout:5/858: dwrite d9/db/d92/dc2/ff9 [0,4194304] 0 2024-04-22T23:35:12.111 INFO:tasks.workunit.client.1.smithi157.stdout:8/745: dwrite d2/d6/d21/d5f/d11/d13/fe8 [0,4194304] 0 2024-04-22T23:35:12.114 INFO:tasks.workunit.client.0.smithi107.stdout:5/859: truncate d9/d76/da0/fbf 647911 0 2024-04-22T23:35:12.121 INFO:tasks.workunit.client.0.smithi107.stdout:9/765: dread d5/d7/d17/f4e [0,4194304] 0 2024-04-22T23:35:12.122 INFO:tasks.workunit.client.1.smithi157.stdout:0/910: unlink fe 0 2024-04-22T23:35:12.122 INFO:tasks.workunit.client.1.smithi157.stdout:8/746: dread d2/d4/d30/f62 [0,4194304] 0 2024-04-22T23:35:12.122 INFO:tasks.workunit.client.0.smithi107.stdout:6/877: write db/d20/d2c/f32 [2124689,118856] 0 2024-04-22T23:35:12.122 INFO:tasks.workunit.client.0.smithi107.stdout:6/878: fsync db/d20/d2a/d38/f3c 0 2024-04-22T23:35:12.124 INFO:tasks.workunit.client.0.smithi107.stdout:8/815: dwrite d0/d73/db0/d107/d33/d37/f80 [4194304,4194304] 0 2024-04-22T23:35:12.124 INFO:tasks.workunit.client.0.smithi107.stdout:2/878: dwrite d1/d12/d35/d102/d7e/da8/fb8 [0,4194304] 0 2024-04-22T23:35:12.125 INFO:tasks.workunit.client.0.smithi107.stdout:6/879: symlink db/d20/d2c/d3b/d40/l10a 0 2024-04-22T23:35:12.125 INFO:tasks.workunit.client.0.smithi107.stdout:9/766: creat d5/d7/d17/de0/f10a x:0 0 0 2024-04-22T23:35:12.125 INFO:tasks.workunit.client.0.smithi107.stdout:6/880: creat db/d20/d2c/d3b/d40/d9e/f10b x:0 0 0 2024-04-22T23:35:12.126 INFO:tasks.workunit.client.1.smithi157.stdout:0/911: mknod d17/d23/d34/d67/d104/c132 0 2024-04-22T23:35:12.126 INFO:tasks.workunit.client.1.smithi157.stdout:9/902: dwrite d4/df/d61/fc1 [0,4194304] 0 2024-04-22T23:35:12.127 INFO:tasks.workunit.client.0.smithi107.stdout:8/816: unlink d0/d85/d51/l58 0 2024-04-22T23:35:12.127 INFO:tasks.workunit.client.0.smithi107.stdout:0/776: dwrite d1/d84/fc2 [0,4194304] 0 2024-04-22T23:35:12.127 INFO:tasks.workunit.client.0.smithi107.stdout:1/735: dwrite d7/f15 [0,4194304] 0 2024-04-22T23:35:12.127 INFO:tasks.workunit.client.0.smithi107.stdout:1/736: chown d7/d19/d2d/da4 66830062 1 2024-04-22T23:35:12.130 INFO:tasks.workunit.client.0.smithi107.stdout:9/767: creat d5/d7/d27/dfc/f10b x:0 0 0 2024-04-22T23:35:12.133 INFO:tasks.workunit.client.1.smithi157.stdout:9/903: link d4/df/f62 d4/d15/d23/d27/dc0/f116 0 2024-04-22T23:35:12.134 INFO:tasks.workunit.client.1.smithi157.stdout:5/966: rmdir d1/d5/dc/d43/d4f/d7f/dab 39 2024-04-22T23:35:12.140 INFO:tasks.workunit.client.1.smithi157.stdout:5/967: write d1/d5/d7/f67 [2030963,109495] 0 2024-04-22T23:35:12.143 INFO:tasks.workunit.client.1.smithi157.stdout:5/968: truncate d1/d5/f14 7396025 0 2024-04-22T23:35:12.144 INFO:tasks.workunit.client.0.smithi107.stdout:0/777: creat d1/d9/d14/d4f/f101 x:0 0 0 2024-04-22T23:35:12.145 INFO:tasks.workunit.client.0.smithi107.stdout:1/737: symlink d7/d19/d42/ded/lf6 0 2024-04-22T23:35:12.145 INFO:tasks.workunit.client.1.smithi157.stdout:5/969: mkdir d1/d5/d158 0 2024-04-22T23:35:12.146 INFO:tasks.workunit.client.1.smithi157.stdout:0/912: write d17/d23/d34/dee/f114 [2125645,109980] 0 2024-04-22T23:35:12.146 INFO:tasks.workunit.client.1.smithi157.stdout:0/913: chown d17/d23/f32 8125701 1 2024-04-22T23:35:12.147 INFO:tasks.workunit.client.0.smithi107.stdout:0/778: symlink d1/d27/d8a/db5/l102 0 2024-04-22T23:35:12.148 INFO:tasks.workunit.client.1.smithi157.stdout:5/970: truncate d1/fe 2554390 0 2024-04-22T23:35:12.150 INFO:tasks.workunit.client.1.smithi157.stdout:0/914: mknod d17/d2b/d4f/d5a/c133 0 2024-04-22T23:35:12.153 INFO:tasks.workunit.client.1.smithi157.stdout:5/971: getdents d1/d5/d104 0 2024-04-22T23:35:12.153 INFO:tasks.workunit.client.1.smithi157.stdout:0/915: mkdir d17/d2b/d52/d134 0 2024-04-22T23:35:12.154 INFO:tasks.workunit.client.0.smithi107.stdout:1/738: rmdir d7/d19/d20/d39/d53/d43/d46/dba/dc4 39 2024-04-22T23:35:12.154 INFO:tasks.workunit.client.0.smithi107.stdout:1/739: write d7/d19/d20/fc3 [971971,94483] 0 2024-04-22T23:35:12.154 INFO:tasks.workunit.client.0.smithi107.stdout:1/740: chown d7/d19/d20/d39/d53/d43/d46/d99/fd2 56 1 2024-04-22T23:35:12.154 INFO:tasks.workunit.client.1.smithi157.stdout:5/972: dread d1/d5/d7/d2f/d45/fd6 [0,4194304] 0 2024-04-22T23:35:12.154 INFO:tasks.workunit.client.1.smithi157.stdout:5/973: dread - d1/d5/dc/d5d/f14d zero size 2024-04-22T23:35:12.154 INFO:tasks.workunit.client.1.smithi157.stdout:0/916: rmdir d17/d2b/d4f/dc5/dcc/de9/df6/dfd 39 2024-04-22T23:35:12.156 INFO:tasks.workunit.client.1.smithi157.stdout:0/917: creat d17/d2b/d4f/d5a/dd2/f135 x:0 0 0 2024-04-22T23:35:12.157 INFO:tasks.workunit.client.0.smithi107.stdout:1/741: unlink d7/d19/d2d/lbc 0 2024-04-22T23:35:12.157 INFO:tasks.workunit.client.0.smithi107.stdout:1/742: write d7/d19/d20/d39/d53/fe2 [2267747,63807] 0 2024-04-22T23:35:12.157 INFO:tasks.workunit.client.0.smithi107.stdout:1/743: dread - d7/d19/d2d/d89/fb5 zero size 2024-04-22T23:35:12.163 INFO:tasks.workunit.client.0.smithi107.stdout:0/779: write d1/f15 [2419240,118513] 0 2024-04-22T23:35:12.164 INFO:tasks.workunit.client.1.smithi157.stdout:0/918: write d17/d2b/d5b/ff4 [368035,92582] 0 2024-04-22T23:35:12.167 INFO:tasks.workunit.client.0.smithi107.stdout:0/780: getdents d1/d84/d99 0 2024-04-22T23:35:12.169 INFO:tasks.workunit.client.0.smithi107.stdout:0/781: symlink d1/d27/d8a/d8b/l103 0 2024-04-22T23:35:12.169 INFO:tasks.workunit.client.0.smithi107.stdout:0/782: truncate d1/d9/d14/d4f/fa2 136611 0 2024-04-22T23:35:12.169 INFO:tasks.workunit.client.0.smithi107.stdout:0/783: truncate d1/d27/d20/d4a/fe9 2752 0 2024-04-22T23:35:12.169 INFO:tasks.workunit.client.0.smithi107.stdout:0/784: dread - d1/d9/ff0 zero size 2024-04-22T23:35:12.169 INFO:tasks.workunit.client.0.smithi107.stdout:0/785: chown d1/d2a/lb8 42 1 2024-04-22T23:35:12.172 INFO:tasks.workunit.client.1.smithi157.stdout:0/919: mkdir d17/d23/d34/d67/d136 0 2024-04-22T23:35:12.173 INFO:tasks.workunit.client.0.smithi107.stdout:4/826: sync 2024-04-22T23:35:12.173 INFO:tasks.workunit.client.0.smithi107.stdout:3/751: sync 2024-04-22T23:35:12.176 INFO:tasks.workunit.client.0.smithi107.stdout:3/752: getdents db/d21/d27/d15/db8 0 2024-04-22T23:35:12.179 INFO:tasks.workunit.client.0.smithi107.stdout:3/753: unlink db/d21/c5a 0 2024-04-22T23:35:12.179 INFO:tasks.workunit.client.0.smithi107.stdout:3/754: dread - db/d21/fd9 zero size 2024-04-22T23:35:12.190 INFO:tasks.workunit.client.1.smithi157.stdout:8/747: dwrite d2/d6/d21/d5f/d17/f49 [0,4194304] 0 2024-04-22T23:35:12.191 INFO:tasks.workunit.client.0.smithi107.stdout:4/827: dread d1/d1b/d44/fbe [4194304,4194304] 0 2024-04-22T23:35:12.191 INFO:tasks.workunit.client.0.smithi107.stdout:4/828: creat d1/f127 x:0 0 0 2024-04-22T23:35:12.192 INFO:tasks.workunit.client.1.smithi157.stdout:8/748: creat d2/db1/f102 x:0 0 0 2024-04-22T23:35:12.193 INFO:tasks.workunit.client.0.smithi107.stdout:4/829: mknod d1/d1b/c128 0 2024-04-22T23:35:12.193 INFO:tasks.workunit.client.0.smithi107.stdout:4/830: write d1/d7/d63/d78/da4/fd0 [1442123,45661] 0 2024-04-22T23:35:12.194 INFO:tasks.workunit.client.1.smithi157.stdout:8/749: creat d2/f103 x:0 0 0 2024-04-22T23:35:12.194 INFO:tasks.workunit.client.1.smithi157.stdout:8/750: creat d2/d4/dbd/dd8/f104 x:0 0 0 2024-04-22T23:35:12.194 INFO:tasks.workunit.client.1.smithi157.stdout:8/751: dread - d2/f103 zero size 2024-04-22T23:35:12.195 INFO:tasks.workunit.client.0.smithi107.stdout:4/831: creat d1/d21/d29/d110/f129 x:0 0 0 2024-04-22T23:35:12.195 INFO:tasks.workunit.client.0.smithi107.stdout:4/832: readlink d1/d7/d63/d78/d96/da0/lc5 0 2024-04-22T23:35:12.196 INFO:tasks.workunit.client.1.smithi157.stdout:8/752: rename d2/d6/d6b/ccc to d2/d34/d6a/c105 0 2024-04-22T23:35:12.196 INFO:tasks.workunit.client.0.smithi107.stdout:4/833: rename d1/d7/d63/d78/d96/fa6 to d1/d21/d29/d2e/d3b/f12a 0 2024-04-22T23:35:12.196 INFO:tasks.workunit.client.1.smithi157.stdout:8/753: mkdir d2/d4/dad/d106 0 2024-04-22T23:35:12.197 INFO:tasks.workunit.client.0.smithi107.stdout:4/834: rmdir d1/d7/d63/d78/d111 0 2024-04-22T23:35:12.198 INFO:tasks.workunit.client.0.smithi107.stdout:4/835: mknod d1/d21/d29/d2e/d88/d99/c12b 0 2024-04-22T23:35:12.210 INFO:tasks.workunit.client.0.smithi107.stdout:2/879: dwrite d1/d12/d35/d54/fd5 [0,4194304] 0 2024-04-22T23:35:12.210 INFO:tasks.workunit.client.0.smithi107.stdout:2/880: truncate d1/f4c 750100 0 2024-04-22T23:35:12.212 INFO:tasks.workunit.client.0.smithi107.stdout:2/881: mkdir d1/d12/d35/d102/d7e/da8/d125 0 2024-04-22T23:35:12.232 INFO:tasks.workunit.client.0.smithi107.stdout:4/836: dread d1/f28 [0,4194304] 0 2024-04-22T23:35:12.232 INFO:tasks.workunit.client.0.smithi107.stdout:4/837: readlink d1/d21/d29/d2e/d88/d99/dd2/daf/dc1/l77 0 2024-04-22T23:35:12.233 INFO:tasks.workunit.client.0.smithi107.stdout:4/838: getdents d1/d7/d63/d78/da4 0 2024-04-22T23:35:12.234 INFO:tasks.workunit.client.0.smithi107.stdout:4/839: mkdir d1/d21/d29/d2e/d88/d99/d12c 0 2024-04-22T23:35:12.236 INFO:tasks.workunit.client.0.smithi107.stdout:4/840: symlink d1/d21/d29/d2e/d88/d99/d12c/l12d 0 2024-04-22T23:35:12.236 INFO:tasks.workunit.client.0.smithi107.stdout:4/841: truncate d1/d7/f32 6539345 0 2024-04-22T23:35:12.237 INFO:tasks.workunit.client.0.smithi107.stdout:4/842: symlink d1/d21/d29/d2e/d88/d99/l12e 0 2024-04-22T23:35:12.238 INFO:tasks.workunit.client.1.smithi157.stdout:8/754: dread d2/d6/d21/d5f/d11/f4b [0,4194304] 0 2024-04-22T23:35:12.239 INFO:tasks.workunit.client.1.smithi157.stdout:8/755: creat d2/d4/d30/f107 x:0 0 0 2024-04-22T23:35:12.239 INFO:tasks.workunit.client.1.smithi157.stdout:8/756: fdatasync d2/d4/dbd/fb4 0 2024-04-22T23:35:12.241 INFO:tasks.workunit.client.1.smithi157.stdout:8/757: creat d2/d4/f108 x:0 0 0 2024-04-22T23:35:12.241 INFO:tasks.workunit.client.1.smithi157.stdout:8/758: fsync d2/d6/f3c 0 2024-04-22T23:35:12.242 INFO:tasks.workunit.client.1.smithi157.stdout:8/759: creat d2/d4/dbd/d67/f109 x:0 0 0 2024-04-22T23:35:12.243 INFO:tasks.workunit.client.1.smithi157.stdout:8/760: symlink d2/d4/dbd/l10a 0 2024-04-22T23:35:12.244 INFO:tasks.workunit.client.0.smithi107.stdout:6/881: dwrite db/d20/d2c/f44 [0,4194304] 0 2024-04-22T23:35:12.245 INFO:tasks.workunit.client.0.smithi107.stdout:6/882: link db/d20/d2a/d38/d61/d6f/da2/d84/fd6 db/d20/d2a/f10c 0 2024-04-22T23:35:12.256 INFO:tasks.workunit.client.1.smithi157.stdout:9/904: dwrite d4/d15/d23/d27/ff1 [0,4194304] 0 2024-04-22T23:35:12.256 INFO:tasks.workunit.client.0.smithi107.stdout:8/817: dwrite d0/d73/db0/d107/d3b/f2b [0,4194304] 0 2024-04-22T23:35:12.256 INFO:tasks.workunit.client.0.smithi107.stdout:8/818: truncate d0/d73/db0/d107/d33/faa 1493721 0 2024-04-22T23:35:12.256 INFO:tasks.workunit.client.0.smithi107.stdout:9/768: dwrite d5/d7/d17/d1b/d1f/feb [0,4194304] 0 2024-04-22T23:35:12.260 INFO:tasks.workunit.client.0.smithi107.stdout:8/819: dread d0/d73/db0/fbc [0,4194304] 0 2024-04-22T23:35:12.262 INFO:tasks.workunit.client.1.smithi157.stdout:9/905: dread d4/d15/fc5 [0,4194304] 0 2024-04-22T23:35:12.262 INFO:tasks.workunit.client.0.smithi107.stdout:8/820: creat d0/d73/db0/d107/d3b/dd9/f11d x:0 0 0 2024-04-22T23:35:12.262 INFO:tasks.workunit.client.0.smithi107.stdout:8/821: fsync d0/d73/db0/d107/d33/d9b/df3/f112 0 2024-04-22T23:35:12.263 INFO:tasks.workunit.client.1.smithi157.stdout:9/906: truncate d4/df/fb 2329930 0 2024-04-22T23:35:12.263 INFO:tasks.workunit.client.1.smithi157.stdout:8/761: write d2/d6/d21/d5f/d11/f4a [2824432,125929] 0 2024-04-22T23:35:12.263 INFO:tasks.workunit.client.1.smithi157.stdout:8/762: chown d2/d34/d6a/d77/cd9 362 1 2024-04-22T23:35:12.264 INFO:tasks.workunit.client.0.smithi107.stdout:8/822: getdents d0/d3a/d46 0 2024-04-22T23:35:12.264 INFO:tasks.workunit.client.0.smithi107.stdout:8/823: write d0/d85/d51/f101 [631184,98048] 0 2024-04-22T23:35:12.264 INFO:tasks.workunit.client.1.smithi157.stdout:9/907: creat d4/df/f117 x:0 0 0 2024-04-22T23:35:12.264 INFO:tasks.workunit.client.1.smithi157.stdout:9/908: write d4/df/d57/dbd/dc6/ffa [649231,93564] 0 2024-04-22T23:35:12.266 INFO:tasks.workunit.client.0.smithi107.stdout:8/824: rename d0/d7/f5e to d0/d7/d20/df9/f11e 0 2024-04-22T23:35:12.266 INFO:tasks.workunit.client.1.smithi157.stdout:8/763: link d2/d4/dbd/d67/cb9 d2/d4/dad/c10b 0 2024-04-22T23:35:12.268 INFO:tasks.workunit.client.0.smithi107.stdout:8/825: creat d0/d7/d20/de4/f11f x:0 0 0 2024-04-22T23:35:12.268 INFO:tasks.workunit.client.0.smithi107.stdout:8/826: dread - d0/d73/db0/d107/d33/d37/d9f/feb zero size 2024-04-22T23:35:12.268 INFO:tasks.workunit.client.1.smithi157.stdout:8/764: symlink d2/d4/dad/l10c 0 2024-04-22T23:35:12.269 INFO:tasks.workunit.client.0.smithi107.stdout:8/827: unlink d0/d73/cc2 0 2024-04-22T23:35:12.269 INFO:tasks.workunit.client.0.smithi107.stdout:8/828: chown d0 10 1 2024-04-22T23:35:12.270 INFO:tasks.workunit.client.0.smithi107.stdout:8/829: mknod d0/d3a/d46/d59/c120 0 2024-04-22T23:35:12.270 INFO:tasks.workunit.client.0.smithi107.stdout:9/769: dread d5/d7/d27/d46/f88 [4194304,4194304] 0 2024-04-22T23:35:12.271 INFO:tasks.workunit.client.0.smithi107.stdout:9/770: creat d5/d14/d28/f10c x:0 0 0 2024-04-22T23:35:12.274 INFO:tasks.workunit.client.0.smithi107.stdout:9/771: link d5/d7/l85 d5/d7/d27/d46/d4b/def/l10d 0 2024-04-22T23:35:12.274 INFO:tasks.workunit.client.0.smithi107.stdout:9/772: write d5/d24/f77 [839653,16698] 0 2024-04-22T23:35:12.275 INFO:tasks.workunit.client.0.smithi107.stdout:9/773: mknod d5/d14/d2e/d4f/d83/c10e 0 2024-04-22T23:35:12.276 INFO:tasks.workunit.client.0.smithi107.stdout:9/774: getdents d5/d7/d27 0 2024-04-22T23:35:12.279 INFO:tasks.workunit.client.1.smithi157.stdout:8/765: write d2/d4/d73/fc7 [3789340,57866] 0 2024-04-22T23:35:12.279 INFO:tasks.workunit.client.1.smithi157.stdout:8/766: chown d2/d6/d21/d5f/d17/dc2 507771324 1 2024-04-22T23:35:12.283 INFO:tasks.workunit.client.1.smithi157.stdout:8/767: symlink d2/d6/d21/d5f/d17/d7f/l10d 0 2024-04-22T23:35:12.284 INFO:tasks.workunit.client.1.smithi157.stdout:8/768: unlink d2/fa9 0 2024-04-22T23:35:12.284 INFO:tasks.workunit.client.0.smithi107.stdout:9/775: truncate d5/d29/f3a 966509 0 2024-04-22T23:35:12.288 INFO:tasks.workunit.client.0.smithi107.stdout:6/883: dwrite db/d20/d2c/d3b/fa8 [0,4194304] 0 2024-04-22T23:35:12.288 INFO:tasks.workunit.client.0.smithi107.stdout:6/884: chown db/d20/d2a/d38/d4b/c66 0 1 2024-04-22T23:35:12.289 INFO:tasks.workunit.client.0.smithi107.stdout:2/882: dwrite d1/d67/f9f [0,4194304] 0 2024-04-22T23:35:12.293 INFO:tasks.workunit.client.1.smithi157.stdout:5/974: dwrite d1/d5/d7/d2f/d45/d54/f125 [0,4194304] 0 2024-04-22T23:35:12.293 INFO:tasks.workunit.client.1.smithi157.stdout:8/769: write d2/d6/d21/d5f/d11/f4b [3755768,46704] 0 2024-04-22T23:35:12.315 INFO:tasks.workunit.client.0.smithi107.stdout:1/744: dwrite d7/d19/d20/d39/d53/d43/d46/d5c/dac/fea [0,4194304] 0 2024-04-22T23:35:12.316 INFO:tasks.workunit.client.1.smithi157.stdout:0/920: dwrite fc [4194304,4194304] 0 2024-04-22T23:35:12.316 INFO:tasks.workunit.client.1.smithi157.stdout:0/921: readlink d17/d2b/d4f/dc5/dcc/de9/df6/l8c 0 2024-04-22T23:35:12.316 INFO:tasks.workunit.client.0.smithi107.stdout:0/786: dwrite d1/d27/d20/d2c/d2e/f39 [0,4194304] 0 2024-04-22T23:35:12.317 INFO:tasks.workunit.client.0.smithi107.stdout:1/745: rename d7/d19/d20/d39/d93/df4/db8/fe8 to d7/d19/d20/d39/d53/ff7 0 2024-04-22T23:35:12.317 INFO:tasks.workunit.client.0.smithi107.stdout:1/746: fdatasync d7/d19/d2d/d89/fb1 0 2024-04-22T23:35:12.318 INFO:tasks.workunit.client.0.smithi107.stdout:3/755: dwrite db/d21/fda [0,4194304] 0 2024-04-22T23:35:12.319 INFO:tasks.workunit.client.0.smithi107.stdout:4/843: dwrite d1/d21/d29/d2e/d88/d99/dd2/daf/dc1/d11f/ff6 [0,4194304] 0 2024-04-22T23:35:12.323 INFO:tasks.workunit.client.0.smithi107.stdout:0/787: dread d1/d9/d14/d4f/fe1 [0,4194304] 0 2024-04-22T23:35:12.324 INFO:tasks.workunit.client.1.smithi157.stdout:0/922: dread d17/f25 [0,4194304] 0 2024-04-22T23:35:12.325 INFO:tasks.workunit.client.0.smithi107.stdout:1/747: dread d7/f10 [0,4194304] 0 2024-04-22T23:35:12.325 INFO:tasks.workunit.client.0.smithi107.stdout:0/788: symlink d1/d9/d14/d4f/l104 0 2024-04-22T23:35:12.325 INFO:tasks.workunit.client.0.smithi107.stdout:0/789: chown d1/d9/d14/d4f/fe1 0 1 2024-04-22T23:35:12.325 INFO:tasks.workunit.client.1.smithi157.stdout:5/975: write d1/db0/d7a/f87 [8384244,88644] 0 2024-04-22T23:35:12.325 INFO:tasks.workunit.client.1.smithi157.stdout:0/923: mknod d17/d2b/d4f/dc5/dcc/de9/c137 0 2024-04-22T23:35:12.326 INFO:tasks.workunit.client.1.smithi157.stdout:5/976: creat d1/db0/f159 x:0 0 0 2024-04-22T23:35:12.326 INFO:tasks.workunit.client.0.smithi107.stdout:1/748: truncate d7/d19/d20/d39/d53/f5f 2259285 0 2024-04-22T23:35:12.327 INFO:tasks.workunit.client.0.smithi107.stdout:0/790: rmdir d1/d7b/d85 39 2024-04-22T23:35:12.328 INFO:tasks.workunit.client.1.smithi157.stdout:5/977: unlink d1/d27/d29/fc2 0 2024-04-22T23:35:12.329 INFO:tasks.workunit.client.0.smithi107.stdout:9/776: dwrite d5/d29/f4d [0,4194304] 0 2024-04-22T23:35:12.329 INFO:tasks.workunit.client.0.smithi107.stdout:1/749: mkdir d7/d19/d20/d39/d93/df4/db8/df8 0 2024-04-22T23:35:12.331 INFO:tasks.workunit.client.0.smithi107.stdout:0/791: mkdir d1/d27/d8a/db5/d105 0 2024-04-22T23:35:12.333 INFO:tasks.workunit.client.1.smithi157.stdout:0/924: dread d17/d23/fda [0,4194304] 0 2024-04-22T23:35:12.333 INFO:tasks.workunit.client.0.smithi107.stdout:3/756: write db/d21/d27/d15/d29/d2a/d45/ddd/d25/fe2 [3759159,8043] 0 2024-04-22T23:35:12.334 INFO:tasks.workunit.client.1.smithi157.stdout:8/770: dwrite d2/d34/d6a/d77/f9f [0,4194304] 0 2024-04-22T23:35:12.334 INFO:tasks.workunit.client.0.smithi107.stdout:9/777: chown d5/d7/d17/d1b/d1f/d36/fb8 1694231 1 2024-04-22T23:35:12.342 INFO:tasks.workunit.client.0.smithi107.stdout:1/750: symlink d7/d19/d20/d39/d53/d43/d46/d5c/db7/lf9 0 2024-04-22T23:35:12.342 INFO:tasks.workunit.client.0.smithi107.stdout:7/769: sync 2024-04-22T23:35:12.342 INFO:tasks.workunit.client.0.smithi107.stdout:0/792: symlink d1/d27/d20/d2c/d41/l106 0 2024-04-22T23:35:12.342 INFO:tasks.workunit.client.0.smithi107.stdout:2/883: dwrite d1/d12/d35/d48/f9d [0,4194304] 0 2024-04-22T23:35:12.342 INFO:tasks.workunit.client.0.smithi107.stdout:2/884: chown d1/d12/d35/d102/d87/c8d 1152989 1 2024-04-22T23:35:12.342 INFO:tasks.workunit.client.0.smithi107.stdout:3/757: creat db/d21/d27/d15/d24/d52/d60/dca/dfc/f101 x:0 0 0 2024-04-22T23:35:12.342 INFO:tasks.workunit.client.0.smithi107.stdout:4/844: write d1/d21/d29/d2e/d88/d99/dd2/f7f [1216061,86154] 0 2024-04-22T23:35:12.342 INFO:tasks.workunit.client.0.smithi107.stdout:6/885: dwrite db/d20/d2c/d3b/f45 [0,4194304] 0 2024-04-22T23:35:12.343 INFO:tasks.workunit.client.0.smithi107.stdout:1/751: rename d7/d19/d54/fdb to d7/d19/d20/d39/d53/ffa 0 2024-04-22T23:35:12.343 INFO:tasks.workunit.client.1.smithi157.stdout:8/771: mknod d2/d6/d21/d5f/d17/dc8/c10e 0 2024-04-22T23:35:12.343 INFO:tasks.workunit.client.1.smithi157.stdout:8/772: stat d2/d6/d21/f98 0 2024-04-22T23:35:12.343 INFO:tasks.workunit.client.1.smithi157.stdout:8/773: mkdir d2/d4/dbd/d67/d10f 0 2024-04-22T23:35:12.343 INFO:tasks.workunit.client.1.smithi157.stdout:8/774: creat d2/db1/d58/f110 x:0 0 0 2024-04-22T23:35:12.343 INFO:tasks.workunit.client.1.smithi157.stdout:8/775: chown d2/d6/d6b/fe3 1 1 2024-04-22T23:35:12.344 INFO:tasks.workunit.client.1.smithi157.stdout:8/776: readlink d2/d6/d21/d5f/d11/le7 0 2024-04-22T23:35:12.344 INFO:tasks.workunit.client.1.smithi157.stdout:8/777: chown d2/lb0 14 1 2024-04-22T23:35:12.344 INFO:tasks.workunit.client.1.smithi157.stdout:8/778: stat d2/d4/dda/fef 0 2024-04-22T23:35:12.345 INFO:tasks.workunit.client.0.smithi107.stdout:0/793: mknod d1/d27/d20/d40/dbd/df2/c107 0 2024-04-22T23:35:12.345 INFO:tasks.workunit.client.0.smithi107.stdout:0/794: chown d1/d9/d14/d4f/ld8 11548530 1 2024-04-22T23:35:12.350 INFO:tasks.workunit.client.0.smithi107.stdout:7/770: dread d7/d21/f22 [0,4194304] 0 2024-04-22T23:35:12.350 INFO:tasks.workunit.client.0.smithi107.stdout:7/771: stat d7/d4c/da1/fd9 0 2024-04-22T23:35:12.350 INFO:tasks.workunit.client.0.smithi107.stdout:7/772: chown d7/d21/da4/c3d 2358947 1 2024-04-22T23:35:12.350 INFO:tasks.workunit.client.0.smithi107.stdout:2/885: getdents d1/d12/d35/d102/d7e/da8 0 2024-04-22T23:35:12.354 INFO:tasks.workunit.client.1.smithi157.stdout:8/779: dread d2/d34/d6a/d77/f9f [0,4194304] 0 2024-04-22T23:35:12.354 INFO:tasks.workunit.client.1.smithi157.stdout:8/780: dread - d2/d4/dbd/ff3 zero size 2024-04-22T23:35:12.354 INFO:tasks.workunit.client.0.smithi107.stdout:1/752: dread d7/f67 [4194304,4194304] 0 2024-04-22T23:35:12.355 INFO:tasks.workunit.client.0.smithi107.stdout:7/773: dread d7/d21/d48/f55 [0,4194304] 0 2024-04-22T23:35:12.360 INFO:tasks.workunit.client.1.smithi157.stdout:7/921: sync 2024-04-22T23:35:12.360 INFO:tasks.workunit.client.1.smithi157.stdout:0/925: dwrite d17/d2b/d4f/d5a/dd2/f135 [0,4194304] 0 2024-04-22T23:35:12.361 INFO:tasks.workunit.client.1.smithi157.stdout:5/978: dwrite d1/d5/d7/d2f/d14c/f122 [0,4194304] 0 2024-04-22T23:35:12.363 INFO:tasks.workunit.client.1.smithi157.stdout:7/922: dread d0/d9/d29/d98/fc4 [0,4194304] 0 2024-04-22T23:35:12.363 INFO:tasks.workunit.client.1.smithi157.stdout:0/926: chown d17/c1c 0 1 2024-04-22T23:35:12.363 INFO:tasks.workunit.client.1.smithi157.stdout:0/927: write d17/d38/fb9 [174222,21226] 0 2024-04-22T23:35:12.363 INFO:tasks.workunit.client.1.smithi157.stdout:0/928: write d17/d23/d34/d67/d7f/d69/f89 [1143240,346] 0 2024-04-22T23:35:12.363 INFO:tasks.workunit.client.1.smithi157.stdout:0/929: readlink d17/d2b/d4f/dc5/dcc/de9/df6/dfd/l129 0 2024-04-22T23:35:12.365 INFO:tasks.workunit.client.1.smithi157.stdout:0/930: fsync d17/d2b/d71/fc2 0 2024-04-22T23:35:12.365 INFO:tasks.workunit.client.1.smithi157.stdout:0/931: write d17/d2b/d52/f42 [279471,100091] 0 2024-04-22T23:35:12.365 INFO:tasks.workunit.client.1.smithi157.stdout:0/932: fdatasync d17/d2b/d4f/d5a/f81 0 2024-04-22T23:35:12.365 INFO:tasks.workunit.client.1.smithi157.stdout:0/933: write d17/d2b/fd7 [3720774,50116] 0 2024-04-22T23:35:12.365 INFO:tasks.workunit.client.1.smithi157.stdout:0/934: chown d17/d38 6742603 1 2024-04-22T23:35:12.366 INFO:tasks.workunit.client.0.smithi107.stdout:4/845: truncate d1/d21/d85/f9c 2513871 0 2024-04-22T23:35:12.366 INFO:tasks.workunit.client.0.smithi107.stdout:4/846: chown d1/d21/d29 59 1 2024-04-22T23:35:12.367 INFO:tasks.workunit.client.0.smithi107.stdout:6/886: mkdir db/d20/d2c/de8/d10d 0 2024-04-22T23:35:12.367 INFO:tasks.workunit.client.0.smithi107.stdout:6/887: chown db/d20/d2a/d38/d4b/d9d 1295731839 1 2024-04-22T23:35:12.370 INFO:tasks.workunit.client.0.smithi107.stdout:9/778: mkdir d5/d7/d10f 0 2024-04-22T23:35:12.370 INFO:tasks.workunit.client.0.smithi107.stdout:9/779: dread - d5/d14/d28/fc1 zero size 2024-04-22T23:35:12.370 INFO:tasks.workunit.client.0.smithi107.stdout:9/780: truncate d5/d14/d2e/d4f/d73/df6/ff8 332951 0 2024-04-22T23:35:12.370 INFO:tasks.workunit.client.0.smithi107.stdout:9/781: chown d5/d24/ca5 355848 1 2024-04-22T23:35:12.370 INFO:tasks.workunit.client.0.smithi107.stdout:9/782: chown d5/d7/d27/d46/d4b/c67 180926283 1 2024-04-22T23:35:12.370 INFO:tasks.workunit.client.0.smithi107.stdout:9/783: dread - d5/d14/d28/f10c zero size 2024-04-22T23:35:12.372 INFO:tasks.workunit.client.1.smithi157.stdout:0/935: dread d17/d23/d34/d67/d7f/f5f [4194304,4194304] 0 2024-04-22T23:35:12.372 INFO:tasks.workunit.client.1.smithi157.stdout:0/936: chown d17/d38/f80 5 1 2024-04-22T23:35:12.373 INFO:tasks.workunit.client.0.smithi107.stdout:0/795: link d1/d27/d20/d2c/d41/dbe/fd5 d1/d27/d20/d40/db4/f108 0 2024-04-22T23:35:12.374 INFO:tasks.workunit.client.1.smithi157.stdout:8/781: dwrite d2/d4/dbd/d97/fea [0,4194304] 0 2024-04-22T23:35:12.374 INFO:tasks.workunit.client.1.smithi157.stdout:8/782: write d2/d6/d21/d5f/d17/dc8/ffa [17428,7073] 0 2024-04-22T23:35:12.375 INFO:tasks.workunit.client.1.smithi157.stdout:8/783: stat d2/d4/dda/cf7 0 2024-04-22T23:35:12.375 INFO:tasks.workunit.client.1.smithi157.stdout:0/937: rename d17/d23/d34/dee/da8/c8e to d17/d2b/d4f/dc5/c138 0 2024-04-22T23:35:12.376 INFO:tasks.workunit.client.0.smithi107.stdout:0/796: dread d1/d84/f6c [0,4194304] 0 2024-04-22T23:35:12.376 INFO:tasks.workunit.client.0.smithi107.stdout:2/886: link d1/d2b/f80 d1/d12/d35/d102/dba/df1/f126 0 2024-04-22T23:35:12.377 INFO:tasks.workunit.client.0.smithi107.stdout:2/887: chown d1/d12/l38 124031587 1 2024-04-22T23:35:12.377 INFO:tasks.workunit.client.0.smithi107.stdout:2/888: creat d1/d12/d35/d102/d87/f127 x:0 0 0 2024-04-22T23:35:12.378 INFO:tasks.workunit.client.0.smithi107.stdout:3/758: getdents db/d21/d27/d91 0 2024-04-22T23:35:12.378 INFO:tasks.workunit.client.0.smithi107.stdout:3/759: dread - db/d21/d27/d15/d29/d2a/f100 zero size 2024-04-22T23:35:12.378 INFO:tasks.workunit.client.0.smithi107.stdout:6/888: write db/d20/d2a/d38/d61/d6f/da2/fac [3153310,58072] 0 2024-04-22T23:35:12.380 INFO:tasks.workunit.client.1.smithi157.stdout:5/979: write d1/d27/d29/d4b/f5c [1901333,5075] 0 2024-04-22T23:35:12.381 INFO:tasks.workunit.client.1.smithi157.stdout:5/980: readlink d1/d5/l71 0 2024-04-22T23:35:12.381 INFO:tasks.workunit.client.1.smithi157.stdout:5/981: dread - d1/db0/dff/d102/f120 zero size 2024-04-22T23:35:12.381 INFO:tasks.workunit.client.0.smithi107.stdout:0/797: dread d1/d27/d20/d2c/d2e/f71 [0,4194304] 0 2024-04-22T23:35:12.382 INFO:tasks.workunit.client.0.smithi107.stdout:1/753: unlink d7/d19/d20/d39/d93/df4/db8/cc7 0 2024-04-22T23:35:12.383 INFO:tasks.workunit.client.0.smithi107.stdout:7/774: truncate d7/d21/d48/fc2 1192587 0 2024-04-22T23:35:12.383 INFO:tasks.workunit.client.0.smithi107.stdout:7/775: chown d7/d1b/l77 5 1 2024-04-22T23:35:12.385 INFO:tasks.workunit.client.0.smithi107.stdout:4/847: mkdir d1/d7/d5b/dbf/d27/d12f 0 2024-04-22T23:35:12.386 INFO:tasks.workunit.client.0.smithi107.stdout:5/860: sync 2024-04-22T23:35:12.387 INFO:tasks.workunit.client.0.smithi107.stdout:9/784: rmdir d5/d7/d17/d1b/d1f/d36/d53 39 2024-04-22T23:35:12.387 INFO:tasks.workunit.client.1.smithi157.stdout:5/982: mknod d1/d5/dc/d43/d5e/d145/c15a 0 2024-04-22T23:35:12.387 INFO:tasks.workunit.client.1.smithi157.stdout:5/983: chown d1/db0/dff/d102/f124 46407989 1 2024-04-22T23:35:12.388 INFO:tasks.workunit.client.1.smithi157.stdout:8/784: dread d2/d6/d21/d5f/d17/f49 [0,4194304] 0 2024-04-22T23:35:12.388 INFO:tasks.workunit.client.1.smithi157.stdout:8/785: read - d2/d4/dbd/d67/f95 zero size 2024-04-22T23:35:12.389 INFO:tasks.workunit.client.0.smithi107.stdout:6/889: write db/f2d [797574,110843] 0 2024-04-22T23:35:12.389 INFO:tasks.workunit.client.0.smithi107.stdout:6/890: chown db/d20/d49/d7b/l96 4050 1 2024-04-22T23:35:12.389 INFO:tasks.workunit.client.1.smithi157.stdout:5/984: symlink d1/d5/d7/d10/d14f/dc6/d148/l15b 0 2024-04-22T23:35:12.390 INFO:tasks.workunit.client.1.smithi157.stdout:7/923: dwrite d0/d9/db/f110 [0,4194304] 0 2024-04-22T23:35:12.390 INFO:tasks.workunit.client.1.smithi157.stdout:7/924: readlink d0/d9/d29/d33/d7c/lea 0 2024-04-22T23:35:12.391 INFO:tasks.workunit.client.0.smithi107.stdout:3/760: rmdir db/d21/d27/d15/d29/d2a/d45/deb 39 2024-04-22T23:35:12.391 INFO:tasks.workunit.client.0.smithi107.stdout:3/761: chown db/d21/d27/d15/d24/d52/d60/dca 218461 1 2024-04-22T23:35:12.391 INFO:tasks.workunit.client.1.smithi157.stdout:5/985: symlink d1/d5/dc/d5d/d77/l15c 0 2024-04-22T23:35:12.394 INFO:tasks.workunit.client.0.smithi107.stdout:7/776: getdents d7/d21/d7e/d94 0 2024-04-22T23:35:12.394 INFO:tasks.workunit.client.0.smithi107.stdout:7/777: chown d7/d21/d54/f90 1426 1 2024-04-22T23:35:12.395 INFO:tasks.workunit.client.1.smithi157.stdout:7/925: rename d0/d9/d42/f8c to d0/d9/de/f118 0 2024-04-22T23:35:12.396 INFO:tasks.workunit.client.1.smithi157.stdout:5/986: symlink d1/d5/d7/dbd/l15d 0 2024-04-22T23:35:12.396 INFO:tasks.workunit.client.1.smithi157.stdout:5/987: fdatasync d1/d5/d7/d2f/d14c/f122 0 2024-04-22T23:35:12.396 INFO:tasks.workunit.client.0.smithi107.stdout:4/848: creat d1/d21/d29/d2e/d88/d99/dd2/daf/f130 x:0 0 0 2024-04-22T23:35:12.397 INFO:tasks.workunit.client.0.smithi107.stdout:5/861: link d9/db/cc d9/db/de/d5e/d124/c128 0 2024-04-22T23:35:12.398 INFO:tasks.workunit.client.1.smithi157.stdout:7/926: rename d0/d9/d29/d5e/da9 to d0/d9/d29/d119 0 2024-04-22T23:35:12.398 INFO:tasks.workunit.client.1.smithi157.stdout:7/927: dread - d0/d9/d42/d55/d80/d95/dce/f112 zero size 2024-04-22T23:35:12.398 INFO:tasks.workunit.client.1.smithi157.stdout:7/928: fdatasync d0/d9/d29/f4d 0 2024-04-22T23:35:12.398 INFO:tasks.workunit.client.1.smithi157.stdout:7/929: fdatasync d0/d9/d29/dbd/f10e 0 2024-04-22T23:35:12.398 INFO:tasks.workunit.client.1.smithi157.stdout:7/930: dread - d0/d9/d29/d5e/d9e/fc5 zero size 2024-04-22T23:35:12.398 INFO:tasks.workunit.client.1.smithi157.stdout:7/931: stat d0/d9/d29/c69 0 2024-04-22T23:35:12.399 INFO:tasks.workunit.client.0.smithi107.stdout:9/785: creat d5/d7/d27/d46/d4b/f110 x:0 0 0 2024-04-22T23:35:12.399 INFO:tasks.workunit.client.1.smithi157.stdout:5/988: link d1/d5/dc/d43/d4f/ff5 d1/d5/dc/f15e 0 2024-04-22T23:35:12.399 INFO:tasks.workunit.client.1.smithi157.stdout:5/989: rmdir d1/d5/dc/d43/d4f 39 2024-04-22T23:35:12.400 INFO:tasks.workunit.client.0.smithi107.stdout:6/891: creat db/d20/d2a/d35/d69/db3/d100/f10e x:0 0 0 2024-04-22T23:35:12.402 INFO:tasks.workunit.client.0.smithi107.stdout:2/889: rmdir d1/d2b/d88 39 2024-04-22T23:35:12.406 INFO:tasks.workunit.client.0.smithi107.stdout:3/762: mkdir db/d21/d27/d15/d29/d2a/d45/ddd/d102 0 2024-04-22T23:35:12.406 INFO:tasks.workunit.client.1.smithi157.stdout:5/990: dread d1/fb4 [0,4194304] 0 2024-04-22T23:35:12.407 INFO:tasks.workunit.client.1.smithi157.stdout:5/991: dread d1/d5/d7/d2f/d45/fd6 [0,4194304] 0 2024-04-22T23:35:12.408 INFO:tasks.workunit.client.0.smithi107.stdout:7/778: write d7/d21/f96 [632906,32164] 0 2024-04-22T23:35:12.408 INFO:tasks.workunit.client.0.smithi107.stdout:7/779: truncate d7/d21/da4/d81/fc5 15902 0 2024-04-22T23:35:12.409 INFO:tasks.workunit.client.0.smithi107.stdout:4/849: dread d1/d21/d52/f97 [0,4194304] 0 2024-04-22T23:35:12.409 INFO:tasks.workunit.client.0.smithi107.stdout:4/850: dread - d1/d21/d29/d110/f129 zero size 2024-04-22T23:35:12.412 INFO:tasks.workunit.client.1.smithi157.stdout:0/938: dwrite fc [4194304,4194304] 0 2024-04-22T23:35:12.412 INFO:tasks.workunit.client.0.smithi107.stdout:0/798: dwrite d1/d9/d14/d4f/d59/fe2 [0,4194304] 0 2024-04-22T23:35:12.413 INFO:tasks.workunit.client.0.smithi107.stdout:5/862: rename d9/d76/l91 to d9/d26/d79/l129 0 2024-04-22T23:35:12.414 INFO:tasks.workunit.client.1.smithi157.stdout:7/932: dread d0/d9/d29/d5e/f6a [0,4194304] 0 2024-04-22T23:35:12.414 INFO:tasks.workunit.client.1.smithi157.stdout:5/992: write d1/d5/dc/f89 [1113410,60158] 0 2024-04-22T23:35:12.415 INFO:tasks.workunit.client.0.smithi107.stdout:9/786: symlink d5/d7/d17/d1b/l111 0 2024-04-22T23:35:12.416 INFO:tasks.workunit.client.0.smithi107.stdout:6/892: mkdir db/d20/d2a/dca/d10f 0 2024-04-22T23:35:12.416 INFO:tasks.workunit.client.0.smithi107.stdout:6/893: fsync db/d20/d2a/d38/d61/d6f/da2/fac 0 2024-04-22T23:35:12.416 INFO:tasks.workunit.client.0.smithi107.stdout:2/890: symlink d1/dcd/df0/l128 0 2024-04-22T23:35:12.418 INFO:tasks.workunit.client.0.smithi107.stdout:3/763: chown db/d21/d27/ce 216516 1 2024-04-22T23:35:12.420 INFO:tasks.workunit.client.0.smithi107.stdout:7/780: creat d7/d21/d54/dd3/f107 x:0 0 0 2024-04-22T23:35:12.420 INFO:tasks.workunit.client.0.smithi107.stdout:7/781: dread - d7/d4c/da1/fad zero size 2024-04-22T23:35:12.420 INFO:tasks.workunit.client.0.smithi107.stdout:7/782: fsync d7/d21/d48/fb6 0 2024-04-22T23:35:12.420 INFO:tasks.workunit.client.0.smithi107.stdout:7/783: write d7/d21/d48/d5c/d9d/d6e/db2/f100 [72428,20579] 0 2024-04-22T23:35:12.421 INFO:tasks.workunit.client.0.smithi107.stdout:7/784: getdents d7/d21/d7e/d94 0 2024-04-22T23:35:12.421 INFO:tasks.workunit.client.0.smithi107.stdout:4/851: unlink d1/d21/d29/d2e/d88/d99/dd2/lcd 0 2024-04-22T23:35:12.421 INFO:tasks.workunit.client.0.smithi107.stdout:4/852: readlink d1/d21/d29/d2e/d88/d99/dd2/ld8 0 2024-04-22T23:35:12.421 INFO:tasks.workunit.client.0.smithi107.stdout:4/853: truncate d1/d21/d29/d2e/d88/d99/dd2/fcf 1531991 0 2024-04-22T23:35:12.422 INFO:tasks.workunit.client.1.smithi157.stdout:8/786: dwrite d2/d34/f78 [4194304,4194304] 0 2024-04-22T23:35:12.425 INFO:tasks.workunit.client.1.smithi157.stdout:0/939: symlink d17/d2b/d52/ddc/l139 0 2024-04-22T23:35:12.426 INFO:tasks.workunit.client.0.smithi107.stdout:0/799: dread d1/d27/d20/d2c/d2e/f39 [0,4194304] 0 2024-04-22T23:35:12.426 INFO:tasks.workunit.client.1.smithi157.stdout:5/993: link d1/db0/ca8 d1/d5/d7/dbd/c15f 0 2024-04-22T23:35:12.426 INFO:tasks.workunit.client.1.smithi157.stdout:5/994: dread - d1/d5/dc/d43/d4f/ff5 zero size 2024-04-22T23:35:12.427 INFO:tasks.workunit.client.0.smithi107.stdout:5/863: mknod d9/db/de/d72/d7c/d109/c12a 0 2024-04-22T23:35:12.428 INFO:tasks.workunit.client.1.smithi157.stdout:7/933: dread d0/d9/d29/d49/d4a/f111 [0,4194304] 0 2024-04-22T23:35:12.429 INFO:tasks.workunit.client.0.smithi107.stdout:9/787: creat d5/d7/d17/d1b/d1f/f112 x:0 0 0 2024-04-22T23:35:12.429 INFO:tasks.workunit.client.0.smithi107.stdout:9/788: fdatasync d5/d7/d27/d46/fd5 0 2024-04-22T23:35:12.430 INFO:tasks.workunit.client.1.smithi157.stdout:0/940: creat d17/d2b/d4f/dc5/dcc/de9/df6/dfd/f13a x:0 0 0 2024-04-22T23:35:12.430 INFO:tasks.workunit.client.0.smithi107.stdout:3/764: creat db/d21/d27/d15/d24/d66/da5/def/f103 x:0 0 0 2024-04-22T23:35:12.435 INFO:tasks.workunit.client.0.smithi107.stdout:2/891: dwrite d1/d12/d35/d102/d7e/dc6/fed [0,4194304] 0 2024-04-22T23:35:12.436 INFO:tasks.workunit.client.0.smithi107.stdout:7/785: symlink d7/d4c/da1/dce/l108 0 2024-04-22T23:35:12.436 INFO:tasks.workunit.client.0.smithi107.stdout:7/786: write d7/d21/f6d [4648518,99521] 0 2024-04-22T23:35:12.438 INFO:tasks.workunit.client.0.smithi107.stdout:4/854: mkdir d1/d21/d29/d2e/d88/d99/dd2/daf/d131 0 2024-04-22T23:35:12.438 INFO:tasks.workunit.client.0.smithi107.stdout:4/855: fdatasync d1/d21/d52/f73 0 2024-04-22T23:35:12.438 INFO:tasks.workunit.client.0.smithi107.stdout:4/856: write d1/d1b/fa9 [760968,74154] 0 2024-04-22T23:35:12.440 INFO:tasks.workunit.client.0.smithi107.stdout:9/789: unlink d5/d7/d27/d46/d4b/c67 0 2024-04-22T23:35:12.442 INFO:tasks.workunit.client.0.smithi107.stdout:3/765: chown db/d21/d27/l35 85 1 2024-04-22T23:35:12.442 INFO:tasks.workunit.client.0.smithi107.stdout:3/766: creat db/d21/d27/d15/d24/d87/f104 x:0 0 0 2024-04-22T23:35:12.445 INFO:tasks.workunit.client.1.smithi157.stdout:8/787: getdents d2/d4/dda 0 2024-04-22T23:35:12.446 INFO:tasks.workunit.client.1.smithi157.stdout:7/934: mkdir d0/d9/d29/d49/d4a/d11a 0 2024-04-22T23:35:12.447 INFO:tasks.workunit.client.1.smithi157.stdout:0/941: mkdir d17/d2b/d4f/d5a/dd2/d10a/d13b 0 2024-04-22T23:35:12.447 INFO:tasks.workunit.client.1.smithi157.stdout:0/942: creat d17/d23/d34/d67/d7f/d69/dce/de3/f13c x:0 0 0 2024-04-22T23:35:12.447 INFO:tasks.workunit.client.1.smithi157.stdout:5/995: getdents d1/db0/dff/d102 0 2024-04-22T23:35:12.448 INFO:tasks.workunit.client.1.smithi157.stdout:7/935: read d0/d9/db/d35/fba [29403,11212] 0 2024-04-22T23:35:12.448 INFO:tasks.workunit.client.1.smithi157.stdout:7/936: creat d0/d9/d42/d55/d80/d95/f11b x:0 0 0 2024-04-22T23:35:12.448 INFO:tasks.workunit.client.1.smithi157.stdout:8/788: symlink d2/d6/d21/d5f/d11/l111 0 2024-04-22T23:35:12.449 INFO:tasks.workunit.client.1.smithi157.stdout:0/943: mkdir d17/d38/d13d 0 2024-04-22T23:35:12.450 INFO:tasks.workunit.client.0.smithi107.stdout:0/800: dwrite d1/d27/d20/d2c/d41/f58 [0,4194304] 0 2024-04-22T23:35:12.451 INFO:tasks.workunit.client.1.smithi157.stdout:5/996: creat d1/d5/d7/d2f/d51/f160 x:0 0 0 2024-04-22T23:35:12.451 INFO:tasks.workunit.client.0.smithi107.stdout:7/787: mkdir d7/d21/d54/dcc/d109 0 2024-04-22T23:35:12.451 INFO:tasks.workunit.client.0.smithi107.stdout:7/788: creat d7/d21/d48/d5c/d9d/db1/f10a x:0 0 0 2024-04-22T23:35:12.453 INFO:tasks.workunit.client.0.smithi107.stdout:5/864: mknod d9/db/d92/ddb/c12b 0 2024-04-22T23:35:12.454 INFO:tasks.workunit.client.0.smithi107.stdout:9/790: creat d5/d7/d17/d1b/f113 x:0 0 0 2024-04-22T23:35:12.454 INFO:tasks.workunit.client.0.smithi107.stdout:3/767: dread db/d21/d27/d36/f7e [0,4194304] 0 2024-04-22T23:35:12.457 INFO:tasks.workunit.client.0.smithi107.stdout:9/791: write d5/d14/f15 [4157749,130080] 0 2024-04-22T23:35:12.458 INFO:tasks.workunit.client.0.smithi107.stdout:0/801: mkdir d1/d27/d20/d40/dbd/df2/d109 0 2024-04-22T23:35:12.462 INFO:tasks.workunit.client.1.smithi157.stdout:0/944: rename d17/d2b/d4f/d9b/le8 to d17/d2b/d4f/dc5/l13e 0 2024-04-22T23:35:12.462 INFO:tasks.workunit.client.1.smithi157.stdout:9/909: sync 2024-04-22T23:35:12.462 INFO:tasks.workunit.client.1.smithi157.stdout:7/937: symlink d0/d9/d29/d49/l11c 0 2024-04-22T23:35:12.462 INFO:tasks.workunit.client.1.smithi157.stdout:0/945: read - d17/d2b/d52/dd1/ff0 zero size 2024-04-22T23:35:12.462 INFO:tasks.workunit.client.1.smithi157.stdout:9/910: fsync d4/df/d57/d6b/dc4/fd1 0 2024-04-22T23:35:12.462 INFO:tasks.workunit.client.1.smithi157.stdout:0/946: creat d17/d23/d34/d67/d78/f13f x:0 0 0 2024-04-22T23:35:12.464 INFO:tasks.workunit.client.1.smithi157.stdout:7/938: mknod d0/d9/db/df/dac/c11d 0 2024-04-22T23:35:12.464 INFO:tasks.workunit.client.1.smithi157.stdout:7/939: chown d0/d9/d29/d33 11186671 1 2024-04-22T23:35:12.464 INFO:tasks.workunit.client.1.smithi157.stdout:7/940: chown d0/d9/db/df/d68/fca 4149285 1 2024-04-22T23:35:12.466 INFO:tasks.workunit.client.1.smithi157.stdout:9/911: rmdir d4/d15/d23/d27/db5/de2 0 2024-04-22T23:35:12.466 INFO:tasks.workunit.client.0.smithi107.stdout:2/892: dwrite d1/d12/d35/d54/fd5 [0,4194304] 0 2024-04-22T23:35:12.466 INFO:tasks.workunit.client.0.smithi107.stdout:4/857: dwrite d1/d21/d29/d2e/d88/f8b [0,4194304] 0 2024-04-22T23:35:12.466 INFO:tasks.workunit.client.0.smithi107.stdout:7/789: truncate d7/f13 141503 0 2024-04-22T23:35:12.466 INFO:tasks.workunit.client.0.smithi107.stdout:4/858: write d1/fea [679689,5994] 0 2024-04-22T23:35:12.466 INFO:tasks.workunit.client.0.smithi107.stdout:7/790: stat d7/d21/d48/d5c/d9d/l65 0 2024-04-22T23:35:12.468 INFO:tasks.workunit.client.0.smithi107.stdout:3/768: creat db/d21/d27/d15/d24/d52/f105 x:0 0 0 2024-04-22T23:35:12.468 INFO:tasks.workunit.client.0.smithi107.stdout:3/769: write db/d21/d27/d15/d29/d2a/d45/ddd/d25/fe2 [3634425,127500] 0 2024-04-22T23:35:12.468 INFO:tasks.workunit.client.1.smithi157.stdout:7/941: symlink d0/d9/d29/d33/d37/l11e 0 2024-04-22T23:35:12.470 INFO:tasks.workunit.client.0.smithi107.stdout:0/802: rmdir d1/d7b/d85/ded 39 2024-04-22T23:35:12.470 INFO:tasks.workunit.client.0.smithi107.stdout:2/893: dread d1/d2b/d3d/d57/fa0 [0,4194304] 0 2024-04-22T23:35:12.470 INFO:tasks.workunit.client.0.smithi107.stdout:0/803: write d1/d2a/fee [581858,35921] 0 2024-04-22T23:35:12.471 INFO:tasks.workunit.client.1.smithi157.stdout:9/912: link d4/df/d57/dbd/cbf d4/d2d/c118 0 2024-04-22T23:35:12.472 INFO:tasks.workunit.client.1.smithi157.stdout:9/913: symlink d4/d2d/dfe/l119 0 2024-04-22T23:35:12.473 INFO:tasks.workunit.client.1.smithi157.stdout:9/914: readlink d4/df/d57/dbd/dc6/l100 0 2024-04-22T23:35:12.473 INFO:tasks.workunit.client.1.smithi157.stdout:9/915: readlink d4/df/d57/dbd/dc6/l100 0 2024-04-22T23:35:12.474 INFO:tasks.workunit.client.1.smithi157.stdout:9/916: symlink d4/d15/d23/dad/l11a 0 2024-04-22T23:35:12.474 INFO:tasks.workunit.client.1.smithi157.stdout:8/789: write d2/d34/d6a/d77/f9f [3417752,77892] 0 2024-04-22T23:35:12.475 INFO:tasks.workunit.client.1.smithi157.stdout:8/790: write d2/d4/d73/fe9 [1029020,120874] 0 2024-04-22T23:35:12.475 INFO:tasks.workunit.client.1.smithi157.stdout:9/917: creat d4/df/d57/db7/f11b x:0 0 0 2024-04-22T23:35:12.477 INFO:tasks.workunit.client.1.smithi157.stdout:8/791: unlink d2/d6/d6b/f8b 0 2024-04-22T23:35:12.477 INFO:tasks.workunit.client.1.smithi157.stdout:7/942: write d0/d9/db/df/f9d [583838,47935] 0 2024-04-22T23:35:12.479 INFO:tasks.workunit.client.1.smithi157.stdout:9/918: unlink d4/d15/d55/c56 0 2024-04-22T23:35:12.480 INFO:tasks.workunit.client.1.smithi157.stdout:0/947: getdents d17/d38 0 2024-04-22T23:35:12.481 INFO:tasks.workunit.client.0.smithi107.stdout:4/859: symlink d1/d7/d53/d66/d6b/l132 0 2024-04-22T23:35:12.481 INFO:tasks.workunit.client.0.smithi107.stdout:4/860: chown d1/d7/d63/d78/d80/lcb 109 1 2024-04-22T23:35:12.482 INFO:tasks.workunit.client.1.smithi157.stdout:0/948: mkdir d17/d23/d34/d140 0 2024-04-22T23:35:12.482 INFO:tasks.workunit.client.1.smithi157.stdout:0/949: chown d17/d23/d34/d67/d7f/d69/d103/f122 10681 1 2024-04-22T23:35:12.483 INFO:tasks.workunit.client.1.smithi157.stdout:0/950: getdents d17/d23/d9c 0 2024-04-22T23:35:12.484 INFO:tasks.workunit.client.0.smithi107.stdout:7/791: rename d7/d21/d48/d5c/d9d/db1/f10a to d7/d21/d48/d5c/d9d/db1/f10b 0 2024-04-22T23:35:12.484 INFO:tasks.workunit.client.0.smithi107.stdout:7/792: write d7/d4c/da1/fd9 [560466,106060] 0 2024-04-22T23:35:12.484 INFO:tasks.workunit.client.1.smithi157.stdout:8/792: dread d2/d34/d6a/f76 [0,4194304] 0 2024-04-22T23:35:12.484 INFO:tasks.workunit.client.1.smithi157.stdout:9/919: dread d4/d15/f7a [0,4194304] 0 2024-04-22T23:35:12.484 INFO:tasks.workunit.client.1.smithi157.stdout:8/793: creat d2/d4/f112 x:0 0 0 2024-04-22T23:35:12.485 INFO:tasks.workunit.client.0.smithi107.stdout:5/865: dwrite d9/db/de/d5e/fe2 [0,4194304] 0 2024-04-22T23:35:12.485 INFO:tasks.workunit.client.1.smithi157.stdout:0/951: creat d17/d2b/d4f/d9b/f141 x:0 0 0 2024-04-22T23:35:12.486 INFO:tasks.workunit.client.0.smithi107.stdout:9/792: dwrite d5/d7/d27/d46/f88 [8388608,4194304] 0 2024-04-22T23:35:12.486 INFO:tasks.workunit.client.0.smithi107.stdout:3/770: truncate f0 907819 0 2024-04-22T23:35:12.486 INFO:tasks.workunit.client.0.smithi107.stdout:3/771: write db/f6f [2034622,31061] 0 2024-04-22T23:35:12.488 INFO:tasks.workunit.client.1.smithi157.stdout:9/920: rmdir d4/df/d57/db7/df9 39 2024-04-22T23:35:12.488 INFO:tasks.workunit.client.1.smithi157.stdout:9/921: chown d4/d15/d23/c89 116669 1 2024-04-22T23:35:12.489 INFO:tasks.workunit.client.0.smithi107.stdout:0/804: creat d1/d27/d20/d2c/d41/f10a x:0 0 0 2024-04-22T23:35:12.491 INFO:tasks.workunit.client.1.smithi157.stdout:8/794: symlink d2/d34/d6a/d77/l113 0 2024-04-22T23:35:12.492 INFO:tasks.workunit.client.0.smithi107.stdout:0/805: dread d1/d7b/fbf [0,4194304] 0 2024-04-22T23:35:12.492 INFO:tasks.workunit.client.0.smithi107.stdout:4/861: creat d1/d7/d5b/dbf/d27/d12f/f133 x:0 0 0 2024-04-22T23:35:12.493 INFO:tasks.workunit.client.1.smithi157.stdout:0/952: symlink d17/d2b/d4f/d5a/dd2/d10a/d13b/l142 0 2024-04-22T23:35:12.495 INFO:tasks.workunit.client.0.smithi107.stdout:7/793: mknod d7/d21/da4/d81/c10c 0 2024-04-22T23:35:12.495 INFO:tasks.workunit.client.0.smithi107.stdout:7/794: read d7/d21/d7e/fb3 [2174336,57283] 0 2024-04-22T23:35:12.495 INFO:tasks.workunit.client.1.smithi157.stdout:8/795: creat d2/d34/d6a/d77/df2/f114 x:0 0 0 2024-04-22T23:35:12.501 INFO:tasks.workunit.client.1.smithi157.stdout:9/922: dread d4/d15/d23/d27/ff1 [0,4194304] 0 2024-04-22T23:35:12.503 INFO:tasks.workunit.client.0.smithi107.stdout:5/866: symlink d9/db/de/l12c 0 2024-04-22T23:35:12.510 INFO:tasks.workunit.client.0.smithi107.stdout:0/806: creat d1/d7b/f10b x:0 0 0 2024-04-22T23:35:12.510 INFO:tasks.workunit.client.0.smithi107.stdout:0/807: chown d1/d27/d20/d40/dbd/cc1 4295467 1 2024-04-22T23:35:12.511 INFO:tasks.workunit.client.0.smithi107.stdout:4/862: link d1/d21/d29/d2e/d3b/f90 d1/d7/d63/d78/f134 0 2024-04-22T23:35:12.511 INFO:tasks.workunit.client.0.smithi107.stdout:4/863: chown d1/d21/d29/d2e/d88/d99/dd2/f7f 28308 1 2024-04-22T23:35:12.511 INFO:tasks.workunit.client.0.smithi107.stdout:9/793: dwrite d5/d7/d17/d1b/d1f/d36/d96/fb2 [0,4194304] 0 2024-04-22T23:35:12.511 INFO:tasks.workunit.client.0.smithi107.stdout:9/794: fdatasync d5/d7/f16 0 2024-04-22T23:35:12.513 INFO:tasks.workunit.client.0.smithi107.stdout:5/867: mknod d9/db/de/d72/d7c/da4/c12d 0 2024-04-22T23:35:12.513 INFO:tasks.workunit.client.0.smithi107.stdout:5/868: dread - d9/db/de/f11e zero size 2024-04-22T23:35:12.514 INFO:tasks.workunit.client.1.smithi157.stdout:0/953: rename d17/d2b/d4f/d5a/dd2 to d17/d23/d34/d67/d136/d143 0 2024-04-22T23:35:12.515 INFO:tasks.workunit.client.0.smithi107.stdout:3/772: rename db/d21/d27/l3a to db/d21/db1/dcd/l106 0 2024-04-22T23:35:12.515 INFO:tasks.workunit.client.0.smithi107.stdout:3/773: fdatasync db/d21/fda 0 2024-04-22T23:35:12.515 INFO:tasks.workunit.client.0.smithi107.stdout:3/774: write db/d21/d27/d15/d24/d52/d60/fd8 [940395,87772] 0 2024-04-22T23:35:12.516 INFO:tasks.workunit.client.1.smithi157.stdout:9/923: rename d4/df/d57/dd0/f114 to d4/d15/d23/d27/db5/db9/f11c 0 2024-04-22T23:35:12.516 INFO:tasks.workunit.client.0.smithi107.stdout:9/795: creat d5/d7/d17/d1b/dcb/f114 x:0 0 0 2024-04-22T23:35:12.516 INFO:tasks.workunit.client.0.smithi107.stdout:9/796: readlink d5/d7/d27/d69/lc4 0 2024-04-22T23:35:12.516 INFO:tasks.workunit.client.0.smithi107.stdout:9/797: dread - d5/d14/d28/fc1 zero size 2024-04-22T23:35:12.517 INFO:tasks.workunit.client.0.smithi107.stdout:7/795: mknod d7/d21/d48/c10d 0 2024-04-22T23:35:12.517 INFO:tasks.workunit.client.0.smithi107.stdout:7/796: read - d7/d4c/fe7 zero size 2024-04-22T23:35:12.517 INFO:tasks.workunit.client.1.smithi157.stdout:0/954: unlink d17/d23/d34/dee/da8/fe7 0 2024-04-22T23:35:12.519 INFO:tasks.workunit.client.0.smithi107.stdout:0/808: rename d1/d27/d20/d40/dbd/df2/c107 to d1/d27/d20/d40/dbd/df2/d109/c10c 0 2024-04-22T23:35:12.519 INFO:tasks.workunit.client.0.smithi107.stdout:0/809: stat d1/d27/f38 0 2024-04-22T23:35:12.519 INFO:tasks.workunit.client.0.smithi107.stdout:0/810: readlink d1/d9/d14/d4f/d59/l95 0 2024-04-22T23:35:12.519 INFO:tasks.workunit.client.0.smithi107.stdout:0/811: chown d1/d27/d20/f60 51 1 2024-04-22T23:35:12.519 INFO:tasks.workunit.client.1.smithi157.stdout:0/955: creat d17/d23/d34/d67/d78/dc7/d12f/f144 x:0 0 0 2024-04-22T23:35:12.520 INFO:tasks.workunit.client.1.smithi157.stdout:8/796: dwrite d2/d6/fa2 [4194304,4194304] 0 2024-04-22T23:35:12.522 INFO:tasks.workunit.client.0.smithi107.stdout:9/798: link d5/d7/d27/d46/f74 d5/d29/d7b/f115 0 2024-04-22T23:35:12.523 INFO:tasks.workunit.client.1.smithi157.stdout:0/956: mkdir d17/d2b/d4f/d5a/d145 0 2024-04-22T23:35:12.524 INFO:tasks.workunit.client.0.smithi107.stdout:0/812: truncate d1/d27/d20/d2c/d41/f54 2343012 0 2024-04-22T23:35:12.524 INFO:tasks.workunit.client.0.smithi107.stdout:0/813: write d1/d27/d20/d2c/d41/f10a [342999,89954] 0 2024-04-22T23:35:12.525 INFO:tasks.workunit.client.0.smithi107.stdout:0/814: symlink d1/d84/d99/l10d 0 2024-04-22T23:35:12.525 INFO:tasks.workunit.client.0.smithi107.stdout:0/815: fsync d1/d84/d99/feb 0 2024-04-22T23:35:12.526 INFO:tasks.workunit.client.0.smithi107.stdout:3/775: write db/d21/d27/d15/fb4 [2481528,12223] 0 2024-04-22T23:35:12.526 INFO:tasks.workunit.client.0.smithi107.stdout:3/776: truncate db/d21/d27/d15/d24/d52/fbd 4906602 0 2024-04-22T23:35:12.527 INFO:tasks.workunit.client.0.smithi107.stdout:3/777: creat db/d21/d27/d15/d24/d52/d60/dca/dfc/f107 x:0 0 0 2024-04-22T23:35:12.531 INFO:tasks.workunit.client.1.smithi157.stdout:0/957: link d17/d23/d34/d67/d136/d143/de4/l12e d17/d38/d13d/l146 0 2024-04-22T23:35:12.533 INFO:tasks.workunit.client.0.smithi107.stdout:5/869: dread d9/db/d92/dc2/ff9 [0,4194304] 0 2024-04-22T23:35:12.533 INFO:tasks.workunit.client.0.smithi107.stdout:5/870: creat d9/d26/d79/f12e x:0 0 0 2024-04-22T23:35:12.534 INFO:tasks.workunit.client.0.smithi107.stdout:7/797: dwrite d7/d21/d48/d5c/d9d/d6e/f70 [0,4194304] 0 2024-04-22T23:35:12.534 INFO:tasks.workunit.client.0.smithi107.stdout:7/798: write d7/d21/f7f [556416,95857] 0 2024-04-22T23:35:12.546 INFO:tasks.workunit.client.0.smithi107.stdout:4/864: dwrite d1/d21/d29/d2e/dfa/f10b [0,4194304] 0 2024-04-22T23:35:12.546 INFO:tasks.workunit.client.1.smithi157.stdout:9/924: rename d4/d15 to d4/df/d11d 0 2024-04-22T23:35:12.547 INFO:tasks.workunit.client.0.smithi107.stdout:3/778: dwrite db/d21/d27/d15/d24/d87/f89 [4194304,4194304] 0 2024-04-22T23:35:12.547 INFO:tasks.workunit.client.0.smithi107.stdout:9/799: dwrite d5/d14/d2e/f3f [0,4194304] 0 2024-04-22T23:35:12.547 INFO:tasks.workunit.client.0.smithi107.stdout:9/800: fsync d5/d7/d17/d1b/f30 0 2024-04-22T23:35:12.547 INFO:tasks.workunit.client.0.smithi107.stdout:9/801: dread - d5/d7/d27/d69/db6/fea zero size 2024-04-22T23:35:12.554 INFO:tasks.workunit.client.0.smithi107.stdout:4/865: dread d1/d21/d52/f97 [0,4194304] 0 2024-04-22T23:35:12.573 INFO:tasks.workunit.client.0.smithi107.stdout:4/866: dwrite d1/d7/d5b/dbf/d27/f115 [0,4194304] 0 2024-04-22T23:35:12.573 INFO:tasks.workunit.client.1.smithi157.stdout:9/925: dwrite d4/df/d11d/d23/f97 [0,4194304] 0 2024-04-22T23:35:12.576 INFO:tasks.workunit.client.1.smithi157.stdout:9/926: creat d4/df/d11d/d23/d27/db5/db9/f11e x:0 0 0 2024-04-22T23:35:12.577 INFO:tasks.workunit.client.1.smithi157.stdout:9/927: symlink d4/df/d57/dd0/l11f 0 2024-04-22T23:35:12.577 INFO:tasks.workunit.client.1.smithi157.stdout:9/928: stat d4/df/d11d/d23/d3f 0 2024-04-22T23:35:12.577 INFO:tasks.workunit.client.1.smithi157.stdout:9/929: write d4/df/d57/d6b/fe8 [552966,46743] 0 2024-04-22T23:35:12.577 INFO:tasks.workunit.client.1.smithi157.stdout:9/930: fsync d4/df/d11d/d23/d27/d75/ff0 0 2024-04-22T23:35:12.578 INFO:tasks.workunit.client.1.smithi157.stdout:9/931: mknod d4/d2d/dfe/c120 0 2024-04-22T23:35:12.580 INFO:tasks.workunit.client.1.smithi157.stdout:9/932: symlink d4/l121 0 2024-04-22T23:35:12.583 INFO:tasks.workunit.client.1.smithi157.stdout:9/933: mknod d4/df/d57/db7/df9/c122 0 2024-04-22T23:35:12.583 INFO:tasks.workunit.client.1.smithi157.stdout:9/934: write d4/df/d57/dbd/dc6/ffa [1759008,59883] 0 2024-04-22T23:35:12.583 INFO:tasks.workunit.client.1.smithi157.stdout:9/935: symlink d4/df/d72/l123 0 2024-04-22T23:35:12.584 INFO:tasks.workunit.client.1.smithi157.stdout:9/936: creat d4/df/d57/db7/df9/f124 x:0 0 0 2024-04-22T23:35:12.585 INFO:tasks.workunit.client.1.smithi157.stdout:9/937: creat d4/df/d11d/d23/d3f/f125 x:0 0 0 2024-04-22T23:35:12.586 INFO:tasks.workunit.client.1.smithi157.stdout:9/938: unlink d4/ccc 0 2024-04-22T23:35:12.586 INFO:tasks.workunit.client.1.smithi157.stdout:9/939: chown d4/f29 3162 1 2024-04-22T23:35:12.587 INFO:tasks.workunit.client.1.smithi157.stdout:9/940: rename d4/df/d11d/d23/d67/f95 to d4/df/d11d/d55/d8d/f126 0 2024-04-22T23:35:12.587 INFO:tasks.workunit.client.1.smithi157.stdout:9/941: chown d4/df/d11d/d23/d27/d75/fea 919701 1 2024-04-22T23:35:12.588 INFO:tasks.workunit.client.1.smithi157.stdout:9/942: truncate d4/df/d11d/d55/d8d/f126 4295144 0 2024-04-22T23:35:12.589 INFO:tasks.workunit.client.1.smithi157.stdout:9/943: link d4/df/d11d/d55/f76 d4/df/d11d/d55/f127 0 2024-04-22T23:35:12.589 INFO:tasks.workunit.client.1.smithi157.stdout:9/944: fsync d4/df/d11d/d23/d27/d75/ff0 0 2024-04-22T23:35:12.589 INFO:tasks.workunit.client.1.smithi157.stdout:9/945: truncate d4/df/d57/dbd/dc6/fd6 764078 0 2024-04-22T23:35:12.590 INFO:tasks.workunit.client.1.smithi157.stdout:9/946: truncate d4/df/d11d/d23/d27/db5/f105 1977414 0 2024-04-22T23:35:12.590 INFO:tasks.workunit.client.1.smithi157.stdout:9/947: truncate d4/df/d11d/d23/d27/d75/fea 4700050 0 2024-04-22T23:35:12.590 INFO:tasks.workunit.client.0.smithi107.stdout:4/867: dwrite d1/d21/d29/d2e/d88/d99/dd2/daf/dc1/f116 [0,4194304] 0 2024-04-22T23:35:12.590 INFO:tasks.workunit.client.1.smithi157.stdout:9/948: creat d4/df/d11d/d23/d67/f128 x:0 0 0 2024-04-22T23:35:12.591 INFO:tasks.workunit.client.1.smithi157.stdout:9/949: chown d4/df/d61/fc1 47432 1 2024-04-22T23:35:12.591 INFO:tasks.workunit.client.1.smithi157.stdout:9/950: mkdir d4/df/d11d/d23/d27/db5/db9/dcd/d129 0 2024-04-22T23:35:12.592 INFO:tasks.workunit.client.1.smithi157.stdout:9/951: creat d4/df/d11d/d23/d3f/dde/f12a x:0 0 0 2024-04-22T23:35:12.592 INFO:tasks.workunit.client.1.smithi157.stdout:9/952: truncate d4/df/d11d/d23/d27/fb1 1408545 0 2024-04-22T23:35:12.599 INFO:tasks.workunit.client.1.smithi157.stdout:9/953: dread d4/df/d11d/d23/f88 [0,4194304] 0 2024-04-22T23:35:12.600 INFO:tasks.workunit.client.1.smithi157.stdout:9/954: creat d4/df/d11d/d23/d27/d75/f12b x:0 0 0 2024-04-22T23:35:12.600 INFO:tasks.workunit.client.1.smithi157.stdout:9/955: read d4/df/d11d/d23/d27/d47/f7f [859224,18942] 0 2024-04-22T23:35:12.614 INFO:tasks.workunit.client.1.smithi157.stdout:9/956: dwrite d4/df/d57/db7/f11b [0,4194304] 0 2024-04-22T23:35:12.628 INFO:tasks.workunit.client.1.smithi157.stdout:9/957: dwrite d4/df/d57/d6b/dc4/fe4 [0,4194304] 0 2024-04-22T23:35:12.630 INFO:tasks.workunit.client.1.smithi157.stdout:9/958: symlink d4/df/d11d/d23/l12c 0 2024-04-22T23:35:12.632 INFO:tasks.workunit.client.1.smithi157.stdout:9/959: truncate d4/df/d11d/d55/d8d/f126 1521555 0 2024-04-22T23:35:12.633 INFO:tasks.workunit.client.1.smithi157.stdout:9/960: truncate d4/df/d61/f84 2100051 0 2024-04-22T23:35:12.634 INFO:tasks.workunit.client.1.smithi157.stdout:9/961: mkdir d4/df/d11d/d12d 0 2024-04-22T23:35:12.634 INFO:tasks.workunit.client.1.smithi157.stdout:9/962: truncate d4/df/f117 627845 0 2024-04-22T23:35:12.636 INFO:tasks.workunit.client.1.smithi157.stdout:9/963: creat d4/df/d11d/d23/d27/dc0/f12e x:0 0 0 2024-04-22T23:35:12.636 INFO:tasks.workunit.client.1.smithi157.stdout:9/964: readlink d4/df/la5 0 2024-04-22T23:35:12.637 INFO:tasks.workunit.client.1.smithi157.stdout:9/965: stat d4/df/l17 0 2024-04-22T23:35:12.639 INFO:tasks.workunit.client.1.smithi157.stdout:9/966: unlink d4/df/d11d/d23/f88 0 2024-04-22T23:35:12.639 INFO:tasks.workunit.client.1.smithi157.stdout:9/967: fsync d4/df/d11d/d23/d27/f4f 0 2024-04-22T23:35:12.640 INFO:tasks.workunit.client.1.smithi157.stdout:9/968: symlink d4/df/d11d/d23/d27/db5/l12f 0 2024-04-22T23:35:12.642 INFO:tasks.workunit.client.1.smithi157.stdout:9/969: dread d4/df/d11d/d23/d27/fb1 [0,4194304] 0 2024-04-22T23:35:12.644 INFO:tasks.workunit.client.1.smithi157.stdout:9/970: mkdir d4/df/d52/d130 0 2024-04-22T23:35:12.645 INFO:tasks.workunit.client.1.smithi157.stdout:9/971: link d4/df/d11d/d23/d27/d75/f12b d4/df/d11d/d12d/f131 0 2024-04-22T23:35:12.648 INFO:tasks.workunit.client.1.smithi157.stdout:9/972: dread d4/df/f6a [0,4194304] 0 2024-04-22T23:35:12.649 INFO:tasks.workunit.client.1.smithi157.stdout:9/973: link d4/df/l17 d4/df/d11d/d23/dad/df2/l132 0 2024-04-22T23:35:12.651 INFO:tasks.workunit.client.1.smithi157.stdout:9/974: rename d4/df/d11d/d23/c89 to d4/df/d11d/d23/dad/c133 0 2024-04-22T23:35:12.651 INFO:tasks.workunit.client.1.smithi157.stdout:9/975: chown d4/df/d11d/d23/d27/d75/f12b 134909981 1 2024-04-22T23:35:12.656 INFO:tasks.workunit.client.1.smithi157.stdout:9/976: creat d4/df/d11d/d55/d103/f134 x:0 0 0 2024-04-22T23:35:12.656 INFO:tasks.workunit.client.1.smithi157.stdout:9/977: creat d4/df/d57/d6b/f135 x:0 0 0 2024-04-22T23:35:12.656 INFO:tasks.workunit.client.1.smithi157.stdout:9/978: fsync d4/df/f26 0 2024-04-22T23:35:12.657 INFO:tasks.workunit.client.1.smithi157.stdout:9/979: write d4/df/d57/d6b/f135 [116213,114579] 0 2024-04-22T23:35:12.658 INFO:tasks.workunit.client.0.smithi107.stdout:5/871: creat d9/db/de/d72/d7c/d109/f12f x:0 0 0 2024-04-22T23:35:12.660 INFO:tasks.workunit.client.1.smithi157.stdout:9/980: creat d4/df/d11d/d55/d8d/f136 x:0 0 0 2024-04-22T23:35:12.699 INFO:tasks.workunit.client.0.smithi107.stdout:3/779: mkdir db/d21/d108 0 2024-04-22T23:35:12.700 INFO:tasks.workunit.client.0.smithi107.stdout:9/802: creat d5/d14/d2e/d4f/d73/df6/f116 x:0 0 0 2024-04-22T23:35:12.702 INFO:tasks.workunit.client.0.smithi107.stdout:8/830: sync 2024-04-22T23:35:12.703 INFO:tasks.workunit.client.1.smithi157.stdout:5/997: sync 2024-04-22T23:35:12.703 INFO:tasks.workunit.client.1.smithi157.stdout:7/943: sync 2024-04-22T23:35:12.703 INFO:tasks.workunit.client.1.smithi157.stdout:7/944: fsync d0/d9/f3c 0 2024-04-22T23:35:12.704 INFO:tasks.workunit.client.1.smithi157.stdout:5/998: symlink d1/d5/d7/d2f/d45/d54/d59/l161 0 2024-04-22T23:35:12.704 INFO:tasks.workunit.client.1.smithi157.stdout:5/999: dread - d1/d5/dc/d43/d4f/fec zero size 2024-04-22T23:35:12.705 INFO:tasks.workunit.client.1.smithi157.stdout:7/945: mkdir d0/d9/d29/d49/d64/d11f 0 2024-04-22T23:35:12.706 INFO:tasks.workunit.client.1.smithi157.stdout:7/946: creat d0/d9/db/da3/d10c/f120 x:0 0 0 2024-04-22T23:35:12.707 INFO:tasks.workunit.client.0.smithi107.stdout:4/868: mkdir d1/d7/d5b/dbf/da8/d135 0 2024-04-22T23:35:12.707 INFO:tasks.workunit.client.0.smithi107.stdout:4/869: dread - d1/d7/d5b/de3/f11c zero size 2024-04-22T23:35:12.711 INFO:tasks.workunit.client.0.smithi107.stdout:7/799: getdents d7/d21/d48/d95 0 2024-04-22T23:35:12.712 INFO:tasks.workunit.client.1.smithi157.stdout:7/947: dread d0/d9/de/f5f [4194304,4194304] 0 2024-04-22T23:35:12.713 INFO:tasks.workunit.client.1.smithi157.stdout:7/948: truncate d0/f46 4448268 0 2024-04-22T23:35:12.713 INFO:tasks.workunit.client.0.smithi107.stdout:3/780: mkdir db/d21/d108/d109 0 2024-04-22T23:35:12.715 INFO:tasks.workunit.client.0.smithi107.stdout:9/803: creat d5/d7/d27/d46/d4b/def/f117 x:0 0 0 2024-04-22T23:35:12.715 INFO:tasks.workunit.client.0.smithi107.stdout:9/804: write d5/d7/d17/d1b/d1f/d36/fb8 [214263,76399] 0 2024-04-22T23:35:12.715 INFO:tasks.workunit.client.0.smithi107.stdout:9/805: chown d5/d29/f4d 490 1 2024-04-22T23:35:12.716 INFO:tasks.workunit.client.0.smithi107.stdout:8/831: truncate d0/f5 2636056 0 2024-04-22T23:35:12.717 INFO:tasks.workunit.client.1.smithi157.stdout:7/949: write d0/d9/d29/d33/d37/f50 [1788364,42321] 0 2024-04-22T23:35:12.723 INFO:tasks.workunit.client.0.smithi107.stdout:3/781: dread db/d21/d27/d15/d29/f6d [0,4194304] 0 2024-04-22T23:35:12.730 INFO:tasks.workunit.client.0.smithi107.stdout:9/806: dwrite f4 [0,4194304] 0 2024-04-22T23:35:12.732 INFO:tasks.workunit.client.0.smithi107.stdout:7/800: rmdir d7/d21/d7e/d94 39 2024-04-22T23:35:12.733 INFO:tasks.workunit.client.0.smithi107.stdout:8/832: rename d0/d7/d20/da2/cfb to d0/d7/d20/da2/dc3/c121 0 2024-04-22T23:35:12.741 INFO:tasks.workunit.client.0.smithi107.stdout:1/754: sync 2024-04-22T23:35:12.741 INFO:tasks.workunit.client.0.smithi107.stdout:1/755: write d7/d19/d20/d39/f7e [862044,8562] 0 2024-04-22T23:35:12.742 INFO:tasks.workunit.client.0.smithi107.stdout:2/894: sync 2024-04-22T23:35:12.742 INFO:tasks.workunit.client.0.smithi107.stdout:6/894: sync 2024-04-22T23:35:12.743 INFO:tasks.workunit.client.0.smithi107.stdout:0/816: sync 2024-04-22T23:35:12.743 INFO:tasks.workunit.client.0.smithi107.stdout:4/870: rmdir d1 39 2024-04-22T23:35:12.743 INFO:tasks.workunit.client.0.smithi107.stdout:3/782: getdents db/d21/d27/d15/d29/d2a/d45/ddd/d25 0 2024-04-22T23:35:12.765 INFO:tasks.workunit.client.0.smithi107.stdout:0/817: dwrite d1/d27/d20/d2c/d41/f54 [0,4194304] 0 2024-04-22T23:35:12.765 INFO:tasks.workunit.client.0.smithi107.stdout:0/818: chown d1/d27/d20/d2c/l2f 138 1 2024-04-22T23:35:12.765 INFO:tasks.workunit.client.0.smithi107.stdout:3/783: dwrite db/d21/d27/d15/d24/d87/f89 [0,4194304] 0 2024-04-22T23:35:12.778 INFO:tasks.workunit.client.0.smithi107.stdout:9/807: chown d5/d29/c41 66535 1 2024-04-22T23:35:12.778 INFO:tasks.workunit.client.0.smithi107.stdout:9/808: fsync d5/d29/d7b/f115 0 2024-04-22T23:35:12.840 INFO:tasks.workunit.client.0.smithi107.stdout:7/801: symlink d7/d21/d48/daf/l10e 0 2024-04-22T23:35:12.841 INFO:tasks.workunit.client.0.smithi107.stdout:8/833: creat d0/d73/db0/d107/f122 x:0 0 0 2024-04-22T23:35:12.845 INFO:tasks.workunit.client.0.smithi107.stdout:1/756: symlink d7/d19/d20/lfb 0 2024-04-22T23:35:12.845 INFO:tasks.workunit.client.0.smithi107.stdout:7/802: dread d7/d21/d48/fc2 [0,4194304] 0 2024-04-22T23:35:12.845 INFO:tasks.workunit.client.0.smithi107.stdout:6/895: getdents db/d20/d2c 0 2024-04-22T23:35:12.848 INFO:tasks.workunit.client.0.smithi107.stdout:2/895: rename d1/d2b/d88/ld6 to d1/d12/d35/d102/d7e/da8/l129 0 2024-04-22T23:35:12.848 INFO:tasks.workunit.client.0.smithi107.stdout:2/896: stat d1/d12/f31 0 2024-04-22T23:35:12.848 INFO:tasks.workunit.client.0.smithi107.stdout:2/897: stat d1/d12/d35/d48/d13/cfc 0 2024-04-22T23:35:12.849 INFO:tasks.workunit.client.0.smithi107.stdout:4/871: truncate d1/d21/d52/f70 2568428 0 2024-04-22T23:35:12.851 INFO:tasks.workunit.client.0.smithi107.stdout:0/819: symlink d1/d7b/l10e 0 2024-04-22T23:35:12.854 INFO:tasks.workunit.client.0.smithi107.stdout:3/784: mkdir db/d21/d27/d15/d24/d87/d10a 0 2024-04-22T23:35:12.854 INFO:tasks.workunit.client.0.smithi107.stdout:3/785: dread - db/d21/d27/d15/d24/d87/faf zero size 2024-04-22T23:35:12.854 INFO:tasks.workunit.client.0.smithi107.stdout:3/786: truncate db/d21/d27/d15/d29/d2a/d45/deb/fd6 658932 0 2024-04-22T23:35:12.856 INFO:tasks.workunit.client.0.smithi107.stdout:3/787: write db/d21/d27/d15/d24/d52/d60/f93 [369769,113702] 0 2024-04-22T23:35:12.859 INFO:tasks.workunit.client.0.smithi107.stdout:9/809: write d5/d29/f61 [2068108,49203] 0 2024-04-22T23:35:12.859 INFO:tasks.workunit.client.0.smithi107.stdout:9/810: dread - d5/d7/d27/d46/d4b/def/f117 zero size 2024-04-22T23:35:12.860 INFO:tasks.workunit.client.0.smithi107.stdout:2/898: mknod d1/d2b/c12a 0 2024-04-22T23:35:12.863 INFO:tasks.workunit.client.0.smithi107.stdout:1/757: rename d7/d19/d20/d39/d53/d43/d46/d68/l87 to d7/d19/lfc 0 2024-04-22T23:35:12.863 INFO:tasks.workunit.client.0.smithi107.stdout:1/758: fsync d7/d19/d20/d39/d53/d43/d63/f74 0 2024-04-22T23:35:12.863 INFO:tasks.workunit.client.0.smithi107.stdout:1/759: fsync d7/f17 0 2024-04-22T23:35:12.863 INFO:tasks.workunit.client.0.smithi107.stdout:1/760: creat d7/d19/d42/ffd x:0 0 0 2024-04-22T23:35:12.863 INFO:tasks.workunit.client.0.smithi107.stdout:1/761: write d7/d19/d20/d39/d53/ff7 [1035178,18706] 0 2024-04-22T23:35:12.868 INFO:tasks.workunit.client.0.smithi107.stdout:6/896: dwrite db/d20/d58/fb9 [0,4194304] 0 2024-04-22T23:35:12.868 INFO:tasks.workunit.client.0.smithi107.stdout:6/897: write db/d20/d2a/d38/f53 [4668530,118267] 0 2024-04-22T23:35:12.869 INFO:tasks.workunit.client.0.smithi107.stdout:8/834: dwrite d0/d73/db0/d107/f122 [0,4194304] 0 2024-04-22T23:35:12.869 INFO:tasks.workunit.client.0.smithi107.stdout:8/835: write d0/d7/d20/da2/f105 [646987,61352] 0 2024-04-22T23:35:12.869 INFO:tasks.workunit.client.0.smithi107.stdout:8/836: chown d0/c6d 6828 1 2024-04-22T23:35:12.870 INFO:tasks.workunit.client.0.smithi107.stdout:0/820: unlink d1/d84/f6c 0 2024-04-22T23:35:12.873 INFO:tasks.workunit.client.0.smithi107.stdout:2/899: mknod d1/d12/d35/d102/d7e/dae/de8/c12b 0 2024-04-22T23:35:12.873 INFO:tasks.workunit.client.0.smithi107.stdout:2/900: readlink d1/d67/ld0 0 2024-04-22T23:35:12.877 INFO:tasks.workunit.client.0.smithi107.stdout:9/811: dwrite d5/d29/d7b/d91/dde/dc9/daf/fd0 [0,4194304] 0 2024-04-22T23:35:12.879 INFO:tasks.workunit.client.0.smithi107.stdout:8/837: write d0/d7/fa [3978720,19942] 0 2024-04-22T23:35:12.882 INFO:tasks.workunit.client.0.smithi107.stdout:0/821: dread d1/d27/d20/f36 [0,4194304] 0 2024-04-22T23:35:12.882 INFO:tasks.workunit.client.0.smithi107.stdout:0/822: write d1/d84/fc8 [681854,10632] 0 2024-04-22T23:35:12.979 INFO:tasks.workunit.client.1.smithi157.stdout:9/981: sync 2024-04-22T23:35:12.979 INFO:tasks.workunit.client.1.smithi157.stdout:7/950: sync 2024-04-22T23:35:12.979 INFO:tasks.workunit.client.1.smithi157.stdout:8/797: sync 2024-04-22T23:35:12.979 INFO:tasks.workunit.client.1.smithi157.stdout:0/958: sync 2024-04-22T23:35:12.980 INFO:tasks.workunit.client.1.smithi157.stdout:0/959: write d17/d2b/d52/dd1/ff0 [814839,78933] 0 2024-04-22T23:35:12.980 INFO:tasks.workunit.client.1.smithi157.stdout:7/951: write d0/d9/d42/d55/d80/fa7 [332120,120551] 0 2024-04-22T23:35:12.980 INFO:tasks.workunit.client.1.smithi157.stdout:0/960: write d17/d2b/d5b/f124 [1007358,119431] 0 2024-04-22T23:35:12.991 INFO:tasks.workunit.client.1.smithi157.stdout:9/982: dread d4/df/d61/fcb [0,4194304] 0 2024-04-22T23:35:12.991 INFO:tasks.workunit.client.1.smithi157.stdout:9/983: dread - d4/df/d11d/d12d/f131 zero size 2024-04-22T23:35:12.991 INFO:tasks.workunit.client.1.smithi157.stdout:9/984: chown d4/df/d57/db7/df9 6107356 1 2024-04-22T23:35:12.991 INFO:tasks.workunit.client.1.smithi157.stdout:9/985: getdents d4/df/d57/df5 0 2024-04-22T23:35:13.054 INFO:tasks.workunit.client.0.smithi107.stdout:6/898: mkdir db/d20/d2a/d38/d4b/d110 0 2024-04-22T23:35:13.054 INFO:tasks.workunit.client.0.smithi107.stdout:6/899: getdents db/d20/d2a/d38/d61/d6f/da2/d84 0 2024-04-22T23:35:13.054 INFO:tasks.workunit.client.0.smithi107.stdout:6/900: chown db/d20/d2c/d3b/l90 336131 1 2024-04-22T23:35:13.059 INFO:tasks.workunit.client.0.smithi107.stdout:3/788: rename db/d21/cd4 to db/d21/d27/d36/c10b 0 2024-04-22T23:35:13.060 INFO:tasks.workunit.client.0.smithi107.stdout:2/901: rename d1 to d1/dcd/df0/d12c 22 2024-04-22T23:35:13.060 INFO:tasks.workunit.client.0.smithi107.stdout:2/902: dread - d1/d63/fb6 zero size 2024-04-22T23:35:13.060 INFO:tasks.workunit.client.0.smithi107.stdout:2/903: truncate d1/d2b/d3d/fca 610011 0 2024-04-22T23:35:13.060 INFO:tasks.workunit.client.0.smithi107.stdout:1/762: getdents d7/d19/d20/d39/d93/df4/db8 0 2024-04-22T23:35:13.065 INFO:tasks.workunit.client.0.smithi107.stdout:8/838: mknod d0/d7/d20/df9/c123 0 2024-04-22T23:35:13.065 INFO:tasks.workunit.client.0.smithi107.stdout:8/839: chown d0/c6d 12902565 1 2024-04-22T23:35:13.067 INFO:tasks.workunit.client.0.smithi107.stdout:3/789: dread db/d21/db1/dcd/ff8 [0,4194304] 0 2024-04-22T23:35:13.068 INFO:tasks.workunit.client.0.smithi107.stdout:3/790: write db/d21/d27/d15/d29/d2a/d45/f8a [492157,18125] 0 2024-04-22T23:35:13.074 INFO:tasks.workunit.client.0.smithi107.stdout:6/901: rmdir db/d20/d2c 39 2024-04-22T23:35:13.076 INFO:tasks.workunit.client.0.smithi107.stdout:9/812: rename d5/d14/f15 to d5/d7/d27/d46/dc7/f118 0 2024-04-22T23:35:13.076 INFO:tasks.workunit.client.0.smithi107.stdout:9/813: creat d5/d7/d17/d1b/f119 x:0 0 0 2024-04-22T23:35:13.079 INFO:tasks.workunit.client.0.smithi107.stdout:2/904: unlink d1/l4d 0 2024-04-22T23:35:13.083 INFO:tasks.workunit.client.0.smithi107.stdout:1/763: rename d7/d19/d20/d39/d53/d43/f7a to d7/d19/d42/ded/ffe 0 2024-04-22T23:35:13.085 INFO:tasks.workunit.client.0.smithi107.stdout:3/791: dwrite db/d21/d27/d15/d29/d2a/d45/ddd/d25/fb6 [0,4194304] 0 2024-04-22T23:35:13.085 INFO:tasks.workunit.client.0.smithi107.stdout:3/792: chown db/d21/d27/d15/d24/d52/d60/f92 3408 1 2024-04-22T23:35:13.085 INFO:tasks.workunit.client.0.smithi107.stdout:3/793: truncate db/d21/d27/d15/d24/d87/f104 178997 0 2024-04-22T23:35:13.086 INFO:tasks.workunit.client.0.smithi107.stdout:2/905: mkdir d1/d12/d35/d102/d7e/da8/d125/d12d 0 2024-04-22T23:35:13.088 INFO:tasks.workunit.client.0.smithi107.stdout:1/764: mknod d7/d19/d20/d39/d53/d43/d46/d5c/cff 0 2024-04-22T23:35:13.088 INFO:tasks.workunit.client.0.smithi107.stdout:1/765: readlink d7/d19/lca 0 2024-04-22T23:35:13.092 INFO:tasks.workunit.client.0.smithi107.stdout:1/766: rmdir d7/d19/d20/d39/d53/d43/d46/d5c 39 2024-04-22T23:35:13.092 INFO:tasks.workunit.client.0.smithi107.stdout:1/767: read - d7/d19/d20/fdc zero size 2024-04-22T23:35:13.092 INFO:tasks.workunit.client.0.smithi107.stdout:7/803: sync 2024-04-22T23:35:13.093 INFO:tasks.workunit.client.0.smithi107.stdout:1/768: creat d7/d19/f100 x:0 0 0 2024-04-22T23:35:13.094 INFO:tasks.workunit.client.0.smithi107.stdout:1/769: truncate d7/f15 3525614 0 2024-04-22T23:35:13.094 INFO:tasks.workunit.client.0.smithi107.stdout:4/872: sync 2024-04-22T23:35:13.095 INFO:tasks.workunit.client.0.smithi107.stdout:0/823: sync 2024-04-22T23:35:13.095 INFO:tasks.workunit.client.0.smithi107.stdout:0/824: write d1/d27/d20/d2c/d41/dbe/fdf [1400457,95592] 0 2024-04-22T23:35:13.095 INFO:tasks.workunit.client.0.smithi107.stdout:1/770: mknod d7/d19/d20/d39/d53/d43/d46/d68/c101 0 2024-04-22T23:35:13.095 INFO:tasks.workunit.client.0.smithi107.stdout:0/825: mkdir d1/d27/d8a/db5/d10f 0 2024-04-22T23:35:13.098 INFO:tasks.workunit.client.0.smithi107.stdout:6/902: dwrite db/d20/d2a/d38/d61/d6f/da2/fdb [0,4194304] 0 2024-04-22T23:35:13.098 INFO:tasks.workunit.client.0.smithi107.stdout:6/903: truncate db/d20/d2a/d35/d69/db3/dbc/ffb 23198 0 2024-04-22T23:35:13.098 INFO:tasks.workunit.client.0.smithi107.stdout:6/904: fdatasync db/d20/d2a/f10c 0 2024-04-22T23:35:13.099 INFO:tasks.workunit.client.0.smithi107.stdout:9/814: dwrite d5/f11 [0,4194304] 0 2024-04-22T23:35:13.099 INFO:tasks.workunit.client.0.smithi107.stdout:9/815: dread - d5/d7/d27/d46/d4b/f110 zero size 2024-04-22T23:35:13.099 INFO:tasks.workunit.client.0.smithi107.stdout:9/816: read d5/d14/d28/f59 [3349331,126209] 0 2024-04-22T23:35:13.101 INFO:tasks.workunit.client.0.smithi107.stdout:6/905: link db/d20/d49/d7b/c80 db/d20/d2c/c111 0 2024-04-22T23:35:13.101 INFO:tasks.workunit.client.0.smithi107.stdout:3/794: dread db/d21/d27/d36/fb0 [0,4194304] 0 2024-04-22T23:35:13.102 INFO:tasks.workunit.client.0.smithi107.stdout:9/817: stat d5/d29/d7b/d91/dde/l72 0 2024-04-22T23:35:13.102 INFO:tasks.workunit.client.0.smithi107.stdout:9/818: chown d5/d14/d28/de3/ce6 4 1 2024-04-22T23:35:13.104 INFO:tasks.workunit.client.0.smithi107.stdout:3/795: creat db/d21/d108/d109/f10c x:0 0 0 2024-04-22T23:35:13.106 INFO:tasks.workunit.client.0.smithi107.stdout:9/819: creat d5/d14/d2e/d4f/d83/f11a x:0 0 0 2024-04-22T23:35:13.106 INFO:tasks.workunit.client.0.smithi107.stdout:9/820: readlink d5/d7/l37 0 2024-04-22T23:35:13.106 INFO:tasks.workunit.client.0.smithi107.stdout:4/873: dread d1/d1b/f104 [0,4194304] 0 2024-04-22T23:35:13.108 INFO:tasks.workunit.client.0.smithi107.stdout:3/796: truncate db/d21/d27/d15/d29/d2a/f5b 3959347 0 2024-04-22T23:35:13.108 INFO:tasks.workunit.client.0.smithi107.stdout:3/797: write db/d21/d27/d15/d29/d2a/d45/ddd/d25/fe2 [2082812,109001] 0 2024-04-22T23:35:13.108 INFO:tasks.workunit.client.0.smithi107.stdout:3/798: chown db/d21/d27/d15/d24/d66/fc6 1789424 1 2024-04-22T23:35:13.108 INFO:tasks.workunit.client.0.smithi107.stdout:3/799: chown db/d21/d27/d15/d29/d2a/d45/ddd/d25/lff 157145777 1 2024-04-22T23:35:13.109 INFO:tasks.workunit.client.0.smithi107.stdout:9/821: creat d5/d14/df7/f11b x:0 0 0 2024-04-22T23:35:13.110 INFO:tasks.workunit.client.0.smithi107.stdout:9/822: write d5/d29/f61 [3259946,119304] 0 2024-04-22T23:35:13.110 INFO:tasks.workunit.client.0.smithi107.stdout:9/823: truncate d5/d7/d27/d46/fbd 770336 0 2024-04-22T23:35:13.110 INFO:tasks.workunit.client.0.smithi107.stdout:1/771: dread d7/d19/d2d/f31 [0,4194304] 0 2024-04-22T23:35:13.110 INFO:tasks.workunit.client.0.smithi107.stdout:1/772: stat d7/d19/d42/c50 0 2024-04-22T23:35:13.110 INFO:tasks.workunit.client.0.smithi107.stdout:4/874: creat d1/d21/f136 x:0 0 0 2024-04-22T23:35:13.114 INFO:tasks.workunit.client.0.smithi107.stdout:9/824: symlink d5/d14/d28/de3/l11c 0 2024-04-22T23:35:13.114 INFO:tasks.workunit.client.0.smithi107.stdout:9/825: write d5/d14/d2e/d4f/fb7 [874880,66457] 0 2024-04-22T23:35:13.116 INFO:tasks.workunit.client.0.smithi107.stdout:9/826: creat d5/d14/d2e/d4f/d73/f11d x:0 0 0 2024-04-22T23:35:13.116 INFO:tasks.workunit.client.0.smithi107.stdout:9/827: fdatasync d5/d7/d17/d1b/d1f/f112 0 2024-04-22T23:35:13.116 INFO:tasks.workunit.client.0.smithi107.stdout:0/826: dwrite d1/d9/f3e [4194304,4194304] 0 2024-04-22T23:35:13.116 INFO:tasks.workunit.client.0.smithi107.stdout:0/827: truncate d1/d84/fc8 1288245 0 2024-04-22T23:35:13.120 INFO:tasks.workunit.client.0.smithi107.stdout:9/828: rename d5/d29/ldc to d5/d14/l11e 0 2024-04-22T23:35:13.121 INFO:tasks.workunit.client.0.smithi107.stdout:9/829: getdents d5/d7/d27/d46/dd3 0 2024-04-22T23:35:13.121 INFO:tasks.workunit.client.0.smithi107.stdout:9/830: write d5/d7/d27/d46/d4b/f56 [194561,61678] 0 2024-04-22T23:35:13.121 INFO:tasks.workunit.client.0.smithi107.stdout:9/831: fdatasync d5/d7/f16 0 2024-04-22T23:35:13.122 INFO:tasks.workunit.client.0.smithi107.stdout:9/832: mkdir d5/d14/d28/d11f 0 2024-04-22T23:35:13.126 INFO:tasks.workunit.client.0.smithi107.stdout:4/875: getdents d1/d21 0 2024-04-22T23:35:13.127 INFO:tasks.workunit.client.0.smithi107.stdout:4/876: stat d1/d21/d29/d2e/d88/d99/dd2/daf/fe8 0 2024-04-22T23:35:13.127 INFO:tasks.workunit.client.0.smithi107.stdout:9/833: symlink d5/d7/d27/l120 0 2024-04-22T23:35:13.127 INFO:tasks.workunit.client.0.smithi107.stdout:9/834: dread - d5/d7/d17/d1b/fc5 zero size 2024-04-22T23:35:13.127 INFO:tasks.workunit.client.0.smithi107.stdout:0/828: write d1/d9/d14/d4f/d59/fe2 [749725,12728] 0 2024-04-22T23:35:13.127 INFO:tasks.workunit.client.0.smithi107.stdout:4/877: mknod d1/d7/d63/d78/d96/da0/de9/c137 0 2024-04-22T23:35:13.127 INFO:tasks.workunit.client.0.smithi107.stdout:4/878: chown d1/c11b 53747910 1 2024-04-22T23:35:13.128 INFO:tasks.workunit.client.0.smithi107.stdout:0/829: dread d1/d27/d20/d2c/d41/dbe/fdf [0,4194304] 0 2024-04-22T23:35:13.128 INFO:tasks.workunit.client.0.smithi107.stdout:0/830: chown d1/d2a/dad/c8e 167155 1 2024-04-22T23:35:13.128 INFO:tasks.workunit.client.0.smithi107.stdout:0/831: write d1/d9/d14/d4f/f101 [902016,27906] 0 2024-04-22T23:35:13.128 INFO:tasks.workunit.client.0.smithi107.stdout:0/832: chown d1/d27/d20/d2c/fbb 190743 1 2024-04-22T23:35:13.129 INFO:tasks.workunit.client.0.smithi107.stdout:9/835: creat d5/d7/d27/d46/d4b/dff/f121 x:0 0 0 2024-04-22T23:35:13.129 INFO:tasks.workunit.client.0.smithi107.stdout:9/836: dread - d5/d14/d2e/ffa zero size 2024-04-22T23:35:13.130 INFO:tasks.workunit.client.0.smithi107.stdout:4/879: symlink d1/d7/d5b/l138 0 2024-04-22T23:35:13.136 INFO:tasks.workunit.client.1.smithi157.stdout:8/798: unlink d2/d6/d21/l39 0 2024-04-22T23:35:13.136 INFO:tasks.workunit.client.1.smithi157.stdout:8/799: write d2/d4/f112 [883644,82622] 0 2024-04-22T23:35:13.140 INFO:tasks.workunit.client.0.smithi107.stdout:9/837: dread d5/d29/f4d [0,4194304] 0 2024-04-22T23:35:13.148 INFO:tasks.workunit.client.0.smithi107.stdout:9/838: read d5/d6c/f87 [2329791,27854] 0 2024-04-22T23:35:13.149 INFO:tasks.workunit.client.0.smithi107.stdout:0/833: dwrite d1/d9/d14/d4f/fe1 [0,4194304] 0 2024-04-22T23:35:13.149 INFO:tasks.workunit.client.0.smithi107.stdout:4/880: dwrite d1/d21/d29/d2e/d88/d99/dd2/fcf [0,4194304] 0 2024-04-22T23:35:13.149 INFO:tasks.workunit.client.0.smithi107.stdout:4/881: write d1/d21/d29/d2e/d88/d99/fb5 [4947012,11368] 0 2024-04-22T23:35:13.151 INFO:tasks.workunit.client.0.smithi107.stdout:9/839: unlink d5/d7/d17/d1b/d1f/l103 0 2024-04-22T23:35:13.151 INFO:tasks.workunit.client.0.smithi107.stdout:9/840: chown d5/d7/d17/d1b/dcb/f114 13205 1 2024-04-22T23:35:13.151 INFO:tasks.workunit.client.0.smithi107.stdout:9/841: write d5/d7/d17/d1b/f30 [5380150,5106] 0 2024-04-22T23:35:13.151 INFO:tasks.workunit.client.0.smithi107.stdout:9/842: chown d5/d7/d27/d69/db6/da8 4 1 2024-04-22T23:35:13.152 INFO:tasks.workunit.client.0.smithi107.stdout:0/834: creat d1/d27/d20/f110 x:0 0 0 2024-04-22T23:35:13.153 INFO:tasks.workunit.client.0.smithi107.stdout:0/835: chown d1/d84/d99/fe6 67 1 2024-04-22T23:35:13.158 INFO:tasks.workunit.client.0.smithi107.stdout:4/882: write d1/d7/d5b/dbf/fdc [1864121,32321] 0 2024-04-22T23:35:13.163 INFO:tasks.workunit.client.0.smithi107.stdout:9/843: dwrite d5/d7/d27/d46/d4b/f110 [0,4194304] 0 2024-04-22T23:35:13.249 INFO:tasks.workunit.client.1.smithi157.stdout:7/952: mkdir d0/d9/d42/d55/d80/d95/df8/d121 0 2024-04-22T23:35:13.250 INFO:tasks.workunit.client.1.smithi157.stdout:0/961: rename d17/d2b/d4f/d5a/d7c/l8a to d17/d2b/d4f/d5a/d145/l147 0 2024-04-22T23:35:13.434 INFO:tasks.workunit.client.1.smithi157.stdout:9/986: mkdir d4/df/d11d/d23/d27/d137 0 2024-04-22T23:35:13.435 INFO:tasks.workunit.client.1.smithi157.stdout:8/800: mknod d2/d34/c115 0 2024-04-22T23:35:13.436 INFO:tasks.workunit.client.1.smithi157.stdout:7/953: rename d0/d9/db/df/l99 to d0/d9/d29/d33/l122 0 2024-04-22T23:35:13.437 INFO:tasks.workunit.client.1.smithi157.stdout:0/962: creat d17/d23/d34/d67/d136/d143/de4/f148 x:0 0 0 2024-04-22T23:35:13.438 INFO:tasks.workunit.client.1.smithi157.stdout:9/987: creat d4/df/d52/d130/f138 x:0 0 0 2024-04-22T23:35:13.439 INFO:tasks.workunit.client.1.smithi157.stdout:8/801: creat d2/d4/dbd/d67/d10f/f116 x:0 0 0 2024-04-22T23:35:13.439 INFO:tasks.workunit.client.1.smithi157.stdout:7/954: symlink d0/d9/d36/d105/l123 0 2024-04-22T23:35:13.440 INFO:tasks.workunit.client.1.smithi157.stdout:7/955: truncate d0/d9/db/df/d68/ff4 682768 0 2024-04-22T23:35:13.441 INFO:tasks.workunit.client.1.smithi157.stdout:0/963: creat d17/d2b/d52/d134/f149 x:0 0 0 2024-04-22T23:35:13.441 INFO:tasks.workunit.client.1.smithi157.stdout:0/964: chown d17/d23/d34/d67/d136/d143/d10a/d13b/l142 16778060 1 2024-04-22T23:35:13.442 INFO:tasks.workunit.client.1.smithi157.stdout:9/988: symlink d4/df/d11d/d23/d27/d75/l139 0 2024-04-22T23:35:13.443 INFO:tasks.workunit.client.1.smithi157.stdout:7/956: creat d0/d9/db/df/d68/dc6/f124 x:0 0 0 2024-04-22T23:35:13.443 INFO:tasks.workunit.client.1.smithi157.stdout:7/957: chown d0/d9/d42/d55/d80/d95/dce 4 1 2024-04-22T23:35:13.443 INFO:tasks.workunit.client.1.smithi157.stdout:7/958: stat d0/d9/d29/d5e 0 2024-04-22T23:35:13.445 INFO:tasks.workunit.client.1.smithi157.stdout:0/965: truncate d17/d2b/d71/fac 3677083 0 2024-04-22T23:35:13.449 INFO:tasks.workunit.client.1.smithi157.stdout:8/802: read d2/d6/d21/d5f/d11/d13/fff [499947,7862] 0 2024-04-22T23:35:13.449 INFO:tasks.workunit.client.1.smithi157.stdout:0/966: symlink d17/d2b/d52/ddc/l14a 0 2024-04-22T23:35:13.452 INFO:tasks.workunit.client.1.smithi157.stdout:9/989: dread d4/df/d11d/d55/f9b [0,4194304] 0 2024-04-22T23:35:13.486 INFO:tasks.workunit.client.0.smithi107.stdout:0/836: symlink d1/d27/d20/l111 0 2024-04-22T23:35:13.486 INFO:tasks.workunit.client.0.smithi107.stdout:0/837: truncate d1/d9/d14/d4f/fe8 738660 0 2024-04-22T23:35:13.487 INFO:tasks.workunit.client.0.smithi107.stdout:4/883: rename d1/d21/d29/d2e/d88/d99/dd2/f1f to d1/d21/d29/d2e/dfa/f139 0 2024-04-22T23:35:13.488 INFO:tasks.workunit.client.0.smithi107.stdout:4/884: truncate d1/d21/d29/d2e/fac 1622798 0 2024-04-22T23:35:13.488 INFO:tasks.workunit.client.0.smithi107.stdout:4/885: read d1/d7/fc [699329,69747] 0 2024-04-22T23:35:13.491 INFO:tasks.workunit.client.0.smithi107.stdout:4/886: dread d1/d21/d29/d2e/fac [0,4194304] 0 2024-04-22T23:35:13.491 INFO:tasks.workunit.client.0.smithi107.stdout:4/887: dread - d1/d21/d52/f9f zero size 2024-04-22T23:35:13.501 INFO:tasks.workunit.client.0.smithi107.stdout:9/844: write d5/d7/d27/d46/d4b/fa9 [282364,35129] 0 2024-04-22T23:35:13.501 INFO:tasks.workunit.client.0.smithi107.stdout:9/845: chown d5/d14/fca 225530 1 2024-04-22T23:35:13.501 INFO:tasks.workunit.client.0.smithi107.stdout:4/888: getdents d1/d1b/d44 0 2024-04-22T23:35:13.502 INFO:tasks.workunit.client.0.smithi107.stdout:4/889: unlink d1/d21/d29/d2e/d88/d99/l12e 0 2024-04-22T23:35:13.505 INFO:tasks.workunit.client.0.smithi107.stdout:5/872: dwrite d9/db/d92/dc2/ff9 [0,4194304] 0 2024-04-22T23:35:13.509 INFO:tasks.workunit.client.0.smithi107.stdout:3/800: sync 2024-04-22T23:35:13.509 INFO:tasks.workunit.client.0.smithi107.stdout:8/840: sync 2024-04-22T23:35:13.509 INFO:tasks.workunit.client.0.smithi107.stdout:6/906: sync 2024-04-22T23:35:13.509 INFO:tasks.workunit.client.0.smithi107.stdout:1/773: sync 2024-04-22T23:35:13.509 INFO:tasks.workunit.client.0.smithi107.stdout:7/804: sync 2024-04-22T23:35:13.509 INFO:tasks.workunit.client.0.smithi107.stdout:2/906: sync 2024-04-22T23:35:13.509 INFO:tasks.workunit.client.0.smithi107.stdout:6/907: chown db/d20/d2a/d5f/cef 19551 1 2024-04-22T23:35:13.509 INFO:tasks.workunit.client.0.smithi107.stdout:3/801: stat db/d21/d27/d15/d29/d2a/d45/ddd/d25/fb6 0 2024-04-22T23:35:13.509 INFO:tasks.workunit.client.0.smithi107.stdout:1/774: fdatasync d7/d19/d20/d39/d53/d43/d46/dba/fe5 0 2024-04-22T23:35:13.509 INFO:tasks.workunit.client.0.smithi107.stdout:6/908: chown db/d20/l36 9 1 2024-04-22T23:35:13.509 INFO:tasks.workunit.client.0.smithi107.stdout:8/841: truncate d0/d7/d20/df9/f100 4460766 0 2024-04-22T23:35:13.509 INFO:tasks.workunit.client.0.smithi107.stdout:7/805: chown d7/d21/d48/d5c/f75 129198253 1 2024-04-22T23:35:13.510 INFO:tasks.workunit.client.0.smithi107.stdout:7/806: chown d7/d21/da4/c3d 45 1 2024-04-22T23:35:13.513 INFO:tasks.workunit.client.0.smithi107.stdout:2/907: link d1/d12/d35/d102/dba/df1/c112 d1/d2b/d59/c12e 0 2024-04-22T23:35:13.513 INFO:tasks.workunit.client.0.smithi107.stdout:2/908: creat d1/d12/d35/d48/d13/f12f x:0 0 0 2024-04-22T23:35:13.513 INFO:tasks.workunit.client.0.smithi107.stdout:2/909: truncate d1/d12/d35/d48/f9d 4556697 0 2024-04-22T23:35:13.513 INFO:tasks.workunit.client.0.smithi107.stdout:2/910: creat d1/d2b/d3d/d57/f130 x:0 0 0 2024-04-22T23:35:13.514 INFO:tasks.workunit.client.0.smithi107.stdout:3/802: mknod db/d21/db1/dbb/c10d 0 2024-04-22T23:35:13.514 INFO:tasks.workunit.client.0.smithi107.stdout:3/803: write db/d21/d27/d15/d29/d2a/d45/deb/fd6 [1481866,72865] 0 2024-04-22T23:35:13.514 INFO:tasks.workunit.client.0.smithi107.stdout:9/846: write d5/d7/d27/d46/dd3/fd1 [2778349,87602] 0 2024-04-22T23:35:13.515 INFO:tasks.workunit.client.0.smithi107.stdout:1/775: creat d7/d19/d2d/d89/f102 x:0 0 0 2024-04-22T23:35:13.520 INFO:tasks.workunit.client.0.smithi107.stdout:2/911: rename d1/d2b/d88/fab to d1/d12/d35/d102/dba/df1/f131 0 2024-04-22T23:35:13.523 INFO:tasks.workunit.client.0.smithi107.stdout:3/804: link db/d21/d27/d15/d24/d52/d60/lc7 db/d21/d27/d15/d24/d87/d10a/l10e 0 2024-04-22T23:35:13.524 INFO:tasks.workunit.client.0.smithi107.stdout:9/847: mknod d5/d7/d27/d46/d4b/c122 0 2024-04-22T23:35:13.525 INFO:tasks.workunit.client.0.smithi107.stdout:1/776: dread d7/fa0 [0,4194304] 0 2024-04-22T23:35:13.531 INFO:tasks.workunit.client.0.smithi107.stdout:8/842: dread d0/d7/d20/da2/fa9 [0,4194304] 0 2024-04-22T23:35:13.531 INFO:tasks.workunit.client.0.smithi107.stdout:8/843: stat d0/fb 0 2024-04-22T23:35:13.531 INFO:tasks.workunit.client.0.smithi107.stdout:8/844: chown d0/d7/d8/d41/d53/f67 94898 1 2024-04-22T23:35:13.533 INFO:tasks.workunit.client.0.smithi107.stdout:9/848: read f4 [2246129,38029] 0 2024-04-22T23:35:13.533 INFO:tasks.workunit.client.0.smithi107.stdout:8/845: getdents d0/d3a/d46/d59/d78 0 2024-04-22T23:35:13.536 INFO:tasks.workunit.client.0.smithi107.stdout:7/807: dread d7/d21/f6d [0,4194304] 0 2024-04-22T23:35:13.538 INFO:tasks.workunit.client.0.smithi107.stdout:5/873: dwrite d9/db/de/d72/d7c/ffc [0,4194304] 0 2024-04-22T23:35:13.538 INFO:tasks.workunit.client.0.smithi107.stdout:5/874: stat d9/d26/d79/f114 0 2024-04-22T23:35:13.538 INFO:tasks.workunit.client.0.smithi107.stdout:4/890: read d1/d7/d5b/dbf/d27/f115 [96648,66161] 0 2024-04-22T23:35:13.538 INFO:tasks.workunit.client.0.smithi107.stdout:4/891: write d1/d21/d29/d2e/d88/d99/dd2/df0/ff5 [162651,58691] 0 2024-04-22T23:35:13.538 INFO:tasks.workunit.client.0.smithi107.stdout:4/892: write d1/d7/d5b/dbf/d27/f115 [4778391,41950] 0 2024-04-22T23:35:13.539 INFO:tasks.workunit.client.0.smithi107.stdout:8/846: link d0/d7/d87/dc8/fee d0/d7/d20/d89/f124 0 2024-04-22T23:35:13.539 INFO:tasks.workunit.client.0.smithi107.stdout:8/847: stat d0/d7/d8/d41/d53/d76/f114 0 2024-04-22T23:35:13.542 INFO:tasks.workunit.client.0.smithi107.stdout:7/808: rename d7/d21/f87 to d7/d21/da4/f10f 0 2024-04-22T23:35:13.542 INFO:tasks.workunit.client.0.smithi107.stdout:9/849: dread d5/d14/d2e/d4f/fa4 [0,4194304] 0 2024-04-22T23:35:13.544 INFO:tasks.workunit.client.0.smithi107.stdout:4/893: creat d1/d21/f13a x:0 0 0 2024-04-22T23:35:13.545 INFO:tasks.workunit.client.0.smithi107.stdout:8/848: rmdir d0/d73/db0/d107/d33 39 2024-04-22T23:35:13.545 INFO:tasks.workunit.client.0.smithi107.stdout:8/849: fdatasync d0/d73/db0/d107/d3b/f10 0 2024-04-22T23:35:13.546 INFO:tasks.workunit.client.0.smithi107.stdout:6/909: dwrite db/fd [0,4194304] 0 2024-04-22T23:35:13.550 INFO:tasks.workunit.client.0.smithi107.stdout:9/850: link d5/d7/d27/d46/fc6 d5/d29/d7b/d91/dde/f123 0 2024-04-22T23:35:13.550 INFO:tasks.workunit.client.0.smithi107.stdout:9/851: read d5/d7/fd [277587,123676] 0 2024-04-22T23:35:13.553 INFO:tasks.workunit.client.0.smithi107.stdout:4/894: creat d1/d7/d63/f13b x:0 0 0 2024-04-22T23:35:13.554 INFO:tasks.workunit.client.0.smithi107.stdout:8/850: creat d0/f125 x:0 0 0 2024-04-22T23:35:13.554 INFO:tasks.workunit.client.0.smithi107.stdout:8/851: truncate d0/f81 40663 0 2024-04-22T23:35:13.554 INFO:tasks.workunit.client.0.smithi107.stdout:8/852: truncate d0/d73/db0/d107/f110 687637 0 2024-04-22T23:35:13.554 INFO:tasks.workunit.client.0.smithi107.stdout:8/853: chown d0/d7/d20/lf2 412 1 2024-04-22T23:35:13.555 INFO:tasks.workunit.client.0.smithi107.stdout:6/910: creat db/d20/f112 x:0 0 0 2024-04-22T23:35:13.555 INFO:tasks.workunit.client.0.smithi107.stdout:6/911: chown db/d20/d2a/d38/d59/f93 804522 1 2024-04-22T23:35:13.557 INFO:tasks.workunit.client.0.smithi107.stdout:8/854: mknod d0/d7/d8/c126 0 2024-04-22T23:35:13.559 INFO:tasks.workunit.client.0.smithi107.stdout:5/875: dread d9/d26/d6a/f6f [0,4194304] 0 2024-04-22T23:35:13.559 INFO:tasks.workunit.client.0.smithi107.stdout:5/876: chown d9/d76/fc8 23858054 1 2024-04-22T23:35:13.559 INFO:tasks.workunit.client.0.smithi107.stdout:2/912: dwrite d1/d67/f6f [0,4194304] 0 2024-04-22T23:35:13.561 INFO:tasks.workunit.client.1.smithi157.stdout:8/803: creat d2/d6/d21/d8d/f117 x:0 0 0 2024-04-22T23:35:13.563 INFO:tasks.workunit.client.0.smithi107.stdout:2/913: symlink d1/d12/d35/d102/d7e/da8/d10e/l132 0 2024-04-22T23:35:13.563 INFO:tasks.workunit.client.0.smithi107.stdout:2/914: write d1/d2b/f111 [730790,10776] 0 2024-04-22T23:35:13.565 INFO:tasks.workunit.client.0.smithi107.stdout:2/915: mknod d1/d12/d35/d102/d7e/da8/d125/d12d/c133 0 2024-04-22T23:35:13.565 INFO:tasks.workunit.client.0.smithi107.stdout:2/916: write d1/d12/f95 [161930,51875] 0 2024-04-22T23:35:13.565 INFO:tasks.workunit.client.0.smithi107.stdout:2/917: readlink d1/d12/d35/d102/dba/df1/l11a 0 2024-04-22T23:35:13.565 INFO:tasks.workunit.client.0.smithi107.stdout:2/918: readlink d1/d12/l4a 0 2024-04-22T23:35:13.565 INFO:tasks.workunit.client.0.smithi107.stdout:2/919: write d1/d2b/d59/f115 [638578,25181] 0 2024-04-22T23:35:13.567 INFO:tasks.workunit.client.0.smithi107.stdout:2/920: symlink d1/d2b/d3d/d57/l134 0 2024-04-22T23:35:13.567 INFO:tasks.workunit.client.1.smithi157.stdout:0/967: mknod d17/d2b/d52/c14b 0 2024-04-22T23:35:13.569 INFO:tasks.workunit.client.1.smithi157.stdout:9/990: rmdir d4/df/d80 39 2024-04-22T23:35:13.571 INFO:tasks.workunit.client.1.smithi157.stdout:9/991: dread d4/df/d11d/d23/d27/db5/f105 [0,4194304] 0 2024-04-22T23:35:13.572 INFO:tasks.workunit.client.0.smithi107.stdout:5/877: creat d9/d76/f130 x:0 0 0 2024-04-22T23:35:13.573 INFO:tasks.workunit.client.1.smithi157.stdout:9/992: mkdir d4/df/d57/dbd/dc6/d13a 0 2024-04-22T23:35:13.575 INFO:tasks.workunit.client.1.smithi157.stdout:0/968: dread d17/d38/f6d [0,4194304] 0 2024-04-22T23:35:13.575 INFO:tasks.workunit.client.1.smithi157.stdout:0/969: write d17/d23/fc9 [38508,80689] 0 2024-04-22T23:35:13.575 INFO:tasks.workunit.client.0.smithi107.stdout:5/878: mknod d9/d76/da0/c131 0 2024-04-22T23:35:13.576 INFO:tasks.workunit.client.0.smithi107.stdout:5/879: truncate d9/de6/f110 2658329 0 2024-04-22T23:35:13.576 INFO:tasks.workunit.client.0.smithi107.stdout:5/880: truncate d9/db/de/d5e/fe0 1288328 0 2024-04-22T23:35:13.578 INFO:tasks.workunit.client.0.smithi107.stdout:5/881: rename d9/db/de/fe5 to d9/d26/d6a/d100/f132 0 2024-04-22T23:35:13.578 INFO:tasks.workunit.client.0.smithi107.stdout:5/882: symlink d9/db/dc1/dcb/de9/d11b/l133 0 2024-04-22T23:35:13.579 INFO:tasks.workunit.client.0.smithi107.stdout:5/883: creat d9/d76/da0/f134 x:0 0 0 2024-04-22T23:35:13.581 INFO:tasks.workunit.client.0.smithi107.stdout:5/884: creat d9/db/de/d72/d7c/da4/f135 x:0 0 0 2024-04-22T23:35:13.581 INFO:tasks.workunit.client.0.smithi107.stdout:5/885: readlink d9/l5d 0 2024-04-22T23:35:13.581 INFO:tasks.workunit.client.0.smithi107.stdout:5/886: chown d9/d26/d79 187720719 1 2024-04-22T23:35:13.581 INFO:tasks.workunit.client.0.smithi107.stdout:5/887: dread - d9/fb2 zero size 2024-04-22T23:35:13.581 INFO:tasks.workunit.client.0.smithi107.stdout:5/888: write d9/fb2 [457321,15527] 0 2024-04-22T23:35:13.583 INFO:tasks.workunit.client.1.smithi157.stdout:9/993: dread d4/df/d11d/d23/d27/d47/db3/fb8 [0,4194304] 0 2024-04-22T23:35:13.583 INFO:tasks.workunit.client.1.smithi157.stdout:9/994: fsync d4/df/d11d/d23/d27/d47/f4a 0 2024-04-22T23:35:13.584 INFO:tasks.workunit.client.1.smithi157.stdout:0/970: dread d17/d2b/d4f/d5a/f63 [0,4194304] 0 2024-04-22T23:35:13.584 INFO:tasks.workunit.client.1.smithi157.stdout:9/995: mkdir d4/df/d61/d9c/d13b 0 2024-04-22T23:35:13.586 INFO:tasks.workunit.client.0.smithi107.stdout:2/921: write d1/d12/d35/d48/d13/d4f/feb [398052,59292] 0 2024-04-22T23:35:13.587 INFO:tasks.workunit.client.0.smithi107.stdout:2/922: fdatasync d1/d67/f11f 0 2024-04-22T23:35:13.592 INFO:tasks.workunit.client.1.smithi157.stdout:0/971: dread d17/d23/d34/d67/d7f/f74 [0,4194304] 0 2024-04-22T23:35:13.592 INFO:tasks.workunit.client.1.smithi157.stdout:0/972: write d17/d23/d34/d67/d104/fc8 [4457390,62428] 0 2024-04-22T23:35:13.594 INFO:tasks.workunit.client.1.smithi157.stdout:0/973: symlink d17/d23/d34/d67/d7f/l14c 0 2024-04-22T23:35:13.594 INFO:tasks.workunit.client.0.smithi107.stdout:5/889: dread d9/db/de/d72/d7c/ffc [0,4194304] 0 2024-04-22T23:35:13.595 INFO:tasks.workunit.client.0.smithi107.stdout:7/809: dwrite d7/d21/d48/d5c/d9d/f62 [0,4194304] 0 2024-04-22T23:35:13.596 INFO:tasks.workunit.client.1.smithi157.stdout:0/974: mkdir d17/d23/d34/d67/d104/d121/d14d 0 2024-04-22T23:35:13.596 INFO:tasks.workunit.client.1.smithi157.stdout:0/975: write d17/d2b/d71/fc2 [361608,79230] 0 2024-04-22T23:35:13.597 INFO:tasks.workunit.client.0.smithi107.stdout:7/810: rename d7/d21/d48/f6b to d7/d21/d54/dcc/d109/f110 0 2024-04-22T23:35:13.597 INFO:tasks.workunit.client.0.smithi107.stdout:5/890: creat d9/db/de/d5e/d122/f136 x:0 0 0 2024-04-22T23:35:13.598 INFO:tasks.workunit.client.1.smithi157.stdout:0/976: mkdir d17/d2b/d52/d14e 0 2024-04-22T23:35:13.598 INFO:tasks.workunit.client.0.smithi107.stdout:7/811: unlink d7/d21/d54/fea 0 2024-04-22T23:35:13.599 INFO:tasks.workunit.client.0.smithi107.stdout:7/812: read - d7/d4c/da1/fad zero size 2024-04-22T23:35:13.599 INFO:tasks.workunit.client.0.smithi107.stdout:5/891: symlink d9/db/d94/l137 0 2024-04-22T23:35:13.600 INFO:tasks.workunit.client.0.smithi107.stdout:2/923: write d1/d12/d35/d102/d7e/da8/fb8 [1479749,50261] 0 2024-04-22T23:35:13.602 INFO:tasks.workunit.client.0.smithi107.stdout:7/813: rename d7/df7 to d7/d21/da4/dbf/d111 0 2024-04-22T23:35:13.603 INFO:tasks.workunit.client.0.smithi107.stdout:3/805: dwrite db/d21/d27/d36/fe4 [0,4194304] 0 2024-04-22T23:35:13.604 INFO:tasks.workunit.client.0.smithi107.stdout:8/855: dwrite d0/f96 [0,4194304] 0 2024-04-22T23:35:13.604 INFO:tasks.workunit.client.0.smithi107.stdout:2/924: unlink d1/f11 0 2024-04-22T23:35:13.604 INFO:tasks.workunit.client.0.smithi107.stdout:2/925: write d1/f5 [3077672,94762] 0 2024-04-22T23:35:13.605 INFO:tasks.workunit.client.0.smithi107.stdout:7/814: getdents d7/d21/d7e/ddf 0 2024-04-22T23:35:13.605 INFO:tasks.workunit.client.0.smithi107.stdout:7/815: read d7/f11 [3522293,34703] 0 2024-04-22T23:35:13.606 INFO:tasks.workunit.client.0.smithi107.stdout:1/777: dwrite d7/d19/d20/d39/d53/d43/d46/d5c/dac/fbf [0,4194304] 0 2024-04-22T23:35:13.606 INFO:tasks.workunit.client.0.smithi107.stdout:9/852: dwrite d5/d29/d7b/d91/dde/dc9/fae [0,4194304] 0 2024-04-22T23:35:13.606 INFO:tasks.workunit.client.1.smithi157.stdout:8/804: dwrite d2/d4/d73/fc7 [0,4194304] 0 2024-04-22T23:35:13.606 INFO:tasks.workunit.client.1.smithi157.stdout:8/805: chown d2/d6/fe0 81 1 2024-04-22T23:35:13.606 INFO:tasks.workunit.client.1.smithi157.stdout:8/806: write d2/db1/d58/f110 [513495,85271] 0 2024-04-22T23:35:13.607 INFO:tasks.workunit.client.1.smithi157.stdout:9/996: dwrite d4/df/d11d/d23/d27/dc0/f12e [0,4194304] 0 2024-04-22T23:35:13.607 INFO:tasks.workunit.client.1.smithi157.stdout:8/807: dread d2/d4/f5b [0,4194304] 0 2024-04-22T23:35:13.612 INFO:tasks.workunit.client.0.smithi107.stdout:8/856: truncate d0/d7/d8/d41/dec/f40 3256682 0 2024-04-22T23:35:13.615 INFO:tasks.workunit.client.1.smithi157.stdout:8/808: mknod d2/d34/d6a/d77/df2/c118 0 2024-04-22T23:35:13.615 INFO:tasks.workunit.client.1.smithi157.stdout:8/809: chown d2/d6/d21/d5f/d11/d13/f6c 1 1 2024-04-22T23:35:13.615 INFO:tasks.workunit.client.1.smithi157.stdout:8/810: fdatasync d2/d6/fa2 0 2024-04-22T23:35:13.616 INFO:tasks.workunit.client.0.smithi107.stdout:2/926: read d1/d12/d35/d48/df3/f119 [949675,69365] 0 2024-04-22T23:35:13.618 INFO:tasks.workunit.client.0.smithi107.stdout:7/816: mkdir d7/db4/d112 0 2024-04-22T23:35:13.618 INFO:tasks.workunit.client.0.smithi107.stdout:7/817: fsync d7/d21/da4/d81/fc5 0 2024-04-22T23:35:13.631 INFO:tasks.workunit.client.0.smithi107.stdout:3/806: mknod db/d21/d27/d36/c10f 0 2024-04-22T23:35:13.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:35:13 smithi157 ceph-mon[33877]: pgmap v263: 65 pgs: 65 active+clean; 3.4 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 330 MiB/s rd, 397 MiB/s wr, 552 op/s 2024-04-22T23:35:13.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:35:13 smithi157 ceph-mon[33877]: fsmap cephfs:2 {0=cephfs.smithi107.ktuxjl=up:active,1=cephfs.smithi157.rcaris=up:active} 2 up:standby 2024-04-22T23:35:13.634 INFO:tasks.workunit.client.0.smithi107.stdout:8/857: symlink d0/d3a/d46/l127 0 2024-04-22T23:35:13.634 INFO:tasks.workunit.client.0.smithi107.stdout:2/927: rmdir d1/d12/d35/d102/d7e/da8/d10e 39 2024-04-22T23:35:13.634 INFO:tasks.workunit.client.0.smithi107.stdout:2/928: fdatasync d1/d12/d35/d54/ffe 0 2024-04-22T23:35:13.636 INFO:tasks.workunit.client.0.smithi107.stdout:9/853: getdents d5/d14 0 2024-04-22T23:35:13.637 INFO:tasks.workunit.client.0.smithi107.stdout:3/807: rmdir db/d21/d27/d15/d29/d2a/d45/ddd 39 2024-04-22T23:35:13.637 INFO:tasks.workunit.client.0.smithi107.stdout:3/808: chown db/d21 0 1 2024-04-22T23:35:13.637 INFO:tasks.workunit.client.0.smithi107.stdout:3/809: stat db/d21/d27/d15/d24/d52/d60/f6b 0 2024-04-22T23:35:13.637 INFO:tasks.workunit.client.0.smithi107.stdout:3/810: write db/d21/d27/d36/d9c/fa3 [703445,99455] 0 2024-04-22T23:35:13.637 INFO:tasks.workunit.client.0.smithi107.stdout:3/811: readlink db/d21/d27/l2c 0 2024-04-22T23:35:13.638 INFO:tasks.workunit.client.0.smithi107.stdout:1/778: dwrite d7/d19/d20/fa7 [4194304,4194304] 0 2024-04-22T23:35:13.638 INFO:tasks.workunit.client.0.smithi107.stdout:8/858: symlink d0/d3a/d46/d59/l128 0 2024-04-22T23:35:13.639 INFO:tasks.workunit.client.0.smithi107.stdout:1/779: dread d7/d19/d54/fb2 [0,4194304] 0 2024-04-22T23:35:13.641 INFO:tasks.workunit.client.0.smithi107.stdout:3/812: rename db/d21/d27/d15/d24/d52/c88 to db/d21/d27/d15/db8/c110 0 2024-04-22T23:35:13.643 INFO:tasks.workunit.client.0.smithi107.stdout:9/854: getdents d5/d7/d17/d1b/dcb 0 2024-04-22T23:35:13.643 INFO:tasks.workunit.client.0.smithi107.stdout:9/855: readlink d5/d29/d7b/d91/dde/l72 0 2024-04-22T23:35:13.644 INFO:tasks.workunit.client.0.smithi107.stdout:9/856: fdatasync f4 0 2024-04-22T23:35:13.644 INFO:tasks.workunit.client.0.smithi107.stdout:9/857: creat d5/d14/d28/d11f/f124 x:0 0 0 2024-04-22T23:35:13.645 INFO:tasks.workunit.client.0.smithi107.stdout:9/858: mknod d5/d7/d17/c125 0 2024-04-22T23:35:13.647 INFO:tasks.workunit.client.0.smithi107.stdout:9/859: creat d5/d7/d17/d1b/d1f/d36/f126 x:0 0 0 2024-04-22T23:35:13.648 INFO:tasks.workunit.client.0.smithi107.stdout:3/813: dread db/d21/db1/dcd/ff8 [0,4194304] 0 2024-04-22T23:35:13.648 INFO:tasks.workunit.client.0.smithi107.stdout:8/859: read d0/d73/f94 [2590914,28973] 0 2024-04-22T23:35:13.649 INFO:tasks.workunit.client.0.smithi107.stdout:9/860: rename d5/d14/d2e/ce8 to d5/d7/d27/d69/c127 0 2024-04-22T23:35:13.649 INFO:tasks.workunit.client.0.smithi107.stdout:9/861: fsync d5/d7/d17/d1b/d1f/f112 0 2024-04-22T23:35:13.649 INFO:tasks.workunit.client.0.smithi107.stdout:8/860: rmdir d0/d73/db0/d107/d3b 39 2024-04-22T23:35:13.651 INFO:tasks.workunit.client.0.smithi107.stdout:3/814: rmdir db/d21/db1/dbb 39 2024-04-22T23:35:13.651 INFO:tasks.workunit.client.0.smithi107.stdout:3/815: chown db/d21/d27/d36/fb0 1147 1 2024-04-22T23:35:13.651 INFO:tasks.workunit.client.0.smithi107.stdout:3/816: write db/d21/d27/d15/d29/d2a/d45/ddd/d25/fe2 [2648529,67905] 0 2024-04-22T23:35:13.651 INFO:tasks.workunit.client.0.smithi107.stdout:3/817: stat db/d21/d27/d15/d29/d2a/caa 0 2024-04-22T23:35:13.651 INFO:tasks.workunit.client.0.smithi107.stdout:3/818: write db/d21/d27/d15/d24/d52/d60/f6b [3700279,113461] 0 2024-04-22T23:35:13.651 INFO:tasks.workunit.client.0.smithi107.stdout:3/819: read db/d21/d27/f23 [1995467,1824] 0 2024-04-22T23:35:13.652 INFO:tasks.workunit.client.0.smithi107.stdout:8/861: rmdir d0/d85 39 2024-04-22T23:35:13.653 INFO:tasks.workunit.client.0.smithi107.stdout:8/862: truncate d0/d7/d20/d89/fc5 2341272 0 2024-04-22T23:35:13.654 INFO:tasks.workunit.client.0.smithi107.stdout:8/863: stat d0/d7/d20/de4/fd7 0 2024-04-22T23:35:13.656 INFO:tasks.workunit.client.0.smithi107.stdout:1/780: dread d7/d19/d20/d39/d53/d43/f97 [0,4194304] 0 2024-04-22T23:35:13.656 INFO:tasks.workunit.client.0.smithi107.stdout:8/864: link d0/d73/db0/d107/d3b/c86 d0/d7/d8/d41/d53/d76/def/c129 0 2024-04-22T23:35:13.656 INFO:tasks.workunit.client.0.smithi107.stdout:1/781: write d7/d19/d20/f58 [390742,55800] 0 2024-04-22T23:35:13.656 INFO:tasks.workunit.client.0.smithi107.stdout:1/782: chown d7/d19/d20/d39/d53/d43/f97 106373 1 2024-04-22T23:35:13.658 INFO:tasks.workunit.client.0.smithi107.stdout:1/783: rename d7/d19/lfc to d7/d19/d20/d39/d53/d43/d46/d5c/db7/l103 0 2024-04-22T23:35:13.658 INFO:tasks.workunit.client.0.smithi107.stdout:1/784: creat d7/d19/d54/f104 x:0 0 0 2024-04-22T23:35:13.662 INFO:tasks.workunit.client.0.smithi107.stdout:1/785: creat d7/d19/d20/d39/d53/d43/d46/f105 x:0 0 0 2024-04-22T23:35:13.662 INFO:tasks.workunit.client.0.smithi107.stdout:1/786: fdatasync d7/d19/d54/fb2 0 2024-04-22T23:35:13.664 INFO:tasks.workunit.client.1.smithi157.stdout:9/997: dwrite d4/df/d11d/d23/d3f/f42 [0,4194304] 0 2024-04-22T23:35:13.664 INFO:tasks.workunit.client.1.smithi157.stdout:9/998: chown d4/df/d57/dbd/dc6 58 1 2024-04-22T23:35:13.666 INFO:tasks.workunit.client.1.smithi157.stdout:9/999: mkdir d4/df/d57/d6b/d13c 0 2024-04-22T23:35:13.671 INFO:tasks.workunit.client.0.smithi107.stdout:1/787: dread d7/d19/d2d/f31 [0,4194304] 0 2024-04-22T23:35:13.672 INFO:tasks.workunit.client.0.smithi107.stdout:1/788: symlink d7/d19/d20/d39/d53/l106 0 2024-04-22T23:35:13.672 INFO:tasks.workunit.client.0.smithi107.stdout:1/789: chown d7/d19/d54/ld7 26701 1 2024-04-22T23:35:13.672 INFO:tasks.workunit.client.0.smithi107.stdout:1/790: readlink d7/d19/d20/l7b 0 2024-04-22T23:35:13.673 INFO:tasks.workunit.client.0.smithi107.stdout:1/791: link d7/d19/d20/d39/d53/d43/d46/d5c/c6a d7/d19/d20/d39/d53/d43/d46/d68/dde/c107 0 2024-04-22T23:35:13.673 INFO:tasks.workunit.client.0.smithi107.stdout:1/792: chown d7/l82 3252991 1 2024-04-22T23:35:13.675 INFO:tasks.workunit.client.0.smithi107.stdout:1/793: link d7/d19/d20/fd6 d7/dce/f108 0 2024-04-22T23:35:13.675 INFO:tasks.workunit.client.0.smithi107.stdout:1/794: write d7/d19/d20/d39/d53/d43/d46/dba/fe5 [869355,47387] 0 2024-04-22T23:35:13.675 INFO:tasks.workunit.client.0.smithi107.stdout:6/912: dwrite db/f1f [0,4194304] 0 2024-04-22T23:35:13.675 INFO:tasks.workunit.client.0.smithi107.stdout:6/913: dread - db/d20/d2c/d3b/d40/d9e/fd0 zero size 2024-04-22T23:35:13.675 INFO:tasks.workunit.client.0.smithi107.stdout:6/914: fsync db/d20/d2a/d35/d69/db3/d100/f10e 0 2024-04-22T23:35:13.675 INFO:tasks.workunit.client.0.smithi107.stdout:1/795: stat d7/d19/d20/d39/d53/d43/f49 0 2024-04-22T23:35:13.676 INFO:tasks.workunit.client.0.smithi107.stdout:6/915: symlink db/d20/d2a/d38/d4b/d110/l113 0 2024-04-22T23:35:13.677 INFO:tasks.workunit.client.0.smithi107.stdout:1/796: mkdir d7/d19/d20/d39/d53/d43/d46/d5c/d109 0 2024-04-22T23:35:13.678 INFO:tasks.workunit.client.0.smithi107.stdout:8/865: dwrite d0/f83 [0,4194304] 0 2024-04-22T23:35:13.678 INFO:tasks.workunit.client.0.smithi107.stdout:6/916: link db/d20/l27 db/d20/d2c/de8/l114 0 2024-04-22T23:35:13.679 INFO:tasks.workunit.client.0.smithi107.stdout:1/797: symlink d7/d19/d20/d39/d93/df4/l10a 0 2024-04-22T23:35:13.680 INFO:tasks.workunit.client.1.smithi157.stdout:8/811: dwrite d2/d4/dbd/fb4 [8388608,4194304] 0 2024-04-22T23:35:13.681 INFO:tasks.workunit.client.0.smithi107.stdout:2/929: dwrite d1/d12/d35/d54/fd5 [0,4194304] 0 2024-04-22T23:35:13.681 INFO:tasks.workunit.client.0.smithi107.stdout:2/930: fsync d1/fad 0 2024-04-22T23:35:13.681 INFO:tasks.workunit.client.0.smithi107.stdout:7/818: dwrite d7/d21/da4/d81/fc5 [0,4194304] 0 2024-04-22T23:35:13.681 INFO:tasks.workunit.client.0.smithi107.stdout:2/931: dread - d1/d12/d35/d102/d87/f11e zero size 2024-04-22T23:35:13.684 INFO:tasks.workunit.client.1.smithi157.stdout:8/812: rename d2/d6/d21/d5f/d11 to d2/d4/dbd/dd8/d119 0 2024-04-22T23:35:13.685 INFO:tasks.workunit.client.1.smithi157.stdout:8/813: creat d2/d4/dad/f11a x:0 0 0 2024-04-22T23:35:13.686 INFO:tasks.workunit.client.0.smithi107.stdout:7/819: creat d7/d21/d48/f113 x:0 0 0 2024-04-22T23:35:13.686 INFO:tasks.workunit.client.0.smithi107.stdout:7/820: fsync d7/d21/da4/f33 0 2024-04-22T23:35:13.688 INFO:tasks.workunit.client.0.smithi107.stdout:2/932: creat d1/d12/d35/d48/f135 x:0 0 0 2024-04-22T23:35:13.689 INFO:tasks.workunit.client.0.smithi107.stdout:2/933: readlink d1/d67/l42 0 2024-04-22T23:35:13.689 INFO:tasks.workunit.client.0.smithi107.stdout:2/934: creat d1/d12/d35/d102/d7e/dae/f136 x:0 0 0 2024-04-22T23:35:13.692 INFO:tasks.workunit.client.0.smithi107.stdout:1/798: dread d7/d19/d20/d39/d53/d43/d46/d5c/dac/fea [0,4194304] 0 2024-04-22T23:35:13.692 INFO:tasks.workunit.client.0.smithi107.stdout:3/820: dwrite db/d21/d27/d15/fb4 [0,4194304] 0 2024-04-22T23:35:13.692 INFO:tasks.workunit.client.0.smithi107.stdout:9/862: dwrite d5/d7/d17/d1b/dcb/ffe [0,4194304] 0 2024-04-22T23:35:13.692 INFO:tasks.workunit.client.0.smithi107.stdout:8/866: rename d0/d7/d8/d41/dec/f48 to d0/d73/db0/d107/d33/d9b/f12a 0 2024-04-22T23:35:13.694 INFO:tasks.workunit.client.0.smithi107.stdout:7/821: link d7/f76 d7/d21/d48/d5c/d9d/d6e/f114 0 2024-04-22T23:35:13.697 INFO:tasks.workunit.client.0.smithi107.stdout:2/935: creat d1/d2b/d88/f137 x:0 0 0 2024-04-22T23:35:13.697 INFO:tasks.workunit.client.0.smithi107.stdout:2/936: write d1/d2b/d3d/d57/f97 [578356,59116] 0 2024-04-22T23:35:13.697 INFO:tasks.workunit.client.0.smithi107.stdout:6/917: dwrite db/f15 [0,4194304] 0 2024-04-22T23:35:13.697 INFO:tasks.workunit.client.0.smithi107.stdout:6/918: write db/d20/d2a/d38/d61/d6f/da2/fe6 [720626,85103] 0 2024-04-22T23:35:13.698 INFO:tasks.workunit.client.0.smithi107.stdout:6/919: truncate db/f8a 873762 0 2024-04-22T23:35:13.703 INFO:tasks.workunit.client.0.smithi107.stdout:1/799: symlink d7/d19/d20/d39/d53/d43/d63/l10b 0 2024-04-22T23:35:13.704 INFO:tasks.workunit.client.0.smithi107.stdout:1/800: readlink d7/d19/d20/d39/d53/d43/d46/d68/l79 0 2024-04-22T23:35:13.705 INFO:tasks.workunit.client.0.smithi107.stdout:3/821: mkdir db/d21/d27/d15/d24/d87/d10a/d111 0 2024-04-22T23:35:13.706 INFO:tasks.workunit.client.0.smithi107.stdout:2/937: write d1/d63/f69 [3287694,90083] 0 2024-04-22T23:35:13.708 INFO:tasks.workunit.client.0.smithi107.stdout:7/822: truncate d7/d21/d48/d5c/d9d/d6e/f114 1633779 0 2024-04-22T23:35:13.708 INFO:tasks.workunit.client.0.smithi107.stdout:7/823: fdatasync d7/d21/d48/d95/ff6 0 2024-04-22T23:35:13.708 INFO:tasks.workunit.client.0.smithi107.stdout:7/824: chown d7/d21/da4/cc8 278 1 2024-04-22T23:35:13.709 INFO:tasks.workunit.client.0.smithi107.stdout:7/825: stat d7/d21/d48/cd6 0 2024-04-22T23:35:13.709 INFO:tasks.workunit.client.0.smithi107.stdout:7/826: write d7/d21/d54/f83 [1047313,70051] 0 2024-04-22T23:35:13.709 INFO:tasks.workunit.client.0.smithi107.stdout:7/827: truncate d7/d21/d48/d5c/d9d/d6e/db2/f100 558921 0 2024-04-22T23:35:13.709 INFO:tasks.workunit.client.0.smithi107.stdout:7/828: readlink d7/d4c/da1/dce/l108 0 2024-04-22T23:35:13.709 INFO:tasks.workunit.client.0.smithi107.stdout:7/829: write d7/d21/d54/ff9 [1022919,58176] 0 2024-04-22T23:35:13.712 INFO:tasks.workunit.client.0.smithi107.stdout:1/801: rmdir d7/d19/d20/d39/d53/d43/d63 39 2024-04-22T23:35:13.712 INFO:tasks.workunit.client.0.smithi107.stdout:1/802: write d7/f10 [3993443,96481] 0 2024-04-22T23:35:13.712 INFO:tasks.workunit.client.1.smithi157.stdout:8/814: dwrite d2/d6/f3c [0,4194304] 0 2024-04-22T23:35:13.716 INFO:tasks.workunit.client.0.smithi107.stdout:2/938: mkdir d1/d12/d35/d48/d138 0 2024-04-22T23:35:13.717 INFO:tasks.workunit.client.1.smithi157.stdout:8/815: dread d2/d4/dbd/dd8/d119/d13/fff [0,4194304] 0 2024-04-22T23:35:13.717 INFO:tasks.workunit.client.1.smithi157.stdout:8/816: chown d2/d4/dbd/d97/fea 968 1 2024-04-22T23:35:13.718 INFO:tasks.workunit.client.0.smithi107.stdout:8/867: dwrite d0/d7/d8/f1e [0,4194304] 0 2024-04-22T23:35:13.718 INFO:tasks.workunit.client.0.smithi107.stdout:9/863: dwrite d5/d14/d28/de3/f107 [0,4194304] 0 2024-04-22T23:35:13.718 INFO:tasks.workunit.client.0.smithi107.stdout:9/864: write d5/d14/fca [1828906,32258] 0 2024-04-22T23:35:13.718 INFO:tasks.workunit.client.0.smithi107.stdout:9/865: chown d5/d14/d2e/c90 0 1 2024-04-22T23:35:13.719 INFO:tasks.workunit.client.1.smithi157.stdout:8/817: truncate d2/d6/d21/d5f/f3f 2489143 0 2024-04-22T23:35:13.719 INFO:tasks.workunit.client.1.smithi157.stdout:8/818: dread - d2/db1/f102 zero size 2024-04-22T23:35:13.721 INFO:tasks.workunit.client.0.smithi107.stdout:7/830: unlink d7/f3b 0 2024-04-22T23:35:13.721 INFO:tasks.workunit.client.0.smithi107.stdout:7/831: readlink d7/d21/l9a 0 2024-04-22T23:35:13.722 INFO:tasks.workunit.client.1.smithi157.stdout:8/819: getdents d2/db1 0 2024-04-22T23:35:13.724 INFO:tasks.workunit.client.1.smithi157.stdout:8/820: unlink d2/d6/d21/d5f/fd 0 2024-04-22T23:35:13.727 INFO:tasks.workunit.client.1.smithi157.stdout:8/821: rename d2/d6/d21/d5f/d17/fcf to d2/d34/f11b 0 2024-04-22T23:35:13.728 INFO:tasks.workunit.client.0.smithi107.stdout:1/803: rename d7/d19/d20/d39/d53/d43/f97 to d7/d19/d2d/da4/f10c 0 2024-04-22T23:35:13.729 INFO:tasks.workunit.client.0.smithi107.stdout:7/832: write d7/d21/d54/ff9 [308284,21909] 0 2024-04-22T23:35:13.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:35:13 smithi107 ceph-mon[26355]: pgmap v263: 65 pgs: 65 active+clean; 3.4 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 330 MiB/s rd, 397 MiB/s wr, 552 op/s 2024-04-22T23:35:13.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:35:13 smithi107 ceph-mon[26355]: fsmap cephfs:2 {0=cephfs.smithi107.ktuxjl=up:active,1=cephfs.smithi157.rcaris=up:active} 2 up:standby 2024-04-22T23:35:13.734 INFO:tasks.workunit.client.0.smithi107.stdout:1/804: dread d7/d19/d20/d39/faf [0,4194304] 0 2024-04-22T23:35:13.734 INFO:tasks.workunit.client.0.smithi107.stdout:2/939: dwrite d1/d12/d35/d54/ffe [0,4194304] 0 2024-04-22T23:35:13.742 INFO:tasks.workunit.client.1.smithi157.stdout:8/822: write d2/d4/dbd/dd8/d119/d13/fe8 [1190457,14031] 0 2024-04-22T23:35:13.742 INFO:tasks.workunit.client.1.smithi157.stdout:8/823: fsync d2/d6/d21/d5f/f1b 0 2024-04-22T23:35:13.742 INFO:tasks.workunit.client.1.smithi157.stdout:8/824: creat d2/d4/dda/f11c x:0 0 0 2024-04-22T23:35:13.742 INFO:tasks.workunit.client.1.smithi157.stdout:8/825: symlink d2/d6/d21/d5f/d17/l11d 0 2024-04-22T23:35:13.744 INFO:tasks.workunit.client.0.smithi107.stdout:1/805: dread d7/d19/d42/f6d [4194304,4194304] 0 2024-04-22T23:35:13.744 INFO:tasks.workunit.client.0.smithi107.stdout:1/806: truncate d7/d19/d20/d39/d53/d43/d46/d5c/fb9 573039 0 2024-04-22T23:35:13.744 INFO:tasks.workunit.client.0.smithi107.stdout:1/807: write d7/f2a [1538759,126825] 0 2024-04-22T23:35:13.745 INFO:tasks.workunit.client.0.smithi107.stdout:8/868: mkdir d0/d73/db0/d107/d33/d37/d12b 0 2024-04-22T23:35:13.749 INFO:tasks.workunit.client.0.smithi107.stdout:9/866: getdents d5/d7/d27/d46 0 2024-04-22T23:35:13.751 INFO:tasks.workunit.client.0.smithi107.stdout:7/833: dwrite d7/d21/da4/fab [0,4194304] 0 2024-04-22T23:35:13.751 INFO:tasks.workunit.client.0.smithi107.stdout:7/834: chown d7/d21/l40 245 1 2024-04-22T23:35:13.751 INFO:tasks.workunit.client.0.smithi107.stdout:7/835: chown d7/d21/d48/d5c/d9d/d6e 5437592 1 2024-04-22T23:35:13.754 INFO:tasks.workunit.client.1.smithi157.stdout:8/826: dread d2/d6/d21/d5f/d17/f49 [0,4194304] 0 2024-04-22T23:35:13.755 INFO:tasks.workunit.client.0.smithi107.stdout:2/940: symlink d1/d12/d35/d48/l139 0 2024-04-22T23:35:13.755 INFO:tasks.workunit.client.1.smithi157.stdout:8/827: write d2/d4/dbd/dd8/d119/f4b [4441863,104622] 0 2024-04-22T23:35:13.755 INFO:tasks.workunit.client.1.smithi157.stdout:8/828: creat d2/db1/d58/f11e x:0 0 0 2024-04-22T23:35:13.756 INFO:tasks.workunit.client.0.smithi107.stdout:6/920: truncate db/f1f 2931006 0 2024-04-22T23:35:13.758 INFO:tasks.workunit.client.1.smithi157.stdout:8/829: rmdir d2/d34/d6a/d77/df0 0 2024-04-22T23:35:13.758 INFO:tasks.workunit.client.1.smithi157.stdout:8/830: write d2/d4/f108 [759287,100080] 0 2024-04-22T23:35:13.758 INFO:tasks.workunit.client.1.smithi157.stdout:8/831: chown d2/d4/d73/fe9 175256 1 2024-04-22T23:35:13.758 INFO:tasks.workunit.client.1.smithi157.stdout:8/832: write d2/d34/f60 [1253040,116945] 0 2024-04-22T23:35:13.760 INFO:tasks.workunit.client.0.smithi107.stdout:8/869: truncate d0/d3a/f4a 2858054 0 2024-04-22T23:35:13.762 INFO:tasks.workunit.client.0.smithi107.stdout:9/867: symlink d5/d14/d28/de3/l128 0 2024-04-22T23:35:13.765 INFO:tasks.workunit.client.0.smithi107.stdout:2/941: read d1/d12/d35/d54/ffe [195275,97297] 0 2024-04-22T23:35:13.765 INFO:tasks.workunit.client.0.smithi107.stdout:2/942: readlink d1/d12/d35/d102/dba/df1/l11a 0 2024-04-22T23:35:13.765 INFO:tasks.workunit.client.0.smithi107.stdout:7/836: mknod d7/db4/c115 0 2024-04-22T23:35:13.768 INFO:tasks.workunit.client.0.smithi107.stdout:6/921: symlink db/d20/d2a/d35/d69/db3/d100/l115 0 2024-04-22T23:35:13.769 INFO:tasks.workunit.client.0.smithi107.stdout:1/808: symlink d7/d19/d20/d39/d53/d43/l10d 0 2024-04-22T23:35:13.769 INFO:tasks.workunit.client.0.smithi107.stdout:1/809: readlink d7/d19/d42/ded/lf6 0 2024-04-22T23:35:13.771 INFO:tasks.workunit.client.0.smithi107.stdout:8/870: symlink d0/d7/d8/l12c 0 2024-04-22T23:35:13.771 INFO:tasks.workunit.client.0.smithi107.stdout:7/837: dread d7/d21/da4/d81/f8f [0,4194304] 0 2024-04-22T23:35:13.771 INFO:tasks.workunit.client.0.smithi107.stdout:7/838: fdatasync d7/d4c/da1/fd9 0 2024-04-22T23:35:13.772 INFO:tasks.workunit.client.0.smithi107.stdout:9/868: mknod d5/d7/d17/d1b/c129 0 2024-04-22T23:35:13.772 INFO:tasks.workunit.client.0.smithi107.stdout:9/869: write d5/d7/d27/d46/dd3/fd1 [3889424,113831] 0 2024-04-22T23:35:13.772 INFO:tasks.workunit.client.0.smithi107.stdout:9/870: write d5/d7/d27/d46/fc6 [305951,122565] 0 2024-04-22T23:35:13.774 INFO:tasks.workunit.client.0.smithi107.stdout:2/943: link d1/d12/d35/d48/d13/l28 d1/d12/d35/d102/dba/l13a 0 2024-04-22T23:35:13.775 INFO:tasks.workunit.client.1.smithi157.stdout:8/833: dwrite d2/d4/dbd/dd8/d119/d13/fe8 [0,4194304] 0 2024-04-22T23:35:13.775 INFO:tasks.workunit.client.1.smithi157.stdout:8/834: truncate d2/d6/d21/d8d/f117 324517 0 2024-04-22T23:35:13.776 INFO:tasks.workunit.client.0.smithi107.stdout:6/922: symlink db/d20/d2a/d35/d69/d79/l116 0 2024-04-22T23:35:13.776 INFO:tasks.workunit.client.0.smithi107.stdout:6/923: creat db/d20/d2c/d3b/d40/d9e/f117 x:0 0 0 2024-04-22T23:35:13.776 INFO:tasks.workunit.client.0.smithi107.stdout:6/924: fdatasync db/fd 0 2024-04-22T23:35:13.777 INFO:tasks.workunit.client.0.smithi107.stdout:1/810: symlink d7/d19/d20/d39/d53/d43/d46/d5c/l10e 0 2024-04-22T23:35:13.777 INFO:tasks.workunit.client.0.smithi107.stdout:1/811: chown d7/d19/d20/fa7 98559003 1 2024-04-22T23:35:13.780 INFO:tasks.workunit.client.0.smithi107.stdout:7/839: link d7/f63 d7/d21/d48/d5c/f116 0 2024-04-22T23:35:13.781 INFO:tasks.workunit.client.0.smithi107.stdout:7/840: readlink d7/d21/l7b 0 2024-04-22T23:35:13.781 INFO:tasks.workunit.client.0.smithi107.stdout:7/841: chown d7/d1b/f4d 0 1 2024-04-22T23:35:13.781 INFO:tasks.workunit.client.0.smithi107.stdout:7/842: chown d7/d4c/da1/dce 332 1 2024-04-22T23:35:13.782 INFO:tasks.workunit.client.0.smithi107.stdout:9/871: rmdir d5/d14/d2e/d4f 39 2024-04-22T23:35:13.783 INFO:tasks.workunit.client.0.smithi107.stdout:2/944: symlink d1/d12/d35/d102/d7e/da8/d10e/d11c/l13b 0 2024-04-22T23:35:13.785 INFO:tasks.workunit.client.1.smithi157.stdout:8/835: write d2/d6/d21/d5f/d17/f49 [3330054,14449] 0 2024-04-22T23:35:13.785 INFO:tasks.workunit.client.0.smithi107.stdout:1/812: write d7/d19/d54/f70 [4877342,54776] 0 2024-04-22T23:35:13.786 INFO:tasks.workunit.client.1.smithi157.stdout:8/836: truncate d2/d4/d73/f99 1279097 0 2024-04-22T23:35:13.786 INFO:tasks.workunit.client.1.smithi157.stdout:8/837: dread - d2/d4/dbd/d67/f95 zero size 2024-04-22T23:35:13.786 INFO:tasks.workunit.client.1.smithi157.stdout:8/838: creat d2/d4/dbd/dd8/f11f x:0 0 0 2024-04-22T23:35:13.787 INFO:tasks.workunit.client.0.smithi107.stdout:7/843: mkdir d7/d21/d7e/d94/d117 0 2024-04-22T23:35:13.787 INFO:tasks.workunit.client.0.smithi107.stdout:8/871: dwrite d0/d7/d20/f104 [0,4194304] 0 2024-04-22T23:35:13.787 INFO:tasks.workunit.client.0.smithi107.stdout:8/872: fsync d0/d7/d20/f39 0 2024-04-22T23:35:13.789 INFO:tasks.workunit.client.0.smithi107.stdout:9/872: mkdir d5/d12a 0 2024-04-22T23:35:13.789 INFO:tasks.workunit.client.0.smithi107.stdout:9/873: write d5/d7/d27/d46/d4b/f56 [1328784,104856] 0 2024-04-22T23:35:13.789 INFO:tasks.workunit.client.0.smithi107.stdout:9/874: write d5/d7/fb9 [1194080,51493] 0 2024-04-22T23:35:13.790 INFO:tasks.workunit.client.0.smithi107.stdout:1/813: mknod d7/d19/d2d/da4/c10f 0 2024-04-22T23:35:13.790 INFO:tasks.workunit.client.1.smithi157.stdout:8/839: link d2/d6/d21/d5f/f56 d2/d4/dbd/f120 0 2024-04-22T23:35:13.791 INFO:tasks.workunit.client.0.smithi107.stdout:7/844: rename d7/d4c/f6c to d7/d21/d48/d5c/f118 0 2024-04-22T23:35:13.792 INFO:tasks.workunit.client.1.smithi157.stdout:8/840: mknod d2/d34/c121 0 2024-04-22T23:35:13.792 INFO:tasks.workunit.client.1.smithi157.stdout:8/841: chown d2/d34/d6a/c105 6 1 2024-04-22T23:35:13.793 INFO:tasks.workunit.client.1.smithi157.stdout:8/842: dread d2/d4/dbd/dd8/d119/fdc [0,4194304] 0 2024-04-22T23:35:13.794 INFO:tasks.workunit.client.1.smithi157.stdout:8/843: mknod d2/d6/d21/d5f/d17/dc8/c122 0 2024-04-22T23:35:13.795 INFO:tasks.workunit.client.0.smithi107.stdout:8/873: creat d0/d73/db0/d107/d33/f12d x:0 0 0 2024-04-22T23:35:13.796 INFO:tasks.workunit.client.1.smithi157.stdout:8/844: rename d2/d34/d6a/c105 to d2/d6/d21/d5f/d17/d7f/c123 0 2024-04-22T23:35:13.798 INFO:tasks.workunit.client.0.smithi107.stdout:9/875: rename d5/d29/d7b/d91/dde/dc9/c8a to d5/d14/df7/c12b 0 2024-04-22T23:35:13.799 INFO:tasks.workunit.client.0.smithi107.stdout:9/876: fsync d5/d29/d7b/d91/dde/dc9/fae 0 2024-04-22T23:35:13.799 INFO:tasks.workunit.client.0.smithi107.stdout:7/845: rmdir d7/d21/d7e 39 2024-04-22T23:35:13.799 INFO:tasks.workunit.client.0.smithi107.stdout:7/846: chown d7/d1b/le9 10 1 2024-04-22T23:35:13.801 INFO:tasks.workunit.client.0.smithi107.stdout:2/945: dwrite d1/d12/d35/d102/d7e/dae/f109 [0,4194304] 0 2024-04-22T23:35:13.801 INFO:tasks.workunit.client.0.smithi107.stdout:1/814: mkdir d7/d19/d20/d39/d53/d43/d63/d91/d110 0 2024-04-22T23:35:13.801 INFO:tasks.workunit.client.0.smithi107.stdout:1/815: readlink d7/d19/d20/d39/d53/d43/d46/d5c/db7/lcd 0 2024-04-22T23:35:13.804 INFO:tasks.workunit.client.0.smithi107.stdout:7/847: dread d7/d21/d54/f83 [0,4194304] 0 2024-04-22T23:35:13.804 INFO:tasks.workunit.client.0.smithi107.stdout:6/925: dwrite db/d20/d2c/d3b/feb [0,4194304] 0 2024-04-22T23:35:13.805 INFO:tasks.workunit.client.0.smithi107.stdout:6/926: write db/d20/d49/f6e [1114088,84326] 0 2024-04-22T23:35:13.805 INFO:tasks.workunit.client.0.smithi107.stdout:9/877: creat d5/d7/d27/d46/dd3/f12c x:0 0 0 2024-04-22T23:35:13.806 INFO:tasks.workunit.client.0.smithi107.stdout:8/874: dread d0/d7/d8/d41/d53/f67 [0,4194304] 0 2024-04-22T23:35:13.810 INFO:tasks.workunit.client.0.smithi107.stdout:1/816: dread d7/d19/d20/f58 [0,4194304] 0 2024-04-22T23:35:13.813 INFO:tasks.workunit.client.0.smithi107.stdout:1/817: dread d7/d19/d20/fd6 [0,4194304] 0 2024-04-22T23:35:13.813 INFO:tasks.workunit.client.0.smithi107.stdout:1/818: chown d7/d19/d20/d39/d53/d43/fab 3577 1 2024-04-22T23:35:13.823 INFO:tasks.workunit.client.0.smithi107.stdout:7/848: truncate d7/d21/d7e/fb3 390467 0 2024-04-22T23:35:13.826 INFO:tasks.workunit.client.0.smithi107.stdout:8/875: mkdir d0/d7/d12e 0 2024-04-22T23:35:13.831 INFO:tasks.workunit.client.0.smithi107.stdout:2/946: write d1/d12/d35/d102/f50 [5240245,24813] 0 2024-04-22T23:35:13.833 INFO:tasks.workunit.client.0.smithi107.stdout:9/878: dwrite f4 [0,4194304] 0 2024-04-22T23:35:13.834 INFO:tasks.workunit.client.0.smithi107.stdout:2/947: symlink d1/d2b/d59/d123/l13c 0 2024-04-22T23:35:13.834 INFO:tasks.workunit.client.0.smithi107.stdout:2/948: creat d1/d12/d35/d48/d13/d4f/f13d x:0 0 0 2024-04-22T23:35:13.834 INFO:tasks.workunit.client.0.smithi107.stdout:2/949: fsync d1/d12/d35/d102/d7e/dae/f136 0 2024-04-22T23:35:13.836 INFO:tasks.workunit.client.0.smithi107.stdout:1/819: dwrite d7/d19/d20/d39/d93/df4/db8/fd5 [0,4194304] 0 2024-04-22T23:35:13.837 INFO:tasks.workunit.client.0.smithi107.stdout:9/879: link d5/d7/d27/d46/d4b/ce4 d5/d7/d17/d1b/d1f/dda/c12d 0 2024-04-22T23:35:13.837 INFO:tasks.workunit.client.0.smithi107.stdout:9/880: write d5/d7/d27/d46/dd3/f12c [794020,36836] 0 2024-04-22T23:35:13.838 INFO:tasks.workunit.client.0.smithi107.stdout:2/950: mkdir d1/d2b/d3d/d121/d13e 0 2024-04-22T23:35:13.838 INFO:tasks.workunit.client.0.smithi107.stdout:2/951: write d1/d2b/d3d/d57/fa2 [769128,53291] 0 2024-04-22T23:35:13.840 INFO:tasks.workunit.client.0.smithi107.stdout:6/927: dwrite db/fd [0,4194304] 0 2024-04-22T23:35:13.841 INFO:tasks.workunit.client.0.smithi107.stdout:1/820: mkdir d7/d19/d20/d39/d93/d111 0 2024-04-22T23:35:13.841 INFO:tasks.workunit.client.0.smithi107.stdout:1/821: readlink d7/d19/d20/d39/d93/df4/db8/le3 0 2024-04-22T23:35:13.843 INFO:tasks.workunit.client.0.smithi107.stdout:9/881: unlink d5/d7/f16 0 2024-04-22T23:35:13.844 INFO:tasks.workunit.client.0.smithi107.stdout:2/952: symlink d1/d63/l13f 0 2024-04-22T23:35:13.846 INFO:tasks.workunit.client.0.smithi107.stdout:7/849: dwrite d7/d21/da4/f33 [0,4194304] 0 2024-04-22T23:35:13.847 INFO:tasks.workunit.client.0.smithi107.stdout:8/876: dwrite d0/d73/db0/d107/d33/f118 [0,4194304] 0 2024-04-22T23:35:13.848 INFO:tasks.workunit.client.0.smithi107.stdout:7/850: dread d7/d21/d48/fc2 [0,4194304] 0 2024-04-22T23:35:13.849 INFO:tasks.workunit.client.0.smithi107.stdout:7/851: truncate d7/d21/d48/d5c/d9d/db1/f10b 906370 0 2024-04-22T23:35:13.849 INFO:tasks.workunit.client.0.smithi107.stdout:7/852: write d7/d4c/fe7 [961521,83616] 0 2024-04-22T23:35:13.849 INFO:tasks.workunit.client.0.smithi107.stdout:0/838: sync 2024-04-22T23:35:13.851 INFO:tasks.workunit.client.0.smithi107.stdout:7/853: write d7/d21/d48/d5c/f118 [4352465,66561] 0 2024-04-22T23:35:13.851 INFO:tasks.workunit.client.0.smithi107.stdout:7/854: fsync d7/d4c/da1/dce/fe5 0 2024-04-22T23:35:13.853 INFO:tasks.workunit.client.0.smithi107.stdout:6/928: symlink db/d20/d49/d7b/dea/l118 0 2024-04-22T23:35:13.855 INFO:tasks.workunit.client.0.smithi107.stdout:9/882: dread d5/d7/d27/d46/fd5 [0,4194304] 0 2024-04-22T23:35:13.855 INFO:tasks.workunit.client.0.smithi107.stdout:0/839: symlink d1/d27/d8a/db5/d10f/l112 0 2024-04-22T23:35:13.860 INFO:tasks.workunit.client.0.smithi107.stdout:2/953: dread d1/d12/fa4 [0,4194304] 0 2024-04-22T23:35:13.860 INFO:tasks.workunit.client.0.smithi107.stdout:2/954: readlink d1/d12/d35/d48/d13/d4f/lc1 0 2024-04-22T23:35:13.862 INFO:tasks.workunit.client.0.smithi107.stdout:9/883: creat d5/d12a/f12e x:0 0 0 2024-04-22T23:35:13.862 INFO:tasks.workunit.client.0.smithi107.stdout:9/884: getdents d5/d7/d27/d69/db6/d104 0 2024-04-22T23:35:13.864 INFO:tasks.workunit.client.0.smithi107.stdout:6/929: rmdir db/d20/d2a/d35/d69/db3/dbc 39 2024-04-22T23:35:13.868 INFO:tasks.workunit.client.0.smithi107.stdout:9/885: rename d5/d7/d17/d1b/d1f/d36/d53/c82 to d5/d7/d27/d46/d4b/def/c12f 0 2024-04-22T23:35:13.868 INFO:tasks.workunit.client.0.smithi107.stdout:9/886: creat d5/d7/d27/d46/d4b/f130 x:0 0 0 2024-04-22T23:35:13.868 INFO:tasks.workunit.client.0.smithi107.stdout:9/887: readlink d5/d7/d17/l5d 0 2024-04-22T23:35:13.874 INFO:tasks.workunit.client.0.smithi107.stdout:9/888: creat d5/d7/d17/d1b/d1f/d36/d53/f131 x:0 0 0 2024-04-22T23:35:13.874 INFO:tasks.workunit.client.0.smithi107.stdout:9/889: read - d5/d14/d28/fc1 zero size 2024-04-22T23:35:13.875 INFO:tasks.workunit.client.0.smithi107.stdout:3/822: sync 2024-04-22T23:35:13.875 INFO:tasks.workunit.client.0.smithi107.stdout:4/895: sync 2024-04-22T23:35:13.875 INFO:tasks.workunit.client.0.smithi107.stdout:3/823: fsync db/d21/d27/d15/d29/d2a/d40/f80 0 2024-04-22T23:35:13.875 INFO:tasks.workunit.client.0.smithi107.stdout:3/824: readlink db/d21/d27/d15/d24/d52/l5c 0 2024-04-22T23:35:13.875 INFO:tasks.workunit.client.0.smithi107.stdout:5/892: sync 2024-04-22T23:35:13.875 INFO:tasks.workunit.client.0.smithi107.stdout:3/825: write db/d21/d27/f39 [4939549,24130] 0 2024-04-22T23:35:13.876 INFO:tasks.workunit.client.0.smithi107.stdout:9/890: mknod d5/d7/d27/d46/c132 0 2024-04-22T23:35:13.876 INFO:tasks.workunit.client.0.smithi107.stdout:9/891: truncate d5/d7/d17/d1b/dcb/f114 1032644 0 2024-04-22T23:35:13.876 INFO:tasks.workunit.client.0.smithi107.stdout:9/892: truncate d5/d14/f2f 689129 0 2024-04-22T23:35:13.877 INFO:tasks.workunit.client.0.smithi107.stdout:4/896: mknod d1/d21/d29/d2e/d88/d99/dd2/df0/c13c 0 2024-04-22T23:35:13.879 INFO:tasks.workunit.client.0.smithi107.stdout:7/855: dwrite d7/d4c/da1/fcd [0,4194304] 0 2024-04-22T23:35:13.879 INFO:tasks.workunit.client.0.smithi107.stdout:7/856: chown d7/d21/da4/d66/l79 129947274 1 2024-04-22T23:35:13.879 INFO:tasks.workunit.client.0.smithi107.stdout:7/857: write d7/d21/da4/dbf/ff2 [800788,117301] 0 2024-04-22T23:35:13.879 INFO:tasks.workunit.client.0.smithi107.stdout:7/858: chown d7/d4c 56522 1 2024-04-22T23:35:13.879 INFO:tasks.workunit.client.0.smithi107.stdout:7/859: chown d7/l34 117443809 1 2024-04-22T23:35:13.879 INFO:tasks.workunit.client.0.smithi107.stdout:4/897: mknod d1/d1b/d44/c13d 0 2024-04-22T23:35:13.879 INFO:tasks.workunit.client.0.smithi107.stdout:4/898: dread - d1/d7/ff1 zero size 2024-04-22T23:35:13.879 INFO:tasks.workunit.client.0.smithi107.stdout:4/899: fsync d1/d7/f32 0 2024-04-22T23:35:13.880 INFO:tasks.workunit.client.0.smithi107.stdout:6/930: read db/f21 [933053,53013] 0 2024-04-22T23:35:13.880 INFO:tasks.workunit.client.0.smithi107.stdout:6/931: readlink db/d20/d2a/d38/d4b/l67 0 2024-04-22T23:35:13.880 INFO:tasks.workunit.client.0.smithi107.stdout:5/893: dread d9/db/de/faf [0,4194304] 0 2024-04-22T23:35:13.881 INFO:tasks.workunit.client.0.smithi107.stdout:1/822: dwrite d7/d19/d20/d39/d93/df4/f96 [0,4194304] 0 2024-04-22T23:35:13.883 INFO:tasks.workunit.client.0.smithi107.stdout:7/860: creat d7/d21/f119 x:0 0 0 2024-04-22T23:35:13.884 INFO:tasks.workunit.client.0.smithi107.stdout:4/900: getdents d1/d7/d63/d78/d96 0 2024-04-22T23:35:13.887 INFO:tasks.workunit.client.0.smithi107.stdout:6/932: symlink db/d20/d2a/d5f/l119 0 2024-04-22T23:35:13.887 INFO:tasks.workunit.client.0.smithi107.stdout:6/933: chown db/d20/d2a/d35/ce1 1 1 2024-04-22T23:35:13.887 INFO:tasks.workunit.client.0.smithi107.stdout:6/934: write db/d20/d2a/d35/d69/db3/ff0 [209153,44077] 0 2024-04-22T23:35:13.890 INFO:tasks.workunit.client.0.smithi107.stdout:2/955: dwrite d1/d12/d35/d48/d13/d4f/f53 [0,4194304] 0 2024-04-22T23:35:13.890 INFO:tasks.workunit.client.0.smithi107.stdout:2/956: creat d1/d63/f140 x:0 0 0 2024-04-22T23:35:13.891 INFO:tasks.workunit.client.0.smithi107.stdout:2/957: read d1/d12/d35/d102/f51 [38910,69864] 0 2024-04-22T23:35:13.892 INFO:tasks.workunit.client.0.smithi107.stdout:0/840: dwrite d1/d27/d20/d2c/d41/fac [0,4194304] 0 2024-04-22T23:35:13.893 INFO:tasks.workunit.client.0.smithi107.stdout:7/861: creat d7/db4/d112/f11a x:0 0 0 2024-04-22T23:35:13.894 INFO:tasks.workunit.client.0.smithi107.stdout:7/862: readlink d7/d21/d7e/d94/lc6 0 2024-04-22T23:35:13.895 INFO:tasks.workunit.client.0.smithi107.stdout:4/901: rename d1/d7/d63/cad to d1/c13e 0 2024-04-22T23:35:13.895 INFO:tasks.workunit.client.0.smithi107.stdout:4/902: write d1/d21/d29/d2e/dfa/f139 [5182839,67847] 0 2024-04-22T23:35:13.895 INFO:tasks.workunit.client.0.smithi107.stdout:4/903: stat d1/d7/d53/c5a 0 2024-04-22T23:35:13.900 INFO:tasks.workunit.client.0.smithi107.stdout:2/958: truncate d1/d12/f34 223606 0 2024-04-22T23:35:13.905 INFO:tasks.workunit.client.0.smithi107.stdout:7/863: mkdir d7/d21/d48/d5c/d9d/d6e/db2/d11b 0 2024-04-22T23:35:13.905 INFO:tasks.workunit.client.0.smithi107.stdout:1/823: dwrite d7/d19/d20/d39/d53/ff7 [0,4194304] 0 2024-04-22T23:35:13.905 INFO:tasks.workunit.client.0.smithi107.stdout:1/824: chown d7/d19/d20/d39/d53/d43/d63/d91/lf2 18 1 2024-04-22T23:35:13.906 INFO:tasks.workunit.client.0.smithi107.stdout:1/825: stat d7/d19/d20/d39/d53/d43/d63/d91/lf2 0 2024-04-22T23:35:13.906 INFO:tasks.workunit.client.0.smithi107.stdout:2/959: creat d1/d12/d35/d102/d7e/f141 x:0 0 0 2024-04-22T23:35:13.907 INFO:tasks.workunit.client.0.smithi107.stdout:7/864: mknod d7/db4/d112/c11c 0 2024-04-22T23:35:13.907 INFO:tasks.workunit.client.0.smithi107.stdout:7/865: dread - d7/d21/d7e/d94/fba zero size 2024-04-22T23:35:13.907 INFO:tasks.workunit.client.0.smithi107.stdout:7/866: chown d7/db4/d112 611173105 1 2024-04-22T23:35:13.911 INFO:tasks.workunit.client.0.smithi107.stdout:6/935: dread db/fd [0,4194304] 0 2024-04-22T23:35:13.911 INFO:tasks.workunit.client.0.smithi107.stdout:1/826: rename d7/d19/d20/d39/de0/df3 to d7/d19/d112 0 2024-04-22T23:35:13.912 INFO:tasks.workunit.client.0.smithi107.stdout:2/960: creat d1/d12/d35/d102/dba/f142 x:0 0 0 2024-04-22T23:35:13.912 INFO:tasks.workunit.client.0.smithi107.stdout:2/961: write d1/d12/d35/d102/d7e/fbe [4365363,84866] 0 2024-04-22T23:35:13.912 INFO:tasks.workunit.client.0.smithi107.stdout:9/893: dwrite d5/f10 [0,4194304] 0 2024-04-22T23:35:13.913 INFO:tasks.workunit.client.0.smithi107.stdout:7/867: symlink d7/d21/da4/d66/l11d 0 2024-04-22T23:35:13.913 INFO:tasks.workunit.client.0.smithi107.stdout:7/868: fsync d7/d21/d48/d95/ff6 0 2024-04-22T23:35:13.914 INFO:tasks.workunit.client.0.smithi107.stdout:1/827: rmdir d7/def 0 2024-04-22T23:35:13.915 INFO:tasks.workunit.client.0.smithi107.stdout:2/962: creat d1/d2b/d3d/f143 x:0 0 0 2024-04-22T23:35:13.915 INFO:tasks.workunit.client.0.smithi107.stdout:2/963: dread - d1/d67/f11f zero size 2024-04-22T23:35:13.915 INFO:tasks.workunit.client.0.smithi107.stdout:2/964: creat d1/d63/dbb/f144 x:0 0 0 2024-04-22T23:35:13.916 INFO:tasks.workunit.client.0.smithi107.stdout:7/869: mkdir d7/d21/d7e/ddf/d11e 0 2024-04-22T23:35:13.918 INFO:tasks.workunit.client.0.smithi107.stdout:1/828: rename d7/f15 to d7/d19/d20/d39/d93/df4/db8/f113 0 2024-04-22T23:35:13.919 INFO:tasks.workunit.client.0.smithi107.stdout:0/841: dwrite d1/d27/d20/d2c/fbb [0,4194304] 0 2024-04-22T23:35:13.919 INFO:tasks.workunit.client.0.smithi107.stdout:4/904: dwrite d1/d1b/d44/f95 [4194304,4194304] 0 2024-04-22T23:35:13.920 INFO:tasks.workunit.client.0.smithi107.stdout:7/870: creat d7/d21/d7e/ddf/d11e/f11f x:0 0 0 2024-04-22T23:35:13.922 INFO:tasks.workunit.client.0.smithi107.stdout:1/829: dread d7/d19/d2d/da4/f10c [0,4194304] 0 2024-04-22T23:35:13.922 INFO:tasks.workunit.client.0.smithi107.stdout:5/894: dwrite d9/db/de/d5e/f66 [0,4194304] 0 2024-04-22T23:35:13.922 INFO:tasks.workunit.client.0.smithi107.stdout:5/895: stat d9/f36 0 2024-04-22T23:35:13.923 INFO:tasks.workunit.client.0.smithi107.stdout:6/936: write db/d20/d2a/d38/d4b/f54 [5388750,91666] 0 2024-04-22T23:35:13.924 INFO:tasks.workunit.client.0.smithi107.stdout:9/894: read d5/d29/d7b/d91/dde/dc9/daf/fd0 [2674066,21890] 0 2024-04-22T23:35:13.924 INFO:tasks.workunit.client.0.smithi107.stdout:5/896: link d9/d26/d79/f12e d9/db/d94/f138 0 2024-04-22T23:35:13.924 INFO:tasks.workunit.client.0.smithi107.stdout:0/842: symlink d1/d27/d20/d40/dbd/df2/l113 0 2024-04-22T23:35:13.925 INFO:tasks.workunit.client.0.smithi107.stdout:5/897: mknod d9/db/de/d5e/d122/c139 0 2024-04-22T23:35:13.926 INFO:tasks.workunit.client.0.smithi107.stdout:4/905: creat d1/d21/d52/f13f x:0 0 0 2024-04-22T23:35:13.930 INFO:tasks.workunit.client.0.smithi107.stdout:1/830: unlink d7/d19/d20/d39/d53/d43/d46/d68/l8d 0 2024-04-22T23:35:13.934 INFO:tasks.workunit.client.0.smithi107.stdout:7/871: write d7/d21/d48/f61 [3045257,4404] 0 2024-04-22T23:35:13.934 INFO:tasks.workunit.client.0.smithi107.stdout:7/872: write d7/d1b/f68 [1200835,123401] 0 2024-04-22T23:35:13.934 INFO:tasks.workunit.client.0.smithi107.stdout:6/937: link db/d20/d2c/f32 db/d20/d2a/d38/f11a 0 2024-04-22T23:35:13.934 INFO:tasks.workunit.client.0.smithi107.stdout:6/938: fdatasync db/d20/d2a/d38/d59/d70/fb7 0 2024-04-22T23:35:13.934 INFO:tasks.workunit.client.0.smithi107.stdout:6/939: creat db/d20/d2a/d38/d61/d6f/da2/f11b x:0 0 0 2024-04-22T23:35:13.934 INFO:tasks.workunit.client.0.smithi107.stdout:6/940: readlink db/d20/d2a/d38/l7c 0 2024-04-22T23:35:13.934 INFO:tasks.workunit.client.0.smithi107.stdout:6/941: truncate db/d20/d2a/d38/d61/d6f/da2/f11b 548347 0 2024-04-22T23:35:13.935 INFO:tasks.workunit.client.0.smithi107.stdout:4/906: dread d1/d21/d29/d2e/d88/d99/dd2/daf/f112 [0,4194304] 0 2024-04-22T23:35:13.935 INFO:tasks.workunit.client.0.smithi107.stdout:4/907: chown d1/d21/c47 48 1 2024-04-22T23:35:13.937 INFO:tasks.workunit.client.0.smithi107.stdout:0/843: unlink d1/d27/d20/d40/fa0 0 2024-04-22T23:35:13.937 INFO:tasks.workunit.client.0.smithi107.stdout:0/844: write d1/d9/d14/d4f/d59/fe2 [5148923,36766] 0 2024-04-22T23:35:13.939 INFO:tasks.workunit.client.0.smithi107.stdout:1/831: rmdir d7/d19/d20/d39/d53/d43/d46/d5c 39 2024-04-22T23:35:13.940 INFO:tasks.workunit.client.0.smithi107.stdout:5/898: rmdir d9 39 2024-04-22T23:35:13.941 INFO:tasks.workunit.client.0.smithi107.stdout:9/895: dwrite d5/d29/f2c [4194304,4194304] 0 2024-04-22T23:35:13.942 INFO:tasks.workunit.client.0.smithi107.stdout:5/899: mknod d9/d76/da0/c13a 0 2024-04-22T23:35:13.942 INFO:tasks.workunit.client.0.smithi107.stdout:7/873: symlink d7/d21/d7e/ddf/l120 0 2024-04-22T23:35:13.943 INFO:tasks.workunit.client.0.smithi107.stdout:5/900: creat d9/db/de/d5e/f13b x:0 0 0 2024-04-22T23:35:13.944 INFO:tasks.workunit.client.0.smithi107.stdout:5/901: read d9/db/d42/d75/db6/fb9 [4518212,96505] 0 2024-04-22T23:35:13.944 INFO:tasks.workunit.client.0.smithi107.stdout:6/942: link db/d20/d2a/d38/d61/d6f/da2/fd9 db/d20/d2c/d3b/d40/f11c 0 2024-04-22T23:35:13.944 INFO:tasks.workunit.client.0.smithi107.stdout:6/943: getdents db/d20/d2a/d38/d59/d106 0 2024-04-22T23:35:13.945 INFO:tasks.workunit.client.0.smithi107.stdout:5/902: symlink d9/db/d42/l13c 0 2024-04-22T23:35:13.945 INFO:tasks.workunit.client.0.smithi107.stdout:5/903: stat d9/d26/d79 0 2024-04-22T23:35:13.945 INFO:tasks.workunit.client.0.smithi107.stdout:5/904: write d9/d26/d79/fc3 [595788,63218] 0 2024-04-22T23:35:13.946 INFO:tasks.workunit.client.0.smithi107.stdout:0/845: mkdir d1/d9/d14/d4f/d59/d114 0 2024-04-22T23:35:13.946 INFO:tasks.workunit.client.0.smithi107.stdout:0/846: stat d1/d27/d20/d2c/d2e/ccf 0 2024-04-22T23:35:13.946 INFO:tasks.workunit.client.0.smithi107.stdout:0/847: fdatasync d1/d9/f5e 0 2024-04-22T23:35:13.946 INFO:tasks.workunit.client.0.smithi107.stdout:0/848: chown d1/d27/d20/d2c/d2e/ccf 428368 1 2024-04-22T23:35:13.946 INFO:tasks.workunit.client.0.smithi107.stdout:5/905: creat d9/db/de/d72/f13d x:0 0 0 2024-04-22T23:35:13.946 INFO:tasks.workunit.client.0.smithi107.stdout:5/906: readlink d9/db/d42/lae 0 2024-04-22T23:35:13.950 INFO:tasks.workunit.client.0.smithi107.stdout:5/907: mknod d9/db/d42/d75/db6/c13e 0 2024-04-22T23:35:13.950 INFO:tasks.workunit.client.0.smithi107.stdout:5/908: dread d9/db/d42/fcc [0,4194304] 0 2024-04-22T23:35:13.950 INFO:tasks.workunit.client.0.smithi107.stdout:7/874: mkdir d7/d21/d7e/ddf/d121 0 2024-04-22T23:35:13.951 INFO:tasks.workunit.client.0.smithi107.stdout:5/909: link d9/db/d42/l13c d9/db/de/d72/d9e/dee/l13f 0 2024-04-22T23:35:13.951 INFO:tasks.workunit.client.0.smithi107.stdout:5/910: readlink d9/db/d42/l8e 0 2024-04-22T23:35:13.951 INFO:tasks.workunit.client.0.smithi107.stdout:9/896: dread d5/d14/d2e/d4f/fa4 [0,4194304] 0 2024-04-22T23:35:13.951 INFO:tasks.workunit.client.0.smithi107.stdout:9/897: read - d5/d7/d17/d1b/fc5 zero size 2024-04-22T23:35:13.952 INFO:tasks.workunit.client.0.smithi107.stdout:7/875: truncate d7/d1b/f24 3063899 0 2024-04-22T23:35:13.952 INFO:tasks.workunit.client.0.smithi107.stdout:4/908: dwrite d1/d21/d29/f39 [0,4194304] 0 2024-04-22T23:35:13.952 INFO:tasks.workunit.client.0.smithi107.stdout:4/909: stat d1/d7/d63/c69 0 2024-04-22T23:35:13.957 INFO:tasks.workunit.client.0.smithi107.stdout:6/944: dread db/f12 [0,4194304] 0 2024-04-22T23:35:13.959 INFO:tasks.workunit.client.0.smithi107.stdout:7/876: creat d7/d21/d48/d5c/d9d/d6e/db2/de0/f122 x:0 0 0 2024-04-22T23:35:13.961 INFO:tasks.workunit.client.0.smithi107.stdout:4/910: rename d1/d21/d29/d2e/d3b/c6e to d1/d7/d5b/dbf/da8/d135/c140 0 2024-04-22T23:35:13.962 INFO:tasks.workunit.client.0.smithi107.stdout:4/911: chown d1/d21/d29/d2e/d88/d99/dd2 720 1 2024-04-22T23:35:13.962 INFO:tasks.workunit.client.0.smithi107.stdout:1/832: dwrite d7/f17 [0,4194304] 0 2024-04-22T23:35:13.962 INFO:tasks.workunit.client.0.smithi107.stdout:1/833: readlink d7/d19/d20/d39/d53/d43/d63/l10b 0 2024-04-22T23:35:13.962 INFO:tasks.workunit.client.0.smithi107.stdout:1/834: creat d7/d19/d2d/d89/f114 x:0 0 0 2024-04-22T23:35:13.964 INFO:tasks.workunit.client.0.smithi107.stdout:6/945: unlink db/la3 0 2024-04-22T23:35:13.964 INFO:tasks.workunit.client.0.smithi107.stdout:6/946: fdatasync db/d20/d2a/d38/d4b/f6b 0 2024-04-22T23:35:13.964 INFO:tasks.workunit.client.0.smithi107.stdout:6/947: creat db/d20/f11d x:0 0 0 2024-04-22T23:35:13.965 INFO:tasks.workunit.client.0.smithi107.stdout:7/877: mkdir d7/d21/d7e/d94/d123 0 2024-04-22T23:35:13.965 INFO:tasks.workunit.client.0.smithi107.stdout:7/878: readlink d7/d1b/l77 0 2024-04-22T23:35:13.969 INFO:tasks.workunit.client.0.smithi107.stdout:4/912: truncate d1/d21/d52/fec 749688 0 2024-04-22T23:35:13.969 INFO:tasks.workunit.client.0.smithi107.stdout:4/913: truncate d1/d21/d29/d2e/d3b/f8a 2262676 0 2024-04-22T23:35:13.969 INFO:tasks.workunit.client.0.smithi107.stdout:4/914: truncate d1/d7/d5b/dbf/d27/d12f/f133 816995 0 2024-04-22T23:35:13.970 INFO:tasks.workunit.client.0.smithi107.stdout:9/898: dwrite d5/d29/f8c [0,4194304] 0 2024-04-22T23:35:13.971 INFO:tasks.workunit.client.0.smithi107.stdout:5/911: dwrite d9/db/d94/fd3 [4194304,4194304] 0 2024-04-22T23:35:13.971 INFO:tasks.workunit.client.0.smithi107.stdout:1/835: rmdir d7/d19/d20/d39/d93 39 2024-04-22T23:35:13.973 INFO:tasks.workunit.client.0.smithi107.stdout:6/948: mknod db/d20/d2a/d35/d69/db3/c11e 0 2024-04-22T23:35:13.977 INFO:tasks.workunit.client.0.smithi107.stdout:5/912: dread f7 [0,4194304] 0 2024-04-22T23:35:13.977 INFO:tasks.workunit.client.0.smithi107.stdout:5/913: write d9/db/f8c [1921597,78768] 0 2024-04-22T23:35:13.978 INFO:tasks.workunit.client.0.smithi107.stdout:4/915: unlink d1/d7/c1a 0 2024-04-22T23:35:13.981 INFO:tasks.workunit.client.0.smithi107.stdout:9/899: truncate d5/d7/d17/d1b/dcb/ffe 2652052 0 2024-04-22T23:35:13.984 INFO:tasks.workunit.client.0.smithi107.stdout:4/916: unlink d1/d7/d63/d78/da4/lba 0 2024-04-22T23:35:13.984 INFO:tasks.workunit.client.0.smithi107.stdout:4/917: chown d1/d21/d29/d2e/lf7 1723242 1 2024-04-22T23:35:13.984 INFO:tasks.workunit.client.0.smithi107.stdout:9/900: truncate d5/d24/f3c 501405 0 2024-04-22T23:35:13.986 INFO:tasks.workunit.client.0.smithi107.stdout:1/836: dwrite d7/d19/d54/f70 [0,4194304] 0 2024-04-22T23:35:13.987 INFO:tasks.workunit.client.0.smithi107.stdout:4/918: link d1/c13e d1/d1b/d44/c141 0 2024-04-22T23:35:13.987 INFO:tasks.workunit.client.0.smithi107.stdout:9/901: rmdir d5/d14/d2e/d4f 39 2024-04-22T23:35:13.988 INFO:tasks.workunit.client.0.smithi107.stdout:5/914: rmdir d9/d76/da0 39 2024-04-22T23:35:13.989 INFO:tasks.workunit.client.0.smithi107.stdout:5/915: symlink d9/db/d42/d75/l140 0 2024-04-22T23:35:13.990 INFO:tasks.workunit.client.0.smithi107.stdout:5/916: mknod d9/de6/c141 0 2024-04-22T23:35:13.990 INFO:tasks.workunit.client.0.smithi107.stdout:5/917: chown d9/db/d42/fcc 31653 1 2024-04-22T23:35:13.990 INFO:tasks.workunit.client.0.smithi107.stdout:9/902: mkdir d5/d7/d17/d1b/d1f/d36/d96/d133 0 2024-04-22T23:35:13.990 INFO:tasks.workunit.client.0.smithi107.stdout:5/918: truncate f7 2647589 0 2024-04-22T23:35:13.990 INFO:tasks.workunit.client.0.smithi107.stdout:5/919: write d9/db/dc1/dcb/de9/f11c [896451,91976] 0 2024-04-22T23:35:13.992 INFO:tasks.workunit.client.0.smithi107.stdout:7/879: rename d7/d21/d48/d5c/d9d/ddd/fe3 to d7/d21/d7e/f124 0 2024-04-22T23:35:13.993 INFO:tasks.workunit.client.0.smithi107.stdout:5/920: dread d9/f9d [0,4194304] 0 2024-04-22T23:35:13.993 INFO:tasks.workunit.client.0.smithi107.stdout:5/921: chown d9/d26/f3b 197768 1 2024-04-22T23:35:13.997 INFO:tasks.workunit.client.0.smithi107.stdout:7/880: creat d7/d21/d48/d5c/d9d/d6e/db2/d11b/f125 x:0 0 0 2024-04-22T23:35:13.997 INFO:tasks.workunit.client.0.smithi107.stdout:7/881: dread - d7/d21/d48/f113 zero size 2024-04-22T23:35:13.999 INFO:tasks.workunit.client.0.smithi107.stdout:5/922: dread d9/d26/d6a/f6f [0,4194304] 0 2024-04-22T23:35:13.999 INFO:tasks.workunit.client.0.smithi107.stdout:5/923: write d9/db/de/d72/f13d [469950,94350] 0 2024-04-22T23:35:13.999 INFO:tasks.workunit.client.1.smithi157.stdout:0/977: sync 2024-04-22T23:35:13.999 INFO:tasks.workunit.client.1.smithi157.stdout:8/845: sync 2024-04-22T23:35:13.999 INFO:tasks.workunit.client.1.smithi157.stdout:7/959: sync 2024-04-22T23:35:13.999 INFO:tasks.workunit.client.1.smithi157.stdout:8/846: chown d2/d34/d6a/d77/df2/c118 82116 1 2024-04-22T23:35:13.999 INFO:tasks.workunit.client.1.smithi157.stdout:0/978: read - d17/d23/d34/d67/d78/f127 zero size 2024-04-22T23:35:13.999 INFO:tasks.workunit.client.1.smithi157.stdout:7/960: dread - d0/d9/d29/d5e/da6/ffe zero size 2024-04-22T23:35:14.000 INFO:tasks.workunit.client.1.smithi157.stdout:8/847: mknod d2/d4/c124 0 2024-04-22T23:35:14.000 INFO:tasks.workunit.client.1.smithi157.stdout:8/848: truncate d2/d6/d21/d5f/f1b 1245199 0 2024-04-22T23:35:14.000 INFO:tasks.workunit.client.0.smithi107.stdout:5/924: write d9/d26/d6a/d100/f132 [1103542,93997] 0 2024-04-22T23:35:14.001 INFO:tasks.workunit.client.0.smithi107.stdout:5/925: truncate d9/db/d92/fea 770666 0 2024-04-22T23:35:14.001 INFO:tasks.workunit.client.0.smithi107.stdout:5/926: chown d9/db/de/l12c 308226482 1 2024-04-22T23:35:14.001 INFO:tasks.workunit.client.0.smithi107.stdout:5/927: creat d9/db/de/d72/d7c/da4/f142 x:0 0 0 2024-04-22T23:35:14.002 INFO:tasks.workunit.client.1.smithi157.stdout:7/961: symlink d0/d9/d10/l125 0 2024-04-22T23:35:14.002 INFO:tasks.workunit.client.1.smithi157.stdout:8/849: mkdir d2/d4/dbd/d97/d125 0 2024-04-22T23:35:14.002 INFO:tasks.workunit.client.1.smithi157.stdout:8/850: write d2/d4/dbd/dd8/d119/f4b [3901554,19529] 0 2024-04-22T23:35:14.002 INFO:tasks.workunit.client.1.smithi157.stdout:8/851: fdatasync d2/d6/d21/d5f/f1b 0 2024-04-22T23:35:14.002 INFO:tasks.workunit.client.1.smithi157.stdout:8/852: truncate d2/f103 1016378 0 2024-04-22T23:35:14.002 INFO:tasks.workunit.client.1.smithi157.stdout:8/853: creat d2/d34/d6a/d77/df2/f126 x:0 0 0 2024-04-22T23:35:14.004 INFO:tasks.workunit.client.0.smithi107.stdout:6/949: rename db/d20/d2a/d38/d4b/d9d to db/d20/d2c/d3b/d40/d9e/ded/d11f 0 2024-04-22T23:35:14.004 INFO:tasks.workunit.client.0.smithi107.stdout:6/950: chown db/d20/d58/f65 0 1 2024-04-22T23:35:14.005 INFO:tasks.workunit.client.0.smithi107.stdout:7/882: mkdir d7/d21/d48/d126 0 2024-04-22T23:35:14.007 INFO:tasks.workunit.client.0.smithi107.stdout:1/837: rename d7/cb to d7/d19/d2d/da4/c115 0 2024-04-22T23:35:14.007 INFO:tasks.workunit.client.0.smithi107.stdout:1/838: chown d7/d19 0 1 2024-04-22T23:35:14.008 INFO:tasks.workunit.client.1.smithi157.stdout:7/962: dread d0/d9/d42/d55/d80/d82/fe0 [0,4194304] 0 2024-04-22T23:35:14.008 INFO:tasks.workunit.client.1.smithi157.stdout:7/963: fsync d0/d9/d29/d49/d4a/ff6 0 2024-04-22T23:35:14.010 INFO:tasks.workunit.client.0.smithi107.stdout:7/883: link d7/d21/l69 d7/d1b/l127 0 2024-04-22T23:35:14.010 INFO:tasks.workunit.client.0.smithi107.stdout:9/903: dwrite d5/d7/d27/d46/fd5 [0,4194304] 0 2024-04-22T23:35:14.011 INFO:tasks.workunit.client.0.smithi107.stdout:9/904: read - d5/d14/d2e/ffa zero size 2024-04-22T23:35:14.011 INFO:tasks.workunit.client.0.smithi107.stdout:4/919: rename d1/d1b/d44/c13d to d1/d21/d29/d2e/d88/d99/c142 0 2024-04-22T23:35:14.011 INFO:tasks.workunit.client.0.smithi107.stdout:4/920: stat d1/d21/d52/fec 0 2024-04-22T23:35:14.011 INFO:tasks.workunit.client.0.smithi107.stdout:4/921: dread - d1/d7/d5b/dbf/d27/fb3 zero size 2024-04-22T23:35:14.011 INFO:tasks.workunit.client.0.smithi107.stdout:4/922: write d1/d21/d29/d2e/dfa/f11d [55285,25330] 0 2024-04-22T23:35:14.013 INFO:tasks.workunit.client.0.smithi107.stdout:1/839: mknod d7/d19/d54/c116 0 2024-04-22T23:35:14.014 INFO:tasks.workunit.client.1.smithi157.stdout:7/964: mknod d0/d9/d36/c126 0 2024-04-22T23:35:14.016 INFO:tasks.workunit.client.0.smithi107.stdout:5/928: dread d9/f4e [0,4194304] 0 2024-04-22T23:35:14.016 INFO:tasks.workunit.client.0.smithi107.stdout:5/929: creat d9/db/de/d72/d7c/da4/f143 x:0 0 0 2024-04-22T23:35:14.016 INFO:tasks.workunit.client.1.smithi157.stdout:7/965: symlink d0/d9/d42/d55/d80/d95/df8/d121/l127 0 2024-04-22T23:35:14.017 INFO:tasks.workunit.client.0.smithi107.stdout:5/930: stat d9/db/f3f 0 2024-04-22T23:35:14.017 INFO:tasks.workunit.client.0.smithi107.stdout:5/931: chown d9/db/de/l1c 7 1 2024-04-22T23:35:14.017 INFO:tasks.workunit.client.0.smithi107.stdout:5/932: write d9/f43 [374327,113850] 0 2024-04-22T23:35:14.018 INFO:tasks.workunit.client.0.smithi107.stdout:5/933: stat d9/db/l77 0 2024-04-22T23:35:14.018 INFO:tasks.workunit.client.0.smithi107.stdout:5/934: write d9/d26/f55 [4606569,86801] 0 2024-04-22T23:35:14.018 INFO:tasks.workunit.client.0.smithi107.stdout:9/905: creat d5/d7/d27/dfc/f134 x:0 0 0 2024-04-22T23:35:14.021 INFO:tasks.workunit.client.0.smithi107.stdout:9/906: mkdir d5/d135 0 2024-04-22T23:35:14.021 INFO:tasks.workunit.client.0.smithi107.stdout:9/907: chown d5/d7/d27/d46/d4b/f130 81322810 1 2024-04-22T23:35:14.026 INFO:tasks.workunit.client.0.smithi107.stdout:9/908: stat d5/d29/dbb/c102 0 2024-04-22T23:35:14.029 INFO:tasks.workunit.client.1.smithi157.stdout:8/854: write d2/d4/d30/f62 [4364125,51300] 0 2024-04-22T23:35:14.042 INFO:tasks.workunit.client.0.smithi107.stdout:4/923: dread d1/f5 [0,4194304] 0 2024-04-22T23:35:14.042 INFO:tasks.workunit.client.0.smithi107.stdout:6/951: dwrite db/d20/d2a/d38/d4b/fda [0,4194304] 0 2024-04-22T23:35:14.042 INFO:tasks.workunit.client.0.smithi107.stdout:6/952: write db/d20/d2a/d5d/fa9 [578992,129923] 0 2024-04-22T23:35:14.044 INFO:tasks.workunit.client.1.smithi157.stdout:7/966: dwrite d0/d9/d36/dbb/fe1 [0,4194304] 0 2024-04-22T23:35:14.045 INFO:tasks.workunit.client.0.smithi107.stdout:4/924: truncate d1/d1b/f9d 946148 0 2024-04-22T23:35:14.045 INFO:tasks.workunit.client.0.smithi107.stdout:4/925: chown d1/d7/d53/f84 99 1 2024-04-22T23:35:14.045 INFO:tasks.workunit.client.0.smithi107.stdout:4/926: readlink d1/d7/d5b/dbf/d27/l5f 0 2024-04-22T23:35:14.045 INFO:tasks.workunit.client.0.smithi107.stdout:4/927: dread - d1/d7/d5b/dbf/da8/f121 zero size 2024-04-22T23:35:14.045 INFO:tasks.workunit.client.0.smithi107.stdout:8/877: sync 2024-04-22T23:35:14.045 INFO:tasks.workunit.client.0.smithi107.stdout:3/826: sync 2024-04-22T23:35:14.045 INFO:tasks.workunit.client.0.smithi107.stdout:8/878: readlink d0/d7/la5 0 2024-04-22T23:35:14.045 INFO:tasks.workunit.client.0.smithi107.stdout:8/879: truncate d0/d7/d8/d41/d53/f77 4863877 0 2024-04-22T23:35:14.045 INFO:tasks.workunit.client.0.smithi107.stdout:8/880: read - d0/d73/db0/d107/d33/d37/d9f/feb zero size 2024-04-22T23:35:14.047 INFO:tasks.workunit.client.0.smithi107.stdout:4/928: mkdir d1/d7/d5b/dbf/da8/d135/d143 0 2024-04-22T23:35:14.048 INFO:tasks.workunit.client.0.smithi107.stdout:3/827: mknod db/d21/d27/d15/d24/d87/d10a/d111/c112 0 2024-04-22T23:35:14.048 INFO:tasks.workunit.client.1.smithi157.stdout:7/967: symlink d0/d9/d29/d5e/l128 0 2024-04-22T23:35:14.048 INFO:tasks.workunit.client.1.smithi157.stdout:7/968: write d0/d9/d10/f18 [1161518,62904] 0 2024-04-22T23:35:14.049 INFO:tasks.workunit.client.0.smithi107.stdout:8/881: rmdir d0/d73 39 2024-04-22T23:35:14.050 INFO:tasks.workunit.client.1.smithi157.stdout:7/969: creat d0/d9/db/df/d60/f129 x:0 0 0 2024-04-22T23:35:14.051 INFO:tasks.workunit.client.0.smithi107.stdout:7/884: dwrite d7/d21/da4/d81/fc5 [0,4194304] 0 2024-04-22T23:35:14.051 INFO:tasks.workunit.client.0.smithi107.stdout:7/885: creat d7/d21/d7e/ddf/d11e/f128 x:0 0 0 2024-04-22T23:35:14.051 INFO:tasks.workunit.client.0.smithi107.stdout:4/929: link d1/d7/d5b/dbf/fdc d1/d7/f144 0 2024-04-22T23:35:14.052 INFO:tasks.workunit.client.1.smithi157.stdout:0/979: dwrite d17/d38/f80 [4194304,4194304] 0 2024-04-22T23:35:14.053 INFO:tasks.workunit.client.1.smithi157.stdout:7/970: symlink d0/d9/d29/dde/l12a 0 2024-04-22T23:35:14.053 INFO:tasks.workunit.client.0.smithi107.stdout:3/828: unlink c7 0 2024-04-22T23:35:14.056 INFO:tasks.workunit.client.0.smithi107.stdout:6/953: dread db/d20/d2c/d3b/f60 [4194304,4194304] 0 2024-04-22T23:35:14.056 INFO:tasks.workunit.client.1.smithi157.stdout:7/971: creat d0/d9/d29/d49/d4a/d11a/f12b x:0 0 0 2024-04-22T23:35:14.056 INFO:tasks.workunit.client.1.smithi157.stdout:7/972: write d0/d9/de/fb9 [1464498,37273] 0 2024-04-22T23:35:14.058 INFO:tasks.workunit.client.0.smithi107.stdout:7/886: mkdir d7/d21/d54/dcc/d109/d129 0 2024-04-22T23:35:14.058 INFO:tasks.workunit.client.0.smithi107.stdout:7/887: write d7/d21/d48/daf/fdc [150119,97924] 0 2024-04-22T23:35:14.060 INFO:tasks.workunit.client.0.smithi107.stdout:3/829: creat db/d21/d27/d15/d24/d52/d60/dca/f113 x:0 0 0 2024-04-22T23:35:14.062 INFO:tasks.workunit.client.0.smithi107.stdout:6/954: creat db/d20/d2a/d38/d61/d6f/d105/f120 x:0 0 0 2024-04-22T23:35:14.063 INFO:tasks.workunit.client.0.smithi107.stdout:7/888: creat d7/d21/d54/dcc/d109/d129/f12a x:0 0 0 2024-04-22T23:35:14.065 INFO:tasks.workunit.client.0.smithi107.stdout:3/830: mkdir db/d21/d27/d15/d24/d87/d114 0 2024-04-22T23:35:14.065 INFO:tasks.workunit.client.0.smithi107.stdout:3/831: getdents db/d21/d27/d15/d29/d2a/d45/ddd/d102 0 2024-04-22T23:35:14.068 INFO:tasks.workunit.client.1.smithi157.stdout:8/855: dwrite d2/d6/d21/d5f/f6f [0,4194304] 0 2024-04-22T23:35:14.069 INFO:tasks.workunit.client.0.smithi107.stdout:8/882: dwrite d0/f5 [0,4194304] 0 2024-04-22T23:35:14.069 INFO:tasks.workunit.client.0.smithi107.stdout:3/832: rmdir db/d21/d27/d15/d24/d52/d60/dca/dfc 39 2024-04-22T23:35:14.069 INFO:tasks.workunit.client.0.smithi107.stdout:1/840: dwrite d7/d19/d20/d39/d53/d43/ff0 [0,4194304] 0 2024-04-22T23:35:14.070 INFO:tasks.workunit.client.0.smithi107.stdout:4/930: dread d1/d1b/f9d [0,4194304] 0 2024-04-22T23:35:14.070 INFO:tasks.workunit.client.0.smithi107.stdout:4/931: stat d1/d7/lf4 0 2024-04-22T23:35:14.071 INFO:tasks.workunit.client.1.smithi157.stdout:8/856: rename d2/d6/d21/d5f/d17/d7f/dfe to d2/d4/dbd/d67/d127 0 2024-04-22T23:35:14.072 INFO:tasks.workunit.client.1.smithi157.stdout:7/973: truncate d0/d9/d36/dbb/fe1 879923 0 2024-04-22T23:35:14.072 INFO:tasks.workunit.client.1.smithi157.stdout:7/974: truncate d0/d9/d29/dbd/f10e 131461 0 2024-04-22T23:35:14.073 INFO:tasks.workunit.client.0.smithi107.stdout:8/883: link d0/fb d0/f12f 0 2024-04-22T23:35:14.073 INFO:tasks.workunit.client.0.smithi107.stdout:8/884: write d0/d7/d20/de4/f111 [178426,67589] 0 2024-04-22T23:35:14.073 INFO:tasks.workunit.client.0.smithi107.stdout:5/935: dwrite d9/db/d94/fd3 [4194304,4194304] 0 2024-04-22T23:35:14.073 INFO:tasks.workunit.client.0.smithi107.stdout:6/955: dread db/d20/d2c/d3b/feb [0,4194304] 0 2024-04-22T23:35:14.074 INFO:tasks.workunit.client.1.smithi157.stdout:8/857: rename d2/d4/dbd/d67/f95 to d2/d4/dbd/d67/d127/f128 0 2024-04-22T23:35:14.074 INFO:tasks.workunit.client.1.smithi157.stdout:8/858: truncate d2/db1/d58/f110 1178036 0 2024-04-22T23:35:14.074 INFO:tasks.workunit.client.1.smithi157.stdout:8/859: write d2/fdb [418055,65655] 0 2024-04-22T23:35:14.074 INFO:tasks.workunit.client.0.smithi107.stdout:3/833: getdents db/d21/d27/d15/d24 0 2024-04-22T23:35:14.074 INFO:tasks.workunit.client.0.smithi107.stdout:3/834: write db/d21/d27/d15/d24/d52/f105 [165979,70712] 0 2024-04-22T23:35:14.076 INFO:tasks.workunit.client.1.smithi157.stdout:7/975: symlink d0/d9/db/df/d60/l12c 0 2024-04-22T23:35:14.079 INFO:tasks.workunit.client.1.smithi157.stdout:0/980: write d17/d38/f65 [855020,54238] 0 2024-04-22T23:35:14.081 INFO:tasks.workunit.client.1.smithi157.stdout:7/976: dread d0/d9/db/df/f9d [0,4194304] 0 2024-04-22T23:35:14.082 INFO:tasks.workunit.client.1.smithi157.stdout:0/981: mknod d17/d23/d9c/c14f 0 2024-04-22T23:35:14.083 INFO:tasks.workunit.client.0.smithi107.stdout:5/936: dread d9/db/de/d72/d7c/da4/fb3 [0,4194304] 0 2024-04-22T23:35:14.085 INFO:tasks.workunit.client.0.smithi107.stdout:1/841: rename d7/d19/d20/d39/d53/d43/d46/d68/dde to d7/d117 0 2024-04-22T23:35:14.085 INFO:tasks.workunit.client.0.smithi107.stdout:1/842: chown d7/d19/d20/d39/d53/d43/d46/d5c/db7/l103 4904401 1 2024-04-22T23:35:14.086 INFO:tasks.workunit.client.1.smithi157.stdout:0/982: mknod d17/d2b/d4f/d5a/c150 0 2024-04-22T23:35:14.087 INFO:tasks.workunit.client.0.smithi107.stdout:4/932: mkdir d1/d21/d29/d110/d145 0 2024-04-22T23:35:14.087 INFO:tasks.workunit.client.0.smithi107.stdout:4/933: write d1/fea [1563157,102612] 0 2024-04-22T23:35:14.089 INFO:tasks.workunit.client.1.smithi157.stdout:0/983: unlink d17/d23/d34/dee/f114 0 2024-04-22T23:35:14.091 INFO:tasks.workunit.client.1.smithi157.stdout:7/977: dread d0/d9/db/f110 [0,4194304] 0 2024-04-22T23:35:14.091 INFO:tasks.workunit.client.0.smithi107.stdout:7/889: dwrite d7/d21/da4/d81/fc5 [0,4194304] 0 2024-04-22T23:35:14.093 INFO:tasks.workunit.client.0.smithi107.stdout:3/835: rename db/d21/d27/fcb to db/d21/db1/f115 0 2024-04-22T23:35:14.093 INFO:tasks.workunit.client.1.smithi157.stdout:7/978: getdents d0/d9/d29/d49/d4a 0 2024-04-22T23:35:14.096 INFO:tasks.workunit.client.0.smithi107.stdout:1/843: mkdir d7/d19/d20/d39/d93/df4/d118 0 2024-04-22T23:35:14.096 INFO:tasks.workunit.client.0.smithi107.stdout:1/844: chown d7/d19/d20/d39/d53/d43/d46/dba/fe5 29516 1 2024-04-22T23:35:14.097 INFO:tasks.workunit.client.0.smithi107.stdout:7/890: dread d7/d21/d48/f9c [0,4194304] 0 2024-04-22T23:35:14.097 INFO:tasks.workunit.client.1.smithi157.stdout:7/979: dread d0/d9/d29/d49/d4a/fe5 [0,4194304] 0 2024-04-22T23:35:14.098 INFO:tasks.workunit.client.0.smithi107.stdout:4/934: link d1/d7/d53/d66/c74 d1/d7/d63/d78/d96/da0/c146 0 2024-04-22T23:35:14.098 INFO:tasks.workunit.client.0.smithi107.stdout:8/885: getdents d0/d7/d87/dc8 0 2024-04-22T23:35:14.100 INFO:tasks.workunit.client.0.smithi107.stdout:6/956: rmdir db/d20/d2c/d3b/d40/d9e 39 2024-04-22T23:35:14.100 INFO:tasks.workunit.client.0.smithi107.stdout:6/957: chown db/d20/d2a/d35/d69/d79/l89 1356477 1 2024-04-22T23:35:14.100 INFO:tasks.workunit.client.0.smithi107.stdout:6/958: stat db/d20/d2c/d3b/d40/d9e/f117 0 2024-04-22T23:35:14.100 INFO:tasks.workunit.client.0.smithi107.stdout:0/849: sync 2024-04-22T23:35:14.101 INFO:tasks.workunit.client.0.smithi107.stdout:2/965: sync 2024-04-22T23:35:14.102 INFO:tasks.workunit.client.0.smithi107.stdout:7/891: symlink d7/d21/d7e/deb/l12b 0 2024-04-22T23:35:14.102 INFO:tasks.workunit.client.0.smithi107.stdout:8/886: truncate d0/d7/f11 669232 0 2024-04-22T23:35:14.103 INFO:tasks.workunit.client.0.smithi107.stdout:6/959: unlink db/d20/d2a/cc4 0 2024-04-22T23:35:14.105 INFO:tasks.workunit.client.1.smithi157.stdout:7/980: dread d0/d9/d29/dbd/f10e [0,4194304] 0 2024-04-22T23:35:14.105 INFO:tasks.workunit.client.0.smithi107.stdout:5/937: rmdir d9/db/d92/dc2 39 2024-04-22T23:35:14.105 INFO:tasks.workunit.client.0.smithi107.stdout:5/938: chown d9/de6/f110 515768260 1 2024-04-22T23:35:14.106 INFO:tasks.workunit.client.0.smithi107.stdout:5/939: mknod d9/db/dc1/dcb/de9/d11b/c144 0 2024-04-22T23:35:14.107 INFO:tasks.workunit.client.0.smithi107.stdout:0/850: mkdir d1/d27/d20/d40/d75/d115 0 2024-04-22T23:35:14.108 INFO:tasks.workunit.client.0.smithi107.stdout:2/966: mknod d1/d12/d35/d48/d13/c145 0 2024-04-22T23:35:14.108 INFO:tasks.workunit.client.0.smithi107.stdout:5/940: creat d9/db/dc1/dcb/f145 x:0 0 0 2024-04-22T23:35:14.109 INFO:tasks.workunit.client.0.smithi107.stdout:6/960: mkdir db/d20/d2a/d38/d61/d6f/d121 0 2024-04-22T23:35:14.109 INFO:tasks.workunit.client.0.smithi107.stdout:5/941: dread - d9/db/d92/dc2/f120 zero size 2024-04-22T23:35:14.110 INFO:tasks.workunit.client.0.smithi107.stdout:0/851: unlink d1/d9/dd4/cd7 0 2024-04-22T23:35:14.111 INFO:tasks.workunit.client.0.smithi107.stdout:6/961: dread db/fd1 [0,4194304] 0 2024-04-22T23:35:14.111 INFO:tasks.workunit.client.0.smithi107.stdout:5/942: creat d9/f146 x:0 0 0 2024-04-22T23:35:14.111 INFO:tasks.workunit.client.0.smithi107.stdout:5/943: stat d9/d26/d6a/d100/f112 0 2024-04-22T23:35:14.111 INFO:tasks.workunit.client.0.smithi107.stdout:5/944: chown d9/db/de/c40 39421680 1 2024-04-22T23:35:14.111 INFO:tasks.workunit.client.0.smithi107.stdout:5/945: readlink d9/db/de/d72/l88 0 2024-04-22T23:35:14.112 INFO:tasks.workunit.client.0.smithi107.stdout:2/967: truncate d1/d2b/d59/f115 2240820 0 2024-04-22T23:35:14.112 INFO:tasks.workunit.client.0.smithi107.stdout:4/935: write d1/d21/d29/d2e/d88/d99/dd2/daf/dc1/d11f/f125 [2747258,31297] 0 2024-04-22T23:35:14.113 INFO:tasks.workunit.client.0.smithi107.stdout:0/852: mknod d1/d7b/d85/c116 0 2024-04-22T23:35:14.113 INFO:tasks.workunit.client.0.smithi107.stdout:6/962: mknod db/d20/d2c/db5/dd2/c122 0 2024-04-22T23:35:14.113 INFO:tasks.workunit.client.0.smithi107.stdout:6/963: chown db/d20/d2a/d38/d4b/f81 420 1 2024-04-22T23:35:14.114 INFO:tasks.workunit.client.0.smithi107.stdout:2/968: mkdir d1/d12/d35/d48/d13/d4f/d146 0 2024-04-22T23:35:14.115 INFO:tasks.workunit.client.0.smithi107.stdout:4/936: rmdir d1/d21/d29 39 2024-04-22T23:35:14.116 INFO:tasks.workunit.client.0.smithi107.stdout:0/853: creat d1/d9/d14/f117 x:0 0 0 2024-04-22T23:35:14.116 INFO:tasks.workunit.client.0.smithi107.stdout:0/854: readlink d1/d9/d14/d4f/l66 0 2024-04-22T23:35:14.116 INFO:tasks.workunit.client.0.smithi107.stdout:4/937: truncate d1/d7/d53/d66/d6b/fdf 84654 0 2024-04-22T23:35:14.116 INFO:tasks.workunit.client.0.smithi107.stdout:4/938: chown d1/d21/d29/d110/f129 39 1 2024-04-22T23:35:14.117 INFO:tasks.workunit.client.0.smithi107.stdout:0/855: creat d1/d27/d20/d40/dbd/f118 x:0 0 0 2024-04-22T23:35:14.118 INFO:tasks.workunit.client.0.smithi107.stdout:4/939: getdents d1/d1b 0 2024-04-22T23:35:14.119 INFO:tasks.workunit.client.0.smithi107.stdout:4/940: chown d1/d21/d29/d2e/d88/d99/dd2/ld8 21902649 1 2024-04-22T23:35:14.119 INFO:tasks.workunit.client.0.smithi107.stdout:0/856: link d1/d9/c23 d1/d27/d20/d40/dbd/df2/d109/c119 0 2024-04-22T23:35:14.122 INFO:tasks.workunit.client.0.smithi107.stdout:0/857: read d1/d9/d14/d4f/fa2 [26279,99127] 0 2024-04-22T23:35:14.122 INFO:tasks.workunit.client.0.smithi107.stdout:0/858: read d1/d9/d14/f4d [2548898,12190] 0 2024-04-22T23:35:14.122 INFO:tasks.workunit.client.0.smithi107.stdout:8/887: write d0/d7/d8/d41/d53/f57 [960008,2047] 0 2024-04-22T23:35:14.126 INFO:tasks.workunit.client.0.smithi107.stdout:4/941: write d1/d1b/f104 [323907,103161] 0 2024-04-22T23:35:14.127 INFO:tasks.workunit.client.1.smithi157.stdout:0/984: dwrite d17/d23/f32 [0,4194304] 0 2024-04-22T23:35:14.129 INFO:tasks.workunit.client.1.smithi157.stdout:0/985: symlink d17/d23/d34/d67/d7f/l151 0 2024-04-22T23:35:14.129 INFO:tasks.workunit.client.0.smithi107.stdout:7/892: dread d7/d21/da4/fab [0,4194304] 0 2024-04-22T23:35:14.131 INFO:tasks.workunit.client.1.smithi157.stdout:0/986: read d17/f22 [4265822,51377] 0 2024-04-22T23:35:14.131 INFO:tasks.workunit.client.0.smithi107.stdout:7/893: mknod d7/d21/d54/c12c 0 2024-04-22T23:35:14.132 INFO:tasks.workunit.client.1.smithi157.stdout:0/987: write d17/f1a [3328640,52206] 0 2024-04-22T23:35:14.133 INFO:tasks.workunit.client.0.smithi107.stdout:7/894: truncate d7/d21/d48/fc2 695559 0 2024-04-22T23:35:14.133 INFO:tasks.workunit.client.0.smithi107.stdout:7/895: dread - d7/d1b/f103 zero size 2024-04-22T23:35:14.159 INFO:tasks.workunit.client.0.smithi107.stdout:3/836: dwrite db/d21/d27/d15/d24/d52/d60/f92 [0,4194304] 0 2024-04-22T23:35:14.160 INFO:tasks.workunit.client.0.smithi107.stdout:1/845: dwrite d7/d19/d20/fc3 [0,4194304] 0 2024-04-22T23:35:14.160 INFO:tasks.workunit.client.0.smithi107.stdout:0/859: dwrite d1/d27/d20/d2c/d2e/f39 [4194304,4194304] 0 2024-04-22T23:35:14.160 INFO:tasks.workunit.client.0.smithi107.stdout:0/860: creat d1/d7b/f11a x:0 0 0 2024-04-22T23:35:14.160 INFO:tasks.workunit.client.0.smithi107.stdout:0/861: dread - d1/d9/ff6 zero size 2024-04-22T23:35:14.160 INFO:tasks.workunit.client.0.smithi107.stdout:5/946: dwrite d9/d26/f38 [8388608,4194304] 0 2024-04-22T23:35:14.161 INFO:tasks.workunit.client.1.smithi157.stdout:7/981: dwrite d0/d9/db/d35/f76 [0,4194304] 0 2024-04-22T23:35:14.161 INFO:tasks.workunit.client.0.smithi107.stdout:8/888: dwrite d0/d73/db0/d107/f110 [0,4194304] 0 2024-04-22T23:35:14.162 INFO:tasks.workunit.client.0.smithi107.stdout:8/889: creat d0/d73/db0/d107/d33/f130 x:0 0 0 2024-04-22T23:35:14.162 INFO:tasks.workunit.client.0.smithi107.stdout:6/964: dwrite db/d20/d2a/d38/d4b/fe5 [0,4194304] 0 2024-04-22T23:35:14.162 INFO:tasks.workunit.client.0.smithi107.stdout:6/965: chown db/d20/d2c/f32 3907361 1 2024-04-22T23:35:14.162 INFO:tasks.workunit.client.0.smithi107.stdout:2/969: dwrite d1/d2b/d59/ff2 [0,4194304] 0 2024-04-22T23:35:14.162 INFO:tasks.workunit.client.0.smithi107.stdout:2/970: dread - d1/d12/d35/d102/fa3 zero size 2024-04-22T23:35:14.162 INFO:tasks.workunit.client.0.smithi107.stdout:2/971: readlink d1/d2b/d3d/d57/l134 0 2024-04-22T23:35:14.162 INFO:tasks.workunit.client.0.smithi107.stdout:2/972: truncate d1/d12/d35/d102/f51 863864 0 2024-04-22T23:35:14.162 INFO:tasks.workunit.client.0.smithi107.stdout:2/973: chown d1/d12/d35/d102/dba/df1/f126 806344523 1 2024-04-22T23:35:14.163 INFO:tasks.workunit.client.1.smithi157.stdout:0/988: dwrite d17/ddb/f107 [0,4194304] 0 2024-04-22T23:35:14.164 INFO:tasks.workunit.client.1.smithi157.stdout:0/989: dread d17/d2b/f3d [4194304,4194304] 0 2024-04-22T23:35:14.165 INFO:tasks.workunit.client.1.smithi157.stdout:7/982: unlink d0/d9/d29/d33/l9f 0 2024-04-22T23:35:14.165 INFO:tasks.workunit.client.1.smithi157.stdout:7/983: write d0/d9/d29/d33/d37/fcd [2001784,103014] 0 2024-04-22T23:35:14.165 INFO:tasks.workunit.client.1.smithi157.stdout:7/984: write d0/d9/db/df/dac/fdb [1030752,109741] 0 2024-04-22T23:35:14.166 INFO:tasks.workunit.client.0.smithi107.stdout:3/837: creat db/d21/d27/d15/db8/f116 x:0 0 0 2024-04-22T23:35:14.166 INFO:tasks.workunit.client.0.smithi107.stdout:3/838: readlink db/d21/d27/d15/d29/la9 0 2024-04-22T23:35:14.166 INFO:tasks.workunit.client.0.smithi107.stdout:3/839: chown db/d21/d27/d15/d29/d2a/d45/ddd/d25/d53/c63 10917 1 2024-04-22T23:35:14.167 INFO:tasks.workunit.client.0.smithi107.stdout:5/947: creat d9/db/de/d72/d7c/f147 x:0 0 0 2024-04-22T23:35:14.167 INFO:tasks.workunit.client.0.smithi107.stdout:5/948: truncate d9/d26/d79/f114 769638 0 2024-04-22T23:35:14.172 INFO:tasks.workunit.client.1.smithi157.stdout:7/985: rmdir d0/d9/d29/d49/d64 39 2024-04-22T23:35:14.172 INFO:tasks.workunit.client.1.smithi157.stdout:7/986: chown d0/d9/d29/d33/d37/c6d 0 1 2024-04-22T23:35:14.175 INFO:tasks.workunit.client.1.smithi157.stdout:7/987: creat d0/d9/d42/d55/d80/d95/df8/f12d x:0 0 0 2024-04-22T23:35:14.179 INFO:tasks.workunit.client.0.smithi107.stdout:4/942: write d1/d7/d53/d66/d6b/fdf [881451,60994] 0 2024-04-22T23:35:14.180 INFO:tasks.workunit.client.0.smithi107.stdout:8/890: rmdir d0/d73/db0/d107/d3b/d7b 39 2024-04-22T23:35:14.182 INFO:tasks.workunit.client.0.smithi107.stdout:3/840: dread db/d21/d27/d15/d29/d2a/d40/ff2 [0,4194304] 0 2024-04-22T23:35:14.186 INFO:tasks.workunit.client.0.smithi107.stdout:2/974: creat d1/d63/de6/f147 x:0 0 0 2024-04-22T23:35:14.187 INFO:tasks.workunit.client.0.smithi107.stdout:1/846: rmdir d7/d19/d20/d39/d53/d43/d46/d5c/dac 39 2024-04-22T23:35:14.189 INFO:tasks.workunit.client.1.smithi157.stdout:7/988: write d0/d9/d29/f4d [693873,14610] 0 2024-04-22T23:35:14.190 INFO:tasks.workunit.client.0.smithi107.stdout:4/943: creat d1/d21/d29/d2e/d88/d99/d12c/f147 x:0 0 0 2024-04-22T23:35:14.190 INFO:tasks.workunit.client.0.smithi107.stdout:8/891: symlink d0/l131 0 2024-04-22T23:35:14.192 INFO:tasks.workunit.client.0.smithi107.stdout:2/975: rename d1/d12/d35/d48/d13/f32 to d1/d12/d35/d48/d138/f148 0 2024-04-22T23:35:14.193 INFO:tasks.workunit.client.0.smithi107.stdout:4/944: rename d1/d21/d29/d110/d145 to d1/d21/d29/d2e/d88/d99/d148 0 2024-04-22T23:35:14.194 INFO:tasks.workunit.client.0.smithi107.stdout:2/976: rename d1/cbf to d1/d2b/d3d/d57/c149 0 2024-04-22T23:35:14.195 INFO:tasks.workunit.client.0.smithi107.stdout:4/945: creat d1/d7/d5b/dbf/d27/f149 x:0 0 0 2024-04-22T23:35:14.196 INFO:tasks.workunit.client.0.smithi107.stdout:2/977: mkdir d1/d12/d35/d102/d7e/da8/d10e/d11c/d14a 0 2024-04-22T23:35:14.196 INFO:tasks.workunit.client.0.smithi107.stdout:4/946: creat d1/d21/d29/f14a x:0 0 0 2024-04-22T23:35:14.197 INFO:tasks.workunit.client.0.smithi107.stdout:3/841: dread db/d21/d27/f39 [0,4194304] 0 2024-04-22T23:35:14.205 INFO:tasks.workunit.client.0.smithi107.stdout:8/892: write d0/d73/db0/d107/d3b/faf [3834961,17038] 0 2024-04-22T23:35:14.205 INFO:tasks.workunit.client.0.smithi107.stdout:8/893: dread - d0/d73/db0/d107/d33/d37/d9f/feb zero size 2024-04-22T23:35:14.205 INFO:tasks.workunit.client.0.smithi107.stdout:8/894: truncate d0/d7/d87/dc8/fee 449763 0 2024-04-22T23:35:14.205 INFO:tasks.workunit.client.0.smithi107.stdout:8/895: stat d0/d7/d20/da2/da3/fbb 0 2024-04-22T23:35:14.205 INFO:tasks.workunit.client.0.smithi107.stdout:8/896: chown d0/d7/d8/d41/dec/l60 3087 1 2024-04-22T23:35:14.206 INFO:tasks.workunit.client.1.smithi157.stdout:7/989: dread d0/d9/f1d [0,4194304] 0 2024-04-22T23:35:14.207 INFO:tasks.workunit.client.0.smithi107.stdout:7/896: dwrite d7/d21/d7e/fb3 [0,4194304] 0 2024-04-22T23:35:14.207 INFO:tasks.workunit.client.0.smithi107.stdout:7/897: write d7/d21/d48/d5c/d9d/db1/ffe [902376,11338] 0 2024-04-22T23:35:14.216 INFO:tasks.workunit.client.0.smithi107.stdout:8/897: read d0/d7/d20/da2/dc3/fcf [1852529,62354] 0 2024-04-22T23:35:14.217 INFO:tasks.workunit.client.0.smithi107.stdout:3/842: dwrite db/d21/d27/d15/d24/d87/f89 [4194304,4194304] 0 2024-04-22T23:35:14.220 INFO:tasks.workunit.client.0.smithi107.stdout:3/843: truncate db/d21/d27/d15/d29/d4f/f77 1416048 0 2024-04-22T23:35:14.220 INFO:tasks.workunit.client.0.smithi107.stdout:3/844: chown db/d21/d27/d15/d24/d52/d60/dca/dcc/cdf 35464 1 2024-04-22T23:35:14.220 INFO:tasks.workunit.client.0.smithi107.stdout:3/845: write db/d21/d27/d15/d24/d52/f105 [1099862,38575] 0 2024-04-22T23:35:14.220 INFO:tasks.workunit.client.0.smithi107.stdout:3/846: read - db/d21/fd9 zero size 2024-04-22T23:35:14.220 INFO:tasks.workunit.client.0.smithi107.stdout:0/862: dwrite d1/d84/d99/fe6 [0,4194304] 0 2024-04-22T23:35:14.221 INFO:tasks.workunit.client.0.smithi107.stdout:6/966: dwrite db/d20/f112 [0,4194304] 0 2024-04-22T23:35:14.221 INFO:tasks.workunit.client.1.smithi157.stdout:0/990: dwrite d17/d2b/fb4 [0,4194304] 0 2024-04-22T23:35:14.222 INFO:tasks.workunit.client.0.smithi107.stdout:3/847: mknod db/d21/d27/d15/d29/c117 0 2024-04-22T23:35:14.224 INFO:tasks.workunit.client.0.smithi107.stdout:1/847: dwrite d7/d19/d2d/fee [0,4194304] 0 2024-04-22T23:35:14.226 INFO:tasks.workunit.client.1.smithi157.stdout:0/991: stat d17/f19 0 2024-04-22T23:35:14.226 INFO:tasks.workunit.client.1.smithi157.stdout:0/992: fsync d17/d23/fc9 0 2024-04-22T23:35:14.227 INFO:tasks.workunit.client.0.smithi107.stdout:0/863: getdents d1/d27/d20/d40/dbd/ddc 0 2024-04-22T23:35:14.227 INFO:tasks.workunit.client.0.smithi107.stdout:6/967: dread db/d20/d2a/d35/d69/d79/f95 [0,4194304] 0 2024-04-22T23:35:14.229 INFO:tasks.workunit.client.1.smithi157.stdout:7/990: rmdir d0/d9/d42/d55/d80/d95/df8 39 2024-04-22T23:35:14.229 INFO:tasks.workunit.client.1.smithi157.stdout:7/991: stat d0/d9/d36/dbb/fe1 0 2024-04-22T23:35:14.229 INFO:tasks.workunit.client.1.smithi157.stdout:7/992: fdatasync d0/d9/d29/d49/d64/fb5 0 2024-04-22T23:35:14.229 INFO:tasks.workunit.client.1.smithi157.stdout:7/993: fsync d0/d9/de/f23 0 2024-04-22T23:35:14.232 INFO:tasks.workunit.client.0.smithi107.stdout:3/848: truncate db/d21/d27/d15/d29/d2a/f43 5106283 0 2024-04-22T23:35:14.234 INFO:tasks.workunit.client.1.smithi157.stdout:7/994: rename d0/d9/d29/d49/l11c to d0/d9/d29/d49/d64/l12e 0 2024-04-22T23:35:14.234 INFO:tasks.workunit.client.1.smithi157.stdout:7/995: chown d0/f4 50665023 1 2024-04-22T23:35:14.235 INFO:tasks.workunit.client.1.smithi157.stdout:7/996: creat d0/d9/d42/d55/d80/de8/f12f x:0 0 0 2024-04-22T23:35:14.235 INFO:tasks.workunit.client.1.smithi157.stdout:7/997: chown d0/d9/db/dc3/ff0 5 1 2024-04-22T23:35:14.236 INFO:tasks.workunit.client.0.smithi107.stdout:8/898: dwrite d0/d7/d20/de4/f11f [0,4194304] 0 2024-04-22T23:35:14.236 INFO:tasks.workunit.client.0.smithi107.stdout:7/898: dwrite d7/d21/d54/f98 [0,4194304] 0 2024-04-22T23:35:14.236 INFO:tasks.workunit.client.0.smithi107.stdout:7/899: write d7/d21/d48/f55 [1935454,105517] 0 2024-04-22T23:35:14.236 INFO:tasks.workunit.client.0.smithi107.stdout:7/900: chown d7/d1b/f103 3641201 1 2024-04-22T23:35:14.239 INFO:tasks.workunit.client.0.smithi107.stdout:6/968: link db/d20/d2a/d35/d69/d79/cc6 db/d20/d2a/d35/d69/db3/d100/c123 0 2024-04-22T23:35:14.248 INFO:tasks.workunit.client.1.smithi157.stdout:0/993: dwrite d17/d2b/f47 [0,4194304] 0 2024-04-22T23:35:14.249 INFO:tasks.workunit.client.1.smithi157.stdout:0/994: dread - d17/d2b/d4f/d9b/f141 zero size 2024-04-22T23:35:14.249 INFO:tasks.workunit.client.0.smithi107.stdout:6/969: getdents db/d20/d2a/d35 0 2024-04-22T23:35:14.249 INFO:tasks.workunit.client.0.smithi107.stdout:6/970: chown db/d20/d2c/d3b/f43 33525 1 2024-04-22T23:35:14.250 INFO:tasks.workunit.client.0.smithi107.stdout:7/901: dread d7/d21/da4/f43 [0,4194304] 0 2024-04-22T23:35:14.252 INFO:tasks.workunit.client.0.smithi107.stdout:2/978: rename d1/d63/de6 to d1/d12/d35/d48/d13/d14b 0 2024-04-22T23:35:14.253 INFO:tasks.workunit.client.0.smithi107.stdout:1/848: dwrite d7/d19/d20/f52 [0,4194304] 0 2024-04-22T23:35:14.254 INFO:tasks.workunit.client.0.smithi107.stdout:0/864: dwrite d1/d27/d20/f60 [0,4194304] 0 2024-04-22T23:35:14.255 INFO:tasks.workunit.client.1.smithi157.stdout:7/998: dread d0/d9/d10/f30 [4194304,4194304] 0 2024-04-22T23:35:14.257 INFO:tasks.workunit.client.1.smithi157.stdout:7/999: creat d0/d9/d72/f130 x:0 0 0 2024-04-22T23:35:14.258 INFO:tasks.workunit.client.0.smithi107.stdout:2/979: creat d1/d12/d35/d102/dba/df1/f14c x:0 0 0 2024-04-22T23:35:14.259 INFO:tasks.workunit.client.0.smithi107.stdout:3/849: rename db/d21/d27/d15/d24/d52/d60/dca/dfc/f107 to db/d21/d27/d15/d29/d2a/d99/f118 0 2024-04-22T23:35:14.263 INFO:tasks.workunit.client.0.smithi107.stdout:1/849: mkdir d7/d19/d20/d39/d53/d43/d46/d119 0 2024-04-22T23:35:14.264 INFO:tasks.workunit.client.0.smithi107.stdout:0/865: dread d1/d9/f5e [0,4194304] 0 2024-04-22T23:35:14.265 INFO:tasks.workunit.client.0.smithi107.stdout:6/971: mkdir db/d20/d2a/d38/d61/d6f/da2/d84/d124 0 2024-04-22T23:35:14.268 INFO:tasks.workunit.client.0.smithi107.stdout:2/980: mknod d1/d12/d35/d102/d7e/da8/d125/c14d 0 2024-04-22T23:35:14.268 INFO:tasks.workunit.client.0.smithi107.stdout:2/981: dread - d1/d12/d35/d102/d87/f127 zero size 2024-04-22T23:35:14.270 INFO:tasks.workunit.client.0.smithi107.stdout:1/850: mkdir d7/d19/d20/d39/d93/df4/d11a 0 2024-04-22T23:35:14.270 INFO:tasks.workunit.client.0.smithi107.stdout:1/851: dread - d7/d19/d2d/d89/f102 zero size 2024-04-22T23:35:14.270 INFO:tasks.workunit.client.1.smithi157.stdout:0/995: dwrite d17/d23/d34/d67/d78/fbb [0,4194304] 0 2024-04-22T23:35:14.271 INFO:tasks.workunit.client.0.smithi107.stdout:8/899: dwrite d0/d73/db0/d107/f23 [4194304,4194304] 0 2024-04-22T23:35:14.271 INFO:tasks.workunit.client.0.smithi107.stdout:8/900: stat d0/d7/d20/de4/f11f 0 2024-04-22T23:35:14.272 INFO:tasks.workunit.client.0.smithi107.stdout:6/972: mknod db/d20/d2a/dca/d10f/c125 0 2024-04-22T23:35:14.272 INFO:tasks.workunit.client.0.smithi107.stdout:6/973: chown db/d20/d49/d7b/l96 115566 1 2024-04-22T23:35:14.273 INFO:tasks.workunit.client.1.smithi157.stdout:0/996: creat d17/d23/d34/d67/d136/d143/f152 x:0 0 0 2024-04-22T23:35:14.274 INFO:tasks.workunit.client.1.smithi157.stdout:0/997: mknod d17/d23/d34/d67/d104/d121/c153 0 2024-04-22T23:35:14.276 INFO:tasks.workunit.client.1.smithi157.stdout:0/998: rmdir d17/d2b/d4f/dc5/dcc/de9/df6/d12b 39 2024-04-22T23:35:14.276 INFO:tasks.workunit.client.0.smithi107.stdout:2/982: creat d1/d12/d35/d102/d7e/dae/de8/f14e x:0 0 0 2024-04-22T23:35:14.277 INFO:tasks.workunit.client.0.smithi107.stdout:1/852: mkdir d7/d19/d20/d39/d11b 0 2024-04-22T23:35:14.277 INFO:tasks.workunit.client.0.smithi107.stdout:1/853: readlink d7/d19/d2d/l6c 0 2024-04-22T23:35:14.278 INFO:tasks.workunit.client.1.smithi157.stdout:0/999: creat d17/d23/d34/d67/d78/dc7/f154 x:0 0 0 2024-04-22T23:35:14.278 INFO:tasks.workunit.client.0.smithi107.stdout:1/854: read d7/d19/d2d/d89/fd0 [124826,101138] 0 2024-04-22T23:35:14.283 INFO:tasks.workunit.client.0.smithi107.stdout:1/855: dread d7/d19/d20/d39/d53/d43/f49 [4194304,4194304] 0 2024-04-22T23:35:14.284 INFO:tasks.workunit.client.0.smithi107.stdout:3/850: dwrite db/d21/d27/f23 [0,4194304] 0 2024-04-22T23:35:14.287 INFO:tasks.workunit.client.0.smithi107.stdout:1/856: rename d7/d19/d20/f38 to d7/d19/d42/f11c 0 2024-04-22T23:35:14.287 INFO:tasks.workunit.client.0.smithi107.stdout:6/974: getdents db/d20/d2a/d35/d69/db3 0 2024-04-22T23:35:14.289 INFO:tasks.workunit.client.0.smithi107.stdout:3/851: symlink db/d21/d27/d15/d29/d2a/l119 0 2024-04-22T23:35:14.290 INFO:tasks.workunit.client.0.smithi107.stdout:0/866: dwrite d1/d27/f38 [0,4194304] 0 2024-04-22T23:35:14.292 INFO:tasks.workunit.client.0.smithi107.stdout:8/901: dwrite d0/d73/db0/d107/d33/d9b/f12a [0,4194304] 0 2024-04-22T23:35:14.292 INFO:tasks.workunit.client.0.smithi107.stdout:8/902: dread - d0/d7/d87/f10e zero size 2024-04-22T23:35:14.293 INFO:tasks.workunit.client.0.smithi107.stdout:6/975: rename db/d20/d2a/d35/d69/db3/ff0 to db/d20/d2c/f126 0 2024-04-22T23:35:14.294 INFO:tasks.workunit.client.0.smithi107.stdout:3/852: mknod db/d21/d27/d15/d24/d52/d60/dca/dfc/c11a 0 2024-04-22T23:35:14.294 INFO:tasks.workunit.client.0.smithi107.stdout:3/853: write db/d21/d27/d15/db8/f116 [1031568,118402] 0 2024-04-22T23:35:14.294 INFO:tasks.workunit.client.0.smithi107.stdout:3/854: fdatasync db/d21/fd9 0 2024-04-22T23:35:14.295 INFO:tasks.workunit.client.1.smithi157.stdout:8/860: sync 2024-04-22T23:35:14.297 INFO:tasks.workunit.client.1.smithi157.stdout:8/861: rename d2/d4/l83 to d2/d4/d73/l129 0 2024-04-22T23:35:14.297 INFO:tasks.workunit.client.1.smithi157.stdout:8/862: chown d2/d6/d21/d5f 210779562 1 2024-04-22T23:35:14.298 INFO:tasks.workunit.client.0.smithi107.stdout:8/903: unlink d0/d7/d20/d89/fe2 0 2024-04-22T23:35:14.298 INFO:tasks.workunit.client.0.smithi107.stdout:8/904: chown d0/d3a 1 1 2024-04-22T23:35:14.299 INFO:tasks.workunit.client.0.smithi107.stdout:8/905: readlink d0/d7/d8/d41/l11a 0 2024-04-22T23:35:14.299 INFO:tasks.workunit.client.0.smithi107.stdout:6/976: mknod db/d20/d2a/d5d/c127 0 2024-04-22T23:35:14.300 INFO:tasks.workunit.client.0.smithi107.stdout:6/977: write db/d20/d2c/d3b/fa8 [4236916,103762] 0 2024-04-22T23:35:14.300 INFO:tasks.workunit.client.0.smithi107.stdout:8/906: mkdir d0/d7/d20/d2d/d132 0 2024-04-22T23:35:14.300 INFO:tasks.workunit.client.0.smithi107.stdout:0/867: dread d1/d27/d20/d2c/fbb [0,4194304] 0 2024-04-22T23:35:14.302 INFO:tasks.workunit.client.0.smithi107.stdout:8/907: mknod d0/d3a/d46/d59/ddd/c133 0 2024-04-22T23:35:14.302 INFO:tasks.workunit.client.0.smithi107.stdout:6/978: dread db/d20/d49/f6e [0,4194304] 0 2024-04-22T23:35:14.302 INFO:tasks.workunit.client.0.smithi107.stdout:8/908: chown d0/d73/db0/d107/c95 1893002768 1 2024-04-22T23:35:14.303 INFO:tasks.workunit.client.0.smithi107.stdout:8/909: creat d0/d85/d51/f134 x:0 0 0 2024-04-22T23:35:14.303 INFO:tasks.workunit.client.0.smithi107.stdout:8/910: write d0/f125 [968260,7672] 0 2024-04-22T23:35:14.306 INFO:tasks.workunit.client.0.smithi107.stdout:3/855: write db/d21/d27/d15/db8/f82 [697367,16692] 0 2024-04-22T23:35:14.306 INFO:tasks.workunit.client.0.smithi107.stdout:3/856: chown db/d21/d27/d36/cc0 279 1 2024-04-22T23:35:14.307 INFO:tasks.workunit.client.0.smithi107.stdout:9/909: sync 2024-04-22T23:35:14.310 INFO:tasks.workunit.client.0.smithi107.stdout:9/910: rename d5/d7/d17/c125 to d5/d24/c136 0 2024-04-22T23:35:14.310 INFO:tasks.workunit.client.0.smithi107.stdout:1/857: dwrite d7/d19/d20/d39/faf [0,4194304] 0 2024-04-22T23:35:14.310 INFO:tasks.workunit.client.0.smithi107.stdout:8/911: dread d0/d7/d20/da2/fa9 [0,4194304] 0 2024-04-22T23:35:14.313 INFO:tasks.workunit.client.0.smithi107.stdout:1/858: unlink d7/d19/d20/d39/d53/l106 0 2024-04-22T23:35:14.313 INFO:tasks.workunit.client.0.smithi107.stdout:0/868: dread d1/d27/d20/d2c/d41/f64 [0,4194304] 0 2024-04-22T23:35:14.314 INFO:tasks.workunit.client.0.smithi107.stdout:6/979: dread db/d20/d2a/f2f [0,4194304] 0 2024-04-22T23:35:14.315 INFO:tasks.workunit.client.0.smithi107.stdout:8/912: unlink d0/d73/db0/d107/d3b/d7b/ca1 0 2024-04-22T23:35:14.320 INFO:tasks.workunit.client.0.smithi107.stdout:0/869: mknod d1/d27/d8a/db5/d10f/c11b 0 2024-04-22T23:35:14.320 INFO:tasks.workunit.client.0.smithi107.stdout:2/983: rmdir d1/d12/d35/d102/d7e/dae/de8 39 2024-04-22T23:35:14.321 INFO:tasks.workunit.client.0.smithi107.stdout:1/859: rmdir d7/d19/d20/d39/d53/d43/d63/d91 39 2024-04-22T23:35:14.321 INFO:tasks.workunit.client.0.smithi107.stdout:9/911: dread d5/d29/d7b/f115 [0,4194304] 0 2024-04-22T23:35:14.321 INFO:tasks.workunit.client.0.smithi107.stdout:9/912: fsync d5/d14/f2f 0 2024-04-22T23:35:14.321 INFO:tasks.workunit.client.0.smithi107.stdout:9/913: readlink d5/d14/d2e/d4f/la0 0 2024-04-22T23:35:14.321 INFO:tasks.workunit.client.0.smithi107.stdout:9/914: chown d5/d7/d27/d46/d4b/fa9 6 1 2024-04-22T23:35:14.321 INFO:tasks.workunit.client.0.smithi107.stdout:2/984: symlink d1/d12/d35/d48/d13/d4f/da5/l14f 0 2024-04-22T23:35:14.325 INFO:tasks.workunit.client.0.smithi107.stdout:9/915: creat d5/d7/d27/d69/f137 x:0 0 0 2024-04-22T23:35:14.326 INFO:tasks.workunit.client.0.smithi107.stdout:3/857: truncate db/d21/d27/d15/d29/d2a/f5b 1981951 0 2024-04-22T23:35:14.328 INFO:tasks.workunit.client.0.smithi107.stdout:9/916: mkdir d5/d7/d27/d46/d4b/d138 0 2024-04-22T23:35:14.328 INFO:tasks.workunit.client.0.smithi107.stdout:9/917: chown d5/d7/d27/d46/fc6 102 1 2024-04-22T23:35:14.330 INFO:tasks.workunit.client.0.smithi107.stdout:3/858: getdents db/d21/d27/d15/d24/d52/d60/dca/dfc 0 2024-04-22T23:35:14.331 INFO:tasks.workunit.client.0.smithi107.stdout:3/859: truncate db/d21/d27/d15/d29/d2a/d45/ddd/d25/fe2 1743556 0 2024-04-22T23:35:14.331 INFO:tasks.workunit.client.0.smithi107.stdout:3/860: write db/d21/d27/d36/d9c/fa3 [1729653,94879] 0 2024-04-22T23:35:14.333 INFO:tasks.workunit.client.0.smithi107.stdout:2/985: dread d1/d12/d35/d102/d7e/f8e [0,4194304] 0 2024-04-22T23:35:14.333 INFO:tasks.workunit.client.0.smithi107.stdout:0/870: write d1/d27/d8a/db5/fc9 [2495262,86961] 0 2024-04-22T23:35:14.333 INFO:tasks.workunit.client.0.smithi107.stdout:0/871: readlink d1/d27/d20/d40/dbd/df2/l113 0 2024-04-22T23:35:14.334 INFO:tasks.workunit.client.0.smithi107.stdout:3/861: chown db/d21/d27/d36/l78 5318503 1 2024-04-22T23:35:14.336 INFO:tasks.workunit.client.0.smithi107.stdout:8/913: write d0/f96 [2003175,67702] 0 2024-04-22T23:35:14.337 INFO:tasks.workunit.client.0.smithi107.stdout:2/986: mknod d1/d12/d35/d102/dba/df1/d113/c150 0 2024-04-22T23:35:14.338 INFO:tasks.workunit.client.0.smithi107.stdout:0/872: creat d1/d7b/d85/ded/f11c x:0 0 0 2024-04-22T23:35:14.340 INFO:tasks.workunit.client.0.smithi107.stdout:8/914: link d0/d73/db0/d107/d33/d37/fcc d0/d7/dfa/f135 0 2024-04-22T23:35:14.340 INFO:tasks.workunit.client.0.smithi107.stdout:8/915: creat d0/d7/d20/de4/f136 x:0 0 0 2024-04-22T23:35:14.342 INFO:tasks.workunit.client.0.smithi107.stdout:2/987: rename d1/d12/d35/d102/d7e/dae/f10b to d1/d12/d35/d102/f151 0 2024-04-22T23:35:14.343 INFO:tasks.workunit.client.0.smithi107.stdout:0/873: mknod d1/d27/d20/d40/dbd/c11d 0 2024-04-22T23:35:14.344 INFO:tasks.workunit.client.1.smithi157.stdout:8/863: dwrite d2/d4/d73/f99 [0,4194304] 0 2024-04-22T23:35:14.347 INFO:tasks.workunit.client.0.smithi107.stdout:0/874: symlink d1/d9/d14/d4f/da6/l11e 0 2024-04-22T23:35:14.347 INFO:tasks.workunit.client.0.smithi107.stdout:0/875: chown d1/d27/d20/d40/dbd 2085090 1 2024-04-22T23:35:14.347 INFO:tasks.workunit.client.0.smithi107.stdout:0/876: chown d1/d9/c23 142 1 2024-04-22T23:35:14.348 INFO:tasks.workunit.client.0.smithi107.stdout:8/916: getdents d0 0 2024-04-22T23:35:14.349 INFO:tasks.workunit.client.1.smithi157.stdout:8/864: unlink d2/d6/cc1 0 2024-04-22T23:35:14.349 INFO:tasks.workunit.client.1.smithi157.stdout:8/865: readlink d2/d34/d6a/d77/lee 0 2024-04-22T23:35:14.350 INFO:tasks.workunit.client.0.smithi107.stdout:1/860: write d7/fa0 [1059106,76393] 0 2024-04-22T23:35:14.350 INFO:tasks.workunit.client.0.smithi107.stdout:1/861: fsync d7/f1b 0 2024-04-22T23:35:14.350 INFO:tasks.workunit.client.0.smithi107.stdout:8/917: rmdir d0/d7/d8/d41 39 2024-04-22T23:35:14.350 INFO:tasks.workunit.client.0.smithi107.stdout:8/918: chown d0/d7/l84 2138195539 1 2024-04-22T23:35:14.351 INFO:tasks.workunit.client.1.smithi157.stdout:8/866: link d2/d6/d21/d5f/d17/dc8/c122 d2/d6/d21/d5f/c12a 0 2024-04-22T23:35:14.352 INFO:tasks.workunit.client.1.smithi157.stdout:8/867: creat d2/d6/f12b x:0 0 0 2024-04-22T23:35:14.353 INFO:tasks.workunit.client.0.smithi107.stdout:1/862: rmdir d7/d19/d20/d39/d53/d43/d46/d5c/db7 39 2024-04-22T23:35:14.353 INFO:tasks.workunit.client.0.smithi107.stdout:1/863: stat d7/d19/d2d/fcf 0 2024-04-22T23:35:14.353 INFO:tasks.workunit.client.0.smithi107.stdout:8/919: mkdir d0/d7/d8/d41/d53/d137 0 2024-04-22T23:35:14.359 INFO:tasks.workunit.client.0.smithi107.stdout:1/864: truncate d7/d19/d20/d39/f7e 147624 0 2024-04-22T23:35:14.359 INFO:tasks.workunit.client.0.smithi107.stdout:8/920: mkdir d0/d73/db0/d107/d33/d9b/d138 0 2024-04-22T23:35:14.359 INFO:tasks.workunit.client.0.smithi107.stdout:8/921: write d0/d73/db0/d107/d3b/d7b/fcb [597360,5044] 0 2024-04-22T23:35:14.359 INFO:tasks.workunit.client.0.smithi107.stdout:8/922: mkdir d0/d7/d20/df9/d139 0 2024-04-22T23:35:14.359 INFO:tasks.workunit.client.0.smithi107.stdout:9/918: dwrite d5/d14/d2e/d4f/fb7 [0,4194304] 0 2024-04-22T23:35:14.359 INFO:tasks.workunit.client.0.smithi107.stdout:9/919: readlink d5/d7/d27/d46/d4b/def/l10d 0 2024-04-22T23:35:14.361 INFO:tasks.workunit.client.0.smithi107.stdout:9/920: symlink d5/d7/d27/d69/db6/l139 0 2024-04-22T23:35:14.362 INFO:tasks.workunit.client.0.smithi107.stdout:6/980: dwrite db/d20/d2a/d38/d61/f6d [0,4194304] 0 2024-04-22T23:35:14.363 INFO:tasks.workunit.client.1.smithi157.stdout:8/868: write d2/d4/dbd/dd8/d119/d13/f6c [697692,78683] 0 2024-04-22T23:35:14.363 INFO:tasks.workunit.client.1.smithi157.stdout:8/869: truncate d2/d4/dad/f11a 315411 0 2024-04-22T23:35:14.363 INFO:tasks.workunit.client.1.smithi157.stdout:8/870: readlink d2/db1/d58/l96 0 2024-04-22T23:35:14.363 INFO:tasks.workunit.client.0.smithi107.stdout:9/921: rename d5/d29/d7b/d91/c9e to d5/d7/d27/d46/d4b/dff/c13a 0 2024-04-22T23:35:14.367 INFO:tasks.workunit.client.0.smithi107.stdout:8/923: write d0/d7/d8/f1e [859094,127960] 0 2024-04-22T23:35:14.367 INFO:tasks.workunit.client.0.smithi107.stdout:8/924: chown d0/d73/db0/d107/d3b/f10 1054703 1 2024-04-22T23:35:14.376 INFO:tasks.workunit.client.0.smithi107.stdout:3/862: dwrite db/d21/d27/d15/f8c [0,4194304] 0 2024-04-22T23:35:14.377 INFO:tasks.workunit.client.0.smithi107.stdout:2/988: dwrite d1/d63/f69 [4194304,4194304] 0 2024-04-22T23:35:14.377 INFO:tasks.workunit.client.0.smithi107.stdout:2/989: dread - d1/d12/d35/d48/d13/d14b/f147 zero size 2024-04-22T23:35:14.377 INFO:tasks.workunit.client.1.smithi157.stdout:8/871: write d2/d4/dbd/fb4 [11445337,97605] 0 2024-04-22T23:35:14.378 INFO:tasks.workunit.client.0.smithi107.stdout:1/865: dwrite d7/d19/d2d/d89/f90 [0,4194304] 0 2024-04-22T23:35:14.378 INFO:tasks.workunit.client.0.smithi107.stdout:8/925: rmdir d0/d3a/d46 39 2024-04-22T23:35:14.378 INFO:tasks.workunit.client.0.smithi107.stdout:8/926: chown d0/d7/d20/d2d/cb6 1 1 2024-04-22T23:35:14.378 INFO:tasks.workunit.client.0.smithi107.stdout:8/927: chown d0/d73/db0/d107/d33/d9b/df3/df8 8072 1 2024-04-22T23:35:14.380 INFO:tasks.workunit.client.1.smithi157.stdout:8/872: unlink d2/d4/f108 0 2024-04-22T23:35:14.381 INFO:tasks.workunit.client.0.smithi107.stdout:3/863: truncate db/d21/d27/d15/d29/d2a/d40/ff2 2969029 0 2024-04-22T23:35:14.382 INFO:tasks.workunit.client.0.smithi107.stdout:2/990: dread d1/d12/d35/d54/f5f [0,4194304] 0 2024-04-22T23:35:14.382 INFO:tasks.workunit.client.0.smithi107.stdout:2/991: readlink d1/d2b/d3d/d57/l134 0 2024-04-22T23:35:14.382 INFO:tasks.workunit.client.0.smithi107.stdout:6/981: link db/d20/d2a/d35/d69/d79/c87 db/d20/d2a/d35/d69/db3/dbc/dd3/c128 0 2024-04-22T23:35:14.387 INFO:tasks.workunit.client.0.smithi107.stdout:1/866: link d7/d19/d54/fb2 d7/d19/d20/d39/d53/d43/d46/d5c/db7/f11d 0 2024-04-22T23:35:14.390 INFO:tasks.workunit.client.0.smithi107.stdout:8/928: truncate d0/d73/db0/d107/d3b/d7b/fe6 176921 0 2024-04-22T23:35:14.390 INFO:tasks.workunit.client.1.smithi157.stdout:8/873: dread d2/d6/d21/d5f/d17/f1e [0,4194304] 0 2024-04-22T23:35:14.390 INFO:tasks.workunit.client.1.smithi157.stdout:8/874: chown d2/d4/dbd/d67/d127 22 1 2024-04-22T23:35:14.392 INFO:tasks.workunit.client.0.smithi107.stdout:3/864: rename db/d21/d27/d15/d29/d2a/d45/ddd/d25/d53/cf9 to db/d21/d27/d15/d29/d2a/d45/ddd/da7/c11b 0 2024-04-22T23:35:14.392 INFO:tasks.workunit.client.1.smithi157.stdout:8/875: symlink d2/d4/dad/l12c 0 2024-04-22T23:35:14.392 INFO:tasks.workunit.client.1.smithi157.stdout:8/876: readlink d2/db1/d58/l8f 0 2024-04-22T23:35:14.393 INFO:tasks.workunit.client.0.smithi107.stdout:9/922: dwrite d5/d14/d2e/d4f/d83/f11a [0,4194304] 0 2024-04-22T23:35:14.393 INFO:tasks.workunit.client.0.smithi107.stdout:9/923: chown d5/d7/d17/de0/f10a 3 1 2024-04-22T23:35:14.393 INFO:tasks.workunit.client.0.smithi107.stdout:9/924: dread - d5/d14/d2e/d4f/d73/f11d zero size 2024-04-22T23:35:14.393 INFO:tasks.workunit.client.0.smithi107.stdout:6/982: stat db/d20/d2a/d38/f9f 0 2024-04-22T23:35:14.395 INFO:tasks.workunit.client.0.smithi107.stdout:1/867: unlink d7/d19/d20/d39/d53/d43/d46/d5c/dac/fbf 0 2024-04-22T23:35:14.395 INFO:tasks.workunit.client.0.smithi107.stdout:1/868: truncate d7/d19/d20/d39/d53/f4e 8601060 0 2024-04-22T23:35:14.396 INFO:tasks.workunit.client.0.smithi107.stdout:8/929: rmdir d0/d73/db0/d107/d3b/dd9/dda 39 2024-04-22T23:35:14.396 INFO:tasks.workunit.client.0.smithi107.stdout:8/930: fsync d0/f12f 0 2024-04-22T23:35:14.398 INFO:tasks.workunit.client.0.smithi107.stdout:2/992: rename d1/l9 to d1/d12/d35/d102/d7e/da8/d10e/d11c/l152 0 2024-04-22T23:35:14.398 INFO:tasks.workunit.client.0.smithi107.stdout:2/993: read d1/d12/f95 [170941,51443] 0 2024-04-22T23:35:14.400 INFO:tasks.workunit.client.0.smithi107.stdout:3/865: mkdir db/d21/d27/d15/d29/d2a/d45/deb/d11c 0 2024-04-22T23:35:14.404 INFO:tasks.workunit.client.0.smithi107.stdout:6/983: mkdir db/d20/d49/d129 0 2024-04-22T23:35:14.404 INFO:tasks.workunit.client.0.smithi107.stdout:6/984: truncate db/d20/d2c/d3b/d40/d9e/fd0 936374 0 2024-04-22T23:35:14.405 INFO:tasks.workunit.client.1.smithi157.stdout:8/877: dread d2/d4/dbd/dd8/d119/d13/f94 [0,4194304] 0 2024-04-22T23:35:14.405 INFO:tasks.workunit.client.0.smithi107.stdout:2/994: rmdir d1/d2b/d88/dc9 39 2024-04-22T23:35:14.406 INFO:tasks.workunit.client.0.smithi107.stdout:3/866: mknod db/d21/d27/d15/d24/d87/d114/c11d 0 2024-04-22T23:35:14.407 INFO:tasks.workunit.client.1.smithi157.stdout:8/878: creat d2/d4/dbd/dd8/d119/f12d x:0 0 0 2024-04-22T23:35:14.407 INFO:tasks.workunit.client.0.smithi107.stdout:8/931: dread d0/d73/db0/d107/f122 [0,4194304] 0 2024-04-22T23:35:14.407 INFO:tasks.workunit.client.0.smithi107.stdout:6/985: truncate db/d20/d2a/d38/d61/d6f/da2/f11b 76686 0 2024-04-22T23:35:14.409 INFO:tasks.workunit.client.0.smithi107.stdout:2/995: mkdir d1/d12/d35/d102/d7e/da8/d125/d12d/d153 0 2024-04-22T23:35:14.411 INFO:tasks.workunit.client.0.smithi107.stdout:3/867: mknod db/d21/d27/d15/d24/de1/c11e 0 2024-04-22T23:35:14.411 INFO:tasks.workunit.client.0.smithi107.stdout:3/868: read - db/d21/d27/d15/d24/d52/d60/d96/ff1 zero size 2024-04-22T23:35:14.412 INFO:tasks.workunit.client.0.smithi107.stdout:5/949: sync 2024-04-22T23:35:14.412 INFO:tasks.workunit.client.0.smithi107.stdout:4/947: sync 2024-04-22T23:35:14.412 INFO:tasks.workunit.client.0.smithi107.stdout:4/948: chown d1/d21/d29/d2e/d3b/f93 30 1 2024-04-22T23:35:14.413 INFO:tasks.workunit.client.1.smithi157.stdout:8/879: dread d2/d6/d21/d8d/f9c [0,4194304] 0 2024-04-22T23:35:14.413 INFO:tasks.workunit.client.1.smithi157.stdout:8/880: fsync d2/d6/d21/f54 0 2024-04-22T23:35:14.414 INFO:tasks.workunit.client.0.smithi107.stdout:6/986: dread db/d20/d2c/d3b/f60 [4194304,4194304] 0 2024-04-22T23:35:14.414 INFO:tasks.workunit.client.0.smithi107.stdout:8/932: rename d0/d3a/cb4 to d0/d73/db0/d107/d33/d9b/c13a 0 2024-04-22T23:35:14.415 INFO:tasks.workunit.client.0.smithi107.stdout:2/996: mknod d1/d12/d35/d102/dba/def/c154 0 2024-04-22T23:35:14.417 INFO:tasks.workunit.client.0.smithi107.stdout:4/949: symlink d1/d7/d63/d78/l14b 0 2024-04-22T23:35:14.417 INFO:tasks.workunit.client.0.smithi107.stdout:4/950: dread - d1/d7/d63/d78/d80/fb7 zero size 2024-04-22T23:35:14.418 INFO:tasks.workunit.client.0.smithi107.stdout:7/902: sync 2024-04-22T23:35:14.418 INFO:tasks.workunit.client.0.smithi107.stdout:3/869: dread db/d21/d27/d15/d29/d2a/d45/ddd/d25/fe2 [0,4194304] 0 2024-04-22T23:35:14.418 INFO:tasks.workunit.client.0.smithi107.stdout:3/870: write db/d21/db1/dbb/fe8 [754343,51063] 0 2024-04-22T23:35:14.418 INFO:tasks.workunit.client.0.smithi107.stdout:1/869: dwrite d7/d19/d20/d39/d53/d43/d46/d99/fd2 [4194304,4194304] 0 2024-04-22T23:35:14.419 INFO:tasks.workunit.client.0.smithi107.stdout:1/870: getdents d7/d19/d20/d39/d93/d111 0 2024-04-22T23:35:14.419 INFO:tasks.workunit.client.0.smithi107.stdout:8/933: unlink d0/d73/db0/d107/d3b/l43 0 2024-04-22T23:35:14.422 INFO:tasks.workunit.client.0.smithi107.stdout:2/997: rename d1/d12/d35/d54/c71 to d1/d12/d35/d102/d7e/da8/d10e/d11c/c155 0 2024-04-22T23:35:14.423 INFO:tasks.workunit.client.0.smithi107.stdout:4/951: creat d1/f14c x:0 0 0 2024-04-22T23:35:14.426 INFO:tasks.workunit.client.0.smithi107.stdout:3/871: creat db/d21/db1/f11f x:0 0 0 2024-04-22T23:35:14.426 INFO:tasks.workunit.client.0.smithi107.stdout:3/872: chown db/d21/d27/d15/d24/d87 3 1 2024-04-22T23:35:14.426 INFO:tasks.workunit.client.0.smithi107.stdout:3/873: dread - db/fe5 zero size 2024-04-22T23:35:14.429 INFO:tasks.workunit.client.0.smithi107.stdout:8/934: unlink d0/d7/d20/da2/dc3/c10d 0 2024-04-22T23:35:14.429 INFO:tasks.workunit.client.0.smithi107.stdout:8/935: chown d0/d7/d8/d41/d53/d76/c108 167 1 2024-04-22T23:35:14.429 INFO:tasks.workunit.client.0.smithi107.stdout:6/987: read db/d20/d2a/d38/d59/f93 [2601255,69034] 0 2024-04-22T23:35:14.429 INFO:tasks.workunit.client.0.smithi107.stdout:6/988: chown db/d20/d2a/d5d/cee 27069 1 2024-04-22T23:35:14.429 INFO:tasks.workunit.client.0.smithi107.stdout:6/989: readlink db/d20/d2a/d38/d61/d6f/da2/d84/l92 0 2024-04-22T23:35:14.430 INFO:tasks.workunit.client.0.smithi107.stdout:1/871: rename d7/dce/de7 to d7/d19/d20/d39/d93/df4/d11e 0 2024-04-22T23:35:14.430 INFO:tasks.workunit.client.0.smithi107.stdout:2/998: stat d1/d12/d35/d48/d13/l76 0 2024-04-22T23:35:14.431 INFO:tasks.workunit.client.0.smithi107.stdout:5/950: dwrite d9/db/de/d5e/f71 [0,4194304] 0 2024-04-22T23:35:14.432 INFO:tasks.workunit.client.0.smithi107.stdout:5/951: write d9/db/de/d5e/d122/f136 [546930,57763] 0 2024-04-22T23:35:14.432 INFO:tasks.workunit.client.0.smithi107.stdout:4/952: mkdir d1/d21/d29/d2e/d88/d99/d148/d14d 0 2024-04-22T23:35:14.432 INFO:tasks.workunit.client.0.smithi107.stdout:4/953: fsync d1/d21/d29/d2e/d88/d99/dd2/daf/dc1/d11f/ff6 0 2024-04-22T23:35:14.433 INFO:tasks.workunit.client.0.smithi107.stdout:4/954: write d1/d21/d52/f9f [188107,67820] 0 2024-04-22T23:35:14.434 INFO:tasks.workunit.client.0.smithi107.stdout:3/874: unlink db/d21/d27/d15/c72 0 2024-04-22T23:35:14.436 INFO:tasks.workunit.client.0.smithi107.stdout:5/952: unlink d9/db/dc1/dcb/de9/d11b/c144 0 2024-04-22T23:35:14.436 INFO:tasks.workunit.client.0.smithi107.stdout:8/936: creat d0/d7/d20/d2d/f13b x:0 0 0 2024-04-22T23:35:14.438 INFO:tasks.workunit.client.1.smithi157.stdout:8/881: dwrite d2/d4/dbd/dd8/d119/d13/f94 [0,4194304] 0 2024-04-22T23:35:14.440 INFO:tasks.workunit.client.1.smithi157.stdout:8/882: dread d2/d34/f60 [0,4194304] 0 2024-04-22T23:35:14.442 INFO:tasks.workunit.client.1.smithi157.stdout:8/883: creat d2/db1/f12e x:0 0 0 2024-04-22T23:35:14.442 INFO:tasks.workunit.client.1.smithi157.stdout:8/884: chown d2/d34/d6a/d77 81748508 1 2024-04-22T23:35:14.443 INFO:tasks.workunit.client.1.smithi157.stdout:8/885: write d2/db1/d58/fd6 [162278,121518] 0 2024-04-22T23:35:14.445 INFO:tasks.workunit.client.0.smithi107.stdout:1/872: dread d7/d19/d54/f66 [0,4194304] 0 2024-04-22T23:35:14.445 INFO:tasks.workunit.client.0.smithi107.stdout:5/953: write d9/f58 [1505964,32280] 0 2024-04-22T23:35:14.446 INFO:tasks.workunit.client.0.smithi107.stdout:4/955: unlink d1/d21/d29/d2e/d88/d99/dd2/daf/dc1/f116 0 2024-04-22T23:35:14.446 INFO:tasks.workunit.client.0.smithi107.stdout:4/956: chown d1/d21/d29/d2e/d3b/c40 30967008 1 2024-04-22T23:35:14.447 INFO:tasks.workunit.client.1.smithi157.stdout:8/886: link d2/d6/d21/d5f/c12a d2/c12f 0 2024-04-22T23:35:14.447 INFO:tasks.workunit.client.0.smithi107.stdout:6/990: rmdir db/d20/d2a/d5d 39 2024-04-22T23:35:14.450 INFO:tasks.workunit.client.0.smithi107.stdout:1/873: mknod d7/d19/d20/d39/d93/df4/db8/c11f 0 2024-04-22T23:35:14.451 INFO:tasks.workunit.client.1.smithi157.stdout:8/887: rename d2/d34/f60 to d2/d4/dbd/d67/f130 0 2024-04-22T23:35:14.453 INFO:tasks.workunit.client.0.smithi107.stdout:4/957: symlink d1/d21/d52/l14e 0 2024-04-22T23:35:14.454 INFO:tasks.workunit.client.1.smithi157.stdout:8/888: mknod d2/d6/c131 0 2024-04-22T23:35:14.454 INFO:tasks.workunit.client.1.smithi157.stdout:8/889: fdatasync d2/d4/dbd/ff3 0 2024-04-22T23:35:14.455 INFO:tasks.workunit.client.1.smithi157.stdout:8/890: readlink d2/d4/d73/l84 0 2024-04-22T23:35:14.455 INFO:tasks.workunit.client.0.smithi107.stdout:5/954: write d9/db/de/d5e/f61 [720603,71510] 0 2024-04-22T23:35:14.456 INFO:tasks.workunit.client.0.smithi107.stdout:5/955: write d9/db/dc1/dcb/de9/f11c [487940,67104] 0 2024-04-22T23:35:14.457 INFO:tasks.workunit.client.0.smithi107.stdout:1/874: rename d7/d19/d20/d39/d93/df4/fa5 to d7/d19/d20/d39/d93/d111/f120 0 2024-04-22T23:35:14.458 INFO:tasks.workunit.client.0.smithi107.stdout:5/956: write d9/d76/da0/fd0 [543633,87072] 0 2024-04-22T23:35:14.458 INFO:tasks.workunit.client.0.smithi107.stdout:5/957: write d9/d26/d6a/f6f [6132740,61946] 0 2024-04-22T23:35:14.462 INFO:tasks.workunit.client.0.smithi107.stdout:2/999: dwrite d1/d12/d35/d48/f1f [0,4194304] 0 2024-04-22T23:35:14.462 INFO:tasks.workunit.client.0.smithi107.stdout:5/958: mkdir d9/db/de/d72/d9e/d148 0 2024-04-22T23:35:14.464 INFO:tasks.workunit.client.0.smithi107.stdout:5/959: creat d9/db/de/d72/d9e/d148/f149 x:0 0 0 2024-04-22T23:35:14.465 INFO:tasks.workunit.client.0.smithi107.stdout:5/960: write d9/d76/da0/f134 [386704,14314] 0 2024-04-22T23:35:14.465 INFO:tasks.workunit.client.0.smithi107.stdout:5/961: creat d9/db/de/d72/f14a x:0 0 0 2024-04-22T23:35:14.467 INFO:tasks.workunit.client.0.smithi107.stdout:5/962: unlink d9/db/d42/l48 0 2024-04-22T23:35:14.468 INFO:tasks.workunit.client.0.smithi107.stdout:3/875: dwrite db/d21/d27/d15/d24/d87/f89 [0,4194304] 0 2024-04-22T23:35:14.471 INFO:tasks.workunit.client.0.smithi107.stdout:3/876: truncate db/d21/d27/d15/fb4 2717275 0 2024-04-22T23:35:14.476 INFO:tasks.workunit.client.0.smithi107.stdout:3/877: creat db/d21/d27/d15/d24/d66/f120 x:0 0 0 2024-04-22T23:35:14.477 INFO:tasks.workunit.client.0.smithi107.stdout:3/878: mkdir db/d21/d27/d15/d29/d2a/d45/ddd/da7/d121 0 2024-04-22T23:35:14.477 INFO:tasks.workunit.client.0.smithi107.stdout:6/991: dwrite db/d20/f4f [0,4194304] 0 2024-04-22T23:35:14.477 INFO:tasks.workunit.client.0.smithi107.stdout:6/992: write db/d20/d2a/d38/d59/d70/fb7 [317958,75641] 0 2024-04-22T23:35:14.479 INFO:tasks.workunit.client.0.smithi107.stdout:6/993: link db/d20/d2a/d38/d61/d6f/c8d db/d20/d2c/db5/dd2/c12a 0 2024-04-22T23:35:14.482 INFO:tasks.workunit.client.0.smithi107.stdout:6/994: rename db/f15 to db/f12b 0 2024-04-22T23:35:14.483 INFO:tasks.workunit.client.0.smithi107.stdout:6/995: creat db/d20/d2c/de8/d10d/f12c x:0 0 0 2024-04-22T23:35:14.486 INFO:tasks.workunit.client.0.smithi107.stdout:1/875: dwrite d7/d19/d42/f6e [4194304,4194304] 0 2024-04-22T23:35:14.486 INFO:tasks.workunit.client.0.smithi107.stdout:1/876: getdents d7/d19/d20/d39/d53/d43/d63/d91/d110 0 2024-04-22T23:35:14.486 INFO:tasks.workunit.client.0.smithi107.stdout:4/958: dwrite d1/d21/d29/d2e/d88/d99/dd2/df0/ff5 [0,4194304] 0 2024-04-22T23:35:14.487 INFO:tasks.workunit.client.0.smithi107.stdout:5/963: dwrite d9/db/de/d72/f14a [0,4194304] 0 2024-04-22T23:35:14.489 INFO:tasks.workunit.client.0.smithi107.stdout:1/877: creat d7/d19/d20/f121 x:0 0 0 2024-04-22T23:35:14.489 INFO:tasks.workunit.client.0.smithi107.stdout:4/959: dread d1/d7/d63/d78/d80/fe2 [0,4194304] 0 2024-04-22T23:35:14.498 INFO:tasks.workunit.client.0.smithi107.stdout:5/964: truncate d9/d26/f3b 3191144 0 2024-04-22T23:35:14.499 INFO:tasks.workunit.client.0.smithi107.stdout:5/965: mkdir d9/db/dc1/dcb/de9/d14b 0 2024-04-22T23:35:14.500 INFO:tasks.workunit.client.0.smithi107.stdout:5/966: mknod d9/de6/c14c 0 2024-04-22T23:35:14.500 INFO:tasks.workunit.client.0.smithi107.stdout:3/879: dwrite db/d21/db1/f115 [0,4194304] 0 2024-04-22T23:35:14.502 INFO:tasks.workunit.client.0.smithi107.stdout:5/967: link d9/db/l77 d9/db/d42/l14d 0 2024-04-22T23:35:14.515 INFO:tasks.workunit.client.0.smithi107.stdout:4/960: dwrite d1/d21/d29/d2e/d88/d99/dd2/fcf [0,4194304] 0 2024-04-22T23:35:14.517 INFO:tasks.workunit.client.0.smithi107.stdout:4/961: creat d1/d7/d63/d78/d96/da0/de9/d124/f14f x:0 0 0 2024-04-22T23:35:14.517 INFO:tasks.workunit.client.0.smithi107.stdout:6/996: dwrite db/f5e [0,4194304] 0 2024-04-22T23:35:14.518 INFO:tasks.workunit.client.0.smithi107.stdout:6/997: write db/d20/d2a/d38/d61/d6f/d9b/ff8 [1021868,36331] 0 2024-04-22T23:35:14.518 INFO:tasks.workunit.client.0.smithi107.stdout:1/878: dwrite d7/d19/d54/f66 [0,4194304] 0 2024-04-22T23:35:14.518 INFO:tasks.workunit.client.0.smithi107.stdout:1/879: fsync d7/d19/d20/d39/d53/d43/d46/f105 0 2024-04-22T23:35:14.519 INFO:tasks.workunit.client.0.smithi107.stdout:5/968: dwrite d9/d76/da0/fbf [0,4194304] 0 2024-04-22T23:35:14.524 INFO:tasks.workunit.client.0.smithi107.stdout:1/880: mknod d7/d19/d20/d39/d93/df4/c122 0 2024-04-22T23:35:14.527 INFO:tasks.workunit.client.0.smithi107.stdout:6/998: truncate db/d20/d2a/d38/d4b/fe5 1770222 0 2024-04-22T23:35:14.530 INFO:tasks.workunit.client.0.smithi107.stdout:5/969: dread d9/db/d94/f99 [0,4194304] 0 2024-04-22T23:35:14.530 INFO:tasks.workunit.client.0.smithi107.stdout:5/970: stat d9/db/d92/dc2/f121 0 2024-04-22T23:35:14.530 INFO:tasks.workunit.client.0.smithi107.stdout:5/971: fdatasync d9/db/de/d5e/f61 0 2024-04-22T23:35:14.531 INFO:tasks.workunit.client.0.smithi107.stdout:5/972: creat d9/db/de/f14e x:0 0 0 2024-04-22T23:35:14.531 INFO:tasks.workunit.client.0.smithi107.stdout:5/973: dread - d9/db/de/d72/ff8 zero size 2024-04-22T23:35:14.533 INFO:tasks.workunit.client.0.smithi107.stdout:5/974: rename d9/l4c to d9/db/d42/d75/db6/l14f 0 2024-04-22T23:35:14.534 INFO:tasks.workunit.client.0.smithi107.stdout:5/975: mkdir d9/d76/d150 0 2024-04-22T23:35:14.534 INFO:tasks.workunit.client.0.smithi107.stdout:5/976: fdatasync d9/f58 0 2024-04-22T23:35:14.535 INFO:tasks.workunit.client.0.smithi107.stdout:4/962: dwrite d1/d21/d29/d2e/d7e/fef [8388608,4194304] 0 2024-04-22T23:35:14.536 INFO:tasks.workunit.client.0.smithi107.stdout:4/963: creat d1/d7/d5b/de3/f150 x:0 0 0 2024-04-22T23:35:14.536 INFO:tasks.workunit.client.0.smithi107.stdout:4/964: truncate d1/f127 545428 0 2024-04-22T23:35:14.542 INFO:tasks.workunit.client.0.smithi107.stdout:6/999: write db/d20/d2c/d3b/f60 [7729163,125075] 0 2024-04-22T23:35:14.552 INFO:tasks.workunit.client.0.smithi107.stdout:1/881: dwrite d7/d19/d20/d39/d53/d43/d46/dba/fe5 [0,4194304] 0 2024-04-22T23:35:14.553 INFO:tasks.workunit.client.0.smithi107.stdout:1/882: chown d7/d19/d20/d39/d93/df4/l10a 241429 1 2024-04-22T23:35:14.560 INFO:tasks.workunit.client.0.smithi107.stdout:5/977: dwrite f3 [4194304,4194304] 0 2024-04-22T23:35:14.562 INFO:tasks.workunit.client.0.smithi107.stdout:5/978: unlink d9/db/de/f14e 0 2024-04-22T23:35:14.566 INFO:tasks.workunit.client.0.smithi107.stdout:5/979: rename d9/db/de/d72/d9e/f10a to d9/d76/da0/f151 0 2024-04-22T23:35:14.566 INFO:tasks.workunit.client.0.smithi107.stdout:5/980: chown d9/db/de/d5e/f13b 4 1 2024-04-22T23:35:14.566 INFO:tasks.workunit.client.0.smithi107.stdout:1/883: dread d7/d19/d20/d39/d93/d111/f120 [0,4194304] 0 2024-04-22T23:35:14.566 INFO:tasks.workunit.client.0.smithi107.stdout:1/884: mkdir d7/d19/d20/d39/d53/d43/d123 0 2024-04-22T23:35:14.567 INFO:tasks.workunit.client.0.smithi107.stdout:1/885: mkdir d7/d19/d20/d39/d93/d124 0 2024-04-22T23:35:14.568 INFO:tasks.workunit.client.0.smithi107.stdout:1/886: rename d7/d19/d20/d39/d53/d43/d46/dba/dc4 to d7/d19/d20/d39/d93/df4/db8/d125 0 2024-04-22T23:35:14.569 INFO:tasks.workunit.client.0.smithi107.stdout:9/925: sync 2024-04-22T23:35:14.569 INFO:tasks.workunit.client.0.smithi107.stdout:0/877: sync 2024-04-22T23:35:14.569 INFO:tasks.workunit.client.0.smithi107.stdout:0/878: fdatasync d1/d7b/d85/ded/f11c 0 2024-04-22T23:35:14.571 INFO:tasks.workunit.client.0.smithi107.stdout:9/926: getdents d5/d7/d27/dfc 0 2024-04-22T23:35:14.571 INFO:tasks.workunit.client.0.smithi107.stdout:9/927: fsync d5/d14/fa2 0 2024-04-22T23:35:14.571 INFO:tasks.workunit.client.0.smithi107.stdout:9/928: readlink d5/d14/d28/de3/lf2 0 2024-04-22T23:35:14.572 INFO:tasks.workunit.client.0.smithi107.stdout:0/879: unlink d1/d9/d14/d4f/d59/l95 0 2024-04-22T23:35:14.575 INFO:tasks.workunit.client.0.smithi107.stdout:1/887: getdents d7/d19/d20/d39/d53/d43 0 2024-04-22T23:35:14.575 INFO:tasks.workunit.client.0.smithi107.stdout:1/888: fsync d7/d19/d20/d39/d93/d111/f120 0 2024-04-22T23:35:14.575 INFO:tasks.workunit.client.0.smithi107.stdout:1/889: readlink d7/l86 0 2024-04-22T23:35:14.575 INFO:tasks.workunit.client.0.smithi107.stdout:1/890: chown d7/d19/d20/d39/d93/d111/f120 1930620 1 2024-04-22T23:35:14.578 INFO:tasks.workunit.client.0.smithi107.stdout:0/880: mkdir d1/d27/d20/d40/dbd/df2/d11f 0 2024-04-22T23:35:14.579 INFO:tasks.workunit.client.0.smithi107.stdout:5/981: dwrite d9/db/d42/d75/db6/fb9 [0,4194304] 0 2024-04-22T23:35:14.580 INFO:tasks.workunit.client.0.smithi107.stdout:1/891: rename d7/d19/d20/d39/d93/df4/db8 to d7/d19/d20/d39/d53/d43/d63/d126 0 2024-04-22T23:35:14.580 INFO:tasks.workunit.client.0.smithi107.stdout:1/892: readlink d7/d19/d20/d39/d53/d43/ld9 0 2024-04-22T23:35:14.582 INFO:tasks.workunit.client.0.smithi107.stdout:0/881: creat d1/d27/d20/d40/dbd/df2/d11f/f120 x:0 0 0 2024-04-22T23:35:14.583 INFO:tasks.workunit.client.0.smithi107.stdout:1/893: mkdir d7/d19/d20/d39/d53/d43/d63/d126/df8/d127 0 2024-04-22T23:35:14.583 INFO:tasks.workunit.client.0.smithi107.stdout:1/894: write d7/d19/d42/f8f [684463,4844] 0 2024-04-22T23:35:14.583 INFO:tasks.workunit.client.0.smithi107.stdout:1/895: dread - d7/d19/d20/d39/d53/d43/d63/d126/d125/fc8 zero size 2024-04-22T23:35:14.584 INFO:tasks.workunit.client.0.smithi107.stdout:0/882: creat d1/d2a/dad/f121 x:0 0 0 2024-04-22T23:35:14.585 INFO:tasks.workunit.client.0.smithi107.stdout:1/896: mkdir d7/d19/d20/d39/d53/d43/d63/d126/d128 0 2024-04-22T23:35:14.595 INFO:tasks.workunit.client.0.smithi107.stdout:0/883: write d1/d27/d20/f60 [3459341,35172] 0 2024-04-22T23:35:14.595 INFO:tasks.workunit.client.0.smithi107.stdout:0/884: creat d1/d27/d20/d40/dbd/df2/d11f/f122 x:0 0 0 2024-04-22T23:35:14.599 INFO:tasks.workunit.client.0.smithi107.stdout:5/982: dwrite d9/d26/d6a/d100/f132 [0,4194304] 0 2024-04-22T23:35:14.604 INFO:tasks.workunit.client.0.smithi107.stdout:1/897: dwrite d7/fc1 [0,4194304] 0 2024-04-22T23:35:14.605 INFO:tasks.workunit.client.0.smithi107.stdout:1/898: write d7/d19/d42/f64 [892464,105456] 0 2024-04-22T23:35:14.606 INFO:tasks.workunit.client.0.smithi107.stdout:1/899: mknod d7/d19/d20/d39/d53/d43/d63/d126/c129 0 2024-04-22T23:35:14.607 INFO:tasks.workunit.client.0.smithi107.stdout:1/900: creat d7/d19/d20/d39/d93/df4/d11e/f12a x:0 0 0 2024-04-22T23:35:14.608 INFO:tasks.workunit.client.0.smithi107.stdout:5/983: read d9/f12 [2538984,44686] 0 2024-04-22T23:35:14.608 INFO:tasks.workunit.client.0.smithi107.stdout:5/984: fdatasync d9/d26/f3b 0 2024-04-22T23:35:14.608 INFO:tasks.workunit.client.0.smithi107.stdout:1/901: symlink d7/d19/d112/l12b 0 2024-04-22T23:35:14.611 INFO:tasks.workunit.client.0.smithi107.stdout:0/885: dwrite d1/d9/d14/d4f/fa2 [0,4194304] 0 2024-04-22T23:35:14.615 INFO:tasks.workunit.client.0.smithi107.stdout:0/886: creat d1/d9/dd4/f123 x:0 0 0 2024-04-22T23:35:14.616 INFO:tasks.workunit.client.0.smithi107.stdout:1/902: dread d7/d19/d42/f64 [0,4194304] 0 2024-04-22T23:35:14.617 INFO:tasks.workunit.client.0.smithi107.stdout:0/887: link d1/d27/d8a/d8b/fa4 d1/d9/d14/d4f/d59/d114/f124 0 2024-04-22T23:35:14.624 INFO:tasks.workunit.client.0.smithi107.stdout:5/985: dwrite d9/db/dc1/dcb/f145 [0,4194304] 0 2024-04-22T23:35:14.627 INFO:tasks.workunit.client.0.smithi107.stdout:1/903: write d7/d19/d20/d39/d53/d43/ff0 [610044,39615] 0 2024-04-22T23:35:14.627 INFO:tasks.workunit.client.0.smithi107.stdout:1/904: chown d7/d19/d20/d39/d53/d43/d46/d5c/db7/lf9 636 1 2024-04-22T23:35:14.629 INFO:tasks.workunit.client.0.smithi107.stdout:1/905: creat d7/d19/d20/d39/d53/f12c x:0 0 0 2024-04-22T23:35:14.629 INFO:tasks.workunit.client.0.smithi107.stdout:1/906: stat d7/d19/d42/c50 0 2024-04-22T23:35:14.632 INFO:tasks.workunit.client.0.smithi107.stdout:1/907: link d7/fa0 d7/d19/d20/d39/d53/d43/d46/d5c/f12d 0 2024-04-22T23:35:14.632 INFO:tasks.workunit.client.0.smithi107.stdout:7/903: sync 2024-04-22T23:35:14.632 INFO:tasks.workunit.client.0.smithi107.stdout:8/937: sync 2024-04-22T23:35:14.634 INFO:tasks.workunit.client.0.smithi107.stdout:1/908: mkdir d7/d117/d12e 0 2024-04-22T23:35:14.634 INFO:tasks.workunit.client.0.smithi107.stdout:1/909: stat d7/d19/d54/f66 0 2024-04-22T23:35:14.634 INFO:tasks.workunit.client.0.smithi107.stdout:1/910: dread - d7/d19/d20/d39/d53/d43/d46/f105 zero size 2024-04-22T23:35:14.634 INFO:tasks.workunit.client.0.smithi107.stdout:1/911: fsync d7/d19/d42/f8f 0 2024-04-22T23:35:14.635 INFO:tasks.workunit.client.0.smithi107.stdout:8/938: truncate d0/d7/d8/d41/fad 361738 0 2024-04-22T23:35:14.635 INFO:tasks.workunit.client.0.smithi107.stdout:8/939: truncate d0/d7/d20/df9/f11c 578235 0 2024-04-22T23:35:14.635 INFO:tasks.workunit.client.0.smithi107.stdout:8/940: write d0/f125 [1225188,27333] 0 2024-04-22T23:35:14.637 INFO:tasks.workunit.client.0.smithi107.stdout:1/912: rmdir d7/d19/d20/d39/d53/d43/d46/dba 39 2024-04-22T23:35:14.648 INFO:tasks.workunit.client.0.smithi107.stdout:5/986: dwrite d9/d26/f41 [0,4194304] 0 2024-04-22T23:35:14.648 INFO:tasks.workunit.client.0.smithi107.stdout:5/987: creat d9/db/de/d5e/f152 x:0 0 0 2024-04-22T23:35:14.648 INFO:tasks.workunit.client.0.smithi107.stdout:5/988: chown d9/db/d94/fd3 3798147 1 2024-04-22T23:35:14.650 INFO:tasks.workunit.client.0.smithi107.stdout:5/989: mkdir d9/db/d42/d75/db6/d153 0 2024-04-22T23:35:14.651 INFO:tasks.workunit.client.0.smithi107.stdout:8/941: dread d0/f12f [0,4194304] 0 2024-04-22T23:35:14.651 INFO:tasks.workunit.client.0.smithi107.stdout:7/904: dwrite d7/d21/d48/d5c/f118 [0,4194304] 0 2024-04-22T23:35:14.651 INFO:tasks.workunit.client.0.smithi107.stdout:5/990: truncate d9/f31 2718300 0 2024-04-22T23:35:14.651 INFO:tasks.workunit.client.0.smithi107.stdout:8/942: dread d0/d7/d20/df9/f11c [0,4194304] 0 2024-04-22T23:35:14.652 INFO:tasks.workunit.client.0.smithi107.stdout:7/905: creat d7/d21/da4/d81/f12d x:0 0 0 2024-04-22T23:35:14.653 INFO:tasks.workunit.client.0.smithi107.stdout:7/906: creat d7/d21/d48/d5c/d9d/db1/f12e x:0 0 0 2024-04-22T23:35:14.654 INFO:tasks.workunit.client.0.smithi107.stdout:8/943: mkdir d0/d7/d13c 0 2024-04-22T23:35:14.654 INFO:tasks.workunit.client.0.smithi107.stdout:8/944: dread - d0/d7/d20/d2d/fd6 zero size 2024-04-22T23:35:14.654 INFO:tasks.workunit.client.0.smithi107.stdout:8/945: chown d0/d7/d8/d41/f9a 1 1 2024-04-22T23:35:14.654 INFO:tasks.workunit.client.0.smithi107.stdout:8/946: write d0/d7/dfa/f135 [365251,55369] 0 2024-04-22T23:35:14.654 INFO:tasks.workunit.client.0.smithi107.stdout:8/947: getdents d0/d7/d8/de0 0 2024-04-22T23:35:14.654 INFO:tasks.workunit.client.0.smithi107.stdout:5/991: unlink d9/db/de/d5e/f13b 0 2024-04-22T23:35:14.654 INFO:tasks.workunit.client.0.smithi107.stdout:5/992: chown d9/db/dc1/dcb 227127397 1 2024-04-22T23:35:14.655 INFO:tasks.workunit.client.0.smithi107.stdout:8/948: symlink d0/d7/d8/d41/l13d 0 2024-04-22T23:35:14.655 INFO:tasks.workunit.client.0.smithi107.stdout:8/949: chown d0/d73/db0/d107/d33/d9b/d138 518 1 2024-04-22T23:35:14.655 INFO:tasks.workunit.client.0.smithi107.stdout:8/950: mknod d0/d7/d87/c13e 0 2024-04-22T23:35:14.658 INFO:tasks.workunit.client.0.smithi107.stdout:5/993: dread f7 [0,4194304] 0 2024-04-22T23:35:14.658 INFO:tasks.workunit.client.0.smithi107.stdout:5/994: read - d9/db/de/d72/d9e/dee/f126 zero size 2024-04-22T23:35:14.658 INFO:tasks.workunit.client.0.smithi107.stdout:7/907: dread d7/d21/d48/f55 [0,4194304] 0 2024-04-22T23:35:14.659 INFO:tasks.workunit.client.0.smithi107.stdout:1/913: dwrite d7/d19/d20/d39/d93/df4/d11e/f12a [0,4194304] 0 2024-04-22T23:35:14.659 INFO:tasks.workunit.client.0.smithi107.stdout:7/908: truncate d7/d4c/da1/fd9 3142342 0 2024-04-22T23:35:14.659 INFO:tasks.workunit.client.0.smithi107.stdout:7/909: stat d7/d4c/da1/fd0 0 2024-04-22T23:35:14.659 INFO:tasks.workunit.client.0.smithi107.stdout:7/910: readlink d7/d1b/l77 0 2024-04-22T23:35:14.659 INFO:tasks.workunit.client.0.smithi107.stdout:5/995: creat d9/db/d42/d75/db6/d153/f154 x:0 0 0 2024-04-22T23:35:14.660 INFO:tasks.workunit.client.0.smithi107.stdout:5/996: read d9/f37 [157885,46138] 0 2024-04-22T23:35:14.660 INFO:tasks.workunit.client.0.smithi107.stdout:5/997: mkdir d9/db/de/d72/d7c/da4/d155 0 2024-04-22T23:35:14.661 INFO:tasks.workunit.client.0.smithi107.stdout:5/998: unlink d9/f101 0 2024-04-22T23:35:14.661 INFO:tasks.workunit.client.0.smithi107.stdout:5/999: readlink d9/db/d92/ddb/lfe 0 2024-04-22T23:35:14.662 INFO:tasks.workunit.client.0.smithi107.stdout:1/914: creat d7/d19/d20/d39/d93/f12f x:0 0 0 2024-04-22T23:35:14.663 INFO:tasks.workunit.client.0.smithi107.stdout:1/915: write d7/d19/d20/d39/d53/d43/d46/d5c/f8a [1999964,15637] 0 2024-04-22T23:35:14.665 INFO:tasks.workunit.client.0.smithi107.stdout:1/916: rename d7/d19/d20/d39/d93/df4/d11a to d7/d19/d20/d39/d93/df4/d11e/d130 0 2024-04-22T23:35:14.669 INFO:tasks.workunit.client.0.smithi107.stdout:1/917: mknod d7/d19/d20/d39/d93/c131 0 2024-04-22T23:35:14.671 INFO:tasks.workunit.client.0.smithi107.stdout:1/918: getdents d7/d19/d20/d39/d53/d43/d63/d91 0 2024-04-22T23:35:14.671 INFO:tasks.workunit.client.0.smithi107.stdout:1/919: chown d7/d19/d20/d39/d93/df4/l10a 31498 1 2024-04-22T23:35:14.672 INFO:tasks.workunit.client.1.smithi157.stdout:8/891: sync 2024-04-22T23:35:14.672 INFO:tasks.workunit.client.0.smithi107.stdout:1/920: creat d7/dce/f132 x:0 0 0 2024-04-22T23:35:14.674 INFO:tasks.workunit.client.1.smithi157.stdout:8/892: symlink d2/d4/d30/db8/l132 0 2024-04-22T23:35:14.675 INFO:tasks.workunit.client.1.smithi157.stdout:8/893: symlink d2/d4/dbd/d97/ddd/l133 0 2024-04-22T23:35:14.675 INFO:tasks.workunit.client.1.smithi157.stdout:8/894: truncate d2/d4/dbd/ff3 42769 0 2024-04-22T23:35:14.676 INFO:tasks.workunit.client.1.smithi157.stdout:8/895: rename d2/d6/d21/d5f/d17/c53 to d2/d6/d21/d5f/dcd/c134 0 2024-04-22T23:35:14.682 INFO:tasks.workunit.client.0.smithi107.stdout:1/921: dread d7/d19/d20/d39/d53/fe2 [0,4194304] 0 2024-04-22T23:35:14.682 INFO:tasks.workunit.client.0.smithi107.stdout:1/922: fsync d7/d19/d20/d39/d53/d43/ff0 0 2024-04-22T23:35:14.684 INFO:tasks.workunit.client.0.smithi107.stdout:8/951: dwrite d0/d73/db0/d107/d33/f12d [0,4194304] 0 2024-04-22T23:35:14.684 INFO:tasks.workunit.client.0.smithi107.stdout:1/923: mknod d7/d19/d20/d39/d53/d43/d63/d126/df8/c133 0 2024-04-22T23:35:14.687 INFO:tasks.workunit.client.0.smithi107.stdout:8/952: mknod d0/d7/d8/ded/c13f 0 2024-04-22T23:35:14.687 INFO:tasks.workunit.client.0.smithi107.stdout:8/953: truncate d0/f96 4668302 0 2024-04-22T23:35:14.688 INFO:tasks.workunit.client.0.smithi107.stdout:1/924: symlink d7/d19/d20/d39/d53/d43/d46/d119/l134 0 2024-04-22T23:35:14.689 INFO:tasks.workunit.client.1.smithi157.stdout:8/896: dwrite d2/d4/dbd/fb3 [0,4194304] 0 2024-04-22T23:35:14.689 INFO:tasks.workunit.client.1.smithi157.stdout:8/897: chown d2/d4/f5b 97502272 1 2024-04-22T23:35:14.689 INFO:tasks.workunit.client.1.smithi157.stdout:8/898: write d2/d4/dbd/d67/d10f/f116 [184316,66736] 0 2024-04-22T23:35:14.690 INFO:tasks.workunit.client.0.smithi107.stdout:8/954: rmdir d0/d73/db0/d107/d33 39 2024-04-22T23:35:14.691 INFO:tasks.workunit.client.1.smithi157.stdout:8/899: creat d2/d34/d6a/d77/f135 x:0 0 0 2024-04-22T23:35:14.696 INFO:tasks.workunit.client.0.smithi107.stdout:1/925: write d7/d19/d20/d39/d53/d43/d46/d99/fd2 [5554484,63659] 0 2024-04-22T23:35:14.698 INFO:tasks.workunit.client.0.smithi107.stdout:1/926: truncate d7/d19/d20/d39/d53/f33 1093369 0 2024-04-22T23:35:14.701 INFO:tasks.workunit.client.0.smithi107.stdout:1/927: rename d7/d19/d2d/d89/f114 to d7/d19/d20/d39/d93/df4/d11e/f135 0 2024-04-22T23:35:14.701 INFO:tasks.workunit.client.0.smithi107.stdout:1/928: fdatasync d7/d19/d2d/fcf 0 2024-04-22T23:35:14.701 INFO:tasks.workunit.client.0.smithi107.stdout:1/929: write d7/d19/d20/fdc [788120,109826] 0 2024-04-22T23:35:14.701 INFO:tasks.workunit.client.0.smithi107.stdout:1/930: dread - d7/d19/d2d/f9b zero size 2024-04-22T23:35:14.701 INFO:tasks.workunit.client.0.smithi107.stdout:8/955: dread d0/d7/d20/df9/f11e [0,4194304] 0 2024-04-22T23:35:14.702 INFO:tasks.workunit.client.0.smithi107.stdout:8/956: write d0/d7/d20/d2d/f13b [685761,95139] 0 2024-04-22T23:35:14.702 INFO:tasks.workunit.client.0.smithi107.stdout:8/957: write d0/d73/db0/d107/d3b/d7b/fe6 [415616,6154] 0 2024-04-22T23:35:14.702 INFO:tasks.workunit.client.0.smithi107.stdout:8/958: chown d0/d7/l32 37582 1 2024-04-22T23:35:14.702 INFO:tasks.workunit.client.0.smithi107.stdout:8/959: chown d0/d7/d20/d89/f124 241763614 1 2024-04-22T23:35:14.702 INFO:tasks.workunit.client.0.smithi107.stdout:8/960: getdents d0/d3a/d46/d59/d78 0 2024-04-22T23:35:14.702 INFO:tasks.workunit.client.0.smithi107.stdout:8/961: write d0/f125 [1228793,50348] 0 2024-04-22T23:35:14.704 INFO:tasks.workunit.client.0.smithi107.stdout:1/931: rmdir d7/d19/d20/d39/d53/d43/d63/d91/d110 0 2024-04-22T23:35:14.704 INFO:tasks.workunit.client.0.smithi107.stdout:1/932: write d7/fa0 [2473678,83724] 0 2024-04-22T23:35:14.704 INFO:tasks.workunit.client.0.smithi107.stdout:1/933: creat d7/d19/d20/d39/d53/f136 x:0 0 0 2024-04-22T23:35:14.715 INFO:tasks.workunit.client.1.smithi157.stdout:8/900: dwrite d2/f5 [4194304,4194304] 0 2024-04-22T23:35:14.715 INFO:tasks.workunit.client.1.smithi157.stdout:8/901: chown d2/d34/d6a/d77 4380 1 2024-04-22T23:35:14.716 INFO:tasks.workunit.client.1.smithi157.stdout:8/902: creat d2/d6/d21/d8d/f136 x:0 0 0 2024-04-22T23:35:14.716 INFO:tasks.workunit.client.1.smithi157.stdout:8/903: creat d2/d4/dbd/d67/f137 x:0 0 0 2024-04-22T23:35:14.717 INFO:tasks.workunit.client.1.smithi157.stdout:8/904: symlink d2/d6/d21/d5f/d17/l138 0 2024-04-22T23:35:14.718 INFO:tasks.workunit.client.1.smithi157.stdout:8/905: write d2/d4/dbd/dd8/d119/fdc [913224,59273] 0 2024-04-22T23:35:14.718 INFO:tasks.workunit.client.0.smithi107.stdout:4/965: sync 2024-04-22T23:35:14.718 INFO:tasks.workunit.client.0.smithi107.stdout:3/880: sync 2024-04-22T23:35:14.721 INFO:tasks.workunit.client.0.smithi107.stdout:8/962: dwrite d0/d7/d20/da2/fa9 [4194304,4194304] 0 2024-04-22T23:35:14.721 INFO:tasks.workunit.client.0.smithi107.stdout:8/963: write d0/d73/db0/d107/d33/d9b/fae [349400,9014] 0 2024-04-22T23:35:14.721 INFO:tasks.workunit.client.1.smithi157.stdout:8/906: dread d2/d34/d6a/f76 [0,4194304] 0 2024-04-22T23:35:14.722 INFO:tasks.workunit.client.0.smithi107.stdout:4/966: rename d1/d21/d52/l14e to d1/d7/d63/d78/dfc/l151 0 2024-04-22T23:35:14.723 INFO:tasks.workunit.client.0.smithi107.stdout:8/964: dread d0/d73/db0/d107/d33/d9b/fb8 [0,4194304] 0 2024-04-22T23:35:14.723 INFO:tasks.workunit.client.0.smithi107.stdout:4/967: write d1/d7/ff1 [671877,8236] 0 2024-04-22T23:35:14.723 INFO:tasks.workunit.client.1.smithi157.stdout:8/907: creat d2/d6/d21/d5f/d17/dc8/f139 x:0 0 0 2024-04-22T23:35:14.724 INFO:tasks.workunit.client.1.smithi157.stdout:8/908: symlink d2/d34/d6a/d77/l13a 0 2024-04-22T23:35:14.724 INFO:tasks.workunit.client.1.smithi157.stdout:8/909: creat d2/d4/dbd/d67/d10f/f13b x:0 0 0 2024-04-22T23:35:14.724 INFO:tasks.workunit.client.1.smithi157.stdout:8/910: chown d2/d34/d6a/d77/la7 593441 1 2024-04-22T23:35:14.724 INFO:tasks.workunit.client.1.smithi157.stdout:8/911: chown d2/fdb 2600 1 2024-04-22T23:35:14.725 INFO:tasks.workunit.client.1.smithi157.stdout:8/912: fsync d2/d34/d6a/d77/df2/f126 0 2024-04-22T23:35:14.725 INFO:tasks.workunit.client.1.smithi157.stdout:8/913: stat d2/db1/d58/f110 0 2024-04-22T23:35:14.725 INFO:tasks.workunit.client.0.smithi107.stdout:1/934: dwrite d7/d19/d20/d39/d93/f9e [0,4194304] 0 2024-04-22T23:35:14.727 INFO:tasks.workunit.client.0.smithi107.stdout:8/965: getdents d0/d7/d8/d41/dec 0 2024-04-22T23:35:14.730 INFO:tasks.workunit.client.1.smithi157.stdout:8/914: dread d2/d6/d21/d5f/f1b [0,4194304] 0 2024-04-22T23:35:14.732 INFO:tasks.workunit.client.1.smithi157.stdout:8/915: mkdir d2/d4/d73/d13c 0 2024-04-22T23:35:14.732 INFO:tasks.workunit.client.1.smithi157.stdout:8/916: readlink d2/d4/dbd/dd8/d119/d13/l72 0 2024-04-22T23:35:14.742 INFO:tasks.workunit.client.0.smithi107.stdout:1/935: dread d7/d19/d20/d39/d53/d43/d46/d99/fd2 [0,4194304] 0 2024-04-22T23:35:14.749 INFO:tasks.workunit.client.0.smithi107.stdout:4/968: dwrite d1/d21/d29/d2e/f4f [0,4194304] 0 2024-04-22T23:35:14.749 INFO:tasks.workunit.client.0.smithi107.stdout:4/969: readlink d1/d7/d63/d78/d80/lcb 0 2024-04-22T23:35:14.750 INFO:tasks.workunit.client.0.smithi107.stdout:8/966: dwrite d0/d73/db0/d107/d33/f10f [0,4194304] 0 2024-04-22T23:35:14.750 INFO:tasks.workunit.client.0.smithi107.stdout:8/967: chown d0/d73/db0/d107/d33/d37 10846 1 2024-04-22T23:35:14.750 INFO:tasks.workunit.client.0.smithi107.stdout:4/970: stat d1/d7/d63/d78/dfc/l151 0 2024-04-22T23:35:14.751 INFO:tasks.workunit.client.0.smithi107.stdout:4/971: chown d1/d7/d63/d78/d96/da0/de9/c137 12181 1 2024-04-22T23:35:14.751 INFO:tasks.workunit.client.0.smithi107.stdout:3/881: dwrite db/d21/db1/dbb/fe8 [0,4194304] 0 2024-04-22T23:35:14.751 INFO:tasks.workunit.client.0.smithi107.stdout:8/968: dread d0/d73/db0/d107/d33/d9b/fb8 [0,4194304] 0 2024-04-22T23:35:14.756 INFO:tasks.workunit.client.1.smithi157.stdout:8/917: rmdir d2/d4/d73 39 2024-04-22T23:35:14.756 INFO:tasks.workunit.client.1.smithi157.stdout:8/918: stat d2/d6/d21/d5f/d17 0 2024-04-22T23:35:14.757 INFO:tasks.workunit.client.0.smithi107.stdout:4/972: symlink d1/d7/d5b/dbf/d27/d12f/l152 0 2024-04-22T23:35:14.757 INFO:tasks.workunit.client.0.smithi107.stdout:4/973: chown d1/d7/d63/c69 85 1 2024-04-22T23:35:14.757 INFO:tasks.workunit.client.0.smithi107.stdout:4/974: read d1/f127 [267330,92087] 0 2024-04-22T23:35:14.757 INFO:tasks.workunit.client.0.smithi107.stdout:3/882: mkdir db/d21/d27/d15/d24/d87/d114/d122 0 2024-04-22T23:35:14.757 INFO:tasks.workunit.client.1.smithi157.stdout:8/919: symlink d2/d4/d30/db8/l13d 0 2024-04-22T23:35:14.758 INFO:tasks.workunit.client.0.smithi107.stdout:0/888: sync 2024-04-22T23:35:14.758 INFO:tasks.workunit.client.0.smithi107.stdout:9/929: sync 2024-04-22T23:35:14.758 INFO:tasks.workunit.client.0.smithi107.stdout:7/911: sync 2024-04-22T23:35:14.758 INFO:tasks.workunit.client.0.smithi107.stdout:7/912: readlink d7/d21/d48/d5c/d9d/l65 0 2024-04-22T23:35:14.758 INFO:tasks.workunit.client.0.smithi107.stdout:9/930: fsync d5/d7/d27/d46/ff0 0 2024-04-22T23:35:14.758 INFO:tasks.workunit.client.0.smithi107.stdout:9/931: chown d5/d7/d27/dfc/f134 484 1 2024-04-22T23:35:14.759 INFO:tasks.workunit.client.1.smithi157.stdout:8/920: mknod d2/d34/d6a/d77/df2/c13e 0 2024-04-22T23:35:14.759 INFO:tasks.workunit.client.1.smithi157.stdout:8/921: write d2/d4/dbd/f120 [1044688,89188] 0 2024-04-22T23:35:14.759 INFO:tasks.workunit.client.1.smithi157.stdout:8/922: write d2/d4/dbd/d67/f130 [1794162,46438] 0 2024-04-22T23:35:14.760 INFO:tasks.workunit.client.0.smithi107.stdout:1/936: dwrite d7/d19/d20/d39/f4b [0,4194304] 0 2024-04-22T23:35:14.760 INFO:tasks.workunit.client.0.smithi107.stdout:1/937: chown d7/d117/c107 0 1 2024-04-22T23:35:14.761 INFO:tasks.workunit.client.1.smithi157.stdout:8/923: rename d2/d6/d21/d5f/c12a to d2/d4/dda/c13f 0 2024-04-22T23:35:14.762 INFO:tasks.workunit.client.0.smithi107.stdout:0/889: rename d1/d27/l72 to d1/d27/d20/d40/d75/l125 0 2024-04-22T23:35:14.763 INFO:tasks.workunit.client.1.smithi157.stdout:8/924: truncate d2/fdb 229375 0 2024-04-22T23:35:14.765 INFO:tasks.workunit.client.1.smithi157.stdout:8/925: mkdir d2/d4/dbd/dd8/d140 0 2024-04-22T23:35:14.766 INFO:tasks.workunit.client.0.smithi107.stdout:9/932: dread d5/f31 [0,4194304] 0 2024-04-22T23:35:14.766 INFO:tasks.workunit.client.0.smithi107.stdout:9/933: dread - d5/d7/d17/d1b/fc5 zero size 2024-04-22T23:35:14.769 INFO:tasks.workunit.client.1.smithi157.stdout:8/926: truncate d2/d6/d21/d5f/f1b 637777 0 2024-04-22T23:35:14.770 INFO:tasks.workunit.client.0.smithi107.stdout:9/934: truncate d5/d7/d17/d1b/f30 1272274 0 2024-04-22T23:35:14.770 INFO:tasks.workunit.client.0.smithi107.stdout:7/913: write d7/d21/d48/d5c/d9d/db1/fdb [4365546,43479] 0 2024-04-22T23:35:14.770 INFO:tasks.workunit.client.0.smithi107.stdout:7/914: fdatasync d7/d21/da4/d81/fc5 0 2024-04-22T23:35:14.772 INFO:tasks.workunit.client.0.smithi107.stdout:7/915: creat d7/d21/d7e/d94/f12f x:0 0 0 2024-04-22T23:35:14.772 INFO:tasks.workunit.client.0.smithi107.stdout:7/916: write d7/fd [1428246,65723] 0 2024-04-22T23:35:14.773 INFO:tasks.workunit.client.0.smithi107.stdout:9/935: rmdir d5/d7/d27/d69 39 2024-04-22T23:35:14.773 INFO:tasks.workunit.client.0.smithi107.stdout:9/936: read d5/d29/f2c [284007,1935] 0 2024-04-22T23:35:14.775 INFO:tasks.workunit.client.0.smithi107.stdout:9/937: rmdir d5/d29/d7b 39 2024-04-22T23:35:14.776 INFO:tasks.workunit.client.0.smithi107.stdout:7/917: rmdir d7/d21/d48/d5c/d9d/d6e/db2/de0 39 2024-04-22T23:35:14.777 INFO:tasks.workunit.client.0.smithi107.stdout:8/969: dwrite d0/d85/d51/f134 [0,4194304] 0 2024-04-22T23:35:14.779 INFO:tasks.workunit.client.0.smithi107.stdout:9/938: mkdir d5/d7/de9/d13b 0 2024-04-22T23:35:14.784 INFO:tasks.workunit.client.0.smithi107.stdout:9/939: creat d5/d7/f13c x:0 0 0 2024-04-22T23:35:14.784 INFO:tasks.workunit.client.0.smithi107.stdout:1/938: dwrite d7/d19/d42/f6f [0,4194304] 0 2024-04-22T23:35:14.787 INFO:tasks.workunit.client.0.smithi107.stdout:9/940: getdents d5/d29/d7b/d91/dde/dc9 0 2024-04-22T23:35:14.788 INFO:tasks.workunit.client.0.smithi107.stdout:9/941: readlink d5/d29/dbb/le1 0 2024-04-22T23:35:14.788 INFO:tasks.workunit.client.0.smithi107.stdout:9/942: fdatasync d5/d7/d17/f100 0 2024-04-22T23:35:14.788 INFO:tasks.workunit.client.0.smithi107.stdout:9/943: write d5/d14/df7/f11b [1045107,41685] 0 2024-04-22T23:35:14.788 INFO:tasks.workunit.client.0.smithi107.stdout:9/944: write d5/d12a/f12e [319014,67641] 0 2024-04-22T23:35:14.788 INFO:tasks.workunit.client.0.smithi107.stdout:9/945: chown d5/d7/d27/d46/dd3/fc2 46 1 2024-04-22T23:35:14.789 INFO:tasks.workunit.client.0.smithi107.stdout:9/946: link d5/d7/d27/d46/dd3/f12c d5/d7/de9/d13b/f13d 0 2024-04-22T23:35:14.790 INFO:tasks.workunit.client.0.smithi107.stdout:8/970: dread d0/d3a/d46/f8a [0,4194304] 0 2024-04-22T23:35:14.791 INFO:tasks.workunit.client.0.smithi107.stdout:4/975: dwrite d1/d21/d29/d2e/d88/d99/dd2/f122 [0,4194304] 0 2024-04-22T23:35:14.791 INFO:tasks.workunit.client.0.smithi107.stdout:4/976: write d1/d21/d29/d2e/dfa/f139 [1589947,17214] 0 2024-04-22T23:35:14.791 INFO:tasks.workunit.client.0.smithi107.stdout:4/977: getdents d1/d1b 0 2024-04-22T23:35:14.791 INFO:tasks.workunit.client.0.smithi107.stdout:4/978: stat d1/d21/d52 0 2024-04-22T23:35:14.792 INFO:tasks.workunit.client.0.smithi107.stdout:9/947: link d5/d29/d7b/d91/f95 d5/d29/dbb/f13e 0 2024-04-22T23:35:14.792 INFO:tasks.workunit.client.0.smithi107.stdout:9/948: write d5/d29/dbb/f13e [3363575,59419] 0 2024-04-22T23:35:14.800 INFO:tasks.workunit.client.0.smithi107.stdout:3/883: dwrite db/ff [0,4194304] 0 2024-04-22T23:35:14.801 INFO:tasks.workunit.client.0.smithi107.stdout:8/971: mkdir d0/d73/db0/d107/d33/d37/d140 0 2024-04-22T23:35:14.801 INFO:tasks.workunit.client.0.smithi107.stdout:8/972: write d0/d7/d87/dc8/fee [686590,121522] 0 2024-04-22T23:35:14.802 INFO:tasks.workunit.client.0.smithi107.stdout:1/939: dread d7/d19/d20/f52 [0,4194304] 0 2024-04-22T23:35:14.802 INFO:tasks.workunit.client.0.smithi107.stdout:1/940: write d7/d19/d20/f121 [971133,74978] 0 2024-04-22T23:35:14.802 INFO:tasks.workunit.client.0.smithi107.stdout:1/941: chown d7/d19/d20/d39/d93 2114068 1 2024-04-22T23:35:14.803 INFO:tasks.workunit.client.1.smithi157.stdout:8/927: dwrite d2/db1/d58/f110 [0,4194304] 0 2024-04-22T23:35:14.803 INFO:tasks.workunit.client.0.smithi107.stdout:4/979: rename d1/d21/d52/f70 to d1/d21/d29/d2e/d88/d99/dd2/daf/dc1/d11f/f153 0 2024-04-22T23:35:14.806 INFO:tasks.workunit.client.0.smithi107.stdout:3/884: symlink db/d21/d27/d15/d29/d2a/d40/l123 0 2024-04-22T23:35:14.806 INFO:tasks.workunit.client.0.smithi107.stdout:3/885: readlink db/d21/d27/d91/l98 0 2024-04-22T23:35:14.808 INFO:tasks.workunit.client.0.smithi107.stdout:0/890: dwrite d1/d9/d14/d4f/d59/f5c [0,4194304] 0 2024-04-22T23:35:14.809 INFO:tasks.workunit.client.0.smithi107.stdout:8/973: mknod d0/d3a/d46/d59/c141 0 2024-04-22T23:35:14.810 INFO:tasks.workunit.client.0.smithi107.stdout:7/918: dwrite d7/d21/f96 [4194304,4194304] 0 2024-04-22T23:35:14.812 INFO:tasks.workunit.client.0.smithi107.stdout:4/980: mkdir d1/d7/d63/d113/d154 0 2024-04-22T23:35:14.813 INFO:tasks.workunit.client.0.smithi107.stdout:3/886: dread db/d21/d27/d36/fd0 [0,4194304] 0 2024-04-22T23:35:14.813 INFO:tasks.workunit.client.0.smithi107.stdout:1/942: dread d7/d19/d20/d39/d53/d43/d46/fc9 [0,4194304] 0 2024-04-22T23:35:14.818 INFO:tasks.workunit.client.1.smithi157.stdout:8/928: dwrite d2/d4/dda/fef [0,4194304] 0 2024-04-22T23:35:14.819 INFO:tasks.workunit.client.0.smithi107.stdout:8/974: creat d0/d3a/d46/f142 x:0 0 0 2024-04-22T23:35:14.822 INFO:tasks.workunit.client.1.smithi157.stdout:8/929: link d2/d6/d21/d5f/d17/fe4 d2/d4/dbd/d67/d127/f141 0 2024-04-22T23:35:14.824 INFO:tasks.workunit.client.0.smithi107.stdout:7/919: link d7/d21/d48/d5c/d9d/l82 d7/d21/d48/d5c/l130 0 2024-04-22T23:35:14.824 INFO:tasks.workunit.client.0.smithi107.stdout:7/920: creat d7/d21/d7e/ddf/d11e/f131 x:0 0 0 2024-04-22T23:35:14.824 INFO:tasks.workunit.client.0.smithi107.stdout:7/921: dread - d7/d21/d48/d5c/d9d/db1/fec zero size 2024-04-22T23:35:14.824 INFO:tasks.workunit.client.1.smithi157.stdout:8/930: creat d2/d6/d6b/f142 x:0 0 0 2024-04-22T23:35:14.831 INFO:tasks.workunit.client.0.smithi107.stdout:4/981: rmdir d1/d7/d53/d66/d6b 39 2024-04-22T23:35:14.833 INFO:tasks.workunit.client.0.smithi107.stdout:3/887: link db/d21/d27/f39 db/d21/d27/d15/d24/d52/d60/f124 0 2024-04-22T23:35:14.834 INFO:tasks.workunit.client.0.smithi107.stdout:4/982: dread d1/d21/d29/d2e/d88/d99/fb5 [4194304,4194304] 0 2024-04-22T23:35:14.837 INFO:tasks.workunit.client.1.smithi157.stdout:8/931: write d2/db1/d58/f110 [2332121,52892] 0 2024-04-22T23:35:14.837 INFO:tasks.workunit.client.1.smithi157.stdout:8/932: creat d2/d4/dad/f143 x:0 0 0 2024-04-22T23:35:14.837 INFO:tasks.workunit.client.1.smithi157.stdout:8/933: write d2/d4/d73/fe9 [1037801,123212] 0 2024-04-22T23:35:14.837 INFO:tasks.workunit.client.0.smithi107.stdout:0/891: rename d1/d27 to d1/d126 0 2024-04-22T23:35:14.839 INFO:tasks.workunit.client.0.smithi107.stdout:0/892: link d1/d126/d20/c78 d1/d126/d20/d40/d75/c127 0 2024-04-22T23:35:14.839 INFO:tasks.workunit.client.1.smithi157.stdout:8/934: unlink d2/d4/f1c 0 2024-04-22T23:35:14.840 INFO:tasks.workunit.client.0.smithi107.stdout:3/888: symlink db/d21/d27/d36/l125 0 2024-04-22T23:35:14.840 INFO:tasks.workunit.client.0.smithi107.stdout:3/889: readlink db/d21/d27/d15/ld5 0 2024-04-22T23:35:14.840 INFO:tasks.workunit.client.0.smithi107.stdout:3/890: truncate db/d21/d27/d15/d24/d52/d60/d96/ff1 350362 0 2024-04-22T23:35:14.841 INFO:tasks.workunit.client.1.smithi157.stdout:8/935: rename d2/d4/dbd/dd8/d119/d13/f94 to d2/db1/d58/f144 0 2024-04-22T23:35:14.844 INFO:tasks.workunit.client.1.smithi157.stdout:8/936: rename d2/d4/dbd/dd8/d119/f27 to d2/d6/f145 0 2024-04-22T23:35:14.844 INFO:tasks.workunit.client.1.smithi157.stdout:8/937: write d2/db1/d58/f110 [5177129,8359] 0 2024-04-22T23:35:14.844 INFO:tasks.workunit.client.1.smithi157.stdout:8/938: stat d2/d6/d21/d5f/d17/dc8 0 2024-04-22T23:35:14.846 INFO:tasks.workunit.client.1.smithi157.stdout:8/939: rename d2/f5 to d2/d4/d73/f146 0 2024-04-22T23:35:14.846 INFO:tasks.workunit.client.1.smithi157.stdout:8/940: chown d2/d4/d30/db8/l13d 104 1 2024-04-22T23:35:14.847 INFO:tasks.workunit.client.0.smithi107.stdout:1/943: dwrite d7/d19/d20/d39/d53/fe2 [0,4194304] 0 2024-04-22T23:35:14.847 INFO:tasks.workunit.client.0.smithi107.stdout:8/975: dwrite d0/f72 [0,4194304] 0 2024-04-22T23:35:14.847 INFO:tasks.workunit.client.0.smithi107.stdout:8/976: write d0/d3a/d46/f142 [701357,88307] 0 2024-04-22T23:35:14.847 INFO:tasks.workunit.client.0.smithi107.stdout:8/977: stat d0/d73/db0/d107/d33/d37/d140 0 2024-04-22T23:35:14.847 INFO:tasks.workunit.client.0.smithi107.stdout:8/978: readlink d0/d73/ld3 0 2024-04-22T23:35:14.848 INFO:tasks.workunit.client.0.smithi107.stdout:7/922: dwrite d7/db4/d112/f11a [0,4194304] 0 2024-04-22T23:35:14.848 INFO:tasks.workunit.client.0.smithi107.stdout:1/944: symlink d7/dce/l137 0 2024-04-22T23:35:14.848 INFO:tasks.workunit.client.0.smithi107.stdout:1/945: dread - d7/d19/d20/d39/d53/d43/d63/d126/d125/fc8 zero size 2024-04-22T23:35:14.848 INFO:tasks.workunit.client.0.smithi107.stdout:1/946: chown d7/l34 8187110 1 2024-04-22T23:35:14.852 INFO:tasks.workunit.client.0.smithi107.stdout:0/893: read d1/d9/d14/d4f/da6/faa [1497537,117260] 0 2024-04-22T23:35:14.852 INFO:tasks.workunit.client.0.smithi107.stdout:8/979: dread d0/d7/d87/dc8/fee [0,4194304] 0 2024-04-22T23:35:14.854 INFO:tasks.workunit.client.0.smithi107.stdout:4/983: dwrite d1/d7/d53/d66/d6b/fdf [0,4194304] 0 2024-04-22T23:35:14.854 INFO:tasks.workunit.client.0.smithi107.stdout:4/984: write d1/d21/d29/d2e/d88/d99/dd2/daf/dc1/d11f/fbb [759757,43536] 0 2024-04-22T23:35:14.855 INFO:tasks.workunit.client.0.smithi107.stdout:4/985: dread - d1/d7/d63/d78/d80/fff zero size 2024-04-22T23:35:14.856 INFO:tasks.workunit.client.0.smithi107.stdout:7/923: symlink d7/d21/d7e/deb/l132 0 2024-04-22T23:35:14.858 INFO:tasks.workunit.client.0.smithi107.stdout:4/986: dread d1/d21/d29/d2e/f46 [0,4194304] 0 2024-04-22T23:35:14.859 INFO:tasks.workunit.client.0.smithi107.stdout:7/924: creat d7/db4/f133 x:0 0 0 2024-04-22T23:35:14.859 INFO:tasks.workunit.client.0.smithi107.stdout:7/925: readlink d7/d21/d7e/la3 0 2024-04-22T23:35:14.860 INFO:tasks.workunit.client.0.smithi107.stdout:1/947: dread d7/d19/d20/d39/d53/d43/f49 [0,4194304] 0 2024-04-22T23:35:14.860 INFO:tasks.workunit.client.0.smithi107.stdout:1/948: write d7/d19/d20/d39/d53/f5f [3039428,87633] 0 2024-04-22T23:35:14.860 INFO:tasks.workunit.client.0.smithi107.stdout:7/926: mkdir d7/d21/d7e/ddf/d11e/d134 0 2024-04-22T23:35:14.865 INFO:tasks.workunit.client.0.smithi107.stdout:1/949: getdents d7/d19/d42/ded 0 2024-04-22T23:35:14.868 INFO:tasks.workunit.client.0.smithi107.stdout:7/927: rmdir d7/d21/d7e/d94/d123 0 2024-04-22T23:35:14.868 INFO:tasks.workunit.client.0.smithi107.stdout:7/928: chown d7/d21/d54/dcc/cf1 23204 1 2024-04-22T23:35:14.870 INFO:tasks.workunit.client.0.smithi107.stdout:8/980: write d0/d7/d8/d41/d53/d76/f114 [200628,10032] 0 2024-04-22T23:35:14.870 INFO:tasks.workunit.client.0.smithi107.stdout:1/950: getdents d7/d19/d20/d39/d53 0 2024-04-22T23:35:14.873 INFO:tasks.workunit.client.1.smithi157.stdout:8/941: dwrite d2/d4/dbd/ff3 [0,4194304] 0 2024-04-22T23:35:14.873 INFO:tasks.workunit.client.1.smithi157.stdout:8/942: chown d2/d4/dbd/dd8/d119/f4a 23 1 2024-04-22T23:35:14.874 INFO:tasks.workunit.client.0.smithi107.stdout:8/981: mkdir d0/d7/d8/ded/d143 0 2024-04-22T23:35:14.874 INFO:tasks.workunit.client.0.smithi107.stdout:8/982: truncate d0/d73/db0/d107/d33/d37/d9f/feb 56269 0 2024-04-22T23:35:14.874 INFO:tasks.workunit.client.0.smithi107.stdout:0/894: dwrite d1/d2a/f77 [0,4194304] 0 2024-04-22T23:35:14.875 INFO:tasks.workunit.client.0.smithi107.stdout:8/983: symlink d0/d7/l144 0 2024-04-22T23:35:14.875 INFO:tasks.workunit.client.0.smithi107.stdout:8/984: fsync d0/d7/d87/dc8/fee 0 2024-04-22T23:35:14.877 INFO:tasks.workunit.client.0.smithi107.stdout:0/895: creat d1/d126/d20/d40/dbd/f128 x:0 0 0 2024-04-22T23:35:14.882 INFO:tasks.workunit.client.0.smithi107.stdout:0/896: link d1/d126/d20/d2c/d41/fcd d1/d2a/dad/f129 0 2024-04-22T23:35:14.887 INFO:tasks.workunit.client.0.smithi107.stdout:7/929: dread d7/d1b/f38 [0,4194304] 0 2024-04-22T23:35:14.889 INFO:tasks.workunit.client.0.smithi107.stdout:7/930: link d7/d21/d7e/la3 d7/d21/d48/d5c/d9d/d6e/db2/l135 0 2024-04-22T23:35:14.891 INFO:tasks.workunit.client.0.smithi107.stdout:7/931: creat d7/d21/d48/d126/f136 x:0 0 0 2024-04-22T23:35:14.892 INFO:tasks.workunit.client.0.smithi107.stdout:7/932: truncate d7/d1b/f38 2467077 0 2024-04-22T23:35:14.892 INFO:tasks.workunit.client.0.smithi107.stdout:7/933: fdatasync d7/d4c/da1/fd9 0 2024-04-22T23:35:14.893 INFO:tasks.workunit.client.0.smithi107.stdout:0/897: dread d1/d9/d14/f8c [0,4194304] 0 2024-04-22T23:35:14.894 INFO:tasks.workunit.client.0.smithi107.stdout:0/898: symlink d1/d2a/dad/l12a 0 2024-04-22T23:35:14.894 INFO:tasks.workunit.client.0.smithi107.stdout:1/951: dwrite d7/d19/d20/f58 [0,4194304] 0 2024-04-22T23:35:14.894 INFO:tasks.workunit.client.0.smithi107.stdout:1/952: truncate d7/d19/d42/f64 1510682 0 2024-04-22T23:35:14.895 INFO:tasks.workunit.client.0.smithi107.stdout:7/934: symlink d7/d21/d48/l137 0 2024-04-22T23:35:14.895 INFO:tasks.workunit.client.0.smithi107.stdout:7/935: read - d7/d4c/da1/fad zero size 2024-04-22T23:35:14.896 INFO:tasks.workunit.client.0.smithi107.stdout:1/953: link d7/d19/d20/d39/d93/df4/fc2 d7/d19/d112/f138 0 2024-04-22T23:35:14.897 INFO:tasks.workunit.client.0.smithi107.stdout:7/936: unlink d7/d21/d49/c4a 0 2024-04-22T23:35:14.898 INFO:tasks.workunit.client.0.smithi107.stdout:7/937: truncate d7/d21/d48/d5c/d9d/db1/f10b 223691 0 2024-04-22T23:35:14.901 INFO:tasks.workunit.client.1.smithi157.stdout:8/943: dwrite d2/db1/f102 [0,4194304] 0 2024-04-22T23:35:14.901 INFO:tasks.workunit.client.0.smithi107.stdout:8/985: dwrite d0/d7/d8/d41/dec/f2a [0,4194304] 0 2024-04-22T23:35:14.901 INFO:tasks.workunit.client.0.smithi107.stdout:7/938: link d7/d21/d48/d5c/c8c d7/d21/d7e/ddf/d121/c138 0 2024-04-22T23:35:14.903 INFO:tasks.workunit.client.0.smithi107.stdout:8/986: symlink d0/d7/d8/l145 0 2024-04-22T23:35:14.904 INFO:tasks.workunit.client.1.smithi157.stdout:8/944: unlink d2/d6/d21/d5f/d17/fe4 0 2024-04-22T23:35:14.904 INFO:tasks.workunit.client.0.smithi107.stdout:7/939: mkdir d7/d21/d7e/d139 0 2024-04-22T23:35:14.906 INFO:tasks.workunit.client.1.smithi157.stdout:8/945: rmdir d2/db1/d58 39 2024-04-22T23:35:14.906 INFO:tasks.workunit.client.0.smithi107.stdout:7/940: creat d7/d21/d48/d95/f13a x:0 0 0 2024-04-22T23:35:14.907 INFO:tasks.workunit.client.1.smithi157.stdout:8/946: rename d2/d4/d30/l74 to d2/d6/d21/d5f/dcd/l147 0 2024-04-22T23:35:14.908 INFO:tasks.workunit.client.0.smithi107.stdout:7/941: symlink d7/d21/d7e/deb/l13b 0 2024-04-22T23:35:14.909 INFO:tasks.workunit.client.1.smithi157.stdout:8/947: mknod d2/d6/d21/d5f/c148 0 2024-04-22T23:35:14.910 INFO:tasks.workunit.client.0.smithi107.stdout:7/942: symlink d7/d21/d49/l13c 0 2024-04-22T23:35:14.910 INFO:tasks.workunit.client.1.smithi157.stdout:8/948: rename d2/db1/d58/l8f to d2/d6/d21/d5f/d17/dc2/l149 0 2024-04-22T23:35:14.911 INFO:tasks.workunit.client.0.smithi107.stdout:7/943: fsync d7/d4c/fe7 0 2024-04-22T23:35:14.911 INFO:tasks.workunit.client.0.smithi107.stdout:7/944: truncate d7/d21/d48/daf/fe1 731427 0 2024-04-22T23:35:14.911 INFO:tasks.workunit.client.0.smithi107.stdout:0/899: dwrite d1/d9/dd4/f123 [0,4194304] 0 2024-04-22T23:35:14.911 INFO:tasks.workunit.client.0.smithi107.stdout:0/900: chown d1/d126/d20/f60 44271083 1 2024-04-22T23:35:14.912 INFO:tasks.workunit.client.1.smithi157.stdout:8/949: creat d2/d4/dbd/d67/d127/f14a x:0 0 0 2024-04-22T23:35:14.912 INFO:tasks.workunit.client.0.smithi107.stdout:0/901: dread d1/d126/d20/fda [0,4194304] 0 2024-04-22T23:35:14.913 INFO:tasks.workunit.client.0.smithi107.stdout:0/902: chown d1/d9/c32 2391 1 2024-04-22T23:35:14.913 INFO:tasks.workunit.client.0.smithi107.stdout:0/903: fdatasync d1/d7b/f10b 0 2024-04-22T23:35:14.913 INFO:tasks.workunit.client.0.smithi107.stdout:0/904: dread - d1/d126/d20/d40/dbd/df2/d11f/f122 zero size 2024-04-22T23:35:14.913 INFO:tasks.workunit.client.1.smithi157.stdout:8/950: creat d2/d4/dbd/d97/ddd/f14b x:0 0 0 2024-04-22T23:35:14.917 INFO:tasks.workunit.client.0.smithi107.stdout:0/905: creat d1/d126/d20/f12b x:0 0 0 2024-04-22T23:35:14.918 INFO:tasks.workunit.client.0.smithi107.stdout:0/906: write d1/d126/d20/d4a/fe9 [1021474,40631] 0 2024-04-22T23:35:14.920 INFO:tasks.workunit.client.0.smithi107.stdout:0/907: unlink d1/d9/ff6 0 2024-04-22T23:35:14.922 INFO:tasks.workunit.client.0.smithi107.stdout:0/908: rename d1/l56 to d1/d9/d14/d4f/da6/l12c 0 2024-04-22T23:35:14.923 INFO:tasks.workunit.client.0.smithi107.stdout:1/954: dwrite d7/d19/d20/d39/f3c [0,4194304] 0 2024-04-22T23:35:14.923 INFO:tasks.workunit.client.0.smithi107.stdout:1/955: dread - d7/d19/d2d/f9b zero size 2024-04-22T23:35:14.924 INFO:tasks.workunit.client.0.smithi107.stdout:7/945: rmdir d7/d21/d48/d126 39 2024-04-22T23:35:14.924 INFO:tasks.workunit.client.0.smithi107.stdout:7/946: chown d7/d21/d48/daf 477 1 2024-04-22T23:35:14.925 INFO:tasks.workunit.client.1.smithi157.stdout:8/951: dread d2/d6/d21/d5f/f6f [0,4194304] 0 2024-04-22T23:35:14.925 INFO:tasks.workunit.client.1.smithi157.stdout:8/952: fdatasync d2/d4/dbd/dd8/d119/d13/f7c 0 2024-04-22T23:35:14.925 INFO:tasks.workunit.client.1.smithi157.stdout:8/953: creat d2/d4/dbd/f14c x:0 0 0 2024-04-22T23:35:14.925 INFO:tasks.workunit.client.1.smithi157.stdout:8/954: truncate d2/d6/f12b 109116 0 2024-04-22T23:35:14.925 INFO:tasks.workunit.client.0.smithi107.stdout:0/909: creat d1/d126/d20/d2c/d2e/f12d x:0 0 0 2024-04-22T23:35:14.926 INFO:tasks.workunit.client.0.smithi107.stdout:8/987: dwrite d0/d73/f115 [0,4194304] 0 2024-04-22T23:35:14.926 INFO:tasks.workunit.client.0.smithi107.stdout:8/988: read - d0/d7/d87/f10e zero size 2024-04-22T23:35:14.928 INFO:tasks.workunit.client.1.smithi157.stdout:8/955: rename d2/d4/dbd/dd8/d119/d13/f26 to d2/d4/dad/f14d 0 2024-04-22T23:35:14.930 INFO:tasks.workunit.client.1.smithi157.stdout:8/956: creat d2/db1/dfb/f14e x:0 0 0 2024-04-22T23:35:14.931 INFO:tasks.workunit.client.1.smithi157.stdout:8/957: rmdir d2/d4/d73 39 2024-04-22T23:35:14.932 INFO:tasks.workunit.client.1.smithi157.stdout:8/958: truncate d2/d6/f46 5021163 0 2024-04-22T23:35:14.933 INFO:tasks.workunit.client.1.smithi157.stdout:8/959: symlink d2/d4/dbd/d67/d10f/l14f 0 2024-04-22T23:35:14.934 INFO:tasks.workunit.client.1.smithi157.stdout:8/960: unlink d2/d6/d21/d5f/d17/dc2/l149 0 2024-04-22T23:35:14.936 INFO:tasks.workunit.client.0.smithi107.stdout:1/956: truncate d7/f67 2363763 0 2024-04-22T23:35:14.936 INFO:tasks.workunit.client.0.smithi107.stdout:1/957: creat d7/d19/d2d/d89/f139 x:0 0 0 2024-04-22T23:35:14.937 INFO:tasks.workunit.client.0.smithi107.stdout:7/947: truncate d7/fd 2227481 0 2024-04-22T23:35:14.940 INFO:tasks.workunit.client.0.smithi107.stdout:0/910: creat d1/d9/d14/d4f/d59/d114/f12e x:0 0 0 2024-04-22T23:35:14.942 INFO:tasks.workunit.client.0.smithi107.stdout:8/989: creat d0/d7/d8/d41/dec/f146 x:0 0 0 2024-04-22T23:35:14.945 INFO:tasks.workunit.client.0.smithi107.stdout:1/958: link d7/d19/d20/d39/d53/cd8 d7/d19/d20/d39/d53/d43/d46/d5c/c13a 0 2024-04-22T23:35:14.948 INFO:tasks.workunit.client.0.smithi107.stdout:8/990: unlink d0/d73/db0/d107/f5c 0 2024-04-22T23:35:14.950 INFO:tasks.workunit.client.0.smithi107.stdout:1/959: link d7/d19/d20/d39/f7e d7/d19/d20/d39/d53/d43/f13b 0 2024-04-22T23:35:14.950 INFO:tasks.workunit.client.0.smithi107.stdout:7/948: getdents d7/d21/da4/d81 0 2024-04-22T23:35:14.952 INFO:tasks.workunit.client.0.smithi107.stdout:8/991: dread d0/d73/db0/d107/d3b/d7b/fcb [0,4194304] 0 2024-04-22T23:35:14.953 INFO:tasks.workunit.client.0.smithi107.stdout:1/960: link d7/d19/d20/d39/d53/d43/fab d7/d19/d20/d39/d93/df4/d11e/f13c 0 2024-04-22T23:35:14.953 INFO:tasks.workunit.client.0.smithi107.stdout:1/961: truncate d7/d19/d20/d39/d93/f12f 800712 0 2024-04-22T23:35:14.953 INFO:tasks.workunit.client.0.smithi107.stdout:1/962: fsync d7/d19/d20/d39/d93/df4/d11e/f12a 0 2024-04-22T23:35:14.954 INFO:tasks.workunit.client.0.smithi107.stdout:7/949: mkdir d7/d21/d7e/d94/d117/d13d 0 2024-04-22T23:35:14.954 INFO:tasks.workunit.client.0.smithi107.stdout:7/950: fsync d7/d1b/f4d 0 2024-04-22T23:35:14.955 INFO:tasks.workunit.client.0.smithi107.stdout:1/963: creat d7/d19/d20/d39/d53/d43/d46/d99/f13d x:0 0 0 2024-04-22T23:35:14.956 INFO:tasks.workunit.client.0.smithi107.stdout:1/964: write d7/d19/d20/fdc [1470679,59149] 0 2024-04-22T23:35:14.956 INFO:tasks.workunit.client.0.smithi107.stdout:1/965: write d7/fa0 [5085555,71337] 0 2024-04-22T23:35:14.956 INFO:tasks.workunit.client.0.smithi107.stdout:7/951: mknod d7/d21/d7e/deb/c13e 0 2024-04-22T23:35:14.957 INFO:tasks.workunit.client.0.smithi107.stdout:0/911: dwrite d1/d2a/dad/f121 [0,4194304] 0 2024-04-22T23:35:14.957 INFO:tasks.workunit.client.1.smithi157.stdout:8/961: dwrite d2/d4/dbd/d97/fea [0,4194304] 0 2024-04-22T23:35:14.957 INFO:tasks.workunit.client.1.smithi157.stdout:8/962: dread - d2/d6/d6b/fe3 zero size 2024-04-22T23:35:14.957 INFO:tasks.workunit.client.0.smithi107.stdout:1/966: link d7/d19/d2d/da4/f10c d7/d19/d54/f13e 0 2024-04-22T23:35:14.959 INFO:tasks.workunit.client.0.smithi107.stdout:7/952: mkdir d7/d21/d48/d5c/d13f 0 2024-04-22T23:35:14.962 INFO:tasks.workunit.client.1.smithi157.stdout:8/963: rmdir d2/db1/dfb 39 2024-04-22T23:35:14.962 INFO:tasks.workunit.client.1.smithi157.stdout:8/964: chown d2/d6/d21/d5f/d17/f49 14204 1 2024-04-22T23:35:14.963 INFO:tasks.workunit.client.0.smithi107.stdout:1/967: mknod d7/d19/d20/d39/d53/c13f 0 2024-04-22T23:35:14.973 INFO:tasks.workunit.client.0.smithi107.stdout:9/949: sync 2024-04-22T23:35:14.974 INFO:tasks.workunit.client.0.smithi107.stdout:9/950: mknod d5/d7/d27/d46/d4b/d138/c13f 0 2024-04-22T23:35:14.975 INFO:tasks.workunit.client.0.smithi107.stdout:9/951: creat d5/d7/d17/d1b/d1f/d36/d96/d133/f140 x:0 0 0 2024-04-22T23:35:14.975 INFO:tasks.workunit.client.0.smithi107.stdout:9/952: readlink d5/d24/lcd 0 2024-04-22T23:35:14.979 INFO:tasks.workunit.client.0.smithi107.stdout:0/912: dwrite d1/d126/d20/f36 [8388608,4194304] 0 2024-04-22T23:35:14.981 INFO:tasks.workunit.client.0.smithi107.stdout:0/913: mknod d1/d126/d8a/db5/d10f/c12f 0 2024-04-22T23:35:14.990 INFO:tasks.workunit.client.0.smithi107.stdout:0/914: write d1/d2a/f77 [3127509,110147] 0 2024-04-22T23:35:14.991 INFO:tasks.workunit.client.0.smithi107.stdout:1/968: dwrite d7/d19/d20/d39/d53/d43/d63/d126/f113 [0,4194304] 0 2024-04-22T23:35:14.991 INFO:tasks.workunit.client.0.smithi107.stdout:1/969: write d7/d19/f80 [1748710,128805] 0 2024-04-22T23:35:14.991 INFO:tasks.workunit.client.0.smithi107.stdout:1/970: getdents d7/d19/d42/ded 0 2024-04-22T23:35:14.991 INFO:tasks.workunit.client.0.smithi107.stdout:1/971: creat d7/d19/d54/f140 x:0 0 0 2024-04-22T23:35:14.991 INFO:tasks.workunit.client.0.smithi107.stdout:9/953: dwrite d5/d14/d28/f10c [0,4194304] 0 2024-04-22T23:35:14.992 INFO:tasks.workunit.client.0.smithi107.stdout:0/915: mknod d1/d9/d14/d4f/d59/d114/c130 0 2024-04-22T23:35:14.992 INFO:tasks.workunit.client.1.smithi157.stdout:8/965: truncate d2/d6/f46 1978995 0 2024-04-22T23:35:14.995 INFO:tasks.workunit.client.0.smithi107.stdout:9/954: mknod d5/d7/d27/d46/d4b/d138/c141 0 2024-04-22T23:35:14.997 INFO:tasks.workunit.client.0.smithi107.stdout:1/972: dread d7/d19/d54/f13e [0,4194304] 0 2024-04-22T23:35:14.998 INFO:tasks.workunit.client.0.smithi107.stdout:9/955: mknod d5/d29/dbb/c142 0 2024-04-22T23:35:15.001 INFO:tasks.workunit.client.0.smithi107.stdout:0/916: rmdir d1/d9/d14 39 2024-04-22T23:35:15.001 INFO:tasks.workunit.client.0.smithi107.stdout:1/973: truncate d7/d19/d2d/f31 3141933 0 2024-04-22T23:35:15.003 INFO:tasks.workunit.client.0.smithi107.stdout:9/956: rename d5/d29/d7b/f115 to d5/d14/d2e/d4f/f143 0 2024-04-22T23:35:15.003 INFO:tasks.workunit.client.0.smithi107.stdout:9/957: truncate d5/d7/de9/d13b/f13d 1397342 0 2024-04-22T23:35:15.004 INFO:tasks.workunit.client.0.smithi107.stdout:0/917: mkdir d1/d126/d20/d40/dbd/df2/d131 0 2024-04-22T23:35:15.010 INFO:tasks.workunit.client.0.smithi107.stdout:1/974: write d7/d19/d2d/fee [657326,65416] 0 2024-04-22T23:35:15.010 INFO:tasks.workunit.client.0.smithi107.stdout:1/975: fsync d7/d19/d20/d39/d53/d43/d46/d5c/db7/f11d 0 2024-04-22T23:35:15.012 INFO:tasks.workunit.client.0.smithi107.stdout:1/976: unlink d7/d19/d20/d39/d53/d43/d63/l10b 0 2024-04-22T23:35:15.013 INFO:tasks.workunit.client.0.smithi107.stdout:1/977: mknod d7/d19/d20/d39/d53/d43/d63/d126/d128/c141 0 2024-04-22T23:35:15.042 INFO:tasks.workunit.client.0.smithi107.stdout:1/978: dwrite d7/d19/d42/f6f [4194304,4194304] 0 2024-04-22T23:35:15.042 INFO:tasks.workunit.client.0.smithi107.stdout:1/979: fdatasync d7/d19/f95 0 2024-04-22T23:35:15.044 INFO:tasks.workunit.client.0.smithi107.stdout:1/980: creat d7/d19/d20/d39/d53/d43/d46/d5c/dac/f142 x:0 0 0 2024-04-22T23:35:15.044 INFO:tasks.workunit.client.0.smithi107.stdout:9/958: dwrite d5/fc [0,4194304] 0 2024-04-22T23:35:15.044 INFO:tasks.workunit.client.0.smithi107.stdout:1/981: chown d7/fa0 4653 1 2024-04-22T23:35:15.044 INFO:tasks.workunit.client.0.smithi107.stdout:1/982: write d7/d19/d20/d39/d93/df4/f96 [4343477,120835] 0 2024-04-22T23:35:15.044 INFO:tasks.workunit.client.0.smithi107.stdout:1/983: creat d7/d19/d20/d39/d53/d43/d46/d5c/dac/f143 x:0 0 0 2024-04-22T23:35:15.044 INFO:tasks.workunit.client.0.smithi107.stdout:0/918: dwrite d1/d9/d14/d4f/d59/f5c [0,4194304] 0 2024-04-22T23:35:15.047 INFO:tasks.workunit.client.0.smithi107.stdout:1/984: mkdir d7/d19/d42/d144 0 2024-04-22T23:35:15.047 INFO:tasks.workunit.client.0.smithi107.stdout:1/985: fsync d7/d19/d20/d39/d53/fe2 0 2024-04-22T23:35:15.048 INFO:tasks.workunit.client.0.smithi107.stdout:0/919: rmdir d1/d126/d20/d40/dbd/df2 39 2024-04-22T23:35:15.048 INFO:tasks.workunit.client.0.smithi107.stdout:0/920: fdatasync d1/d9/d14/f4d 0 2024-04-22T23:35:15.053 INFO:tasks.workunit.client.0.smithi107.stdout:0/921: rename d1/lb2 to d1/d126/d20/d40/dbd/df2/d131/l132 0 2024-04-22T23:35:15.054 INFO:tasks.workunit.client.0.smithi107.stdout:0/922: mknod d1/d126/d20/d4a/c133 0 2024-04-22T23:35:15.055 INFO:tasks.workunit.client.0.smithi107.stdout:9/959: write f4 [2346272,60879] 0 2024-04-22T23:35:15.056 INFO:tasks.workunit.client.0.smithi107.stdout:0/923: mkdir d1/d126/d20/d2c/d41/dbe/d134 0 2024-04-22T23:35:15.056 INFO:tasks.workunit.client.0.smithi107.stdout:1/986: dread d7/d19/d20/d39/d93/df4/f78 [0,4194304] 0 2024-04-22T23:35:15.056 INFO:tasks.workunit.client.0.smithi107.stdout:0/924: creat d1/d126/d20/f135 x:0 0 0 2024-04-22T23:35:15.059 INFO:tasks.workunit.client.0.smithi107.stdout:0/925: rmdir d1/d126/d8a/db5/d105 0 2024-04-22T23:35:15.059 INFO:tasks.workunit.client.0.smithi107.stdout:0/926: write d1/d84/f92 [1618677,18234] 0 2024-04-22T23:35:15.061 INFO:tasks.workunit.client.0.smithi107.stdout:1/987: rename d7/d19/d20/d39/d53/fbe to d7/d19/d20/d39/d53/d43/d46/d68/f145 0 2024-04-22T23:35:15.062 INFO:tasks.workunit.client.0.smithi107.stdout:9/960: read d5/d14/d2e/ff3 [3203885,8626] 0 2024-04-22T23:35:15.063 INFO:tasks.workunit.client.0.smithi107.stdout:9/961: mkdir d5/d144 0 2024-04-22T23:35:15.064 INFO:tasks.workunit.client.0.smithi107.stdout:9/962: creat d5/d7/d17/d1b/f145 x:0 0 0 2024-04-22T23:35:15.064 INFO:tasks.workunit.client.0.smithi107.stdout:9/963: stat d5/d7/d27/d46/d4b/def/f117 0 2024-04-22T23:35:15.064 INFO:tasks.workunit.client.0.smithi107.stdout:9/964: write d5/d7/d27/f9d [548174,112728] 0 2024-04-22T23:35:15.064 INFO:tasks.workunit.client.0.smithi107.stdout:9/965: fdatasync d5/d7/d17/d1b/d1f/d36/d96/fb2 0 2024-04-22T23:35:15.070 INFO:tasks.workunit.client.0.smithi107.stdout:0/927: dread d1/d126/d20/d2c/d41/fac [0,4194304] 0 2024-04-22T23:35:15.070 INFO:tasks.workunit.client.0.smithi107.stdout:0/928: chown d1/d9/d14/d4f 64823 1 2024-04-22T23:35:15.071 INFO:tasks.workunit.client.0.smithi107.stdout:0/929: mknod d1/d7b/d85/ded/c136 0 2024-04-22T23:35:15.072 INFO:tasks.workunit.client.0.smithi107.stdout:0/930: truncate d1/d126/d20/d2c/d41/f64 5219970 0 2024-04-22T23:35:15.074 INFO:tasks.workunit.client.0.smithi107.stdout:0/931: creat d1/d126/d20/d40/dbd/df2/d11f/f137 x:0 0 0 2024-04-22T23:35:15.086 INFO:tasks.workunit.client.0.smithi107.stdout:1/988: dwrite d7/d19/d20/d39/d93/df4/f78 [0,4194304] 0 2024-04-22T23:35:15.087 INFO:tasks.workunit.client.0.smithi107.stdout:9/966: dwrite d5/d29/d7b/fd6 [0,4194304] 0 2024-04-22T23:35:15.087 INFO:tasks.workunit.client.0.smithi107.stdout:9/967: fdatasync d5/d7/de9/d13b/f13d 0 2024-04-22T23:35:15.088 INFO:tasks.workunit.client.0.smithi107.stdout:1/989: mknod d7/d19/d20/d39/d53/d43/c146 0 2024-04-22T23:35:15.089 INFO:tasks.workunit.client.0.smithi107.stdout:0/932: dwrite d1/d7b/d85/ded/f11c [0,4194304] 0 2024-04-22T23:35:15.090 INFO:tasks.workunit.client.0.smithi107.stdout:9/968: mkdir d5/d24/d146 0 2024-04-22T23:35:15.090 INFO:tasks.workunit.client.0.smithi107.stdout:1/990: symlink d7/d19/d20/d39/d53/d43/d46/d5c/dac/l147 0 2024-04-22T23:35:15.093 INFO:tasks.workunit.client.0.smithi107.stdout:0/933: mkdir d1/d126/d20/d138 0 2024-04-22T23:35:15.097 INFO:tasks.workunit.client.0.smithi107.stdout:9/969: read d5/d7/d17/d1b/dcb/ffe [2253541,83846] 0 2024-04-22T23:35:15.097 INFO:tasks.workunit.client.0.smithi107.stdout:9/970: truncate d5/d7/f13c 822825 0 2024-04-22T23:35:15.099 INFO:tasks.workunit.client.0.smithi107.stdout:3/891: sync 2024-04-22T23:35:15.099 INFO:tasks.workunit.client.0.smithi107.stdout:4/987: sync 2024-04-22T23:35:15.099 INFO:tasks.workunit.client.0.smithi107.stdout:8/992: sync 2024-04-22T23:35:15.099 INFO:tasks.workunit.client.0.smithi107.stdout:7/953: sync 2024-04-22T23:35:15.099 INFO:tasks.workunit.client.0.smithi107.stdout:3/892: read - db/fe5 zero size 2024-04-22T23:35:15.100 INFO:tasks.workunit.client.0.smithi107.stdout:4/988: dread - d1/d21/d52/f13f zero size 2024-04-22T23:35:15.100 INFO:tasks.workunit.client.0.smithi107.stdout:1/991: dread d7/fc1 [0,4194304] 0 2024-04-22T23:35:15.101 INFO:tasks.workunit.client.0.smithi107.stdout:8/993: rename d0/d7/d8/l44 to d0/d73/db0/d107/d3b/l147 0 2024-04-22T23:35:15.101 INFO:tasks.workunit.client.0.smithi107.stdout:8/994: creat d0/d3a/d46/f148 x:0 0 0 2024-04-22T23:35:15.101 INFO:tasks.workunit.client.0.smithi107.stdout:8/995: read d0/d73/db0/d107/fca [390,88091] 0 2024-04-22T23:35:15.101 INFO:tasks.workunit.client.0.smithi107.stdout:0/934: dread d1/d126/d8a/db5/fc9 [0,4194304] 0 2024-04-22T23:35:15.102 INFO:tasks.workunit.client.0.smithi107.stdout:3/893: mkdir db/d21/d108/d109/d126 0 2024-04-22T23:35:15.103 INFO:tasks.workunit.client.0.smithi107.stdout:4/989: symlink d1/d7/d63/d78/l155 0 2024-04-22T23:35:15.104 INFO:tasks.workunit.client.0.smithi107.stdout:7/954: symlink d7/d21/da4/d81/l140 0 2024-04-22T23:35:15.104 INFO:tasks.workunit.client.0.smithi107.stdout:7/955: dread - d7/d21/d48/d95/fd4 zero size 2024-04-22T23:35:15.105 INFO:tasks.workunit.client.0.smithi107.stdout:9/971: write d5/f11 [3568653,19585] 0 2024-04-22T23:35:15.106 INFO:tasks.workunit.client.0.smithi107.stdout:7/956: stat d7/d21/d7e/f124 0 2024-04-22T23:35:15.106 INFO:tasks.workunit.client.0.smithi107.stdout:7/957: creat d7/db4/d112/f141 x:0 0 0 2024-04-22T23:35:15.111 INFO:tasks.workunit.client.0.smithi107.stdout:9/972: creat d5/d14/f147 x:0 0 0 2024-04-22T23:35:15.112 INFO:tasks.workunit.client.0.smithi107.stdout:8/996: dread d0/d7/d20/f104 [0,4194304] 0 2024-04-22T23:35:15.114 INFO:tasks.workunit.client.0.smithi107.stdout:4/990: dread d1/d7/f11 [0,4194304] 0 2024-04-22T23:35:15.114 INFO:tasks.workunit.client.0.smithi107.stdout:1/992: rename d7/d19/d2d to d7/d19/d42/d144/d148 0 2024-04-22T23:35:15.115 INFO:tasks.workunit.client.0.smithi107.stdout:9/973: stat d5/d7/d27/c45 0 2024-04-22T23:35:15.116 INFO:tasks.workunit.client.0.smithi107.stdout:8/997: symlink d0/d7/d20/d89/l149 0 2024-04-22T23:35:15.119 INFO:tasks.workunit.client.0.smithi107.stdout:1/993: unlink d7/d19/d20/d39/d53/d43/c146 0 2024-04-22T23:35:15.121 INFO:tasks.workunit.client.0.smithi107.stdout:7/958: dwrite d7/d21/da4/d66/fa0 [0,4194304] 0 2024-04-22T23:35:15.121 INFO:tasks.workunit.client.0.smithi107.stdout:7/959: creat d7/d21/d54/dd3/f142 x:0 0 0 2024-04-22T23:35:15.121 INFO:tasks.workunit.client.0.smithi107.stdout:7/960: write d7/f11 [753939,32414] 0 2024-04-22T23:35:15.121 INFO:tasks.workunit.client.0.smithi107.stdout:1/994: dread d7/d19/d20/d39/d93/f12f [0,4194304] 0 2024-04-22T23:35:15.123 INFO:tasks.workunit.client.0.smithi107.stdout:4/991: dread d1/d21/d29/d2e/d3b/f4c [0,4194304] 0 2024-04-22T23:35:15.123 INFO:tasks.workunit.client.0.smithi107.stdout:4/992: write d1/d21/d29/d110/f129 [74576,100875] 0 2024-04-22T23:35:15.126 INFO:tasks.workunit.client.0.smithi107.stdout:9/974: link d5/d7/d27/d46/la3 d5/d7/d17/d1b/d1f/d36/l148 0 2024-04-22T23:35:15.127 INFO:tasks.workunit.client.0.smithi107.stdout:4/993: symlink d1/d21/l156 0 2024-04-22T23:35:15.127 INFO:tasks.workunit.client.0.smithi107.stdout:4/994: chown d1/d21/d85/f9c 2219 1 2024-04-22T23:35:15.128 INFO:tasks.workunit.client.0.smithi107.stdout:4/995: readlink d1/d7/d63/d78/l14b 0 2024-04-22T23:35:15.128 INFO:tasks.workunit.client.0.smithi107.stdout:7/961: read d7/d21/d48/d5c/d9d/d6e/f70 [684179,64737] 0 2024-04-22T23:35:15.128 INFO:tasks.workunit.client.0.smithi107.stdout:7/962: chown d7/d21/d49/f6a 344 1 2024-04-22T23:35:15.128 INFO:tasks.workunit.client.0.smithi107.stdout:4/996: read d1/d1b/f64 [1103858,52661] 0 2024-04-22T23:35:15.129 INFO:tasks.workunit.client.0.smithi107.stdout:9/975: rename d5/d135 to d5/d14/d2e/d4f/d83/d149 0 2024-04-22T23:35:15.129 INFO:tasks.workunit.client.0.smithi107.stdout:9/976: truncate d5/d7/de9/ff4 88428 0 2024-04-22T23:35:15.130 INFO:tasks.workunit.client.0.smithi107.stdout:1/995: write d7/d19/d42/d144/d148/f84 [8127337,119177] 0 2024-04-22T23:35:15.130 INFO:tasks.workunit.client.0.smithi107.stdout:1/996: readlink d7/d19/d20/d39/d93/la6 0 2024-04-22T23:35:15.132 INFO:tasks.workunit.client.0.smithi107.stdout:4/997: creat d1/d7/f157 x:0 0 0 2024-04-22T23:35:15.132 INFO:tasks.workunit.client.0.smithi107.stdout:4/998: truncate d1/d21/d29/d2e/fac 1671128 0 2024-04-22T23:35:15.133 INFO:tasks.workunit.client.0.smithi107.stdout:9/977: symlink d5/d7/d27/d69/l14a 0 2024-04-22T23:35:15.134 INFO:tasks.workunit.client.0.smithi107.stdout:7/963: write d7/f76 [301405,116115] 0 2024-04-22T23:35:15.134 INFO:tasks.workunit.client.0.smithi107.stdout:7/964: stat d7/c45 0 2024-04-22T23:35:15.135 INFO:tasks.workunit.client.0.smithi107.stdout:4/999: symlink d1/d1b/d44/l158 0 2024-04-22T23:35:15.136 INFO:tasks.workunit.client.0.smithi107.stdout:7/965: mknod d7/d21/d48/d5c/d9d/d6e/c143 0 2024-04-22T23:35:15.138 INFO:tasks.workunit.client.0.smithi107.stdout:7/966: mkdir d7/d21/d54/dcc/d109/d144 0 2024-04-22T23:35:15.139 INFO:tasks.workunit.client.0.smithi107.stdout:7/967: symlink d7/d21/d7e/deb/l145 0 2024-04-22T23:35:15.139 INFO:tasks.workunit.client.0.smithi107.stdout:7/968: mknod d7/d21/da4/dbf/d111/c146 0 2024-04-22T23:35:15.140 INFO:tasks.workunit.client.0.smithi107.stdout:7/969: mknod d7/c147 0 2024-04-22T23:35:15.141 INFO:tasks.workunit.client.0.smithi107.stdout:7/970: mkdir d7/d21/da4/d66/d148 0 2024-04-22T23:35:15.142 INFO:tasks.workunit.client.0.smithi107.stdout:7/971: rmdir d7/d21/d48/d5c/d9d 39 2024-04-22T23:35:15.142 INFO:tasks.workunit.client.0.smithi107.stdout:7/972: write d7/d21/d54/dcc/d109/f110 [933981,15340] 0 2024-04-22T23:35:15.144 INFO:tasks.workunit.client.0.smithi107.stdout:7/973: link d7/d21/d7e/deb/c13e d7/d21/d48/c149 0 2024-04-22T23:35:15.144 INFO:tasks.workunit.client.0.smithi107.stdout:7/974: chown d7/d21/d48/d5c/d9d/f62 3134 1 2024-04-22T23:35:15.144 INFO:tasks.workunit.client.0.smithi107.stdout:7/975: readlink d7/d21/d48/d5c/l130 0 2024-04-22T23:35:15.144 INFO:tasks.workunit.client.0.smithi107.stdout:7/976: fdatasync d7/d21/d48/f61 0 2024-04-22T23:35:15.144 INFO:tasks.workunit.client.0.smithi107.stdout:7/977: read d7/d21/da4/d66/fc1 [648264,6369] 0 2024-04-22T23:35:15.150 INFO:tasks.workunit.client.0.smithi107.stdout:8/998: dwrite d0/d73/db0/d107/d3b/dd9/f11d [0,4194304] 0 2024-04-22T23:35:15.150 INFO:tasks.workunit.client.0.smithi107.stdout:9/978: dwrite d5/d7/de9/d13b/f13d [0,4194304] 0 2024-04-22T23:35:15.150 INFO:tasks.workunit.client.0.smithi107.stdout:8/999: dread - d0/d7/d87/dc8/f117 zero size 2024-04-22T23:35:15.152 INFO:tasks.workunit.client.0.smithi107.stdout:7/978: write d7/d21/f22 [1008203,28857] 0 2024-04-22T23:35:15.153 INFO:tasks.workunit.client.0.smithi107.stdout:9/979: unlink d5/d6c/f108 0 2024-04-22T23:35:15.154 INFO:tasks.workunit.client.0.smithi107.stdout:9/980: symlink d5/d14/d28/de3/l14b 0 2024-04-22T23:35:15.161 INFO:tasks.workunit.client.0.smithi107.stdout:7/979: dread d7/d21/d7e/d94/fc4 [0,4194304] 0 2024-04-22T23:35:15.162 INFO:tasks.workunit.client.0.smithi107.stdout:7/980: mknod d7/d21/d7e/c14a 0 2024-04-22T23:35:15.163 INFO:tasks.workunit.client.0.smithi107.stdout:7/981: truncate d7/d21/da4/f43 3944685 0 2024-04-22T23:35:15.163 INFO:tasks.workunit.client.0.smithi107.stdout:7/982: fdatasync d7/d21/d48/d5c/d9d/db1/f12e 0 2024-04-22T23:35:15.163 INFO:tasks.workunit.client.0.smithi107.stdout:7/983: fdatasync d7/d21/d48/d126/f136 0 2024-04-22T23:35:15.168 INFO:tasks.workunit.client.0.smithi107.stdout:7/984: dread d7/d21/f22 [0,4194304] 0 2024-04-22T23:35:15.170 INFO:tasks.workunit.client.0.smithi107.stdout:7/985: rmdir d7/d21/d48/d5c/d9d/db1 39 2024-04-22T23:35:15.173 INFO:tasks.workunit.client.0.smithi107.stdout:7/986: rename d7/d21/d48/d95/cbc to d7/c14b 0 2024-04-22T23:35:15.176 INFO:tasks.workunit.client.0.smithi107.stdout:7/987: creat d7/d21/d7e/deb/f14c x:0 0 0 2024-04-22T23:35:15.183 INFO:tasks.workunit.client.0.smithi107.stdout:7/988: read d7/d1b/f53 [2800223,3078] 0 2024-04-22T23:35:15.199 INFO:tasks.workunit.client.0.smithi107.stdout:7/989: dwrite d7/d21/d48/f9c [0,4194304] 0 2024-04-22T23:35:15.206 INFO:tasks.workunit.client.0.smithi107.stdout:3/894: sync 2024-04-22T23:35:15.206 INFO:tasks.workunit.client.0.smithi107.stdout:0/935: sync 2024-04-22T23:35:15.206 INFO:tasks.workunit.client.0.smithi107.stdout:3/895: read - db/d21/d27/d15/d24/d52/d60/dca/f113 zero size 2024-04-22T23:35:15.229 INFO:tasks.workunit.client.0.smithi107.stdout:0/936: dwrite d1/d2a/f77 [4194304,4194304] 0 2024-04-22T23:35:15.229 INFO:tasks.workunit.client.0.smithi107.stdout:3/896: dwrite db/d21/db1/dcd/ff8 [4194304,4194304] 0 2024-04-22T23:35:15.229 INFO:tasks.workunit.client.0.smithi107.stdout:3/897: write db/d21/f33 [1139141,128498] 0 2024-04-22T23:35:15.230 INFO:tasks.workunit.client.0.smithi107.stdout:0/937: symlink d1/d126/d20/d40/d75/d115/l139 0 2024-04-22T23:35:15.231 INFO:tasks.workunit.client.0.smithi107.stdout:0/938: write d1/d126/d20/f12b [306825,1622] 0 2024-04-22T23:35:15.231 INFO:tasks.workunit.client.0.smithi107.stdout:0/939: write d1/d126/d8a/db5/fc9 [2356851,72097] 0 2024-04-22T23:35:15.231 INFO:tasks.workunit.client.0.smithi107.stdout:3/898: symlink db/d21/d27/d15/d29/d2a/d45/ddd/l127 0 2024-04-22T23:35:15.233 INFO:tasks.workunit.client.0.smithi107.stdout:0/940: truncate d1/d9/d14/f8c 3485480 0 2024-04-22T23:35:15.234 INFO:tasks.workunit.client.0.smithi107.stdout:3/899: getdents db/d21/d27/d15/d29/d2a/d45/ddd/da7 0 2024-04-22T23:35:15.234 INFO:tasks.workunit.client.0.smithi107.stdout:3/900: write db/fe5 [819801,25431] 0 2024-04-22T23:35:15.235 INFO:tasks.workunit.client.0.smithi107.stdout:3/901: link db/d21/fda db/d21/d108/f128 0 2024-04-22T23:35:15.238 INFO:tasks.workunit.client.0.smithi107.stdout:3/902: symlink db/d21/d27/d15/d29/d2a/d45/ddd/d102/l129 0 2024-04-22T23:35:15.238 INFO:tasks.workunit.client.0.smithi107.stdout:3/903: dread - db/d21/d27/d15/d24/d52/d60/dca/dfc/f101 zero size 2024-04-22T23:35:15.238 INFO:tasks.workunit.client.0.smithi107.stdout:3/904: symlink db/d21/d108/l12a 0 2024-04-22T23:35:15.239 INFO:tasks.workunit.client.0.smithi107.stdout:3/905: chown db/d21/d27/d15/d29/d4f/db2/fd1 96238 1 2024-04-22T23:35:15.241 INFO:tasks.workunit.client.0.smithi107.stdout:3/906: mknod db/d21/d27/d15/d24/d52/d60/c12b 0 2024-04-22T23:35:15.241 INFO:tasks.workunit.client.0.smithi107.stdout:3/907: fdatasync db/ff 0 2024-04-22T23:35:15.268 INFO:tasks.workunit.client.0.smithi107.stdout:1/997: sync 2024-04-22T23:35:15.268 INFO:tasks.workunit.client.0.smithi107.stdout:9/981: sync 2024-04-22T23:35:15.269 INFO:tasks.workunit.client.0.smithi107.stdout:1/998: write d7/dce/f108 [527608,84304] 0 2024-04-22T23:35:15.269 INFO:tasks.workunit.client.0.smithi107.stdout:1/999: creat d7/d19/d20/f149 x:0 0 0 2024-04-22T23:35:15.269 INFO:tasks.workunit.client.0.smithi107.stdout:9/982: getdents d5/d14/df7 0 2024-04-22T23:35:15.270 INFO:tasks.workunit.client.0.smithi107.stdout:9/983: mknod d5/d7/d27/d46/d4b/d138/c14c 0 2024-04-22T23:35:15.270 INFO:tasks.workunit.client.0.smithi107.stdout:9/984: chown d5/d7/d27/d46/fc6 187793 1 2024-04-22T23:35:15.271 INFO:tasks.workunit.client.0.smithi107.stdout:9/985: getdents d5/d29/d7b/d91/dde/dc9/daf 0 2024-04-22T23:35:15.287 INFO:tasks.workunit.client.0.smithi107.stdout:9/986: dwrite d5/d29/f3a [4194304,4194304] 0 2024-04-22T23:35:15.288 INFO:tasks.workunit.client.1.smithi157.stdout:8/966: sync 2024-04-22T23:35:15.289 INFO:tasks.workunit.client.0.smithi107.stdout:9/987: mkdir d5/d7/d27/dfc/d14d 0 2024-04-22T23:35:15.289 INFO:tasks.workunit.client.0.smithi107.stdout:9/988: chown d5/d7/d27/d46/dd3/fc2 221 1 2024-04-22T23:35:15.290 INFO:tasks.workunit.client.1.smithi157.stdout:8/967: symlink d2/d4/d30/db8/l150 0 2024-04-22T23:35:15.290 INFO:tasks.workunit.client.1.smithi157.stdout:8/968: readlink d2/d4/d30/db8/l13d 0 2024-04-22T23:35:15.290 INFO:tasks.workunit.client.1.smithi157.stdout:8/969: write d2/d4/dda/f11c [745026,104732] 0 2024-04-22T23:35:15.290 INFO:tasks.workunit.client.1.smithi157.stdout:8/970: chown d2/d4/dbd/d97/ddd/l133 1 1 2024-04-22T23:35:15.290 INFO:tasks.workunit.client.0.smithi107.stdout:9/989: mknod d5/d29/d7b/c14e 0 2024-04-22T23:35:15.291 INFO:tasks.workunit.client.1.smithi157.stdout:8/971: rmdir d2/d34/d6a/d77 39 2024-04-22T23:35:15.291 INFO:tasks.workunit.client.1.smithi157.stdout:8/972: fdatasync d2/d6/d21/d5f/d17/f1e 0 2024-04-22T23:35:15.291 INFO:tasks.workunit.client.1.smithi157.stdout:8/973: dread - d2/d4/dbd/f14c zero size 2024-04-22T23:35:15.291 INFO:tasks.workunit.client.1.smithi157.stdout:8/974: write d2/d6/d21/d5f/d17/dc8/ffa [209761,111811] 0 2024-04-22T23:35:15.292 INFO:tasks.workunit.client.1.smithi157.stdout:8/975: truncate d2/d4/dbd/dd8/d119/f4a 8631810 0 2024-04-22T23:35:15.293 INFO:tasks.workunit.client.1.smithi157.stdout:8/976: mkdir d2/d4/dbd/d67/d127/d151 0 2024-04-22T23:35:15.294 INFO:tasks.workunit.client.1.smithi157.stdout:8/977: mknod d2/d6/d21/d5f/d17/dc8/c152 0 2024-04-22T23:35:15.294 INFO:tasks.workunit.client.1.smithi157.stdout:8/978: readlink d2/d6/d21/d5f/d17/l138 0 2024-04-22T23:35:15.295 INFO:tasks.workunit.client.0.smithi107.stdout:9/990: rmdir d5/d7/d17/d1b/d1f 39 2024-04-22T23:35:15.295 INFO:tasks.workunit.client.0.smithi107.stdout:9/991: write d5/d7/d17/d1b/d1f/d36/d53/f5b [3450279,118134] 0 2024-04-22T23:35:15.295 INFO:tasks.workunit.client.0.smithi107.stdout:9/992: write d5/d7/d17/d1b/f145 [556086,85991] 0 2024-04-22T23:35:15.296 INFO:tasks.workunit.client.0.smithi107.stdout:9/993: mknod d5/d7/d27/d69/db6/d104/c14f 0 2024-04-22T23:35:15.315 INFO:tasks.workunit.client.0.smithi107.stdout:9/994: dwrite d5/d7/d27/d46/f88 [12582912,4194304] 0 2024-04-22T23:35:15.316 INFO:tasks.workunit.client.1.smithi157.stdout:8/979: dwrite d2/d34/d6a/f76 [0,4194304] 0 2024-04-22T23:35:15.318 INFO:tasks.workunit.client.1.smithi157.stdout:8/980: mknod d2/d6/d21/d8d/c153 0 2024-04-22T23:35:15.318 INFO:tasks.workunit.client.1.smithi157.stdout:8/981: symlink d2/db1/dfb/l154 0 2024-04-22T23:35:15.318 INFO:tasks.workunit.client.1.smithi157.stdout:8/982: creat d2/d4/dda/f155 x:0 0 0 2024-04-22T23:35:15.321 INFO:tasks.workunit.client.1.smithi157.stdout:8/983: rename d2/d6/d21/f98 to d2/d6/d21/d5f/d17/dc2/f156 0 2024-04-22T23:35:15.322 INFO:tasks.workunit.client.1.smithi157.stdout:8/984: symlink d2/d34/d6a/l157 0 2024-04-22T23:35:15.323 INFO:tasks.workunit.client.0.smithi107.stdout:9/995: dread d5/d7/f70 [0,4194304] 0 2024-04-22T23:35:15.323 INFO:tasks.workunit.client.1.smithi157.stdout:8/985: symlink d2/d4/dbd/d97/d125/l158 0 2024-04-22T23:35:15.324 INFO:tasks.workunit.client.0.smithi107.stdout:9/996: getdents d5/d14/d28/de3 0 2024-04-22T23:35:15.324 INFO:tasks.workunit.client.1.smithi157.stdout:8/986: mknod d2/d4/dbd/dd8/c159 0 2024-04-22T23:35:15.324 INFO:tasks.workunit.client.1.smithi157.stdout:8/987: chown d2/d4/dbd/d97/ddd/l133 6643 1 2024-04-22T23:35:15.324 INFO:tasks.workunit.client.1.smithi157.stdout:8/988: readlink d2/d4/dbd/d97/ddd/l133 0 2024-04-22T23:35:15.325 INFO:tasks.workunit.client.1.smithi157.stdout:8/989: symlink d2/d34/d6a/d77/l15a 0 2024-04-22T23:35:15.325 INFO:tasks.workunit.client.1.smithi157.stdout:8/990: creat d2/d4/dbd/dd8/f15b x:0 0 0 2024-04-22T23:35:15.325 INFO:tasks.workunit.client.1.smithi157.stdout:8/991: stat d2/d6/lc9 0 2024-04-22T23:35:15.325 INFO:tasks.workunit.client.1.smithi157.stdout:8/992: chown d2/d34/d6a 68938673 1 2024-04-22T23:35:15.325 INFO:tasks.workunit.client.0.smithi107.stdout:9/997: getdents d5/d14/d28/d11f 0 2024-04-22T23:35:15.326 INFO:tasks.workunit.client.1.smithi157.stdout:8/993: getdents d2/db1/d58 0 2024-04-22T23:35:15.326 INFO:tasks.workunit.client.1.smithi157.stdout:8/994: getdents d2/d4/d73/d13c 0 2024-04-22T23:35:15.326 INFO:tasks.workunit.client.0.smithi107.stdout:9/998: creat d5/d7/d17/d1b/d1f/dda/f150 x:0 0 0 2024-04-22T23:35:15.326 INFO:tasks.workunit.client.0.smithi107.stdout:9/999: chown d5/d7/d17/d1b/f145 128380145 1 2024-04-22T23:35:15.327 INFO:tasks.workunit.client.1.smithi157.stdout:8/995: mknod d2/d34/d6a/d77/c15c 0 2024-04-22T23:35:15.327 INFO:tasks.workunit.client.1.smithi157.stdout:8/996: mknod d2/d4/c15d 0 2024-04-22T23:35:15.328 INFO:tasks.workunit.client.1.smithi157.stdout:8/997: mkdir d2/d15e 0 2024-04-22T23:35:15.329 INFO:tasks.workunit.client.1.smithi157.stdout:8/998: rmdir d2/d34/d6a/d77/df2 39 2024-04-22T23:35:15.329 INFO:tasks.workunit.client.1.smithi157.stdout:8/999: readlink d2/d4/d30/l64 0 2024-04-22T23:35:15.331 INFO:tasks.workunit.client.1.smithi157.stderr:+ rm -rf -- ./tmp.3ctMruI8Hs 2024-04-22T23:35:15.336 INFO:tasks.workunit.client.0.smithi107.stdout:7/990: sync 2024-04-22T23:35:15.337 INFO:tasks.workunit.client.0.smithi107.stdout:7/991: link d7/d4c/l8a d7/d21/d48/d5c/d13f/l14d 0 2024-04-22T23:35:15.338 INFO:tasks.workunit.client.0.smithi107.stdout:7/992: truncate d7/d21/d48/d5c/f118 4115000 0 2024-04-22T23:35:15.340 INFO:tasks.workunit.client.0.smithi107.stdout:7/993: unlink d7/d21/d7e/deb/c13e 0 2024-04-22T23:35:15.342 INFO:tasks.workunit.client.0.smithi107.stdout:7/994: write d7/d21/da4/d81/f97 [1150932,73993] 0 2024-04-22T23:35:15.345 INFO:tasks.workunit.client.0.smithi107.stdout:7/995: mknod d7/c14e 0 2024-04-22T23:35:15.345 INFO:tasks.workunit.client.0.smithi107.stdout:7/996: chown d7/d21/da4/cc7 11359968 1 2024-04-22T23:35:15.346 INFO:tasks.workunit.client.0.smithi107.stdout:7/997: truncate d7/d1b/f68 989157 0 2024-04-22T23:35:15.346 INFO:tasks.workunit.client.0.smithi107.stdout:7/998: write d7/d21/d48/d95/ff6 [853746,118323] 0 2024-04-22T23:35:15.347 INFO:tasks.workunit.client.0.smithi107.stdout:7/999: symlink d7/d21/d48/daf/l14f 0 2024-04-22T23:35:15.356 INFO:tasks.workunit.client.0.smithi107.stdout:3/908: sync 2024-04-22T23:35:15.357 INFO:tasks.workunit.client.0.smithi107.stdout:0/941: sync 2024-04-22T23:35:15.357 INFO:tasks.workunit.client.0.smithi107.stdout:0/942: read - d1/d9/d14/d4f/d59/d114/f124 zero size 2024-04-22T23:35:15.357 INFO:tasks.workunit.client.0.smithi107.stdout:0/943: write d1/d126/d20/d40/dbd/df2/d11f/f137 [550471,122426] 0 2024-04-22T23:35:15.357 INFO:tasks.workunit.client.0.smithi107.stdout:3/909: creat db/d21/db1/dbb/f12c x:0 0 0 2024-04-22T23:35:15.358 INFO:tasks.workunit.client.0.smithi107.stdout:0/944: mknod d1/d126/d8a/db5/d10f/c13a 0 2024-04-22T23:35:15.359 INFO:tasks.workunit.client.0.smithi107.stdout:3/910: getdents db/d21/d27/d15/d29/d4f/db2 0 2024-04-22T23:35:15.359 INFO:tasks.workunit.client.0.smithi107.stdout:3/911: write db/d21/d27/d15/d29/d2a/d45/ddd/d25/fe0 [465476,125315] 0 2024-04-22T23:35:15.367 INFO:tasks.workunit.client.0.smithi107.stdout:0/945: write d1/d84/fc2 [4097867,112827] 0 2024-04-22T23:35:15.367 INFO:tasks.workunit.client.0.smithi107.stdout:3/912: dread db/d21/d27/d15/d29/d2a/d45/ddd/d25/fb6 [0,4194304] 0 2024-04-22T23:35:15.369 INFO:tasks.workunit.client.0.smithi107.stdout:0/946: truncate d1/d9/dd4/fe4 4428415 0 2024-04-22T23:35:15.369 INFO:tasks.workunit.client.0.smithi107.stdout:0/947: chown d1/d9/f3e 2 1 2024-04-22T23:35:15.371 INFO:tasks.workunit.client.0.smithi107.stdout:3/913: creat db/d21/d27/d15/d29/d2a/d99/f12d x:0 0 0 2024-04-22T23:35:15.371 INFO:tasks.workunit.client.0.smithi107.stdout:3/914: write db/d21/d27/f23 [2524831,80430] 0 2024-04-22T23:35:15.371 INFO:tasks.workunit.client.0.smithi107.stdout:3/915: dread - db/d21/d27/d15/d29/d2a/f100 zero size 2024-04-22T23:35:15.371 INFO:tasks.workunit.client.0.smithi107.stdout:3/916: stat db/d21/d27/d15/d24/c38 0 2024-04-22T23:35:15.371 INFO:tasks.workunit.client.0.smithi107.stdout:3/917: creat db/d21/d27/d15/d29/d2a/d99/f12e x:0 0 0 2024-04-22T23:35:15.371 INFO:tasks.workunit.client.0.smithi107.stdout:3/918: fsync db/d21/f33 0 2024-04-22T23:35:15.371 INFO:tasks.workunit.client.0.smithi107.stdout:3/919: readlink db/d21/d27/d15/d29/d2a/d45/ddd/lc1 0 2024-04-22T23:35:15.371 INFO:tasks.workunit.client.0.smithi107.stdout:0/948: mknod d1/d84/c13b 0 2024-04-22T23:35:15.375 INFO:tasks.workunit.client.0.smithi107.stdout:3/920: write db/d21/d27/f39 [3837050,57895] 0 2024-04-22T23:35:15.375 INFO:tasks.workunit.client.0.smithi107.stdout:3/921: chown db/d21/d27/d36/c10f 1700396139 1 2024-04-22T23:35:15.378 INFO:tasks.workunit.client.0.smithi107.stdout:3/922: rename db/d21/d27/d15/d24/d52/d60/lc7 to db/d21/d27/d15/d24/d87/d114/d122/l12f 0 2024-04-22T23:35:15.382 INFO:tasks.workunit.client.0.smithi107.stdout:3/923: symlink db/d21/d27/d15/d29/d2a/d45/ddd/d25/l130 0 2024-04-22T23:35:15.391 INFO:tasks.workunit.client.0.smithi107.stdout:3/924: dread db/d21/d27/d15/d29/f42 [0,4194304] 0 2024-04-22T23:35:15.392 INFO:tasks.workunit.client.0.smithi107.stdout:3/925: symlink db/d21/d27/d15/d29/d2a/d45/deb/l131 0 2024-04-22T23:35:15.393 INFO:tasks.workunit.client.0.smithi107.stdout:3/926: mkdir db/d21/d27/d15/d29/d2a/d45/deb/d11c/d132 0 2024-04-22T23:35:15.407 INFO:tasks.workunit.client.0.smithi107.stdout:3/927: dwrite db/d21/d108/d109/f10c [0,4194304] 0 2024-04-22T23:35:15.407 INFO:tasks.workunit.client.0.smithi107.stdout:3/928: chown db/d21/d27/d36/c10b 1393 1 2024-04-22T23:35:15.409 INFO:tasks.workunit.client.0.smithi107.stdout:3/929: write db/d21/d27/d15/db8/f116 [145188,1027] 0 2024-04-22T23:35:15.411 INFO:tasks.workunit.client.0.smithi107.stdout:3/930: dread db/d21/d27/d15/d29/d2a/d45/ddd/d25/fe0 [0,4194304] 0 2024-04-22T23:35:15.411 INFO:tasks.workunit.client.0.smithi107.stdout:3/931: stat db/d21/d108 0 2024-04-22T23:35:15.419 INFO:tasks.workunit.client.0.smithi107.stdout:3/932: write db/d21/db1/dbb/fe8 [613742,21198] 0 2024-04-22T23:35:15.420 INFO:tasks.workunit.client.0.smithi107.stdout:3/933: unlink db/d21/d27/d15/d29/d2a/d99/f118 0 2024-04-22T23:35:15.424 INFO:tasks.workunit.client.0.smithi107.stdout:3/934: dread db/d21/d27/d15/d24/d52/d60/f124 [0,4194304] 0 2024-04-22T23:35:15.425 INFO:tasks.workunit.client.0.smithi107.stdout:3/935: mkdir db/d21/d27/d36/d9c/d133 0 2024-04-22T23:35:15.426 INFO:tasks.workunit.client.0.smithi107.stdout:3/936: mkdir db/d21/d108/d109/d126/d134 0 2024-04-22T23:35:15.427 INFO:tasks.workunit.client.0.smithi107.stdout:3/937: mkdir db/d21/d27/d15/d24/d135 0 2024-04-22T23:35:15.442 INFO:tasks.workunit.client.0.smithi107.stdout:3/938: dwrite db/d21/d27/d15/f8c [0,4194304] 0 2024-04-22T23:35:15.443 INFO:tasks.workunit.client.0.smithi107.stdout:3/939: getdents db/d21/d27/d36/d9c/d133 0 2024-04-22T23:35:15.451 INFO:tasks.workunit.client.0.smithi107.stdout:3/940: dread db/d21/d27/d15/d29/d2a/f58 [0,4194304] 0 2024-04-22T23:35:15.469 INFO:tasks.workunit.client.0.smithi107.stdout:3/941: dwrite db/f6f [0,4194304] 0 2024-04-22T23:35:15.469 INFO:tasks.workunit.client.0.smithi107.stdout:3/942: stat db/d21/d27/d36/fb0 0 2024-04-22T23:35:15.469 INFO:tasks.workunit.client.0.smithi107.stdout:3/943: creat db/d21/d27/d15/d24/d52/f136 x:0 0 0 2024-04-22T23:35:15.472 INFO:tasks.workunit.client.0.smithi107.stdout:3/944: creat db/d21/d27/d15/d29/d2a/d45/ddd/d25/d53/f137 x:0 0 0 2024-04-22T23:35:15.472 INFO:tasks.workunit.client.0.smithi107.stdout:3/945: write db/d21/db1/dbb/fe8 [1799885,93933] 0 2024-04-22T23:35:15.472 INFO:tasks.workunit.client.0.smithi107.stdout:3/946: readlink db/d21/d27/d36/le6 0 2024-04-22T23:35:15.472 INFO:tasks.workunit.client.0.smithi107.stdout:3/947: chown db/d21/d27/l35 16493 1 2024-04-22T23:35:15.473 INFO:tasks.workunit.client.0.smithi107.stdout:3/948: rename db/d21/d27/d15/d29/d2a/d99/f12e to db/d21/db1/f138 0 2024-04-22T23:35:15.473 INFO:tasks.workunit.client.0.smithi107.stdout:3/949: read db/d21/d27/d15/d29/d2a/d45/ddd/d25/fe2 [942499,53950] 0 2024-04-22T23:35:15.473 INFO:tasks.workunit.client.0.smithi107.stdout:3/950: chown db/d21/d27/d15/d24/de1 135904 1 2024-04-22T23:35:15.475 INFO:tasks.workunit.client.0.smithi107.stdout:3/951: rename db/d21/d27/d15/d29/d4f/db2 to db/d21/d27/d15/d24/d52/d60/dca/d139 0 2024-04-22T23:35:15.475 INFO:tasks.workunit.client.0.smithi107.stdout:3/952: fsync db/d21/d27/d15/db8/f82 0 2024-04-22T23:35:15.500 INFO:tasks.workunit.client.0.smithi107.stdout:3/953: dwrite db/d21/fda [0,4194304] 0 2024-04-22T23:35:15.500 INFO:tasks.workunit.client.0.smithi107.stdout:3/954: readlink db/d21/d27/d15/l9e 0 2024-04-22T23:35:15.501 INFO:tasks.workunit.client.0.smithi107.stdout:3/955: rmdir db/d21/d27/d36 39 2024-04-22T23:35:15.503 INFO:tasks.workunit.client.0.smithi107.stdout:3/956: rename db/d21/d27/d15/d24/d52/d60/d96/ff1 to db/d21/d27/d15/d24/d87/f13a 0 2024-04-22T23:35:15.503 INFO:tasks.workunit.client.0.smithi107.stdout:3/957: getdents db/d21/d27 0 2024-04-22T23:35:15.508 INFO:tasks.workunit.client.0.smithi107.stdout:0/949: sync 2024-04-22T23:35:15.508 INFO:tasks.workunit.client.0.smithi107.stdout:0/950: chown d1/d126/d20/d40/dbd/df2/d11f/f137 71 1 2024-04-22T23:35:15.508 INFO:tasks.workunit.client.0.smithi107.stdout:0/951: chown d1/d2a/l9a 39254 1 2024-04-22T23:35:15.508 INFO:tasks.workunit.client.0.smithi107.stdout:0/952: symlink d1/d9/d14/d4f/d59/l13c 0 2024-04-22T23:35:15.511 INFO:tasks.workunit.client.0.smithi107.stdout:0/953: rename d1/d9/d14/d4f/c96 to d1/d126/d20/d4a/c13d 0 2024-04-22T23:35:15.511 INFO:tasks.workunit.client.0.smithi107.stdout:0/954: dread - d1/d126/d20/d40/db4/f108 zero size 2024-04-22T23:35:15.511 INFO:tasks.workunit.client.0.smithi107.stdout:0/955: readlink d1/d9/d14/d4f/d59/lae 0 2024-04-22T23:35:15.513 INFO:tasks.workunit.client.0.smithi107.stdout:0/956: unlink d1/c49 0 2024-04-22T23:35:15.513 INFO:tasks.workunit.client.0.smithi107.stdout:0/957: write d1/d126/d20/d40/dbd/f118 [556825,16329] 0 2024-04-22T23:35:15.515 INFO:tasks.workunit.client.0.smithi107.stdout:0/958: link d1/d126/d20/c50 d1/d126/d8a/db5/c13e 0 2024-04-22T23:35:15.516 INFO:tasks.workunit.client.0.smithi107.stdout:0/959: rename d1/d126/d20/d40/db4/fd0 to d1/d126/d20/d40/dbd/ddc/f13f 0 2024-04-22T23:35:15.517 INFO:tasks.workunit.client.0.smithi107.stdout:0/960: rmdir d1/d126/d20/d40/dbd/df2/d109 39 2024-04-22T23:35:15.517 INFO:tasks.workunit.client.0.smithi107.stdout:0/961: write d1/d9/dd4/fe4 [5031141,92977] 0 2024-04-22T23:35:15.517 INFO:tasks.workunit.client.0.smithi107.stdout:0/962: mknod d1/d126/d20/d40/dbd/df2/d11f/c140 0 2024-04-22T23:35:15.518 INFO:tasks.workunit.client.0.smithi107.stdout:0/963: symlink d1/d9/dd4/l141 0 2024-04-22T23:35:15.520 INFO:tasks.workunit.client.0.smithi107.stdout:3/958: dwrite db/d21/db1/f115 [0,4194304] 0 2024-04-22T23:35:15.520 INFO:tasks.workunit.client.0.smithi107.stdout:3/959: fdatasync db/d21/d27/d15/d29/d2a/fae 0 2024-04-22T23:35:15.523 INFO:tasks.workunit.client.0.smithi107.stdout:3/960: dread db/d21/d27/d15/d29/d4f/f77 [0,4194304] 0 2024-04-22T23:35:15.524 INFO:tasks.workunit.client.0.smithi107.stdout:3/961: mkdir db/d21/d108/d109/d126/d134/d13b 0 2024-04-22T23:35:15.543 INFO:tasks.workunit.client.0.smithi107.stdout:3/962: dwrite db/d21/d27/d15/d29/d2a/d99/f12d [0,4194304] 0 2024-04-22T23:35:15.544 INFO:tasks.workunit.client.0.smithi107.stdout:0/964: dwrite d1/d84/fc2 [0,4194304] 0 2024-04-22T23:35:15.544 INFO:tasks.workunit.client.0.smithi107.stdout:0/965: fsync d1/d126/fb0 0 2024-04-22T23:35:15.547 INFO:tasks.workunit.client.0.smithi107.stdout:0/966: link d1/d126/d8a/db5/cef d1/d126/c142 0 2024-04-22T23:35:15.548 INFO:tasks.workunit.client.0.smithi107.stdout:3/963: dread db/d21/d27/d15/d24/d52/d60/f124 [0,4194304] 0 2024-04-22T23:35:15.551 INFO:tasks.workunit.client.0.smithi107.stdout:0/967: rmdir d1/d7b 39 2024-04-22T23:35:15.551 INFO:tasks.workunit.client.0.smithi107.stdout:0/968: chown d1/d126/d20/c46 1 1 2024-04-22T23:35:15.553 INFO:tasks.workunit.client.0.smithi107.stdout:3/964: creat db/d21/d27/d36/d9c/d133/f13c x:0 0 0 2024-04-22T23:35:15.553 INFO:tasks.workunit.client.0.smithi107.stdout:3/965: stat db/d21/d27/d15/d24/d52/c7c 0 2024-04-22T23:35:15.554 INFO:tasks.workunit.client.0.smithi107.stdout:0/969: mkdir d1/d2a/d143 0 2024-04-22T23:35:15.557 INFO:tasks.workunit.client.0.smithi107.stdout:3/966: link db/d21/d27/l2c db/d21/d27/d15/d29/d2a/d99/l13d 0 2024-04-22T23:35:15.559 INFO:tasks.workunit.client.0.smithi107.stdout:0/970: creat d1/d126/d20/d40/d75/f144 x:0 0 0 2024-04-22T23:35:15.560 INFO:tasks.workunit.client.0.smithi107.stdout:3/967: mkdir db/d21/d27/d15/d24/d52/d60/d96/d13e 0 2024-04-22T23:35:15.560 INFO:tasks.workunit.client.0.smithi107.stdout:3/968: chown db/d21/c3c 10 1 2024-04-22T23:35:15.563 INFO:tasks.workunit.client.0.smithi107.stdout:0/971: rmdir d1/d126/d20/d2c 39 2024-04-22T23:35:15.565 INFO:tasks.workunit.client.0.smithi107.stdout:0/972: dread d1/d126/fdd [0,4194304] 0 2024-04-22T23:35:15.566 INFO:tasks.workunit.client.0.smithi107.stdout:0/973: mkdir d1/d126/d20/d40/dbd/d145 0 2024-04-22T23:35:15.567 INFO:tasks.workunit.client.0.smithi107.stdout:0/974: dread d1/d9/d14/f9c [0,4194304] 0 2024-04-22T23:35:15.574 INFO:tasks.workunit.client.0.smithi107.stdout:0/975: dread d1/d9/d14/d4f/d59/fe2 [0,4194304] 0 2024-04-22T23:35:15.589 INFO:tasks.workunit.client.0.smithi107.stdout:0/976: dwrite d1/d126/fb0 [0,4194304] 0 2024-04-22T23:35:15.589 INFO:tasks.workunit.client.0.smithi107.stdout:0/977: chown d1/d7b/f10b 2549388 1 2024-04-22T23:35:15.589 INFO:tasks.workunit.client.0.smithi107.stdout:0/978: write d1/d126/d20/d2c/d2e/f12d [530668,104983] 0 2024-04-22T23:35:15.590 INFO:tasks.workunit.client.0.smithi107.stdout:0/979: rmdir d1/d126/d8a/db5 39 2024-04-22T23:35:15.592 INFO:tasks.workunit.client.0.smithi107.stdout:0/980: rename d1/d126/d20/d40/dbd/ddc/lfd to d1/d84/d99/l146 0 2024-04-22T23:35:15.594 INFO:tasks.workunit.client.0.smithi107.stdout:0/981: creat d1/d126/d20/d40/dbd/df2/d109/f147 x:0 0 0 2024-04-22T23:35:15.597 INFO:tasks.workunit.client.0.smithi107.stdout:0/982: rmdir d1/d2a 39 2024-04-22T23:35:15.597 INFO:tasks.workunit.client.0.smithi107.stdout:0/983: chown d1/d7b/l7f 9 1 2024-04-22T23:35:15.597 INFO:tasks.workunit.client.0.smithi107.stdout:0/984: write d1/d126/d8a/d8b/fcc [726039,110138] 0 2024-04-22T23:35:15.597 INFO:tasks.workunit.client.0.smithi107.stdout:0/985: stat d1/d9/d14/c1d 0 2024-04-22T23:35:15.597 INFO:tasks.workunit.client.0.smithi107.stdout:0/986: symlink d1/d126/d8a/d8b/l148 0 2024-04-22T23:35:15.598 INFO:tasks.workunit.client.0.smithi107.stdout:0/987: mkdir d1/d126/d20/d149 0 2024-04-22T23:35:15.598 INFO:tasks.workunit.client.0.smithi107.stdout:0/988: truncate d1/d84/d99/feb 488049 0 2024-04-22T23:35:15.599 INFO:tasks.workunit.client.0.smithi107.stdout:0/989: rmdir d1/d126/d20/d40 39 2024-04-22T23:35:15.599 INFO:tasks.workunit.client.0.smithi107.stdout:0/990: stat d1/d126/d20/d2c/db6 0 2024-04-22T23:35:15.600 INFO:tasks.workunit.client.0.smithi107.stdout:0/991: creat d1/d7b/d85/f14a x:0 0 0 2024-04-22T23:35:15.604 INFO:tasks.workunit.client.0.smithi107.stdout:0/992: getdents d1/d126/d20 0 2024-04-22T23:35:15.604 INFO:tasks.workunit.client.0.smithi107.stdout:0/993: chown d1/d9/d14/d4f/c9f 483292 1 2024-04-22T23:35:15.604 INFO:tasks.workunit.client.0.smithi107.stdout:0/994: mknod d1/d126/d20/c14b 0 2024-04-22T23:35:15.606 INFO:tasks.workunit.client.0.smithi107.stdout:0/995: rename d1/d9/d14/l79 to d1/d2a/l14c 0 2024-04-22T23:35:15.606 INFO:tasks.workunit.client.0.smithi107.stdout:0/996: chown d1/d84/d99/lf5 0 1 2024-04-22T23:35:15.607 INFO:tasks.workunit.client.0.smithi107.stdout:0/997: creat d1/d126/d20/d2c/d41/dbe/f14d x:0 0 0 2024-04-22T23:35:15.623 INFO:tasks.workunit.client.0.smithi107.stdout:0/998: dwrite d1/d7b/f10b [0,4194304] 0 2024-04-22T23:35:15.623 INFO:tasks.workunit.client.0.smithi107.stdout:0/999: dread - d1/d9/d14/d4f/d59/d114/f12e zero size 2024-04-22T23:35:15.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:35:15 smithi157 ceph-mon[33877]: pgmap v264: 65 pgs: 65 active+clean; 3.4 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 303 MiB/s rd, 366 MiB/s wr, 507 op/s 2024-04-22T23:35:15.724 INFO:tasks.workunit.client.0.smithi107.stdout:3/969: sync 2024-04-22T23:35:15.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:35:15 smithi107 ceph-mon[26355]: pgmap v264: 65 pgs: 65 active+clean; 3.4 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 303 MiB/s rd, 366 MiB/s wr, 507 op/s 2024-04-22T23:35:15.741 INFO:tasks.workunit.client.0.smithi107.stdout:3/970: dwrite db/d21/db1/f11f [0,4194304] 0 2024-04-22T23:35:15.741 INFO:tasks.workunit.client.0.smithi107.stdout:3/971: creat db/d21/f13f x:0 0 0 2024-04-22T23:35:15.743 INFO:tasks.workunit.client.0.smithi107.stdout:3/972: mkdir db/d21/d27/d36/d9c/d133/d140 0 2024-04-22T23:35:15.743 INFO:tasks.workunit.client.0.smithi107.stdout:3/973: write db/d21/db1/dbb/f12c [408089,69044] 0 2024-04-22T23:35:15.744 INFO:tasks.workunit.client.0.smithi107.stdout:3/974: link db/d21/d27/d15/d24/d52/d60/d96/l9a db/d21/d27/d36/d9c/d133/d140/l141 0 2024-04-22T23:35:15.745 INFO:tasks.workunit.client.0.smithi107.stdout:3/975: dread db/d21/d27/d15/d29/d2a/d45/ddd/d25/fe0 [0,4194304] 0 2024-04-22T23:35:15.760 INFO:tasks.workunit.client.0.smithi107.stdout:3/976: dwrite db/d21/d27/d15/d24/d52/d60/f124 [0,4194304] 0 2024-04-22T23:35:15.762 INFO:tasks.workunit.client.0.smithi107.stdout:3/977: link db/fcf db/d21/d27/d15/d29/f142 0 2024-04-22T23:35:15.763 INFO:tasks.workunit.client.0.smithi107.stdout:3/978: truncate db/d21/d27/d15/d29/d2a/d45/ddd/d25/fe2 425035 0 2024-04-22T23:35:15.764 INFO:tasks.workunit.client.0.smithi107.stdout:3/979: rename db/d21/d27/d15/d29/d2a/c65 to db/d21/d27/d15/d29/d2a/d40/c143 0 2024-04-22T23:35:15.777 INFO:tasks.workunit.client.0.smithi107.stdout:3/980: dwrite db/d21/d27/d15/d24/d66/da5/def/f103 [0,4194304] 0 2024-04-22T23:35:15.777 INFO:tasks.workunit.client.0.smithi107.stdout:3/981: write db/d21/d27/d15/d24/d52/d60/dca/f113 [83974,93511] 0 2024-04-22T23:35:15.777 INFO:tasks.workunit.client.0.smithi107.stdout:3/982: fsync db/d21/d27/d15/d29/d2a/d45/f62 0 2024-04-22T23:35:15.777 INFO:tasks.workunit.client.0.smithi107.stdout:3/983: chown db/d21/d27/d15/d29/d2a/d45/ddd/d25/lff 0 1 2024-04-22T23:35:15.780 INFO:tasks.workunit.client.0.smithi107.stdout:3/984: rename db/d21/d27/d15/d29/d2a/d45/ddd/d25/c84 to db/d21/d27/d36/d9c/d133/c144 0 2024-04-22T23:35:15.781 INFO:tasks.workunit.client.0.smithi107.stdout:3/985: symlink db/d21/l145 0 2024-04-22T23:35:15.781 INFO:tasks.workunit.client.0.smithi107.stdout:3/986: stat db/d21/d27/d15/d29/d2a/d45/ddd/da7/d121 0 2024-04-22T23:35:15.797 INFO:tasks.workunit.client.0.smithi107.stdout:3/987: dwrite db/d21/d27/d15/d24/d66/fc6 [4194304,4194304] 0 2024-04-22T23:35:15.798 INFO:tasks.workunit.client.0.smithi107.stdout:3/988: stat db/d21/d27/d15/d29/d2a/d45/ddd/d25/d53/c63 0 2024-04-22T23:35:15.798 INFO:tasks.workunit.client.0.smithi107.stdout:3/989: stat db/d21/d27/d15/d24/d52/d60/d96/l9a 0 2024-04-22T23:35:15.798 INFO:tasks.workunit.client.0.smithi107.stdout:3/990: chown db/fe5 4951199 1 2024-04-22T23:35:15.806 INFO:tasks.workunit.client.0.smithi107.stdout:3/991: dread db/d21/f33 [4194304,4194304] 0 2024-04-22T23:35:15.807 INFO:tasks.workunit.client.0.smithi107.stdout:3/992: symlink db/d21/d27/d15/d24/d66/da5/def/l146 0 2024-04-22T23:35:15.807 INFO:tasks.workunit.client.0.smithi107.stdout:3/993: dread - db/d21/d27/d15/d29/d2a/fae zero size 2024-04-22T23:35:15.807 INFO:tasks.workunit.client.0.smithi107.stdout:3/994: fsync db/d21/d27/d15/d24/d52/d60/dca/d139/fd1 0 2024-04-22T23:35:15.807 INFO:tasks.workunit.client.0.smithi107.stdout:3/995: creat db/d21/d27/d15/d29/d2a/d99/f147 x:0 0 0 2024-04-22T23:35:15.808 INFO:tasks.workunit.client.0.smithi107.stdout:3/996: dread - db/fcf zero size 2024-04-22T23:35:15.812 INFO:tasks.workunit.client.0.smithi107.stdout:3/997: read db/d21/d27/d15/d29/d2a/f5b [637688,99874] 0 2024-04-22T23:35:15.828 INFO:tasks.workunit.client.0.smithi107.stdout:3/998: dwrite db/d21/d27/d15/d29/d2a/d99/f147 [0,4194304] 0 2024-04-22T23:35:15.828 INFO:tasks.workunit.client.0.smithi107.stdout:3/999: write db/d21/d27/d15/db8/ff6 [1025396,123968] 0 2024-04-22T23:35:15.830 INFO:tasks.workunit.client.0.smithi107.stderr:+ rm -rf -- ./tmp.M51zPrONoe 2024-04-22T23:35:17.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:35:17 smithi157 ceph-mon[33877]: pgmap v265: 65 pgs: 65 active+clean; 3.7 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 339 MiB/s rd, 412 MiB/s wr, 574 op/s 2024-04-22T23:35:17.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:35:17 smithi107 ceph-mon[26355]: pgmap v265: 65 pgs: 65 active+clean; 3.7 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 339 MiB/s rd, 412 MiB/s wr, 574 op/s 2024-04-22T23:35:19.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:35:19 smithi157 ceph-mon[33877]: pgmap v266: 65 pgs: 65 active+clean; 4.0 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 348 MiB/s rd, 428 MiB/s wr, 587 op/s 2024-04-22T23:35:19.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:35:19 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:35:19.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:35:19 smithi107 ceph-mon[26355]: pgmap v266: 65 pgs: 65 active+clean; 4.0 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 348 MiB/s rd, 428 MiB/s wr, 587 op/s 2024-04-22T23:35:19.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:35:19 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:35:21.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:35:21 smithi107 ceph-mon[26355]: pgmap v267: 65 pgs: 65 active+clean; 3.7 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 262 MiB/s rd, 326 MiB/s wr, 507 op/s 2024-04-22T23:35:21.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:35:21 smithi157 ceph-mon[33877]: pgmap v267: 65 pgs: 65 active+clean; 3.7 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 262 MiB/s rd, 326 MiB/s wr, 507 op/s 2024-04-22T23:35:23.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:35:23 smithi157 ceph-mon[33877]: pgmap v268: 65 pgs: 65 active+clean; 2.7 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 203 MiB/s rd, 252 MiB/s wr, 481 op/s 2024-04-22T23:35:23.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:35:23 smithi107 ceph-mon[26355]: pgmap v268: 65 pgs: 65 active+clean; 2.7 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 203 MiB/s rd, 252 MiB/s wr, 481 op/s 2024-04-22T23:35:24.545 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-22T23:35:24.545 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-04-22T23:35:25.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:35:25 smithi157 ceph-mon[33877]: pgmap v269: 65 pgs: 65 active+clean; 2.5 GiB data, 9.1 GiB used, 527 GiB / 536 GiB avail; 124 MiB/s rd, 155 MiB/s wr, 362 op/s 2024-04-22T23:35:25.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:35:25 smithi107 ceph-mon[26355]: pgmap v269: 65 pgs: 65 active+clean; 2.5 GiB data, 9.1 GiB used, 527 GiB / 536 GiB avail; 124 MiB/s rd, 155 MiB/s wr, 362 op/s 2024-04-22T23:35:25.993 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-22T23:35:25.993 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-22T23:35:27.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.483+0000 7f6971f9e700 1 -- 172.21.15.107:0/1732813913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f696c0fee60 msgr2=0x7f696c0fafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:35:27.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.483+0000 7f6971f9e700 1 --2- 172.21.15.107:0/1732813913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f696c0fee60 0x7f696c0fafa0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6954004940 tx=0x7f6954009e30 comp rx=0 tx=0).stop 2024-04-22T23:35:27.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.483+0000 7f6971f9e700 1 -- 172.21.15.107:0/1732813913 shutdown_connections 2024-04-22T23:35:27.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.483+0000 7f6971f9e700 1 --2- 172.21.15.107:0/1732813913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f696c0fee60 0x7f696c0fafa0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:27.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.483+0000 7f6971f9e700 1 --2- 172.21.15.107:0/1732813913 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f696c0fe510 0x7f696c0fe920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:27.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.483+0000 7f6971f9e700 1 -- 172.21.15.107:0/1732813913 >> 172.21.15.107:0/1732813913 conn(0x7f696c0f6bd0 msgr2=0x7f696c0f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:35:27.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.483+0000 7f6971f9e700 1 -- 172.21.15.107:0/1732813913 shutdown_connections 2024-04-22T23:35:27.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.483+0000 7f6971f9e700 1 -- 172.21.15.107:0/1732813913 wait complete. 2024-04-22T23:35:27.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.484+0000 7f6971f9e700 1 Processor -- start 2024-04-22T23:35:27.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.484+0000 7f6971f9e700 1 -- start start 2024-04-22T23:35:27.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.484+0000 7f6971f9e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f696c0fe510 0x7f696c0fdb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:35:27.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.484+0000 7f6971f9e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f696c0fee60 0x7f696c0fc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:35:27.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.484+0000 7f6971f9e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f696c0fe120 con 0x7f696c0fee60 2024-04-22T23:35:27.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.484+0000 7f6971f9e700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f696c0fc710 con 0x7f696c0fe510 2024-04-22T23:35:27.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.484+0000 7f696b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f696c0fee60 0x7f696c0fc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:35:27.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.484+0000 7f6963fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f696c0fe510 0x7f696c0fdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:35:27.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.484+0000 7f696b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f696c0fee60 0x7f696c0fc1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51260/0 (socket says 172.21.15.107:51260) 2024-04-22T23:35:27.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.484+0000 7f6963fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f696c0fe510 0x7f696c0fdb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.107:45500/0 (socket says 172.21.15.107:45500) 2024-04-22T23:35:27.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.484+0000 7f696b7fe700 1 -- 172.21.15.107:0/3179023240 learned_addr learned my addr 172.21.15.107:0/3179023240 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:35:27.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.485+0000 7f6963fff700 1 -- 172.21.15.107:0/3179023240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f696c0fee60 msgr2=0x7f696c0fc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:35:27.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.485+0000 7f6963fff700 1 --2- 172.21.15.107:0/3179023240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f696c0fee60 0x7f696c0fc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:27.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.485+0000 7f6963fff700 1 -- 172.21.15.107:0/3179023240 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6954005040 con 0x7f696c0fe510 2024-04-22T23:35:27.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.485+0000 7f696b7fe700 1 --2- 172.21.15.107:0/3179023240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f696c0fee60 0x7f696c0fc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T23:35:27.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.485+0000 7f6963fff700 1 --2- 172.21.15.107:0/3179023240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f696c0fe510 0x7f696c0fdb20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f695c009fd0 tx=0x7f695c00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:35:27.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.485+0000 7f6968ff9700 1 -- 172.21.15.107:0/3179023240 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f695c004e90 con 0x7f696c0fe510 2024-04-22T23:35:27.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.485+0000 7f6971f9e700 1 -- 172.21.15.107:0/3179023240 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f696c0fc960 con 0x7f696c0fe510 2024-04-22T23:35:27.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.485+0000 7f6968ff9700 1 -- 172.21.15.107:0/3179023240 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f695c00f040 con 0x7f696c0fe510 2024-04-22T23:35:27.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.486+0000 7f6968ff9700 1 -- 172.21.15.107:0/3179023240 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f695c00ca50 con 0x7f696c0fe510 2024-04-22T23:35:27.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.486+0000 7f6971f9e700 1 -- 172.21.15.107:0/3179023240 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f696c0fcea0 con 0x7f696c0fe510 2024-04-22T23:35:27.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.487+0000 7f6971f9e700 1 -- 172.21.15.107:0/3179023240 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f696c04c710 con 0x7f696c0fe510 2024-04-22T23:35:27.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.487+0000 7f6968ff9700 1 -- 172.21.15.107:0/3179023240 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f695c007cc0 con 0x7f696c0fe510 2024-04-22T23:35:27.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.487+0000 7f6968ff9700 1 --2- 172.21.15.107:0/3179023240 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f69580692a0 0x7f695806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:35:27.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.487+0000 7f6968ff9700 1 -- 172.21.15.107:0/3179023240 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f695c086c20 con 0x7f696c0fe510 2024-04-22T23:35:27.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.487+0000 7f696b7fe700 1 --2- 172.21.15.107:0/3179023240 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f69580692a0 0x7f695806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:35:27.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.488+0000 7f696b7fe700 1 --2- 172.21.15.107:0/3179023240 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f69580692a0 0x7f695806b750 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f695400b0d0 tx=0x7f6954009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:35:27.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.495+0000 7f6968ff9700 1 -- 172.21.15.107:0/3179023240 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f695c04e250 con 0x7f696c0fe510 2024-04-22T23:35:27.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:35:27 smithi157 ceph-mon[33877]: pgmap v270: 65 pgs: 65 active+clean; 1.8 GiB data, 7.6 GiB used, 529 GiB / 536 GiB avail; 113 MiB/s rd, 139 MiB/s wr, 396 op/s 2024-04-22T23:35:27.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:35:27 smithi107 ceph-mon[26355]: pgmap v270: 65 pgs: 65 active+clean; 1.8 GiB data, 7.6 GiB used, 529 GiB / 536 GiB avail; 113 MiB/s rd, 139 MiB/s wr, 396 op/s 2024-04-22T23:35:27.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.811+0000 7f6971f9e700 1 -- 172.21.15.107:0/3179023240 --> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f696c102560 con 0x7f69580692a0 2024-04-22T23:35:27.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.813+0000 7f6968ff9700 1 -- 172.21.15.107:0/3179023240 <== mgr.14206 v2:172.21.15.107:6800/2516490111 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7f696c102560 con 0x7f69580692a0 2024-04-22T23:35:27.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.816+0000 7f6971f9e700 1 -- 172.21.15.107:0/3179023240 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f69580692a0 msgr2=0x7f695806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:35:27.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.816+0000 7f6971f9e700 1 --2- 172.21.15.107:0/3179023240 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f69580692a0 0x7f695806b750 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f695400b0d0 tx=0x7f6954009e30 comp rx=0 tx=0).stop 2024-04-22T23:35:27.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.816+0000 7f6971f9e700 1 -- 172.21.15.107:0/3179023240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f696c0fe510 msgr2=0x7f696c0fdb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:35:27.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.816+0000 7f6971f9e700 1 --2- 172.21.15.107:0/3179023240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f696c0fe510 0x7f696c0fdb20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f695c009fd0 tx=0x7f695c00b040 comp rx=0 tx=0).stop 2024-04-22T23:35:27.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.816+0000 7f6971f9e700 1 -- 172.21.15.107:0/3179023240 shutdown_connections 2024-04-22T23:35:27.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.816+0000 7f6971f9e700 1 --2- 172.21.15.107:0/3179023240 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f69580692a0 0x7f695806b750 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:27.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.816+0000 7f6971f9e700 1 --2- 172.21.15.107:0/3179023240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f696c0fe510 0x7f696c0fdb20 secure :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f695c009fd0 tx=0x7f695c00b040 comp rx=0 tx=0).stop 2024-04-22T23:35:27.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.816+0000 7f6971f9e700 1 --2- 172.21.15.107:0/3179023240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f696c0fee60 0x7f696c0fc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:27.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.816+0000 7f6971f9e700 1 -- 172.21.15.107:0/3179023240 >> 172.21.15.107:0/3179023240 conn(0x7f696c0f6bd0 msgr2=0x7f696c0f8240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:35:27.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.817+0000 7f6971f9e700 1 -- 172.21.15.107:0/3179023240 shutdown_connections 2024-04-22T23:35:27.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.818+0000 7f6971f9e700 1 -- 172.21.15.107:0/3179023240 wait complete. 2024-04-22T23:35:27.824 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-04-22T23:35:27.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.925+0000 7f3fb7608700 1 -- 172.21.15.107:0/932293526 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fb00fcfa0 msgr2=0x7f3fb00fd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:35:27.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.925+0000 7f3fb7608700 1 --2- 172.21.15.107:0/932293526 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fb00fcfa0 0x7f3fb00fd410 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0004940 tx=0x7f3fa0009e30 comp rx=0 tx=0).stop 2024-04-22T23:35:27.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.925+0000 7f3fb7608700 1 -- 172.21.15.107:0/932293526 shutdown_connections 2024-04-22T23:35:27.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.925+0000 7f3fb7608700 1 --2- 172.21.15.107:0/932293526 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fb00fcfa0 0x7f3fb00fd410 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:27.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.925+0000 7f3fb7608700 1 --2- 172.21.15.107:0/932293526 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fb00fbcf0 0x7f3fb00fc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:27.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.925+0000 7f3fb7608700 1 -- 172.21.15.107:0/932293526 >> 172.21.15.107:0/932293526 conn(0x7f3fb00f7450 msgr2=0x7f3fb00f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:35:27.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.925+0000 7f3fb7608700 1 -- 172.21.15.107:0/932293526 shutdown_connections 2024-04-22T23:35:27.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.925+0000 7f3fb7608700 1 -- 172.21.15.107:0/932293526 wait complete. 2024-04-22T23:35:27.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.926+0000 7f3fb7608700 1 Processor -- start 2024-04-22T23:35:27.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.926+0000 7f3fb7608700 1 -- start start 2024-04-22T23:35:27.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.926+0000 7f3fb7608700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fb00fbcf0 0x7f3fb010da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:35:27.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.926+0000 7f3fb7608700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fb00fcfa0 0x7f3fb010df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:35:27.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.926+0000 7f3fb7608700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb010e540 con 0x7f3fb00fcfa0 2024-04-22T23:35:27.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.926+0000 7f3fb7608700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb010ed40 con 0x7f3fb00fbcf0 2024-04-22T23:35:27.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.926+0000 7f3fb4ba3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fb00fcfa0 0x7f3fb010df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:35:27.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.926+0000 7f3fb53a4700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fb00fbcf0 0x7f3fb010da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:35:27.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.926+0000 7f3fb4ba3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fb00fcfa0 0x7f3fb010df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51272/0 (socket says 172.21.15.107:51272) 2024-04-22T23:35:27.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.926+0000 7f3fb4ba3700 1 -- 172.21.15.107:0/3253650501 learned_addr learned my addr 172.21.15.107:0/3253650501 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:35:27.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.926+0000 7f3fb53a4700 1 -- 172.21.15.107:0/3253650501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fb00fcfa0 msgr2=0x7f3fb010df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:35:27.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.926+0000 7f3fb53a4700 1 --2- 172.21.15.107:0/3253650501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fb00fcfa0 0x7f3fb010df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:27.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.926+0000 7f3fb53a4700 1 -- 172.21.15.107:0/3253650501 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa0005040 con 0x7f3fb00fbcf0 2024-04-22T23:35:27.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.927+0000 7f3fb53a4700 1 --2- 172.21.15.107:0/3253650501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fb00fbcf0 0x7f3fb010da00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3fac009fd0 tx=0x7f3fac00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:35:27.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.927+0000 7f3fa5ffb700 1 -- 172.21.15.107:0/3253650501 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3fac004e90 con 0x7f3fb00fbcf0 2024-04-22T23:35:27.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.927+0000 7f3fb7608700 1 -- 172.21.15.107:0/3253650501 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fb010efc0 con 0x7f3fb00fbcf0 2024-04-22T23:35:27.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.927+0000 7f3fa5ffb700 1 -- 172.21.15.107:0/3253650501 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3fac00f040 con 0x7f3fb00fbcf0 2024-04-22T23:35:27.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.927+0000 7f3fa5ffb700 1 -- 172.21.15.107:0/3253650501 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3fac00ca50 con 0x7f3fb00fbcf0 2024-04-22T23:35:27.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.927+0000 7f3fb7608700 1 -- 172.21.15.107:0/3253650501 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fb010f500 con 0x7f3fb00fbcf0 2024-04-22T23:35:27.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.928+0000 7f3fa5ffb700 1 -- 172.21.15.107:0/3253650501 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f3fac007cc0 con 0x7f3fb00fbcf0 2024-04-22T23:35:27.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.929+0000 7f3fb7608700 1 -- 172.21.15.107:0/3253650501 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fb0101b30 con 0x7f3fb00fbcf0 2024-04-22T23:35:27.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.929+0000 7f3fa5ffb700 1 --2- 172.21.15.107:0/3253650501 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f3f9c0692a0 0x7f3f9c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:35:27.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.929+0000 7f3fa5ffb700 1 -- 172.21.15.107:0/3253650501 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f3fac086cc0 con 0x7f3fb00fbcf0 2024-04-22T23:35:27.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.929+0000 7f3fb4ba3700 1 --2- 172.21.15.107:0/3253650501 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f3f9c0692a0 0x7f3f9c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:35:27.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.929+0000 7f3fb4ba3700 1 --2- 172.21.15.107:0/3253650501 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f3f9c0692a0 0x7f3f9c06b750 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0009b30 tx=0x7f3fa0009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:35:27.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:27.937+0000 7f3fa5ffb700 1 -- 172.21.15.107:0/3253650501 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f3fac04e250 con 0x7f3fb00fbcf0 2024-04-22T23:35:28.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.165+0000 7f3fb7608700 1 -- 172.21.15.107:0/3253650501 --> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3fb0104950 con 0x7f3f9c0692a0 2024-04-22T23:35:28.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.166+0000 7f3fa5ffb700 1 -- 172.21.15.107:0/3253650501 <== mgr.14206 v2:172.21.15.107:6800/2516490111 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7f3fb0104950 con 0x7f3f9c0692a0 2024-04-22T23:35:28.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.168+0000 7f3fb7608700 1 -- 172.21.15.107:0/3253650501 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f3f9c0692a0 msgr2=0x7f3f9c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:35:28.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.168+0000 7f3fb7608700 1 --2- 172.21.15.107:0/3253650501 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f3f9c0692a0 0x7f3f9c06b750 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0009b30 tx=0x7f3fa0009e30 comp rx=0 tx=0).stop 2024-04-22T23:35:28.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.168+0000 7f3fb7608700 1 -- 172.21.15.107:0/3253650501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fb00fbcf0 msgr2=0x7f3fb010da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:35:28.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.168+0000 7f3fb7608700 1 --2- 172.21.15.107:0/3253650501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fb00fbcf0 0x7f3fb010da00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3fac009fd0 tx=0x7f3fac00b040 comp rx=0 tx=0).stop 2024-04-22T23:35:28.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.168+0000 7f3fb7608700 1 -- 172.21.15.107:0/3253650501 shutdown_connections 2024-04-22T23:35:28.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.168+0000 7f3fb7608700 1 --2- 172.21.15.107:0/3253650501 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f3f9c0692a0 0x7f3f9c06b750 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:28.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.168+0000 7f3fb7608700 1 --2- 172.21.15.107:0/3253650501 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3fb00fbcf0 0x7f3fb010da00 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:28.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.168+0000 7f3fb7608700 1 --2- 172.21.15.107:0/3253650501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fb00fcfa0 0x7f3fb010df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:28.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.168+0000 7f3fb7608700 1 -- 172.21.15.107:0/3253650501 >> 172.21.15.107:0/3253650501 conn(0x7f3fb00f7450 msgr2=0x7f3fb0066420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:35:28.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.168+0000 7f3fb7608700 1 -- 172.21.15.107:0/3253650501 shutdown_connections 2024-04-22T23:35:28.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.168+0000 7f3fb7608700 1 -- 172.21.15.107:0/3253650501 wait complete. 2024-04-22T23:35:28.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.321+0000 7f036dfba700 1 -- 172.21.15.107:0/3651025827 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0360094d90 msgr2=0x7f0360095200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:35:28.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.321+0000 7f036dfba700 1 --2- 172.21.15.107:0/3651025827 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0360094d90 0x7f0360095200 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0354004940 tx=0x7f0354009e30 comp rx=0 tx=0).stop 2024-04-22T23:35:28.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.322+0000 7f036dfba700 1 -- 172.21.15.107:0/3651025827 shutdown_connections 2024-04-22T23:35:28.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.322+0000 7f036dfba700 1 --2- 172.21.15.107:0/3651025827 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0360094d90 0x7f0360095200 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:28.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.322+0000 7f036dfba700 1 --2- 172.21.15.107:0/3651025827 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0360093ae0 0x7f0360093ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:28.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.322+0000 7f036dfba700 1 -- 172.21.15.107:0/3651025827 >> 172.21.15.107:0/3651025827 conn(0x7f036008f220 msgr2=0x7f0360091670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:35:28.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.323+0000 7f036dfba700 1 -- 172.21.15.107:0/3651025827 shutdown_connections 2024-04-22T23:35:28.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.323+0000 7f036dfba700 1 -- 172.21.15.107:0/3651025827 wait complete. 2024-04-22T23:35:28.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.323+0000 7f036dfba700 1 Processor -- start 2024-04-22T23:35:28.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.323+0000 7f036dfba700 1 -- start start 2024-04-22T23:35:28.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.324+0000 7f036dfba700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0360093ae0 0x7f03600abae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:35:28.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.324+0000 7f036dfba700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0360094d90 0x7f03600ac020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:35:28.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.324+0000 7f036dfba700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03600ac620 con 0x7f0360093ae0 2024-04-22T23:35:28.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.324+0000 7f036dfba700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03600ac790 con 0x7f0360094d90 2024-04-22T23:35:28.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.324+0000 7f036cfb8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0360093ae0 0x7f03600abae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:35:28.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.324+0000 7f036cfb8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0360093ae0 0x7f03600abae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51296/0 (socket says 172.21.15.107:51296) 2024-04-22T23:35:28.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.324+0000 7f036cfb8700 1 -- 172.21.15.107:0/40035144 learned_addr learned my addr 172.21.15.107:0/40035144 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:35:28.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.324+0000 7f0367fff700 1 --2- 172.21.15.107:0/40035144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0360094d90 0x7f03600ac020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:35:28.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.325+0000 7f036cfb8700 1 -- 172.21.15.107:0/40035144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0360094d90 msgr2=0x7f03600ac020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:35:28.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.325+0000 7f036cfb8700 1 --2- 172.21.15.107:0/40035144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0360094d90 0x7f03600ac020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:28.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.325+0000 7f036cfb8700 1 -- 172.21.15.107:0/40035144 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0354005040 con 0x7f0360093ae0 2024-04-22T23:35:28.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.325+0000 7f036cfb8700 1 --2- 172.21.15.107:0/40035144 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0360093ae0 0x7f03600abae0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f035c002a50 tx=0x7f035c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:35:28.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.325+0000 7f03657fa700 1 -- 172.21.15.107:0/40035144 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f035c007cc0 con 0x7f0360093ae0 2024-04-22T23:35:28.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.325+0000 7f03657fa700 1 -- 172.21.15.107:0/40035144 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f035c00f040 con 0x7f0360093ae0 2024-04-22T23:35:28.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.325+0000 7f03657fa700 1 -- 172.21.15.107:0/40035144 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f035c00da30 con 0x7f0360093ae0 2024-04-22T23:35:28.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.326+0000 7f036dfba700 1 -- 172.21.15.107:0/40035144 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03600b11f0 con 0x7f0360093ae0 2024-04-22T23:35:28.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.326+0000 7f036dfba700 1 -- 172.21.15.107:0/40035144 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03600b17b0 con 0x7f0360093ae0 2024-04-22T23:35:28.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.327+0000 7f036dfba700 1 -- 172.21.15.107:0/40035144 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0360004470 con 0x7f0360093ae0 2024-04-22T23:35:28.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.328+0000 7f03657fa700 1 -- 172.21.15.107:0/40035144 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f035c00db90 con 0x7f0360093ae0 2024-04-22T23:35:28.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.329+0000 7f03657fa700 1 --2- 172.21.15.107:0/40035144 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f0358069360 0x7f035806b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:35:28.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.329+0000 7f03657fa700 1 -- 172.21.15.107:0/40035144 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f035c086de0 con 0x7f0360093ae0 2024-04-22T23:35:28.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.329+0000 7f0367fff700 1 --2- 172.21.15.107:0/40035144 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f0358069360 0x7f035806b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:35:28.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.330+0000 7f0367fff700 1 --2- 172.21.15.107:0/40035144 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f0358069360 0x7f035806b810 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f035400b970 tx=0x7f0354009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:35:28.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.336+0000 7f03657fa700 1 -- 172.21.15.107:0/40035144 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f035c056bf0 con 0x7f0360093ae0 2024-04-22T23:35:28.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.604+0000 7f036dfba700 1 -- 172.21.15.107:0/40035144 --> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0360097fc0 con 0x7f0358069360 2024-04-22T23:35:28.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.615+0000 7f03657fa700 1 -- 172.21.15.107:0/40035144 <== mgr.14206 v2:172.21.15.107:6800/2516490111 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3344 (secure 0 0 0) 0x7f0360097fc0 con 0x7f0358069360 2024-04-22T23:35:28.610 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-22T23:35:28.610 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (7m) 2m ago 8m 25.5M - 0.25.0 c8568f914cd2 4d90ceb89249 2024-04-22T23:35:28.610 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (8m) 2m ago 8m 6660k - 17.2.7-884-g70d0cb4b cafd8ddac8fa c10611c756e4 2024-04-22T23:35:28.610 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi157 smithi157 running (7m) 2m ago 7m 6669k - 17.2.7-884-g70d0cb4b cafd8ddac8fa 55d2d46294c9 2024-04-22T23:35:28.610 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (7m) 2m ago 8m 86.8M - 9.4.7 954c08fa6188 c2d1f8de1aa7 2024-04-22T23:35:28.610 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.dfvvkw smithi107 running (3m) 2m ago 3m 10.4M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 08d3aa8d717d 2024-04-22T23:35:28.610 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.ktuxjl smithi107 running (2m) 2m ago 2m 16.9M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 762f27fe0f30 2024-04-22T23:35:28.610 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi157.rcaris smithi157 running (3m) 2m ago 3m 16.8M - 17.2.7-884-g70d0cb4b cafd8ddac8fa d23366ac085d 2024-04-22T23:35:28.610 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi157.ufywse smithi157 running (2m) 2m ago 2m 11.0M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 8b00d61eb355 2024-04-22T23:35:28.610 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.uipxvm smithi107 *:9283 running (9m) 2m ago 9m 481M - 17.2.7-884-g70d0cb4b cafd8ddac8fa db96e8cbbdfb 2024-04-22T23:35:28.611 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi157.tuylis smithi157 *:8443,9283 running (7m) 2m ago 7m 431M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 0c12be10ffaf 2024-04-22T23:35:28.611 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (9m) 2m ago 10m 49.6M 2048M 17.2.7-884-g70d0cb4b cafd8ddac8fa 8314962fa1b9 2024-04-22T23:35:28.611 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi157 smithi157 running (7m) 2m ago 7m 36.6M 2048M 17.2.7-884-g70d0cb4b cafd8ddac8fa 0747a85008d9 2024-04-22T23:35:28.611 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (8m) 2m ago 8m 16.0M - 1.5.0 0da6a335fe13 8a7c595c114f 2024-04-22T23:35:28.611 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi157 smithi157 *:9100 running (7m) 2m ago 7m 16.1M - 1.5.0 0da6a335fe13 940164771fe3 2024-04-22T23:35:28.611 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (6m) 2m ago 6m 46.6M 4288M 17.2.7-884-g70d0cb4b cafd8ddac8fa 18a179b0a618 2024-04-22T23:35:28.611 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (6m) 2m ago 6m 45.9M 4288M 17.2.7-884-g70d0cb4b cafd8ddac8fa c7272eb84ddb 2024-04-22T23:35:28.611 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (5m) 2m ago 5m 46.2M 4288M 17.2.7-884-g70d0cb4b cafd8ddac8fa 9db1788096ad 2024-04-22T23:35:28.611 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi157 running (5m) 2m ago 5m 43.9M 5312M 17.2.7-884-g70d0cb4b cafd8ddac8fa c9fb2d1a13dd 2024-04-22T23:35:28.611 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi157 running (4m) 2m ago 4m 44.2M 5312M 17.2.7-884-g70d0cb4b cafd8ddac8fa 24a16e19951f 2024-04-22T23:35:28.611 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi157 running (4m) 2m ago 4m 43.1M 5312M 17.2.7-884-g70d0cb4b cafd8ddac8fa 4c5b7a7ef8bd 2024-04-22T23:35:28.611 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (6m) 2m ago 8m 51.2M - 2.43.0 a07b618ecd1d 8c5df40caabe 2024-04-22T23:35:28.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.618+0000 7f036dfba700 1 -- 172.21.15.107:0/40035144 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f0358069360 msgr2=0x7f035806b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:35:28.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.618+0000 7f036dfba700 1 --2- 172.21.15.107:0/40035144 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f0358069360 0x7f035806b810 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f035400b970 tx=0x7f0354009e30 comp rx=0 tx=0).stop 2024-04-22T23:35:28.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.618+0000 7f036dfba700 1 -- 172.21.15.107:0/40035144 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0360093ae0 msgr2=0x7f03600abae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:35:28.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.618+0000 7f036dfba700 1 --2- 172.21.15.107:0/40035144 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0360093ae0 0x7f03600abae0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f035c002a50 tx=0x7f035c0046b0 comp rx=0 tx=0).stop 2024-04-22T23:35:28.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.618+0000 7f036dfba700 1 -- 172.21.15.107:0/40035144 shutdown_connections 2024-04-22T23:35:28.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.618+0000 7f036dfba700 1 --2- 172.21.15.107:0/40035144 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f0358069360 0x7f035806b810 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:28.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.618+0000 7f036dfba700 1 --2- 172.21.15.107:0/40035144 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0360093ae0 0x7f03600abae0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:28.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.618+0000 7f036dfba700 1 --2- 172.21.15.107:0/40035144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0360094d90 0x7f03600ac020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:28.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.618+0000 7f036dfba700 1 -- 172.21.15.107:0/40035144 >> 172.21.15.107:0/40035144 conn(0x7f036008f220 msgr2=0x7f0360090470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:35:28.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.618+0000 7f036dfba700 1 -- 172.21.15.107:0/40035144 shutdown_connections 2024-04-22T23:35:28.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.618+0000 7f036dfba700 1 -- 172.21.15.107:0/40035144 wait complete. 2024-04-22T23:35:28.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.767+0000 7fbeeb882700 1 -- 172.21.15.107:0/1100387724 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbee40fdff0 msgr2=0x7fbee40fe460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:35:28.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.767+0000 7fbeeb882700 1 --2- 172.21.15.107:0/1100387724 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbee40fdff0 0x7fbee40fe460 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fbed4004940 tx=0x7fbed4009e30 comp rx=0 tx=0).stop 2024-04-22T23:35:28.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.767+0000 7fbeeb882700 1 -- 172.21.15.107:0/1100387724 shutdown_connections 2024-04-22T23:35:28.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.767+0000 7fbeeb882700 1 --2- 172.21.15.107:0/1100387724 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbee40fdff0 0x7fbee40fe460 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:28.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.767+0000 7fbeeb882700 1 --2- 172.21.15.107:0/1100387724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbee40fce80 0x7fbee40fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:28.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.767+0000 7fbeeb882700 1 -- 172.21.15.107:0/1100387724 >> 172.21.15.107:0/1100387724 conn(0x7fbee40f8540 msgr2=0x7fbee40fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:35:28.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.768+0000 7fbeeb882700 1 -- 172.21.15.107:0/1100387724 shutdown_connections 2024-04-22T23:35:28.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.768+0000 7fbeeb882700 1 -- 172.21.15.107:0/1100387724 wait complete. 2024-04-22T23:35:28.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.769+0000 7fbeeb882700 1 Processor -- start 2024-04-22T23:35:28.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.769+0000 7fbeeb882700 1 -- start start 2024-04-22T23:35:28.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.769+0000 7fbeeb882700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbee40fce80 0x7fbee410d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:35:28.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.769+0000 7fbeeb882700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbee40fdff0 0x7fbee410ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:35:28.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.769+0000 7fbeeb882700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee410e3d0 con 0x7fbee40fdff0 2024-04-22T23:35:28.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.769+0000 7fbeeb882700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee410e540 con 0x7fbee40fce80 2024-04-22T23:35:28.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.770+0000 7fbee961e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbee40fce80 0x7fbee410d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:35:28.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.770+0000 7fbee961e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbee40fce80 0x7fbee410d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.107:45538/0 (socket says 172.21.15.107:45538) 2024-04-22T23:35:28.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.770+0000 7fbee961e700 1 -- 172.21.15.107:0/2294315432 learned_addr learned my addr 172.21.15.107:0/2294315432 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:35:28.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.770+0000 7fbee8e1d700 1 --2- 172.21.15.107:0/2294315432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbee40fdff0 0x7fbee410ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:35:28.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.770+0000 7fbee8e1d700 1 -- 172.21.15.107:0/2294315432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbee40fce80 msgr2=0x7fbee410d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:35:28.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.770+0000 7fbee8e1d700 1 --2- 172.21.15.107:0/2294315432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbee40fce80 0x7fbee410d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:28.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.770+0000 7fbee8e1d700 1 -- 172.21.15.107:0/2294315432 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed4005040 con 0x7fbee40fdff0 2024-04-22T23:35:28.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.770+0000 7fbee8e1d700 1 --2- 172.21.15.107:0/2294315432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbee40fdff0 0x7fbee410ddd0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fbed4000c00 tx=0x7fbed4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:35:28.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.771+0000 7fbed9ffb700 1 -- 172.21.15.107:0/2294315432 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbed400e030 con 0x7fbee40fdff0 2024-04-22T23:35:28.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.771+0000 7fbeeb882700 1 -- 172.21.15.107:0/2294315432 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbee4112f40 con 0x7fbee40fdff0 2024-04-22T23:35:28.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.771+0000 7fbed9ffb700 1 -- 172.21.15.107:0/2294315432 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbed401c400 con 0x7fbee40fdff0 2024-04-22T23:35:28.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.771+0000 7fbed9ffb700 1 -- 172.21.15.107:0/2294315432 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbed400e030 con 0x7fbee40fdff0 2024-04-22T23:35:28.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.771+0000 7fbeeb882700 1 -- 172.21.15.107:0/2294315432 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbee4113450 con 0x7fbee40fdff0 2024-04-22T23:35:28.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.774+0000 7fbeeb882700 1 -- 172.21.15.107:0/2294315432 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbee404c710 con 0x7fbee40fdff0 2024-04-22T23:35:28.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.774+0000 7fbed9ffb700 1 -- 172.21.15.107:0/2294315432 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fbed401ca70 con 0x7fbee40fdff0 2024-04-22T23:35:28.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.775+0000 7fbed9ffb700 1 --2- 172.21.15.107:0/2294315432 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fbed00692a0 0x7fbed006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:35:28.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.775+0000 7fbed9ffb700 1 -- 172.21.15.107:0/2294315432 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fbed40887f0 con 0x7fbee40fdff0 2024-04-22T23:35:28.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.775+0000 7fbee961e700 1 --2- 172.21.15.107:0/2294315432 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fbed00692a0 0x7fbed006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:35:28.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.776+0000 7fbee961e700 1 --2- 172.21.15.107:0/2294315432 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fbed00692a0 0x7fbed006b750 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbee0004bc0 tx=0x7fbee000b040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:35:28.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:28.781+0000 7fbed9ffb700 1 -- 172.21.15.107:0/2294315432 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fbed4058620 con 0x7fbee40fdff0 2024-04-22T23:35:29.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.110+0000 7fbeeb882700 1 -- 172.21.15.107:0/2294315432 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbee4113930 con 0x7fbee40fdff0 2024-04-22T23:35:29.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.110+0000 7fbed9ffb700 1 -- 172.21.15.107:0/2294315432 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7fbed40581b0 con 0x7fbee40fdff0 2024-04-22T23:35:29.105 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-22T23:35:29.105 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-04-22T23:35:29.106 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 2 2024-04-22T23:35:29.106 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:35:29.106 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-04-22T23:35:29.106 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 2 2024-04-22T23:35:29.106 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:35:29.106 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-04-22T23:35:29.106 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 6 2024-04-22T23:35:29.106 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:35:29.106 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-04-22T23:35:29.107 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 4 2024-04-22T23:35:29.107 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:35:29.107 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-04-22T23:35:29.107 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 14 2024-04-22T23:35:29.107 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T23:35:29.107 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-22T23:35:29.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.113+0000 7fbeeb882700 1 -- 172.21.15.107:0/2294315432 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fbed00692a0 msgr2=0x7fbed006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:35:29.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.113+0000 7fbeeb882700 1 --2- 172.21.15.107:0/2294315432 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fbed00692a0 0x7fbed006b750 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbee0004bc0 tx=0x7fbee000b040 comp rx=0 tx=0).stop 2024-04-22T23:35:29.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.113+0000 7fbeeb882700 1 -- 172.21.15.107:0/2294315432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbee40fdff0 msgr2=0x7fbee410ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:35:29.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.113+0000 7fbeeb882700 1 --2- 172.21.15.107:0/2294315432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbee40fdff0 0x7fbee410ddd0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fbed4000c00 tx=0x7fbed4009e30 comp rx=0 tx=0).stop 2024-04-22T23:35:29.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.113+0000 7fbeeb882700 1 -- 172.21.15.107:0/2294315432 shutdown_connections 2024-04-22T23:35:29.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.113+0000 7fbeeb882700 1 --2- 172.21.15.107:0/2294315432 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fbed00692a0 0x7fbed006b750 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:29.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.113+0000 7fbeeb882700 1 --2- 172.21.15.107:0/2294315432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbee40fce80 0x7fbee410d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:29.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.113+0000 7fbeeb882700 1 --2- 172.21.15.107:0/2294315432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbee40fdff0 0x7fbee410ddd0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:29.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.113+0000 7fbeeb882700 1 -- 172.21.15.107:0/2294315432 >> 172.21.15.107:0/2294315432 conn(0x7fbee40f8540 msgr2=0x7fbee40f9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:35:29.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.113+0000 7fbeeb882700 1 -- 172.21.15.107:0/2294315432 shutdown_connections 2024-04-22T23:35:29.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.113+0000 7fbeeb882700 1 -- 172.21.15.107:0/2294315432 wait complete. 2024-04-22T23:35:29.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.243+0000 7f9b5d225700 1 -- 172.21.15.107:0/3783195534 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b580fe410 msgr2=0x7f9b580fe820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:35:29.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.243+0000 7f9b5d225700 1 --2- 172.21.15.107:0/3783195534 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b580fe410 0x7f9b580fe820 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f9b48004940 tx=0x7f9b48009e30 comp rx=0 tx=0).stop 2024-04-22T23:35:29.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.243+0000 7f9b5d225700 1 -- 172.21.15.107:0/3783195534 shutdown_connections 2024-04-22T23:35:29.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.243+0000 7f9b5d225700 1 --2- 172.21.15.107:0/3783195534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b580fed60 0x7f9b580faea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:29.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.243+0000 7f9b5d225700 1 --2- 172.21.15.107:0/3783195534 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b580fe410 0x7f9b580fe820 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:29.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.243+0000 7f9b5d225700 1 -- 172.21.15.107:0/3783195534 >> 172.21.15.107:0/3783195534 conn(0x7f9b580f6bd0 msgr2=0x7f9b580f8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:35:29.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.243+0000 7f9b5d225700 1 -- 172.21.15.107:0/3783195534 shutdown_connections 2024-04-22T23:35:29.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.243+0000 7f9b5d225700 1 -- 172.21.15.107:0/3783195534 wait complete. 2024-04-22T23:35:29.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.244+0000 7f9b5d225700 1 Processor -- start 2024-04-22T23:35:29.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.244+0000 7f9b5d225700 1 -- start start 2024-04-22T23:35:29.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.244+0000 7f9b5d225700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b580fe410 0x7f9b580fdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:35:29.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.244+0000 7f9b5d225700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b580fed60 0x7f9b580fc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:35:29.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.244+0000 7f9b5d225700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b580fc680 con 0x7f9b580fed60 2024-04-22T23:35:29.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.244+0000 7f9b5d225700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b580fc7f0 con 0x7f9b580fe410 2024-04-22T23:35:29.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.244+0000 7f9b5659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b580fed60 0x7f9b580fc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:35:29.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.244+0000 7f9b5659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b580fed60 0x7f9b580fc0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51332/0 (socket says 172.21.15.107:51332) 2024-04-22T23:35:29.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.244+0000 7f9b56d9d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b580fe410 0x7f9b580fdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:35:29.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.244+0000 7f9b5659c700 1 -- 172.21.15.107:0/2947334326 learned_addr learned my addr 172.21.15.107:0/2947334326 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:35:29.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.245+0000 7f9b56d9d700 1 -- 172.21.15.107:0/2947334326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b580fed60 msgr2=0x7f9b580fc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:35:29.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.245+0000 7f9b56d9d700 1 --2- 172.21.15.107:0/2947334326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b580fed60 0x7f9b580fc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:29.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.245+0000 7f9b56d9d700 1 -- 172.21.15.107:0/2947334326 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b48005040 con 0x7f9b580fe410 2024-04-22T23:35:29.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.245+0000 7f9b5659c700 1 --2- 172.21.15.107:0/2947334326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b580fed60 0x7f9b580fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T23:35:29.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.245+0000 7f9b56d9d700 1 --2- 172.21.15.107:0/2947334326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b580fe410 0x7f9b580fdc00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9b48009fd0 tx=0x7f9b48009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:35:29.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.245+0000 7f9b3f7fe700 1 -- 172.21.15.107:0/2947334326 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b48021070 con 0x7f9b580fe410 2024-04-22T23:35:29.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.245+0000 7f9b3f7fe700 1 -- 172.21.15.107:0/2947334326 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9b4801c400 con 0x7f9b580fe410 2024-04-22T23:35:29.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.245+0000 7f9b5d225700 1 -- 172.21.15.107:0/2947334326 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b580fca70 con 0x7f9b580fe410 2024-04-22T23:35:29.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.245+0000 7f9b3f7fe700 1 -- 172.21.15.107:0/2947334326 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b4800e070 con 0x7f9b580fe410 2024-04-22T23:35:29.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.245+0000 7f9b5d225700 1 -- 172.21.15.107:0/2947334326 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b581159d0 con 0x7f9b580fe410 2024-04-22T23:35:29.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.246+0000 7f9b5d225700 1 -- 172.21.15.107:0/2947334326 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b5805f5f0 con 0x7f9b580fe410 2024-04-22T23:35:29.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.247+0000 7f9b3f7fe700 1 -- 172.21.15.107:0/2947334326 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f9b4801ca20 con 0x7f9b580fe410 2024-04-22T23:35:29.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.248+0000 7f9b3f7fe700 1 --2- 172.21.15.107:0/2947334326 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f9b400692b0 0x7f9b4006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:35:29.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.248+0000 7f9b3f7fe700 1 -- 172.21.15.107:0/2947334326 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f9b480882d0 con 0x7f9b580fe410 2024-04-22T23:35:29.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.248+0000 7f9b5659c700 1 --2- 172.21.15.107:0/2947334326 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f9b400692b0 0x7f9b4006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:35:29.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.248+0000 7f9b5659c700 1 --2- 172.21.15.107:0/2947334326 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f9b400692b0 0x7f9b4006b760 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c0041d0 tx=0x7f9b4c00b040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:35:29.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.252+0000 7f9b3f7fe700 1 -- 172.21.15.107:0/2947334326 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f9b48054650 con 0x7f9b580fe410 2024-04-22T23:35:29.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.587+0000 7f9b5d225700 1 -- 172.21.15.107:0/2947334326 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9b58115b90 con 0x7f9b580fe410 2024-04-22T23:35:29.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.587+0000 7f9b3f7fe700 1 -- 172.21.15.107:0/2947334326 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) v1 ==== 76+0+1858 (secure 0 0 0) 0x7f9b48057c70 con 0x7f9b580fe410 2024-04-22T23:35:29.582 INFO:teuthology.orchestra.run.smithi107.stdout:e19 2024-04-22T23:35:29.582 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T23:35:29.582 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:35:29.582 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-04-22T23:35:29.582 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:35:29.582 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-04-22T23:35:29.583 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-04-22T23:35:29.583 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 19 2024-04-22T23:35:29.583 INFO:teuthology.orchestra.run.smithi107.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-22T23:35:29.583 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-04-22T23:32:20.146130+0000 2024-04-22T23:35:29.583 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-04-22T23:35:12.338678+0000 2024-04-22T23:35:29.583 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-04-22T23:35:29.583 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-04-22T23:35:29.583 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-04-22T23:35:29.583 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-04-22T23:35:29.583 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-04-22T23:35:29.583 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-04-22T23:35:29.583 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-04-22T23:35:29.583 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 46 2024-04-22T23:35:29.583 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:35:29.583 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 2 2024-04-22T23:35:29.584 INFO:teuthology.orchestra.run.smithi107.stdout:in 0,1 2024-04-22T23:35:29.584 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=14468,1=24265} 2024-04-22T23:35:29.584 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-04-22T23:35:29.584 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-04-22T23:35:29.584 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-04-22T23:35:29.584 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-04-22T23:35:29.584 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-04-22T23:35:29.584 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data disabled 2024-04-22T23:35:29.584 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-04-22T23:35:29.584 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-04-22T23:35:29.584 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.ktuxjl{0:14468} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.107:6828/3335313253,v1:172.21.15.107:6829/3335313253] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:35:29.584 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi157.rcaris{1:24265} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.157:6824/2779251893,v1:172.21.15.157:6825/2779251893] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:35:29.584 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:35:29.584 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:35:29.584 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-04-22T23:35:29.585 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:35:29.585 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.dfvvkw{-1:14484} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/3548717247,v1:172.21.15.107:6827/3548717247] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:35:29.585 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi157.ufywse{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.157:6826/4072444199,v1:172.21.15.157:6827/4072444199] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:35:29.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.589+0000 7f9b5d225700 1 -- 172.21.15.107:0/2947334326 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f9b400692b0 msgr2=0x7f9b4006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:35:29.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.589+0000 7f9b5d225700 1 --2- 172.21.15.107:0/2947334326 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f9b400692b0 0x7f9b4006b760 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c0041d0 tx=0x7f9b4c00b040 comp rx=0 tx=0).stop 2024-04-22T23:35:29.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.589+0000 7f9b5d225700 1 -- 172.21.15.107:0/2947334326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b580fe410 msgr2=0x7f9b580fdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:35:29.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.589+0000 7f9b5d225700 1 --2- 172.21.15.107:0/2947334326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b580fe410 0x7f9b580fdc00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9b48009fd0 tx=0x7f9b48009e30 comp rx=0 tx=0).stop 2024-04-22T23:35:29.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.589+0000 7f9b5d225700 1 -- 172.21.15.107:0/2947334326 shutdown_connections 2024-04-22T23:35:29.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.589+0000 7f9b5d225700 1 --2- 172.21.15.107:0/2947334326 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f9b400692b0 0x7f9b4006b760 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:29.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.589+0000 7f9b5d225700 1 --2- 172.21.15.107:0/2947334326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b580fe410 0x7f9b580fdc00 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:29.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.589+0000 7f9b5d225700 1 --2- 172.21.15.107:0/2947334326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b580fed60 0x7f9b580fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:29.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.589+0000 7f9b5d225700 1 -- 172.21.15.107:0/2947334326 >> 172.21.15.107:0/2947334326 conn(0x7f9b580f6bd0 msgr2=0x7f9b580f81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:35:29.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.589+0000 7f9b5d225700 1 -- 172.21.15.107:0/2947334326 shutdown_connections 2024-04-22T23:35:29.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.589+0000 7f9b5d225700 1 -- 172.21.15.107:0/2947334326 wait complete. 2024-04-22T23:35:29.586 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 19 2024-04-22T23:35:29.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:35:29 smithi157 ceph-mon[33877]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:35:29.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:35:29 smithi157 ceph-mon[33877]: pgmap v271: 65 pgs: 65 active+clean; 780 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 67 MiB/s rd, 84 MiB/s wr, 403 op/s 2024-04-22T23:35:29.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:35:29 smithi157 ceph-mon[33877]: from='client.24419 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:35:29.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:35:29 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/2294315432' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:35:29.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.731+0000 7f580effd700 1 -- 172.21.15.107:0/2560588906 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5800013b70 con 0x7f58100fc2e0 2024-04-22T23:35:29.727 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:35:29 smithi107 ceph-mon[26355]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:35:29.727 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:35:29 smithi107 ceph-mon[26355]: pgmap v271: 65 pgs: 65 active+clean; 780 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 67 MiB/s rd, 84 MiB/s wr, 403 op/s 2024-04-22T23:35:29.727 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:35:29 smithi107 ceph-mon[26355]: from='client.24419 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:35:29.727 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:35:29 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/2294315432' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:35:29.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.732+0000 7f5816779700 1 -- 172.21.15.107:0/2560588906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58100fc2e0 msgr2=0x7f58100fc6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:35:29.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.732+0000 7f5816779700 1 --2- 172.21.15.107:0/2560588906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58100fc2e0 0x7f58100fc6f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5800004940 tx=0x7f5800009e30 comp rx=0 tx=0).stop 2024-04-22T23:35:29.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.732+0000 7f5816779700 1 -- 172.21.15.107:0/2560588906 shutdown_connections 2024-04-22T23:35:29.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.732+0000 7f5816779700 1 --2- 172.21.15.107:0/2560588906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58100fcc30 0x7f58100fd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:29.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.732+0000 7f5816779700 1 --2- 172.21.15.107:0/2560588906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58100fc2e0 0x7f58100fc6f0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:29.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.732+0000 7f5816779700 1 -- 172.21.15.107:0/2560588906 >> 172.21.15.107:0/2560588906 conn(0x7f58100f6bd0 msgr2=0x7f58100f8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:35:29.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.732+0000 7f5816779700 1 -- 172.21.15.107:0/2560588906 shutdown_connections 2024-04-22T23:35:29.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.732+0000 7f5816779700 1 -- 172.21.15.107:0/2560588906 wait complete. 2024-04-22T23:35:29.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.733+0000 7f5816779700 1 Processor -- start 2024-04-22T23:35:29.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.734+0000 7f5816779700 1 -- start start 2024-04-22T23:35:29.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.734+0000 7f5816779700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58100fc2e0 0x7f581010fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:35:29.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.734+0000 7f5816779700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58100fcc30 0x7f58101100d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:35:29.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.734+0000 7f580ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58100fc2e0 0x7f581010fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:35:29.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.734+0000 7f580ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58100fc2e0 0x7f581010fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51358/0 (socket says 172.21.15.107:51358) 2024-04-22T23:35:29.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.735+0000 7f5816779700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58101106d0 con 0x7f58100fc2e0 2024-04-22T23:35:29.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.735+0000 7f5816779700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5810110f20 con 0x7f58100fcc30 2024-04-22T23:35:29.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.735+0000 7f580ffff700 1 -- 172.21.15.107:0/3731100474 learned_addr learned my addr 172.21.15.107:0/3731100474 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:35:29.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.735+0000 7f580f7fe700 1 --2- 172.21.15.107:0/3731100474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58100fcc30 0x7f58101100d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:35:29.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.735+0000 7f580ffff700 1 -- 172.21.15.107:0/3731100474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58100fcc30 msgr2=0x7f58101100d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:35:29.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.735+0000 7f580ffff700 1 --2- 172.21.15.107:0/3731100474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58100fcc30 0x7f58101100d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:29.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.735+0000 7f580ffff700 1 -- 172.21.15.107:0/3731100474 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5800005040 con 0x7f58100fc2e0 2024-04-22T23:35:29.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.736+0000 7f580ffff700 1 --2- 172.21.15.107:0/3731100474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58100fc2e0 0x7f581010fb90 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5800013870 tx=0x7f5800009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:35:29.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.737+0000 7f580cff9700 1 -- 172.21.15.107:0/3731100474 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5800021070 con 0x7f58100fc2e0 2024-04-22T23:35:29.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.737+0000 7f580cff9700 1 -- 172.21.15.107:0/3731100474 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5800009430 con 0x7f58100fc2e0 2024-04-22T23:35:29.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.737+0000 7f5816779700 1 -- 172.21.15.107:0/3731100474 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58101111a0 con 0x7f58100fc2e0 2024-04-22T23:35:29.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.737+0000 7f5816779700 1 -- 172.21.15.107:0/3731100474 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58101116e0 con 0x7f58100fc2e0 2024-04-22T23:35:29.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.737+0000 7f580cff9700 1 -- 172.21.15.107:0/3731100474 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f580000e070 con 0x7f58100fc2e0 2024-04-22T23:35:29.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.739+0000 7f5816779700 1 -- 172.21.15.107:0/3731100474 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57fc0052e0 con 0x7f58100fc2e0 2024-04-22T23:35:29.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.740+0000 7f580cff9700 1 -- 172.21.15.107:0/3731100474 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f5800013af0 con 0x7f58100fc2e0 2024-04-22T23:35:29.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.740+0000 7f580cff9700 1 --2- 172.21.15.107:0/3731100474 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f57f8069440 0x7f57f806b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:35:29.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.740+0000 7f580cff9700 1 -- 172.21.15.107:0/3731100474 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f58000883a0 con 0x7f58100fc2e0 2024-04-22T23:35:29.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.740+0000 7f580f7fe700 1 --2- 172.21.15.107:0/3731100474 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f57f8069440 0x7f57f806b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:35:29.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.741+0000 7f580f7fe700 1 --2- 172.21.15.107:0/3731100474 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f57f8069440 0x7f57f806b8f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5804009b80 tx=0x7f580400b040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:35:29.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.747+0000 7f580cff9700 1 -- 172.21.15.107:0/3731100474 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f58000580f0 con 0x7f58100fc2e0 2024-04-22T23:35:29.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.979+0000 7f5816779700 1 -- 172.21.15.107:0/3731100474 --> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f57fc000bf0 con 0x7f57f8069440 2024-04-22T23:35:29.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.981+0000 7f580cff9700 1 -- 172.21.15.107:0/3731100474 <== mgr.14206 v2:172.21.15.107:6800/2516490111 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7f57fc000bf0 con 0x7f57f8069440 2024-04-22T23:35:29.976 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-22T23:35:29.976 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c", 2024-04-22T23:35:29.976 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-04-22T23:35:29.976 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-22T23:35:29.976 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [], 2024-04-22T23:35:29.976 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "0/21 daemons upgraded", 2024-04-22T23:35:29.976 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c image on host smithi157", 2024-04-22T23:35:29.976 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-04-22T23:35:29.976 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-22T23:35:29.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.983+0000 7f5816779700 1 -- 172.21.15.107:0/3731100474 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f57f8069440 msgr2=0x7f57f806b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:35:29.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.983+0000 7f5816779700 1 --2- 172.21.15.107:0/3731100474 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f57f8069440 0x7f57f806b8f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5804009b80 tx=0x7f580400b040 comp rx=0 tx=0).stop 2024-04-22T23:35:29.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.983+0000 7f5816779700 1 -- 172.21.15.107:0/3731100474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58100fc2e0 msgr2=0x7f581010fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:35:29.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.983+0000 7f5816779700 1 --2- 172.21.15.107:0/3731100474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58100fc2e0 0x7f581010fb90 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5800013870 tx=0x7f5800009e30 comp rx=0 tx=0).stop 2024-04-22T23:35:29.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.983+0000 7f5816779700 1 -- 172.21.15.107:0/3731100474 shutdown_connections 2024-04-22T23:35:29.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.983+0000 7f5816779700 1 --2- 172.21.15.107:0/3731100474 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f57f8069440 0x7f57f806b8f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:29.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.983+0000 7f5816779700 1 --2- 172.21.15.107:0/3731100474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58100fc2e0 0x7f581010fb90 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:29.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.983+0000 7f5816779700 1 --2- 172.21.15.107:0/3731100474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58100fcc30 0x7f58101100d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:29.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.983+0000 7f5816779700 1 -- 172.21.15.107:0/3731100474 >> 172.21.15.107:0/3731100474 conn(0x7f58100f6bd0 msgr2=0x7f58100f7710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:35:29.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.983+0000 7f5816779700 1 -- 172.21.15.107:0/3731100474 shutdown_connections 2024-04-22T23:35:29.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:29.983+0000 7f5816779700 1 -- 172.21.15.107:0/3731100474 wait complete. 2024-04-22T23:35:30.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:30.095+0000 7f4c1e865700 1 -- 172.21.15.107:0/3163669435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c180fcfa0 msgr2=0x7f4c180fd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:35:30.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:30.095+0000 7f4c1e865700 1 --2- 172.21.15.107:0/3163669435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c180fcfa0 0x7f4c180fd410 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c004940 tx=0x7f4c0c009e30 comp rx=0 tx=0).stop 2024-04-22T23:35:30.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:30.095+0000 7f4c1e865700 1 -- 172.21.15.107:0/3163669435 shutdown_connections 2024-04-22T23:35:30.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:30.095+0000 7f4c1e865700 1 --2- 172.21.15.107:0/3163669435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c180fcfa0 0x7f4c180fd410 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:30.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:30.095+0000 7f4c1e865700 1 --2- 172.21.15.107:0/3163669435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c180fbcf0 0x7f4c180fc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:30.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:30.095+0000 7f4c1e865700 1 -- 172.21.15.107:0/3163669435 >> 172.21.15.107:0/3163669435 conn(0x7f4c180f7450 msgr2=0x7f4c180f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:35:30.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:30.096+0000 7f4c1e865700 1 -- 172.21.15.107:0/3163669435 shutdown_connections 2024-04-22T23:35:30.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:30.096+0000 7f4c1e865700 1 -- 172.21.15.107:0/3163669435 wait complete. 2024-04-22T23:35:30.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:30.096+0000 7f4c1e865700 1 Processor -- start 2024-04-22T23:35:30.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:30.097+0000 7f4c1e865700 1 -- start start 2024-04-22T23:35:30.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:30.097+0000 7f4c1e865700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c180fbcf0 0x7f4c18110720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:35:30.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:30.097+0000 7f4c1e865700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c180fcfa0 0x7f4c18112c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:35:30.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:30.097+0000 7f4c1e865700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c18066ba0 con 0x7f4c180fcfa0 2024-04-22T23:35:30.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:30.097+0000 7f4c1e865700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c18066ce0 con 0x7f4c180fbcf0 2024-04-22T23:35:30.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:30.097+0000 7f4c177fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c180fcfa0 0x7f4c18112c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:35:30.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:30.097+0000 7f4c177fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c180fcfa0 0x7f4c18112c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51386/0 (socket says 172.21.15.107:51386) 2024-04-22T23:35:30.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:30.097+0000 7f4c177fe700 1 -- 172.21.15.107:0/2946275734 learned_addr learned my addr 172.21.15.107:0/2946275734 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:35:30.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:30.097+0000 7f4c17fff700 1 --2- 172.21.15.107:0/2946275734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c180fbcf0 0x7f4c18110720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:35:30.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:30.097+0000 7f4c177fe700 1 -- 172.21.15.107:0/2946275734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c180fbcf0 msgr2=0x7f4c18110720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:35:30.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:30.097+0000 7f4c177fe700 1 --2- 172.21.15.107:0/2946275734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c180fbcf0 0x7f4c18110720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:30.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:30.097+0000 7f4c177fe700 1 -- 172.21.15.107:0/2946275734 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c0c005040 con 0x7f4c180fcfa0 2024-04-22T23:35:30.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:30.097+0000 7f4c177fe700 1 --2- 172.21.15.107:0/2946275734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c180fcfa0 0x7f4c18112c70 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c013870 tx=0x7f4c0c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:35:30.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:30.098+0000 7f4c14ff9700 1 -- 172.21.15.107:0/2946275734 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c0c021070 con 0x7f4c180fcfa0 2024-04-22T23:35:30.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:30.098+0000 7f4c14ff9700 1 -- 172.21.15.107:0/2946275734 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4c0c01c400 con 0x7f4c180fcfa0 2024-04-22T23:35:30.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:30.098+0000 7f4c1e865700 1 -- 172.21.15.107:0/2946275734 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c181131b0 con 0x7f4c180fcfa0 2024-04-22T23:35:30.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:30.098+0000 7f4c14ff9700 1 -- 172.21.15.107:0/2946275734 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c0c00e070 con 0x7f4c180fcfa0 2024-04-22T23:35:30.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:30.098+0000 7f4c1e865700 1 -- 172.21.15.107:0/2946275734 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c18113650 con 0x7f4c180fcfa0 2024-04-22T23:35:30.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:30.099+0000 7f4c1e865700 1 -- 172.21.15.107:0/2946275734 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c1805f5f0 con 0x7f4c180fcfa0 2024-04-22T23:35:30.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:30.100+0000 7f4c14ff9700 1 -- 172.21.15.107:0/2946275734 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f4c0c01ca20 con 0x7f4c180fcfa0 2024-04-22T23:35:30.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:30.101+0000 7f4c14ff9700 1 --2- 172.21.15.107:0/2946275734 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f4c000692a0 0x7f4c0006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:35:30.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:30.101+0000 7f4c14ff9700 1 -- 172.21.15.107:0/2946275734 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f4c0c088c70 con 0x7f4c180fcfa0 2024-04-22T23:35:30.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:30.101+0000 7f4c17fff700 1 --2- 172.21.15.107:0/2946275734 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f4c000692a0 0x7f4c0006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:35:30.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:30.101+0000 7f4c17fff700 1 --2- 172.21.15.107:0/2946275734 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f4c000692a0 0x7f4c0006b750 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4c08004600 tx=0x7f4c0800b040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:35:30.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:30.105+0000 7f4c14ff9700 1 -- 172.21.15.107:0/2946275734 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4c0c054ff0 con 0x7f4c180fcfa0 2024-04-22T23:35:30.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:30.470+0000 7f4c1e865700 1 -- 172.21.15.107:0/2946275734 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f4c181047e0 con 0x7f4c180fcfa0 2024-04-22T23:35:30.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:30.470+0000 7f4c14ff9700 1 -- 172.21.15.107:0/2946275734 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f4c0c004490 con 0x7f4c180fcfa0 2024-04-22T23:35:30.465 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_OK 2024-04-22T23:35:30.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:30.473+0000 7f4c1e865700 1 -- 172.21.15.107:0/2946275734 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f4c000692a0 msgr2=0x7f4c0006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:35:30.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:30.473+0000 7f4c1e865700 1 --2- 172.21.15.107:0/2946275734 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f4c000692a0 0x7f4c0006b750 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4c08004600 tx=0x7f4c0800b040 comp rx=0 tx=0).stop 2024-04-22T23:35:30.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:30.473+0000 7f4c1e865700 1 -- 172.21.15.107:0/2946275734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c180fcfa0 msgr2=0x7f4c18112c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:35:30.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:30.473+0000 7f4c1e865700 1 --2- 172.21.15.107:0/2946275734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c180fcfa0 0x7f4c18112c70 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c013870 tx=0x7f4c0c009e30 comp rx=0 tx=0).stop 2024-04-22T23:35:30.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:30.473+0000 7f4c1e865700 1 -- 172.21.15.107:0/2946275734 shutdown_connections 2024-04-22T23:35:30.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:30.473+0000 7f4c1e865700 1 --2- 172.21.15.107:0/2946275734 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f4c000692a0 0x7f4c0006b750 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:30.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:30.473+0000 7f4c1e865700 1 --2- 172.21.15.107:0/2946275734 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c180fbcf0 0x7f4c18110720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:30.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:30.473+0000 7f4c1e865700 1 --2- 172.21.15.107:0/2946275734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c180fcfa0 0x7f4c18112c70 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:35:30.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:30.473+0000 7f4c1e865700 1 -- 172.21.15.107:0/2946275734 >> 172.21.15.107:0/2946275734 conn(0x7f4c180f7450 msgr2=0x7f4c180f8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:35:30.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:30.473+0000 7f4c1e865700 1 -- 172.21.15.107:0/2946275734 shutdown_connections 2024-04-22T23:35:30.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:35:30.473+0000 7f4c1e865700 1 -- 172.21.15.107:0/2946275734 wait complete. 2024-04-22T23:35:30.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:35:30 smithi107 ceph-mon[26355]: from='client.14672 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:35:30.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:35:30 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/2947334326' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:35:30.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:35:30 smithi157 ceph-mon[33877]: from='client.14672 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:35:30.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:35:30 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/2947334326' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:35:31.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:35:31 smithi107 ceph-mon[26355]: from='client.14684 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:35:31.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:35:31 smithi107 ceph-mon[26355]: pgmap v272: 65 pgs: 65 active+clean; 413 MiB data, 3.2 GiB used, 533 GiB / 536 GiB avail; 9.6 MiB/s rd, 18 MiB/s wr, 321 op/s 2024-04-22T23:35:31.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:35:31 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/2946275734' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:35:31.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:35:31 smithi157 ceph-mon[33877]: from='client.14684 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:35:31.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:35:31 smithi157 ceph-mon[33877]: pgmap v272: 65 pgs: 65 active+clean; 413 MiB data, 3.2 GiB used, 533 GiB / 536 GiB avail; 9.6 MiB/s rd, 18 MiB/s wr, 321 op/s 2024-04-22T23:35:31.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:35:31 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/2946275734' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:35:33.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:35:33 smithi107 ceph-mon[26355]: pgmap v273: 65 pgs: 65 active+clean; 290 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 55 KiB/s rd, 3.1 MiB/s wr, 456 op/s 2024-04-22T23:35:33.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:35:33 smithi157 ceph-mon[33877]: pgmap v273: 65 pgs: 65 active+clean; 290 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 55 KiB/s rd, 3.1 MiB/s wr, 456 op/s 2024-04-22T23:35:34.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:35:34 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:35:34.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:35:34 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:35:35.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:35:35 smithi107 ceph-mon[26355]: pgmap v274: 65 pgs: 65 active+clean; 289 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 45 KiB/s rd, 2.5 MiB/s wr, 389 op/s 2024-04-22T23:35:35.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:35:35 smithi157 ceph-mon[33877]: pgmap v274: 65 pgs: 65 active+clean; 289 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 45 KiB/s rd, 2.5 MiB/s wr, 389 op/s 2024-04-22T23:35:37.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:35:37 smithi157 ceph-mon[33877]: pgmap v275: 65 pgs: 65 active+clean; 288 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 59 KiB/s rd, 2.2 MiB/s wr, 451 op/s 2024-04-22T23:35:37.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:35:37 smithi107 ceph-mon[26355]: pgmap v275: 65 pgs: 65 active+clean; 288 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 59 KiB/s rd, 2.2 MiB/s wr, 451 op/s 2024-04-22T23:35:39.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:35:39 smithi107 ceph-mon[26355]: pgmap v276: 65 pgs: 65 active+clean; 297 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 58 KiB/s rd, 2.9 MiB/s wr, 586 op/s 2024-04-22T23:35:39.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:35:39 smithi157 ceph-mon[33877]: pgmap v276: 65 pgs: 65 active+clean; 297 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 58 KiB/s rd, 2.9 MiB/s wr, 586 op/s 2024-04-22T23:35:41.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:35:41 smithi107 ceph-mon[26355]: pgmap v277: 65 pgs: 65 active+clean; 298 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 65 KiB/s rd, 2.6 MiB/s wr, 611 op/s 2024-04-22T23:35:41.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:35:41 smithi157 ceph-mon[33877]: pgmap v277: 65 pgs: 65 active+clean; 298 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 65 KiB/s rd, 2.6 MiB/s wr, 611 op/s 2024-04-22T23:35:43.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:35:43 smithi107 ceph-mon[26355]: pgmap v278: 65 pgs: 65 active+clean; 297 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 65 KiB/s rd, 3.4 MiB/s wr, 711 op/s 2024-04-22T23:35:43.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:35:43 smithi157 ceph-mon[33877]: pgmap v278: 65 pgs: 65 active+clean; 297 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 65 KiB/s rd, 3.4 MiB/s wr, 711 op/s 2024-04-22T23:35:45.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:35:45 smithi107 ceph-mon[26355]: pgmap v279: 65 pgs: 65 active+clean; 291 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 71 KiB/s rd, 2.3 MiB/s wr, 609 op/s 2024-04-22T23:35:45.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:35:45 smithi157 ceph-mon[33877]: pgmap v279: 65 pgs: 65 active+clean; 291 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 71 KiB/s rd, 2.3 MiB/s wr, 609 op/s 2024-04-22T23:35:47.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:35:47 smithi157 ceph-mon[33877]: pgmap v280: 65 pgs: 65 active+clean; 282 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 71 KiB/s rd, 2.3 MiB/s wr, 659 op/s 2024-04-22T23:35:47.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:35:47 smithi107 ceph-mon[26355]: pgmap v280: 65 pgs: 65 active+clean; 282 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 71 KiB/s rd, 2.3 MiB/s wr, 659 op/s 2024-04-22T23:35:49.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:35:49 smithi157 ceph-mon[33877]: pgmap v281: 65 pgs: 65 active+clean; 282 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 70 KiB/s rd, 3.4 MiB/s wr, 753 op/s 2024-04-22T23:35:49.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:35:49 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:35:49.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:35:49 smithi107 ceph-mon[26355]: pgmap v281: 65 pgs: 65 active+clean; 282 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 70 KiB/s rd, 3.4 MiB/s wr, 753 op/s 2024-04-22T23:35:49.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:35:49 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:35:51.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:35:51 smithi107 ceph-mon[26355]: pgmap v282: 65 pgs: 65 active+clean; 283 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 70 KiB/s rd, 2.7 MiB/s wr, 650 op/s 2024-04-22T23:35:51.671 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:35:51 smithi157 ceph-mon[33877]: pgmap v282: 65 pgs: 65 active+clean; 283 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 70 KiB/s rd, 2.7 MiB/s wr, 650 op/s 2024-04-22T23:35:53.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:35:53 smithi107 ceph-mon[26355]: pgmap v283: 65 pgs: 65 active+clean; 284 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 59 KiB/s rd, 2.8 MiB/s wr, 631 op/s 2024-04-22T23:35:53.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:35:53 smithi157 ceph-mon[33877]: pgmap v283: 65 pgs: 65 active+clean; 284 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 59 KiB/s rd, 2.8 MiB/s wr, 631 op/s 2024-04-22T23:35:55.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:35:55 smithi107 ceph-mon[26355]: pgmap v284: 65 pgs: 65 active+clean; 287 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 59 KiB/s rd, 2.3 MiB/s wr, 520 op/s 2024-04-22T23:35:55.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:35:55 smithi157 ceph-mon[33877]: pgmap v284: 65 pgs: 65 active+clean; 287 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 59 KiB/s rd, 2.3 MiB/s wr, 520 op/s 2024-04-22T23:35:57.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:35:57 smithi157 ceph-mon[33877]: pgmap v285: 65 pgs: 65 active+clean; 286 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 42 KiB/s rd, 2.3 MiB/s wr, 491 op/s 2024-04-22T23:35:57.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:35:57 smithi107 ceph-mon[26355]: pgmap v285: 65 pgs: 65 active+clean; 286 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 42 KiB/s rd, 2.3 MiB/s wr, 491 op/s 2024-04-22T23:35:59.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:35:59 smithi107 ceph-mon[26355]: pgmap v286: 65 pgs: 65 active+clean; 287 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 42 KiB/s rd, 2.6 MiB/s wr, 509 op/s 2024-04-22T23:35:59.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:35:59 smithi157 ceph-mon[33877]: pgmap v286: 65 pgs: 65 active+clean; 287 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 42 KiB/s rd, 2.6 MiB/s wr, 509 op/s 2024-04-22T23:36:00.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:00.597+0000 7fb04bdc5700 1 -- 172.21.15.107:0/95974975 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0440fcff0 msgr2=0x7fb0440fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:00.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:00.597+0000 7fb04bdc5700 1 --2- 172.21.15.107:0/95974975 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0440fcff0 0x7fb0440fd460 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb034004940 tx=0x7fb034009e30 comp rx=0 tx=0).stop 2024-04-22T23:36:00.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:00.598+0000 7fb04bdc5700 1 -- 172.21.15.107:0/95974975 shutdown_connections 2024-04-22T23:36:00.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:00.598+0000 7fb04bdc5700 1 --2- 172.21.15.107:0/95974975 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0440fcff0 0x7fb0440fd460 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:00.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:00.598+0000 7fb04bdc5700 1 --2- 172.21.15.107:0/95974975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0440fbd40 0x7fb0440fc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:00.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:00.598+0000 7fb04bdc5700 1 -- 172.21.15.107:0/95974975 >> 172.21.15.107:0/95974975 conn(0x7fb0440f74c0 msgr2=0x7fb0440f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:00.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:00.598+0000 7fb04bdc5700 1 -- 172.21.15.107:0/95974975 shutdown_connections 2024-04-22T23:36:00.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:00.598+0000 7fb04bdc5700 1 -- 172.21.15.107:0/95974975 wait complete. 2024-04-22T23:36:00.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:00.598+0000 7fb04bdc5700 1 Processor -- start 2024-04-22T23:36:00.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:00.598+0000 7fb04bdc5700 1 -- start start 2024-04-22T23:36:00.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:00.598+0000 7fb04bdc5700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0440fbd40 0x7fb04410d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:00.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:00.598+0000 7fb04bdc5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0440fcff0 0x7fb04410df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:00.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:00.598+0000 7fb04bdc5700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04410e520 con 0x7fb0440fcff0 2024-04-22T23:36:00.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:00.598+0000 7fb04bdc5700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04410ed20 con 0x7fb0440fbd40 2024-04-22T23:36:00.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:00.599+0000 7fb049360700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0440fcff0 0x7fb04410df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:00.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:00.599+0000 7fb049b61700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0440fbd40 0x7fb04410d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:00.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:00.599+0000 7fb049360700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0440fcff0 0x7fb04410df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59208/0 (socket says 172.21.15.107:59208) 2024-04-22T23:36:00.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:00.599+0000 7fb049360700 1 -- 172.21.15.107:0/2473750705 learned_addr learned my addr 172.21.15.107:0/2473750705 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:36:00.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:00.599+0000 7fb049360700 1 -- 172.21.15.107:0/2473750705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0440fbd40 msgr2=0x7fb04410d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:00.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:00.599+0000 7fb049360700 1 --2- 172.21.15.107:0/2473750705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0440fbd40 0x7fb04410d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:00.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:00.599+0000 7fb049360700 1 -- 172.21.15.107:0/2473750705 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb034005040 con 0x7fb0440fcff0 2024-04-22T23:36:00.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:00.599+0000 7fb049b61700 1 --2- 172.21.15.107:0/2473750705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0440fbd40 0x7fb04410d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T23:36:00.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:00.599+0000 7fb049360700 1 --2- 172.21.15.107:0/2473750705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0440fcff0 0x7fb04410df20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb034013870 tx=0x7fb034009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:00.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:00.599+0000 7fb03a7fc700 1 -- 172.21.15.107:0/2473750705 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb034021070 con 0x7fb0440fcff0 2024-04-22T23:36:00.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:00.599+0000 7fb04bdc5700 1 -- 172.21.15.107:0/2473750705 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb04410efa0 con 0x7fb0440fcff0 2024-04-22T23:36:00.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:00.599+0000 7fb03a7fc700 1 -- 172.21.15.107:0/2473750705 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb034009430 con 0x7fb0440fcff0 2024-04-22T23:36:00.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:00.599+0000 7fb03a7fc700 1 -- 172.21.15.107:0/2473750705 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb03400e070 con 0x7fb0440fcff0 2024-04-22T23:36:00.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:00.599+0000 7fb04bdc5700 1 -- 172.21.15.107:0/2473750705 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb04410f4e0 con 0x7fb0440fcff0 2024-04-22T23:36:00.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:00.601+0000 7fb04bdc5700 1 -- 172.21.15.107:0/2473750705 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb044101b80 con 0x7fb0440fcff0 2024-04-22T23:36:00.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:00.601+0000 7fb03a7fc700 1 -- 172.21.15.107:0/2473750705 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fb03401c750 con 0x7fb0440fcff0 2024-04-22T23:36:00.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:00.602+0000 7fb03a7fc700 1 --2- 172.21.15.107:0/2473750705 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fb0300692a0 0x7fb03006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:00.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:00.602+0000 7fb03a7fc700 1 -- 172.21.15.107:0/2473750705 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fb034088480 con 0x7fb0440fcff0 2024-04-22T23:36:00.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:00.602+0000 7fb049b61700 1 --2- 172.21.15.107:0/2473750705 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fb0300692a0 0x7fb03006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:00.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:00.602+0000 7fb049b61700 1 --2- 172.21.15.107:0/2473750705 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fb0300692a0 0x7fb03006b750 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb040009b80 tx=0x7fb04000b040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:00.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:00.607+0000 7fb03a7fc700 1 -- 172.21.15.107:0/2473750705 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb0340581d0 con 0x7fb0440fcff0 2024-04-22T23:36:00.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:00.911+0000 7fb04bdc5700 1 -- 172.21.15.107:0/2473750705 --> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb044104930 con 0x7fb0300692a0 2024-04-22T23:36:00.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:00.914+0000 7fb03a7fc700 1 -- 172.21.15.107:0/2473750705 <== mgr.14206 v2:172.21.15.107:6800/2516490111 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7fb044104930 con 0x7fb0300692a0 2024-04-22T23:36:00.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:00.917+0000 7fb04bdc5700 1 -- 172.21.15.107:0/2473750705 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fb0300692a0 msgr2=0x7fb03006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:00.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:00.917+0000 7fb04bdc5700 1 --2- 172.21.15.107:0/2473750705 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fb0300692a0 0x7fb03006b750 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb040009b80 tx=0x7fb04000b040 comp rx=0 tx=0).stop 2024-04-22T23:36:00.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:00.917+0000 7fb04bdc5700 1 -- 172.21.15.107:0/2473750705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0440fcff0 msgr2=0x7fb04410df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:00.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:00.917+0000 7fb04bdc5700 1 --2- 172.21.15.107:0/2473750705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0440fcff0 0x7fb04410df20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb034013870 tx=0x7fb034009e30 comp rx=0 tx=0).stop 2024-04-22T23:36:00.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:00.918+0000 7fb04bdc5700 1 -- 172.21.15.107:0/2473750705 shutdown_connections 2024-04-22T23:36:00.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:00.918+0000 7fb04bdc5700 1 --2- 172.21.15.107:0/2473750705 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7fb0300692a0 0x7fb03006b750 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:00.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:00.918+0000 7fb04bdc5700 1 --2- 172.21.15.107:0/2473750705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb0440fbd40 0x7fb04410d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:00.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:00.918+0000 7fb04bdc5700 1 --2- 172.21.15.107:0/2473750705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0440fcff0 0x7fb04410df20 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:00.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:00.918+0000 7fb04bdc5700 1 -- 172.21.15.107:0/2473750705 >> 172.21.15.107:0/2473750705 conn(0x7fb0440f74c0 msgr2=0x7fb044065b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:00.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:00.918+0000 7fb04bdc5700 1 -- 172.21.15.107:0/2473750705 shutdown_connections 2024-04-22T23:36:00.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:00.918+0000 7fb04bdc5700 1 -- 172.21.15.107:0/2473750705 wait complete. 2024-04-22T23:36:00.932 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-04-22T23:36:01.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.086+0000 7f86c5c85700 1 -- 172.21.15.107:0/2038703105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86c00fce50 msgr2=0x7f86c00fd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:01.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.086+0000 7f86c5c85700 1 --2- 172.21.15.107:0/2038703105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86c00fce50 0x7f86c00fd260 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f86b8004c20 tx=0x7f86b8009ca0 comp rx=0 tx=0).stop 2024-04-22T23:36:01.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.086+0000 7f86c5c85700 1 -- 172.21.15.107:0/2038703105 shutdown_connections 2024-04-22T23:36:01.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.086+0000 7f86c5c85700 1 --2- 172.21.15.107:0/2038703105 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f86c00fe050 0x7f86c00fe4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:01.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.086+0000 7f86c5c85700 1 --2- 172.21.15.107:0/2038703105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86c00fce50 0x7f86c00fd260 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:01.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.086+0000 7f86c5c85700 1 -- 172.21.15.107:0/2038703105 >> 172.21.15.107:0/2038703105 conn(0x7f86c00f8580 msgr2=0x7f86c00fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:01.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.086+0000 7f86c5c85700 1 -- 172.21.15.107:0/2038703105 shutdown_connections 2024-04-22T23:36:01.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.086+0000 7f86c5c85700 1 -- 172.21.15.107:0/2038703105 wait complete. 2024-04-22T23:36:01.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.087+0000 7f86c5c85700 1 Processor -- start 2024-04-22T23:36:01.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.088+0000 7f86c5c85700 1 -- start start 2024-04-22T23:36:01.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.088+0000 7f86c5c85700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86c00fce50 0x7f86c010d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:01.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.088+0000 7f86c5c85700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f86c00fe050 0x7f86c010df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:01.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.088+0000 7f86c5c85700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c010e520 con 0x7f86c00fe050 2024-04-22T23:36:01.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.088+0000 7f86c5c85700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c010ed70 con 0x7f86c00fce50 2024-04-22T23:36:01.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.089+0000 7f86beffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f86c00fe050 0x7f86c010df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:01.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.089+0000 7f86beffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f86c00fe050 0x7f86c010df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59224/0 (socket says 172.21.15.107:59224) 2024-04-22T23:36:01.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.089+0000 7f86beffd700 1 -- 172.21.15.107:0/2008037123 learned_addr learned my addr 172.21.15.107:0/2008037123 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:36:01.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.089+0000 7f86bf7fe700 1 --2- 172.21.15.107:0/2008037123 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86c00fce50 0x7f86c010d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:01.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.089+0000 7f86bf7fe700 1 -- 172.21.15.107:0/2008037123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f86c00fe050 msgr2=0x7f86c010df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:01.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.089+0000 7f86bf7fe700 1 --2- 172.21.15.107:0/2008037123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f86c00fe050 0x7f86c010df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:01.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.089+0000 7f86bf7fe700 1 -- 172.21.15.107:0/2008037123 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86b80048f0 con 0x7f86c00fce50 2024-04-22T23:36:01.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.090+0000 7f86bf7fe700 1 --2- 172.21.15.107:0/2008037123 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86c00fce50 0x7f86c010d9e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f86b800ba20 tx=0x7f86b8009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:01.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.090+0000 7f86a7fff700 1 -- 172.21.15.107:0/2008037123 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86b800e040 con 0x7f86c00fce50 2024-04-22T23:36:01.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.090+0000 7f86a7fff700 1 -- 172.21.15.107:0/2008037123 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f86b801c910 con 0x7f86c00fce50 2024-04-22T23:36:01.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.090+0000 7f86a7fff700 1 -- 172.21.15.107:0/2008037123 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86b80129c0 con 0x7f86c00fce50 2024-04-22T23:36:01.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.090+0000 7f86c5c85700 1 -- 172.21.15.107:0/2008037123 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86c010eff0 con 0x7f86c00fce50 2024-04-22T23:36:01.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.091+0000 7f86c5c85700 1 -- 172.21.15.107:0/2008037123 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86c010f500 con 0x7f86c00fce50 2024-04-22T23:36:01.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.092+0000 7f86c5c85700 1 -- 172.21.15.107:0/2008037123 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86c005f5f0 con 0x7f86c00fce50 2024-04-22T23:36:01.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.093+0000 7f86a7fff700 1 -- 172.21.15.107:0/2008037123 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f86b801ca80 con 0x7f86c00fce50 2024-04-22T23:36:01.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.093+0000 7f86a7fff700 1 --2- 172.21.15.107:0/2008037123 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f86a80692a0 0x7f86a806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:01.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.093+0000 7f86a7fff700 1 -- 172.21.15.107:0/2008037123 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f86b8089210 con 0x7f86c00fce50 2024-04-22T23:36:01.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.094+0000 7f86beffd700 1 --2- 172.21.15.107:0/2008037123 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f86a80692a0 0x7f86a806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:01.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.096+0000 7f86beffd700 1 --2- 172.21.15.107:0/2008037123 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f86a80692a0 0x7f86a806b750 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f86b0009b80 tx=0x7f86b000b040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:01.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.101+0000 7f86a7fff700 1 -- 172.21.15.107:0/2008037123 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f86b8058ee0 con 0x7f86c00fce50 2024-04-22T23:36:01.172 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:01 smithi107 ceph-mon[26355]: Upgrade: Updating mgr.smithi157.tuylis 2024-04-22T23:36:01.172 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:01 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:36:01.172 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:01 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi157.tuylis", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T23:36:01.172 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:01 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T23:36:01.172 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:01 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:36:01.172 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:01 smithi107 ceph-mon[26355]: Deploying daemon mgr.smithi157.tuylis on smithi157 2024-04-22T23:36:01.172 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:01 smithi107 ceph-mon[26355]: pgmap v287: 65 pgs: 65 active+clean; 285 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 43 KiB/s rd, 1.5 MiB/s wr, 379 op/s 2024-04-22T23:36:01.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:01 smithi157 ceph-mon[33877]: Upgrade: Updating mgr.smithi157.tuylis 2024-04-22T23:36:01.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:01 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:36:01.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:01 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi157.tuylis", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T23:36:01.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:01 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T23:36:01.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:01 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:36:01.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:01 smithi157 ceph-mon[33877]: Deploying daemon mgr.smithi157.tuylis on smithi157 2024-04-22T23:36:01.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:01 smithi157 ceph-mon[33877]: pgmap v287: 65 pgs: 65 active+clean; 285 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 43 KiB/s rd, 1.5 MiB/s wr, 379 op/s 2024-04-22T23:36:01.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.405+0000 7f86c5c85700 1 -- 172.21.15.107:0/2008037123 --> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f86c010f7e0 con 0x7f86a80692a0 2024-04-22T23:36:01.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.406+0000 7f86a7fff700 1 -- 172.21.15.107:0/2008037123 <== mgr.14206 v2:172.21.15.107:6800/2516490111 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7f86c010f7e0 con 0x7f86a80692a0 2024-04-22T23:36:01.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.408+0000 7f86c5c85700 1 -- 172.21.15.107:0/2008037123 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f86a80692a0 msgr2=0x7f86a806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:01.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.408+0000 7f86c5c85700 1 --2- 172.21.15.107:0/2008037123 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f86a80692a0 0x7f86a806b750 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f86b0009b80 tx=0x7f86b000b040 comp rx=0 tx=0).stop 2024-04-22T23:36:01.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.408+0000 7f86c5c85700 1 -- 172.21.15.107:0/2008037123 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86c00fce50 msgr2=0x7f86c010d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:01.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.409+0000 7f86c5c85700 1 --2- 172.21.15.107:0/2008037123 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86c00fce50 0x7f86c010d9e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f86b800ba20 tx=0x7f86b8009ca0 comp rx=0 tx=0).stop 2024-04-22T23:36:01.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.409+0000 7f86c5c85700 1 -- 172.21.15.107:0/2008037123 shutdown_connections 2024-04-22T23:36:01.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.409+0000 7f86c5c85700 1 --2- 172.21.15.107:0/2008037123 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f86a80692a0 0x7f86a806b750 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:01.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.409+0000 7f86c5c85700 1 --2- 172.21.15.107:0/2008037123 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86c00fce50 0x7f86c010d9e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:01.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.409+0000 7f86c5c85700 1 --2- 172.21.15.107:0/2008037123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f86c00fe050 0x7f86c010df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:01.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.409+0000 7f86c5c85700 1 -- 172.21.15.107:0/2008037123 >> 172.21.15.107:0/2008037123 conn(0x7f86c00f8580 msgr2=0x7f86c00f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:01.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.409+0000 7f86c5c85700 1 -- 172.21.15.107:0/2008037123 shutdown_connections 2024-04-22T23:36:01.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.409+0000 7f86c5c85700 1 -- 172.21.15.107:0/2008037123 wait complete. 2024-04-22T23:36:01.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.529+0000 7f082a611700 1 -- 172.21.15.107:0/514617238 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08240fff70 msgr2=0x7f0824102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:01.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.529+0000 7f082a611700 1 --2- 172.21.15.107:0/514617238 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08240fff70 0x7f0824102390 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0818004940 tx=0x7f0818009e30 comp rx=0 tx=0).stop 2024-04-22T23:36:01.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.529+0000 7f082a611700 1 -- 172.21.15.107:0/514617238 shutdown_connections 2024-04-22T23:36:01.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.529+0000 7f082a611700 1 --2- 172.21.15.107:0/514617238 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08240fff70 0x7f0824102390 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:01.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.529+0000 7f082a611700 1 --2- 172.21.15.107:0/514617238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08240662f0 0x7f08240ffa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:01.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.529+0000 7f082a611700 1 -- 172.21.15.107:0/514617238 >> 172.21.15.107:0/514617238 conn(0x7f08240f74c0 msgr2=0x7f08240f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:01.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.529+0000 7f082a611700 1 -- 172.21.15.107:0/514617238 shutdown_connections 2024-04-22T23:36:01.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.529+0000 7f082a611700 1 -- 172.21.15.107:0/514617238 wait complete. 2024-04-22T23:36:01.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.530+0000 7f082a611700 1 Processor -- start 2024-04-22T23:36:01.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.530+0000 7f082a611700 1 -- start start 2024-04-22T23:36:01.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.530+0000 7f082a611700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08240662f0 0x7f0824067d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:01.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.530+0000 7f082a611700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08240fff70 0x7f08240682d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:01.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.530+0000 7f082a611700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0824068920 con 0x7f08240fff70 2024-04-22T23:36:01.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.530+0000 7f082a611700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0824069170 con 0x7f08240662f0 2024-04-22T23:36:01.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.531+0000 7f08237fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08240fff70 0x7f08240682d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:01.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.531+0000 7f0823fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08240662f0 0x7f0824067d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:01.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.531+0000 7f0823fff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08240662f0 0x7f0824067d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.107:57184/0 (socket says 172.21.15.107:57184) 2024-04-22T23:36:01.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.531+0000 7f0823fff700 1 -- 172.21.15.107:0/366151929 learned_addr learned my addr 172.21.15.107:0/366151929 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:36:01.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.531+0000 7f0823fff700 1 -- 172.21.15.107:0/366151929 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08240fff70 msgr2=0x7f08240682d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:01.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.531+0000 7f0823fff700 1 --2- 172.21.15.107:0/366151929 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08240fff70 0x7f08240682d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:01.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.531+0000 7f0823fff700 1 -- 172.21.15.107:0/366151929 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0818005040 con 0x7f08240662f0 2024-04-22T23:36:01.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.531+0000 7f08237fe700 1 --2- 172.21.15.107:0/366151929 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08240fff70 0x7f08240682d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T23:36:01.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.531+0000 7f0823fff700 1 --2- 172.21.15.107:0/366151929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08240662f0 0x7f0824067d90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0814002a80 tx=0x7f0814004630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:01.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.531+0000 7f0820ff9700 1 -- 172.21.15.107:0/366151929 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0814004e80 con 0x7f08240662f0 2024-04-22T23:36:01.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.531+0000 7f082a611700 1 -- 172.21.15.107:0/366151929 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0824069450 con 0x7f08240662f0 2024-04-22T23:36:01.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.531+0000 7f0820ff9700 1 -- 172.21.15.107:0/366151929 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f081400f040 con 0x7f08240662f0 2024-04-22T23:36:01.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.531+0000 7f0820ff9700 1 -- 172.21.15.107:0/366151929 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f081400daa0 con 0x7f08240662f0 2024-04-22T23:36:01.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.531+0000 7f082a611700 1 -- 172.21.15.107:0/366151929 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0824069a10 con 0x7f08240662f0 2024-04-22T23:36:01.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.532+0000 7f082a611700 1 -- 172.21.15.107:0/366151929 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f082405f5f0 con 0x7f08240662f0 2024-04-22T23:36:01.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.533+0000 7f0820ff9700 1 -- 172.21.15.107:0/366151929 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f081400dc00 con 0x7f08240662f0 2024-04-22T23:36:01.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.533+0000 7f0820ff9700 1 --2- 172.21.15.107:0/366151929 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f080c0692a0 0x7f080c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:01.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.533+0000 7f0820ff9700 1 -- 172.21.15.107:0/366151929 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f08140868e0 con 0x7f08240662f0 2024-04-22T23:36:01.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.533+0000 7f08237fe700 1 --2- 172.21.15.107:0/366151929 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f080c0692a0 0x7f080c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:01.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.534+0000 7f08237fe700 1 --2- 172.21.15.107:0/366151929 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f080c0692a0 0x7f080c06b750 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f081800b970 tx=0x7f0818009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:01.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.538+0000 7f0820ff9700 1 -- 172.21.15.107:0/366151929 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0814052aa0 con 0x7f08240662f0 2024-04-22T23:36:01.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.760+0000 7f082a611700 1 -- 172.21.15.107:0/366151929 --> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0824069c80 con 0x7f080c0692a0 2024-04-22T23:36:01.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.770+0000 7f0820ff9700 1 -- 172.21.15.107:0/366151929 <== mgr.14206 v2:172.21.15.107:6800/2516490111 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3366 (secure 0 0 0) 0x7f0824069c80 con 0x7f080c0692a0 2024-04-22T23:36:01.765 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-22T23:36:01.765 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (7m) 3m ago 8m 25.5M - 0.25.0 c8568f914cd2 4d90ceb89249 2024-04-22T23:36:01.765 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (9m) 3m ago 9m 6660k - 17.2.7-884-g70d0cb4b cafd8ddac8fa c10611c756e4 2024-04-22T23:36:01.765 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi157 smithi157 running (8m) 3m ago 8m 6669k - 17.2.7-884-g70d0cb4b cafd8ddac8fa 55d2d46294c9 2024-04-22T23:36:01.765 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (7m) 3m ago 8m 86.8M - 9.4.7 954c08fa6188 c2d1f8de1aa7 2024-04-22T23:36:01.765 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.dfvvkw smithi107 running (3m) 3m ago 3m 10.4M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 08d3aa8d717d 2024-04-22T23:36:01.765 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.ktuxjl smithi107 running (3m) 3m ago 3m 16.9M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 762f27fe0f30 2024-04-22T23:36:01.765 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi157.rcaris smithi157 running (3m) 3m ago 3m 16.8M - 17.2.7-884-g70d0cb4b cafd8ddac8fa d23366ac085d 2024-04-22T23:36:01.765 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi157.ufywse smithi157 running (3m) 3m ago 3m 11.0M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 8b00d61eb355 2024-04-22T23:36:01.765 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.uipxvm smithi107 *:9283 running (10m) 3m ago 10m 481M - 17.2.7-884-g70d0cb4b cafd8ddac8fa db96e8cbbdfb 2024-04-22T23:36:01.765 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi157.tuylis smithi157 *:8443,9283 running (8m) 3m ago 8m 431M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 0c12be10ffaf 2024-04-22T23:36:01.765 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (10m) 3m ago 10m 49.6M 2048M 17.2.7-884-g70d0cb4b cafd8ddac8fa 8314962fa1b9 2024-04-22T23:36:01.765 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi157 smithi157 running (8m) 3m ago 8m 36.6M 2048M 17.2.7-884-g70d0cb4b cafd8ddac8fa 0747a85008d9 2024-04-22T23:36:01.765 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (9m) 3m ago 9m 16.0M - 1.5.0 0da6a335fe13 8a7c595c114f 2024-04-22T23:36:01.766 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi157 smithi157 *:9100 running (8m) 3m ago 8m 16.1M - 1.5.0 0da6a335fe13 940164771fe3 2024-04-22T23:36:01.766 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (7m) 3m ago 7m 46.6M 4288M 17.2.7-884-g70d0cb4b cafd8ddac8fa 18a179b0a618 2024-04-22T23:36:01.766 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (6m) 3m ago 6m 45.9M 4288M 17.2.7-884-g70d0cb4b cafd8ddac8fa c7272eb84ddb 2024-04-22T23:36:01.766 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (6m) 3m ago 6m 46.2M 4288M 17.2.7-884-g70d0cb4b cafd8ddac8fa 9db1788096ad 2024-04-22T23:36:01.766 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi157 running (5m) 3m ago 5m 43.9M 5312M 17.2.7-884-g70d0cb4b cafd8ddac8fa c9fb2d1a13dd 2024-04-22T23:36:01.766 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi157 running (5m) 3m ago 5m 44.2M 5312M 17.2.7-884-g70d0cb4b cafd8ddac8fa 24a16e19951f 2024-04-22T23:36:01.766 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi157 running (5m) 3m ago 5m 43.1M 5312M 17.2.7-884-g70d0cb4b cafd8ddac8fa 4c5b7a7ef8bd 2024-04-22T23:36:01.766 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (7m) 3m ago 8m 51.2M - 2.43.0 a07b618ecd1d 8c5df40caabe 2024-04-22T23:36:01.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.772+0000 7f082a611700 1 -- 172.21.15.107:0/366151929 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f080c0692a0 msgr2=0x7f080c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:01.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.772+0000 7f082a611700 1 --2- 172.21.15.107:0/366151929 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f080c0692a0 0x7f080c06b750 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f081800b970 tx=0x7f0818009e30 comp rx=0 tx=0).stop 2024-04-22T23:36:01.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.772+0000 7f082a611700 1 -- 172.21.15.107:0/366151929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08240662f0 msgr2=0x7f0824067d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:01.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.772+0000 7f082a611700 1 --2- 172.21.15.107:0/366151929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08240662f0 0x7f0824067d90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0814002a80 tx=0x7f0814004630 comp rx=0 tx=0).stop 2024-04-22T23:36:01.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.772+0000 7f082a611700 1 -- 172.21.15.107:0/366151929 shutdown_connections 2024-04-22T23:36:01.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.772+0000 7f082a611700 1 --2- 172.21.15.107:0/366151929 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f080c0692a0 0x7f080c06b750 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:01.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.772+0000 7f082a611700 1 --2- 172.21.15.107:0/366151929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08240662f0 0x7f0824067d90 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:01.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.772+0000 7f082a611700 1 --2- 172.21.15.107:0/366151929 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08240fff70 0x7f08240682d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:01.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.772+0000 7f082a611700 1 -- 172.21.15.107:0/366151929 >> 172.21.15.107:0/366151929 conn(0x7f08240f74c0 msgr2=0x7f08240f7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:01.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.772+0000 7f082a611700 1 -- 172.21.15.107:0/366151929 shutdown_connections 2024-04-22T23:36:01.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.772+0000 7f082a611700 1 -- 172.21.15.107:0/366151929 wait complete. 2024-04-22T23:36:01.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.912+0000 7f9bd8301700 1 -- 172.21.15.107:0/3208406035 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bd00fcff0 msgr2=0x7f9bd00fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:01.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.912+0000 7f9bd8301700 1 --2- 172.21.15.107:0/3208406035 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bd00fcff0 0x7f9bd00fd460 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4004940 tx=0x7f9bc4009e30 comp rx=0 tx=0).stop 2024-04-22T23:36:01.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.912+0000 7f9bd8301700 1 -- 172.21.15.107:0/3208406035 shutdown_connections 2024-04-22T23:36:01.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.912+0000 7f9bd8301700 1 --2- 172.21.15.107:0/3208406035 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bd00fcff0 0x7f9bd00fd460 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:01.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.912+0000 7f9bd8301700 1 --2- 172.21.15.107:0/3208406035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bd00fbd40 0x7f9bd00fc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:01.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.912+0000 7f9bd8301700 1 -- 172.21.15.107:0/3208406035 >> 172.21.15.107:0/3208406035 conn(0x7f9bd00f74c0 msgr2=0x7f9bd00f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:01.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.912+0000 7f9bd8301700 1 -- 172.21.15.107:0/3208406035 shutdown_connections 2024-04-22T23:36:01.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.912+0000 7f9bd8301700 1 -- 172.21.15.107:0/3208406035 wait complete. 2024-04-22T23:36:01.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.912+0000 7f9bd8301700 1 Processor -- start 2024-04-22T23:36:01.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.912+0000 7f9bd8301700 1 -- start start 2024-04-22T23:36:01.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.913+0000 7f9bd8301700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bd00fbd40 0x7f9bd010d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:01.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.913+0000 7f9bd8301700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bd00fcff0 0x7f9bd010df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:01.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.913+0000 7f9bd8301700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd010e520 con 0x7f9bd00fcff0 2024-04-22T23:36:01.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.913+0000 7f9bd8301700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd010ed20 con 0x7f9bd00fbd40 2024-04-22T23:36:01.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.913+0000 7f9bd589c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bd00fcff0 0x7f9bd010df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:01.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.913+0000 7f9bd609d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bd00fbd40 0x7f9bd010d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:01.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.913+0000 7f9bd589c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bd00fcff0 0x7f9bd010df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46374/0 (socket says 172.21.15.107:46374) 2024-04-22T23:36:01.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.913+0000 7f9bd589c700 1 -- 172.21.15.107:0/2786259990 learned_addr learned my addr 172.21.15.107:0/2786259990 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:36:01.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.913+0000 7f9bd589c700 1 -- 172.21.15.107:0/2786259990 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bd00fbd40 msgr2=0x7f9bd010d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:01.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.913+0000 7f9bd589c700 1 --2- 172.21.15.107:0/2786259990 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bd00fbd40 0x7f9bd010d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:01.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.913+0000 7f9bd589c700 1 -- 172.21.15.107:0/2786259990 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc4005040 con 0x7f9bd00fcff0 2024-04-22T23:36:01.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.913+0000 7f9bd609d700 1 --2- 172.21.15.107:0/2786259990 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bd00fbd40 0x7f9bd010d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T23:36:01.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.913+0000 7f9bd589c700 1 --2- 172.21.15.107:0/2786259990 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bd00fcff0 0x7f9bd010df20 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4009150 tx=0x7f9bc4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:01.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.913+0000 7f9bceffd700 1 -- 172.21.15.107:0/2786259990 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9bc400e040 con 0x7f9bd00fcff0 2024-04-22T23:36:01.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.914+0000 7f9bd8301700 1 -- 172.21.15.107:0/2786259990 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bd010efa0 con 0x7f9bd00fcff0 2024-04-22T23:36:01.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.914+0000 7f9bceffd700 1 -- 172.21.15.107:0/2786259990 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9bc4004290 con 0x7f9bd00fcff0 2024-04-22T23:36:01.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.914+0000 7f9bceffd700 1 -- 172.21.15.107:0/2786259990 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9bc401c630 con 0x7f9bd00fcff0 2024-04-22T23:36:01.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.914+0000 7f9bd8301700 1 -- 172.21.15.107:0/2786259990 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bd010f4e0 con 0x7f9bd00fcff0 2024-04-22T23:36:01.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.915+0000 7f9bceffd700 1 -- 172.21.15.107:0/2786259990 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f9bc400bbf0 con 0x7f9bd00fcff0 2024-04-22T23:36:01.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.915+0000 7f9bd8301700 1 -- 172.21.15.107:0/2786259990 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bd0101b80 con 0x7f9bd00fcff0 2024-04-22T23:36:01.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.916+0000 7f9bceffd700 1 --2- 172.21.15.107:0/2786259990 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f9bb80692a0 0x7f9bb806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:01.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.916+0000 7f9bceffd700 1 -- 172.21.15.107:0/2786259990 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f9bc4087ec0 con 0x7f9bd00fcff0 2024-04-22T23:36:01.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.916+0000 7f9bd609d700 1 --2- 172.21.15.107:0/2786259990 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f9bb80692a0 0x7f9bb806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:01.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.916+0000 7f9bd609d700 1 --2- 172.21.15.107:0/2786259990 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f9bb80692a0 0x7f9bb806b750 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8007e50 tx=0x7f9bc8009e20 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:01.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:01.921+0000 7f9bceffd700 1 -- 172.21.15.107:0/2786259990 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f9bc4057c30 con 0x7f9bd00fcff0 2024-04-22T23:36:02.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:02 smithi107 ceph-mon[26355]: from='client.14692 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:36:02.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:02 smithi107 ceph-mon[26355]: from='client.24447 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:36:02.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.271+0000 7f9bd8301700 1 -- 172.21.15.107:0/2786259990 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9bd005f5f0 con 0x7f9bd00fcff0 2024-04-22T23:36:02.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.272+0000 7f9bceffd700 1 -- 172.21.15.107:0/2786259990 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7f9bc40577c0 con 0x7f9bd00fcff0 2024-04-22T23:36:02.266 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-22T23:36:02.266 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-04-22T23:36:02.266 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 2 2024-04-22T23:36:02.267 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:36:02.267 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-04-22T23:36:02.267 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 2 2024-04-22T23:36:02.267 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:36:02.267 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-04-22T23:36:02.267 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 6 2024-04-22T23:36:02.267 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:36:02.267 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-04-22T23:36:02.267 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 4 2024-04-22T23:36:02.267 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:36:02.267 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-04-22T23:36:02.267 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 14 2024-04-22T23:36:02.267 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T23:36:02.267 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-22T23:36:02.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.275+0000 7f9bd8301700 1 -- 172.21.15.107:0/2786259990 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f9bb80692a0 msgr2=0x7f9bb806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:02.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.275+0000 7f9bd8301700 1 --2- 172.21.15.107:0/2786259990 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f9bb80692a0 0x7f9bb806b750 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8007e50 tx=0x7f9bc8009e20 comp rx=0 tx=0).stop 2024-04-22T23:36:02.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.275+0000 7f9bd8301700 1 -- 172.21.15.107:0/2786259990 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bd00fcff0 msgr2=0x7f9bd010df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:02.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.275+0000 7f9bd8301700 1 --2- 172.21.15.107:0/2786259990 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bd00fcff0 0x7f9bd010df20 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4009150 tx=0x7f9bc4009e30 comp rx=0 tx=0).stop 2024-04-22T23:36:02.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.275+0000 7f9bd8301700 1 -- 172.21.15.107:0/2786259990 shutdown_connections 2024-04-22T23:36:02.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.275+0000 7f9bd8301700 1 --2- 172.21.15.107:0/2786259990 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f9bb80692a0 0x7f9bb806b750 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:02.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.275+0000 7f9bd8301700 1 --2- 172.21.15.107:0/2786259990 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bd00fbd40 0x7f9bd010d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:02.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.275+0000 7f9bd8301700 1 --2- 172.21.15.107:0/2786259990 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bd00fcff0 0x7f9bd010df20 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:02.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.275+0000 7f9bd8301700 1 -- 172.21.15.107:0/2786259990 >> 172.21.15.107:0/2786259990 conn(0x7f9bd00f74c0 msgr2=0x7f9bd0065b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:02.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.275+0000 7f9bd8301700 1 -- 172.21.15.107:0/2786259990 shutdown_connections 2024-04-22T23:36:02.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.275+0000 7f9bd8301700 1 -- 172.21.15.107:0/2786259990 wait complete. 2024-04-22T23:36:02.354 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:02 smithi157 ceph-mon[33877]: from='client.14692 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:36:02.354 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:02 smithi157 ceph-mon[33877]: from='client.24447 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:36:02.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.393+0000 7f79eb2c1700 1 -- 172.21.15.107:0/3891136066 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79e40fe0e0 msgr2=0x7f79e40fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:02.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.393+0000 7f79eb2c1700 1 --2- 172.21.15.107:0/3891136066 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79e40fe0e0 0x7f79e40fe550 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f79d4004940 tx=0x7f79d4009e30 comp rx=0 tx=0).stop 2024-04-22T23:36:02.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.393+0000 7f79eb2c1700 1 -- 172.21.15.107:0/3891136066 shutdown_connections 2024-04-22T23:36:02.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.393+0000 7f79eb2c1700 1 --2- 172.21.15.107:0/3891136066 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79e40fe0e0 0x7f79e40fe550 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:02.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.393+0000 7f79eb2c1700 1 --2- 172.21.15.107:0/3891136066 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79e40fcef0 0x7f79e40fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:02.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.393+0000 7f79eb2c1700 1 -- 172.21.15.107:0/3891136066 >> 172.21.15.107:0/3891136066 conn(0x7f79e40f8530 msgr2=0x7f79e40fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:02.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.393+0000 7f79eb2c1700 1 -- 172.21.15.107:0/3891136066 shutdown_connections 2024-04-22T23:36:02.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.393+0000 7f79eb2c1700 1 -- 172.21.15.107:0/3891136066 wait complete. 2024-04-22T23:36:02.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.394+0000 7f79eb2c1700 1 Processor -- start 2024-04-22T23:36:02.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.394+0000 7f79eb2c1700 1 -- start start 2024-04-22T23:36:02.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.394+0000 7f79eb2c1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79e40fcef0 0x7f79e410daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:02.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.394+0000 7f79eb2c1700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79e40fe0e0 0x7f79e410dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:02.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.394+0000 7f79eb2c1700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e410e5e0 con 0x7f79e40fcef0 2024-04-22T23:36:02.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.394+0000 7f79eb2c1700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e410ee30 con 0x7f79e40fe0e0 2024-04-22T23:36:02.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.394+0000 7f79e905d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79e40fcef0 0x7f79e410daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:02.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.394+0000 7f79e905d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79e40fcef0 0x7f79e410daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46382/0 (socket says 172.21.15.107:46382) 2024-04-22T23:36:02.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.394+0000 7f79e905d700 1 -- 172.21.15.107:0/1166052558 learned_addr learned my addr 172.21.15.107:0/1166052558 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:36:02.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.394+0000 7f79e885c700 1 --2- 172.21.15.107:0/1166052558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79e40fe0e0 0x7f79e410dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:02.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.394+0000 7f79e905d700 1 -- 172.21.15.107:0/1166052558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79e40fe0e0 msgr2=0x7f79e410dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:02.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.394+0000 7f79e905d700 1 --2- 172.21.15.107:0/1166052558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79e40fe0e0 0x7f79e410dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:02.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.394+0000 7f79e905d700 1 -- 172.21.15.107:0/1166052558 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d4005040 con 0x7f79e40fcef0 2024-04-22T23:36:02.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.394+0000 7f79e905d700 1 --2- 172.21.15.107:0/1166052558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79e40fcef0 0x7f79e410daa0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f79e0002a50 tx=0x7f79e0004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:02.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.395+0000 7f79d9ffb700 1 -- 172.21.15.107:0/1166052558 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79e0007cc0 con 0x7f79e40fcef0 2024-04-22T23:36:02.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.395+0000 7f79d9ffb700 1 -- 172.21.15.107:0/1166052558 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f79e000f040 con 0x7f79e40fcef0 2024-04-22T23:36:02.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.395+0000 7f79eb2c1700 1 -- 172.21.15.107:0/1166052558 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79e410f110 con 0x7f79e40fcef0 2024-04-22T23:36:02.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.395+0000 7f79d9ffb700 1 -- 172.21.15.107:0/1166052558 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79e000da40 con 0x7f79e40fcef0 2024-04-22T23:36:02.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.395+0000 7f79eb2c1700 1 -- 172.21.15.107:0/1166052558 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79e410f650 con 0x7f79e40fcef0 2024-04-22T23:36:02.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.396+0000 7f79eb2c1700 1 -- 172.21.15.107:0/1166052558 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79e405f5f0 con 0x7f79e40fcef0 2024-04-22T23:36:02.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.397+0000 7f79d9ffb700 1 -- 172.21.15.107:0/1166052558 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f79e00097c0 con 0x7f79e40fcef0 2024-04-22T23:36:02.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.397+0000 7f79d9ffb700 1 --2- 172.21.15.107:0/1166052558 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f79d00692b0 0x7f79d006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:02.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.397+0000 7f79d9ffb700 1 -- 172.21.15.107:0/1166052558 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f79e0086f80 con 0x7f79e40fcef0 2024-04-22T23:36:02.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.397+0000 7f79e885c700 1 --2- 172.21.15.107:0/1166052558 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f79d00692b0 0x7f79d006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:02.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.398+0000 7f79e885c700 1 --2- 172.21.15.107:0/1166052558 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f79d00692b0 0x7f79d006b760 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f79d400b3b0 tx=0x7f79d4009e30 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:02.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.405+0000 7f79d9ffb700 1 -- 172.21.15.107:0/1166052558 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f79e00569e0 con 0x7f79e40fcef0 2024-04-22T23:36:02.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.710+0000 7f79eb2c1700 1 -- 172.21.15.107:0/1166052558 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f79e410fa30 con 0x7f79e40fcef0 2024-04-22T23:36:02.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.710+0000 7f79d9ffb700 1 -- 172.21.15.107:0/1166052558 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) v1 ==== 76+0+1858 (secure 0 0 0) 0x7f79e0056800 con 0x7f79e40fcef0 2024-04-22T23:36:02.705 INFO:teuthology.orchestra.run.smithi107.stdout:e19 2024-04-22T23:36:02.705 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T23:36:02.705 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:36:02.706 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-04-22T23:36:02.706 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:36:02.706 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-04-22T23:36:02.706 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-04-22T23:36:02.706 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 19 2024-04-22T23:36:02.706 INFO:teuthology.orchestra.run.smithi107.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-22T23:36:02.706 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-04-22T23:32:20.146130+0000 2024-04-22T23:36:02.706 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-04-22T23:35:12.338678+0000 2024-04-22T23:36:02.706 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-04-22T23:36:02.706 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-04-22T23:36:02.706 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-04-22T23:36:02.706 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-04-22T23:36:02.706 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-04-22T23:36:02.706 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-04-22T23:36:02.707 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-04-22T23:36:02.707 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 46 2024-04-22T23:36:02.707 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:36:02.707 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 2 2024-04-22T23:36:02.707 INFO:teuthology.orchestra.run.smithi107.stdout:in 0,1 2024-04-22T23:36:02.707 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=14468,1=24265} 2024-04-22T23:36:02.707 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-04-22T23:36:02.707 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-04-22T23:36:02.707 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-04-22T23:36:02.707 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-04-22T23:36:02.707 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-04-22T23:36:02.707 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data disabled 2024-04-22T23:36:02.707 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-04-22T23:36:02.707 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-04-22T23:36:02.708 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.ktuxjl{0:14468} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.107:6828/3335313253,v1:172.21.15.107:6829/3335313253] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:36:02.708 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi157.rcaris{1:24265} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.157:6824/2779251893,v1:172.21.15.157:6825/2779251893] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:36:02.708 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:36:02.708 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:36:02.708 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-04-22T23:36:02.708 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:36:02.708 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.dfvvkw{-1:14484} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/3548717247,v1:172.21.15.107:6827/3548717247] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:36:02.708 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi157.ufywse{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.157:6826/4072444199,v1:172.21.15.157:6827/4072444199] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:36:02.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.713+0000 7f79eb2c1700 1 -- 172.21.15.107:0/1166052558 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f79d00692b0 msgr2=0x7f79d006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:02.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.713+0000 7f79eb2c1700 1 --2- 172.21.15.107:0/1166052558 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f79d00692b0 0x7f79d006b760 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f79d400b3b0 tx=0x7f79d4009e30 comp rx=0 tx=0).stop 2024-04-22T23:36:02.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.713+0000 7f79eb2c1700 1 -- 172.21.15.107:0/1166052558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79e40fcef0 msgr2=0x7f79e410daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:02.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.713+0000 7f79eb2c1700 1 --2- 172.21.15.107:0/1166052558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79e40fcef0 0x7f79e410daa0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f79e0002a50 tx=0x7f79e0004600 comp rx=0 tx=0).stop 2024-04-22T23:36:02.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.713+0000 7f79eb2c1700 1 -- 172.21.15.107:0/1166052558 shutdown_connections 2024-04-22T23:36:02.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.713+0000 7f79eb2c1700 1 --2- 172.21.15.107:0/1166052558 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f79d00692b0 0x7f79d006b760 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:02.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.713+0000 7f79eb2c1700 1 --2- 172.21.15.107:0/1166052558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79e40fcef0 0x7f79e410daa0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:02.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.713+0000 7f79eb2c1700 1 --2- 172.21.15.107:0/1166052558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79e40fe0e0 0x7f79e410dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:02.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.713+0000 7f79eb2c1700 1 -- 172.21.15.107:0/1166052558 >> 172.21.15.107:0/1166052558 conn(0x7f79e40f8530 msgr2=0x7f79e40f9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:02.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.713+0000 7f79eb2c1700 1 -- 172.21.15.107:0/1166052558 shutdown_connections 2024-04-22T23:36:02.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.713+0000 7f79eb2c1700 1 -- 172.21.15.107:0/1166052558 wait complete. 2024-04-22T23:36:02.710 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 19 2024-04-22T23:36:02.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.832+0000 7f0b827b7700 1 -- 172.21.15.107:0/2989549431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b7c0fce40 msgr2=0x7f0b7c0fd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:02.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.832+0000 7f0b827b7700 1 --2- 172.21.15.107:0/2989549431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b7c0fce40 0x7f0b7c0fd250 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c004940 tx=0x7f0b6c009e30 comp rx=0 tx=0).stop 2024-04-22T23:36:02.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.832+0000 7f0b827b7700 1 -- 172.21.15.107:0/2989549431 shutdown_connections 2024-04-22T23:36:02.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.832+0000 7f0b827b7700 1 --2- 172.21.15.107:0/2989549431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b7c0fe040 0x7f0b7c0fe4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:02.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.832+0000 7f0b827b7700 1 --2- 172.21.15.107:0/2989549431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b7c0fce40 0x7f0b7c0fd250 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:02.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.832+0000 7f0b827b7700 1 -- 172.21.15.107:0/2989549431 >> 172.21.15.107:0/2989549431 conn(0x7f0b7c0f8570 msgr2=0x7f0b7c0fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:02.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.832+0000 7f0b827b7700 1 -- 172.21.15.107:0/2989549431 shutdown_connections 2024-04-22T23:36:02.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.832+0000 7f0b827b7700 1 -- 172.21.15.107:0/2989549431 wait complete. 2024-04-22T23:36:02.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.832+0000 7f0b827b7700 1 Processor -- start 2024-04-22T23:36:02.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.832+0000 7f0b827b7700 1 -- start start 2024-04-22T23:36:02.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.833+0000 7f0b827b7700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b7c0fce40 0x7f0b7c071e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:02.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.833+0000 7f0b827b7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b7c0fe040 0x7f0b7c070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:02.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.833+0000 7f0b827b7700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7c0708c0 con 0x7f0b7c0fe040 2024-04-22T23:36:02.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.833+0000 7f0b827b7700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7c070a30 con 0x7f0b7c0fce40 2024-04-22T23:36:02.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.833+0000 7f0b7b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b7c0fe040 0x7f0b7c070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:02.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.833+0000 7f0b7bfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b7c0fce40 0x7f0b7c071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:02.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.833+0000 7f0b7b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b7c0fe040 0x7f0b7c070310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46392/0 (socket says 172.21.15.107:46392) 2024-04-22T23:36:02.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.833+0000 7f0b7b7fe700 1 -- 172.21.15.107:0/3210705652 learned_addr learned my addr 172.21.15.107:0/3210705652 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:36:02.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.833+0000 7f0b7bfff700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b7c0fce40 0x7f0b7c071e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.107:57242/0 (socket says 172.21.15.107:57242) 2024-04-22T23:36:02.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.833+0000 7f0b7bfff700 1 -- 172.21.15.107:0/3210705652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b7c0fe040 msgr2=0x7f0b7c070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:02.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.833+0000 7f0b7bfff700 1 --2- 172.21.15.107:0/3210705652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b7c0fe040 0x7f0b7c070310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:02.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.833+0000 7f0b7bfff700 1 -- 172.21.15.107:0/3210705652 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b6c005040 con 0x7f0b7c0fce40 2024-04-22T23:36:02.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.833+0000 7f0b7b7fe700 1 --2- 172.21.15.107:0/3210705652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b7c0fe040 0x7f0b7c070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T23:36:02.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.833+0000 7f0b7bfff700 1 --2- 172.21.15.107:0/3210705652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b7c0fce40 0x7f0b7c071e40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c000c00 tx=0x7f0b6c009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:02.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.833+0000 7f0b78ff9700 1 -- 172.21.15.107:0/3210705652 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b6c00e040 con 0x7f0b7c0fce40 2024-04-22T23:36:02.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.834+0000 7f0b78ff9700 1 -- 172.21.15.107:0/3210705652 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0b6c00bbf0 con 0x7f0b7c0fce40 2024-04-22T23:36:02.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.834+0000 7f0b78ff9700 1 -- 172.21.15.107:0/3210705652 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b6c01c650 con 0x7f0b7c0fce40 2024-04-22T23:36:02.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.834+0000 7f0b827b7700 1 -- 172.21.15.107:0/3210705652 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b7c070cb0 con 0x7f0b7c0fce40 2024-04-22T23:36:02.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.834+0000 7f0b827b7700 1 -- 172.21.15.107:0/3210705652 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b7c115a00 con 0x7f0b7c0fce40 2024-04-22T23:36:02.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.835+0000 7f0b827b7700 1 -- 172.21.15.107:0/3210705652 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b7c05f5f0 con 0x7f0b7c0fce40 2024-04-22T23:36:02.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.835+0000 7f0b78ff9700 1 -- 172.21.15.107:0/3210705652 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f0b6c013420 con 0x7f0b7c0fce40 2024-04-22T23:36:02.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.835+0000 7f0b78ff9700 1 --2- 172.21.15.107:0/3210705652 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f0b640692a0 0x7f0b6406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:02.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.836+0000 7f0b78ff9700 1 -- 172.21.15.107:0/3210705652 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f0b6c088520 con 0x7f0b7c0fce40 2024-04-22T23:36:02.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.836+0000 7f0b7b7fe700 1 --2- 172.21.15.107:0/3210705652 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f0b640692a0 0x7f0b6406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:02.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.836+0000 7f0b7b7fe700 1 --2- 172.21.15.107:0/3210705652 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f0b640692a0 0x7f0b6406b750 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0b70004fd0 tx=0x7f0b7000b040 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:02.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:02.840+0000 7f0b78ff9700 1 -- 172.21.15.107:0/3210705652 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0b6c0547a0 con 0x7f0b7c0fce40 2024-04-22T23:36:03.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.139+0000 7f0b827b7700 1 -- 172.21.15.107:0/3210705652 --> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0b7c102360 con 0x7f0b640692a0 2024-04-22T23:36:03.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.140+0000 7f0b78ff9700 1 -- 172.21.15.107:0/3210705652 <== mgr.14206 v2:172.21.15.107:6800/2516490111 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7f0b7c102360 con 0x7f0b640692a0 2024-04-22T23:36:03.135 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-22T23:36:03.135 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c", 2024-04-22T23:36:03.135 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-04-22T23:36:03.135 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-22T23:36:03.135 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [], 2024-04-22T23:36:03.135 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "0/21 daemons upgraded", 2024-04-22T23:36:03.135 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Currently upgrading mgr daemons", 2024-04-22T23:36:03.135 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-04-22T23:36:03.136 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-22T23:36:03.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.144+0000 7f0b827b7700 1 -- 172.21.15.107:0/3210705652 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f0b640692a0 msgr2=0x7f0b6406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:03.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.144+0000 7f0b827b7700 1 --2- 172.21.15.107:0/3210705652 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f0b640692a0 0x7f0b6406b750 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0b70004fd0 tx=0x7f0b7000b040 comp rx=0 tx=0).stop 2024-04-22T23:36:03.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.144+0000 7f0b827b7700 1 -- 172.21.15.107:0/3210705652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b7c0fce40 msgr2=0x7f0b7c071e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:03.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.144+0000 7f0b827b7700 1 --2- 172.21.15.107:0/3210705652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b7c0fce40 0x7f0b7c071e40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c000c00 tx=0x7f0b6c009e30 comp rx=0 tx=0).stop 2024-04-22T23:36:03.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.144+0000 7f0b827b7700 1 -- 172.21.15.107:0/3210705652 shutdown_connections 2024-04-22T23:36:03.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.144+0000 7f0b827b7700 1 --2- 172.21.15.107:0/3210705652 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7f0b640692a0 0x7f0b6406b750 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:03.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.144+0000 7f0b827b7700 1 --2- 172.21.15.107:0/3210705652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b7c0fce40 0x7f0b7c071e40 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:03.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.144+0000 7f0b827b7700 1 --2- 172.21.15.107:0/3210705652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b7c0fe040 0x7f0b7c070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:03.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.144+0000 7f0b827b7700 1 -- 172.21.15.107:0/3210705652 >> 172.21.15.107:0/3210705652 conn(0x7f0b7c0f8570 msgr2=0x7f0b7c0f9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:03.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.145+0000 7f0b827b7700 1 -- 172.21.15.107:0/3210705652 shutdown_connections 2024-04-22T23:36:03.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.145+0000 7f0b827b7700 1 -- 172.21.15.107:0/3210705652 wait complete. 2024-04-22T23:36:03.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.293+0000 7ff4a1d1f700 1 -- 172.21.15.107:0/977442352 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff49c0662f0 msgr2=0x7ff49c0ffa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:03.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.293+0000 7ff4a1d1f700 1 --2- 172.21.15.107:0/977442352 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff49c0662f0 0x7ff49c0ffa30 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff48c004940 tx=0x7ff48c009e30 comp rx=0 tx=0).stop 2024-04-22T23:36:03.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.293+0000 7ff4a1d1f700 1 -- 172.21.15.107:0/977442352 shutdown_connections 2024-04-22T23:36:03.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.293+0000 7ff4a1d1f700 1 --2- 172.21.15.107:0/977442352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff49c0fff70 0x7ff49c102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:03.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.293+0000 7ff4a1d1f700 1 --2- 172.21.15.107:0/977442352 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff49c0662f0 0x7ff49c0ffa30 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:03.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.293+0000 7ff4a1d1f700 1 -- 172.21.15.107:0/977442352 >> 172.21.15.107:0/977442352 conn(0x7ff49c0f74c0 msgr2=0x7ff49c0f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:03.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.293+0000 7ff4a1d1f700 1 -- 172.21.15.107:0/977442352 shutdown_connections 2024-04-22T23:36:03.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.293+0000 7ff4a1d1f700 1 -- 172.21.15.107:0/977442352 wait complete. 2024-04-22T23:36:03.289 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:03 smithi107 ceph-mon[26355]: from='client.24451 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:36:03.289 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:03 smithi107 ceph-mon[26355]: pgmap v288: 65 pgs: 65 active+clean; 291 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 35 KiB/s rd, 2.0 MiB/s wr, 408 op/s 2024-04-22T23:36:03.289 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:03 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/2786259990' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:36:03.290 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:03 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/1166052558' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:36:03.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.294+0000 7ff4a1d1f700 1 Processor -- start 2024-04-22T23:36:03.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.294+0000 7ff4a1d1f700 1 -- start start 2024-04-22T23:36:03.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.294+0000 7ff4a1d1f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff49c0662f0 0x7ff49c06e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:03.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.294+0000 7ff4a1d1f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff49c0fff70 0x7ff49c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:03.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.294+0000 7ff4a1d1f700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c0700a0 con 0x7ff49c0fff70 2024-04-22T23:36:03.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.294+0000 7ff4a1d1f700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c06eee0 con 0x7ff49c0662f0 2024-04-22T23:36:03.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.294+0000 7ff49b7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff49c0662f0 0x7ff49c06e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:03.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.294+0000 7ff49b7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff49c0662f0 0x7ff49c06e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.107:57256/0 (socket says 172.21.15.107:57256) 2024-04-22T23:36:03.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.294+0000 7ff49b7fe700 1 -- 172.21.15.107:0/4243323006 learned_addr learned my addr 172.21.15.107:0/4243323006 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:36:03.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.295+0000 7ff49affd700 1 --2- 172.21.15.107:0/4243323006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff49c0fff70 0x7ff49c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:03.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.295+0000 7ff49b7fe700 1 -- 172.21.15.107:0/4243323006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff49c0fff70 msgr2=0x7ff49c06e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:03.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.295+0000 7ff49b7fe700 1 --2- 172.21.15.107:0/4243323006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff49c0fff70 0x7ff49c06e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:03.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.295+0000 7ff49b7fe700 1 -- 172.21.15.107:0/4243323006 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff48c005040 con 0x7ff49c0662f0 2024-04-22T23:36:03.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.295+0000 7ff49affd700 1 --2- 172.21.15.107:0/4243323006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff49c0fff70 0x7ff49c06e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T23:36:03.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.295+0000 7ff49b7fe700 1 --2- 172.21.15.107:0/4243323006 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff49c0662f0 0x7ff49c06e460 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff48c000c00 tx=0x7ff48c009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:03.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.295+0000 7ff483fff700 1 -- 172.21.15.107:0/4243323006 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff48c00e040 con 0x7ff49c0662f0 2024-04-22T23:36:03.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.295+0000 7ff4a1d1f700 1 -- 172.21.15.107:0/4243323006 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff49c06f160 con 0x7ff49c0662f0 2024-04-22T23:36:03.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.295+0000 7ff483fff700 1 -- 172.21.15.107:0/4243323006 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff48c00bc20 con 0x7ff49c0662f0 2024-04-22T23:36:03.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.295+0000 7ff483fff700 1 -- 172.21.15.107:0/4243323006 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff48c0136f0 con 0x7ff49c0662f0 2024-04-22T23:36:03.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.296+0000 7ff4a1d1f700 1 -- 172.21.15.107:0/4243323006 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff49c0fbb00 con 0x7ff49c0662f0 2024-04-22T23:36:03.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.297+0000 7ff4a1d1f700 1 -- 172.21.15.107:0/4243323006 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff49c05f5f0 con 0x7ff49c0662f0 2024-04-22T23:36:03.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.297+0000 7ff483fff700 1 -- 172.21.15.107:0/4243323006 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7ff48c01a570 con 0x7ff49c0662f0 2024-04-22T23:36:03.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.297+0000 7ff483fff700 1 --2- 172.21.15.107:0/4243323006 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7ff4840692a0 0x7ff48406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:03.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.298+0000 7ff483fff700 1 -- 172.21.15.107:0/4243323006 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7ff48c08c150 con 0x7ff49c0662f0 2024-04-22T23:36:03.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.298+0000 7ff49affd700 1 --2- 172.21.15.107:0/4243323006 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7ff4840692a0 0x7ff48406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:03.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.298+0000 7ff49affd700 1 --2- 172.21.15.107:0/4243323006 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7ff4840692a0 0x7ff48406b750 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff490004fd0 tx=0x7ff4900043c0 comp rx=0 tx=0).ready entity=mgr.14206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:03.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.302+0000 7ff483fff700 1 -- 172.21.15.107:0/4243323006 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ff48c058430 con 0x7ff49c0662f0 2024-04-22T23:36:03.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:03 smithi157 ceph-mon[33877]: from='client.24451 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:36:03.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:03 smithi157 ceph-mon[33877]: pgmap v288: 65 pgs: 65 active+clean; 291 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 35 KiB/s rd, 2.0 MiB/s wr, 408 op/s 2024-04-22T23:36:03.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:03 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/2786259990' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:36:03.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:03 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/1166052558' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:36:03.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.664+0000 7ff4a1d1f700 1 -- 172.21.15.107:0/4243323006 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff49c04bed0 con 0x7ff49c0662f0 2024-04-22T23:36:03.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.665+0000 7ff483fff700 1 -- 172.21.15.107:0/4243323006 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7ff48c029020 con 0x7ff49c0662f0 2024-04-22T23:36:03.659 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_OK 2024-04-22T23:36:03.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.667+0000 7ff4a1d1f700 1 -- 172.21.15.107:0/4243323006 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7ff4840692a0 msgr2=0x7ff48406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:03.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.667+0000 7ff4a1d1f700 1 --2- 172.21.15.107:0/4243323006 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7ff4840692a0 0x7ff48406b750 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff490004fd0 tx=0x7ff4900043c0 comp rx=0 tx=0).stop 2024-04-22T23:36:03.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.667+0000 7ff4a1d1f700 1 -- 172.21.15.107:0/4243323006 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff49c0662f0 msgr2=0x7ff49c06e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:03.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.667+0000 7ff4a1d1f700 1 --2- 172.21.15.107:0/4243323006 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff49c0662f0 0x7ff49c06e460 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff48c000c00 tx=0x7ff48c009e30 comp rx=0 tx=0).stop 2024-04-22T23:36:03.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.667+0000 7ff4a1d1f700 1 -- 172.21.15.107:0/4243323006 shutdown_connections 2024-04-22T23:36:03.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.667+0000 7ff4a1d1f700 1 --2- 172.21.15.107:0/4243323006 >> [v2:172.21.15.107:6800/2516490111,v1:172.21.15.107:6801/2516490111] conn(0x7ff4840692a0 0x7ff48406b750 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:03.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.667+0000 7ff4a1d1f700 1 --2- 172.21.15.107:0/4243323006 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff49c0662f0 0x7ff49c06e460 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:03.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.667+0000 7ff4a1d1f700 1 --2- 172.21.15.107:0/4243323006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff49c0fff70 0x7ff49c06e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:03.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.667+0000 7ff4a1d1f700 1 -- 172.21.15.107:0/4243323006 >> 172.21.15.107:0/4243323006 conn(0x7ff49c0f74c0 msgr2=0x7ff49c0f7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:03.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.667+0000 7ff4a1d1f700 1 -- 172.21.15.107:0/4243323006 shutdown_connections 2024-04-22T23:36:03.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:03.667+0000 7ff4a1d1f700 1 -- 172.21.15.107:0/4243323006 wait complete. 2024-04-22T23:36:04.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:04 smithi157 ceph-mon[33877]: from='client.24463 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:36:04.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:04 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/4243323006' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:36:04.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:04 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:36:04.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:04 smithi107 ceph-mon[26355]: from='client.24463 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:36:04.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:04 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/4243323006' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:36:04.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:04 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:36:05.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:05 smithi157 ceph-mon[33877]: pgmap v289: 65 pgs: 65 active+clean; 289 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 47 KiB/s rd, 1.6 MiB/s wr, 351 op/s 2024-04-22T23:36:05.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:05 smithi107 ceph-mon[26355]: pgmap v289: 65 pgs: 65 active+clean; 289 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 47 KiB/s rd, 1.6 MiB/s wr, 351 op/s 2024-04-22T23:36:07.287 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:07 smithi157 ceph-mon[33877]: pgmap v290: 65 pgs: 65 active+clean; 285 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 33 KiB/s rd, 1.2 MiB/s wr, 344 op/s 2024-04-22T23:36:07.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:07 smithi107 ceph-mon[26355]: pgmap v290: 65 pgs: 65 active+clean; 285 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 33 KiB/s rd, 1.2 MiB/s wr, 344 op/s 2024-04-22T23:36:09.270 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:09 smithi157 ceph-mon[33877]: pgmap v291: 65 pgs: 65 active+clean; 287 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 34 KiB/s rd, 1.7 MiB/s wr, 409 op/s 2024-04-22T23:36:09.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:09 smithi107 ceph-mon[26355]: pgmap v291: 65 pgs: 65 active+clean; 287 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 34 KiB/s rd, 1.7 MiB/s wr, 409 op/s 2024-04-22T23:36:10.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:10 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:36:10.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:10 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:36:10.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:10 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:36:10.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:10 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:36:10.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:10 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:36:10.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:10 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:36:11.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:11 smithi107 ceph-mon[26355]: pgmap v292: 65 pgs: 65 active+clean; 288 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 46 KiB/s rd, 1.9 MiB/s wr, 395 op/s 2024-04-22T23:36:11.601 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:11 smithi157 ceph-mon[33877]: pgmap v292: 65 pgs: 65 active+clean; 288 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 46 KiB/s rd, 1.9 MiB/s wr, 395 op/s 2024-04-22T23:36:12.474 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-04-22T23:36:12.475 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-04-22T23:36:12.888 DEBUG:teuthology.parallel:result is None 2024-04-22T23:36:13.440 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:13 smithi157 ceph-mon[33877]: pgmap v293: 65 pgs: 65 active+clean; 287 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 41 KiB/s rd, 2.3 MiB/s wr, 443 op/s 2024-04-22T23:36:13.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:13 smithi107 ceph-mon[26355]: pgmap v293: 65 pgs: 65 active+clean; 287 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 41 KiB/s rd, 2.3 MiB/s wr, 443 op/s 2024-04-22T23:36:14.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:14 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:36:14.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:14 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:36:14.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:14 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:36:14.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:14 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:36:14.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:14 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:36:14.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:14 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:36:14.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:14 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:36:14.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:14 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:36:15.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:15 smithi107 ceph-mon[26355]: pgmap v294: 65 pgs: 65 active+clean; 286 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 48 KiB/s rd, 1.7 MiB/s wr, 383 op/s 2024-04-22T23:36:15.785 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:15 smithi157 ceph-mon[33877]: pgmap v294: 65 pgs: 65 active+clean; 286 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 48 KiB/s rd, 1.7 MiB/s wr, 383 op/s 2024-04-22T23:36:16.406 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-04-22T23:36:16.406 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-22T23:36:16.715 DEBUG:teuthology.parallel:result is None 2024-04-22T23:36:16.715 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-22T23:36:16.753 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-22T23:36:16.753 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-04-22T23:36:16.786 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-04-22T23:36:16.787 DEBUG:teuthology.parallel:result is None 2024-04-22T23:36:17.704 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:17 smithi157 ceph-mon[33877]: pgmap v295: 65 pgs: 65 active+clean; 282 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 31 KiB/s rd, 1.7 MiB/s wr, 388 op/s 2024-04-22T23:36:17.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:17 smithi107 ceph-mon[26355]: pgmap v295: 65 pgs: 65 active+clean; 282 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 31 KiB/s rd, 1.7 MiB/s wr, 388 op/s 2024-04-22T23:36:19.234 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:18 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:36:19.234 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:18 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:36:19.234 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:18 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:36:19.235 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:18 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:36:19.235 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:18 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:36:19.235 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:18 smithi107 ceph-mon[26355]: Standby manager daemon smithi157.tuylis restarted 2024-04-22T23:36:19.235 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:18 smithi107 ceph-mon[26355]: Standby manager daemon smithi157.tuylis started 2024-04-22T23:36:19.235 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:18 smithi107 ceph-mon[26355]: from='mgr.? 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi157.tuylis/crt"}]: dispatch 2024-04-22T23:36:19.235 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:18 smithi107 ceph-mon[26355]: from='mgr.? 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T23:36:19.235 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:18 smithi107 ceph-mon[26355]: from='mgr.? 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi157.tuylis/key"}]: dispatch 2024-04-22T23:36:19.235 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:18 smithi107 ceph-mon[26355]: from='mgr.? 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T23:36:19.235 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:18 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:36:19.235 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:18 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mgr fail", "who": "smithi107.uipxvm"}]: dispatch 2024-04-22T23:36:19.235 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:18 smithi107 ceph-mon[26355]: Activating manager daemon smithi157.tuylis 2024-04-22T23:36:19.235 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:18 smithi107 ceph-mon[26355]: osdmap e47: 6 total, 6 up, 6 in 2024-04-22T23:36:19.235 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:18 smithi107 ceph-mon[26355]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "mgr fail", "who": "smithi107.uipxvm"}]': finished 2024-04-22T23:36:19.236 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:18 smithi107 ceph-mon[26355]: mgrmap e20: smithi157.tuylis(active, starting, since 0.00762354s) 2024-04-22T23:36:19.236 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:18 smithi107 ceph-mon[26355]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-04-22T23:36:19.236 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:18 smithi107 ceph-mon[26355]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "mon metadata", "id": "smithi157"}]: dispatch 2024-04-22T23:36:19.236 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:18 smithi107 ceph-mon[26355]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.ktuxjl"}]: dispatch 2024-04-22T23:36:19.236 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:18 smithi107 ceph-mon[26355]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.dfvvkw"}]: dispatch 2024-04-22T23:36:19.236 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:18 smithi107 ceph-mon[26355]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi157.rcaris"}]: dispatch 2024-04-22T23:36:19.236 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:18 smithi107 ceph-mon[26355]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi157.ufywse"}]: dispatch 2024-04-22T23:36:19.236 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:18 smithi107 ceph-mon[26355]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "mgr metadata", "who": "smithi157.tuylis", "id": "smithi157.tuylis"}]: dispatch 2024-04-22T23:36:19.236 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:18 smithi107 ceph-mon[26355]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T23:36:19.236 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:18 smithi107 ceph-mon[26355]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T23:36:19.236 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:18 smithi107 ceph-mon[26355]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T23:36:19.236 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:18 smithi107 ceph-mon[26355]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T23:36:19.236 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:18 smithi107 ceph-mon[26355]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T23:36:19.237 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:18 smithi107 ceph-mon[26355]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T23:36:19.237 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:18 smithi107 ceph-mon[26355]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-22T23:36:19.237 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:18 smithi107 ceph-mon[26355]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-22T23:36:19.237 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:18 smithi107 ceph-mon[26355]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-22T23:36:19.237 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:18 smithi107 ceph-mon[26355]: Manager daemon smithi157.tuylis is now available 2024-04-22T23:36:19.237 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:18 smithi107 ceph-mon[26355]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:36:19.237 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:18 smithi107 ceph-mon[26355]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:36:19.237 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:18 smithi107 ceph-mon[26355]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi157.tuylis/mirror_snapshot_schedule"}]: dispatch 2024-04-22T23:36:19.237 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:18 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi157.tuylis/mirror_snapshot_schedule"}]: dispatch 2024-04-22T23:36:19.237 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:18 smithi107 ceph-mon[26355]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi157.tuylis/trash_purge_schedule"}]: dispatch 2024-04-22T23:36:19.237 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:18 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi157.tuylis/trash_purge_schedule"}]: dispatch 2024-04-22T23:36:19.237 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:18 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:19.237 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:18 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:19.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:18 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:36:19.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:18 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:36:19.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:18 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:36:19.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:18 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:36:19.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:18 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' 2024-04-22T23:36:19.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:18 smithi157 ceph-mon[33877]: Standby manager daemon smithi157.tuylis restarted 2024-04-22T23:36:19.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:18 smithi157 ceph-mon[33877]: Standby manager daemon smithi157.tuylis started 2024-04-22T23:36:19.383 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:18 smithi157 ceph-mon[33877]: from='mgr.? 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi157.tuylis/crt"}]: dispatch 2024-04-22T23:36:19.383 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:18 smithi157 ceph-mon[33877]: from='mgr.? 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T23:36:19.383 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:18 smithi157 ceph-mon[33877]: from='mgr.? 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi157.tuylis/key"}]: dispatch 2024-04-22T23:36:19.383 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:18 smithi157 ceph-mon[33877]: from='mgr.? 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T23:36:19.383 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:18 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:36:19.383 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:18 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mgr fail", "who": "smithi107.uipxvm"}]: dispatch 2024-04-22T23:36:19.383 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:18 smithi157 ceph-mon[33877]: Activating manager daemon smithi157.tuylis 2024-04-22T23:36:19.383 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:18 smithi157 ceph-mon[33877]: osdmap e47: 6 total, 6 up, 6 in 2024-04-22T23:36:19.383 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:18 smithi157 ceph-mon[33877]: from='mgr.14206 172.21.15.107:0/1444715703' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "mgr fail", "who": "smithi107.uipxvm"}]': finished 2024-04-22T23:36:19.383 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:18 smithi157 ceph-mon[33877]: mgrmap e20: smithi157.tuylis(active, starting, since 0.00762354s) 2024-04-22T23:36:19.383 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:18 smithi157 ceph-mon[33877]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-04-22T23:36:19.383 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:18 smithi157 ceph-mon[33877]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "mon metadata", "id": "smithi157"}]: dispatch 2024-04-22T23:36:19.383 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:18 smithi157 ceph-mon[33877]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.ktuxjl"}]: dispatch 2024-04-22T23:36:19.383 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:18 smithi157 ceph-mon[33877]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.dfvvkw"}]: dispatch 2024-04-22T23:36:19.384 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:18 smithi157 ceph-mon[33877]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi157.rcaris"}]: dispatch 2024-04-22T23:36:19.384 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:18 smithi157 ceph-mon[33877]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi157.ufywse"}]: dispatch 2024-04-22T23:36:19.384 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:18 smithi157 ceph-mon[33877]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "mgr metadata", "who": "smithi157.tuylis", "id": "smithi157.tuylis"}]: dispatch 2024-04-22T23:36:19.384 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:18 smithi157 ceph-mon[33877]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T23:36:19.384 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:18 smithi157 ceph-mon[33877]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T23:36:19.384 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:18 smithi157 ceph-mon[33877]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T23:36:19.384 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:18 smithi157 ceph-mon[33877]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T23:36:19.384 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:18 smithi157 ceph-mon[33877]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T23:36:19.384 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:18 smithi157 ceph-mon[33877]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T23:36:19.384 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:18 smithi157 ceph-mon[33877]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-22T23:36:19.384 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:18 smithi157 ceph-mon[33877]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-22T23:36:19.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:18 smithi157 ceph-mon[33877]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-22T23:36:19.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:18 smithi157 ceph-mon[33877]: Manager daemon smithi157.tuylis is now available 2024-04-22T23:36:19.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:18 smithi157 ceph-mon[33877]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:36:19.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:18 smithi157 ceph-mon[33877]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:36:19.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:18 smithi157 ceph-mon[33877]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi157.tuylis/mirror_snapshot_schedule"}]: dispatch 2024-04-22T23:36:19.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:18 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi157.tuylis/mirror_snapshot_schedule"}]: dispatch 2024-04-22T23:36:19.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:18 smithi157 ceph-mon[33877]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi157.tuylis/trash_purge_schedule"}]: dispatch 2024-04-22T23:36:19.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:18 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi157.tuylis/trash_purge_schedule"}]: dispatch 2024-04-22T23:36:19.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:18 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:19.385 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:18 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:20.348 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:20 smithi107 ceph-mon[26355]: Deploying cephadm binary to smithi157 2024-04-22T23:36:20.348 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:20 smithi107 ceph-mon[26355]: Deploying cephadm binary to smithi107 2024-04-22T23:36:20.348 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:20 smithi107 ceph-mon[26355]: [22/Apr/2024:23:36:18] ENGINE Bus STARTING 2024-04-22T23:36:20.348 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:20 smithi107 ceph-mon[26355]: [22/Apr/2024:23:36:19] ENGINE Serving on https://172.21.15.157:7150 2024-04-22T23:36:20.348 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:20 smithi107 ceph-mon[26355]: mgrmap e21: smithi157.tuylis(active, since 1.00931s) 2024-04-22T23:36:20.348 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:20 smithi107 ceph-mon[26355]: pgmap v3: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:36:20.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:20 smithi157 ceph-mon[33877]: Deploying cephadm binary to smithi157 2024-04-22T23:36:20.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:20 smithi157 ceph-mon[33877]: Deploying cephadm binary to smithi107 2024-04-22T23:36:20.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:20 smithi157 ceph-mon[33877]: [22/Apr/2024:23:36:18] ENGINE Bus STARTING 2024-04-22T23:36:20.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:20 smithi157 ceph-mon[33877]: [22/Apr/2024:23:36:19] ENGINE Serving on https://172.21.15.157:7150 2024-04-22T23:36:20.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:20 smithi157 ceph-mon[33877]: mgrmap e21: smithi157.tuylis(active, since 1.00931s) 2024-04-22T23:36:20.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:20 smithi157 ceph-mon[33877]: pgmap v3: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:36:21.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:21 smithi157 ceph-mon[33877]: [22/Apr/2024:23:36:19] ENGINE Serving on http://172.21.15.157:8765 2024-04-22T23:36:21.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:21 smithi157 ceph-mon[33877]: [22/Apr/2024:23:36:19] ENGINE Bus STARTED 2024-04-22T23:36:21.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:21 smithi157 ceph-mon[33877]: pgmap v4: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:36:21.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:21 smithi157 ceph-mon[33877]: mgrmap e22: smithi157.tuylis(active, since 2s) 2024-04-22T23:36:21.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:21 smithi107 ceph-mon[26355]: [22/Apr/2024:23:36:19] ENGINE Serving on http://172.21.15.157:8765 2024-04-22T23:36:21.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:21 smithi107 ceph-mon[26355]: [22/Apr/2024:23:36:19] ENGINE Bus STARTED 2024-04-22T23:36:21.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:21 smithi107 ceph-mon[26355]: pgmap v4: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:36:21.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:21 smithi107 ceph-mon[26355]: mgrmap e22: smithi157.tuylis(active, since 2s) 2024-04-22T23:36:23.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:23 smithi107 ceph-mon[26355]: pgmap v5: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:36:23.234 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:23 smithi107 ceph-mon[26355]: mgrmap e23: smithi157.tuylis(active, since 4s) 2024-04-22T23:36:23.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:23 smithi157 ceph-mon[33877]: pgmap v5: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:36:23.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:23 smithi157 ceph-mon[33877]: mgrmap e23: smithi157.tuylis(active, since 4s) 2024-04-22T23:36:25.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:24 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:25.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:24 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:25.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:24 smithi157 ceph-mon[33877]: pgmap v6: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 409 B/s wr, 1 op/s 2024-04-22T23:36:25.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:24 smithi157 ceph-mon[33877]: Standby manager daemon smithi107.uipxvm started 2024-04-22T23:36:25.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:24 smithi157 ceph-mon[33877]: from='mgr.? 172.21.15.107:0/3971028141' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi107.uipxvm/crt"}]: dispatch 2024-04-22T23:36:25.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:24 smithi157 ceph-mon[33877]: from='mgr.? 172.21.15.107:0/3971028141' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T23:36:25.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:24 smithi157 ceph-mon[33877]: from='mgr.? 172.21.15.107:0/3971028141' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi107.uipxvm/key"}]: dispatch 2024-04-22T23:36:25.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:24 smithi157 ceph-mon[33877]: from='mgr.? 172.21.15.107:0/3971028141' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T23:36:25.234 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:24 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:25.234 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:24 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:25.234 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:24 smithi107 ceph-mon[26355]: pgmap v6: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 409 B/s wr, 1 op/s 2024-04-22T23:36:25.234 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:24 smithi107 ceph-mon[26355]: Standby manager daemon smithi107.uipxvm started 2024-04-22T23:36:25.234 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:24 smithi107 ceph-mon[26355]: from='mgr.? 172.21.15.107:0/3971028141' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi107.uipxvm/crt"}]: dispatch 2024-04-22T23:36:25.234 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:24 smithi107 ceph-mon[26355]: from='mgr.? 172.21.15.107:0/3971028141' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T23:36:25.234 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:24 smithi107 ceph-mon[26355]: from='mgr.? 172.21.15.107:0/3971028141' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi107.uipxvm/key"}]: dispatch 2024-04-22T23:36:25.234 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:24 smithi107 ceph-mon[26355]: from='mgr.? 172.21.15.107:0/3971028141' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T23:36:25.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:25 smithi157 ceph-mon[33877]: mgrmap e24: smithi157.tuylis(active, since 6s), standbys: smithi107.uipxvm 2024-04-22T23:36:25.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:25 smithi157 ceph-mon[33877]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "mgr metadata", "who": "smithi107.uipxvm", "id": "smithi107.uipxvm"}]: dispatch 2024-04-22T23:36:26.050 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:25 smithi107 ceph-mon[26355]: mgrmap e24: smithi157.tuylis(active, since 6s), standbys: smithi107.uipxvm 2024-04-22T23:36:26.050 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:25 smithi107 ceph-mon[26355]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "mgr metadata", "who": "smithi107.uipxvm", "id": "smithi107.uipxvm"}]: dispatch 2024-04-22T23:36:27.100 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:26 smithi157 ceph-mon[33877]: pgmap v7: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 1 KiB/s wr, 18 op/s 2024-04-22T23:36:27.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:26 smithi107 ceph-mon[26355]: pgmap v7: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 1 KiB/s wr, 18 op/s 2024-04-22T23:36:29.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:28 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:29.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:28 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:29.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:28 smithi157 ceph-mon[33877]: pgmap v8: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 796 B/s wr, 14 op/s 2024-04-22T23:36:29.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:28 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:29.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:28 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:29.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:28 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:29.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:28 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:29.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:28 smithi107 ceph-mon[26355]: pgmap v8: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 796 B/s wr, 14 op/s 2024-04-22T23:36:29.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:28 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:29.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:28 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:30.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:29 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:30.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:29 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:30.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:29 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:30.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:29 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:30.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:29 smithi157 ceph-mon[33877]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:36:30.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:29 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:36:30.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:29 smithi157 ceph-mon[33877]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:36:30.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:29 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:36:30.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:29 smithi157 ceph-mon[33877]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:36:30.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:29 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:36:30.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:29 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:30.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:29 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:30.234 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:29 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:30.234 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:29 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:30.234 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:29 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:30.234 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:29 smithi107 ceph-mon[26355]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:36:30.234 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:29 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:36:30.234 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:29 smithi107 ceph-mon[26355]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:36:30.234 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:29 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:36:30.234 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:29 smithi107 ceph-mon[26355]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:36:30.234 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:29 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:36:30.234 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:29 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:31.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:30 smithi157 ceph-mon[33877]: Adjusting osd_memory_target on smithi107 to 1557M 2024-04-22T23:36:31.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:30 smithi157 ceph-mon[33877]: pgmap v9: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 838 B/s wr, 11 op/s 2024-04-22T23:36:31.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:30 smithi107 ceph-mon[26355]: Adjusting osd_memory_target on smithi107 to 1557M 2024-04-22T23:36:31.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:30 smithi107 ceph-mon[26355]: pgmap v9: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 838 B/s wr, 11 op/s 2024-04-22T23:36:32.325 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:32 smithi157 ceph-mon[33877]: pgmap v10: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 767 B/s wr, 10 op/s 2024-04-22T23:36:32.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:32 smithi107 ceph-mon[26355]: pgmap v10: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 767 B/s wr, 10 op/s 2024-04-22T23:36:33.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:33 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:33.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:33 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:33.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:33 smithi157 ceph-mon[33877]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:36:33.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:33 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:36:33.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:33 smithi157 ceph-mon[33877]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:36:33.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:33 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:36:33.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:33 smithi157 ceph-mon[33877]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:36:33.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:33 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:36:33.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:33 smithi157 ceph-mon[33877]: Adjusting osd_memory_target on smithi157 to 2581M 2024-04-22T23:36:33.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:33 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:33.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:33 smithi157 ceph-mon[33877]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:36:33.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:33 smithi157 ceph-mon[33877]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:36:33.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:33 smithi157 ceph-mon[33877]: Updating smithi107:/etc/ceph/ceph.conf 2024-04-22T23:36:33.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:33 smithi157 ceph-mon[33877]: Updating smithi157:/etc/ceph/ceph.conf 2024-04-22T23:36:33.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:33 smithi157 ceph-mon[33877]: Updating smithi157:/var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T23:36:33.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:33 smithi157 ceph-mon[33877]: Updating smithi107:/var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T23:36:33.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:33 smithi157 ceph-mon[33877]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2024-04-22T23:36:33.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:33 smithi157 ceph-mon[33877]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-04-22T23:36:33.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:33 smithi157 ceph-mon[33877]: Updating smithi157:/var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T23:36:33.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:33 smithi157 ceph-mon[33877]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:36:33.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:33 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:33.636 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:33 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:33.636 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:33 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:33.636 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:33 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:33.636 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:33 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:33.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:33 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:33.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:33 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:33.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:33 smithi107 ceph-mon[26355]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:36:33.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:33 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:36:33.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:33 smithi107 ceph-mon[26355]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:36:33.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:33 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:36:33.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:33 smithi107 ceph-mon[26355]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:36:33.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:33 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T23:36:33.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:33 smithi107 ceph-mon[26355]: Adjusting osd_memory_target on smithi157 to 2581M 2024-04-22T23:36:33.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:33 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:33.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:33 smithi107 ceph-mon[26355]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:36:33.736 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:33 smithi107 ceph-mon[26355]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:36:33.736 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:33 smithi107 ceph-mon[26355]: Updating smithi107:/etc/ceph/ceph.conf 2024-04-22T23:36:33.736 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:33 smithi107 ceph-mon[26355]: Updating smithi157:/etc/ceph/ceph.conf 2024-04-22T23:36:33.736 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:33 smithi107 ceph-mon[26355]: Updating smithi157:/var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T23:36:33.736 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:33 smithi107 ceph-mon[26355]: Updating smithi107:/var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T23:36:33.736 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:33 smithi107 ceph-mon[26355]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2024-04-22T23:36:33.737 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:33 smithi107 ceph-mon[26355]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-04-22T23:36:33.737 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:33 smithi107 ceph-mon[26355]: Updating smithi157:/var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T23:36:33.737 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:33 smithi107 ceph-mon[26355]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:36:33.737 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:33 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:33.737 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:33 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:33.737 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:33 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:33.738 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:33 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:33.738 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:33 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:33.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:33.779+0000 7f83ca769700 1 -- 172.21.15.107:0/932410584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83c4066ae0 msgr2=0x7f83c40ffb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:33.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:33.779+0000 7f83ca769700 1 --2- 172.21.15.107:0/932410584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83c4066ae0 0x7f83c40ffb10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f83b8004940 tx=0x7f83b8009e30 comp rx=0 tx=0).stop 2024-04-22T23:36:33.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:33.779+0000 7f83ca769700 1 -- 172.21.15.107:0/932410584 shutdown_connections 2024-04-22T23:36:33.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:33.779+0000 7f83ca769700 1 --2- 172.21.15.107:0/932410584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83c4100050 0x7f83c4102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:33.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:33.779+0000 7f83ca769700 1 --2- 172.21.15.107:0/932410584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83c4066ae0 0x7f83c40ffb10 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:33.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:33.779+0000 7f83ca769700 1 -- 172.21.15.107:0/932410584 >> 172.21.15.107:0/932410584 conn(0x7f83c40f74e0 msgr2=0x7f83c40f9910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:33.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:33.779+0000 7f83ca769700 1 -- 172.21.15.107:0/932410584 shutdown_connections 2024-04-22T23:36:33.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:33.779+0000 7f83ca769700 1 -- 172.21.15.107:0/932410584 wait complete. 2024-04-22T23:36:33.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:33.780+0000 7f83ca769700 1 Processor -- start 2024-04-22T23:36:33.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:33.780+0000 7f83ca769700 1 -- start start 2024-04-22T23:36:33.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:33.781+0000 7f83ca769700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83c4066ae0 0x7f83c410da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:33.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:33.781+0000 7f83ca769700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83c4100050 0x7f83c410dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:33.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:33.781+0000 7f83ca769700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c410e5c0 con 0x7f83c4100050 2024-04-22T23:36:33.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:33.781+0000 7f83ca769700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c410ee10 con 0x7f83c4066ae0 2024-04-22T23:36:33.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:33.781+0000 7f83c37fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83c4100050 0x7f83c410dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:33.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:33.781+0000 7f83c37fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83c4100050 0x7f83c410dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45760/0 (socket says 172.21.15.107:45760) 2024-04-22T23:36:33.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:33.781+0000 7f83c37fe700 1 -- 172.21.15.107:0/3118450862 learned_addr learned my addr 172.21.15.107:0/3118450862 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:36:33.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:33.781+0000 7f83c3fff700 1 --2- 172.21.15.107:0/3118450862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83c4066ae0 0x7f83c410da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:33.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:33.781+0000 7f83c37fe700 1 -- 172.21.15.107:0/3118450862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83c4066ae0 msgr2=0x7f83c410da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:33.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:33.781+0000 7f83c37fe700 1 --2- 172.21.15.107:0/3118450862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83c4066ae0 0x7f83c410da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:33.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:33.781+0000 7f83c37fe700 1 -- 172.21.15.107:0/3118450862 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b8005040 con 0x7f83c4100050 2024-04-22T23:36:33.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:33.781+0000 7f83c37fe700 1 --2- 172.21.15.107:0/3118450862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83c4100050 0x7f83c410dfc0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f83b4002a50 tx=0x7f83b40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:33.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:33.781+0000 7f83c0ff9700 1 -- 172.21.15.107:0/3118450862 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83b4007cc0 con 0x7f83c4100050 2024-04-22T23:36:33.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:33.781+0000 7f83ca769700 1 -- 172.21.15.107:0/3118450862 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83c410f0f0 con 0x7f83c4100050 2024-04-22T23:36:33.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:33.782+0000 7f83c0ff9700 1 -- 172.21.15.107:0/3118450862 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f83b400f040 con 0x7f83c4100050 2024-04-22T23:36:33.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:33.782+0000 7f83c0ff9700 1 -- 172.21.15.107:0/3118450862 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83b400da00 con 0x7f83c4100050 2024-04-22T23:36:33.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:33.782+0000 7f83ca769700 1 -- 172.21.15.107:0/3118450862 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83c410f570 con 0x7f83c4100050 2024-04-22T23:36:33.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:33.783+0000 7f83ca769700 1 -- 172.21.15.107:0/3118450862 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83c405f5f0 con 0x7f83c4100050 2024-04-22T23:36:33.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:33.783+0000 7f83c0ff9700 1 -- 172.21.15.107:0/3118450862 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) v1 ==== 89652+0+0 (secure 0 0 0) 0x7f83b400db60 con 0x7f83c4100050 2024-04-22T23:36:33.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:33.783+0000 7f83c0ff9700 1 --2- 172.21.15.107:0/3118450862 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f83ac06bcf0 0x7f83ac06e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:33.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:33.784+0000 7f83c0ff9700 1 -- 172.21.15.107:0/3118450862 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f83b408a7c0 con 0x7f83c4100050 2024-04-22T23:36:33.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:33.784+0000 7f83c3fff700 1 --2- 172.21.15.107:0/3118450862 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f83ac06bcf0 0x7f83ac06e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:33.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:33.784+0000 7f83c3fff700 1 --2- 172.21.15.107:0/3118450862 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f83ac06bcf0 0x7f83ac06e1a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f83b8004890 tx=0x7f83b8009e30 comp rx=0 tx=0).ready entity=mgr.14720 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:33.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:33.788+0000 7f83c0ff9700 1 -- 172.21.15.107:0/3118450862 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187010 (secure 0 0 0) 0x7f83b4057830 con 0x7f83c4100050 2024-04-22T23:36:34.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.032+0000 7f83ca769700 1 -- 172.21.15.107:0/3118450862 --> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f83c410f9c0 con 0x7f83ac06bcf0 2024-04-22T23:36:34.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.034+0000 7f83c0ff9700 1 -- 172.21.15.107:0/3118450862 <== mgr.14720 v2:172.21.15.157:6828/3332906296 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f83c410f9c0 con 0x7f83ac06bcf0 2024-04-22T23:36:34.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.036+0000 7f83ca769700 1 -- 172.21.15.107:0/3118450862 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f83ac06bcf0 msgr2=0x7f83ac06e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:34.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.036+0000 7f83ca769700 1 --2- 172.21.15.107:0/3118450862 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f83ac06bcf0 0x7f83ac06e1a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f83b8004890 tx=0x7f83b8009e30 comp rx=0 tx=0).stop 2024-04-22T23:36:34.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.036+0000 7f83ca769700 1 -- 172.21.15.107:0/3118450862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83c4100050 msgr2=0x7f83c410dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:34.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.036+0000 7f83ca769700 1 --2- 172.21.15.107:0/3118450862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83c4100050 0x7f83c410dfc0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f83b4002a50 tx=0x7f83b40046b0 comp rx=0 tx=0).stop 2024-04-22T23:36:34.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.036+0000 7f83ca769700 1 -- 172.21.15.107:0/3118450862 shutdown_connections 2024-04-22T23:36:34.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.036+0000 7f83ca769700 1 --2- 172.21.15.107:0/3118450862 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f83ac06bcf0 0x7f83ac06e1a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:34.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.036+0000 7f83ca769700 1 --2- 172.21.15.107:0/3118450862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83c4066ae0 0x7f83c410da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:34.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.036+0000 7f83ca769700 1 --2- 172.21.15.107:0/3118450862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83c4100050 0x7f83c410dfc0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:34.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.036+0000 7f83ca769700 1 -- 172.21.15.107:0/3118450862 >> 172.21.15.107:0/3118450862 conn(0x7f83c40f74e0 msgr2=0x7f83c40f7900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:34.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.036+0000 7f83ca769700 1 -- 172.21.15.107:0/3118450862 shutdown_connections 2024-04-22T23:36:34.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.037+0000 7f83ca769700 1 -- 172.21.15.107:0/3118450862 wait complete. 2024-04-22T23:36:34.042 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-04-22T23:36:34.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.148+0000 7f3efa0ac700 1 -- 172.21.15.107:0/1014035423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ef40fffb0 msgr2=0x7f3ef41023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:34.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.148+0000 7f3efa0ac700 1 --2- 172.21.15.107:0/1014035423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ef40fffb0 0x7f3ef41023d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8004940 tx=0x7f3ee8009e30 comp rx=0 tx=0).stop 2024-04-22T23:36:34.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.148+0000 7f3efa0ac700 1 -- 172.21.15.107:0/1014035423 shutdown_connections 2024-04-22T23:36:34.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.148+0000 7f3efa0ac700 1 --2- 172.21.15.107:0/1014035423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ef40fffb0 0x7f3ef41023d0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:34.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.148+0000 7f3efa0ac700 1 --2- 172.21.15.107:0/1014035423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ef4066b80 0x7f3ef40ffa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:34.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.148+0000 7f3efa0ac700 1 -- 172.21.15.107:0/1014035423 >> 172.21.15.107:0/1014035423 conn(0x7f3ef40f74e0 msgr2=0x7f3ef40f9910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:34.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.148+0000 7f3efa0ac700 1 -- 172.21.15.107:0/1014035423 shutdown_connections 2024-04-22T23:36:34.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.148+0000 7f3efa0ac700 1 -- 172.21.15.107:0/1014035423 wait complete. 2024-04-22T23:36:34.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.149+0000 7f3efa0ac700 1 Processor -- start 2024-04-22T23:36:34.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.149+0000 7f3efa0ac700 1 -- start start 2024-04-22T23:36:34.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.149+0000 7f3efa0ac700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ef4066b80 0x7f3ef410dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:34.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.149+0000 7f3efa0ac700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ef40fffb0 0x7f3ef410e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:34.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.149+0000 7f3efa0ac700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef410e6a0 con 0x7f3ef4066b80 2024-04-22T23:36:34.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.149+0000 7f3efa0ac700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef410ee40 con 0x7f3ef40fffb0 2024-04-22T23:36:34.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.149+0000 7f3ef37fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ef4066b80 0x7f3ef410dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:34.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.149+0000 7f3ef37fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ef4066b80 0x7f3ef410dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45782/0 (socket says 172.21.15.107:45782) 2024-04-22T23:36:34.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.149+0000 7f3ef37fe700 1 -- 172.21.15.107:0/1578540494 learned_addr learned my addr 172.21.15.107:0/1578540494 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:36:34.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.150+0000 7f3ef2ffd700 1 --2- 172.21.15.107:0/1578540494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ef40fffb0 0x7f3ef410e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:34.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.150+0000 7f3ef37fe700 1 -- 172.21.15.107:0/1578540494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ef40fffb0 msgr2=0x7f3ef410e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:34.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.150+0000 7f3ef37fe700 1 --2- 172.21.15.107:0/1578540494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ef40fffb0 0x7f3ef410e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:34.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.150+0000 7f3ef37fe700 1 -- 172.21.15.107:0/1578540494 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee8005040 con 0x7f3ef4066b80 2024-04-22T23:36:34.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.150+0000 7f3ef37fe700 1 --2- 172.21.15.107:0/1578540494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ef4066b80 0x7f3ef410dad0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3ee400b790 tx=0x7f3ee4009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:34.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.150+0000 7f3edbfff700 1 -- 172.21.15.107:0/1578540494 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ee400f6f0 con 0x7f3ef4066b80 2024-04-22T23:36:34.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.150+0000 7f3edbfff700 1 -- 172.21.15.107:0/1578540494 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3ee400fd20 con 0x7f3ef4066b80 2024-04-22T23:36:34.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.150+0000 7f3efa0ac700 1 -- 172.21.15.107:0/1578540494 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ef410f120 con 0x7f3ef4066b80 2024-04-22T23:36:34.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.150+0000 7f3edbfff700 1 -- 172.21.15.107:0/1578540494 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ee40103a0 con 0x7f3ef4066b80 2024-04-22T23:36:34.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.150+0000 7f3efa0ac700 1 -- 172.21.15.107:0/1578540494 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ef410f6e0 con 0x7f3ef4066b80 2024-04-22T23:36:34.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.151+0000 7f3efa0ac700 1 -- 172.21.15.107:0/1578540494 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ef405f5f0 con 0x7f3ef4066b80 2024-04-22T23:36:34.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.151+0000 7f3edbfff700 1 -- 172.21.15.107:0/1578540494 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) v1 ==== 89652+0+0 (secure 0 0 0) 0x7f3ee4011400 con 0x7f3ef4066b80 2024-04-22T23:36:34.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.152+0000 7f3edbfff700 1 --2- 172.21.15.107:0/1578540494 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f3edc06bcf0 0x7f3edc06e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:34.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.152+0000 7f3edbfff700 1 -- 172.21.15.107:0/1578540494 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f3ee408a7c0 con 0x7f3ef4066b80 2024-04-22T23:36:34.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.152+0000 7f3ef2ffd700 1 --2- 172.21.15.107:0/1578540494 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f3edc06bcf0 0x7f3edc06e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:34.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.153+0000 7f3ef2ffd700 1 --2- 172.21.15.107:0/1578540494 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f3edc06bcf0 0x7f3edc06e1a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8000c00 tx=0x7f3ee8009e30 comp rx=0 tx=0).ready entity=mgr.14720 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:34.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.156+0000 7f3edbfff700 1 -- 172.21.15.107:0/1578540494 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187010 (secure 0 0 0) 0x7f3ee40579c0 con 0x7f3ef4066b80 2024-04-22T23:36:34.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.379+0000 7f3efa0ac700 1 -- 172.21.15.107:0/1578540494 --> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ef410fa70 con 0x7f3edc06bcf0 2024-04-22T23:36:34.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.380+0000 7f3edbfff700 1 -- 172.21.15.107:0/1578540494 <== mgr.14720 v2:172.21.15.157:6828/3332906296 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f3ef410fa70 con 0x7f3edc06bcf0 2024-04-22T23:36:34.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.382+0000 7f3efa0ac700 1 -- 172.21.15.107:0/1578540494 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f3edc06bcf0 msgr2=0x7f3edc06e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:34.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.382+0000 7f3efa0ac700 1 --2- 172.21.15.107:0/1578540494 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f3edc06bcf0 0x7f3edc06e1a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8000c00 tx=0x7f3ee8009e30 comp rx=0 tx=0).stop 2024-04-22T23:36:34.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.382+0000 7f3efa0ac700 1 -- 172.21.15.107:0/1578540494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ef4066b80 msgr2=0x7f3ef410dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:34.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.382+0000 7f3efa0ac700 1 --2- 172.21.15.107:0/1578540494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ef4066b80 0x7f3ef410dad0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3ee400b790 tx=0x7f3ee4009220 comp rx=0 tx=0).stop 2024-04-22T23:36:34.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.382+0000 7f3efa0ac700 1 -- 172.21.15.107:0/1578540494 shutdown_connections 2024-04-22T23:36:34.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.382+0000 7f3efa0ac700 1 --2- 172.21.15.107:0/1578540494 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f3edc06bcf0 0x7f3edc06e1a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:34.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.382+0000 7f3efa0ac700 1 --2- 172.21.15.107:0/1578540494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ef4066b80 0x7f3ef410dad0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:34.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.382+0000 7f3efa0ac700 1 --2- 172.21.15.107:0/1578540494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ef40fffb0 0x7f3ef410e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:34.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.382+0000 7f3efa0ac700 1 -- 172.21.15.107:0/1578540494 >> 172.21.15.107:0/1578540494 conn(0x7f3ef40f74e0 msgr2=0x7f3ef40f7900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:34.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.382+0000 7f3efa0ac700 1 -- 172.21.15.107:0/1578540494 shutdown_connections 2024-04-22T23:36:34.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.382+0000 7f3efa0ac700 1 -- 172.21.15.107:0/1578540494 wait complete. 2024-04-22T23:36:34.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.487+0000 7fd2bf1a8700 1 -- 172.21.15.107:0/1230156964 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2b80fdc20 msgr2=0x7fd2b80fc030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:34.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.487+0000 7fd2bf1a8700 1 --2- 172.21.15.107:0/1230156964 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2b80fdc20 0x7fd2b80fc030 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4004940 tx=0x7fd2a4009e30 comp rx=0 tx=0).stop 2024-04-22T23:36:34.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.487+0000 7fd2bf1a8700 1 -- 172.21.15.107:0/1230156964 shutdown_connections 2024-04-22T23:36:34.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.487+0000 7fd2bf1a8700 1 --2- 172.21.15.107:0/1230156964 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2b80fdc20 0x7fd2b80fc030 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:34.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.487+0000 7fd2bf1a8700 1 --2- 172.21.15.107:0/1230156964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2b80fd2d0 0x7fd2b80fd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:34.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.487+0000 7fd2bf1a8700 1 -- 172.21.15.107:0/1230156964 >> 172.21.15.107:0/1230156964 conn(0x7fd2b80f6bd0 msgr2=0x7fd2b80f8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:34.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.487+0000 7fd2bf1a8700 1 -- 172.21.15.107:0/1230156964 shutdown_connections 2024-04-22T23:36:34.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.487+0000 7fd2bf1a8700 1 -- 172.21.15.107:0/1230156964 wait complete. 2024-04-22T23:36:34.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.487+0000 7fd2bf1a8700 1 Processor -- start 2024-04-22T23:36:34.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.487+0000 7fd2bf1a8700 1 -- start start 2024-04-22T23:36:34.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.488+0000 7fd2bf1a8700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2b80fd2d0 0x7fd2b810c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:34.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.488+0000 7fd2bf1a8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2b80fdc20 0x7fd2b810ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:34.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.488+0000 7fd2bf1a8700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b810d480 con 0x7fd2b80fdc20 2024-04-22T23:36:34.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.488+0000 7fd2bf1a8700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b810dcd0 con 0x7fd2b80fd2d0 2024-04-22T23:36:34.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.488+0000 7fd2b7fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2b80fdc20 0x7fd2b810ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:34.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.488+0000 7fd2b7fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2b80fdc20 0x7fd2b810ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45800/0 (socket says 172.21.15.107:45800) 2024-04-22T23:36:34.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.488+0000 7fd2b7fff700 1 -- 172.21.15.107:0/3436085416 learned_addr learned my addr 172.21.15.107:0/3436085416 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:36:34.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.488+0000 7fd2bcf44700 1 --2- 172.21.15.107:0/3436085416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2b80fd2d0 0x7fd2b810c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:34.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.488+0000 7fd2b7fff700 1 -- 172.21.15.107:0/3436085416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2b80fd2d0 msgr2=0x7fd2b810c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:34.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.488+0000 7fd2b7fff700 1 --2- 172.21.15.107:0/3436085416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2b80fd2d0 0x7fd2b810c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:34.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.488+0000 7fd2b7fff700 1 -- 172.21.15.107:0/3436085416 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a4005040 con 0x7fd2b80fdc20 2024-04-22T23:36:34.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.488+0000 7fd2b7fff700 1 --2- 172.21.15.107:0/3436085416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2b80fdc20 0x7fd2b810ce80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd2a400b970 tx=0x7fd2a4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:34.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.488+0000 7fd2b57fa700 1 -- 172.21.15.107:0/3436085416 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2a400e040 con 0x7fd2b80fdc20 2024-04-22T23:36:34.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.489+0000 7fd2bf1a8700 1 -- 172.21.15.107:0/3436085416 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2b810df50 con 0x7fd2b80fdc20 2024-04-22T23:36:34.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.489+0000 7fd2b57fa700 1 -- 172.21.15.107:0/3436085416 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd2a4004290 con 0x7fd2b80fdc20 2024-04-22T23:36:34.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.489+0000 7fd2b57fa700 1 -- 172.21.15.107:0/3436085416 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2a401c630 con 0x7fd2b80fdc20 2024-04-22T23:36:34.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.489+0000 7fd2bf1a8700 1 -- 172.21.15.107:0/3436085416 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2b810e490 con 0x7fd2b80fdc20 2024-04-22T23:36:34.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.489+0000 7fd2bf1a8700 1 -- 172.21.15.107:0/3436085416 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2b805f5f0 con 0x7fd2b80fdc20 2024-04-22T23:36:34.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.490+0000 7fd2b57fa700 1 -- 172.21.15.107:0/3436085416 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) v1 ==== 89652+0+0 (secure 0 0 0) 0x7fd2a400bcc0 con 0x7fd2b80fdc20 2024-04-22T23:36:34.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.490+0000 7fd2b57fa700 1 --2- 172.21.15.107:0/3436085416 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7fd2a806bcf0 0x7fd2a806e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:34.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.491+0000 7fd2b57fa700 1 -- 172.21.15.107:0/3436085416 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fd2a408be10 con 0x7fd2b80fdc20 2024-04-22T23:36:34.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.491+0000 7fd2bcf44700 1 --2- 172.21.15.107:0/3436085416 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7fd2a806bcf0 0x7fd2a806e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:34.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.491+0000 7fd2bcf44700 1 --2- 172.21.15.107:0/3436085416 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7fd2a806bcf0 0x7fd2a806e1a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac007fd0 tx=0x7fd2ac009040 comp rx=0 tx=0).ready entity=mgr.14720 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:34.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.495+0000 7fd2b57fa700 1 -- 172.21.15.107:0/3436085416 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187010 (secure 0 0 0) 0x7fd2a4058f00 con 0x7fd2b80fdc20 2024-04-22T23:36:34.681 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:34 smithi107 ceph-mon[26355]: Updating smithi107:/var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T23:36:34.681 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:34 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:34.681 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:34 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:34.681 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:34 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:34.681 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:34 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:34.681 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:34 smithi107 ceph-mon[26355]: Reconfiguring node-exporter.smithi107 (dependencies changed)... 2024-04-22T23:36:34.681 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:34 smithi107 ceph-mon[26355]: Reconfiguring daemon node-exporter.smithi107 on smithi107 2024-04-22T23:36:34.681 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:34 smithi107 ceph-mon[26355]: from='client.14750 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:36:34.681 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:34 smithi107 ceph-mon[26355]: pgmap v11: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 767 B/s wr, 10 op/s 2024-04-22T23:36:34.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.706+0000 7fd2bf1a8700 1 -- 172.21.15.107:0/3436085416 --> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd2b810e7c0 con 0x7fd2a806bcf0 2024-04-22T23:36:34.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.717+0000 7fd2b57fa700 1 -- 172.21.15.107:0/3436085416 <== mgr.14720 v2:172.21.15.157:6828/3332906296 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7fd2b810e7c0 con 0x7fd2a806bcf0 2024-04-22T23:36:34.712 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-22T23:36:34.712 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (8m) 6s ago 9m 25.8M - 0.25.0 c8568f914cd2 4d90ceb89249 2024-04-22T23:36:34.712 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (9m) 6s ago 9m 6660k - 17.2.7-884-g70d0cb4b cafd8ddac8fa c10611c756e4 2024-04-22T23:36:34.712 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi157 smithi157 running (8m) 6s ago 8m 6669k - 17.2.7-884-g70d0cb4b cafd8ddac8fa 55d2d46294c9 2024-04-22T23:36:34.712 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (8m) 6s ago 9m 86.4M - 9.4.7 954c08fa6188 c2d1f8de1aa7 2024-04-22T23:36:34.712 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.dfvvkw smithi107 running (4m) 6s ago 4m 11.9M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 08d3aa8d717d 2024-04-22T23:36:34.712 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.ktuxjl smithi107 running (4m) 6s ago 4m 125M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 762f27fe0f30 2024-04-22T23:36:34.713 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi157.rcaris smithi157 running (4m) 6s ago 4m 128M - 17.2.7-884-g70d0cb4b cafd8ddac8fa d23366ac085d 2024-04-22T23:36:34.713 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi157.ufywse smithi157 running (4m) 6s ago 4m 12.5M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 8b00d61eb355 2024-04-22T23:36:34.713 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.uipxvm smithi107 *:9283 running (10m) 6s ago 10m 401M - 17.2.7-884-g70d0cb4b cafd8ddac8fa db96e8cbbdfb 2024-04-22T23:36:34.713 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi157.tuylis smithi157 *:8443,9283 running (26s) 6s ago 8m 488M - 19.0.0-3252-g43be0201 a0434699c1d7 351c5ba75c91 2024-04-22T23:36:34.713 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (10m) 6s ago 11m 54.0M 2048M 17.2.7-884-g70d0cb4b cafd8ddac8fa 8314962fa1b9 2024-04-22T23:36:34.713 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi157 smithi157 running (8m) 6s ago 8m 37.9M 2048M 17.2.7-884-g70d0cb4b cafd8ddac8fa 0747a85008d9 2024-04-22T23:36:34.714 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (9m) 6s ago 9m 16.0M - 1.5.0 0da6a335fe13 8a7c595c114f 2024-04-22T23:36:34.714 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi157 smithi157 *:9100 running (8m) 6s ago 8m 16.2M - 1.5.0 0da6a335fe13 940164771fe3 2024-04-22T23:36:34.714 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (7m) 6s ago 7m 228M 1557M 17.2.7-884-g70d0cb4b cafd8ddac8fa 18a179b0a618 2024-04-22T23:36:34.714 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (7m) 6s ago 7m 239M 1557M 17.2.7-884-g70d0cb4b cafd8ddac8fa c7272eb84ddb 2024-04-22T23:36:34.714 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (6m) 6s ago 6m 193M 1557M 17.2.7-884-g70d0cb4b cafd8ddac8fa 9db1788096ad 2024-04-22T23:36:34.714 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi157 running (6m) 6s ago 6m 265M 2581M 17.2.7-884-g70d0cb4b cafd8ddac8fa c9fb2d1a13dd 2024-04-22T23:36:34.714 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi157 running (6m) 6s ago 6m 225M 2581M 17.2.7-884-g70d0cb4b cafd8ddac8fa 24a16e19951f 2024-04-22T23:36:34.715 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi157 running (5m) 6s ago 5m 224M 2581M 17.2.7-884-g70d0cb4b cafd8ddac8fa 4c5b7a7ef8bd 2024-04-22T23:36:34.715 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (7m) 6s ago 9m 57.7M - 2.43.0 a07b618ecd1d 8c5df40caabe 2024-04-22T23:36:34.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.720+0000 7fd2bf1a8700 1 -- 172.21.15.107:0/3436085416 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7fd2a806bcf0 msgr2=0x7fd2a806e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:34.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.720+0000 7fd2bf1a8700 1 --2- 172.21.15.107:0/3436085416 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7fd2a806bcf0 0x7fd2a806e1a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac007fd0 tx=0x7fd2ac009040 comp rx=0 tx=0).stop 2024-04-22T23:36:34.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.720+0000 7fd2bf1a8700 1 -- 172.21.15.107:0/3436085416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2b80fdc20 msgr2=0x7fd2b810ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:34.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.720+0000 7fd2bf1a8700 1 --2- 172.21.15.107:0/3436085416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2b80fdc20 0x7fd2b810ce80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd2a400b970 tx=0x7fd2a4009e30 comp rx=0 tx=0).stop 2024-04-22T23:36:34.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.720+0000 7fd2bf1a8700 1 -- 172.21.15.107:0/3436085416 shutdown_connections 2024-04-22T23:36:34.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.720+0000 7fd2bf1a8700 1 --2- 172.21.15.107:0/3436085416 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7fd2a806bcf0 0x7fd2a806e1a0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:34.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.720+0000 7fd2bf1a8700 1 --2- 172.21.15.107:0/3436085416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2b80fd2d0 0x7fd2b810c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:34.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.720+0000 7fd2bf1a8700 1 --2- 172.21.15.107:0/3436085416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2b80fdc20 0x7fd2b810ce80 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:34.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.720+0000 7fd2bf1a8700 1 -- 172.21.15.107:0/3436085416 >> 172.21.15.107:0/3436085416 conn(0x7fd2b80f6bd0 msgr2=0x7fd2b80f82f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:34.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.720+0000 7fd2bf1a8700 1 -- 172.21.15.107:0/3436085416 shutdown_connections 2024-04-22T23:36:34.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.720+0000 7fd2bf1a8700 1 -- 172.21.15.107:0/3436085416 wait complete. 2024-04-22T23:36:34.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.819+0000 7f42a8eca700 1 -- 172.21.15.107:0/3905600143 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42a40fedc0 msgr2=0x7f42a40faf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:34.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.819+0000 7f42a8eca700 1 --2- 172.21.15.107:0/3905600143 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42a40fedc0 0x7f42a40faf00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4298004940 tx=0x7f4298009e30 comp rx=0 tx=0).stop 2024-04-22T23:36:34.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.819+0000 7f42a8eca700 1 -- 172.21.15.107:0/3905600143 shutdown_connections 2024-04-22T23:36:34.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.819+0000 7f42a8eca700 1 --2- 172.21.15.107:0/3905600143 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42a40fedc0 0x7f42a40faf00 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:34.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.819+0000 7f42a8eca700 1 --2- 172.21.15.107:0/3905600143 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42a40fe470 0x7f42a40fe880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:34.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.819+0000 7f42a8eca700 1 -- 172.21.15.107:0/3905600143 >> 172.21.15.107:0/3905600143 conn(0x7f42a40f6c10 msgr2=0x7f42a40f9040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:34.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.819+0000 7f42a8eca700 1 -- 172.21.15.107:0/3905600143 shutdown_connections 2024-04-22T23:36:34.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.819+0000 7f42a8eca700 1 -- 172.21.15.107:0/3905600143 wait complete. 2024-04-22T23:36:34.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.819+0000 7f42a8eca700 1 Processor -- start 2024-04-22T23:36:34.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.820+0000 7f42a8eca700 1 -- start start 2024-04-22T23:36:34.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.820+0000 7f42a8eca700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42a40fe470 0x7f42a410d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:34.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.820+0000 7f42a8eca700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42a40fedc0 0x7f42a410de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:34.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.820+0000 7f42a8eca700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a410e460 con 0x7f42a40fe470 2024-04-22T23:36:34.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.820+0000 7f42a8eca700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a410e5d0 con 0x7f42a40fedc0 2024-04-22T23:36:34.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.820+0000 7f42a259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42a40fe470 0x7f42a410d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:34.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.820+0000 7f42a259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42a40fe470 0x7f42a410d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45826/0 (socket says 172.21.15.107:45826) 2024-04-22T23:36:34.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.820+0000 7f42a259c700 1 -- 172.21.15.107:0/176959132 learned_addr learned my addr 172.21.15.107:0/176959132 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:36:34.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.820+0000 7f42a1d9b700 1 --2- 172.21.15.107:0/176959132 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42a40fedc0 0x7f42a410de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:34.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.820+0000 7f42a259c700 1 -- 172.21.15.107:0/176959132 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42a40fedc0 msgr2=0x7f42a410de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:34.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.820+0000 7f42a259c700 1 --2- 172.21.15.107:0/176959132 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42a40fedc0 0x7f42a410de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:34.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.820+0000 7f42a259c700 1 -- 172.21.15.107:0/176959132 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4298005040 con 0x7f42a40fe470 2024-04-22T23:36:34.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.820+0000 7f42a1d9b700 1 --2- 172.21.15.107:0/176959132 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42a40fedc0 0x7f42a410de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T23:36:34.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.820+0000 7f42a259c700 1 --2- 172.21.15.107:0/176959132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42a40fe470 0x7f42a410d920 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f429400eb40 tx=0x7f429400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:34.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.820+0000 7f4292ffd700 1 -- 172.21.15.107:0/176959132 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4294007cc0 con 0x7f42a40fe470 2024-04-22T23:36:34.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.820+0000 7f4292ffd700 1 -- 172.21.15.107:0/176959132 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f429400f040 con 0x7f42a40fe470 2024-04-22T23:36:34.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.820+0000 7f42a8eca700 1 -- 172.21.15.107:0/176959132 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42a4112fd0 con 0x7f42a40fe470 2024-04-22T23:36:34.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.820+0000 7f4292ffd700 1 -- 172.21.15.107:0/176959132 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4294009620 con 0x7f42a40fe470 2024-04-22T23:36:34.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.821+0000 7f42a8eca700 1 -- 172.21.15.107:0/176959132 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42a4113480 con 0x7f42a40fe470 2024-04-22T23:36:34.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.821+0000 7f42a8eca700 1 -- 172.21.15.107:0/176959132 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42a404c710 con 0x7f42a40fe470 2024-04-22T23:36:34.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.822+0000 7f4292ffd700 1 -- 172.21.15.107:0/176959132 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) v1 ==== 89652+0+0 (secure 0 0 0) 0x7f4294009780 con 0x7f42a40fe470 2024-04-22T23:36:34.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.822+0000 7f4292ffd700 1 --2- 172.21.15.107:0/176959132 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f428c06bd00 0x7f428c06e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:34.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.822+0000 7f4292ffd700 1 -- 172.21.15.107:0/176959132 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f429408a6b0 con 0x7f42a40fe470 2024-04-22T23:36:34.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.823+0000 7f42a1d9b700 1 --2- 172.21.15.107:0/176959132 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f428c06bd00 0x7f428c06e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:34.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.823+0000 7f42a1d9b700 1 --2- 172.21.15.107:0/176959132 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f428c06bd00 0x7f428c06e1b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4298000c00 tx=0x7f4298009e30 comp rx=0 tx=0).ready entity=mgr.14720 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:34.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:34.827+0000 7f4292ffd700 1 -- 172.21.15.107:0/176959132 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187010 (secure 0 0 0) 0x7f4294050aa0 con 0x7f42a40fe470 2024-04-22T23:36:34.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:34 smithi157 ceph-mon[33877]: Updating smithi107:/var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T23:36:34.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:34 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:34.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:34 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:34.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:34 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:34.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:34 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:34.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:34 smithi157 ceph-mon[33877]: Reconfiguring node-exporter.smithi107 (dependencies changed)... 2024-04-22T23:36:34.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:34 smithi157 ceph-mon[33877]: Reconfiguring daemon node-exporter.smithi107 on smithi107 2024-04-22T23:36:34.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:34 smithi157 ceph-mon[33877]: from='client.14750 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:36:34.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:34 smithi157 ceph-mon[33877]: pgmap v11: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 767 B/s wr, 10 op/s 2024-04-22T23:36:35.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.130+0000 7f42a8eca700 1 -- 172.21.15.107:0/176959132 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f42a41136f0 con 0x7f42a40fe470 2024-04-22T23:36:35.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.130+0000 7f4292ffd700 1 -- 172.21.15.107:0/176959132 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7f4294057850 con 0x7f42a40fe470 2024-04-22T23:36:35.125 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-22T23:36:35.125 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-04-22T23:36:35.126 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 2 2024-04-22T23:36:35.126 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:36:35.126 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-04-22T23:36:35.126 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 1, 2024-04-22T23:36:35.126 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 1 2024-04-22T23:36:35.126 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:36:35.126 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-04-22T23:36:35.126 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 6 2024-04-22T23:36:35.126 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:36:35.126 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-04-22T23:36:35.126 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 4 2024-04-22T23:36:35.127 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:36:35.127 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-04-22T23:36:35.127 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 13, 2024-04-22T23:36:35.127 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 1 2024-04-22T23:36:35.127 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T23:36:35.127 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-22T23:36:35.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.132+0000 7f42a8eca700 1 -- 172.21.15.107:0/176959132 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f428c06bd00 msgr2=0x7f428c06e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:35.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.132+0000 7f42a8eca700 1 --2- 172.21.15.107:0/176959132 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f428c06bd00 0x7f428c06e1b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4298000c00 tx=0x7f4298009e30 comp rx=0 tx=0).stop 2024-04-22T23:36:35.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.132+0000 7f42a8eca700 1 -- 172.21.15.107:0/176959132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42a40fe470 msgr2=0x7f42a410d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:35.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.132+0000 7f42a8eca700 1 --2- 172.21.15.107:0/176959132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42a40fe470 0x7f42a410d920 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f429400eb40 tx=0x7f429400b040 comp rx=0 tx=0).stop 2024-04-22T23:36:35.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.132+0000 7f42a8eca700 1 -- 172.21.15.107:0/176959132 shutdown_connections 2024-04-22T23:36:35.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.132+0000 7f42a8eca700 1 --2- 172.21.15.107:0/176959132 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f428c06bd00 0x7f428c06e1b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:35.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.132+0000 7f42a8eca700 1 --2- 172.21.15.107:0/176959132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42a40fe470 0x7f42a410d920 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:35.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.132+0000 7f42a8eca700 1 --2- 172.21.15.107:0/176959132 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42a40fedc0 0x7f42a410de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:35.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.132+0000 7f42a8eca700 1 -- 172.21.15.107:0/176959132 >> 172.21.15.107:0/176959132 conn(0x7f42a40f6c10 msgr2=0x7f42a40f82c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:35.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.133+0000 7f42a8eca700 1 -- 172.21.15.107:0/176959132 shutdown_connections 2024-04-22T23:36:35.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.133+0000 7f42a8eca700 1 -- 172.21.15.107:0/176959132 wait complete. 2024-04-22T23:36:35.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.242+0000 7f8214619700 1 -- 172.21.15.107:0/3805124504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f820c0fce50 msgr2=0x7f820c0fd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:35.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.242+0000 7f8214619700 1 --2- 172.21.15.107:0/3805124504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f820c0fce50 0x7f820c0fd260 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f81fc004940 tx=0x7f81fc009e30 comp rx=0 tx=0).stop 2024-04-22T23:36:35.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.243+0000 7f8214619700 1 -- 172.21.15.107:0/3805124504 shutdown_connections 2024-04-22T23:36:35.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.243+0000 7f8214619700 1 --2- 172.21.15.107:0/3805124504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f820c0fe050 0x7f820c0fe4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:35.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.243+0000 7f8214619700 1 --2- 172.21.15.107:0/3805124504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f820c0fce50 0x7f820c0fd260 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:35.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.243+0000 7f8214619700 1 -- 172.21.15.107:0/3805124504 >> 172.21.15.107:0/3805124504 conn(0x7f820c0f8540 msgr2=0x7f820c0fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:35.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.243+0000 7f8214619700 1 -- 172.21.15.107:0/3805124504 shutdown_connections 2024-04-22T23:36:35.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.243+0000 7f8214619700 1 -- 172.21.15.107:0/3805124504 wait complete. 2024-04-22T23:36:35.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.243+0000 7f8214619700 1 Processor -- start 2024-04-22T23:36:35.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.243+0000 7f8214619700 1 -- start start 2024-04-22T23:36:35.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.243+0000 7f8214619700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f820c0fce50 0x7f820c10d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:35.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.243+0000 7f8214619700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f820c0fe050 0x7f820c10deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:35.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.243+0000 7f8214619700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820c10e4b0 con 0x7f820c0fce50 2024-04-22T23:36:35.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.243+0000 7f8214619700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820c10e620 con 0x7f820c0fe050 2024-04-22T23:36:35.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.243+0000 7f82123b5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f820c0fce50 0x7f820c10d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:35.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.243+0000 7f82123b5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f820c0fce50 0x7f820c10d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45842/0 (socket says 172.21.15.107:45842) 2024-04-22T23:36:35.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.243+0000 7f82123b5700 1 -- 172.21.15.107:0/400082695 learned_addr learned my addr 172.21.15.107:0/400082695 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:36:35.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.243+0000 7f8211bb4700 1 --2- 172.21.15.107:0/400082695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f820c0fe050 0x7f820c10deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:35.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.244+0000 7f82123b5700 1 -- 172.21.15.107:0/400082695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f820c0fe050 msgr2=0x7f820c10deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:35.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.244+0000 7f82123b5700 1 --2- 172.21.15.107:0/400082695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f820c0fe050 0x7f820c10deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:35.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.244+0000 7f82123b5700 1 -- 172.21.15.107:0/400082695 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81fc005040 con 0x7f820c0fce50 2024-04-22T23:36:35.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.244+0000 7f82123b5700 1 --2- 172.21.15.107:0/400082695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f820c0fce50 0x7f820c10d970 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f81fc00b3b0 tx=0x7f81fc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:35.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.244+0000 7f8202ffd700 1 -- 172.21.15.107:0/400082695 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81fc00e040 con 0x7f820c0fce50 2024-04-22T23:36:35.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.244+0000 7f8202ffd700 1 -- 172.21.15.107:0/400082695 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f81fc00bbf0 con 0x7f820c0fce50 2024-04-22T23:36:35.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.244+0000 7f8214619700 1 -- 172.21.15.107:0/400082695 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f820c112fb0 con 0x7f820c0fce50 2024-04-22T23:36:35.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.244+0000 7f8202ffd700 1 -- 172.21.15.107:0/400082695 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81fc01c630 con 0x7f820c0fce50 2024-04-22T23:36:35.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.244+0000 7f8214619700 1 -- 172.21.15.107:0/400082695 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f820c1134e0 con 0x7f820c0fce50 2024-04-22T23:36:35.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.245+0000 7f8214619700 1 -- 172.21.15.107:0/400082695 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f820c04c710 con 0x7f820c0fce50 2024-04-22T23:36:35.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.246+0000 7f8202ffd700 1 -- 172.21.15.107:0/400082695 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) v1 ==== 89652+0+0 (secure 0 0 0) 0x7f81fc00bd60 con 0x7f820c0fce50 2024-04-22T23:36:35.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.246+0000 7f8202ffd700 1 --2- 172.21.15.107:0/400082695 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f81f806bd00 0x7f81f806e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:35.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.247+0000 7f8202ffd700 1 -- 172.21.15.107:0/400082695 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f81fc08bde0 con 0x7f820c0fce50 2024-04-22T23:36:35.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.247+0000 7f8211bb4700 1 --2- 172.21.15.107:0/400082695 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f81f806bd00 0x7f81f806e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:35.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.247+0000 7f8211bb4700 1 --2- 172.21.15.107:0/400082695 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f81f806bd00 0x7f81f806e1b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8208004f40 tx=0x7f820800b040 comp rx=0 tx=0).ready entity=mgr.14720 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:35.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.250+0000 7f8202ffd700 1 -- 172.21.15.107:0/400082695 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187010 (secure 0 0 0) 0x7f81fc058ff0 con 0x7f820c0fce50 2024-04-22T23:36:35.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.526+0000 7f8214619700 1 -- 172.21.15.107:0/400082695 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f820c113af0 con 0x7f820c0fce50 2024-04-22T23:36:35.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.526+0000 7f8202ffd700 1 -- 172.21.15.107:0/400082695 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) v1 ==== 76+0+1858 (secure 0 0 0) 0x7f81fc058b80 con 0x7f820c0fce50 2024-04-22T23:36:35.521 INFO:teuthology.orchestra.run.smithi107.stdout:e19 2024-04-22T23:36:35.521 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T23:36:35.521 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:36:35.521 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-04-22T23:36:35.521 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:36:35.522 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-04-22T23:36:35.522 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-04-22T23:36:35.522 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 19 2024-04-22T23:36:35.522 INFO:teuthology.orchestra.run.smithi107.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-22T23:36:35.522 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-04-22T23:32:20.146130+0000 2024-04-22T23:36:35.522 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-04-22T23:35:12.338678+0000 2024-04-22T23:36:35.522 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-04-22T23:36:35.522 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-04-22T23:36:35.522 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-04-22T23:36:35.522 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-04-22T23:36:35.522 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-04-22T23:36:35.523 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-04-22T23:36:35.523 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-04-22T23:36:35.523 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 46 2024-04-22T23:36:35.523 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:36:35.523 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 2 2024-04-22T23:36:35.523 INFO:teuthology.orchestra.run.smithi107.stdout:in 0,1 2024-04-22T23:36:35.523 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=14468,1=24265} 2024-04-22T23:36:35.523 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-04-22T23:36:35.523 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-04-22T23:36:35.523 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-04-22T23:36:35.523 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-04-22T23:36:35.523 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-04-22T23:36:35.524 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data disabled 2024-04-22T23:36:35.524 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-04-22T23:36:35.524 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-04-22T23:36:35.524 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.ktuxjl{0:14468} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.107:6828/3335313253,v1:172.21.15.107:6829/3335313253] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:36:35.524 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi157.rcaris{1:24265} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.157:6824/2779251893,v1:172.21.15.157:6825/2779251893] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:36:35.524 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:36:35.524 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:36:35.524 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-04-22T23:36:35.524 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:36:35.524 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.dfvvkw{-1:14484} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/3548717247,v1:172.21.15.107:6827/3548717247] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:36:35.524 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi157.ufywse{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.157:6826/4072444199,v1:172.21.15.157:6827/4072444199] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:36:35.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.528+0000 7f8214619700 1 -- 172.21.15.107:0/400082695 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f81f806bd00 msgr2=0x7f81f806e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:35.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.528+0000 7f8214619700 1 --2- 172.21.15.107:0/400082695 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f81f806bd00 0x7f81f806e1b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8208004f40 tx=0x7f820800b040 comp rx=0 tx=0).stop 2024-04-22T23:36:35.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.529+0000 7f8214619700 1 -- 172.21.15.107:0/400082695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f820c0fce50 msgr2=0x7f820c10d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:35.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.529+0000 7f8214619700 1 --2- 172.21.15.107:0/400082695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f820c0fce50 0x7f820c10d970 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f81fc00b3b0 tx=0x7f81fc009e30 comp rx=0 tx=0).stop 2024-04-22T23:36:35.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.529+0000 7f8214619700 1 -- 172.21.15.107:0/400082695 shutdown_connections 2024-04-22T23:36:35.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.529+0000 7f8214619700 1 --2- 172.21.15.107:0/400082695 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f81f806bd00 0x7f81f806e1b0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:35.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.529+0000 7f8214619700 1 --2- 172.21.15.107:0/400082695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f820c0fce50 0x7f820c10d970 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:35.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.529+0000 7f8214619700 1 --2- 172.21.15.107:0/400082695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f820c0fe050 0x7f820c10deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:35.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.529+0000 7f8214619700 1 -- 172.21.15.107:0/400082695 >> 172.21.15.107:0/400082695 conn(0x7f820c0f8540 msgr2=0x7f820c0f9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:35.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.529+0000 7f8214619700 1 -- 172.21.15.107:0/400082695 shutdown_connections 2024-04-22T23:36:35.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.529+0000 7f8214619700 1 -- 172.21.15.107:0/400082695 wait complete. 2024-04-22T23:36:35.526 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 19 2024-04-22T23:36:35.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.632+0000 7faa225e5700 1 -- 172.21.15.107:0/1896921340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa1c0fbce0 msgr2=0x7faa1c0fc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:35.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.632+0000 7faa225e5700 1 --2- 172.21.15.107:0/1896921340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa1c0fbce0 0x7faa1c0fc0f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7faa0c004940 tx=0x7faa0c009e30 comp rx=0 tx=0).stop 2024-04-22T23:36:35.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.632+0000 7faa225e5700 1 -- 172.21.15.107:0/1896921340 shutdown_connections 2024-04-22T23:36:35.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.632+0000 7faa225e5700 1 --2- 172.21.15.107:0/1896921340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa1c0fcf90 0x7faa1c0fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:35.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.632+0000 7faa225e5700 1 --2- 172.21.15.107:0/1896921340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa1c0fbce0 0x7faa1c0fc0f0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:35.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.632+0000 7faa225e5700 1 -- 172.21.15.107:0/1896921340 >> 172.21.15.107:0/1896921340 conn(0x7faa1c0f7440 msgr2=0x7faa1c0f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:35.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.632+0000 7faa225e5700 1 -- 172.21.15.107:0/1896921340 shutdown_connections 2024-04-22T23:36:35.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.632+0000 7faa225e5700 1 -- 172.21.15.107:0/1896921340 wait complete. 2024-04-22T23:36:35.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.632+0000 7faa225e5700 1 Processor -- start 2024-04-22T23:36:35.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.632+0000 7faa225e5700 1 -- start start 2024-04-22T23:36:35.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.632+0000 7faa225e5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa1c0fbce0 0x7faa1c10d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:35.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.632+0000 7faa225e5700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa1c0fcf90 0x7faa1c10dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:35.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.632+0000 7faa225e5700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c10e4e0 con 0x7faa1c0fbce0 2024-04-22T23:36:35.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.632+0000 7faa225e5700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c10e650 con 0x7faa1c0fcf90 2024-04-22T23:36:35.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.633+0000 7faa1bfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa1c0fbce0 0x7faa1c10d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:35.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.633+0000 7faa1bfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa1c0fbce0 0x7faa1c10d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45858/0 (socket says 172.21.15.107:45858) 2024-04-22T23:36:35.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.633+0000 7faa1bfff700 1 -- 172.21.15.107:0/3674043158 learned_addr learned my addr 172.21.15.107:0/3674043158 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:36:35.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.633+0000 7faa1b7fe700 1 --2- 172.21.15.107:0/3674043158 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa1c0fcf90 0x7faa1c10dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:35.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.633+0000 7faa1bfff700 1 -- 172.21.15.107:0/3674043158 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa1c0fcf90 msgr2=0x7faa1c10dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:35.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.633+0000 7faa1bfff700 1 --2- 172.21.15.107:0/3674043158 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa1c0fcf90 0x7faa1c10dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:35.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.633+0000 7faa1bfff700 1 -- 172.21.15.107:0/3674043158 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa0c005040 con 0x7faa1c0fbce0 2024-04-22T23:36:35.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.633+0000 7faa1bfff700 1 --2- 172.21.15.107:0/3674043158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa1c0fbce0 0x7faa1c10d9a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7faa0c00b3b0 tx=0x7faa0c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:35.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.633+0000 7faa18ff9700 1 -- 172.21.15.107:0/3674043158 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa0c00e040 con 0x7faa1c0fbce0 2024-04-22T23:36:35.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.633+0000 7faa225e5700 1 -- 172.21.15.107:0/3674043158 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa1c113000 con 0x7faa1c0fbce0 2024-04-22T23:36:35.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.633+0000 7faa18ff9700 1 -- 172.21.15.107:0/3674043158 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faa0c00bcc0 con 0x7faa1c0fbce0 2024-04-22T23:36:35.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.633+0000 7faa18ff9700 1 -- 172.21.15.107:0/3674043158 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa0c01c630 con 0x7faa1c0fbce0 2024-04-22T23:36:35.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.633+0000 7faa225e5700 1 -- 172.21.15.107:0/3674043158 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa1c1135c0 con 0x7faa1c0fbce0 2024-04-22T23:36:35.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.634+0000 7faa225e5700 1 -- 172.21.15.107:0/3674043158 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa1c101b20 con 0x7faa1c0fbce0 2024-04-22T23:36:35.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.635+0000 7faa18ff9700 1 -- 172.21.15.107:0/3674043158 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) v1 ==== 89652+0+0 (secure 0 0 0) 0x7faa0c013980 con 0x7faa1c0fbce0 2024-04-22T23:36:35.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.635+0000 7faa18ff9700 1 --2- 172.21.15.107:0/3674043158 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7faa0406bd00 0x7faa0406e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:35.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.635+0000 7faa18ff9700 1 -- 172.21.15.107:0/3674043158 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7faa0c08b7f0 con 0x7faa1c0fbce0 2024-04-22T23:36:35.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.635+0000 7faa1b7fe700 1 --2- 172.21.15.107:0/3674043158 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7faa0406bd00 0x7faa0406e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:35.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.636+0000 7faa1b7fe700 1 --2- 172.21.15.107:0/3674043158 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7faa0406bd00 0x7faa0406e1b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7faa10007fd0 tx=0x7faa10009040 comp rx=0 tx=0).ready entity=mgr.14720 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:35.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.640+0000 7faa18ff9700 1 -- 172.21.15.107:0/3674043158 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187010 (secure 0 0 0) 0x7faa0c058980 con 0x7faa1c0fbce0 2024-04-22T23:36:35.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:35 smithi107 ceph-mon[26355]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:36:35.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:35 smithi107 ceph-mon[26355]: from='client.14758 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:36:35.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:35 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/176959132' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:36:35.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.865+0000 7faa225e5700 1 -- 172.21.15.107:0/3674043158 --> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7faa1c05e400 con 0x7faa0406bd00 2024-04-22T23:36:35.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.867+0000 7faa18ff9700 1 -- 172.21.15.107:0/3674043158 <== mgr.14720 v2:172.21.15.157:6828/3332906296 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7faa1c05e400 con 0x7faa0406bd00 2024-04-22T23:36:35.862 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-22T23:36:35.862 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c", 2024-04-22T23:36:35.862 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-04-22T23:36:35.862 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-22T23:36:35.862 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [], 2024-04-22T23:36:35.862 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "1/21 daemons upgraded", 2024-04-22T23:36:35.862 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "", 2024-04-22T23:36:35.862 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-04-22T23:36:35.862 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-22T23:36:35.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.869+0000 7faa225e5700 1 -- 172.21.15.107:0/3674043158 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7faa0406bd00 msgr2=0x7faa0406e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:35.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.869+0000 7faa225e5700 1 --2- 172.21.15.107:0/3674043158 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7faa0406bd00 0x7faa0406e1b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7faa10007fd0 tx=0x7faa10009040 comp rx=0 tx=0).stop 2024-04-22T23:36:35.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.869+0000 7faa225e5700 1 -- 172.21.15.107:0/3674043158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa1c0fbce0 msgr2=0x7faa1c10d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:35.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.869+0000 7faa225e5700 1 --2- 172.21.15.107:0/3674043158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa1c0fbce0 0x7faa1c10d9a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7faa0c00b3b0 tx=0x7faa0c009e30 comp rx=0 tx=0).stop 2024-04-22T23:36:35.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.870+0000 7faa225e5700 1 -- 172.21.15.107:0/3674043158 shutdown_connections 2024-04-22T23:36:35.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.870+0000 7faa225e5700 1 --2- 172.21.15.107:0/3674043158 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7faa0406bd00 0x7faa0406e1b0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:35.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.870+0000 7faa225e5700 1 --2- 172.21.15.107:0/3674043158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa1c0fbce0 0x7faa1c10d9a0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:35.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.870+0000 7faa225e5700 1 --2- 172.21.15.107:0/3674043158 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa1c0fcf90 0x7faa1c10dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:35.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.870+0000 7faa225e5700 1 -- 172.21.15.107:0/3674043158 >> 172.21.15.107:0/3674043158 conn(0x7faa1c0f7440 msgr2=0x7faa1c066890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:35.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.870+0000 7faa225e5700 1 -- 172.21.15.107:0/3674043158 shutdown_connections 2024-04-22T23:36:35.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.870+0000 7faa225e5700 1 -- 172.21.15.107:0/3674043158 wait complete. 2024-04-22T23:36:35.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:35 smithi157 ceph-mon[33877]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:36:35.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:35 smithi157 ceph-mon[33877]: from='client.14758 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:36:35.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:35 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/176959132' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:36:35.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.975+0000 7fa914174700 1 -- 172.21.15.107:0/3723254675 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa90c0fff20 msgr2=0x7fa90c102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:35.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.975+0000 7fa914174700 1 --2- 172.21.15.107:0/3723254675 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa90c0fff20 0x7fa90c102340 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa900004940 tx=0x7fa900009e30 comp rx=0 tx=0).stop 2024-04-22T23:36:35.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.975+0000 7fa914174700 1 -- 172.21.15.107:0/3723254675 shutdown_connections 2024-04-22T23:36:35.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.975+0000 7fa914174700 1 --2- 172.21.15.107:0/3723254675 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa90c0fff20 0x7fa90c102340 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:35.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.975+0000 7fa914174700 1 --2- 172.21.15.107:0/3723254675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa90c066f60 0x7fa90c0ff9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:35.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.975+0000 7fa914174700 1 -- 172.21.15.107:0/3723254675 >> 172.21.15.107:0/3723254675 conn(0x7fa90c0f7450 msgr2=0x7fa90c0f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:35.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.975+0000 7fa914174700 1 -- 172.21.15.107:0/3723254675 shutdown_connections 2024-04-22T23:36:35.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.975+0000 7fa914174700 1 -- 172.21.15.107:0/3723254675 wait complete. 2024-04-22T23:36:35.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.975+0000 7fa914174700 1 Processor -- start 2024-04-22T23:36:35.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.975+0000 7fa914174700 1 -- start start 2024-04-22T23:36:35.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.975+0000 7fa914174700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa90c066f60 0x7fa90c10dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:35.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.975+0000 7fa914174700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa90c0fff20 0x7fa90c10e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:35.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.975+0000 7fa914174700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa90c10e910 con 0x7fa90c0fff20 2024-04-22T23:36:35.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.975+0000 7fa914174700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa90c113220 con 0x7fa90c066f60 2024-04-22T23:36:35.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.976+0000 7fa91170f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa90c0fff20 0x7fa90c10e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:35.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.976+0000 7fa91170f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa90c0fff20 0x7fa90c10e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45876/0 (socket says 172.21.15.107:45876) 2024-04-22T23:36:35.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.976+0000 7fa91170f700 1 -- 172.21.15.107:0/108926307 learned_addr learned my addr 172.21.15.107:0/108926307 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:36:35.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.976+0000 7fa911f10700 1 --2- 172.21.15.107:0/108926307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa90c066f60 0x7fa90c10dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:35.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.976+0000 7fa91170f700 1 -- 172.21.15.107:0/108926307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa90c066f60 msgr2=0x7fa90c10dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:35.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.976+0000 7fa91170f700 1 --2- 172.21.15.107:0/108926307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa90c066f60 0x7fa90c10dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:35.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.976+0000 7fa91170f700 1 -- 172.21.15.107:0/108926307 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa900005040 con 0x7fa90c0fff20 2024-04-22T23:36:35.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.976+0000 7fa911f10700 1 --2- 172.21.15.107:0/108926307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa90c066f60 0x7fa90c10dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T23:36:35.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.976+0000 7fa91170f700 1 --2- 172.21.15.107:0/108926307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa90c0fff20 0x7fa90c10e200 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa90000b3b0 tx=0x7fa900009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:35.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.976+0000 7fa90a7fc700 1 -- 172.21.15.107:0/108926307 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa90000e040 con 0x7fa90c0fff20 2024-04-22T23:36:35.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.976+0000 7fa90a7fc700 1 -- 172.21.15.107:0/108926307 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa90000bcc0 con 0x7fa90c0fff20 2024-04-22T23:36:35.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.976+0000 7fa914174700 1 -- 172.21.15.107:0/108926307 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa90c113420 con 0x7fa90c0fff20 2024-04-22T23:36:35.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.976+0000 7fa90a7fc700 1 -- 172.21.15.107:0/108926307 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa90001c630 con 0x7fa90c0fff20 2024-04-22T23:36:35.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.976+0000 7fa914174700 1 -- 172.21.15.107:0/108926307 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa90c1139b0 con 0x7fa90c0fff20 2024-04-22T23:36:35.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.977+0000 7fa914174700 1 -- 172.21.15.107:0/108926307 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa90c106180 con 0x7fa90c0fff20 2024-04-22T23:36:35.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.978+0000 7fa90a7fc700 1 -- 172.21.15.107:0/108926307 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) v1 ==== 89652+0+0 (secure 0 0 0) 0x7fa900013980 con 0x7fa90c0fff20 2024-04-22T23:36:35.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.978+0000 7fa90a7fc700 1 --2- 172.21.15.107:0/108926307 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7fa8f406bd00 0x7fa8f406e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:36:35.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.978+0000 7fa90a7fc700 1 -- 172.21.15.107:0/108926307 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fa90008b700 con 0x7fa90c0fff20 2024-04-22T23:36:35.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.978+0000 7fa911f10700 1 --2- 172.21.15.107:0/108926307 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7fa8f406bd00 0x7fa8f406e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:36:35.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.979+0000 7fa911f10700 1 --2- 172.21.15.107:0/108926307 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7fa8f406bd00 0x7fa8f406e1b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa90c0fea90 tx=0x7fa8fc0096a0 comp rx=0 tx=0).ready entity=mgr.14720 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:36:35.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:35.982+0000 7fa90a7fc700 1 -- 172.21.15.107:0/108926307 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187010 (secure 0 0 0) 0x7fa900054de0 con 0x7fa90c0fff20 2024-04-22T23:36:36.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:36.282+0000 7fa914174700 1 -- 172.21.15.107:0/108926307 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa90c0676c0 con 0x7fa90c0fff20 2024-04-22T23:36:36.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:36.282+0000 7fa90a7fc700 1 -- 172.21.15.107:0/108926307 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fa9000584a0 con 0x7fa90c0fff20 2024-04-22T23:36:36.277 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_OK 2024-04-22T23:36:36.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:36.285+0000 7fa914174700 1 -- 172.21.15.107:0/108926307 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7fa8f406bd00 msgr2=0x7fa8f406e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:36.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:36.285+0000 7fa914174700 1 --2- 172.21.15.107:0/108926307 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7fa8f406bd00 0x7fa8f406e1b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa90c0fea90 tx=0x7fa8fc0096a0 comp rx=0 tx=0).stop 2024-04-22T23:36:36.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:36.285+0000 7fa914174700 1 -- 172.21.15.107:0/108926307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa90c0fff20 msgr2=0x7fa90c10e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:36:36.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:36.285+0000 7fa914174700 1 --2- 172.21.15.107:0/108926307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa90c0fff20 0x7fa90c10e200 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa90000b3b0 tx=0x7fa900009e30 comp rx=0 tx=0).stop 2024-04-22T23:36:36.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:36.285+0000 7fa914174700 1 -- 172.21.15.107:0/108926307 shutdown_connections 2024-04-22T23:36:36.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:36.285+0000 7fa914174700 1 --2- 172.21.15.107:0/108926307 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7fa8f406bd00 0x7fa8f406e1b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:36.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:36.285+0000 7fa914174700 1 --2- 172.21.15.107:0/108926307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa90c066f60 0x7fa90c10dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:36.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:36.285+0000 7fa914174700 1 --2- 172.21.15.107:0/108926307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa90c0fff20 0x7fa90c10e200 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:36:36.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:36.285+0000 7fa914174700 1 -- 172.21.15.107:0/108926307 >> 172.21.15.107:0/108926307 conn(0x7fa90c0f7450 msgr2=0x7fa90c0f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:36:36.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:36.285+0000 7fa914174700 1 -- 172.21.15.107:0/108926307 shutdown_connections 2024-04-22T23:36:36.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:36:36.285+0000 7fa914174700 1 -- 172.21.15.107:0/108926307 wait complete. 2024-04-22T23:36:36.556 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:36 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/400082695' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:36:36.556 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:36 smithi107 ceph-mon[26355]: from='client.14770 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:36:36.556 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:36 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:36.556 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:36 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:36.556 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:36 smithi107 ceph-mon[26355]: Reconfiguring alertmanager.smithi107 (dependencies changed)... 2024-04-22T23:36:36.556 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:36 smithi107 ceph-mon[26355]: Reconfiguring daemon alertmanager.smithi107 on smithi107 2024-04-22T23:36:36.557 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:36 smithi107 ceph-mon[26355]: pgmap v12: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-22T23:36:36.557 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:36 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/108926307' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:36:36.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:36 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/400082695' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:36:36.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:36 smithi157 ceph-mon[33877]: from='client.14770 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:36:36.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:36 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:36.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:36 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:36.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:36 smithi157 ceph-mon[33877]: Reconfiguring alertmanager.smithi107 (dependencies changed)... 2024-04-22T23:36:36.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:36 smithi157 ceph-mon[33877]: Reconfiguring daemon alertmanager.smithi107 on smithi107 2024-04-22T23:36:36.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:36 smithi157 ceph-mon[33877]: pgmap v12: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-22T23:36:36.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:36 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/108926307' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:36:38.296 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:38 smithi107 ceph-mon[26355]: pgmap v13: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-04-22T23:36:38.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:38 smithi157 ceph-mon[33877]: pgmap v13: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-04-22T23:36:40.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:40 smithi157 ceph-mon[33877]: pgmap v14: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 85 B/s rd, 341 B/s wr, 0 op/s 2024-04-22T23:36:40.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:40 smithi107 ceph-mon[26355]: pgmap v14: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 85 B/s rd, 341 B/s wr, 0 op/s 2024-04-22T23:36:42.377 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:42 smithi107 ceph-mon[26355]: pgmap v15: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 85 B/s rd, 170 B/s wr, 0 op/s 2024-04-22T23:36:42.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:42 smithi157 ceph-mon[33877]: pgmap v15: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 85 B/s rd, 170 B/s wr, 0 op/s 2024-04-22T23:36:44.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:43 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:44.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:43 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:44.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:43 smithi107 ceph-mon[26355]: Reconfiguring grafana.smithi107 (dependencies changed)... 2024-04-22T23:36:44.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:43 smithi107 ceph-mon[26355]: Reconfiguring daemon grafana.smithi107 on smithi107 2024-04-22T23:36:44.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:43 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:44.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:43 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:44.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:43 smithi157 ceph-mon[33877]: Reconfiguring grafana.smithi107 (dependencies changed)... 2024-04-22T23:36:44.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:43 smithi157 ceph-mon[33877]: Reconfiguring daemon grafana.smithi107 on smithi107 2024-04-22T23:36:45.215 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:44 smithi107 ceph-mon[26355]: pgmap v16: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 85 B/s rd, 170 B/s wr, 0 op/s 2024-04-22T23:36:45.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:44 smithi157 ceph-mon[33877]: pgmap v16: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 85 B/s rd, 170 B/s wr, 0 op/s 2024-04-22T23:36:46.216 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:46 smithi107 ceph-mon[26355]: pgmap v17: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 85 B/s rd, 341 B/s wr, 0 op/s 2024-04-22T23:36:46.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:46 smithi157 ceph-mon[33877]: pgmap v17: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 85 B/s rd, 341 B/s wr, 0 op/s 2024-04-22T23:36:48.862 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:48 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:48.862 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:48 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:48.862 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:48 smithi107 ceph-mon[26355]: Reconfiguring prometheus.smithi107 (dependencies changed)... 2024-04-22T23:36:48.862 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:48 smithi107 ceph-mon[26355]: Reconfiguring daemon prometheus.smithi107 on smithi107 2024-04-22T23:36:48.862 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:48 smithi107 ceph-mon[26355]: pgmap v18: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 85 B/s rd, 341 B/s wr, 0 op/s 2024-04-22T23:36:48.862 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:48 smithi107 ceph-mon[26355]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:36:49.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:48 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:49.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:48 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:49.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:48 smithi157 ceph-mon[33877]: Reconfiguring prometheus.smithi107 (dependencies changed)... 2024-04-22T23:36:49.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:48 smithi157 ceph-mon[33877]: Reconfiguring daemon prometheus.smithi107 on smithi107 2024-04-22T23:36:49.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:48 smithi157 ceph-mon[33877]: pgmap v18: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 85 B/s rd, 341 B/s wr, 0 op/s 2024-04-22T23:36:49.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:48 smithi157 ceph-mon[33877]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:36:50.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:50 smithi107 ceph-mon[26355]: pgmap v19: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 85 B/s rd, 341 B/s wr, 0 op/s 2024-04-22T23:36:50.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:50 smithi157 ceph-mon[33877]: pgmap v19: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 85 B/s rd, 341 B/s wr, 0 op/s 2024-04-22T23:36:52.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:52 smithi157 ceph-mon[33877]: pgmap v20: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-04-22T23:36:52.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:52 smithi107 ceph-mon[26355]: pgmap v20: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-04-22T23:36:54.314 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:54 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:54.314 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:54 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:54.314 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:54 smithi157 ceph-mon[33877]: Reconfiguring node-exporter.smithi157 (dependencies changed)... 2024-04-22T23:36:54.314 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:54 smithi157 ceph-mon[33877]: Reconfiguring daemon node-exporter.smithi157 on smithi157 2024-04-22T23:36:54.314 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:54 smithi157 ceph-mon[33877]: pgmap v21: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-04-22T23:36:54.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:54 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:54.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:54 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:54.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:54 smithi107 ceph-mon[26355]: Reconfiguring node-exporter.smithi157 (dependencies changed)... 2024-04-22T23:36:54.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:54 smithi107 ceph-mon[26355]: Reconfiguring daemon node-exporter.smithi157 on smithi157 2024-04-22T23:36:54.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:54 smithi107 ceph-mon[26355]: pgmap v21: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-04-22T23:36:56.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:56 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:56.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:56 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:56.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:56 smithi157 ceph-mon[33877]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-22T23:36:56.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:56 smithi157 ceph-mon[33877]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-22T23:36:56.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:56 smithi157 ceph-mon[33877]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi107.front.sepia.ceph.com:9093"}]: dispatch 2024-04-22T23:36:56.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:56 smithi157 ceph-mon[33877]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi107.front.sepia.ceph.com:9093"}]: dispatch 2024-04-22T23:36:56.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:56 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:56.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:56 smithi157 ceph-mon[33877]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-22T23:36:56.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:56 smithi157 ceph-mon[33877]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-22T23:36:56.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:56 smithi157 ceph-mon[33877]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi107.front.sepia.ceph.com:3000"}]: dispatch 2024-04-22T23:36:56.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:56 smithi157 ceph-mon[33877]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi107.front.sepia.ceph.com:3000"}]: dispatch 2024-04-22T23:36:56.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:56 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:56.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:56 smithi157 ceph-mon[33877]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-22T23:36:56.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:56 smithi157 ceph-mon[33877]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-22T23:36:56.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:56 smithi157 ceph-mon[33877]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi107.front.sepia.ceph.com:9095"}]: dispatch 2024-04-22T23:36:56.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:56 smithi157 ceph-mon[33877]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi107.front.sepia.ceph.com:9095"}]: dispatch 2024-04-22T23:36:56.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:56 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:56.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:56 smithi157 ceph-mon[33877]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:36:56.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:56 smithi157 ceph-mon[33877]: Upgrade: Need to upgrade myself (mgr.smithi157.tuylis) 2024-04-22T23:36:56.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:56 smithi157 ceph-mon[33877]: Upgrade: Need to upgrade myself (mgr.smithi157.tuylis) 2024-04-22T23:36:56.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:56 smithi157 ceph-mon[33877]: pgmap v22: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-04-22T23:36:56.634 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:56 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:56.634 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:56 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:56.634 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:56 smithi107 ceph-mon[26355]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-22T23:36:56.634 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:56 smithi107 ceph-mon[26355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-22T23:36:56.634 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:56 smithi107 ceph-mon[26355]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi107.front.sepia.ceph.com:9093"}]: dispatch 2024-04-22T23:36:56.634 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:56 smithi107 ceph-mon[26355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi107.front.sepia.ceph.com:9093"}]: dispatch 2024-04-22T23:36:56.634 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:56 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:56.634 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:56 smithi107 ceph-mon[26355]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-22T23:36:56.635 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:56 smithi107 ceph-mon[26355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-22T23:36:56.635 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:56 smithi107 ceph-mon[26355]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi107.front.sepia.ceph.com:3000"}]: dispatch 2024-04-22T23:36:56.635 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:56 smithi107 ceph-mon[26355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi107.front.sepia.ceph.com:3000"}]: dispatch 2024-04-22T23:36:56.635 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:56 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:56.635 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:56 smithi107 ceph-mon[26355]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-22T23:36:56.635 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:56 smithi107 ceph-mon[26355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-22T23:36:56.635 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:56 smithi107 ceph-mon[26355]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi107.front.sepia.ceph.com:9095"}]: dispatch 2024-04-22T23:36:56.635 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:56 smithi107 ceph-mon[26355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi107.front.sepia.ceph.com:9095"}]: dispatch 2024-04-22T23:36:56.635 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:56 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:56.635 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:56 smithi107 ceph-mon[26355]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:36:56.635 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:56 smithi107 ceph-mon[26355]: Upgrade: Need to upgrade myself (mgr.smithi157.tuylis) 2024-04-22T23:36:56.635 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:56 smithi107 ceph-mon[26355]: Upgrade: Need to upgrade myself (mgr.smithi157.tuylis) 2024-04-22T23:36:56.635 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:56 smithi107 ceph-mon[26355]: pgmap v22: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-04-22T23:36:58.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:58 smithi107 ceph-mon[26355]: Upgrade: Updating mgr.smithi107.uipxvm 2024-04-22T23:36:58.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:58 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:58.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:58 smithi107 ceph-mon[26355]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi107.uipxvm", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T23:36:58.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:58 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi107.uipxvm", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T23:36:58.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:58 smithi107 ceph-mon[26355]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T23:36:58.234 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:58 smithi107 ceph-mon[26355]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:36:58.234 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:58 smithi107 ceph-mon[26355]: Deploying daemon mgr.smithi107.uipxvm on smithi107 2024-04-22T23:36:58.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:58 smithi157 ceph-mon[33877]: Upgrade: Updating mgr.smithi107.uipxvm 2024-04-22T23:36:58.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:58 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:36:58.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:58 smithi157 ceph-mon[33877]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi107.uipxvm", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T23:36:58.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:58 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi107.uipxvm", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T23:36:58.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:58 smithi157 ceph-mon[33877]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T23:36:58.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:58 smithi157 ceph-mon[33877]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:36:58.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:58 smithi157 ceph-mon[33877]: Deploying daemon mgr.smithi107.uipxvm on smithi107 2024-04-22T23:36:59.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:36:59 smithi107 ceph-mon[26355]: pgmap v23: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:36:59.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:36:59 smithi157 ceph-mon[33877]: pgmap v23: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:37:00.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:00 smithi157 ceph-mon[33877]: pgmap v24: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:37:00.426 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:00 smithi107 ceph-mon[26355]: pgmap v24: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:37:02.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:01 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:37:02.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:01 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:37:02.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:01 smithi157 ceph-mon[33877]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:37:02.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:01 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:37:02.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:01 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:37:02.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:01 smithi107 ceph-mon[26355]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:37:03.107 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:02 smithi107 ceph-mon[26355]: pgmap v25: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:37:03.128 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:02 smithi157 ceph-mon[33877]: pgmap v25: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:37:04.054 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:03 smithi107 ceph-mon[26355]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:37:04.054 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:03 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:37:04.054 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:03 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:37:04.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:03 smithi157 ceph-mon[33877]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:37:04.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:03 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:37:04.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:03 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:37:05.116 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:04 smithi107 ceph-mon[26355]: pgmap v26: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:37:05.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:04 smithi157 ceph-mon[33877]: pgmap v26: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:37:06.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.393+0000 7f39f12ab700 1 -- 172.21.15.107:0/4158903481 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39ec066f60 msgr2=0x7f39ec0ff9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:06.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.393+0000 7f39f12ab700 1 --2- 172.21.15.107:0/4158903481 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39ec066f60 0x7f39ec0ff9e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f39dc004940 tx=0x7f39dc009e30 comp rx=0 tx=0).stop 2024-04-22T23:37:06.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.393+0000 7f39f12ab700 1 -- 172.21.15.107:0/4158903481 shutdown_connections 2024-04-22T23:37:06.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.393+0000 7f39f12ab700 1 --2- 172.21.15.107:0/4158903481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39ec0fff20 0x7f39ec102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:06.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.393+0000 7f39f12ab700 1 --2- 172.21.15.107:0/4158903481 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39ec066f60 0x7f39ec0ff9e0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:06.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.393+0000 7f39f12ab700 1 -- 172.21.15.107:0/4158903481 >> 172.21.15.107:0/4158903481 conn(0x7f39ec0f7450 msgr2=0x7f39ec0f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:06.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.393+0000 7f39f12ab700 1 -- 172.21.15.107:0/4158903481 shutdown_connections 2024-04-22T23:37:06.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.393+0000 7f39f12ab700 1 -- 172.21.15.107:0/4158903481 wait complete. 2024-04-22T23:37:06.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.394+0000 7f39f12ab700 1 Processor -- start 2024-04-22T23:37:06.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.394+0000 7f39f12ab700 1 -- start start 2024-04-22T23:37:06.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.394+0000 7f39f12ab700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39ec066f60 0x7f39ec0fd230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:06.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.394+0000 7f39f12ab700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39ec0fff20 0x7f39ec0fb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:06.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.394+0000 7f39f12ab700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ec0fd830 con 0x7f39ec0fff20 2024-04-22T23:37:06.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.394+0000 7f39f12ab700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ec0fbe20 con 0x7f39ec066f60 2024-04-22T23:37:06.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.395+0000 7f39ea7fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39ec0fff20 0x7f39ec0fb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:06.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.395+0000 7f39ea7fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39ec0fff20 0x7f39ec0fb8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47704/0 (socket says 172.21.15.107:47704) 2024-04-22T23:37:06.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.395+0000 7f39ea7fc700 1 -- 172.21.15.107:0/917816711 learned_addr learned my addr 172.21.15.107:0/917816711 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:37:06.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.395+0000 7f39eaffd700 1 --2- 172.21.15.107:0/917816711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39ec066f60 0x7f39ec0fd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:06.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.395+0000 7f39ea7fc700 1 -- 172.21.15.107:0/917816711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39ec066f60 msgr2=0x7f39ec0fd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:06.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.395+0000 7f39ea7fc700 1 --2- 172.21.15.107:0/917816711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39ec066f60 0x7f39ec0fd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:06.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.395+0000 7f39ea7fc700 1 -- 172.21.15.107:0/917816711 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39dc005040 con 0x7f39ec0fff20 2024-04-22T23:37:06.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.395+0000 7f39ea7fc700 1 --2- 172.21.15.107:0/917816711 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39ec0fff20 0x7f39ec0fb8e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f39e0002a50 tx=0x7f39e00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:06.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.395+0000 7f39d37fe700 1 -- 172.21.15.107:0/917816711 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39e0007cc0 con 0x7f39ec0fff20 2024-04-22T23:37:06.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.395+0000 7f39d37fe700 1 -- 172.21.15.107:0/917816711 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f39e000f040 con 0x7f39ec0fff20 2024-04-22T23:37:06.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.395+0000 7f39f12ab700 1 -- 172.21.15.107:0/917816711 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39ec0fc0d0 con 0x7f39ec0fff20 2024-04-22T23:37:06.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.395+0000 7f39d37fe700 1 -- 172.21.15.107:0/917816711 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39e0009640 con 0x7f39ec0fff20 2024-04-22T23:37:06.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.395+0000 7f39f12ab700 1 -- 172.21.15.107:0/917816711 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39ec0fc690 con 0x7f39ec0fff20 2024-04-22T23:37:06.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.396+0000 7f39f12ab700 1 -- 172.21.15.107:0/917816711 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39ec04c710 con 0x7f39ec0fff20 2024-04-22T23:37:06.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.398+0000 7f39d37fe700 1 -- 172.21.15.107:0/917816711 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) v1 ==== 89652+0+0 (secure 0 0 0) 0x7f39e00097a0 con 0x7f39ec0fff20 2024-04-22T23:37:06.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.398+0000 7f39d37fe700 1 --2- 172.21.15.107:0/917816711 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f39d406bd00 0x7f39d406e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:06.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.398+0000 7f39d37fe700 1 -- 172.21.15.107:0/917816711 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f39e008aef0 con 0x7f39ec0fff20 2024-04-22T23:37:06.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.399+0000 7f39eaffd700 1 --2- 172.21.15.107:0/917816711 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f39d406bd00 0x7f39d406e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:06.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.399+0000 7f39eaffd700 1 --2- 172.21.15.107:0/917816711 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f39d406bd00 0x7f39d406e1b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f39dc00b970 tx=0x7f39dc009e30 comp rx=0 tx=0).ready entity=mgr.14720 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:06.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.402+0000 7f39d37fe700 1 -- 172.21.15.107:0/917816711 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187010 (secure 0 0 0) 0x7f39e0057fe0 con 0x7f39ec0fff20 2024-04-22T23:37:06.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.680+0000 7f39f12ab700 1 -- 172.21.15.107:0/917816711 --> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f39ec101b70 con 0x7f39d406bd00 2024-04-22T23:37:06.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.682+0000 7f39d37fe700 1 -- 172.21.15.107:0/917816711 <== mgr.14720 v2:172.21.15.157:6828/3332906296 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f39ec101b70 con 0x7f39d406bd00 2024-04-22T23:37:06.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.684+0000 7f39f12ab700 1 -- 172.21.15.107:0/917816711 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f39d406bd00 msgr2=0x7f39d406e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:06.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.684+0000 7f39f12ab700 1 --2- 172.21.15.107:0/917816711 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f39d406bd00 0x7f39d406e1b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f39dc00b970 tx=0x7f39dc009e30 comp rx=0 tx=0).stop 2024-04-22T23:37:06.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.684+0000 7f39f12ab700 1 -- 172.21.15.107:0/917816711 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39ec0fff20 msgr2=0x7f39ec0fb8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:06.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.684+0000 7f39f12ab700 1 --2- 172.21.15.107:0/917816711 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39ec0fff20 0x7f39ec0fb8e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f39e0002a50 tx=0x7f39e00046b0 comp rx=0 tx=0).stop 2024-04-22T23:37:06.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.684+0000 7f39f12ab700 1 -- 172.21.15.107:0/917816711 shutdown_connections 2024-04-22T23:37:06.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.684+0000 7f39f12ab700 1 --2- 172.21.15.107:0/917816711 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f39d406bd00 0x7f39d406e1b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:06.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.684+0000 7f39f12ab700 1 --2- 172.21.15.107:0/917816711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39ec066f60 0x7f39ec0fd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:06.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.684+0000 7f39f12ab700 1 --2- 172.21.15.107:0/917816711 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39ec0fff20 0x7f39ec0fb8e0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:06.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.684+0000 7f39f12ab700 1 -- 172.21.15.107:0/917816711 >> 172.21.15.107:0/917816711 conn(0x7f39ec0f7450 msgr2=0x7f39ec0f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:06.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.684+0000 7f39f12ab700 1 -- 172.21.15.107:0/917816711 shutdown_connections 2024-04-22T23:37:06.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.684+0000 7f39f12ab700 1 -- 172.21.15.107:0/917816711 wait complete. 2024-04-22T23:37:06.689 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-04-22T23:37:06.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.790+0000 7f740c911700 1 -- 172.21.15.107:0/3911100548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74080fc300 msgr2=0x7f74080fc710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:06.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.790+0000 7f740c911700 1 --2- 172.21.15.107:0/3911100548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74080fc300 0x7f74080fc710 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f73f8004940 tx=0x7f73f8009e30 comp rx=0 tx=0).stop 2024-04-22T23:37:06.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.791+0000 7f740c911700 1 -- 172.21.15.107:0/3911100548 shutdown_connections 2024-04-22T23:37:06.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.791+0000 7f740c911700 1 --2- 172.21.15.107:0/3911100548 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74080fcc50 0x7f74080fd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:06.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.791+0000 7f740c911700 1 --2- 172.21.15.107:0/3911100548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74080fc300 0x7f74080fc710 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:06.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.791+0000 7f740c911700 1 -- 172.21.15.107:0/3911100548 >> 172.21.15.107:0/3911100548 conn(0x7f74080f6bd0 msgr2=0x7f74080f8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:06.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.791+0000 7f740c911700 1 -- 172.21.15.107:0/3911100548 shutdown_connections 2024-04-22T23:37:06.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.791+0000 7f740c911700 1 -- 172.21.15.107:0/3911100548 wait complete. 2024-04-22T23:37:06.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.791+0000 7f740c911700 1 Processor -- start 2024-04-22T23:37:06.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.791+0000 7f740c911700 1 -- start start 2024-04-22T23:37:06.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.791+0000 7f740c911700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74080fc300 0x7f74081106e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:06.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.791+0000 7f740c911700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74080fcc50 0x7f7408112c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:06.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.791+0000 7f740c911700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74080fbde0 con 0x7f74080fc300 2024-04-22T23:37:06.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.791+0000 7f740c911700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74080fbf20 con 0x7f74080fcc50 2024-04-22T23:37:06.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.791+0000 7f740659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74080fc300 0x7f74081106e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:06.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.792+0000 7f740659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74080fc300 0x7f74081106e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47724/0 (socket says 172.21.15.107:47724) 2024-04-22T23:37:06.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.792+0000 7f740659c700 1 -- 172.21.15.107:0/146269822 learned_addr learned my addr 172.21.15.107:0/146269822 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:37:06.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.792+0000 7f7405d9b700 1 --2- 172.21.15.107:0/146269822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74080fcc50 0x7f7408112c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:06.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.792+0000 7f740659c700 1 -- 172.21.15.107:0/146269822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74080fcc50 msgr2=0x7f7408112c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:06.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.792+0000 7f740659c700 1 --2- 172.21.15.107:0/146269822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74080fcc50 0x7f7408112c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:06.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.792+0000 7f740659c700 1 -- 172.21.15.107:0/146269822 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73f8005040 con 0x7f74080fc300 2024-04-22T23:37:06.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.792+0000 7f7405d9b700 1 --2- 172.21.15.107:0/146269822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74080fcc50 0x7f7408112c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T23:37:06.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.792+0000 7f740659c700 1 --2- 172.21.15.107:0/146269822 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74080fc300 0x7f74081106e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f73f800b3b0 tx=0x7f73f8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:06.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.792+0000 7f73f6ffd700 1 -- 172.21.15.107:0/146269822 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73f800e040 con 0x7f74080fc300 2024-04-22T23:37:06.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.792+0000 7f73f6ffd700 1 -- 172.21.15.107:0/146269822 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f73f800bcc0 con 0x7f74080fc300 2024-04-22T23:37:06.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.792+0000 7f740c911700 1 -- 172.21.15.107:0/146269822 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7408113260 con 0x7f74080fc300 2024-04-22T23:37:06.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.792+0000 7f740c911700 1 -- 172.21.15.107:0/146269822 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7408113820 con 0x7f74080fc300 2024-04-22T23:37:06.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.792+0000 7f73f6ffd700 1 -- 172.21.15.107:0/146269822 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73f801c630 con 0x7f74080fc300 2024-04-22T23:37:06.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.793+0000 7f740c911700 1 -- 172.21.15.107:0/146269822 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73e80052e0 con 0x7f74080fc300 2024-04-22T23:37:06.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.794+0000 7f73f6ffd700 1 -- 172.21.15.107:0/146269822 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) v1 ==== 89652+0+0 (secure 0 0 0) 0x7f73f8013420 con 0x7f74080fc300 2024-04-22T23:37:06.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.795+0000 7f73f6ffd700 1 --2- 172.21.15.107:0/146269822 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f73f006bcf0 0x7f73f006e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:06.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.795+0000 7f73f6ffd700 1 -- 172.21.15.107:0/146269822 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f73f808b6f0 con 0x7f74080fc300 2024-04-22T23:37:06.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.795+0000 7f7405d9b700 1 --2- 172.21.15.107:0/146269822 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f73f006bcf0 0x7f73f006e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:06.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.795+0000 7f7405d9b700 1 --2- 172.21.15.107:0/146269822 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f73f006bcf0 0x7f73f006e1a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f73fc009b10 tx=0x7f73fc008040 comp rx=0 tx=0).ready entity=mgr.14720 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:06.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:06.799+0000 7f73f6ffd700 1 -- 172.21.15.107:0/146269822 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187010 (secure 0 0 0) 0x7f73f8054d50 con 0x7f74080fc300 2024-04-22T23:37:07.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.035+0000 7f740c911700 1 -- 172.21.15.107:0/146269822 --> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f73e8000bf0 con 0x7f73f006bcf0 2024-04-22T23:37:07.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.036+0000 7f73f6ffd700 1 -- 172.21.15.107:0/146269822 <== mgr.14720 v2:172.21.15.157:6828/3332906296 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f73e8000bf0 con 0x7f73f006bcf0 2024-04-22T23:37:07.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.039+0000 7f740c911700 1 -- 172.21.15.107:0/146269822 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f73f006bcf0 msgr2=0x7f73f006e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:07.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.039+0000 7f740c911700 1 --2- 172.21.15.107:0/146269822 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f73f006bcf0 0x7f73f006e1a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f73fc009b10 tx=0x7f73fc008040 comp rx=0 tx=0).stop 2024-04-22T23:37:07.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.039+0000 7f740c911700 1 -- 172.21.15.107:0/146269822 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74080fc300 msgr2=0x7f74081106e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:07.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.039+0000 7f740c911700 1 --2- 172.21.15.107:0/146269822 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74080fc300 0x7f74081106e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f73f800b3b0 tx=0x7f73f8009e30 comp rx=0 tx=0).stop 2024-04-22T23:37:07.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.039+0000 7f740c911700 1 -- 172.21.15.107:0/146269822 shutdown_connections 2024-04-22T23:37:07.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.039+0000 7f740c911700 1 --2- 172.21.15.107:0/146269822 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f73f006bcf0 0x7f73f006e1a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:07.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.039+0000 7f740c911700 1 --2- 172.21.15.107:0/146269822 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74080fc300 0x7f74081106e0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:07.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.039+0000 7f740c911700 1 --2- 172.21.15.107:0/146269822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74080fcc50 0x7f7408112c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:07.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.039+0000 7f740c911700 1 -- 172.21.15.107:0/146269822 >> 172.21.15.107:0/146269822 conn(0x7f74080f6bd0 msgr2=0x7f74080f8260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:07.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.039+0000 7f740c911700 1 -- 172.21.15.107:0/146269822 shutdown_connections 2024-04-22T23:37:07.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.039+0000 7f740c911700 1 -- 172.21.15.107:0/146269822 wait complete. 2024-04-22T23:37:07.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.149+0000 7f7a5da47700 1 -- 172.21.15.107:0/1503891290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a581002f0 msgr2=0x7f7a581026d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:07.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.149+0000 7f7a5da47700 1 --2- 172.21.15.107:0/1503891290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a581002f0 0x7f7a581026d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c004940 tx=0x7f7a4c009e30 comp rx=0 tx=0).stop 2024-04-22T23:37:07.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.149+0000 7f7a5da47700 1 -- 172.21.15.107:0/1503891290 shutdown_connections 2024-04-22T23:37:07.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.149+0000 7f7a5da47700 1 --2- 172.21.15.107:0/1503891290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a581002f0 0x7f7a581026d0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:07.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.149+0000 7f7a5da47700 1 --2- 172.21.15.107:0/1503891290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a580fd9d0 0x7f7a580ffdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:07.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.149+0000 7f7a5da47700 1 -- 172.21.15.107:0/1503891290 >> 172.21.15.107:0/1503891290 conn(0x7f7a580f7450 msgr2=0x7f7a580f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:07.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.149+0000 7f7a5da47700 1 -- 172.21.15.107:0/1503891290 shutdown_connections 2024-04-22T23:37:07.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.149+0000 7f7a5da47700 1 -- 172.21.15.107:0/1503891290 wait complete. 2024-04-22T23:37:07.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.150+0000 7f7a5da47700 1 Processor -- start 2024-04-22T23:37:07.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.150+0000 7f7a5da47700 1 -- start start 2024-04-22T23:37:07.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.150+0000 7f7a5da47700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a580fd9d0 0x7f7a5810da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:07.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.150+0000 7f7a5da47700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a581002f0 0x7f7a5810dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:07.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.150+0000 7f7a5da47700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5810e5a0 con 0x7f7a580fd9d0 2024-04-22T23:37:07.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.150+0000 7f7a5da47700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5810edf0 con 0x7f7a581002f0 2024-04-22T23:37:07.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.150+0000 7f7a56ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a580fd9d0 0x7f7a5810da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:07.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.150+0000 7f7a56ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a580fd9d0 0x7f7a5810da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47752/0 (socket says 172.21.15.107:47752) 2024-04-22T23:37:07.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.150+0000 7f7a56ffd700 1 -- 172.21.15.107:0/4184860537 learned_addr learned my addr 172.21.15.107:0/4184860537 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:37:07.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.151+0000 7f7a567fc700 1 --2- 172.21.15.107:0/4184860537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a581002f0 0x7f7a5810dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:07.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.151+0000 7f7a56ffd700 1 -- 172.21.15.107:0/4184860537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a581002f0 msgr2=0x7f7a5810dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:07.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.151+0000 7f7a56ffd700 1 --2- 172.21.15.107:0/4184860537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a581002f0 0x7f7a5810dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:07.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.151+0000 7f7a56ffd700 1 -- 172.21.15.107:0/4184860537 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a4c005040 con 0x7f7a580fd9d0 2024-04-22T23:37:07.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.151+0000 7f7a56ffd700 1 --2- 172.21.15.107:0/4184860537 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a580fd9d0 0x7f7a5810da60 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7a48002a50 tx=0x7f7a48004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:07.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.151+0000 7f7a3f7fe700 1 -- 172.21.15.107:0/4184860537 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a48007cc0 con 0x7f7a580fd9d0 2024-04-22T23:37:07.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.151+0000 7f7a5da47700 1 -- 172.21.15.107:0/4184860537 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a5810f0d0 con 0x7f7a580fd9d0 2024-04-22T23:37:07.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.151+0000 7f7a3f7fe700 1 -- 172.21.15.107:0/4184860537 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7a4800f040 con 0x7f7a580fd9d0 2024-04-22T23:37:07.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.151+0000 7f7a3f7fe700 1 -- 172.21.15.107:0/4184860537 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a4800d9c0 con 0x7f7a580fd9d0 2024-04-22T23:37:07.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.151+0000 7f7a5da47700 1 -- 172.21.15.107:0/4184860537 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a5810f610 con 0x7f7a580fd9d0 2024-04-22T23:37:07.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.153+0000 7f7a5da47700 1 -- 172.21.15.107:0/4184860537 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a5805f5f0 con 0x7f7a580fd9d0 2024-04-22T23:37:07.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.153+0000 7f7a3f7fe700 1 -- 172.21.15.107:0/4184860537 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) v1 ==== 89652+0+0 (secure 0 0 0) 0x7f7a4800db20 con 0x7f7a580fd9d0 2024-04-22T23:37:07.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.153+0000 7f7a3f7fe700 1 --2- 172.21.15.107:0/4184860537 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f7a4006bcf0 0x7f7a4006e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:07.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.154+0000 7f7a3f7fe700 1 -- 172.21.15.107:0/4184860537 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f7a4808a720 con 0x7f7a580fd9d0 2024-04-22T23:37:07.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.154+0000 7f7a567fc700 1 --2- 172.21.15.107:0/4184860537 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f7a4006bcf0 0x7f7a4006e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:07.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.154+0000 7f7a567fc700 1 --2- 172.21.15.107:0/4184860537 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f7a4006bcf0 0x7f7a4006e1a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c00b3b0 tx=0x7f7a4c009e30 comp rx=0 tx=0).ready entity=mgr.14720 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:07.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.158+0000 7f7a3f7fe700 1 -- 172.21.15.107:0/4184860537 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187010 (secure 0 0 0) 0x7f7a48057810 con 0x7f7a580fd9d0 2024-04-22T23:37:07.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:06 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:37:07.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:06 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:37:07.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:06 smithi107 ceph-mon[26355]: pgmap v27: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:37:07.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.385+0000 7f7a5da47700 1 -- 172.21.15.107:0/4184860537 --> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7a5810f980 con 0x7f7a4006bcf0 2024-04-22T23:37:07.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:06 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:37:07.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:06 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:37:07.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:06 smithi157 ceph-mon[33877]: pgmap v27: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:37:07.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.397+0000 7f7a3f7fe700 1 -- 172.21.15.107:0/4184860537 <== mgr.14720 v2:172.21.15.157:6828/3332906296 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f7a5810f980 con 0x7f7a4006bcf0 2024-04-22T23:37:07.391 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-22T23:37:07.391 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (25s) 1s ago 10m 16.3M - 0.25.0 c8568f914cd2 ec1a5c6260de 2024-04-22T23:37:07.391 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (10m) 1s ago 10m 6665k - 17.2.7-884-g70d0cb4b cafd8ddac8fa c10611c756e4 2024-04-22T23:37:07.391 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi157 smithi157 running (9m) 3s ago 9m 6669k - 17.2.7-884-g70d0cb4b cafd8ddac8fa 55d2d46294c9 2024-04-22T23:37:07.391 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (20s) 1s ago 9m 77.9M - 9.4.7 954c08fa6188 31449ff442d2 2024-04-22T23:37:07.391 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.dfvvkw smithi107 running (4m) 1s ago 4m 12.1M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 08d3aa8d717d 2024-04-22T23:37:07.392 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.ktuxjl smithi107 running (4m) 1s ago 4m 125M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 762f27fe0f30 2024-04-22T23:37:07.392 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi157.rcaris smithi157 running (4m) 3s ago 4m 128M - 17.2.7-884-g70d0cb4b cafd8ddac8fa d23366ac085d 2024-04-22T23:37:07.392 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi157.ufywse smithi157 running (4m) 3s ago 4m 12.7M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 8b00d61eb355 2024-04-22T23:37:07.392 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.uipxvm smithi107 *:8443,9283,8765 running (7s) 1s ago 11m 67.4M - 19.0.0-3252-g43be0201 a0434699c1d7 d20f05dfe4f2 2024-04-22T23:37:07.392 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi157.tuylis smithi157 *:8443,9283 running (58s) 3s ago 9m 503M - 19.0.0-3252-g43be0201 a0434699c1d7 351c5ba75c91 2024-04-22T23:37:07.392 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (11m) 1s ago 11m 55.5M 2048M 17.2.7-884-g70d0cb4b cafd8ddac8fa 8314962fa1b9 2024-04-22T23:37:07.392 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi157 smithi157 running (9m) 3s ago 9m 40.0M 2048M 17.2.7-884-g70d0cb4b cafd8ddac8fa 0747a85008d9 2024-04-22T23:37:07.392 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (31s) 1s ago 10m 8024k - 1.5.0 0da6a335fe13 5205cfd9bd51 2024-04-22T23:37:07.392 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi157 smithi157 *:9100 running (12s) 3s ago 9m 3615k - 1.5.0 0da6a335fe13 168ca191bb8c 2024-04-22T23:37:07.392 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (8m) 1s ago 8m 229M 1557M 17.2.7-884-g70d0cb4b cafd8ddac8fa 18a179b0a618 2024-04-22T23:37:07.392 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (7m) 1s ago 7m 240M 1557M 17.2.7-884-g70d0cb4b cafd8ddac8fa c7272eb84ddb 2024-04-22T23:37:07.392 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (7m) 1s ago 7m 194M 1557M 17.2.7-884-g70d0cb4b cafd8ddac8fa 9db1788096ad 2024-04-22T23:37:07.392 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi157 running (7m) 3s ago 7m 265M 2581M 17.2.7-884-g70d0cb4b cafd8ddac8fa c9fb2d1a13dd 2024-04-22T23:37:07.392 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi157 running (6m) 3s ago 6m 225M 2581M 17.2.7-884-g70d0cb4b cafd8ddac8fa 24a16e19951f 2024-04-22T23:37:07.392 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi157 running (6m) 3s ago 6m 224M 2581M 17.2.7-884-g70d0cb4b cafd8ddac8fa 4c5b7a7ef8bd 2024-04-22T23:37:07.393 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (14s) 1s ago 9m 38.0M - 2.43.0 a07b618ecd1d a0605d0ba8c8 2024-04-22T23:37:07.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.399+0000 7f7a5da47700 1 -- 172.21.15.107:0/4184860537 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f7a4006bcf0 msgr2=0x7f7a4006e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:07.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.399+0000 7f7a5da47700 1 --2- 172.21.15.107:0/4184860537 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f7a4006bcf0 0x7f7a4006e1a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c00b3b0 tx=0x7f7a4c009e30 comp rx=0 tx=0).stop 2024-04-22T23:37:07.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.399+0000 7f7a5da47700 1 -- 172.21.15.107:0/4184860537 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a580fd9d0 msgr2=0x7f7a5810da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:07.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.399+0000 7f7a5da47700 1 --2- 172.21.15.107:0/4184860537 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a580fd9d0 0x7f7a5810da60 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7a48002a50 tx=0x7f7a48004600 comp rx=0 tx=0).stop 2024-04-22T23:37:07.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.399+0000 7f7a5da47700 1 -- 172.21.15.107:0/4184860537 shutdown_connections 2024-04-22T23:37:07.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.399+0000 7f7a5da47700 1 --2- 172.21.15.107:0/4184860537 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f7a4006bcf0 0x7f7a4006e1a0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:07.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.399+0000 7f7a5da47700 1 --2- 172.21.15.107:0/4184860537 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a580fd9d0 0x7f7a5810da60 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:07.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.399+0000 7f7a5da47700 1 --2- 172.21.15.107:0/4184860537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a581002f0 0x7f7a5810dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:07.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.399+0000 7f7a5da47700 1 -- 172.21.15.107:0/4184860537 >> 172.21.15.107:0/4184860537 conn(0x7f7a580f7450 msgr2=0x7f7a580f7870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:07.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.399+0000 7f7a5da47700 1 -- 172.21.15.107:0/4184860537 shutdown_connections 2024-04-22T23:37:07.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.399+0000 7f7a5da47700 1 -- 172.21.15.107:0/4184860537 wait complete. 2024-04-22T23:37:07.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.507+0000 7f270d0d8700 1 -- 172.21.15.107:0/2448625748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2708066250 msgr2=0x7f27080ffad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:07.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.507+0000 7f270d0d8700 1 --2- 172.21.15.107:0/2448625748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2708066250 0x7f27080ffad0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f26f8004940 tx=0x7f26f8009e30 comp rx=0 tx=0).stop 2024-04-22T23:37:07.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.507+0000 7f270d0d8700 1 -- 172.21.15.107:0/2448625748 shutdown_connections 2024-04-22T23:37:07.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.507+0000 7f270d0d8700 1 --2- 172.21.15.107:0/2448625748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2708100010 0x7f27081023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:07.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.507+0000 7f270d0d8700 1 --2- 172.21.15.107:0/2448625748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2708066250 0x7f27080ffad0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:07.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.507+0000 7f270d0d8700 1 -- 172.21.15.107:0/2448625748 >> 172.21.15.107:0/2448625748 conn(0x7f27080f74c0 msgr2=0x7f27080f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:07.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.508+0000 7f270d0d8700 1 -- 172.21.15.107:0/2448625748 shutdown_connections 2024-04-22T23:37:07.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.508+0000 7f270d0d8700 1 -- 172.21.15.107:0/2448625748 wait complete. 2024-04-22T23:37:07.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.508+0000 7f270d0d8700 1 Processor -- start 2024-04-22T23:37:07.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.508+0000 7f270d0d8700 1 -- start start 2024-04-22T23:37:07.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.509+0000 7f270d0d8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2708066250 0x7f270810da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:07.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.509+0000 7f270d0d8700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2708100010 0x7f270810dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:07.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.509+0000 7f270d0d8700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f270810e5b0 con 0x7f2708066250 2024-04-22T23:37:07.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.509+0000 7f270d0d8700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f270810ee00 con 0x7f2708100010 2024-04-22T23:37:07.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.509+0000 7f2706d9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2708066250 0x7f270810da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:07.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.509+0000 7f2706d9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2708066250 0x7f270810da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47782/0 (socket says 172.21.15.107:47782) 2024-04-22T23:37:07.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.509+0000 7f2706d9d700 1 -- 172.21.15.107:0/3739592797 learned_addr learned my addr 172.21.15.107:0/3739592797 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:37:07.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.509+0000 7f270659c700 1 --2- 172.21.15.107:0/3739592797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2708100010 0x7f270810dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:07.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.509+0000 7f2706d9d700 1 -- 172.21.15.107:0/3739592797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2708100010 msgr2=0x7f270810dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:07.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.509+0000 7f2706d9d700 1 --2- 172.21.15.107:0/3739592797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2708100010 0x7f270810dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:07.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.509+0000 7f2706d9d700 1 -- 172.21.15.107:0/3739592797 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f8005040 con 0x7f2708066250 2024-04-22T23:37:07.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.509+0000 7f2706d9d700 1 --2- 172.21.15.107:0/3739592797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2708066250 0x7f270810da70 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f26f800b970 tx=0x7f26f8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:07.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.510+0000 7f26ef7fe700 1 -- 172.21.15.107:0/3739592797 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26f800e040 con 0x7f2708066250 2024-04-22T23:37:07.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.510+0000 7f26ef7fe700 1 -- 172.21.15.107:0/3739592797 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f26f800bbf0 con 0x7f2708066250 2024-04-22T23:37:07.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.510+0000 7f270d0d8700 1 -- 172.21.15.107:0/3739592797 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f270810f080 con 0x7f2708066250 2024-04-22T23:37:07.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.511+0000 7f26ef7fe700 1 -- 172.21.15.107:0/3739592797 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26f801c630 con 0x7f2708066250 2024-04-22T23:37:07.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.511+0000 7f270d0d8700 1 -- 172.21.15.107:0/3739592797 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f270810f5c0 con 0x7f2708066250 2024-04-22T23:37:07.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.512+0000 7f26ef7fe700 1 -- 172.21.15.107:0/3739592797 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) v1 ==== 89652+0+0 (secure 0 0 0) 0x7f26f8013420 con 0x7f2708066250 2024-04-22T23:37:07.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.512+0000 7f270d0d8700 1 -- 172.21.15.107:0/3739592797 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f270805f5f0 con 0x7f2708066250 2024-04-22T23:37:07.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.512+0000 7f26ef7fe700 1 --2- 172.21.15.107:0/3739592797 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f26f006bcf0 0x7f26f006e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:07.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.512+0000 7f26ef7fe700 1 -- 172.21.15.107:0/3739592797 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f26f808b660 con 0x7f2708066250 2024-04-22T23:37:07.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.513+0000 7f270659c700 1 --2- 172.21.15.107:0/3739592797 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f26f006bcf0 0x7f26f006e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:07.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.513+0000 7f270659c700 1 --2- 172.21.15.107:0/3739592797 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f26f006bcf0 0x7f26f006e1a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f26fc0041d0 tx=0x7f26fc0043c0 comp rx=0 tx=0).ready entity=mgr.14720 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:07.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.517+0000 7f26ef7fe700 1 -- 172.21.15.107:0/3739592797 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187010 (secure 0 0 0) 0x7f26f8090060 con 0x7f2708066250 2024-04-22T23:37:07.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.830+0000 7f270d0d8700 1 -- 172.21.15.107:0/3739592797 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f270810f900 con 0x7f2708066250 2024-04-22T23:37:07.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.831+0000 7f26ef7fe700 1 -- 172.21.15.107:0/3739592797 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7f26f8058740 con 0x7f2708066250 2024-04-22T23:37:07.825 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-22T23:37:07.825 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-04-22T23:37:07.825 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 2 2024-04-22T23:37:07.826 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:37:07.826 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-04-22T23:37:07.826 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 1, 2024-04-22T23:37:07.826 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 1 2024-04-22T23:37:07.826 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:37:07.826 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-04-22T23:37:07.826 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 6 2024-04-22T23:37:07.826 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:37:07.826 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-04-22T23:37:07.826 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 4 2024-04-22T23:37:07.826 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:37:07.826 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-04-22T23:37:07.826 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 13, 2024-04-22T23:37:07.826 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 1 2024-04-22T23:37:07.827 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T23:37:07.827 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-22T23:37:07.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.833+0000 7f270d0d8700 1 -- 172.21.15.107:0/3739592797 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f26f006bcf0 msgr2=0x7f26f006e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:07.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.833+0000 7f270d0d8700 1 --2- 172.21.15.107:0/3739592797 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f26f006bcf0 0x7f26f006e1a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f26fc0041d0 tx=0x7f26fc0043c0 comp rx=0 tx=0).stop 2024-04-22T23:37:07.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.833+0000 7f270d0d8700 1 -- 172.21.15.107:0/3739592797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2708066250 msgr2=0x7f270810da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:07.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.833+0000 7f270d0d8700 1 --2- 172.21.15.107:0/3739592797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2708066250 0x7f270810da70 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f26f800b970 tx=0x7f26f8009e30 comp rx=0 tx=0).stop 2024-04-22T23:37:07.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.833+0000 7f270d0d8700 1 -- 172.21.15.107:0/3739592797 shutdown_connections 2024-04-22T23:37:07.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.833+0000 7f270d0d8700 1 --2- 172.21.15.107:0/3739592797 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f26f006bcf0 0x7f26f006e1a0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:07.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.833+0000 7f270d0d8700 1 --2- 172.21.15.107:0/3739592797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2708066250 0x7f270810da70 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:07.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.833+0000 7f270d0d8700 1 --2- 172.21.15.107:0/3739592797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2708100010 0x7f270810dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:07.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.833+0000 7f270d0d8700 1 -- 172.21.15.107:0/3739592797 >> 172.21.15.107:0/3739592797 conn(0x7f27080f74c0 msgr2=0x7f27080f7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:07.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.833+0000 7f270d0d8700 1 -- 172.21.15.107:0/3739592797 shutdown_connections 2024-04-22T23:37:07.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.833+0000 7f270d0d8700 1 -- 172.21.15.107:0/3739592797 wait complete. 2024-04-22T23:37:07.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.963+0000 7f72c94c1700 1 -- 172.21.15.107:0/279858085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72c40fcf90 msgr2=0x7f72c40fd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:07.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.963+0000 7f72c94c1700 1 --2- 172.21.15.107:0/279858085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72c40fcf90 0x7f72c40fd400 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f72b8004940 tx=0x7f72b8009e30 comp rx=0 tx=0).stop 2024-04-22T23:37:07.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.963+0000 7f72c94c1700 1 -- 172.21.15.107:0/279858085 shutdown_connections 2024-04-22T23:37:07.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.963+0000 7f72c94c1700 1 --2- 172.21.15.107:0/279858085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72c40fcf90 0x7f72c40fd400 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:07.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.963+0000 7f72c94c1700 1 --2- 172.21.15.107:0/279858085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72c40fbe20 0x7f72c40fc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:07.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.963+0000 7f72c94c1700 1 -- 172.21.15.107:0/279858085 >> 172.21.15.107:0/279858085 conn(0x7f72c40f74e0 msgr2=0x7f72c40f9910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:07.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.963+0000 7f72c94c1700 1 -- 172.21.15.107:0/279858085 shutdown_connections 2024-04-22T23:37:07.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.963+0000 7f72c94c1700 1 -- 172.21.15.107:0/279858085 wait complete. 2024-04-22T23:37:07.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.964+0000 7f72c94c1700 1 Processor -- start 2024-04-22T23:37:07.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.964+0000 7f72c94c1700 1 -- start start 2024-04-22T23:37:07.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.964+0000 7f72c94c1700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72c40fbe20 0x7f72c410db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:07.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.964+0000 7f72c94c1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72c40fcf90 0x7f72c410e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:07.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.964+0000 7f72c94c1700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c410e650 con 0x7f72c40fcf90 2024-04-22T23:37:07.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.964+0000 7f72c94c1700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c410ee50 con 0x7f72c40fbe20 2024-04-22T23:37:07.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.964+0000 7f72c27fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72c40fcf90 0x7f72c410e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:07.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.964+0000 7f72c27fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72c40fcf90 0x7f72c410e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47794/0 (socket says 172.21.15.107:47794) 2024-04-22T23:37:07.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.964+0000 7f72c27fc700 1 -- 172.21.15.107:0/1042039940 learned_addr learned my addr 172.21.15.107:0/1042039940 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:37:07.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.964+0000 7f72c2ffd700 1 --2- 172.21.15.107:0/1042039940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72c40fbe20 0x7f72c410db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:07.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.964+0000 7f72c27fc700 1 -- 172.21.15.107:0/1042039940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72c40fbe20 msgr2=0x7f72c410db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:07.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.964+0000 7f72c27fc700 1 --2- 172.21.15.107:0/1042039940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72c40fbe20 0x7f72c410db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:07.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.964+0000 7f72c27fc700 1 -- 172.21.15.107:0/1042039940 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b8005040 con 0x7f72c40fcf90 2024-04-22T23:37:07.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.964+0000 7f72c27fc700 1 --2- 172.21.15.107:0/1042039940 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72c40fcf90 0x7f72c410e050 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f72b8009150 tx=0x7f72b8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:07.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.965+0000 7f72ab7fe700 1 -- 172.21.15.107:0/1042039940 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72b800e040 con 0x7f72c40fcf90 2024-04-22T23:37:07.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.965+0000 7f72ab7fe700 1 -- 172.21.15.107:0/1042039940 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f72b8004290 con 0x7f72c40fcf90 2024-04-22T23:37:07.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.965+0000 7f72c94c1700 1 -- 172.21.15.107:0/1042039940 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72c410f0d0 con 0x7f72c40fcf90 2024-04-22T23:37:07.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.965+0000 7f72ab7fe700 1 -- 172.21.15.107:0/1042039940 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72b801c630 con 0x7f72c40fcf90 2024-04-22T23:37:07.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.965+0000 7f72c94c1700 1 -- 172.21.15.107:0/1042039940 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72c410f610 con 0x7f72c40fcf90 2024-04-22T23:37:07.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.966+0000 7f72c94c1700 1 -- 172.21.15.107:0/1042039940 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72c405f5f0 con 0x7f72c40fcf90 2024-04-22T23:37:07.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.966+0000 7f72ab7fe700 1 -- 172.21.15.107:0/1042039940 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) v1 ==== 89652+0+0 (secure 0 0 0) 0x7f72b800bbf0 con 0x7f72c40fcf90 2024-04-22T23:37:07.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.967+0000 7f72ab7fe700 1 --2- 172.21.15.107:0/1042039940 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f72ac06bcf0 0x7f72ac06e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:07.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.967+0000 7f72ab7fe700 1 -- 172.21.15.107:0/1042039940 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f72b808b680 con 0x7f72c40fcf90 2024-04-22T23:37:07.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.967+0000 7f72c2ffd700 1 --2- 172.21.15.107:0/1042039940 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f72ac06bcf0 0x7f72ac06e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:07.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.967+0000 7f72c2ffd700 1 --2- 172.21.15.107:0/1042039940 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f72ac06bcf0 0x7f72ac06e1a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f72b40041d0 tx=0x7f72b40043c0 comp rx=0 tx=0).ready entity=mgr.14720 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:07.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:07.971+0000 7f72ab7fe700 1 -- 172.21.15.107:0/1042039940 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187010 (secure 0 0 0) 0x7f72b8058890 con 0x7f72c40fcf90 2024-04-22T23:37:08.015 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:07 smithi107 ceph-mon[26355]: from='client.14782 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:37:08.015 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:07 smithi107 ceph-mon[26355]: from='client.14786 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:37:08.016 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:07 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:37:08.016 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:07 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:37:08.016 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:07 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/3739592797' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:37:08.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.227+0000 7f72c94c1700 1 -- 172.21.15.107:0/1042039940 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f72c404bed0 con 0x7f72c40fcf90 2024-04-22T23:37:08.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.228+0000 7f72ab7fe700 1 -- 172.21.15.107:0/1042039940 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) v1 ==== 76+0+1858 (secure 0 0 0) 0x7f72b8058420 con 0x7f72c40fcf90 2024-04-22T23:37:08.222 INFO:teuthology.orchestra.run.smithi107.stdout:e19 2024-04-22T23:37:08.222 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T23:37:08.222 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:37:08.222 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-04-22T23:37:08.222 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:37:08.223 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-04-22T23:37:08.223 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-04-22T23:37:08.223 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 19 2024-04-22T23:37:08.223 INFO:teuthology.orchestra.run.smithi107.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-22T23:37:08.223 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-04-22T23:32:20.146130+0000 2024-04-22T23:37:08.223 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-04-22T23:35:12.338678+0000 2024-04-22T23:37:08.223 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-04-22T23:37:08.223 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-04-22T23:37:08.223 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-04-22T23:37:08.223 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-04-22T23:37:08.223 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-04-22T23:37:08.223 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-04-22T23:37:08.223 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-04-22T23:37:08.223 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 46 2024-04-22T23:37:08.223 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:37:08.224 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 2 2024-04-22T23:37:08.224 INFO:teuthology.orchestra.run.smithi107.stdout:in 0,1 2024-04-22T23:37:08.224 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=14468,1=24265} 2024-04-22T23:37:08.224 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-04-22T23:37:08.224 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-04-22T23:37:08.224 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-04-22T23:37:08.224 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-04-22T23:37:08.224 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-04-22T23:37:08.224 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data disabled 2024-04-22T23:37:08.224 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-04-22T23:37:08.224 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-04-22T23:37:08.224 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.ktuxjl{0:14468} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.107:6828/3335313253,v1:172.21.15.107:6829/3335313253] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:37:08.224 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi157.rcaris{1:24265} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.157:6824/2779251893,v1:172.21.15.157:6825/2779251893] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:37:08.224 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:37:08.224 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:37:08.225 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-04-22T23:37:08.225 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:37:08.225 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.dfvvkw{-1:14484} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/3548717247,v1:172.21.15.107:6827/3548717247] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:37:08.225 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi157.ufywse{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.157:6826/4072444199,v1:172.21.15.157:6827/4072444199] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:37:08.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.230+0000 7f72c94c1700 1 -- 172.21.15.107:0/1042039940 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f72ac06bcf0 msgr2=0x7f72ac06e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:08.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.230+0000 7f72c94c1700 1 --2- 172.21.15.107:0/1042039940 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f72ac06bcf0 0x7f72ac06e1a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f72b40041d0 tx=0x7f72b40043c0 comp rx=0 tx=0).stop 2024-04-22T23:37:08.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.230+0000 7f72c94c1700 1 -- 172.21.15.107:0/1042039940 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72c40fcf90 msgr2=0x7f72c410e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:08.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.230+0000 7f72c94c1700 1 --2- 172.21.15.107:0/1042039940 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72c40fcf90 0x7f72c410e050 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f72b8009150 tx=0x7f72b8009e30 comp rx=0 tx=0).stop 2024-04-22T23:37:08.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.230+0000 7f72c94c1700 1 -- 172.21.15.107:0/1042039940 shutdown_connections 2024-04-22T23:37:08.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.230+0000 7f72c94c1700 1 --2- 172.21.15.107:0/1042039940 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f72ac06bcf0 0x7f72ac06e1a0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:08.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.230+0000 7f72c94c1700 1 --2- 172.21.15.107:0/1042039940 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72c40fbe20 0x7f72c410db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:08.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.230+0000 7f72c94c1700 1 --2- 172.21.15.107:0/1042039940 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72c40fcf90 0x7f72c410e050 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:08.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.230+0000 7f72c94c1700 1 -- 172.21.15.107:0/1042039940 >> 172.21.15.107:0/1042039940 conn(0x7f72c40f74e0 msgr2=0x7f72c4066490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:08.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.230+0000 7f72c94c1700 1 -- 172.21.15.107:0/1042039940 shutdown_connections 2024-04-22T23:37:08.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.230+0000 7f72c94c1700 1 -- 172.21.15.107:0/1042039940 wait complete. 2024-04-22T23:37:08.226 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 19 2024-04-22T23:37:08.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.331+0000 7f68a5796700 1 -- 172.21.15.107:0/3103446146 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68a00fce80 msgr2=0x7f68a00fd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:08.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.331+0000 7f68a5796700 1 --2- 172.21.15.107:0/3103446146 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68a00fce80 0x7f68a00fd290 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6894004940 tx=0x7f6894009e30 comp rx=0 tx=0).stop 2024-04-22T23:37:08.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.331+0000 7f68a5796700 1 -- 172.21.15.107:0/3103446146 shutdown_connections 2024-04-22T23:37:08.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.331+0000 7f68a5796700 1 --2- 172.21.15.107:0/3103446146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68a00fdff0 0x7f68a00fe460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:08.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.331+0000 7f68a5796700 1 --2- 172.21.15.107:0/3103446146 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68a00fce80 0x7f68a00fd290 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:08.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.331+0000 7f68a5796700 1 -- 172.21.15.107:0/3103446146 >> 172.21.15.107:0/3103446146 conn(0x7f68a00f8580 msgr2=0x7f68a00fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:08.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.331+0000 7f68a5796700 1 -- 172.21.15.107:0/3103446146 shutdown_connections 2024-04-22T23:37:08.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.331+0000 7f68a5796700 1 -- 172.21.15.107:0/3103446146 wait complete. 2024-04-22T23:37:08.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.332+0000 7f68a5796700 1 Processor -- start 2024-04-22T23:37:08.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.332+0000 7f68a5796700 1 -- start start 2024-04-22T23:37:08.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.332+0000 7f68a5796700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68a00fce80 0x7f68a010d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:08.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.332+0000 7f68a5796700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68a00fdff0 0x7f68a010de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:08.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.332+0000 7f68a5796700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a010e470 con 0x7f68a00fdff0 2024-04-22T23:37:08.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.332+0000 7f68a5796700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a010e5e0 con 0x7f68a00fce80 2024-04-22T23:37:08.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.332+0000 7f689e7fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68a00fdff0 0x7f68a010de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:08.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.332+0000 7f689e7fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68a00fdff0 0x7f68a010de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47812/0 (socket says 172.21.15.107:47812) 2024-04-22T23:37:08.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.332+0000 7f689e7fc700 1 -- 172.21.15.107:0/1565911363 learned_addr learned my addr 172.21.15.107:0/1565911363 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:37:08.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.332+0000 7f689effd700 1 --2- 172.21.15.107:0/1565911363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68a00fce80 0x7f68a010d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:08.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.333+0000 7f689e7fc700 1 -- 172.21.15.107:0/1565911363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68a00fce80 msgr2=0x7f68a010d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:08.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.333+0000 7f689e7fc700 1 --2- 172.21.15.107:0/1565911363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68a00fce80 0x7f68a010d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:08.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.333+0000 7f689e7fc700 1 -- 172.21.15.107:0/1565911363 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6894005040 con 0x7f68a00fdff0 2024-04-22T23:37:08.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.333+0000 7f689e7fc700 1 --2- 172.21.15.107:0/1565911363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68a00fdff0 0x7f68a010de70 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6890002a50 tx=0x7f68900046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:08.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.333+0000 7f68877fe700 1 -- 172.21.15.107:0/1565911363 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6890007cc0 con 0x7f68a00fdff0 2024-04-22T23:37:08.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.333+0000 7f68877fe700 1 -- 172.21.15.107:0/1565911363 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f689000f040 con 0x7f68a00fdff0 2024-04-22T23:37:08.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.333+0000 7f68a5796700 1 -- 172.21.15.107:0/1565911363 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68a0113040 con 0x7f68a00fdff0 2024-04-22T23:37:08.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.333+0000 7f68877fe700 1 -- 172.21.15.107:0/1565911363 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f689000da30 con 0x7f68a00fdff0 2024-04-22T23:37:08.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.333+0000 7f68a5796700 1 -- 172.21.15.107:0/1565911363 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68a01135d0 con 0x7f68a00fdff0 2024-04-22T23:37:08.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.335+0000 7f68a5796700 1 -- 172.21.15.107:0/1565911363 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68a004c710 con 0x7f68a00fdff0 2024-04-22T23:37:08.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.335+0000 7f68877fe700 1 -- 172.21.15.107:0/1565911363 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) v1 ==== 89652+0+0 (secure 0 0 0) 0x7f689000db90 con 0x7f68a00fdff0 2024-04-22T23:37:08.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.335+0000 7f68877fe700 1 --2- 172.21.15.107:0/1565911363 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f688806bd00 0x7f688806e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:08.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.335+0000 7f68877fe700 1 -- 172.21.15.107:0/1565911363 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f689008a890 con 0x7f68a00fdff0 2024-04-22T23:37:08.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.336+0000 7f689effd700 1 --2- 172.21.15.107:0/1565911363 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f688806bd00 0x7f688806e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:08.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.336+0000 7f689effd700 1 --2- 172.21.15.107:0/1565911363 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f688806bd00 0x7f688806e1b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f689400b970 tx=0x7f6894009e30 comp rx=0 tx=0).ready entity=mgr.14720 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:08.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.340+0000 7f68877fe700 1 -- 172.21.15.107:0/1565911363 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187010 (secure 0 0 0) 0x7f6890057900 con 0x7f68a00fdff0 2024-04-22T23:37:08.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:07 smithi157 ceph-mon[33877]: from='client.14782 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:37:08.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:07 smithi157 ceph-mon[33877]: from='client.14786 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:37:08.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:07 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:37:08.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:07 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:37:08.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:07 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/3739592797' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:37:08.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.563+0000 7f68a5796700 1 -- 172.21.15.107:0/1565911363 --> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f68a0102400 con 0x7f688806bd00 2024-04-22T23:37:08.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.564+0000 7f68877fe700 1 -- 172.21.15.107:0/1565911363 <== mgr.14720 v2:172.21.15.157:6828/3332906296 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f68a0102400 con 0x7f688806bd00 2024-04-22T23:37:08.558 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-22T23:37:08.558 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c", 2024-04-22T23:37:08.559 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-04-22T23:37:08.559 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-22T23:37:08.559 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [ 2024-04-22T23:37:08.559 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr" 2024-04-22T23:37:08.559 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-22T23:37:08.559 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "2/21 daemons upgraded", 2024-04-22T23:37:08.559 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Currently upgrading mgr daemons", 2024-04-22T23:37:08.559 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-04-22T23:37:08.559 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-22T23:37:08.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.566+0000 7f68a5796700 1 -- 172.21.15.107:0/1565911363 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f688806bd00 msgr2=0x7f688806e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:08.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.566+0000 7f68a5796700 1 --2- 172.21.15.107:0/1565911363 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f688806bd00 0x7f688806e1b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f689400b970 tx=0x7f6894009e30 comp rx=0 tx=0).stop 2024-04-22T23:37:08.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.566+0000 7f68a5796700 1 -- 172.21.15.107:0/1565911363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68a00fdff0 msgr2=0x7f68a010de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:08.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.566+0000 7f68a5796700 1 --2- 172.21.15.107:0/1565911363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68a00fdff0 0x7f68a010de70 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6890002a50 tx=0x7f68900046b0 comp rx=0 tx=0).stop 2024-04-22T23:37:08.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.566+0000 7f68a5796700 1 -- 172.21.15.107:0/1565911363 shutdown_connections 2024-04-22T23:37:08.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.566+0000 7f68a5796700 1 --2- 172.21.15.107:0/1565911363 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f688806bd00 0x7f688806e1b0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:08.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.566+0000 7f68a5796700 1 --2- 172.21.15.107:0/1565911363 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68a00fce80 0x7f68a010d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:08.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.566+0000 7f68a5796700 1 --2- 172.21.15.107:0/1565911363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68a00fdff0 0x7f68a010de70 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:08.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.566+0000 7f68a5796700 1 -- 172.21.15.107:0/1565911363 >> 172.21.15.107:0/1565911363 conn(0x7f68a00f8580 msgr2=0x7f68a00f9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:08.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.566+0000 7f68a5796700 1 -- 172.21.15.107:0/1565911363 shutdown_connections 2024-04-22T23:37:08.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.566+0000 7f68a5796700 1 -- 172.21.15.107:0/1565911363 wait complete. 2024-04-22T23:37:08.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.670+0000 7f0735f77700 1 -- 172.21.15.107:0/915997948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07300fbd40 msgr2=0x7f07300fc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:08.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.670+0000 7f0735f77700 1 --2- 172.21.15.107:0/915997948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07300fbd40 0x7f07300fc150 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f0728009ce0 tx=0x7f0728004d10 comp rx=0 tx=0).stop 2024-04-22T23:37:08.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.670+0000 7f0735f77700 1 -- 172.21.15.107:0/915997948 shutdown_connections 2024-04-22T23:37:08.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.670+0000 7f0735f77700 1 --2- 172.21.15.107:0/915997948 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07300fcff0 0x7f07300fd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:08.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.670+0000 7f0735f77700 1 --2- 172.21.15.107:0/915997948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07300fbd40 0x7f07300fc150 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:08.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.670+0000 7f0735f77700 1 -- 172.21.15.107:0/915997948 >> 172.21.15.107:0/915997948 conn(0x7f07300f74c0 msgr2=0x7f07300f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:08.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.670+0000 7f0735f77700 1 -- 172.21.15.107:0/915997948 shutdown_connections 2024-04-22T23:37:08.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.670+0000 7f0735f77700 1 -- 172.21.15.107:0/915997948 wait complete. 2024-04-22T23:37:08.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.670+0000 7f0735f77700 1 Processor -- start 2024-04-22T23:37:08.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.670+0000 7f0735f77700 1 -- start start 2024-04-22T23:37:08.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.671+0000 7f0735f77700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07300fbd40 0x7f07301108b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:08.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.671+0000 7f0735f77700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07300fcff0 0x7f0730112e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:08.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.671+0000 7f0735f77700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0730066360 con 0x7f07300fcff0 2024-04-22T23:37:08.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.671+0000 7f0735f77700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07300664a0 con 0x7f07300fbd40 2024-04-22T23:37:08.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.671+0000 7f072effd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07300fcff0 0x7f0730112e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:08.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.671+0000 7f072effd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07300fcff0 0x7f0730112e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47834/0 (socket says 172.21.15.107:47834) 2024-04-22T23:37:08.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.671+0000 7f072effd700 1 -- 172.21.15.107:0/251939887 learned_addr learned my addr 172.21.15.107:0/251939887 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:37:08.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.671+0000 7f072f7fe700 1 --2- 172.21.15.107:0/251939887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07300fbd40 0x7f07301108b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:08.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.671+0000 7f072effd700 1 -- 172.21.15.107:0/251939887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07300fbd40 msgr2=0x7f07301108b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:08.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.671+0000 7f072effd700 1 --2- 172.21.15.107:0/251939887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07300fbd40 0x7f07301108b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:08.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.671+0000 7f072effd700 1 -- 172.21.15.107:0/251939887 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0728009940 con 0x7f07300fcff0 2024-04-22T23:37:08.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.671+0000 7f072f7fe700 1 --2- 172.21.15.107:0/251939887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07300fbd40 0x7f07301108b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T23:37:08.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.671+0000 7f072effd700 1 --2- 172.21.15.107:0/251939887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07300fcff0 0x7f0730112e00 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f071800c830 tx=0x7f0718004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:08.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.671+0000 7f0727fff700 1 -- 172.21.15.107:0/251939887 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f071800a760 con 0x7f07300fcff0 2024-04-22T23:37:08.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.671+0000 7f0727fff700 1 -- 172.21.15.107:0/251939887 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f071800ad90 con 0x7f07300fcff0 2024-04-22T23:37:08.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.671+0000 7f0735f77700 1 -- 172.21.15.107:0/251939887 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07301133a0 con 0x7f07300fcff0 2024-04-22T23:37:08.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.671+0000 7f0727fff700 1 -- 172.21.15.107:0/251939887 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f07180113a0 con 0x7f07300fcff0 2024-04-22T23:37:08.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.671+0000 7f0735f77700 1 -- 172.21.15.107:0/251939887 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0730113960 con 0x7f07300fcff0 2024-04-22T23:37:08.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.672+0000 7f0735f77700 1 -- 172.21.15.107:0/251939887 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f073005f5f0 con 0x7f07300fcff0 2024-04-22T23:37:08.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.673+0000 7f0727fff700 1 -- 172.21.15.107:0/251939887 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) v1 ==== 89652+0+0 (secure 0 0 0) 0x7f0718012400 con 0x7f07300fcff0 2024-04-22T23:37:08.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.673+0000 7f0727fff700 1 --2- 172.21.15.107:0/251939887 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f071c06bcf0 0x7f071c06e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:08.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.673+0000 7f0727fff700 1 -- 172.21.15.107:0/251939887 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f071808aa70 con 0x7f07300fcff0 2024-04-22T23:37:08.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.673+0000 7f072f7fe700 1 --2- 172.21.15.107:0/251939887 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f071c06bcf0 0x7f071c06e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:08.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.674+0000 7f072f7fe700 1 --2- 172.21.15.107:0/251939887 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f071c06bcf0 0x7f071c06e1a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0728015040 tx=0x7f0728004d10 comp rx=0 tx=0).ready entity=mgr.14720 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:08.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.678+0000 7f0727fff700 1 -- 172.21.15.107:0/251939887 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187010 (secure 0 0 0) 0x7f0718054140 con 0x7f07300fcff0 2024-04-22T23:37:08.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.955+0000 7f0727fff700 1 -- 172.21.15.107:0/251939887 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 25) v1 ==== 92571+0+0 (secure 0 0 0) 0x7f0718088f20 con 0x7f07300fcff0 2024-04-22T23:37:08.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.969+0000 7f0735f77700 1 -- 172.21.15.107:0/251939887 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f073004bed0 con 0x7f07300fcff0 2024-04-22T23:37:08.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.970+0000 7f0727fff700 1 -- 172.21.15.107:0/251939887 <== mon.0 v2:172.21.15.107:3300/0 8 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f0718064050 con 0x7f07300fcff0 2024-04-22T23:37:08.964 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_OK 2024-04-22T23:37:08.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.972+0000 7f0735f77700 1 -- 172.21.15.107:0/251939887 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f071c06bcf0 msgr2=0x7f071c06e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:08.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.972+0000 7f0735f77700 1 --2- 172.21.15.107:0/251939887 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f071c06bcf0 0x7f071c06e1a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0728015040 tx=0x7f0728004d10 comp rx=0 tx=0).stop 2024-04-22T23:37:08.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.972+0000 7f0735f77700 1 -- 172.21.15.107:0/251939887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07300fcff0 msgr2=0x7f0730112e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:08.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.972+0000 7f0735f77700 1 --2- 172.21.15.107:0/251939887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07300fcff0 0x7f0730112e00 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f071800c830 tx=0x7f0718004470 comp rx=0 tx=0).stop 2024-04-22T23:37:08.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.972+0000 7f0735f77700 1 -- 172.21.15.107:0/251939887 shutdown_connections 2024-04-22T23:37:08.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.972+0000 7f0735f77700 1 --2- 172.21.15.107:0/251939887 >> [v2:172.21.15.157:6828/3332906296,v1:172.21.15.157:6829/3332906296] conn(0x7f071c06bcf0 0x7f071c06e1a0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:08.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.972+0000 7f0735f77700 1 --2- 172.21.15.107:0/251939887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07300fbd40 0x7f07301108b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:08.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.972+0000 7f0735f77700 1 --2- 172.21.15.107:0/251939887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07300fcff0 0x7f0730112e00 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:08.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.972+0000 7f0735f77700 1 -- 172.21.15.107:0/251939887 >> 172.21.15.107:0/251939887 conn(0x7f07300f74c0 msgr2=0x7f07300f8bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:08.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.972+0000 7f0735f77700 1 -- 172.21.15.107:0/251939887 shutdown_connections 2024-04-22T23:37:08.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:08.972+0000 7f0735f77700 1 -- 172.21.15.107:0/251939887 wait complete. 2024-04-22T23:37:09.224 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:08 smithi107 ceph-mon[26355]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:37:09.224 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:08 smithi107 ceph-mon[26355]: pgmap v28: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:37:09.224 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:08 smithi107 ceph-mon[26355]: Standby manager daemon smithi107.uipxvm restarted 2024-04-22T23:37:09.224 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:08 smithi107 ceph-mon[26355]: Standby manager daemon smithi107.uipxvm started 2024-04-22T23:37:09.224 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:08 smithi107 ceph-mon[26355]: from='mgr.? 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi107.uipxvm/crt"}]: dispatch 2024-04-22T23:37:09.225 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:08 smithi107 ceph-mon[26355]: from='mgr.? 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T23:37:09.225 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:08 smithi107 ceph-mon[26355]: from='mgr.? 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi107.uipxvm/key"}]: dispatch 2024-04-22T23:37:09.225 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:08 smithi107 ceph-mon[26355]: from='mgr.? 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T23:37:09.225 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:08 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/1042039940' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:37:09.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:08 smithi157 ceph-mon[33877]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:37:09.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:08 smithi157 ceph-mon[33877]: pgmap v28: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:37:09.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:08 smithi157 ceph-mon[33877]: Standby manager daemon smithi107.uipxvm restarted 2024-04-22T23:37:09.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:08 smithi157 ceph-mon[33877]: Standby manager daemon smithi107.uipxvm started 2024-04-22T23:37:09.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:08 smithi157 ceph-mon[33877]: from='mgr.? 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi107.uipxvm/crt"}]: dispatch 2024-04-22T23:37:09.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:08 smithi157 ceph-mon[33877]: from='mgr.? 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T23:37:09.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:08 smithi157 ceph-mon[33877]: from='mgr.? 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi107.uipxvm/key"}]: dispatch 2024-04-22T23:37:09.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:08 smithi157 ceph-mon[33877]: from='mgr.? 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T23:37:09.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:08 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/1042039940' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:37:10.227 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:09 smithi107 ceph-mon[26355]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:37:10.227 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:09 smithi107 ceph-mon[26355]: mgrmap e25: smithi157.tuylis(active, since 50s), standbys: smithi107.uipxvm 2024-04-22T23:37:10.227 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:09 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/251939887' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:37:10.282 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:09 smithi157 ceph-mon[33877]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:37:10.282 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:09 smithi157 ceph-mon[33877]: mgrmap e25: smithi157.tuylis(active, since 50s), standbys: smithi107.uipxvm 2024-04-22T23:37:10.282 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:09 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/251939887' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:37:11.234 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:10 smithi107 ceph-mon[26355]: pgmap v29: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:37:11.234 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:10 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:37:11.234 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:10 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:37:11.234 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:10 smithi107 ceph-mon[26355]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:37:11.234 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:10 smithi107 ceph-mon[26355]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:37:11.234 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:10 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:37:11.234 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:10 smithi107 ceph-mon[26355]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:37:11.234 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:10 smithi107 ceph-mon[26355]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "mgr fail", "who": "smithi157.tuylis"}]: dispatch 2024-04-22T23:37:11.234 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:10 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "mgr fail", "who": "smithi157.tuylis"}]: dispatch 2024-04-22T23:37:11.234 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:10 smithi107 ceph-mon[26355]: Activating manager daemon smithi107.uipxvm 2024-04-22T23:37:11.234 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:10 smithi107 ceph-mon[26355]: osdmap e48: 6 total, 6 up, 6 in 2024-04-22T23:37:11.234 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:10 smithi107 ceph-mon[26355]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' cmd='[{"prefix": "mgr fail", "who": "smithi157.tuylis"}]': finished 2024-04-22T23:37:11.235 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:10 smithi107 ceph-mon[26355]: mgrmap e26: smithi107.uipxvm(active, starting, since 0.00601705s) 2024-04-22T23:37:11.235 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:10 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-04-22T23:37:11.235 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:10 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mon metadata", "id": "smithi157"}]: dispatch 2024-04-22T23:37:11.235 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:10 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.ktuxjl"}]: dispatch 2024-04-22T23:37:11.235 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:10 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.dfvvkw"}]: dispatch 2024-04-22T23:37:11.235 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:10 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi157.rcaris"}]: dispatch 2024-04-22T23:37:11.235 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:10 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi157.ufywse"}]: dispatch 2024-04-22T23:37:11.235 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:10 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mgr metadata", "who": "smithi107.uipxvm", "id": "smithi107.uipxvm"}]: dispatch 2024-04-22T23:37:11.235 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:10 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T23:37:11.235 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:10 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T23:37:11.235 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:10 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T23:37:11.235 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:10 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T23:37:11.235 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:10 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T23:37:11.235 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:10 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T23:37:11.235 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:10 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-22T23:37:11.236 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:10 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-22T23:37:11.236 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:10 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-22T23:37:11.236 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:10 smithi107 ceph-mon[26355]: Manager daemon smithi107.uipxvm is now available 2024-04-22T23:37:11.236 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:10 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:37:11.259 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:10 smithi157 ceph-mon[33877]: pgmap v29: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:37:11.259 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:10 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:37:11.259 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:10 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:37:11.259 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:10 smithi157 ceph-mon[33877]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:37:11.259 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:10 smithi157 ceph-mon[33877]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:37:11.259 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:10 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' 2024-04-22T23:37:11.259 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:10 smithi157 ceph-mon[33877]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:37:11.259 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:10 smithi157 ceph-mon[33877]: from='mgr.14720 172.21.15.157:0/3331699018' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "mgr fail", "who": "smithi157.tuylis"}]: dispatch 2024-04-22T23:37:11.259 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:10 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "mgr fail", "who": "smithi157.tuylis"}]: dispatch 2024-04-22T23:37:11.260 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:10 smithi157 ceph-mon[33877]: Activating manager daemon smithi107.uipxvm 2024-04-22T23:37:11.260 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:10 smithi157 ceph-mon[33877]: osdmap e48: 6 total, 6 up, 6 in 2024-04-22T23:37:11.260 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:10 smithi157 ceph-mon[33877]: from='mgr.14720 ' entity='mgr.smithi157.tuylis' cmd='[{"prefix": "mgr fail", "who": "smithi157.tuylis"}]': finished 2024-04-22T23:37:11.260 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:10 smithi157 ceph-mon[33877]: mgrmap e26: smithi107.uipxvm(active, starting, since 0.00601705s) 2024-04-22T23:37:11.260 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:10 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-04-22T23:37:11.260 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:10 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mon metadata", "id": "smithi157"}]: dispatch 2024-04-22T23:37:11.260 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:10 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.ktuxjl"}]: dispatch 2024-04-22T23:37:11.260 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:10 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.dfvvkw"}]: dispatch 2024-04-22T23:37:11.260 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:10 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi157.rcaris"}]: dispatch 2024-04-22T23:37:11.260 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:10 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi157.ufywse"}]: dispatch 2024-04-22T23:37:11.260 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:10 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mgr metadata", "who": "smithi107.uipxvm", "id": "smithi107.uipxvm"}]: dispatch 2024-04-22T23:37:11.260 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:10 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T23:37:11.260 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:10 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T23:37:11.260 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:10 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T23:37:11.260 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:10 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T23:37:11.260 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:10 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T23:37:11.261 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:10 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T23:37:11.261 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:10 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-22T23:37:11.261 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:10 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-22T23:37:11.261 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:10 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-22T23:37:11.261 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:10 smithi157 ceph-mon[33877]: Manager daemon smithi107.uipxvm is now available 2024-04-22T23:37:11.261 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:10 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:37:12.060 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:11 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:37:12.060 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:11 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi107.uipxvm/mirror_snapshot_schedule"}]: dispatch 2024-04-22T23:37:12.060 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:11 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi107.uipxvm/trash_purge_schedule"}]: dispatch 2024-04-22T23:37:12.060 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:11 smithi157 ceph-mon[33877]: [22/Apr/2024:23:37:11] ENGINE Bus STARTING 2024-04-22T23:37:12.060 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:11 smithi157 ceph-mon[33877]: [22/Apr/2024:23:37:11] ENGINE Serving on https://172.21.15.107:7150 2024-04-22T23:37:12.060 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:11 smithi157 ceph-mon[33877]: [22/Apr/2024:23:37:11] ENGINE Serving on http://172.21.15.107:8765 2024-04-22T23:37:12.060 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:11 smithi157 ceph-mon[33877]: [22/Apr/2024:23:37:11] ENGINE Bus STARTED 2024-04-22T23:37:12.061 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:11 smithi157 ceph-mon[33877]: mgrmap e27: smithi107.uipxvm(active, since 1.00779s) 2024-04-22T23:37:12.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:11 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:37:12.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:11 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi107.uipxvm/mirror_snapshot_schedule"}]: dispatch 2024-04-22T23:37:12.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:11 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi107.uipxvm/trash_purge_schedule"}]: dispatch 2024-04-22T23:37:12.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:11 smithi107 ceph-mon[26355]: [22/Apr/2024:23:37:11] ENGINE Bus STARTING 2024-04-22T23:37:12.234 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:11 smithi107 ceph-mon[26355]: [22/Apr/2024:23:37:11] ENGINE Serving on https://172.21.15.107:7150 2024-04-22T23:37:12.234 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:11 smithi107 ceph-mon[26355]: [22/Apr/2024:23:37:11] ENGINE Serving on http://172.21.15.107:8765 2024-04-22T23:37:12.234 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:11 smithi107 ceph-mon[26355]: [22/Apr/2024:23:37:11] ENGINE Bus STARTED 2024-04-22T23:37:12.234 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:11 smithi107 ceph-mon[26355]: mgrmap e27: smithi107.uipxvm(active, since 1.00779s) 2024-04-22T23:37:13.052 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:12 smithi157 ceph-mon[33877]: pgmap v3: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:37:13.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:12 smithi107 ceph-mon[26355]: pgmap v3: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:37:14.222 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:13 smithi107 ceph-mon[26355]: pgmap v4: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:37:14.222 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:13 smithi107 ceph-mon[26355]: mgrmap e28: smithi107.uipxvm(active, since 2s) 2024-04-22T23:37:14.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:13 smithi157 ceph-mon[33877]: pgmap v4: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:37:14.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:13 smithi157 ceph-mon[33877]: mgrmap e28: smithi107.uipxvm(active, since 2s) 2024-04-22T23:37:15.627 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:15 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:15.628 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:15 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:15.628 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:15 smithi157 ceph-mon[33877]: mgrmap e29: smithi107.uipxvm(active, since 4s) 2024-04-22T23:37:15.646 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:15 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:15.646 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:15 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:15.646 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:15 smithi107 ceph-mon[26355]: mgrmap e29: smithi107.uipxvm(active, since 4s) 2024-04-22T23:37:16.849 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:16 smithi157 ceph-mon[33877]: pgmap v5: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:37:16.849 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:16 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:16.849 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:16 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:16.849 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:16 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:16.849 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:16 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:16.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:16 smithi107 ceph-mon[26355]: pgmap v5: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:37:16.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:16 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:16.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:16 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:16.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:16 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:16.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:16 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:18.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:18 smithi107 ceph-mon[26355]: pgmap v6: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:37:18.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:18 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:18.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:18 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:18.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:18 smithi107 ceph-mon[26355]: Standby manager daemon smithi157.tuylis started 2024-04-22T23:37:18.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:18 smithi107 ceph-mon[26355]: from='mgr.? 172.21.15.157:0/265044449' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi157.tuylis/crt"}]: dispatch 2024-04-22T23:37:18.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:18 smithi107 ceph-mon[26355]: from='mgr.? 172.21.15.157:0/265044449' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T23:37:18.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:18 smithi107 ceph-mon[26355]: from='mgr.? 172.21.15.157:0/265044449' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi157.tuylis/key"}]: dispatch 2024-04-22T23:37:18.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:18 smithi107 ceph-mon[26355]: from='mgr.? 172.21.15.157:0/265044449' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T23:37:18.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:18 smithi157 ceph-mon[33877]: pgmap v6: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:37:18.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:18 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:18.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:18 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:18.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:18 smithi157 ceph-mon[33877]: Standby manager daemon smithi157.tuylis started 2024-04-22T23:37:18.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:18 smithi157 ceph-mon[33877]: from='mgr.? 172.21.15.157:0/265044449' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi157.tuylis/crt"}]: dispatch 2024-04-22T23:37:18.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:18 smithi157 ceph-mon[33877]: from='mgr.? 172.21.15.157:0/265044449' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T23:37:18.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:18 smithi157 ceph-mon[33877]: from='mgr.? 172.21.15.157:0/265044449' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi157.tuylis/key"}]: dispatch 2024-04-22T23:37:18.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:18 smithi157 ceph-mon[33877]: from='mgr.? 172.21.15.157:0/265044449' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T23:37:19.660 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:19 smithi107 ceph-mon[26355]: mgrmap e30: smithi107.uipxvm(active, since 7s), standbys: smithi157.tuylis 2024-04-22T23:37:19.660 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:19 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mgr metadata", "who": "smithi157.tuylis", "id": "smithi157.tuylis"}]: dispatch 2024-04-22T23:37:19.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:19 smithi157 ceph-mon[33877]: mgrmap e30: smithi107.uipxvm(active, since 7s), standbys: smithi157.tuylis 2024-04-22T23:37:19.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:19 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mgr metadata", "who": "smithi157.tuylis", "id": "smithi157.tuylis"}]: dispatch 2024-04-22T23:37:20.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:20 smithi157 ceph-mon[33877]: pgmap v7: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-04-22T23:37:20.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:20 smithi107 ceph-mon[26355]: pgmap v7: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-04-22T23:37:22.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:22 smithi107 ceph-mon[26355]: pgmap v8: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-22T23:37:22.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:22 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:22.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:22 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:22.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:22 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:37:22.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:22 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:37:22.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:22 smithi107 ceph-mon[26355]: Updating smithi107:/etc/ceph/ceph.conf 2024-04-22T23:37:22.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:22 smithi107 ceph-mon[26355]: Updating smithi157:/etc/ceph/ceph.conf 2024-04-22T23:37:22.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:22 smithi107 ceph-mon[26355]: Updating smithi107:/var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T23:37:22.484 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:22 smithi107 ceph-mon[26355]: Updating smithi157:/var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T23:37:22.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:22 smithi157 ceph-mon[33877]: pgmap v8: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-22T23:37:22.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:22 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:22.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:22 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:22.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:22 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:37:22.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:22 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:37:22.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:22 smithi157 ceph-mon[33877]: Updating smithi107:/etc/ceph/ceph.conf 2024-04-22T23:37:22.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:22 smithi157 ceph-mon[33877]: Updating smithi157:/etc/ceph/ceph.conf 2024-04-22T23:37:22.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:22 smithi157 ceph-mon[33877]: Updating smithi107:/var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T23:37:22.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:22 smithi157 ceph-mon[33877]: Updating smithi157:/var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T23:37:23.413 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:23 smithi107 ceph-mon[26355]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2024-04-22T23:37:23.414 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:23 smithi107 ceph-mon[26355]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-04-22T23:37:23.414 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:23 smithi107 ceph-mon[26355]: Updating smithi157:/var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T23:37:23.414 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:23 smithi107 ceph-mon[26355]: Updating smithi107:/var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T23:37:23.414 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:23 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:23.414 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:23 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:23.414 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:23 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:23.414 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:23 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:23.414 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:23 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:23.414 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:23 smithi107 ceph-mon[26355]: Reconfiguring prometheus.smithi107 (dependencies changed)... 2024-04-22T23:37:23.414 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:23 smithi107 ceph-mon[26355]: Reconfiguring daemon prometheus.smithi107 on smithi107 2024-04-22T23:37:23.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:23 smithi157 ceph-mon[33877]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2024-04-22T23:37:23.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:23 smithi157 ceph-mon[33877]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-04-22T23:37:23.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:23 smithi157 ceph-mon[33877]: Updating smithi157:/var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T23:37:23.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:23 smithi157 ceph-mon[33877]: Updating smithi107:/var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T23:37:23.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:23 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:23.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:23 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:23.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:23 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:23.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:23 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:23.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:23 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:23.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:23 smithi157 ceph-mon[33877]: Reconfiguring prometheus.smithi107 (dependencies changed)... 2024-04-22T23:37:23.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:23 smithi157 ceph-mon[33877]: Reconfiguring daemon prometheus.smithi107 on smithi107 2024-04-22T23:37:24.525 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:24 smithi107 ceph-mon[26355]: pgmap v9: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-22T23:37:24.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:24 smithi157 ceph-mon[33877]: pgmap v9: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-22T23:37:26.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:26 smithi157 ceph-mon[33877]: pgmap v10: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-22T23:37:26.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:26 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:37:26.683 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:26 smithi107 ceph-mon[26355]: pgmap v10: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-22T23:37:26.683 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:26 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:37:28.611 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:28 smithi157 ceph-mon[33877]: pgmap v11: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-22T23:37:28.612 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:28 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:28.612 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:28 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:28.612 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:28 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-22T23:37:28.612 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:28 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi107:9095"}]: dispatch 2024-04-22T23:37:28.612 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:28 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:28.612 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:28 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:37:28.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:28 smithi107 ceph-mon[26355]: pgmap v11: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-22T23:37:28.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:28 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:28.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:28 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:28.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:28 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-22T23:37:28.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:28 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi107:9095"}]: dispatch 2024-04-22T23:37:28.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:28 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:28.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:28 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:37:29.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:29 smithi157 ceph-mon[33877]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-22T23:37:29.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:29 smithi157 ceph-mon[33877]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi107:9095"}]: dispatch 2024-04-22T23:37:29.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:29 smithi107 ceph-mon[26355]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-22T23:37:29.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:29 smithi107 ceph-mon[26355]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi107:9095"}]: dispatch 2024-04-22T23:37:30.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:30 smithi157 ceph-mon[33877]: pgmap v12: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-22T23:37:30.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:30 smithi157 ceph-mon[33877]: Upgrade: Updating mgr.smithi157.tuylis 2024-04-22T23:37:30.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:30 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi157.tuylis", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T23:37:30.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:30 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T23:37:30.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:30 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:37:30.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:30 smithi157 ceph-mon[33877]: Deploying daemon mgr.smithi157.tuylis on smithi157 2024-04-22T23:37:30.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:30 smithi107 ceph-mon[26355]: pgmap v12: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-22T23:37:30.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:30 smithi107 ceph-mon[26355]: Upgrade: Updating mgr.smithi157.tuylis 2024-04-22T23:37:30.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:30 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi157.tuylis", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T23:37:30.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:30 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T23:37:30.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:30 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:37:30.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:30 smithi107 ceph-mon[26355]: Deploying daemon mgr.smithi157.tuylis on smithi157 2024-04-22T23:37:32.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:32 smithi157 ceph-mon[33877]: pgmap v13: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:37:32.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:32 smithi107 ceph-mon[26355]: pgmap v13: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:37:34.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:34 smithi107 ceph-mon[26355]: pgmap v14: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:37:34.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:34 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:34.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:34 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:34.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:34 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:37:34.866 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:34 smithi157 ceph-mon[33877]: pgmap v14: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:37:34.866 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:34 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:34.866 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:34 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:34.866 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:34 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:37:36.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:36 smithi107 ceph-mon[26355]: pgmap v15: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:37:36.770 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:36 smithi157 ceph-mon[33877]: pgmap v15: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:37:37.965 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:37 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:37.965 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:37 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:37.965 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:37 smithi107 ceph-mon[26355]: pgmap v16: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:37:37.973 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:37 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:37.973 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:37 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:37.973 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:37 smithi157 ceph-mon[33877]: pgmap v16: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:37:38.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:38 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:38.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:38 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:38.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:38 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:38.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:38 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:39.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.080+0000 7fb81da4b700 1 -- 172.21.15.107:0/3953318783 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8180fc320 msgr2=0x7fb8180fc730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:39.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.080+0000 7fb81da4b700 1 --2- 172.21.15.107:0/3953318783 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8180fc320 0x7fb8180fc730 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb80c004940 tx=0x7fb80c009e30 comp rx=0 tx=0).stop 2024-04-22T23:37:39.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.080+0000 7fb81da4b700 1 -- 172.21.15.107:0/3953318783 shutdown_connections 2024-04-22T23:37:39.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.080+0000 7fb81da4b700 1 --2- 172.21.15.107:0/3953318783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8180fcc70 0x7fb8180fd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:39.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.080+0000 7fb81da4b700 1 --2- 172.21.15.107:0/3953318783 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8180fc320 0x7fb8180fc730 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:39.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.080+0000 7fb81da4b700 1 -- 172.21.15.107:0/3953318783 >> 172.21.15.107:0/3953318783 conn(0x7fb8180f6bd0 msgr2=0x7fb8180f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:39.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.080+0000 7fb81da4b700 1 -- 172.21.15.107:0/3953318783 shutdown_connections 2024-04-22T23:37:39.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.080+0000 7fb81da4b700 1 -- 172.21.15.107:0/3953318783 wait complete. 2024-04-22T23:37:39.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.081+0000 7fb81da4b700 1 Processor -- start 2024-04-22T23:37:39.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.081+0000 7fb81da4b700 1 -- start start 2024-04-22T23:37:39.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.081+0000 7fb81da4b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8180fc320 0x7fb81810fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:39.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.081+0000 7fb81da4b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8180fcc70 0x7fb818110120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:39.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.081+0000 7fb81da4b700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb818110720 con 0x7fb8180fc320 2024-04-22T23:37:39.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.081+0000 7fb81da4b700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb818110890 con 0x7fb8180fcc70 2024-04-22T23:37:39.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.082+0000 7fb816ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8180fc320 0x7fb81810fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:39.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.082+0000 7fb816ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8180fc320 0x7fb81810fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53166/0 (socket says 172.21.15.107:53166) 2024-04-22T23:37:39.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.082+0000 7fb816ffd700 1 -- 172.21.15.107:0/3224609229 learned_addr learned my addr 172.21.15.107:0/3224609229 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:37:39.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.082+0000 7fb8167fc700 1 --2- 172.21.15.107:0/3224609229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8180fcc70 0x7fb818110120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:39.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.082+0000 7fb816ffd700 1 -- 172.21.15.107:0/3224609229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8180fcc70 msgr2=0x7fb818110120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:39.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.082+0000 7fb816ffd700 1 --2- 172.21.15.107:0/3224609229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8180fcc70 0x7fb818110120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:39.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.082+0000 7fb816ffd700 1 -- 172.21.15.107:0/3224609229 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb80c005040 con 0x7fb8180fc320 2024-04-22T23:37:39.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.082+0000 7fb816ffd700 1 --2- 172.21.15.107:0/3224609229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8180fc320 0x7fb81810fbe0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb80c00b3b0 tx=0x7fb80c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:39.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.082+0000 7fb7ff7fe700 1 -- 172.21.15.107:0/3224609229 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb80c00e040 con 0x7fb8180fc320 2024-04-22T23:37:39.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.082+0000 7fb81da4b700 1 -- 172.21.15.107:0/3224609229 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb818115240 con 0x7fb8180fc320 2024-04-22T23:37:39.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.082+0000 7fb7ff7fe700 1 -- 172.21.15.107:0/3224609229 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb80c00bbf0 con 0x7fb8180fc320 2024-04-22T23:37:39.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.082+0000 7fb7ff7fe700 1 -- 172.21.15.107:0/3224609229 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb80c01c630 con 0x7fb8180fc320 2024-04-22T23:37:39.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.082+0000 7fb81da4b700 1 -- 172.21.15.107:0/3224609229 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb818115800 con 0x7fb8180fc320 2024-04-22T23:37:39.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.083+0000 7fb81da4b700 1 -- 172.21.15.107:0/3224609229 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb818101a70 con 0x7fb8180fc320 2024-04-22T23:37:39.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.084+0000 7fb7ff7fe700 1 -- 172.21.15.107:0/3224609229 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 30) v1 ==== 92571+0+0 (secure 0 0 0) 0x7fb80c00bd60 con 0x7fb8180fc320 2024-04-22T23:37:39.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.084+0000 7fb7ff7fe700 1 --2- 172.21.15.107:0/3224609229 >> [v2:172.21.15.107:6800/2362628021,v1:172.21.15.107:6801/2362628021] conn(0x7fb80006e630 0x7fb800070ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:39.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.084+0000 7fb7ff7fe700 1 -- 172.21.15.107:0/3224609229 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6152+0+0 (secure 0 0 0) 0x7fb80c08ed70 con 0x7fb8180fc320 2024-04-22T23:37:39.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.084+0000 7fb8167fc700 1 --2- 172.21.15.107:0/3224609229 >> [v2:172.21.15.107:6800/2362628021,v1:172.21.15.107:6801/2362628021] conn(0x7fb80006e630 0x7fb800070ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:39.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.085+0000 7fb8167fc700 1 --2- 172.21.15.107:0/3224609229 >> [v2:172.21.15.107:6800/2362628021,v1:172.21.15.107:6801/2362628021] conn(0x7fb80006e630 0x7fb800070ae0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb808004f40 tx=0x7fb80800b040 comp rx=0 tx=0).ready entity=mgr.14778 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:39.081 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:38 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:39.081 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:38 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:39.082 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:38 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:39.082 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:38 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:39.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.089+0000 7fb7ff7fe700 1 -- 172.21.15.107:0/3224609229 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187010 (secure 0 0 0) 0x7fb80c05bd00 con 0x7fb8180fc320 2024-04-22T23:37:39.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.306+0000 7fb81da4b700 1 -- 172.21.15.107:0/3224609229 --> [v2:172.21.15.107:6800/2362628021,v1:172.21.15.107:6801/2362628021] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb81805e400 con 0x7fb80006e630 2024-04-22T23:37:39.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.308+0000 7fb7ff7fe700 1 -- 172.21.15.107:0/3224609229 <== mgr.14778 v2:172.21.15.107:6800/2362628021 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fb81805e400 con 0x7fb80006e630 2024-04-22T23:37:39.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.310+0000 7fb81da4b700 1 -- 172.21.15.107:0/3224609229 >> [v2:172.21.15.107:6800/2362628021,v1:172.21.15.107:6801/2362628021] conn(0x7fb80006e630 msgr2=0x7fb800070ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:39.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.310+0000 7fb81da4b700 1 --2- 172.21.15.107:0/3224609229 >> [v2:172.21.15.107:6800/2362628021,v1:172.21.15.107:6801/2362628021] conn(0x7fb80006e630 0x7fb800070ae0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb808004f40 tx=0x7fb80800b040 comp rx=0 tx=0).stop 2024-04-22T23:37:39.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.310+0000 7fb81da4b700 1 -- 172.21.15.107:0/3224609229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8180fc320 msgr2=0x7fb81810fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:39.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.310+0000 7fb81da4b700 1 --2- 172.21.15.107:0/3224609229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8180fc320 0x7fb81810fbe0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb80c00b3b0 tx=0x7fb80c009e30 comp rx=0 tx=0).stop 2024-04-22T23:37:39.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.310+0000 7fb81da4b700 1 -- 172.21.15.107:0/3224609229 shutdown_connections 2024-04-22T23:37:39.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.310+0000 7fb81da4b700 1 --2- 172.21.15.107:0/3224609229 >> [v2:172.21.15.107:6800/2362628021,v1:172.21.15.107:6801/2362628021] conn(0x7fb80006e630 0x7fb800070ae0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:39.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.310+0000 7fb81da4b700 1 --2- 172.21.15.107:0/3224609229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8180fc320 0x7fb81810fbe0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:39.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.310+0000 7fb81da4b700 1 --2- 172.21.15.107:0/3224609229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8180fcc70 0x7fb818110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:39.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.310+0000 7fb81da4b700 1 -- 172.21.15.107:0/3224609229 >> 172.21.15.107:0/3224609229 conn(0x7fb8180f6bd0 msgr2=0x7fb8180fb5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:39.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.310+0000 7fb81da4b700 1 -- 172.21.15.107:0/3224609229 shutdown_connections 2024-04-22T23:37:39.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.310+0000 7fb81da4b700 1 -- 172.21.15.107:0/3224609229 wait complete. 2024-04-22T23:37:39.314 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-04-22T23:37:39.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.412+0000 7fa6741c2700 1 -- 172.21.15.107:0/2685889089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa66c066f60 msgr2=0x7fa66c0ffa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:39.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.412+0000 7fa6741c2700 1 --2- 172.21.15.107:0/2685889089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa66c066f60 0x7fa66c0ffa10 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa668004940 tx=0x7fa668009e30 comp rx=0 tx=0).stop 2024-04-22T23:37:39.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.412+0000 7fa6741c2700 1 -- 172.21.15.107:0/2685889089 shutdown_connections 2024-04-22T23:37:39.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.412+0000 7fa6741c2700 1 --2- 172.21.15.107:0/2685889089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa66c0fff50 0x7fa66c102370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:39.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.412+0000 7fa6741c2700 1 --2- 172.21.15.107:0/2685889089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa66c066f60 0x7fa66c0ffa10 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:39.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.412+0000 7fa6741c2700 1 -- 172.21.15.107:0/2685889089 >> 172.21.15.107:0/2685889089 conn(0x7fa66c0f7480 msgr2=0x7fa66c0f98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:39.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.412+0000 7fa6741c2700 1 -- 172.21.15.107:0/2685889089 shutdown_connections 2024-04-22T23:37:39.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.412+0000 7fa6741c2700 1 -- 172.21.15.107:0/2685889089 wait complete. 2024-04-22T23:37:39.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.412+0000 7fa6741c2700 1 Processor -- start 2024-04-22T23:37:39.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.412+0000 7fa6741c2700 1 -- start start 2024-04-22T23:37:39.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.413+0000 7fa6741c2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa66c066f60 0x7fa66c10dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:39.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.413+0000 7fa6741c2700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa66c0fff50 0x7fa66c10e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:39.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.413+0000 7fa6741c2700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66c10e860 con 0x7fa66c066f60 2024-04-22T23:37:39.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.413+0000 7fa6741c2700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66c113200 con 0x7fa66c0fff50 2024-04-22T23:37:39.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.413+0000 7fa671f5e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa66c066f60 0x7fa66c10dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:39.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.413+0000 7fa671f5e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa66c066f60 0x7fa66c10dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53174/0 (socket says 172.21.15.107:53174) 2024-04-22T23:37:39.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.413+0000 7fa671f5e700 1 -- 172.21.15.107:0/2496043298 learned_addr learned my addr 172.21.15.107:0/2496043298 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:37:39.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.413+0000 7fa67175d700 1 --2- 172.21.15.107:0/2496043298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa66c0fff50 0x7fa66c10e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:39.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.413+0000 7fa671f5e700 1 -- 172.21.15.107:0/2496043298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa66c0fff50 msgr2=0x7fa66c10e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:39.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.413+0000 7fa671f5e700 1 --2- 172.21.15.107:0/2496043298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa66c0fff50 0x7fa66c10e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:39.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.413+0000 7fa671f5e700 1 -- 172.21.15.107:0/2496043298 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa668005040 con 0x7fa66c066f60 2024-04-22T23:37:39.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.413+0000 7fa671f5e700 1 --2- 172.21.15.107:0/2496043298 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa66c066f60 0x7fa66c10dca0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa66800b3b0 tx=0x7fa668009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:39.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.413+0000 7fa6627fc700 1 -- 172.21.15.107:0/2496043298 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa66800e040 con 0x7fa66c066f60 2024-04-22T23:37:39.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.413+0000 7fa6627fc700 1 -- 172.21.15.107:0/2496043298 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa66800bbf0 con 0x7fa66c066f60 2024-04-22T23:37:39.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.413+0000 7fa6741c2700 1 -- 172.21.15.107:0/2496043298 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa66c1133a0 con 0x7fa66c066f60 2024-04-22T23:37:39.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.413+0000 7fa6741c2700 1 -- 172.21.15.107:0/2496043298 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa66c113910 con 0x7fa66c066f60 2024-04-22T23:37:39.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.413+0000 7fa6627fc700 1 -- 172.21.15.107:0/2496043298 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa66801c630 con 0x7fa66c066f60 2024-04-22T23:37:39.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.414+0000 7fa6741c2700 1 -- 172.21.15.107:0/2496043298 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa66c04c710 con 0x7fa66c066f60 2024-04-22T23:37:39.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.415+0000 7fa6627fc700 1 -- 172.21.15.107:0/2496043298 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 30) v1 ==== 92571+0+0 (secure 0 0 0) 0x7fa66800bd60 con 0x7fa66c066f60 2024-04-22T23:37:39.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.415+0000 7fa6627fc700 1 --2- 172.21.15.107:0/2496043298 >> [v2:172.21.15.107:6800/2362628021,v1:172.21.15.107:6801/2362628021] conn(0x7fa65806e630 0x7fa658070ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:39.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.416+0000 7fa6627fc700 1 -- 172.21.15.107:0/2496043298 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6152+0+0 (secure 0 0 0) 0x7fa66808ed40 con 0x7fa66c066f60 2024-04-22T23:37:39.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.416+0000 7fa67175d700 1 --2- 172.21.15.107:0/2496043298 >> [v2:172.21.15.107:6800/2362628021,v1:172.21.15.107:6801/2362628021] conn(0x7fa65806e630 0x7fa658070ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:39.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.416+0000 7fa67175d700 1 --2- 172.21.15.107:0/2496043298 >> [v2:172.21.15.107:6800/2362628021,v1:172.21.15.107:6801/2362628021] conn(0x7fa65806e630 0x7fa658070ae0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa66c10f170 tx=0x7fa65c00b040 comp rx=0 tx=0).ready entity=mgr.14778 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:39.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.420+0000 7fa6627fc700 1 -- 172.21.15.107:0/2496043298 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187010 (secure 0 0 0) 0x7fa6680581a0 con 0x7fa66c066f60 2024-04-22T23:37:39.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.636+0000 7fa6741c2700 1 -- 172.21.15.107:0/2496043298 --> [v2:172.21.15.107:6800/2362628021,v1:172.21.15.107:6801/2362628021] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa66c101be0 con 0x7fa65806e630 2024-04-22T23:37:39.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.638+0000 7fa6627fc700 1 -- 172.21.15.107:0/2496043298 <== mgr.14778 v2:172.21.15.107:6800/2362628021 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fa66c101be0 con 0x7fa65806e630 2024-04-22T23:37:39.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.640+0000 7fa6741c2700 1 -- 172.21.15.107:0/2496043298 >> [v2:172.21.15.107:6800/2362628021,v1:172.21.15.107:6801/2362628021] conn(0x7fa65806e630 msgr2=0x7fa658070ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:39.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.640+0000 7fa6741c2700 1 --2- 172.21.15.107:0/2496043298 >> [v2:172.21.15.107:6800/2362628021,v1:172.21.15.107:6801/2362628021] conn(0x7fa65806e630 0x7fa658070ae0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa66c10f170 tx=0x7fa65c00b040 comp rx=0 tx=0).stop 2024-04-22T23:37:39.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.640+0000 7fa6741c2700 1 -- 172.21.15.107:0/2496043298 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa66c066f60 msgr2=0x7fa66c10dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:39.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.640+0000 7fa6741c2700 1 --2- 172.21.15.107:0/2496043298 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa66c066f60 0x7fa66c10dca0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa66800b3b0 tx=0x7fa668009e30 comp rx=0 tx=0).stop 2024-04-22T23:37:39.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.640+0000 7fa6741c2700 1 -- 172.21.15.107:0/2496043298 shutdown_connections 2024-04-22T23:37:39.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.640+0000 7fa6741c2700 1 --2- 172.21.15.107:0/2496043298 >> [v2:172.21.15.107:6800/2362628021,v1:172.21.15.107:6801/2362628021] conn(0x7fa65806e630 0x7fa658070ae0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:39.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.640+0000 7fa6741c2700 1 --2- 172.21.15.107:0/2496043298 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa66c066f60 0x7fa66c10dca0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:39.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.640+0000 7fa6741c2700 1 --2- 172.21.15.107:0/2496043298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa66c0fff50 0x7fa66c10e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:39.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.640+0000 7fa6741c2700 1 -- 172.21.15.107:0/2496043298 >> 172.21.15.107:0/2496043298 conn(0x7fa66c0f7480 msgr2=0x7fa66c0f98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:39.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.640+0000 7fa6741c2700 1 -- 172.21.15.107:0/2496043298 shutdown_connections 2024-04-22T23:37:39.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.640+0000 7fa6741c2700 1 -- 172.21.15.107:0/2496043298 wait complete. 2024-04-22T23:37:39.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.739+0000 7f77e95d4700 1 -- 172.21.15.107:0/4124772331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77e40fce40 msgr2=0x7f77e40fd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:39.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.739+0000 7f77e95d4700 1 --2- 172.21.15.107:0/4124772331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77e40fce40 0x7f77e40fd250 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f77d4004940 tx=0x7f77d4009e30 comp rx=0 tx=0).stop 2024-04-22T23:37:39.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.739+0000 7f77e95d4700 1 -- 172.21.15.107:0/4124772331 shutdown_connections 2024-04-22T23:37:39.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.739+0000 7f77e95d4700 1 --2- 172.21.15.107:0/4124772331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77e40fe040 0x7f77e40fe4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:39.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.739+0000 7f77e95d4700 1 --2- 172.21.15.107:0/4124772331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77e40fce40 0x7f77e40fd250 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:39.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.739+0000 7f77e95d4700 1 -- 172.21.15.107:0/4124772331 >> 172.21.15.107:0/4124772331 conn(0x7f77e40f8530 msgr2=0x7f77e40fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:39.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.739+0000 7f77e95d4700 1 -- 172.21.15.107:0/4124772331 shutdown_connections 2024-04-22T23:37:39.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.739+0000 7f77e95d4700 1 -- 172.21.15.107:0/4124772331 wait complete. 2024-04-22T23:37:39.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.740+0000 7f77e95d4700 1 Processor -- start 2024-04-22T23:37:39.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.740+0000 7f77e95d4700 1 -- start start 2024-04-22T23:37:39.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.740+0000 7f77e95d4700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77e40fce40 0x7f77e4071e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:39.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.740+0000 7f77e95d4700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77e40fe040 0x7f77e4070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:39.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.740+0000 7f77e95d4700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e40708c0 con 0x7f77e40fe040 2024-04-22T23:37:39.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.740+0000 7f77e95d4700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e4070a30 con 0x7f77e40fce40 2024-04-22T23:37:39.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.740+0000 7f77e27fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77e40fe040 0x7f77e4070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:39.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.740+0000 7f77e2ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77e40fce40 0x7f77e4071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:39.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.740+0000 7f77e2ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77e40fce40 0x7f77e4071e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.107:60372/0 (socket says 172.21.15.107:60372) 2024-04-22T23:37:39.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.740+0000 7f77e2ffd700 1 -- 172.21.15.107:0/266270703 learned_addr learned my addr 172.21.15.107:0/266270703 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:37:39.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.741+0000 7f77e27fc700 1 -- 172.21.15.107:0/266270703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77e40fce40 msgr2=0x7f77e4071e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:39.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.741+0000 7f77e27fc700 1 --2- 172.21.15.107:0/266270703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77e40fce40 0x7f77e4071e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:39.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.741+0000 7f77e27fc700 1 -- 172.21.15.107:0/266270703 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d4005040 con 0x7f77e40fe040 2024-04-22T23:37:39.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.741+0000 7f77e27fc700 1 --2- 172.21.15.107:0/266270703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77e40fe040 0x7f77e4070310 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f77d8002820 tx=0x7f77d8004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:39.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.741+0000 7f77cb7fe700 1 -- 172.21.15.107:0/266270703 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77d8007cc0 con 0x7f77e40fe040 2024-04-22T23:37:39.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.741+0000 7f77cb7fe700 1 -- 172.21.15.107:0/266270703 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f77d800f040 con 0x7f77e40fe040 2024-04-22T23:37:39.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.741+0000 7f77e95d4700 1 -- 172.21.15.107:0/266270703 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77e4070d10 con 0x7f77e40fe040 2024-04-22T23:37:39.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.741+0000 7f77cb7fe700 1 -- 172.21.15.107:0/266270703 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77d8013920 con 0x7f77e40fe040 2024-04-22T23:37:39.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.741+0000 7f77e95d4700 1 -- 172.21.15.107:0/266270703 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77e4115ac0 con 0x7f77e40fe040 2024-04-22T23:37:39.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.742+0000 7f77e95d4700 1 -- 172.21.15.107:0/266270703 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77e405f5f0 con 0x7f77e40fe040 2024-04-22T23:37:39.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.743+0000 7f77cb7fe700 1 -- 172.21.15.107:0/266270703 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 30) v1 ==== 92571+0+0 (secure 0 0 0) 0x7f77d801c400 con 0x7f77e40fe040 2024-04-22T23:37:39.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.743+0000 7f77cb7fe700 1 --2- 172.21.15.107:0/266270703 >> [v2:172.21.15.107:6800/2362628021,v1:172.21.15.107:6801/2362628021] conn(0x7f77cc06e620 0x7f77cc070ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:39.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.743+0000 7f77cb7fe700 1 -- 172.21.15.107:0/266270703 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6152+0+0 (secure 0 0 0) 0x7f77d808ddb0 con 0x7f77e40fe040 2024-04-22T23:37:39.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.743+0000 7f77e2ffd700 1 --2- 172.21.15.107:0/266270703 >> [v2:172.21.15.107:6800/2362628021,v1:172.21.15.107:6801/2362628021] conn(0x7f77cc06e620 0x7f77cc070ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:39.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.744+0000 7f77e2ffd700 1 --2- 172.21.15.107:0/266270703 >> [v2:172.21.15.107:6800/2362628021,v1:172.21.15.107:6801/2362628021] conn(0x7f77cc06e620 0x7f77cc070ad0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f77d4004f90 tx=0x7f77d4009e30 comp rx=0 tx=0).ready entity=mgr.14778 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:39.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.748+0000 7f77cb7fe700 1 -- 172.21.15.107:0/266270703 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187010 (secure 0 0 0) 0x7f77d8057210 con 0x7f77e40fe040 2024-04-22T23:37:39.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.957+0000 7f77e95d4700 1 -- 172.21.15.107:0/266270703 --> [v2:172.21.15.107:6800/2362628021,v1:172.21.15.107:6801/2362628021] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f77e41023d0 con 0x7f77cc06e620 2024-04-22T23:37:39.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.967+0000 7f77cb7fe700 1 -- 172.21.15.107:0/266270703 <== mgr.14778 v2:172.21.15.107:6800/2362628021 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f77e41023d0 con 0x7f77cc06e620 2024-04-22T23:37:39.961 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-22T23:37:39.961 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (57s) 1s ago 10m 16.3M - 0.25.0 c8568f914cd2 ec1a5c6260de 2024-04-22T23:37:39.961 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (10m) 1s ago 10m 6665k - 17.2.7-884-g70d0cb4b cafd8ddac8fa c10611c756e4 2024-04-22T23:37:39.961 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi157 smithi157 running (9m) 3s ago 9m 6669k - 17.2.7-884-g70d0cb4b cafd8ddac8fa 55d2d46294c9 2024-04-22T23:37:39.961 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (52s) 1s ago 10m 80.1M - 9.4.7 954c08fa6188 31449ff442d2 2024-04-22T23:37:39.961 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.dfvvkw smithi107 running (5m) 1s ago 5m 12.6M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 08d3aa8d717d 2024-04-22T23:37:39.961 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.ktuxjl smithi107 running (5m) 1s ago 5m 124M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 762f27fe0f30 2024-04-22T23:37:39.961 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi157.rcaris smithi157 running (5m) 3s ago 5m 128M - 17.2.7-884-g70d0cb4b cafd8ddac8fa d23366ac085d 2024-04-22T23:37:39.962 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi157.ufywse smithi157 running (5m) 3s ago 5m 13.2M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 8b00d61eb355 2024-04-22T23:37:39.962 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.uipxvm smithi107 *:8443,9283,8765 running (39s) 1s ago 11m 494M - 19.0.0-3252-g43be0201 a0434699c1d7 d20f05dfe4f2 2024-04-22T23:37:39.962 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi157.tuylis smithi157 *:8443,9283,8765 running (7s) 3s ago 9m 63.4M - 19.0.0-3252-g43be0201 a0434699c1d7 957e598c9001 2024-04-22T23:37:39.962 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (12m) 1s ago 12m 58.4M 2048M 17.2.7-884-g70d0cb4b cafd8ddac8fa 8314962fa1b9 2024-04-22T23:37:39.962 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi157 smithi157 running (9m) 3s ago 9m 40.4M 2048M 17.2.7-884-g70d0cb4b cafd8ddac8fa 0747a85008d9 2024-04-22T23:37:39.962 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (64s) 1s ago 10m 8312k - 1.5.0 0da6a335fe13 5205cfd9bd51 2024-04-22T23:37:39.962 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi157 smithi157 *:9100 running (45s) 3s ago 9m 8686k - 1.5.0 0da6a335fe13 168ca191bb8c 2024-04-22T23:37:39.962 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (8m) 1s ago 8m 230M 1557M 17.2.7-884-g70d0cb4b cafd8ddac8fa 18a179b0a618 2024-04-22T23:37:39.962 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (8m) 1s ago 8m 241M 1557M 17.2.7-884-g70d0cb4b cafd8ddac8fa c7272eb84ddb 2024-04-22T23:37:39.962 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (7m) 1s ago 7m 193M 1557M 17.2.7-884-g70d0cb4b cafd8ddac8fa 9db1788096ad 2024-04-22T23:37:39.962 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi157 running (7m) 3s ago 7m 265M 2581M 17.2.7-884-g70d0cb4b cafd8ddac8fa c9fb2d1a13dd 2024-04-22T23:37:39.962 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi157 running (7m) 3s ago 7m 225M 2581M 17.2.7-884-g70d0cb4b cafd8ddac8fa 24a16e19951f 2024-04-22T23:37:39.962 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi157 running (6m) 3s ago 6m 225M 2581M 17.2.7-884-g70d0cb4b cafd8ddac8fa 4c5b7a7ef8bd 2024-04-22T23:37:39.962 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (12s) 1s ago 10m 46.4M - 2.43.0 a07b618ecd1d 99a4ea8594da 2024-04-22T23:37:39.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.969+0000 7f77e95d4700 1 -- 172.21.15.107:0/266270703 >> [v2:172.21.15.107:6800/2362628021,v1:172.21.15.107:6801/2362628021] conn(0x7f77cc06e620 msgr2=0x7f77cc070ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:39.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.969+0000 7f77e95d4700 1 --2- 172.21.15.107:0/266270703 >> [v2:172.21.15.107:6800/2362628021,v1:172.21.15.107:6801/2362628021] conn(0x7f77cc06e620 0x7f77cc070ad0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f77d4004f90 tx=0x7f77d4009e30 comp rx=0 tx=0).stop 2024-04-22T23:37:39.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.969+0000 7f77e95d4700 1 -- 172.21.15.107:0/266270703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77e40fe040 msgr2=0x7f77e4070310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:39.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.969+0000 7f77e95d4700 1 --2- 172.21.15.107:0/266270703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77e40fe040 0x7f77e4070310 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f77d8002820 tx=0x7f77d8004570 comp rx=0 tx=0).stop 2024-04-22T23:37:39.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.969+0000 7f77e95d4700 1 -- 172.21.15.107:0/266270703 shutdown_connections 2024-04-22T23:37:39.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.969+0000 7f77e95d4700 1 --2- 172.21.15.107:0/266270703 >> [v2:172.21.15.107:6800/2362628021,v1:172.21.15.107:6801/2362628021] conn(0x7f77cc06e620 0x7f77cc070ad0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:39.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.969+0000 7f77e95d4700 1 --2- 172.21.15.107:0/266270703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77e40fce40 0x7f77e4071e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:39.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.969+0000 7f77e95d4700 1 --2- 172.21.15.107:0/266270703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77e40fe040 0x7f77e4070310 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:39.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.969+0000 7f77e95d4700 1 -- 172.21.15.107:0/266270703 >> 172.21.15.107:0/266270703 conn(0x7f77e40f8530 msgr2=0x7f77e40f9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:39.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.969+0000 7f77e95d4700 1 -- 172.21.15.107:0/266270703 shutdown_connections 2024-04-22T23:37:39.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:39.969+0000 7f77e95d4700 1 -- 172.21.15.107:0/266270703 wait complete. 2024-04-22T23:37:40.051 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:39 smithi157 ceph-mon[33877]: pgmap v17: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:37:40.052 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:39 smithi157 ceph-mon[33877]: from='client.14836 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:37:40.052 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:39 smithi157 ceph-mon[33877]: from='client.14840 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:37:40.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.069+0000 7f5573e8c700 1 -- 172.21.15.107:0/2832848060 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f556c0fe090 msgr2=0x7f556c0fe500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:40.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.069+0000 7f5573e8c700 1 --2- 172.21.15.107:0/2832848060 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f556c0fe090 0x7f556c0fe500 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f555c004940 tx=0x7f555c009e30 comp rx=0 tx=0).stop 2024-04-22T23:37:40.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.069+0000 7f5573e8c700 1 -- 172.21.15.107:0/2832848060 shutdown_connections 2024-04-22T23:37:40.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.069+0000 7f5573e8c700 1 --2- 172.21.15.107:0/2832848060 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f556c0fe090 0x7f556c0fe500 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:40.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.069+0000 7f5573e8c700 1 --2- 172.21.15.107:0/2832848060 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f556c0fcf20 0x7f556c0fd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:40.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.069+0000 7f5573e8c700 1 -- 172.21.15.107:0/2832848060 >> 172.21.15.107:0/2832848060 conn(0x7f556c0f85c0 msgr2=0x7f556c0faa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:40.064 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:39 smithi107 ceph-mon[26355]: pgmap v17: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:37:40.064 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:39 smithi107 ceph-mon[26355]: from='client.14836 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:37:40.064 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:39 smithi107 ceph-mon[26355]: from='client.14840 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:37:40.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.069+0000 7f5573e8c700 1 -- 172.21.15.107:0/2832848060 shutdown_connections 2024-04-22T23:37:40.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.069+0000 7f5573e8c700 1 -- 172.21.15.107:0/2832848060 wait complete. 2024-04-22T23:37:40.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.070+0000 7f5573e8c700 1 Processor -- start 2024-04-22T23:37:40.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.070+0000 7f5573e8c700 1 -- start start 2024-04-22T23:37:40.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.070+0000 7f5573e8c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f556c0fcf20 0x7f556c10da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:40.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.070+0000 7f5573e8c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f556c0fe090 0x7f556c10dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:40.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.070+0000 7f5573e8c700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f556c10e5b0 con 0x7f556c0fcf20 2024-04-22T23:37:40.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.070+0000 7f5573e8c700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f556c10ee00 con 0x7f556c0fe090 2024-04-22T23:37:40.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.070+0000 7f5571c28700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f556c0fcf20 0x7f556c10da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:40.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.070+0000 7f5571c28700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f556c0fcf20 0x7f556c10da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53198/0 (socket says 172.21.15.107:53198) 2024-04-22T23:37:40.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.070+0000 7f5571c28700 1 -- 172.21.15.107:0/2588259690 learned_addr learned my addr 172.21.15.107:0/2588259690 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:37:40.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.070+0000 7f5571427700 1 --2- 172.21.15.107:0/2588259690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f556c0fe090 0x7f556c10dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:40.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.071+0000 7f5571c28700 1 -- 172.21.15.107:0/2588259690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f556c0fe090 msgr2=0x7f556c10dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:40.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.071+0000 7f5571c28700 1 --2- 172.21.15.107:0/2588259690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f556c0fe090 0x7f556c10dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:40.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.071+0000 7f5571c28700 1 -- 172.21.15.107:0/2588259690 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f555c005040 con 0x7f556c0fcf20 2024-04-22T23:37:40.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.071+0000 7f5571c28700 1 --2- 172.21.15.107:0/2588259690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f556c0fcf20 0x7f556c10da70 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f5568002a50 tx=0x7f55680046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:40.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.071+0000 7f55627fc700 1 -- 172.21.15.107:0/2588259690 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5568007cc0 con 0x7f556c0fcf20 2024-04-22T23:37:40.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.071+0000 7f55627fc700 1 -- 172.21.15.107:0/2588259690 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f556800f040 con 0x7f556c0fcf20 2024-04-22T23:37:40.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.071+0000 7f5573e8c700 1 -- 172.21.15.107:0/2588259690 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f556c10f0e0 con 0x7f556c0fcf20 2024-04-22T23:37:40.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.071+0000 7f55627fc700 1 -- 172.21.15.107:0/2588259690 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f556800da30 con 0x7f556c0fcf20 2024-04-22T23:37:40.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.071+0000 7f5573e8c700 1 -- 172.21.15.107:0/2588259690 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f556c10f620 con 0x7f556c0fcf20 2024-04-22T23:37:40.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.072+0000 7f5573e8c700 1 -- 172.21.15.107:0/2588259690 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f556c05f5f0 con 0x7f556c0fcf20 2024-04-22T23:37:40.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.072+0000 7f55627fc700 1 -- 172.21.15.107:0/2588259690 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 30) v1 ==== 92571+0+0 (secure 0 0 0) 0x7f556800db90 con 0x7f556c0fcf20 2024-04-22T23:37:40.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.073+0000 7f55627fc700 1 --2- 172.21.15.107:0/2588259690 >> [v2:172.21.15.107:6800/2362628021,v1:172.21.15.107:6801/2362628021] conn(0x7f555806e620 0x7f5558070ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:40.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.073+0000 7f55627fc700 1 -- 172.21.15.107:0/2588259690 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6152+0+0 (secure 0 0 0) 0x7f556808df40 con 0x7f556c0fcf20 2024-04-22T23:37:40.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.073+0000 7f5571427700 1 --2- 172.21.15.107:0/2588259690 >> [v2:172.21.15.107:6800/2362628021,v1:172.21.15.107:6801/2362628021] conn(0x7f555806e620 0x7f5558070ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:40.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.074+0000 7f5571427700 1 --2- 172.21.15.107:0/2588259690 >> [v2:172.21.15.107:6800/2362628021,v1:172.21.15.107:6801/2362628021] conn(0x7f555806e620 0x7f5558070ad0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f555c00b970 tx=0x7f555c009e30 comp rx=0 tx=0).ready entity=mgr.14778 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:40.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.078+0000 7f55627fc700 1 -- 172.21.15.107:0/2588259690 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187010 (secure 0 0 0) 0x7f55680c2aa0 con 0x7f556c0fcf20 2024-04-22T23:37:40.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.368+0000 7f5573e8c700 1 -- 172.21.15.107:0/2588259690 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f556c105910 con 0x7f556c0fcf20 2024-04-22T23:37:40.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.368+0000 7f55627fc700 1 -- 172.21.15.107:0/2588259690 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f556805abc0 con 0x7f556c0fcf20 2024-04-22T23:37:40.362 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-22T23:37:40.362 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-04-22T23:37:40.362 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 2 2024-04-22T23:37:40.362 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:37:40.363 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-04-22T23:37:40.363 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:37:40.363 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:37:40.363 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-04-22T23:37:40.363 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 6 2024-04-22T23:37:40.363 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:37:40.363 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-04-22T23:37:40.363 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 4 2024-04-22T23:37:40.363 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:37:40.363 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-04-22T23:37:40.363 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 12, 2024-04-22T23:37:40.363 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:37:40.363 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T23:37:40.364 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-22T23:37:40.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.370+0000 7f5573e8c700 1 -- 172.21.15.107:0/2588259690 >> [v2:172.21.15.107:6800/2362628021,v1:172.21.15.107:6801/2362628021] conn(0x7f555806e620 msgr2=0x7f5558070ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:40.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.370+0000 7f5573e8c700 1 --2- 172.21.15.107:0/2588259690 >> [v2:172.21.15.107:6800/2362628021,v1:172.21.15.107:6801/2362628021] conn(0x7f555806e620 0x7f5558070ad0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f555c00b970 tx=0x7f555c009e30 comp rx=0 tx=0).stop 2024-04-22T23:37:40.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.370+0000 7f5573e8c700 1 -- 172.21.15.107:0/2588259690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f556c0fcf20 msgr2=0x7f556c10da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:40.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.370+0000 7f5573e8c700 1 --2- 172.21.15.107:0/2588259690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f556c0fcf20 0x7f556c10da70 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f5568002a50 tx=0x7f55680046b0 comp rx=0 tx=0).stop 2024-04-22T23:37:40.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.370+0000 7f5573e8c700 1 -- 172.21.15.107:0/2588259690 shutdown_connections 2024-04-22T23:37:40.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.370+0000 7f5573e8c700 1 --2- 172.21.15.107:0/2588259690 >> [v2:172.21.15.107:6800/2362628021,v1:172.21.15.107:6801/2362628021] conn(0x7f555806e620 0x7f5558070ad0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:40.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.370+0000 7f5573e8c700 1 --2- 172.21.15.107:0/2588259690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f556c0fcf20 0x7f556c10da70 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:40.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.370+0000 7f5573e8c700 1 --2- 172.21.15.107:0/2588259690 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f556c0fe090 0x7f556c10dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:40.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.370+0000 7f5573e8c700 1 -- 172.21.15.107:0/2588259690 >> 172.21.15.107:0/2588259690 conn(0x7f556c0f85c0 msgr2=0x7f556c0f9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:40.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.370+0000 7f5573e8c700 1 -- 172.21.15.107:0/2588259690 shutdown_connections 2024-04-22T23:37:40.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.370+0000 7f5573e8c700 1 -- 172.21.15.107:0/2588259690 wait complete. 2024-04-22T23:37:40.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.471+0000 7fc90acfb700 1 -- 172.21.15.107:0/3264731807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9040fce40 msgr2=0x7fc9040fd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:40.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.471+0000 7fc90acfb700 1 --2- 172.21.15.107:0/3264731807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9040fce40 0x7fc9040fd250 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8004940 tx=0x7fc8f8009e30 comp rx=0 tx=0).stop 2024-04-22T23:37:40.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.471+0000 7fc90acfb700 1 -- 172.21.15.107:0/3264731807 shutdown_connections 2024-04-22T23:37:40.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.471+0000 7fc90acfb700 1 --2- 172.21.15.107:0/3264731807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9040fe040 0x7fc9040fe4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:40.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.471+0000 7fc90acfb700 1 --2- 172.21.15.107:0/3264731807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9040fce40 0x7fc9040fd250 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:40.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.471+0000 7fc90acfb700 1 -- 172.21.15.107:0/3264731807 >> 172.21.15.107:0/3264731807 conn(0x7fc9040f8570 msgr2=0x7fc9040fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:40.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.471+0000 7fc90acfb700 1 -- 172.21.15.107:0/3264731807 shutdown_connections 2024-04-22T23:37:40.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.471+0000 7fc90acfb700 1 -- 172.21.15.107:0/3264731807 wait complete. 2024-04-22T23:37:40.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.471+0000 7fc90acfb700 1 Processor -- start 2024-04-22T23:37:40.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.471+0000 7fc90acfb700 1 -- start start 2024-04-22T23:37:40.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.472+0000 7fc90acfb700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9040fce40 0x7fc90410d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:40.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.472+0000 7fc90acfb700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9040fe040 0x7fc90410dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:40.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.472+0000 7fc90acfb700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90410e4e0 con 0x7fc9040fce40 2024-04-22T23:37:40.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.472+0000 7fc90acfb700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90410ed30 con 0x7fc9040fe040 2024-04-22T23:37:40.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.472+0000 7fc908a97700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9040fce40 0x7fc90410d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:40.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.472+0000 7fc908a97700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9040fce40 0x7fc90410d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53212/0 (socket says 172.21.15.107:53212) 2024-04-22T23:37:40.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.472+0000 7fc908a97700 1 -- 172.21.15.107:0/3429422921 learned_addr learned my addr 172.21.15.107:0/3429422921 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:37:40.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.472+0000 7fc903fff700 1 --2- 172.21.15.107:0/3429422921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9040fe040 0x7fc90410dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:40.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.472+0000 7fc908a97700 1 -- 172.21.15.107:0/3429422921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9040fe040 msgr2=0x7fc90410dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:40.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.472+0000 7fc908a97700 1 --2- 172.21.15.107:0/3429422921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9040fe040 0x7fc90410dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:40.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.472+0000 7fc908a97700 1 -- 172.21.15.107:0/3429422921 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f8005040 con 0x7fc9040fce40 2024-04-22T23:37:40.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.472+0000 7fc908a97700 1 --2- 172.21.15.107:0/3429422921 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9040fce40 0x7fc90410d9a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8000c00 tx=0x7fc8f8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:40.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.473+0000 7fc9017fa700 1 -- 172.21.15.107:0/3429422921 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8f800e040 con 0x7fc9040fce40 2024-04-22T23:37:40.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.473+0000 7fc9017fa700 1 -- 172.21.15.107:0/3429422921 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc8f800bbf0 con 0x7fc9040fce40 2024-04-22T23:37:40.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.473+0000 7fc90acfb700 1 -- 172.21.15.107:0/3429422921 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc90410efb0 con 0x7fc9040fce40 2024-04-22T23:37:40.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.473+0000 7fc9017fa700 1 -- 172.21.15.107:0/3429422921 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8f801c630 con 0x7fc9040fce40 2024-04-22T23:37:40.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.473+0000 7fc90acfb700 1 -- 172.21.15.107:0/3429422921 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc90410f4f0 con 0x7fc9040fce40 2024-04-22T23:37:40.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.474+0000 7fc90acfb700 1 -- 172.21.15.107:0/3429422921 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc90405f5f0 con 0x7fc9040fce40 2024-04-22T23:37:40.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.474+0000 7fc9017fa700 1 -- 172.21.15.107:0/3429422921 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 30) v1 ==== 92571+0+0 (secure 0 0 0) 0x7fc8f800bd60 con 0x7fc9040fce40 2024-04-22T23:37:40.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.474+0000 7fc9017fa700 1 --2- 172.21.15.107:0/3429422921 >> [v2:172.21.15.107:6800/2362628021,v1:172.21.15.107:6801/2362628021] conn(0x7fc8f406e620 0x7fc8f4070ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:40.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.475+0000 7fc9017fa700 1 -- 172.21.15.107:0/3429422921 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6152+0+0 (secure 0 0 0) 0x7fc8f808f450 con 0x7fc9040fce40 2024-04-22T23:37:40.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.475+0000 7fc903fff700 1 --2- 172.21.15.107:0/3429422921 >> [v2:172.21.15.107:6800/2362628021,v1:172.21.15.107:6801/2362628021] conn(0x7fc8f406e620 0x7fc8f4070ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:40.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.475+0000 7fc903fff700 1 --2- 172.21.15.107:0/3429422921 >> [v2:172.21.15.107:6800/2362628021,v1:172.21.15.107:6801/2362628021] conn(0x7fc8f406e620 0x7fc8f4070ad0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc8f0004f40 tx=0x7fc8f000b040 comp rx=0 tx=0).ready entity=mgr.14778 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:40.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.479+0000 7fc9017fa700 1 -- 172.21.15.107:0/3429422921 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187010 (secure 0 0 0) 0x7fc8f805c3e0 con 0x7fc9040fce40 2024-04-22T23:37:40.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.724+0000 7fc90acfb700 1 -- 172.21.15.107:0/3429422921 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc904105860 con 0x7fc9040fce40 2024-04-22T23:37:40.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.725+0000 7fc9017fa700 1 -- 172.21.15.107:0/3429422921 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) v1 ==== 76+0+1858 (secure 0 0 0) 0x7fc8f805bf70 con 0x7fc9040fce40 2024-04-22T23:37:40.719 INFO:teuthology.orchestra.run.smithi107.stdout:e19 2024-04-22T23:37:40.719 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T23:37:40.719 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:37:40.719 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-04-22T23:37:40.719 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:37:40.719 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-04-22T23:37:40.719 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-04-22T23:37:40.719 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 19 2024-04-22T23:37:40.719 INFO:teuthology.orchestra.run.smithi107.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-22T23:37:40.720 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-04-22T23:32:20.146130+0000 2024-04-22T23:37:40.720 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-04-22T23:35:12.338678+0000 2024-04-22T23:37:40.720 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-04-22T23:37:40.720 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-04-22T23:37:40.720 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-04-22T23:37:40.720 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-04-22T23:37:40.720 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-04-22T23:37:40.720 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-04-22T23:37:40.720 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-04-22T23:37:40.720 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 46 2024-04-22T23:37:40.720 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:37:40.720 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 2 2024-04-22T23:37:40.721 INFO:teuthology.orchestra.run.smithi107.stdout:in 0,1 2024-04-22T23:37:40.721 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=14468,1=24265} 2024-04-22T23:37:40.721 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-04-22T23:37:40.721 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-04-22T23:37:40.721 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-04-22T23:37:40.721 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-04-22T23:37:40.721 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-04-22T23:37:40.721 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data disabled 2024-04-22T23:37:40.721 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-04-22T23:37:40.721 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-04-22T23:37:40.721 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.ktuxjl{0:14468} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.107:6828/3335313253,v1:172.21.15.107:6829/3335313253] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:37:40.721 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi157.rcaris{1:24265} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.157:6824/2779251893,v1:172.21.15.157:6825/2779251893] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:37:40.722 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:37:40.722 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:37:40.722 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-04-22T23:37:40.722 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:37:40.722 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.dfvvkw{-1:14484} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/3548717247,v1:172.21.15.107:6827/3548717247] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:37:40.722 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi157.ufywse{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.157:6826/4072444199,v1:172.21.15.157:6827/4072444199] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:37:40.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.727+0000 7fc90acfb700 1 -- 172.21.15.107:0/3429422921 >> [v2:172.21.15.107:6800/2362628021,v1:172.21.15.107:6801/2362628021] conn(0x7fc8f406e620 msgr2=0x7fc8f4070ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:40.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.727+0000 7fc90acfb700 1 --2- 172.21.15.107:0/3429422921 >> [v2:172.21.15.107:6800/2362628021,v1:172.21.15.107:6801/2362628021] conn(0x7fc8f406e620 0x7fc8f4070ad0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc8f0004f40 tx=0x7fc8f000b040 comp rx=0 tx=0).stop 2024-04-22T23:37:40.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.727+0000 7fc90acfb700 1 -- 172.21.15.107:0/3429422921 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9040fce40 msgr2=0x7fc90410d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:40.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.727+0000 7fc90acfb700 1 --2- 172.21.15.107:0/3429422921 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9040fce40 0x7fc90410d9a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8000c00 tx=0x7fc8f8009e30 comp rx=0 tx=0).stop 2024-04-22T23:37:40.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.727+0000 7fc90acfb700 1 -- 172.21.15.107:0/3429422921 shutdown_connections 2024-04-22T23:37:40.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.727+0000 7fc90acfb700 1 --2- 172.21.15.107:0/3429422921 >> [v2:172.21.15.107:6800/2362628021,v1:172.21.15.107:6801/2362628021] conn(0x7fc8f406e620 0x7fc8f4070ad0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:40.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.727+0000 7fc90acfb700 1 --2- 172.21.15.107:0/3429422921 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9040fce40 0x7fc90410d9a0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:40.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.727+0000 7fc90acfb700 1 --2- 172.21.15.107:0/3429422921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9040fe040 0x7fc90410dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:40.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.727+0000 7fc90acfb700 1 -- 172.21.15.107:0/3429422921 >> 172.21.15.107:0/3429422921 conn(0x7fc9040f8570 msgr2=0x7fc9040f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:40.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.727+0000 7fc90acfb700 1 -- 172.21.15.107:0/3429422921 shutdown_connections 2024-04-22T23:37:40.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.727+0000 7fc90acfb700 1 -- 172.21.15.107:0/3429422921 wait complete. 2024-04-22T23:37:40.723 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 19 2024-04-22T23:37:40.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.829+0000 7fb64424d700 1 -- 172.21.15.107:0/3293395051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb63c0fbcf0 msgr2=0x7fb63c0fc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:40.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.829+0000 7fb64424d700 1 --2- 172.21.15.107:0/3293395051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb63c0fbcf0 0x7fb63c0fc100 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb6380049c0 tx=0x7fb638009e30 comp rx=0 tx=0).stop 2024-04-22T23:37:40.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.829+0000 7fb64424d700 1 -- 172.21.15.107:0/3293395051 shutdown_connections 2024-04-22T23:37:40.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.829+0000 7fb64424d700 1 --2- 172.21.15.107:0/3293395051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb63c0fcfa0 0x7fb63c0fd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:40.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.829+0000 7fb64424d700 1 --2- 172.21.15.107:0/3293395051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb63c0fbcf0 0x7fb63c0fc100 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:40.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.829+0000 7fb64424d700 1 -- 172.21.15.107:0/3293395051 >> 172.21.15.107:0/3293395051 conn(0x7fb63c0f7450 msgr2=0x7fb63c0f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:40.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.829+0000 7fb64424d700 1 -- 172.21.15.107:0/3293395051 shutdown_connections 2024-04-22T23:37:40.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.829+0000 7fb64424d700 1 -- 172.21.15.107:0/3293395051 wait complete. 2024-04-22T23:37:40.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.829+0000 7fb64424d700 1 Processor -- start 2024-04-22T23:37:40.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.829+0000 7fb64424d700 1 -- start start 2024-04-22T23:37:40.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.829+0000 7fb64424d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb63c0fbcf0 0x7fb63c10ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:40.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.829+0000 7fb64424d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb63c0fcfa0 0x7fb63c10bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:40.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.830+0000 7fb64424d700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63c10c590 con 0x7fb63c0fbcf0 2024-04-22T23:37:40.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.830+0000 7fb64424d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63c10c700 con 0x7fb63c0fcfa0 2024-04-22T23:37:40.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.830+0000 7fb641fe9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb63c0fbcf0 0x7fb63c10ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:40.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.830+0000 7fb641fe9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb63c0fbcf0 0x7fb63c10ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53230/0 (socket says 172.21.15.107:53230) 2024-04-22T23:37:40.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.830+0000 7fb641fe9700 1 -- 172.21.15.107:0/2823349037 learned_addr learned my addr 172.21.15.107:0/2823349037 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:37:40.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.830+0000 7fb6417e8700 1 --2- 172.21.15.107:0/2823349037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb63c0fcfa0 0x7fb63c10bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:40.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.830+0000 7fb641fe9700 1 -- 172.21.15.107:0/2823349037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb63c0fcfa0 msgr2=0x7fb63c10bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:40.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.830+0000 7fb641fe9700 1 --2- 172.21.15.107:0/2823349037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb63c0fcfa0 0x7fb63c10bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:40.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.830+0000 7fb641fe9700 1 -- 172.21.15.107:0/2823349037 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb638005040 con 0x7fb63c0fbcf0 2024-04-22T23:37:40.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.830+0000 7fb641fe9700 1 --2- 172.21.15.107:0/2823349037 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb63c0fbcf0 0x7fb63c10ba50 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb638009370 tx=0x7fb638009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:40.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.830+0000 7fb6327fc700 1 -- 172.21.15.107:0/2823349037 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb638009af0 con 0x7fb63c0fbcf0 2024-04-22T23:37:40.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.830+0000 7fb6327fc700 1 -- 172.21.15.107:0/2823349037 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb638009620 con 0x7fb63c0fbcf0 2024-04-22T23:37:40.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.830+0000 7fb64424d700 1 -- 172.21.15.107:0/2823349037 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb63c111100 con 0x7fb63c0fbcf0 2024-04-22T23:37:40.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.830+0000 7fb6327fc700 1 -- 172.21.15.107:0/2823349037 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb63801b670 con 0x7fb63c0fbcf0 2024-04-22T23:37:40.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.830+0000 7fb64424d700 1 -- 172.21.15.107:0/2823349037 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb63c1116c0 con 0x7fb63c0fbcf0 2024-04-22T23:37:40.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.831+0000 7fb64424d700 1 -- 172.21.15.107:0/2823349037 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb63c04c710 con 0x7fb63c0fbcf0 2024-04-22T23:37:40.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.832+0000 7fb6327fc700 1 -- 172.21.15.107:0/2823349037 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 30) v1 ==== 92571+0+0 (secure 0 0 0) 0x7fb638019040 con 0x7fb63c0fbcf0 2024-04-22T23:37:40.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.832+0000 7fb6327fc700 1 --2- 172.21.15.107:0/2823349037 >> [v2:172.21.15.107:6800/2362628021,v1:172.21.15.107:6801/2362628021] conn(0x7fb62806e630 0x7fb628070ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:40.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.833+0000 7fb6327fc700 1 -- 172.21.15.107:0/2823349037 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6152+0+0 (secure 0 0 0) 0x7fb63808f580 con 0x7fb63c0fbcf0 2024-04-22T23:37:40.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.833+0000 7fb6417e8700 1 --2- 172.21.15.107:0/2823349037 >> [v2:172.21.15.107:6800/2362628021,v1:172.21.15.107:6801/2362628021] conn(0x7fb62806e630 0x7fb628070ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:40.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.833+0000 7fb6417e8700 1 --2- 172.21.15.107:0/2823349037 >> [v2:172.21.15.107:6800/2362628021,v1:172.21.15.107:6801/2362628021] conn(0x7fb62806e630 0x7fb628070ae0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb62c009b80 tx=0x7fb62c011040 comp rx=0 tx=0).ready entity=mgr.14778 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:40.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.837+0000 7fb6327fc700 1 -- 172.21.15.107:0/2823349037 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mgrmap(e 31) v1 ==== 92571+0+0 (secure 0 0 0) 0x7fb638009790 con 0x7fb63c0fbcf0 2024-04-22T23:37:40.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:40.843+0000 7fb6327fc700 1 -- 172.21.15.107:0/2823349037 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187010 (secure 0 0 0) 0x7fb638073cc0 con 0x7fb63c0fbcf0 2024-04-22T23:37:40.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:40 smithi107 ceph-mon[26355]: from='client.14844 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:37:40.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:40 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/2588259690' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:37:40.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:40 smithi107 ceph-mon[26355]: Standby manager daemon smithi157.tuylis restarted 2024-04-22T23:37:40.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:40 smithi107 ceph-mon[26355]: Standby manager daemon smithi157.tuylis started 2024-04-22T23:37:40.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:40 smithi107 ceph-mon[26355]: from='mgr.? 172.21.15.157:0/4232616347' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi157.tuylis/crt"}]: dispatch 2024-04-22T23:37:40.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:40 smithi107 ceph-mon[26355]: from='mgr.? 172.21.15.157:0/4232616347' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T23:37:40.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:40 smithi107 ceph-mon[26355]: from='mgr.? 172.21.15.157:0/4232616347' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi157.tuylis/key"}]: dispatch 2024-04-22T23:37:40.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:40 smithi107 ceph-mon[26355]: from='mgr.? 172.21.15.157:0/4232616347' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T23:37:40.984 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:40 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/3429422921' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:37:41.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.073+0000 7fb64424d700 1 -- 172.21.15.107:0/2823349037 --> [v2:172.21.15.107:6800/2362628021,v1:172.21.15.107:6801/2362628021] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb63c101400 con 0x7fb62806e630 2024-04-22T23:37:41.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.074+0000 7fb6327fc700 1 -- 172.21.15.107:0/2823349037 <== mgr.14778 v2:172.21.15.107:6800/2362628021 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fb63c101400 con 0x7fb62806e630 2024-04-22T23:37:41.068 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-22T23:37:41.068 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c", 2024-04-22T23:37:41.068 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-04-22T23:37:41.068 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-22T23:37:41.068 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [ 2024-04-22T23:37:41.068 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr" 2024-04-22T23:37:41.068 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-22T23:37:41.068 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "2/21 daemons upgraded", 2024-04-22T23:37:41.068 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Currently upgrading mgr daemons", 2024-04-22T23:37:41.069 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-04-22T23:37:41.069 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-22T23:37:41.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.076+0000 7fb64424d700 1 -- 172.21.15.107:0/2823349037 >> [v2:172.21.15.107:6800/2362628021,v1:172.21.15.107:6801/2362628021] conn(0x7fb62806e630 msgr2=0x7fb628070ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:41.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.076+0000 7fb64424d700 1 --2- 172.21.15.107:0/2823349037 >> [v2:172.21.15.107:6800/2362628021,v1:172.21.15.107:6801/2362628021] conn(0x7fb62806e630 0x7fb628070ae0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb62c009b80 tx=0x7fb62c011040 comp rx=0 tx=0).stop 2024-04-22T23:37:41.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.076+0000 7fb64424d700 1 -- 172.21.15.107:0/2823349037 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb63c0fbcf0 msgr2=0x7fb63c10ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:41.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.076+0000 7fb64424d700 1 --2- 172.21.15.107:0/2823349037 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb63c0fbcf0 0x7fb63c10ba50 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb638009370 tx=0x7fb638009e30 comp rx=0 tx=0).stop 2024-04-22T23:37:41.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.076+0000 7fb64424d700 1 -- 172.21.15.107:0/2823349037 shutdown_connections 2024-04-22T23:37:41.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.076+0000 7fb64424d700 1 --2- 172.21.15.107:0/2823349037 >> [v2:172.21.15.107:6800/2362628021,v1:172.21.15.107:6801/2362628021] conn(0x7fb62806e630 0x7fb628070ae0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:41.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.076+0000 7fb64424d700 1 --2- 172.21.15.107:0/2823349037 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb63c0fbcf0 0x7fb63c10ba50 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:41.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.076+0000 7fb64424d700 1 --2- 172.21.15.107:0/2823349037 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb63c0fcfa0 0x7fb63c10bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:41.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.076+0000 7fb64424d700 1 -- 172.21.15.107:0/2823349037 >> 172.21.15.107:0/2823349037 conn(0x7fb63c0f7450 msgr2=0x7fb63c0f8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:41.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.077+0000 7fb64424d700 1 -- 172.21.15.107:0/2823349037 shutdown_connections 2024-04-22T23:37:41.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.077+0000 7fb64424d700 1 -- 172.21.15.107:0/2823349037 wait complete. 2024-04-22T23:37:41.115 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:40 smithi157 ceph-mon[33877]: from='client.14844 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:37:41.116 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:40 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/2588259690' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:37:41.116 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:40 smithi157 ceph-mon[33877]: Standby manager daemon smithi157.tuylis restarted 2024-04-22T23:37:41.116 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:40 smithi157 ceph-mon[33877]: Standby manager daemon smithi157.tuylis started 2024-04-22T23:37:41.116 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:40 smithi157 ceph-mon[33877]: from='mgr.? 172.21.15.157:0/4232616347' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi157.tuylis/crt"}]: dispatch 2024-04-22T23:37:41.116 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:40 smithi157 ceph-mon[33877]: from='mgr.? 172.21.15.157:0/4232616347' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T23:37:41.116 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:40 smithi157 ceph-mon[33877]: from='mgr.? 172.21.15.157:0/4232616347' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi157.tuylis/key"}]: dispatch 2024-04-22T23:37:41.116 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:40 smithi157 ceph-mon[33877]: from='mgr.? 172.21.15.157:0/4232616347' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T23:37:41.116 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:40 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/3429422921' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:37:41.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.177+0000 7f3e2296b700 1 -- 172.21.15.107:0/1660136159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e1c100320 msgr2=0x7f3e1c100790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:41.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.177+0000 7f3e2296b700 1 --2- 172.21.15.107:0/1660136159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e1c100320 0x7f3e1c100790 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f3e10004940 tx=0x7f3e10009e30 comp rx=0 tx=0).stop 2024-04-22T23:37:41.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.177+0000 7f3e2296b700 1 -- 172.21.15.107:0/1660136159 shutdown_connections 2024-04-22T23:37:41.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.177+0000 7f3e2296b700 1 --2- 172.21.15.107:0/1660136159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e1c100320 0x7f3e1c100790 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:41.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.177+0000 7f3e2296b700 1 --2- 172.21.15.107:0/1660136159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e1c0ff070 0x7f3e1c0ff480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:41.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.177+0000 7f3e2296b700 1 -- 172.21.15.107:0/1660136159 >> 172.21.15.107:0/1660136159 conn(0x7f3e1c0fa950 msgr2=0x7f3e1c0fcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:41.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.177+0000 7f3e2296b700 1 -- 172.21.15.107:0/1660136159 shutdown_connections 2024-04-22T23:37:41.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.177+0000 7f3e2296b700 1 -- 172.21.15.107:0/1660136159 wait complete. 2024-04-22T23:37:41.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.178+0000 7f3e2296b700 1 Processor -- start 2024-04-22T23:37:41.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.178+0000 7f3e2296b700 1 -- start start 2024-04-22T23:37:41.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.178+0000 7f3e2296b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e1c0ff070 0x7f3e1c10fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:41.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.178+0000 7f3e2296b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e1c100320 0x7f3e1c110160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:41.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.178+0000 7f3e2296b700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e1c110760 con 0x7f3e1c0ff070 2024-04-22T23:37:41.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.178+0000 7f3e2296b700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e1c110fb0 con 0x7f3e1c100320 2024-04-22T23:37:41.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.178+0000 7f3e1bfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e1c0ff070 0x7f3e1c10fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:41.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.178+0000 7f3e1bfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e1c0ff070 0x7f3e1c10fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53248/0 (socket says 172.21.15.107:53248) 2024-04-22T23:37:41.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.178+0000 7f3e1bfff700 1 -- 172.21.15.107:0/2093238379 learned_addr learned my addr 172.21.15.107:0/2093238379 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:37:41.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.179+0000 7f3e1b7fe700 1 --2- 172.21.15.107:0/2093238379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e1c100320 0x7f3e1c110160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:41.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.179+0000 7f3e1bfff700 1 -- 172.21.15.107:0/2093238379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e1c100320 msgr2=0x7f3e1c110160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:41.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.179+0000 7f3e1bfff700 1 --2- 172.21.15.107:0/2093238379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e1c100320 0x7f3e1c110160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:41.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.179+0000 7f3e1bfff700 1 -- 172.21.15.107:0/2093238379 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e10005040 con 0x7f3e1c0ff070 2024-04-22T23:37:41.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.179+0000 7f3e1bfff700 1 --2- 172.21.15.107:0/2093238379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e1c0ff070 0x7f3e1c10fc20 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c00c830 tx=0x7f3e0c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:41.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.179+0000 7f3e18ff9700 1 -- 172.21.15.107:0/2093238379 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e0c0106f0 con 0x7f3e1c0ff070 2024-04-22T23:37:41.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.179+0000 7f3e18ff9700 1 -- 172.21.15.107:0/2093238379 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3e0c010d20 con 0x7f3e1c0ff070 2024-04-22T23:37:41.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.179+0000 7f3e2296b700 1 -- 172.21.15.107:0/2093238379 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e1c111290 con 0x7f3e1c0ff070 2024-04-22T23:37:41.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.179+0000 7f3e18ff9700 1 -- 172.21.15.107:0/2093238379 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e0c0113a0 con 0x7f3e1c0ff070 2024-04-22T23:37:41.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.179+0000 7f3e2296b700 1 -- 172.21.15.107:0/2093238379 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e1c1117d0 con 0x7f3e1c0ff070 2024-04-22T23:37:41.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.180+0000 7f3e2296b700 1 -- 172.21.15.107:0/2093238379 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e1c05f5f0 con 0x7f3e1c0ff070 2024-04-22T23:37:41.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.180+0000 7f3e18ff9700 1 -- 172.21.15.107:0/2093238379 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 31) v1 ==== 92571+0+0 (secure 0 0 0) 0x7f3e0c012400 con 0x7f3e1c0ff070 2024-04-22T23:37:41.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.181+0000 7f3e18ff9700 1 --2- 172.21.15.107:0/2093238379 >> [v2:172.21.15.107:6800/2362628021,v1:172.21.15.107:6801/2362628021] conn(0x7f3e0406e620 0x7f3e04070ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:37:41.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.181+0000 7f3e18ff9700 1 -- 172.21.15.107:0/2093238379 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6152+0+0 (secure 0 0 0) 0x7f3e0c08e020 con 0x7f3e1c0ff070 2024-04-22T23:37:41.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.181+0000 7f3e1b7fe700 1 --2- 172.21.15.107:0/2093238379 >> [v2:172.21.15.107:6800/2362628021,v1:172.21.15.107:6801/2362628021] conn(0x7f3e0406e620 0x7f3e04070ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:37:41.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.181+0000 7f3e1b7fe700 1 --2- 172.21.15.107:0/2093238379 >> [v2:172.21.15.107:6800/2362628021,v1:172.21.15.107:6801/2362628021] conn(0x7f3e0406e620 0x7f3e04070ad0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3e1000b970 tx=0x7f3e10009e30 comp rx=0 tx=0).ready entity=mgr.14778 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:37:41.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.185+0000 7f3e18ff9700 1 -- 172.21.15.107:0/2093238379 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+187010 (secure 0 0 0) 0x7f3e0c05b0c0 con 0x7f3e1c0ff070 2024-04-22T23:37:41.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.470+0000 7f3e2296b700 1 -- 172.21.15.107:0/2093238379 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f3e1c107af0 con 0x7f3e1c0ff070 2024-04-22T23:37:41.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.471+0000 7f3e18ff9700 1 -- 172.21.15.107:0/2093238379 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f3e0c055e60 con 0x7f3e1c0ff070 2024-04-22T23:37:41.465 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_OK 2024-04-22T23:37:41.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.473+0000 7f3e2296b700 1 -- 172.21.15.107:0/2093238379 >> [v2:172.21.15.107:6800/2362628021,v1:172.21.15.107:6801/2362628021] conn(0x7f3e0406e620 msgr2=0x7f3e04070ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:41.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.473+0000 7f3e2296b700 1 --2- 172.21.15.107:0/2093238379 >> [v2:172.21.15.107:6800/2362628021,v1:172.21.15.107:6801/2362628021] conn(0x7f3e0406e620 0x7f3e04070ad0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3e1000b970 tx=0x7f3e10009e30 comp rx=0 tx=0).stop 2024-04-22T23:37:41.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.473+0000 7f3e2296b700 1 -- 172.21.15.107:0/2093238379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e1c0ff070 msgr2=0x7f3e1c10fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:37:41.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.473+0000 7f3e2296b700 1 --2- 172.21.15.107:0/2093238379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e1c0ff070 0x7f3e1c10fc20 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c00c830 tx=0x7f3e0c004470 comp rx=0 tx=0).stop 2024-04-22T23:37:41.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.473+0000 7f3e2296b700 1 -- 172.21.15.107:0/2093238379 shutdown_connections 2024-04-22T23:37:41.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.473+0000 7f3e2296b700 1 --2- 172.21.15.107:0/2093238379 >> [v2:172.21.15.107:6800/2362628021,v1:172.21.15.107:6801/2362628021] conn(0x7f3e0406e620 0x7f3e04070ad0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:41.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.473+0000 7f3e2296b700 1 --2- 172.21.15.107:0/2093238379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e1c0ff070 0x7f3e1c10fc20 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:41.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.473+0000 7f3e2296b700 1 --2- 172.21.15.107:0/2093238379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e1c100320 0x7f3e1c110160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:37:41.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.473+0000 7f3e2296b700 1 -- 172.21.15.107:0/2093238379 >> 172.21.15.107:0/2093238379 conn(0x7f3e1c0fa950 msgr2=0x7f3e1c0fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:37:41.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.473+0000 7f3e2296b700 1 -- 172.21.15.107:0/2093238379 shutdown_connections 2024-04-22T23:37:41.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:37:41.473+0000 7f3e2296b700 1 -- 172.21.15.107:0/2093238379 wait complete. 2024-04-22T23:37:42.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:41 smithi157 ceph-mon[33877]: mgrmap e31: smithi107.uipxvm(active, since 29s), standbys: smithi157.tuylis 2024-04-22T23:37:42.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:41 smithi157 ceph-mon[33877]: pgmap v18: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:37:42.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:41 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:37:42.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:41 smithi157 ceph-mon[33877]: from='client.14856 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:37:42.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:41 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:42.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:41 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:42.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:41 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:37:42.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:41 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:37:42.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:41 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:42.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:41 smithi157 ceph-mon[33877]: from='client.? 172.21.15.107:0/2093238379' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:37:42.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:41 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:37:42.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:41 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:37:42.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:41 smithi157 ceph-mon[33877]: Upgrade: Setting container_image for all mgr 2024-04-22T23:37:42.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:41 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:42.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:41 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi107.uipxvm"}]: dispatch 2024-04-22T23:37:42.133 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:41 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi107.uipxvm"}]': finished 2024-04-22T23:37:42.133 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:41 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi157.tuylis"}]: dispatch 2024-04-22T23:37:42.133 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:41 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi157.tuylis"}]': finished 2024-04-22T23:37:42.133 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:41 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-22T23:37:42.192 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:41 smithi107 ceph-mon[26355]: mgrmap e31: smithi107.uipxvm(active, since 29s), standbys: smithi157.tuylis 2024-04-22T23:37:42.192 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:41 smithi107 ceph-mon[26355]: pgmap v18: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:37:42.192 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:41 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:37:42.192 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:41 smithi107 ceph-mon[26355]: from='client.14856 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:37:42.193 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:41 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:42.193 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:41 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:42.193 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:41 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:37:42.193 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:41 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:37:42.193 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:41 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:42.193 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:41 smithi107 ceph-mon[26355]: from='client.? 172.21.15.107:0/2093238379' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:37:42.193 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:41 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:37:42.193 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:41 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:37:42.193 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:41 smithi107 ceph-mon[26355]: Upgrade: Setting container_image for all mgr 2024-04-22T23:37:42.193 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:41 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:42.193 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:41 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi107.uipxvm"}]: dispatch 2024-04-22T23:37:42.193 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:41 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi107.uipxvm"}]': finished 2024-04-22T23:37:42.193 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:41 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi157.tuylis"}]: dispatch 2024-04-22T23:37:42.193 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:41 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi157.tuylis"}]': finished 2024-04-22T23:37:42.194 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:41 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-22T23:37:44.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:44 smithi107 ceph-mon[26355]: pgmap v19: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:37:44.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:44 smithi107 ceph-mon[26355]: Upgrade: Updating mon.smithi107 2024-04-22T23:37:44.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:44 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:44.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:44 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T23:37:44.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:44 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-22T23:37:44.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:44 smithi107 ceph-mon[26355]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:37:44.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:44 smithi107 ceph-mon[26355]: Deploying daemon mon.smithi107 on smithi107 2024-04-22T23:37:44.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:44 smithi157 ceph-mon[33877]: pgmap v19: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:37:44.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:44 smithi157 ceph-mon[33877]: Upgrade: Updating mon.smithi107 2024-04-22T23:37:44.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:44 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:44.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:44 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T23:37:44.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:44 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-22T23:37:44.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:44 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:37:44.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:44 smithi157 ceph-mon[33877]: Deploying daemon mon.smithi107 on smithi107 2024-04-22T23:37:45.554 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:45 smithi107 systemd[1]: Stopping Ceph mon.smithi107 for 8252f6f2-00ff-11ef-bc93-c7b262605968... 2024-04-22T23:37:45.871 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:45 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-mon-smithi107[26351]: 2024-04-22T23:37:45.559+0000 7fb41f26d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi107 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T23:37:45.871 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:45 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-mon-smithi107[26351]: 2024-04-22T23:37:45.559+0000 7fb41f26d700 -1 mon.smithi107@0(leader) e2 *** Got Signal Terminated *** 2024-04-22T23:37:45.871 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:45 smithi107 podman[90007]: 2024-04-22 23:37:45.610985612 +0000 UTC m=+0.159594885 container died 8314962fa1b933e18567c31af8eaa108e6792030a367debbd28d139d120a8450 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-mon-smithi107, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=quincy-70d0cb4, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T23:37:46.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:46 smithi107 podman[90007]: 2024-04-22 23:37:46.085742757 +0000 UTC m=+0.634352065 container cleanup 8314962fa1b933e18567c31af8eaa108e6792030a367debbd28d139d120a8450 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-mon-smithi107, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=quincy-70d0cb4, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-22T23:37:46.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:46 smithi107 bash[90007]: ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-mon-smithi107 2024-04-22T23:37:46.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:46 smithi107 podman[90032]: 2024-04-22 23:37:46.421352697 +0000 UTC m=+0.807408095 container remove 8314962fa1b933e18567c31af8eaa108e6792030a367debbd28d139d120a8450 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-mon-smithi107, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=quincy-70d0cb4, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-22T23:37:46.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:46 smithi107 systemd[1]: ceph-8252f6f2-00ff-11ef-bc93-c7b262605968@mon.smithi107.service: Deactivated successfully. 2024-04-22T23:37:46.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:46 smithi107 systemd[1]: Stopped Ceph mon.smithi107 for 8252f6f2-00ff-11ef-bc93-c7b262605968. 2024-04-22T23:37:46.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:46 smithi107 systemd[1]: ceph-8252f6f2-00ff-11ef-bc93-c7b262605968@mon.smithi107.service: Consumed 11.085s CPU time. 2024-04-22T23:37:47.007 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:46 smithi107 systemd[1]: Starting Ceph mon.smithi107 for 8252f6f2-00ff-11ef-bc93-c7b262605968... 2024-04-22T23:37:47.008 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:46 smithi107 podman[90160]: 2024-04-22 23:37:46.874447007 +0000 UTC m=+0.018362398 image pull a0434699c1d76af4eea1fc95ff0a22ea73d1e04e609464fbf658d6f39bc7f2fc quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c 2024-04-22T23:37:47.008 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 podman[90160]: 2024-04-22 23:37:47.014327678 +0000 UTC m=+0.158243060 container create bc72bbbe2c399f31aa0a3d630e519517ed8839fd6fe6e73777b79263f7812193 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-mon-smithi107, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True) 2024-04-22T23:37:47.258 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 podman[90160]: 2024-04-22 23:37:47.23411461 +0000 UTC m=+0.378029991 container init bc72bbbe2c399f31aa0a3d630e519517ed8839fd6fe6e73777b79263f7812193 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-mon-smithi107, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T23:37:47.258 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 podman[90160]: 2024-04-22 23:37:47.237522497 +0000 UTC m=+0.381437879 container start bc72bbbe2c399f31aa0a3d630e519517ed8839fd6fe6e73777b79263f7812193 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-mon-smithi107, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-22T23:37:47.258 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: set uid:gid to 167:167 (ceph:ceph) 2024-04-22T23:37:47.258 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev), process ceph-mon, pid 2 2024-04-22T23:37:47.259 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: pidfile_write: ignore empty --pid-file 2024-04-22T23:37:47.259 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: load: jerasure load: lrc 2024-04-22T23:37:47.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: RocksDB version: 7.9.2 2024-04-22T23:37:47.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Git sha 0 2024-04-22T23:37:47.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Compile date 2024-04-22 21:33:31 2024-04-22T23:37:47.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: DB SUMMARY 2024-04-22T23:37:47.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: DB Session ID: EHB80FKQXW2I8LZTCKLF 2024-04-22T23:37:47.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: CURRENT file: CURRENT 2024-04-22T23:37:47.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: IDENTITY file: IDENTITY 2024-04-22T23:37:47.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: MANIFEST file: MANIFEST-000015 size: 972 Bytes 2024-04-22T23:37:47.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi107/store.db dir, Total Num: 1, files: 000030.sst 2024-04-22T23:37:47.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi107/store.db: 000028.log size: 2296821 ; 2024-04-22T23:37:47.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.error_if_exists: 0 2024-04-22T23:37:47.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.create_if_missing: 0 2024-04-22T23:37:47.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.paranoid_checks: 1 2024-04-22T23:37:47.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-22T23:37:47.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-22T23:37:47.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-22T23:37:47.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.env: 0x55717b3a0cc0 2024-04-22T23:37:47.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.fs: PosixFileSystem 2024-04-22T23:37:47.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.info_log: 0x55717d43faa0 2024-04-22T23:37:47.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.max_file_opening_threads: 16 2024-04-22T23:37:47.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.statistics: (nil) 2024-04-22T23:37:47.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.use_fsync: 0 2024-04-22T23:37:47.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.max_log_file_size: 0 2024-04-22T23:37:47.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-22T23:37:47.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-22T23:37:47.736 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.keep_log_file_num: 1000 2024-04-22T23:37:47.736 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.recycle_log_file_num: 0 2024-04-22T23:37:47.736 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.allow_fallocate: 1 2024-04-22T23:37:47.736 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.allow_mmap_reads: 0 2024-04-22T23:37:47.736 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.allow_mmap_writes: 0 2024-04-22T23:37:47.736 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.use_direct_reads: 0 2024-04-22T23:37:47.736 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-22T23:37:47.736 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.create_missing_column_families: 0 2024-04-22T23:37:47.736 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.db_log_dir: 2024-04-22T23:37:47.736 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.wal_dir: 2024-04-22T23:37:47.736 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-22T23:37:47.736 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-22T23:37:47.736 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-22T23:37:47.736 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-22T23:37:47.738 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-22T23:37:47.738 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-22T23:37:47.738 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.advise_random_on_open: 1 2024-04-22T23:37:47.738 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.db_write_buffer_size: 0 2024-04-22T23:37:47.738 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.write_buffer_manager: 0x55717d3d30e0 2024-04-22T23:37:47.738 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-22T23:37:47.738 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-22T23:37:47.739 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-22T23:37:47.739 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.rate_limiter: (nil) 2024-04-22T23:37:47.739 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-22T23:37:47.739 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.wal_recovery_mode: 2 2024-04-22T23:37:47.739 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.enable_thread_tracking: 0 2024-04-22T23:37:47.739 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.enable_pipelined_write: 0 2024-04-22T23:37:47.739 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.unordered_write: 0 2024-04-22T23:37:47.739 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-22T23:37:47.739 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-22T23:37:47.739 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-22T23:37:47.739 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-22T23:37:47.739 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.row_cache: None 2024-04-22T23:37:47.739 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.wal_filter: None 2024-04-22T23:37:47.739 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-22T23:37:47.740 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.allow_ingest_behind: 0 2024-04-22T23:37:47.740 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.two_write_queues: 0 2024-04-22T23:37:47.740 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.manual_wal_flush: 0 2024-04-22T23:37:47.740 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.wal_compression: 0 2024-04-22T23:37:47.740 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.atomic_flush: 0 2024-04-22T23:37:47.740 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-22T23:37:47.740 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-22T23:37:47.740 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-22T23:37:47.740 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.log_readahead_size: 0 2024-04-22T23:37:47.740 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-22T23:37:47.740 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.best_efforts_recovery: 0 2024-04-22T23:37:47.740 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-22T23:37:47.740 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-22T23:37:47.740 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.allow_data_in_errors: 0 2024-04-22T23:37:47.741 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.db_host_id: __hostname__ 2024-04-22T23:37:47.741 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-22T23:37:47.741 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.max_background_jobs: 2 2024-04-22T23:37:47.741 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.max_background_compactions: -1 2024-04-22T23:37:47.741 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.max_subcompactions: 1 2024-04-22T23:37:47.741 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-22T23:37:47.741 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-22T23:37:47.741 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-22T23:37:47.741 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.max_total_wal_size: 0 2024-04-22T23:37:47.741 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-22T23:37:47.741 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-22T23:37:47.741 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-22T23:37:47.741 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-22T23:37:47.741 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.max_open_files: -1 2024-04-22T23:37:47.741 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.bytes_per_sync: 0 2024-04-22T23:37:47.742 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-22T23:37:47.742 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-22T23:37:47.742 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.compaction_readahead_size: 0 2024-04-22T23:37:47.742 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.max_background_flushes: -1 2024-04-22T23:37:47.742 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Compression algorithms supported: 2024-04-22T23:37:47.742 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-22T23:37:47.742 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: kZSTD supported: 0 2024-04-22T23:37:47.742 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: kXpressCompression supported: 0 2024-04-22T23:37:47.742 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: kLZ4HCCompression supported: 1 2024-04-22T23:37:47.742 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: kZlibCompression supported: 1 2024-04-22T23:37:47.742 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: kSnappyCompression supported: 1 2024-04-22T23:37:47.742 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: kLZ4Compression supported: 1 2024-04-22T23:37:47.742 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: kBZip2Compression supported: 0 2024-04-22T23:37:47.743 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-22T23:37:47.743 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-22T23:37:47.743 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi107/store.db/MANIFEST-000015 2024-04-22T23:37:47.743 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-22T23:37:47.743 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-22T23:37:47.743 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.merge_operator: 2024-04-22T23:37:47.743 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.compaction_filter: None 2024-04-22T23:37:47.743 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.compaction_filter_factory: None 2024-04-22T23:37:47.743 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.sst_partitioner_factory: None 2024-04-22T23:37:47.743 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-22T23:37:47.743 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-22T23:37:47.744 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55717d43fc80) 2024-04-22T23:37:47.744 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: cache_index_and_filter_blocks: 1 2024-04-22T23:37:47.744 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-22T23:37:47.744 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-22T23:37:47.744 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: pin_top_level_index_and_filter: 1 2024-04-22T23:37:47.744 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: index_type: 0 2024-04-22T23:37:47.744 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: data_block_index_type: 0 2024-04-22T23:37:47.744 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: index_shortening: 1 2024-04-22T23:37:47.744 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-22T23:37:47.744 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: checksum: 4 2024-04-22T23:37:47.744 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: no_block_cache: 0 2024-04-22T23:37:47.744 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: block_cache: 0x55717d4051f0 2024-04-22T23:37:47.744 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: block_cache_name: BinnedLRUCache 2024-04-22T23:37:47.744 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: block_cache_options: 2024-04-22T23:37:47.745 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: capacity : 536870912 2024-04-22T23:37:47.745 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: num_shard_bits : 4 2024-04-22T23:37:47.745 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: strict_capacity_limit : 0 2024-04-22T23:37:47.745 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: high_pri_pool_ratio: 0.000 2024-04-22T23:37:47.745 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: block_cache_compressed: (nil) 2024-04-22T23:37:47.745 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: persistent_cache: (nil) 2024-04-22T23:37:47.745 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: block_size: 4096 2024-04-22T23:37:47.745 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: block_size_deviation: 10 2024-04-22T23:37:47.745 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: block_restart_interval: 16 2024-04-22T23:37:47.745 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: index_block_restart_interval: 1 2024-04-22T23:37:47.745 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: metadata_block_size: 4096 2024-04-22T23:37:47.745 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: partition_filters: 0 2024-04-22T23:37:47.745 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: use_delta_encoding: 1 2024-04-22T23:37:47.745 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: filter_policy: bloomfilter 2024-04-22T23:37:47.745 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: whole_key_filtering: 1 2024-04-22T23:37:47.745 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: verify_compression: 0 2024-04-22T23:37:47.746 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: read_amp_bytes_per_bit: 0 2024-04-22T23:37:47.746 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: format_version: 5 2024-04-22T23:37:47.746 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: enable_index_compression: 1 2024-04-22T23:37:47.746 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: block_align: 0 2024-04-22T23:37:47.746 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: max_auto_readahead_size: 262144 2024-04-22T23:37:47.746 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: prepopulate_block_cache: 0 2024-04-22T23:37:47.746 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: initial_auto_readahead_size: 8192 2024-04-22T23:37:47.746 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: num_file_reads_for_auto_readahead: 2 2024-04-22T23:37:47.746 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.write_buffer_size: 33554432 2024-04-22T23:37:47.746 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.max_write_buffer_number: 2 2024-04-22T23:37:47.746 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.compression: NoCompression 2024-04-22T23:37:47.746 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.bottommost_compression: Disabled 2024-04-22T23:37:47.746 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.prefix_extractor: nullptr 2024-04-22T23:37:47.746 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-22T23:37:47.746 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.num_levels: 7 2024-04-22T23:37:47.747 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-22T23:37:47.747 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-22T23:37:47.747 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-22T23:37:47.747 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-22T23:37:47.747 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-22T23:37:47.747 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-22T23:37:47.747 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-22T23:37:47.747 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-22T23:37:47.747 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-22T23:37:47.747 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-22T23:37:47.747 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-22T23:37:47.747 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-22T23:37:47.747 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-22T23:37:47.747 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.compression_opts.level: 32767 2024-04-22T23:37:47.747 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.compression_opts.strategy: 0 2024-04-22T23:37:47.748 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-22T23:37:47.748 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-22T23:37:47.748 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-22T23:37:47.748 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-22T23:37:47.748 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.compression_opts.enabled: false 2024-04-22T23:37:47.748 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-22T23:37:47.748 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-22T23:37:47.748 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-22T23:37:47.748 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-22T23:37:47.748 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.target_file_size_base: 67108864 2024-04-22T23:37:47.748 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-22T23:37:47.748 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-22T23:37:47.748 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-22T23:37:47.748 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-22T23:37:47.748 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-22T23:37:47.748 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-22T23:37:47.749 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-22T23:37:47.749 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-22T23:37:47.749 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-22T23:37:47.749 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-22T23:37:47.749 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-22T23:37:47.749 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-22T23:37:47.749 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-22T23:37:47.749 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-22T23:37:47.749 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.arena_block_size: 1048576 2024-04-22T23:37:47.749 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-22T23:37:47.749 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-22T23:37:47.749 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.disable_auto_compactions: 0 2024-04-22T23:37:47.749 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-22T23:37:47.749 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-22T23:37:47.749 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-22T23:37:47.749 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-22T23:37:47.750 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-22T23:37:47.750 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-22T23:37:47.750 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-22T23:37:47.750 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-22T23:37:47.750 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-22T23:37:47.750 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-22T23:37:47.750 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-22T23:37:47.750 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.inplace_update_support: 0 2024-04-22T23:37:47.750 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-22T23:37:47.750 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-22T23:37:47.750 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-22T23:37:47.750 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-22T23:37:47.750 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.bloom_locality: 0 2024-04-22T23:37:47.750 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.max_successive_merges: 0 2024-04-22T23:37:47.751 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-22T23:37:47.751 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.paranoid_file_checks: 0 2024-04-22T23:37:47.751 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.force_consistency_checks: 1 2024-04-22T23:37:47.751 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.report_bg_io_stats: 0 2024-04-22T23:37:47.751 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.ttl: 2592000 2024-04-22T23:37:47.751 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-22T23:37:47.751 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-22T23:37:47.751 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-22T23:37:47.751 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.enable_blob_files: false 2024-04-22T23:37:47.751 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.min_blob_size: 0 2024-04-22T23:37:47.751 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.blob_file_size: 268435456 2024-04-22T23:37:47.751 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-22T23:37:47.751 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-22T23:37:47.751 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-22T23:37:47.751 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-22T23:37:47.752 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-22T23:37:47.752 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.blob_file_starting_level: 0 2024-04-22T23:37:47.752 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-22T23:37:47.752 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 30.sst 2024-04-22T23:37:47.752 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-22T23:37:47.752 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi107/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 32, last_sequence is 10834, log_number is 28,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-22T23:37:47.752 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 28 2024-04-22T23:37:47.752 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: e7f29ee8-a57f-4b35-a184-082fab568ff3 2024-04-22T23:37:47.752 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713829067267130, "job": 1, "event": "recovery_started", "wal_files": [28]} 2024-04-22T23:37:47.752 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #28 mode 2 2024-04-22T23:37:47.752 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713829067273575, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 33, "file_size": 2122141, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 10835, "largest_seqno": 11641, "table_properties": {"data_size": 2117426, "index_size": 2647, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 1029, "raw_key_size": 9392, "raw_average_key_size": 24, "raw_value_size": 2108789, "raw_average_value_size": 5435, "num_data_blocks": 121, "num_entries": 388, "num_filter_entries": 388, "num_deletions": 2, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713829067, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "e7f29ee8-a57f-4b35-a184-082fab568ff3", "db_session_id": "EHB80FKQXW2I8LZTCKLF", "orig_file_number": 33, "seqno_to_time_mapping": "N/A"}} 2024-04-22T23:37:47.752 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713829067273646, "job": 1, "event": "recovery_finished"} 2024-04-22T23:37:47.752 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: [db/version_set.cc:5047] Creating manifest 35 2024-04-22T23:37:47.752 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-22T23:37:47.753 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi107/store.db/000028.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-22T23:37:47.753 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55717d51a000 2024-04-22T23:37:47.753 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: DB pointer 0x55717d504000 2024-04-22T23:37:47.753 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-22T23:37:47.753 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-22T23:37:47.753 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: ** DB Stats ** 2024-04-22T23:37:47.753 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-22T23:37:47.753 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-22T23:37:47.753 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-22T23:37:47.753 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-22T23:37:47.753 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-22T23:37:47.754 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-22T23:37:47.754 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-22T23:37:47.754 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: 2024-04-22T23:37:47.754 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: ** Compaction Stats [default] ** 2024-04-22T23:37:47.754 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-22T23:37:47.754 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-22T23:37:47.754 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: L0 1/0 2.02 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 622.1 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-04-22T23:37:47.754 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: L6 1/0 5.47 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-04-22T23:37:47.754 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Sum 2/0 7.50 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 622.1 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-04-22T23:37:47.754 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 622.1 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-04-22T23:37:47.754 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: 2024-04-22T23:37:47.754 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: ** Compaction Stats [default] ** 2024-04-22T23:37:47.754 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-22T23:37:47.755 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-22T23:37:47.755 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 622.1 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-04-22T23:37:47.755 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: 2024-04-22T23:37:47.755 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-22T23:37:47.755 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: 2024-04-22T23:37:47.755 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-22T23:37:47.755 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Flush(GB): cumulative 0.002, interval 0.002 2024-04-22T23:37:47.755 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-22T23:37:47.755 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-22T23:37:47.755 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-22T23:37:47.755 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-22T23:37:47.755 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Cumulative compaction: 0.00 GB write, 188.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-22T23:37:47.755 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Interval compaction: 0.00 GB write, 188.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-22T23:37:47.755 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-22T23:37:47.756 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Block cache BinnedLRUCache@0x55717d4051f0#2 capacity: 512.00 MB usage: 3.98 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-04-22T23:37:47.756 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,1.16 KB,0.000220537%) IndexBlock(1,2.83 KB,0.000539422%) Misc(1,0.00 KB,0%) 2024-04-22T23:37:47.756 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: 2024-04-22T23:37:47.756 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-22T23:37:47.756 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: starting mon.smithi107 rank 0 at public addrs [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] at bind addrs [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi107 fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 2024-04-22T23:37:47.756 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: mon.smithi107@-1(???) e2 preinit fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 2024-04-22T23:37:47.756 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: mon.smithi107@-1(???).mds e19 new map 2024-04-22T23:37:47.756 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: mon.smithi107@-1(???).mds e19 print_map 2024-04-22T23:37:47.756 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: e19 2024-04-22T23:37:47.756 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T23:37:47.756 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:37:47.756 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: legacy client fscid: 1 2024-04-22T23:37:47.756 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: 2024-04-22T23:37:47.756 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Filesystem 'cephfs' (1) 2024-04-22T23:37:47.756 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: fs_name cephfs 2024-04-22T23:37:47.757 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: epoch 19 2024-04-22T23:37:47.757 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-22T23:37:47.757 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: created 2024-04-22T23:32:20.146130+0000 2024-04-22T23:37:47.757 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: modified 2024-04-22T23:35:12.338678+0000 2024-04-22T23:37:47.757 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: tableserver 0 2024-04-22T23:37:47.757 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: root 0 2024-04-22T23:37:47.757 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: session_timeout 60 2024-04-22T23:37:47.757 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: session_autoclose 300 2024-04-22T23:37:47.757 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: max_file_size 1099511627776 2024-04-22T23:37:47.757 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: max_xattr_size 65536 2024-04-22T23:37:47.757 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: required_client_features {} 2024-04-22T23:37:47.757 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: last_failure 0 2024-04-22T23:37:47.757 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: last_failure_osd_epoch 46 2024-04-22T23:37:47.757 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:37:47.757 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: max_mds 2 2024-04-22T23:37:47.758 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: in 0,1 2024-04-22T23:37:47.758 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: up {0=14468,1=24265} 2024-04-22T23:37:47.758 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: failed 2024-04-22T23:37:47.758 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: damaged 2024-04-22T23:37:47.758 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: stopped 2024-04-22T23:37:47.758 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: data_pools [3] 2024-04-22T23:37:47.758 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: metadata_pool 2 2024-04-22T23:37:47.758 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: inline_data disabled 2024-04-22T23:37:47.758 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: balancer 2024-04-22T23:37:47.758 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: bal_rank_mask -1 2024-04-22T23:37:47.758 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: standby_count_wanted 1 2024-04-22T23:37:47.758 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: qdb_cluster leader: 0 members: 2024-04-22T23:37:47.758 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: [mds.cephfs.smithi107.ktuxjl{0:14468} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.107:6828/3335313253,v1:172.21.15.107:6829/3335313253] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:37:47.758 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: [mds.cephfs.smithi157.rcaris{1:24265} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.157:6824/2779251893,v1:172.21.15.157:6825/2779251893] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:37:47.759 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: 2024-04-22T23:37:47.759 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: 2024-04-22T23:37:47.759 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: Standby daemons: 2024-04-22T23:37:47.759 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: 2024-04-22T23:37:47.759 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: [mds.cephfs.smithi107.dfvvkw{-1:14484} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/3548717247,v1:172.21.15.107:6827/3548717247] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:37:47.759 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout: [mds.cephfs.smithi157.ufywse{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.157:6826/4072444199,v1:172.21.15.157:6827/4072444199] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:37:47.759 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: mon.smithi107@-1(???).osd e48 crush map has features 3314933000852226048, adjusting msgr requires 2024-04-22T23:37:47.759 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: mon.smithi107@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-04-22T23:37:47.759 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: mon.smithi107@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-04-22T23:37:47.759 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: mon.smithi107@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-04-22T23:37:47.759 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 ceph-mon[90179]: mon.smithi107@-1(???).paxosservice(auth 1..21) refresh upgraded, format 0 -> 3 2024-04-22T23:37:47.759 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 bash[90160]: bc72bbbe2c399f31aa0a3d630e519517ed8839fd6fe6e73777b79263f7812193 2024-04-22T23:37:47.759 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:47 smithi107 systemd[1]: Started Ceph mon.smithi107 for 8252f6f2-00ff-11ef-bc93-c7b262605968. 2024-04-22T23:37:48.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:48 smithi107 ceph-mon[90179]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-04-22T23:37:48.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:48 smithi107 ceph-mon[90179]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mon metadata", "id": "smithi157"}]: dispatch 2024-04-22T23:37:48.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:48 smithi107 ceph-mon[90179]: pgmap v21: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:37:48.984 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:48 smithi107 ceph-mon[90179]: mon.smithi107 calling monitor election 2024-04-22T23:37:48.984 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:48 smithi107 ceph-mon[90179]: mon.smithi107 is new leader, mons smithi107,smithi157 in quorum (ranks 0,1) 2024-04-22T23:37:48.984 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:48 smithi107 ceph-mon[90179]: monmap epoch 2 2024-04-22T23:37:48.984 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:48 smithi107 ceph-mon[90179]: fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 2024-04-22T23:37:48.984 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:48 smithi107 ceph-mon[90179]: last_changed 2024-04-22T23:27:58.719644+0000 2024-04-22T23:37:48.984 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:48 smithi107 ceph-mon[90179]: created 2024-04-22T23:25:22.712092+0000 2024-04-22T23:37:48.984 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:48 smithi107 ceph-mon[90179]: min_mon_release 17 (quincy) 2024-04-22T23:37:48.984 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:48 smithi107 ceph-mon[90179]: election_strategy: 1 2024-04-22T23:37:48.984 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:48 smithi107 ceph-mon[90179]: 0: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.smithi107 2024-04-22T23:37:48.984 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:48 smithi107 ceph-mon[90179]: 1: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.smithi157 2024-04-22T23:37:48.984 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:48 smithi107 ceph-mon[90179]: fsmap cephfs:2 {0=cephfs.smithi107.ktuxjl=up:active,1=cephfs.smithi157.rcaris=up:active} 2 up:standby 2024-04-22T23:37:48.984 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:48 smithi107 ceph-mon[90179]: osdmap e48: 6 total, 6 up, 6 in 2024-04-22T23:37:48.984 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:48 smithi107 ceph-mon[90179]: mgrmap e31: smithi107.uipxvm(active, since 37s), standbys: smithi157.tuylis 2024-04-22T23:37:48.984 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:48 smithi107 ceph-mon[90179]: overall HEALTH_OK 2024-04-22T23:37:48.984 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:48 smithi107 ceph-mon[90179]: from='mgr.14778 ' entity='' 2024-04-22T23:37:48.985 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:48 smithi107 ceph-mon[90179]: mgrmap e32: smithi107.uipxvm(active, since 37s), standbys: smithi157.tuylis 2024-04-22T23:37:49.060 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:48 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-04-22T23:37:49.060 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:48 smithi157 ceph-mon[33877]: from='mgr.14778 172.21.15.107:0/2431225221' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mon metadata", "id": "smithi157"}]: dispatch 2024-04-22T23:37:49.060 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:48 smithi157 ceph-mon[33877]: pgmap v21: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:37:49.060 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:48 smithi157 ceph-mon[33877]: mon.smithi107 calling monitor election 2024-04-22T23:37:49.060 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:48 smithi157 ceph-mon[33877]: mon.smithi107 is new leader, mons smithi107,smithi157 in quorum (ranks 0,1) 2024-04-22T23:37:49.061 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:48 smithi157 ceph-mon[33877]: monmap epoch 2 2024-04-22T23:37:49.061 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:48 smithi157 ceph-mon[33877]: fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 2024-04-22T23:37:49.061 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:48 smithi157 ceph-mon[33877]: last_changed 2024-04-22T23:27:58.719644+0000 2024-04-22T23:37:49.061 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:48 smithi157 ceph-mon[33877]: created 2024-04-22T23:25:22.712092+0000 2024-04-22T23:37:49.061 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:48 smithi157 ceph-mon[33877]: min_mon_release 17 (quincy) 2024-04-22T23:37:49.061 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:48 smithi157 ceph-mon[33877]: election_strategy: 1 2024-04-22T23:37:49.061 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:48 smithi157 ceph-mon[33877]: 0: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.smithi107 2024-04-22T23:37:49.061 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:48 smithi157 ceph-mon[33877]: 1: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.smithi157 2024-04-22T23:37:49.061 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:48 smithi157 ceph-mon[33877]: fsmap cephfs:2 {0=cephfs.smithi107.ktuxjl=up:active,1=cephfs.smithi157.rcaris=up:active} 2 up:standby 2024-04-22T23:37:49.062 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:48 smithi157 ceph-mon[33877]: osdmap e48: 6 total, 6 up, 6 in 2024-04-22T23:37:49.062 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:48 smithi157 ceph-mon[33877]: mgrmap e31: smithi107.uipxvm(active, since 37s), standbys: smithi157.tuylis 2024-04-22T23:37:49.062 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:48 smithi157 ceph-mon[33877]: overall HEALTH_OK 2024-04-22T23:37:49.062 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:48 smithi157 ceph-mon[33877]: from='mgr.14778 ' entity='' 2024-04-22T23:37:49.062 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:48 smithi157 ceph-mon[33877]: mgrmap e32: smithi107.uipxvm(active, since 37s), standbys: smithi157.tuylis 2024-04-22T23:37:55.705 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:55 smithi107 ceph-mon[90179]: Standby manager daemon smithi157.tuylis restarted 2024-04-22T23:37:55.706 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:55 smithi107 ceph-mon[90179]: Standby manager daemon smithi157.tuylis started 2024-04-22T23:37:55.706 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:55 smithi107 ceph-mon[90179]: from='mgr.? 172.21.15.157:0/2706976439' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi157.tuylis/crt"}]: dispatch 2024-04-22T23:37:55.706 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:55 smithi107 ceph-mon[90179]: from='mgr.? 172.21.15.157:0/2706976439' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T23:37:55.706 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:55 smithi107 ceph-mon[90179]: from='mgr.? 172.21.15.157:0/2706976439' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi157.tuylis/key"}]: dispatch 2024-04-22T23:37:55.706 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:55 smithi107 ceph-mon[90179]: from='mgr.? 172.21.15.157:0/2706976439' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T23:37:55.706 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:55 smithi107 ceph-mon[90179]: Active manager daemon smithi107.uipxvm restarted 2024-04-22T23:37:55.706 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:55 smithi107 ceph-mon[90179]: Activating manager daemon smithi107.uipxvm 2024-04-22T23:37:55.706 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:55 smithi107 ceph-mon[90179]: osdmap e49: 6 total, 6 up, 6 in 2024-04-22T23:37:55.706 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:55 smithi107 ceph-mon[90179]: mgrmap e33: smithi107.uipxvm(active, starting, since 0.00911148s), standbys: smithi157.tuylis 2024-04-22T23:37:55.706 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:55 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-04-22T23:37:55.706 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:55 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mon metadata", "id": "smithi157"}]: dispatch 2024-04-22T23:37:55.706 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:55 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.ktuxjl"}]: dispatch 2024-04-22T23:37:55.706 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:55 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.dfvvkw"}]: dispatch 2024-04-22T23:37:56.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:55 smithi157 ceph-mon[33877]: Standby manager daemon smithi157.tuylis restarted 2024-04-22T23:37:56.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:55 smithi157 ceph-mon[33877]: Standby manager daemon smithi157.tuylis started 2024-04-22T23:37:56.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:55 smithi157 ceph-mon[33877]: from='mgr.? 172.21.15.157:0/2706976439' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi157.tuylis/crt"}]: dispatch 2024-04-22T23:37:56.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:55 smithi157 ceph-mon[33877]: from='mgr.? 172.21.15.157:0/2706976439' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T23:37:56.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:55 smithi157 ceph-mon[33877]: from='mgr.? 172.21.15.157:0/2706976439' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi157.tuylis/key"}]: dispatch 2024-04-22T23:37:56.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:55 smithi157 ceph-mon[33877]: from='mgr.? 172.21.15.157:0/2706976439' entity='mgr.smithi157.tuylis' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T23:37:56.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:55 smithi157 ceph-mon[33877]: Active manager daemon smithi107.uipxvm restarted 2024-04-22T23:37:56.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:55 smithi157 ceph-mon[33877]: Activating manager daemon smithi107.uipxvm 2024-04-22T23:37:56.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:55 smithi157 ceph-mon[33877]: osdmap e49: 6 total, 6 up, 6 in 2024-04-22T23:37:56.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:55 smithi157 ceph-mon[33877]: mgrmap e33: smithi107.uipxvm(active, starting, since 0.00911148s), standbys: smithi157.tuylis 2024-04-22T23:37:56.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:55 smithi157 ceph-mon[33877]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-04-22T23:37:56.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:55 smithi157 ceph-mon[33877]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mon metadata", "id": "smithi157"}]: dispatch 2024-04-22T23:37:56.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:55 smithi157 ceph-mon[33877]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.ktuxjl"}]: dispatch 2024-04-22T23:37:56.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:55 smithi157 ceph-mon[33877]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.dfvvkw"}]: dispatch 2024-04-22T23:37:56.972 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:56 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi157.rcaris"}]: dispatch 2024-04-22T23:37:56.972 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:56 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi157.ufywse"}]: dispatch 2024-04-22T23:37:56.972 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:56 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mgr metadata", "who": "smithi107.uipxvm", "id": "smithi107.uipxvm"}]: dispatch 2024-04-22T23:37:56.972 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:56 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mgr metadata", "who": "smithi157.tuylis", "id": "smithi157.tuylis"}]: dispatch 2024-04-22T23:37:56.972 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:56 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T23:37:56.973 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:56 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T23:37:56.973 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:56 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T23:37:56.973 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:56 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T23:37:56.973 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:56 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T23:37:56.973 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:56 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T23:37:56.973 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:56 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-22T23:37:56.973 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:56 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-22T23:37:56.973 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:56 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-22T23:37:56.973 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:56 smithi107 ceph-mon[90179]: Manager daemon smithi107.uipxvm is now available 2024-04-22T23:37:56.973 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:56 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:37:56.973 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:56 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:37:56.973 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:56 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi107.uipxvm/mirror_snapshot_schedule"}]: dispatch 2024-04-22T23:37:56.973 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:56 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi107.uipxvm/trash_purge_schedule"}]: dispatch 2024-04-22T23:37:56.973 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:56 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:56.974 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:56 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:57.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:56 smithi157 ceph-mon[33877]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi157.rcaris"}]: dispatch 2024-04-22T23:37:57.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:56 smithi157 ceph-mon[33877]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi157.ufywse"}]: dispatch 2024-04-22T23:37:57.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:56 smithi157 ceph-mon[33877]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mgr metadata", "who": "smithi107.uipxvm", "id": "smithi107.uipxvm"}]: dispatch 2024-04-22T23:37:57.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:56 smithi157 ceph-mon[33877]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mgr metadata", "who": "smithi157.tuylis", "id": "smithi157.tuylis"}]: dispatch 2024-04-22T23:37:57.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:56 smithi157 ceph-mon[33877]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T23:37:57.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:56 smithi157 ceph-mon[33877]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T23:37:57.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:56 smithi157 ceph-mon[33877]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T23:37:57.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:56 smithi157 ceph-mon[33877]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T23:37:57.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:56 smithi157 ceph-mon[33877]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T23:37:57.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:56 smithi157 ceph-mon[33877]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T23:37:57.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:56 smithi157 ceph-mon[33877]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-22T23:37:57.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:56 smithi157 ceph-mon[33877]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-22T23:37:57.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:56 smithi157 ceph-mon[33877]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-22T23:37:57.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:56 smithi157 ceph-mon[33877]: Manager daemon smithi107.uipxvm is now available 2024-04-22T23:37:57.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:56 smithi157 ceph-mon[33877]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:37:57.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:56 smithi157 ceph-mon[33877]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:37:57.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:56 smithi157 ceph-mon[33877]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi107.uipxvm/mirror_snapshot_schedule"}]: dispatch 2024-04-22T23:37:57.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:56 smithi157 ceph-mon[33877]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi107.uipxvm/trash_purge_schedule"}]: dispatch 2024-04-22T23:37:57.133 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:56 smithi157 ceph-mon[33877]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:57.133 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:56 smithi157 ceph-mon[33877]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:37:57.868 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:57 smithi107 ceph-mon[90179]: [22/Apr/2024:23:37:56] ENGINE Bus STARTING 2024-04-22T23:37:57.868 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:57 smithi107 ceph-mon[90179]: [22/Apr/2024:23:37:56] ENGINE Serving on https://172.21.15.107:7150 2024-04-22T23:37:57.868 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:57 smithi107 ceph-mon[90179]: [22/Apr/2024:23:37:56] ENGINE Serving on http://172.21.15.107:8765 2024-04-22T23:37:57.868 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:57 smithi107 ceph-mon[90179]: [22/Apr/2024:23:37:56] ENGINE Bus STARTED 2024-04-22T23:37:57.868 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:57 smithi107 ceph-mon[90179]: mgrmap e34: smithi107.uipxvm(active, since 1.02496s), standbys: smithi157.tuylis 2024-04-22T23:37:57.868 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:57 smithi107 ceph-mon[90179]: pgmap v3: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:37:58.010 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:57 smithi157 ceph-mon[33877]: [22/Apr/2024:23:37:56] ENGINE Bus STARTING 2024-04-22T23:37:58.010 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:57 smithi157 ceph-mon[33877]: [22/Apr/2024:23:37:56] ENGINE Serving on https://172.21.15.107:7150 2024-04-22T23:37:58.010 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:57 smithi157 ceph-mon[33877]: [22/Apr/2024:23:37:56] ENGINE Serving on http://172.21.15.107:8765 2024-04-22T23:37:58.010 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:57 smithi157 ceph-mon[33877]: [22/Apr/2024:23:37:56] ENGINE Bus STARTED 2024-04-22T23:37:58.010 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:57 smithi157 ceph-mon[33877]: mgrmap e34: smithi107.uipxvm(active, since 1.02496s), standbys: smithi157.tuylis 2024-04-22T23:37:58.010 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:57 smithi157 ceph-mon[33877]: pgmap v3: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:37:58.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:58 smithi107 ceph-mon[90179]: pgmap v4: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:37:58.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:37:58 smithi107 ceph-mon[90179]: mgrmap e35: smithi107.uipxvm(active, since 2s), standbys: smithi157.tuylis 2024-04-22T23:37:59.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:58 smithi157 ceph-mon[33877]: pgmap v4: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:37:59.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:37:58 smithi157 ceph-mon[33877]: mgrmap e35: smithi107.uipxvm(active, since 2s), standbys: smithi157.tuylis 2024-04-22T23:38:00.684 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:00 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:00.684 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:00 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:00.684 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:00 smithi107 ceph-mon[90179]: pgmap v5: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:38:00.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:00 smithi157 ceph-mon[33877]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:00.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:00 smithi157 ceph-mon[33877]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:00.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:00 smithi157 ceph-mon[33877]: pgmap v5: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:38:01.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:01 smithi157 ceph-mon[33877]: mgrmap e36: smithi107.uipxvm(active, since 4s), standbys: smithi157.tuylis 2024-04-22T23:38:01.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:01 smithi157 ceph-mon[33877]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:01.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:01 smithi157 ceph-mon[33877]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:01.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:01 smithi157 ceph-mon[33877]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:01.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:01 smithi157 ceph-mon[33877]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:01.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:01 smithi107 ceph-mon[90179]: mgrmap e36: smithi107.uipxvm(active, since 4s), standbys: smithi157.tuylis 2024-04-22T23:38:01.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:01 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:01.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:01 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:01.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:01 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:01.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:01 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:02.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:02 smithi107 ceph-mon[90179]: pgmap v6: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:38:02.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:02 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:02.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:02 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:02.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:02 smithi157 ceph-mon[33877]: pgmap v6: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:38:02.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:02 smithi157 ceph-mon[33877]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:02.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:02 smithi157 ceph-mon[33877]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:04.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:04 smithi107 ceph-mon[90179]: pgmap v7: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-04-22T23:38:04.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:04 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:04.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:04 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:04.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:04 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:04.984 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:04 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:38:05.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:04 smithi157 ceph-mon[33877]: pgmap v7: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-04-22T23:38:05.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:04 smithi157 ceph-mon[33877]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:05.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:04 smithi157 ceph-mon[33877]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:05.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:04 smithi157 ceph-mon[33877]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:05.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:04 smithi157 ceph-mon[33877]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:38:05.926 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:05 smithi157 ceph-mon[33877]: Updating smithi107:/etc/ceph/ceph.conf 2024-04-22T23:38:05.926 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:05 smithi157 ceph-mon[33877]: Updating smithi157:/etc/ceph/ceph.conf 2024-04-22T23:38:05.926 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:05 smithi157 ceph-mon[33877]: Updating smithi107:/var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T23:38:05.926 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:05 smithi157 ceph-mon[33877]: Updating smithi157:/var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T23:38:05.926 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:05 smithi157 ceph-mon[33877]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-04-22T23:38:05.926 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:05 smithi157 ceph-mon[33877]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2024-04-22T23:38:05.926 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:05 smithi157 ceph-mon[33877]: Updating smithi107:/var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T23:38:05.926 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:05 smithi157 ceph-mon[33877]: Updating smithi157:/var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T23:38:05.927 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:05 smithi157 ceph-mon[33877]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:05.927 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:05 smithi157 ceph-mon[33877]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:05.927 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:05 smithi157 ceph-mon[33877]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:05.927 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:05 smithi157 ceph-mon[33877]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:05.927 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:05 smithi157 ceph-mon[33877]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:05.927 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:05 smithi157 ceph-mon[33877]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:38:05.927 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:05 smithi157 ceph-mon[33877]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:38:05.927 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:05 smithi157 ceph-mon[33877]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-22T23:38:05.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:05 smithi107 ceph-mon[90179]: Updating smithi107:/etc/ceph/ceph.conf 2024-04-22T23:38:05.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:05 smithi107 ceph-mon[90179]: Updating smithi157:/etc/ceph/ceph.conf 2024-04-22T23:38:05.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:05 smithi107 ceph-mon[90179]: Updating smithi107:/var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T23:38:05.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:05 smithi107 ceph-mon[90179]: Updating smithi157:/var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T23:38:05.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:05 smithi107 ceph-mon[90179]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-04-22T23:38:05.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:05 smithi107 ceph-mon[90179]: Updating smithi157:/etc/ceph/ceph.client.admin.keyring 2024-04-22T23:38:05.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:05 smithi107 ceph-mon[90179]: Updating smithi107:/var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T23:38:05.984 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:05 smithi107 ceph-mon[90179]: Updating smithi157:/var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T23:38:05.984 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:05 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:05.984 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:05 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:05.984 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:05 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:05.984 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:05 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:05.984 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:05 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:05.984 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:05 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:38:05.984 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:05 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:38:05.984 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:05 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-22T23:38:06.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:06 smithi157 ceph-mon[33877]: pgmap v8: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-22T23:38:06.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:06 smithi157 ceph-mon[33877]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:06.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:06 smithi157 ceph-mon[33877]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T23:38:06.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:06 smithi157 ceph-mon[33877]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-22T23:38:06.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:06 smithi157 ceph-mon[33877]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:06.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:06 smithi107 ceph-mon[90179]: pgmap v8: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-22T23:38:06.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:06 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:06.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:06 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T23:38:06.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:06 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-22T23:38:06.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:06 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:07.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:07 smithi157 ceph-mon[33877]: Upgrade: Updating mon.smithi157 2024-04-22T23:38:07.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:07 smithi157 ceph-mon[33877]: Deploying daemon mon.smithi157 on smithi157 2024-04-22T23:38:07.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:07 smithi107 ceph-mon[90179]: Upgrade: Updating mon.smithi157 2024-04-22T23:38:07.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:07 smithi107 ceph-mon[90179]: Deploying daemon mon.smithi157 on smithi157 2024-04-22T23:38:08.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:08 smithi157 systemd[1]: Stopping Ceph mon.smithi157 for 8252f6f2-00ff-11ef-bc93-c7b262605968... 2024-04-22T23:38:08.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:08 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-mon-smithi157[33856]: 2024-04-22T23:38:08.703+0000 7fe7be5f9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi157 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T23:38:08.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:08 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-mon-smithi157[33856]: 2024-04-22T23:38:08.703+0000 7fe7be5f9700 -1 mon.smithi157@1(peon) e2 *** Got Signal Terminated *** 2024-04-22T23:38:09.199 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:08 smithi157 podman[79241]: 2024-04-22 23:38:08.939623833 +0000 UTC m=+0.338675073 container died 0747a85008d9f4a7781d82c00da641ad5c340e116ad04926a3459889ec06a0c5 (image=quay.ceph.io/ceph-ci/ceph@sha256:3890cd726bd5e73ae623dabdf4ed76dd5df81c936b8bd8d761026babda969466, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-mon-smithi157, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=quincy-70d0cb4, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-22T23:38:09.540 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:09 smithi157 podman[79241]: 2024-04-22 23:38:09.313083118 +0000 UTC m=+0.712134354 container cleanup 0747a85008d9f4a7781d82c00da641ad5c340e116ad04926a3459889ec06a0c5 (image=quay.ceph.io/ceph-ci/ceph@sha256:3890cd726bd5e73ae623dabdf4ed76dd5df81c936b8bd8d761026babda969466, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-mon-smithi157, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=quincy-70d0cb4, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, ceph=True) 2024-04-22T23:38:09.541 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:09 smithi157 bash[79241]: ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-mon-smithi157 2024-04-22T23:38:09.870 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:09 smithi157 podman[79255]: 2024-04-22 23:38:09.547594062 +0000 UTC m=+0.602326961 container remove 0747a85008d9f4a7781d82c00da641ad5c340e116ad04926a3459889ec06a0c5 (image=quay.ceph.io/ceph-ci/ceph@sha256:3890cd726bd5e73ae623dabdf4ed76dd5df81c936b8bd8d761026babda969466, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-mon-smithi157, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=quincy-70d0cb4, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-22T23:38:09.870 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:09 smithi157 systemd[1]: ceph-8252f6f2-00ff-11ef-bc93-c7b262605968@mon.smithi157.service: Deactivated successfully. 2024-04-22T23:38:09.870 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:09 smithi157 systemd[1]: Stopped Ceph mon.smithi157 for 8252f6f2-00ff-11ef-bc93-c7b262605968. 2024-04-22T23:38:09.870 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:09 smithi157 systemd[1]: ceph-8252f6f2-00ff-11ef-bc93-c7b262605968@mon.smithi157.service: Consumed 5.964s CPU time. 2024-04-22T23:38:10.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:09 smithi157 systemd[1]: Starting Ceph mon.smithi157 for 8252f6f2-00ff-11ef-bc93-c7b262605968... 2024-04-22T23:38:10.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 podman[79384]: 2024-04-22 23:38:09.996656 +0000 UTC m=+0.018739000 image pull a0434699c1d76af4eea1fc95ff0a22ea73d1e04e609464fbf658d6f39bc7f2fc quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c 2024-04-22T23:38:10.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 podman[79384]: 2024-04-22 23:38:10.098417131 +0000 UTC m=+0.120500128 container create ea88de8be4fe435a0feb9f285fb754e48d54b8968e6b04287acea8a05fb49275 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-mon-smithi157, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-22T23:38:10.436 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 podman[79384]: 2024-04-22 23:38:10.344242624 +0000 UTC m=+0.366325640 container init ea88de8be4fe435a0feb9f285fb754e48d54b8968e6b04287acea8a05fb49275 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-mon-smithi157, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-22T23:38:10.436 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 podman[79384]: 2024-04-22 23:38:10.348429343 +0000 UTC m=+0.370512350 container start ea88de8be4fe435a0feb9f285fb754e48d54b8968e6b04287acea8a05fb49275 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-mon-smithi157, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-22T23:38:10.437 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: set uid:gid to 167:167 (ceph:ceph) 2024-04-22T23:38:10.437 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev), process ceph-mon, pid 2 2024-04-22T23:38:10.437 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: pidfile_write: ignore empty --pid-file 2024-04-22T23:38:10.437 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: load: jerasure load: lrc 2024-04-22T23:38:10.437 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: RocksDB version: 7.9.2 2024-04-22T23:38:10.437 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Git sha 0 2024-04-22T23:38:10.437 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Compile date 2024-04-22 21:33:31 2024-04-22T23:38:10.437 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: DB SUMMARY 2024-04-22T23:38:10.437 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: DB Session ID: SYVQULDZLU02Y3XUW88V 2024-04-22T23:38:10.437 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: CURRENT file: CURRENT 2024-04-22T23:38:10.437 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: IDENTITY file: IDENTITY 2024-04-22T23:38:10.437 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: MANIFEST file: MANIFEST-000009 size: 1084 Bytes 2024-04-22T23:38:10.437 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi157/store.db dir, Total Num: 1, files: 000027.sst 2024-04-22T23:38:10.437 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi157/store.db: 000025.log size: 913862 ; 2024-04-22T23:38:10.437 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.error_if_exists: 0 2024-04-22T23:38:10.438 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.create_if_missing: 0 2024-04-22T23:38:10.438 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.paranoid_checks: 1 2024-04-22T23:38:10.438 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-22T23:38:10.438 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-22T23:38:10.438 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-22T23:38:10.438 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.env: 0x55ae933edcc0 2024-04-22T23:38:10.438 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.fs: PosixFileSystem 2024-04-22T23:38:10.438 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.info_log: 0x55ae94011aa0 2024-04-22T23:38:10.438 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.max_file_opening_threads: 16 2024-04-22T23:38:10.438 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.statistics: (nil) 2024-04-22T23:38:10.438 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.use_fsync: 0 2024-04-22T23:38:10.438 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.max_log_file_size: 0 2024-04-22T23:38:10.438 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-22T23:38:10.439 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-22T23:38:10.439 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.keep_log_file_num: 1000 2024-04-22T23:38:10.439 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.recycle_log_file_num: 0 2024-04-22T23:38:10.439 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.allow_fallocate: 1 2024-04-22T23:38:10.439 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.allow_mmap_reads: 0 2024-04-22T23:38:10.439 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.allow_mmap_writes: 0 2024-04-22T23:38:10.439 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.use_direct_reads: 0 2024-04-22T23:38:10.439 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-22T23:38:10.439 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.create_missing_column_families: 0 2024-04-22T23:38:10.439 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.db_log_dir: 2024-04-22T23:38:10.439 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.wal_dir: 2024-04-22T23:38:10.439 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-22T23:38:10.439 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-22T23:38:10.439 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-22T23:38:10.439 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-22T23:38:10.440 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-22T23:38:10.440 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-22T23:38:10.440 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.advise_random_on_open: 1 2024-04-22T23:38:10.440 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.db_write_buffer_size: 0 2024-04-22T23:38:10.440 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.write_buffer_manager: 0x55ae93fa50e0 2024-04-22T23:38:10.440 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-22T23:38:10.440 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-22T23:38:10.440 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-22T23:38:10.440 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.rate_limiter: (nil) 2024-04-22T23:38:10.440 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-22T23:38:10.440 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.wal_recovery_mode: 2 2024-04-22T23:38:10.440 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.enable_thread_tracking: 0 2024-04-22T23:38:10.440 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.enable_pipelined_write: 0 2024-04-22T23:38:10.441 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.unordered_write: 0 2024-04-22T23:38:10.441 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-22T23:38:10.441 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-22T23:38:10.441 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-22T23:38:10.441 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-22T23:38:10.441 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.row_cache: None 2024-04-22T23:38:10.441 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.wal_filter: None 2024-04-22T23:38:10.442 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-22T23:38:10.442 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.allow_ingest_behind: 0 2024-04-22T23:38:10.443 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.two_write_queues: 0 2024-04-22T23:38:10.443 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.manual_wal_flush: 0 2024-04-22T23:38:10.443 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.wal_compression: 0 2024-04-22T23:38:10.443 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.atomic_flush: 0 2024-04-22T23:38:10.443 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-22T23:38:10.443 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-22T23:38:10.443 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-22T23:38:10.443 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.log_readahead_size: 0 2024-04-22T23:38:10.443 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-22T23:38:10.443 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.best_efforts_recovery: 0 2024-04-22T23:38:10.443 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-22T23:38:10.443 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-22T23:38:10.443 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.allow_data_in_errors: 0 2024-04-22T23:38:10.444 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.db_host_id: __hostname__ 2024-04-22T23:38:10.444 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-22T23:38:10.444 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.max_background_jobs: 2 2024-04-22T23:38:10.444 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.max_background_compactions: -1 2024-04-22T23:38:10.444 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.max_subcompactions: 1 2024-04-22T23:38:10.444 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-22T23:38:10.444 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-22T23:38:10.444 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-22T23:38:10.444 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.max_total_wal_size: 0 2024-04-22T23:38:10.444 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-22T23:38:10.444 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-22T23:38:10.444 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-22T23:38:10.444 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-22T23:38:10.444 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.max_open_files: -1 2024-04-22T23:38:10.445 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.bytes_per_sync: 0 2024-04-22T23:38:10.445 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-22T23:38:10.445 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-22T23:38:10.445 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.compaction_readahead_size: 0 2024-04-22T23:38:10.445 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.max_background_flushes: -1 2024-04-22T23:38:10.445 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Compression algorithms supported: 2024-04-22T23:38:10.445 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-22T23:38:10.445 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: kZSTD supported: 0 2024-04-22T23:38:10.445 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: kXpressCompression supported: 0 2024-04-22T23:38:10.445 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: kLZ4HCCompression supported: 1 2024-04-22T23:38:10.445 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: kZlibCompression supported: 1 2024-04-22T23:38:10.445 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: kSnappyCompression supported: 1 2024-04-22T23:38:10.445 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: kLZ4Compression supported: 1 2024-04-22T23:38:10.445 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: kBZip2Compression supported: 0 2024-04-22T23:38:10.446 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-22T23:38:10.446 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-22T23:38:10.446 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi157/store.db/MANIFEST-000009 2024-04-22T23:38:10.446 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-22T23:38:10.446 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-22T23:38:10.446 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.merge_operator: 2024-04-22T23:38:10.446 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.compaction_filter: None 2024-04-22T23:38:10.446 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.compaction_filter_factory: None 2024-04-22T23:38:10.446 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.sst_partitioner_factory: None 2024-04-22T23:38:10.446 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-22T23:38:10.446 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-22T23:38:10.447 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55ae94011c80) 2024-04-22T23:38:10.447 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: cache_index_and_filter_blocks: 1 2024-04-22T23:38:10.447 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-22T23:38:10.447 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-22T23:38:10.447 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: pin_top_level_index_and_filter: 1 2024-04-22T23:38:10.447 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: index_type: 0 2024-04-22T23:38:10.447 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: data_block_index_type: 0 2024-04-22T23:38:10.447 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: index_shortening: 1 2024-04-22T23:38:10.447 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-22T23:38:10.447 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: checksum: 4 2024-04-22T23:38:10.447 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: no_block_cache: 0 2024-04-22T23:38:10.447 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: block_cache: 0x55ae93fd71f0 2024-04-22T23:38:10.447 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: block_cache_name: BinnedLRUCache 2024-04-22T23:38:10.448 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: block_cache_options: 2024-04-22T23:38:10.448 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: capacity : 536870912 2024-04-22T23:38:10.448 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: num_shard_bits : 4 2024-04-22T23:38:10.448 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: strict_capacity_limit : 0 2024-04-22T23:38:10.448 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: high_pri_pool_ratio: 0.000 2024-04-22T23:38:10.448 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: block_cache_compressed: (nil) 2024-04-22T23:38:10.448 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: persistent_cache: (nil) 2024-04-22T23:38:10.448 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: block_size: 4096 2024-04-22T23:38:10.448 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: block_size_deviation: 10 2024-04-22T23:38:10.448 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: block_restart_interval: 16 2024-04-22T23:38:10.448 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: index_block_restart_interval: 1 2024-04-22T23:38:10.448 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: metadata_block_size: 4096 2024-04-22T23:38:10.448 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: partition_filters: 0 2024-04-22T23:38:10.448 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: use_delta_encoding: 1 2024-04-22T23:38:10.449 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: filter_policy: bloomfilter 2024-04-22T23:38:10.449 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: whole_key_filtering: 1 2024-04-22T23:38:10.449 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: verify_compression: 0 2024-04-22T23:38:10.449 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: read_amp_bytes_per_bit: 0 2024-04-22T23:38:10.449 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: format_version: 5 2024-04-22T23:38:10.449 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: enable_index_compression: 1 2024-04-22T23:38:10.449 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: block_align: 0 2024-04-22T23:38:10.449 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: max_auto_readahead_size: 262144 2024-04-22T23:38:10.449 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: prepopulate_block_cache: 0 2024-04-22T23:38:10.449 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: initial_auto_readahead_size: 8192 2024-04-22T23:38:10.449 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: num_file_reads_for_auto_readahead: 2 2024-04-22T23:38:10.449 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.write_buffer_size: 33554432 2024-04-22T23:38:10.449 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.max_write_buffer_number: 2 2024-04-22T23:38:10.449 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.compression: NoCompression 2024-04-22T23:38:10.450 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.bottommost_compression: Disabled 2024-04-22T23:38:10.450 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.prefix_extractor: nullptr 2024-04-22T23:38:10.450 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-22T23:38:10.450 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.num_levels: 7 2024-04-22T23:38:10.450 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-22T23:38:10.450 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-22T23:38:10.450 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-22T23:38:10.450 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-22T23:38:10.450 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-22T23:38:10.450 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-22T23:38:10.450 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-22T23:38:10.450 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-22T23:38:10.450 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-22T23:38:10.451 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-22T23:38:10.451 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-22T23:38:10.451 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-22T23:38:10.451 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-22T23:38:10.451 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.compression_opts.level: 32767 2024-04-22T23:38:10.451 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.compression_opts.strategy: 0 2024-04-22T23:38:10.451 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-22T23:38:10.451 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-22T23:38:10.451 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-22T23:38:10.451 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-22T23:38:10.451 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.compression_opts.enabled: false 2024-04-22T23:38:10.451 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-22T23:38:10.451 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-22T23:38:10.451 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-22T23:38:10.451 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-22T23:38:10.452 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.target_file_size_base: 67108864 2024-04-22T23:38:10.452 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-22T23:38:10.452 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-22T23:38:10.452 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-22T23:38:10.452 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-22T23:38:10.452 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-22T23:38:10.452 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-22T23:38:10.452 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-22T23:38:10.452 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-22T23:38:10.452 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-22T23:38:10.452 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-22T23:38:10.452 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-22T23:38:10.452 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-22T23:38:10.452 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-22T23:38:10.453 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-22T23:38:10.453 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.arena_block_size: 1048576 2024-04-22T23:38:10.453 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-22T23:38:10.453 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-22T23:38:10.453 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.disable_auto_compactions: 0 2024-04-22T23:38:10.453 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-22T23:38:10.453 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-22T23:38:10.453 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-22T23:38:10.453 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-22T23:38:10.453 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-22T23:38:10.453 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-22T23:38:10.453 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-22T23:38:10.453 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-22T23:38:10.453 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-22T23:38:10.454 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-22T23:38:10.454 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-22T23:38:10.454 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.inplace_update_support: 0 2024-04-22T23:38:10.454 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-22T23:38:10.454 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-22T23:38:10.455 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-22T23:38:10.455 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-22T23:38:10.455 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.bloom_locality: 0 2024-04-22T23:38:10.455 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.max_successive_merges: 0 2024-04-22T23:38:10.455 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-22T23:38:10.455 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.paranoid_file_checks: 0 2024-04-22T23:38:10.455 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.force_consistency_checks: 1 2024-04-22T23:38:10.455 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.report_bg_io_stats: 0 2024-04-22T23:38:10.455 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.ttl: 2592000 2024-04-22T23:38:10.455 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-22T23:38:10.456 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-22T23:38:10.456 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-22T23:38:10.456 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.enable_blob_files: false 2024-04-22T23:38:10.456 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.min_blob_size: 0 2024-04-22T23:38:10.456 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.blob_file_size: 268435456 2024-04-22T23:38:10.456 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-22T23:38:10.456 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-22T23:38:10.456 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-22T23:38:10.456 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-22T23:38:10.456 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-22T23:38:10.456 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.blob_file_starting_level: 0 2024-04-22T23:38:10.456 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-22T23:38:10.456 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 27.sst 2024-04-22T23:38:10.457 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-22T23:38:10.457 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi157/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 29, last_sequence is 12361, log_number is 25,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-22T23:38:10.457 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 25 2024-04-22T23:38:10.457 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 3918de62-4480-4708-91e2-b55eba1e9e72 2024-04-22T23:38:10.457 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713829090377497, "job": 1, "event": "recovery_started", "wal_files": [25]} 2024-04-22T23:38:10.457 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #25 mode 2 2024-04-22T23:38:10.457 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713829090380544, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 30, "file_size": 552548, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12366, "largest_seqno": 12614, "table_properties": {"data_size": 550386, "index_size": 804, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 325, "raw_key_size": 2742, "raw_average_key_size": 24, "raw_value_size": 547689, "raw_average_value_size": 4846, "num_data_blocks": 35, "num_entries": 113, "num_filter_entries": 113, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713829090, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "3918de62-4480-4708-91e2-b55eba1e9e72", "db_session_id": "SYVQULDZLU02Y3XUW88V", "orig_file_number": 30, "seqno_to_time_mapping": "N/A"}} 2024-04-22T23:38:10.457 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713829090380639, "job": 1, "event": "recovery_finished"} 2024-04-22T23:38:10.457 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: [db/version_set.cc:5047] Creating manifest 32 2024-04-22T23:38:10.457 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-22T23:38:10.457 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi157/store.db/000025.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-22T23:38:10.457 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55ae940ec000 2024-04-22T23:38:10.457 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: DB pointer 0x55ae940d6000 2024-04-22T23:38:10.457 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-22T23:38:10.458 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-22T23:38:10.458 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: ** DB Stats ** 2024-04-22T23:38:10.458 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-22T23:38:10.458 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-22T23:38:10.458 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-22T23:38:10.458 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-22T23:38:10.458 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-22T23:38:10.458 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-22T23:38:10.458 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-22T23:38:10.459 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: 2024-04-22T23:38:10.459 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: ** Compaction Stats [default] ** 2024-04-22T23:38:10.459 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-22T23:38:10.459 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-22T23:38:10.459 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: L0 1/0 539.60 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 321.1 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-04-22T23:38:10.459 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: L6 1/0 7.51 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-04-22T23:38:10.459 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: Sum 2/0 8.04 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 321.1 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-04-22T23:38:10.459 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 321.1 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-04-22T23:38:10.459 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: 2024-04-22T23:38:10.459 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: ** Compaction Stats [default] ** 2024-04-22T23:38:10.459 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-22T23:38:10.459 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-22T23:38:10.459 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 321.1 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-04-22T23:38:10.459 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: 2024-04-22T23:38:10.460 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-22T23:38:10.460 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: 2024-04-22T23:38:10.460 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-22T23:38:10.460 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: Flush(GB): cumulative 0.001, interval 0.001 2024-04-22T23:38:10.460 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-22T23:38:10.460 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-22T23:38:10.460 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-22T23:38:10.460 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-22T23:38:10.460 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: Cumulative compaction: 0.00 GB write, 99.71 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-22T23:38:10.460 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: Interval compaction: 0.00 GB write, 99.71 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-22T23:38:10.460 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-22T23:38:10.460 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: Block cache BinnedLRUCache@0x55ae93fd71f0#2 capacity: 512.00 MB usage: 1.33 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-04-22T23:38:10.460 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.38 KB,7.15256e-05%) IndexBlock(1,0.95 KB,0.000181794%) Misc(1,0.00 KB,0%) 2024-04-22T23:38:10.461 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: 2024-04-22T23:38:10.461 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-22T23:38:10.461 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: starting mon.smithi157 rank 1 at public addrs [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] at bind addrs [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi157 fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 2024-04-22T23:38:10.461 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: mon.smithi157@-1(???) e2 preinit fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 2024-04-22T23:38:10.461 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: mon.smithi157@-1(???).mds e19 new map 2024-04-22T23:38:10.461 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: mon.smithi157@-1(???).mds e19 print_map 2024-04-22T23:38:10.461 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: e19 2024-04-22T23:38:10.461 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T23:38:10.461 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:38:10.461 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: legacy client fscid: 1 2024-04-22T23:38:10.461 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: 2024-04-22T23:38:10.461 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: Filesystem 'cephfs' (1) 2024-04-22T23:38:10.461 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: fs_name cephfs 2024-04-22T23:38:10.461 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: epoch 19 2024-04-22T23:38:10.461 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-22T23:38:10.461 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: created 2024-04-22T23:32:20.146130+0000 2024-04-22T23:38:10.462 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: modified 2024-04-22T23:35:12.338678+0000 2024-04-22T23:38:10.462 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: tableserver 0 2024-04-22T23:38:10.462 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: root 0 2024-04-22T23:38:10.462 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: session_timeout 60 2024-04-22T23:38:10.462 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: session_autoclose 300 2024-04-22T23:38:10.462 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: max_file_size 1099511627776 2024-04-22T23:38:10.462 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: max_xattr_size 65536 2024-04-22T23:38:10.462 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: required_client_features {} 2024-04-22T23:38:10.462 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: last_failure 0 2024-04-22T23:38:10.462 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: last_failure_osd_epoch 46 2024-04-22T23:38:10.462 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:38:10.462 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: max_mds 2 2024-04-22T23:38:10.462 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: in 0,1 2024-04-22T23:38:10.462 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: up {0=14468,1=24265} 2024-04-22T23:38:10.463 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: failed 2024-04-22T23:38:10.463 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: damaged 2024-04-22T23:38:10.463 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: stopped 2024-04-22T23:38:10.463 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: data_pools [3] 2024-04-22T23:38:10.463 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: metadata_pool 2 2024-04-22T23:38:10.463 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: inline_data disabled 2024-04-22T23:38:10.463 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: balancer 2024-04-22T23:38:10.463 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: bal_rank_mask -1 2024-04-22T23:38:10.463 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: standby_count_wanted 1 2024-04-22T23:38:10.463 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: qdb_cluster leader: 0 members: 2024-04-22T23:38:10.463 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: [mds.cephfs.smithi107.ktuxjl{0:14468} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.107:6828/3335313253,v1:172.21.15.107:6829/3335313253] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:38:10.463 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: [mds.cephfs.smithi157.rcaris{1:24265} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.157:6824/2779251893,v1:172.21.15.157:6825/2779251893] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:38:10.464 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: 2024-04-22T23:38:10.464 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: 2024-04-22T23:38:10.464 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: Standby daemons: 2024-04-22T23:38:10.464 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: 2024-04-22T23:38:10.464 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: [mds.cephfs.smithi107.dfvvkw{-1:14484} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/3548717247,v1:172.21.15.107:6827/3548717247] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:38:10.464 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout: [mds.cephfs.smithi157.ufywse{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.157:6826/4072444199,v1:172.21.15.157:6827/4072444199] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:38:10.464 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: mon.smithi157@-1(???).osd e49 crush map has features 3314933000852226048, adjusting msgr requires 2024-04-22T23:38:10.464 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: mon.smithi157@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-04-22T23:38:10.464 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: mon.smithi157@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-04-22T23:38:10.464 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: mon.smithi157@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-04-22T23:38:10.464 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 ceph-mon[79403]: mon.smithi157@-1(???).paxosservice(auth 1..22) refresh upgraded, format 0 -> 3 2024-04-22T23:38:10.464 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 bash[79384]: ea88de8be4fe435a0feb9f285fb754e48d54b8968e6b04287acea8a05fb49275 2024-04-22T23:38:10.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:10 smithi157 systemd[1]: Started Ceph mon.smithi157 for 8252f6f2-00ff-11ef-bc93-c7b262605968. 2024-04-22T23:38:11.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.589+0000 7f2d6196e700 1 -- 172.21.15.107:0/1411771222 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d5c0fce40 msgr2=0x7f2d5c0fd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:11.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.589+0000 7f2d6196e700 1 --2- 172.21.15.107:0/1411771222 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d5c0fce40 0x7f2d5c0fd250 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c004940 tx=0x7f2d4c009e30 comp rx=0 tx=0).stop 2024-04-22T23:38:11.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.589+0000 7f2d6196e700 1 -- 172.21.15.107:0/1411771222 shutdown_connections 2024-04-22T23:38:11.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.589+0000 7f2d6196e700 1 --2- 172.21.15.107:0/1411771222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d5c0fe040 0x7f2d5c0fe4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:11.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.589+0000 7f2d6196e700 1 --2- 172.21.15.107:0/1411771222 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d5c0fce40 0x7f2d5c0fd250 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:11.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.589+0000 7f2d6196e700 1 -- 172.21.15.107:0/1411771222 >> 172.21.15.107:0/1411771222 conn(0x7f2d5c0f8530 msgr2=0x7f2d5c0fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:38:11.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.589+0000 7f2d6196e700 1 -- 172.21.15.107:0/1411771222 shutdown_connections 2024-04-22T23:38:11.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.589+0000 7f2d6196e700 1 -- 172.21.15.107:0/1411771222 wait complete. 2024-04-22T23:38:11.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.590+0000 7f2d6196e700 1 Processor -- start 2024-04-22T23:38:11.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.590+0000 7f2d6196e700 1 -- start start 2024-04-22T23:38:11.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.590+0000 7f2d6196e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d5c0fce40 0x7f2d5c10da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:11.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.590+0000 7f2d6196e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d5c0fe040 0x7f2d5c10dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:11.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.590+0000 7f2d6196e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5c10e5d0 con 0x7f2d5c0fce40 2024-04-22T23:38:11.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.590+0000 7f2d6196e700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5c10ee20 con 0x7f2d5c0fe040 2024-04-22T23:38:11.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.590+0000 7f2d5affd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d5c0fce40 0x7f2d5c10da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:38:11.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.590+0000 7f2d5affd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d5c0fce40 0x7f2d5c10da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35800/0 (socket says 172.21.15.107:35800) 2024-04-22T23:38:11.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.591+0000 7f2d5affd700 1 -- 172.21.15.107:0/1044775004 learned_addr learned my addr 172.21.15.107:0/1044775004 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:38:11.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.591+0000 7f2d527fc700 1 --2- 172.21.15.107:0/1044775004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d5c0fe040 0x7f2d5c10dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:38:11.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.591+0000 7f2d5affd700 1 -- 172.21.15.107:0/1044775004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d5c0fe040 msgr2=0x7f2d5c10dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:11.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.591+0000 7f2d5affd700 1 --2- 172.21.15.107:0/1044775004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d5c0fe040 0x7f2d5c10dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:11.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.591+0000 7f2d5affd700 1 -- 172.21.15.107:0/1044775004 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d4c005040 con 0x7f2d5c0fce40 2024-04-22T23:38:11.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.591+0000 7f2d527fc700 1 --2- 172.21.15.107:0/1044775004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d5c0fe040 0x7f2d5c10dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T23:38:11.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.591+0000 7f2d5affd700 1 --2- 172.21.15.107:0/1044775004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d5c0fce40 0x7f2d5c10da90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c00b3b0 tx=0x7f2d4c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:38:11.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.591+0000 7f2d53fff700 1 -- 172.21.15.107:0/1044775004 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d4c00e040 con 0x7f2d5c0fce40 2024-04-22T23:38:11.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.591+0000 7f2d53fff700 1 -- 172.21.15.107:0/1044775004 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2d4c00bcc0 con 0x7f2d5c0fce40 2024-04-22T23:38:11.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.591+0000 7f2d6196e700 1 -- 172.21.15.107:0/1044775004 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d5c10f0a0 con 0x7f2d5c0fce40 2024-04-22T23:38:11.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.591+0000 7f2d53fff700 1 -- 172.21.15.107:0/1044775004 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d4c01c630 con 0x7f2d5c0fce40 2024-04-22T23:38:11.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.591+0000 7f2d6196e700 1 -- 172.21.15.107:0/1044775004 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d5c10f5e0 con 0x7f2d5c0fce40 2024-04-22T23:38:11.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.592+0000 7f2d6196e700 1 -- 172.21.15.107:0/1044775004 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d5c05f5f0 con 0x7f2d5c0fce40 2024-04-22T23:38:11.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.593+0000 7f2d53fff700 1 -- 172.21.15.107:0/1044775004 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 36) v1 ==== 92849+0+0 (secure 0 0 0) 0x7f2d4c0139f0 con 0x7f2d5c0fce40 2024-04-22T23:38:11.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.593+0000 7f2d53fff700 1 --2- 172.21.15.107:0/1044775004 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f2d4806ed20 0x7f2d480711d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:11.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.593+0000 7f2d53fff700 1 -- 172.21.15.107:0/1044775004 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f2d4c08f810 con 0x7f2d5c0fce40 2024-04-22T23:38:11.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.593+0000 7f2d527fc700 1 --2- 172.21.15.107:0/1044775004 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f2d4806ed20 0x7f2d480711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:38:11.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.594+0000 7f2d527fc700 1 --2- 172.21.15.107:0/1044775004 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f2d4806ed20 0x7f2d480711d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2d44009fd0 tx=0x7f2d4400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:38:11.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.598+0000 7f2d53fff700 1 -- 172.21.15.107:0/1044775004 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f2d4c05bf50 con 0x7f2d5c0fce40 2024-04-22T23:38:11.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.725+0000 7f2d53fff700 1 -- 172.21.15.107:0/1044775004 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d4c094020 con 0x7f2d5c0fce40 2024-04-22T23:38:11.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.832+0000 7f2d6196e700 1 -- 172.21.15.107:0/1044775004 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2d5c10f970 con 0x7f2d4806ed20 2024-04-22T23:38:11.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.833+0000 7f2d53fff700 1 -- 172.21.15.107:0/1044775004 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f2d5c10f970 con 0x7f2d4806ed20 2024-04-22T23:38:11.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.835+0000 7f2d6196e700 1 -- 172.21.15.107:0/1044775004 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f2d4806ed20 msgr2=0x7f2d480711d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:11.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.835+0000 7f2d6196e700 1 --2- 172.21.15.107:0/1044775004 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f2d4806ed20 0x7f2d480711d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2d44009fd0 tx=0x7f2d4400b040 comp rx=0 tx=0).stop 2024-04-22T23:38:11.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.835+0000 7f2d6196e700 1 -- 172.21.15.107:0/1044775004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d5c0fce40 msgr2=0x7f2d5c10da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:11.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.835+0000 7f2d6196e700 1 --2- 172.21.15.107:0/1044775004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d5c0fce40 0x7f2d5c10da90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c00b3b0 tx=0x7f2d4c009e30 comp rx=0 tx=0).stop 2024-04-22T23:38:11.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.835+0000 7f2d6196e700 1 -- 172.21.15.107:0/1044775004 shutdown_connections 2024-04-22T23:38:11.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.835+0000 7f2d6196e700 1 --2- 172.21.15.107:0/1044775004 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f2d4806ed20 0x7f2d480711d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:11.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.835+0000 7f2d6196e700 1 --2- 172.21.15.107:0/1044775004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d5c0fce40 0x7f2d5c10da90 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:11.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.835+0000 7f2d6196e700 1 --2- 172.21.15.107:0/1044775004 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2d5c0fe040 0x7f2d5c10dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:11.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.835+0000 7f2d6196e700 1 -- 172.21.15.107:0/1044775004 >> 172.21.15.107:0/1044775004 conn(0x7f2d5c0f8530 msgr2=0x7f2d5c0f9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:38:11.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.836+0000 7f2d6196e700 1 -- 172.21.15.107:0/1044775004 shutdown_connections 2024-04-22T23:38:11.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.836+0000 7f2d6196e700 1 -- 172.21.15.107:0/1044775004 wait complete. 2024-04-22T23:38:11.840 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-04-22T23:38:11.840 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:11 smithi107 ceph-mon[90179]: pgmap v9: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-22T23:38:11.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.938+0000 7f91f044d700 1 -- 172.21.15.107:0/4135942659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91e80fbd40 msgr2=0x7f91e80fc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:11.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.938+0000 7f91f044d700 1 --2- 172.21.15.107:0/4135942659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91e80fbd40 0x7f91e80fc150 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f91d8004940 tx=0x7f91d8009e30 comp rx=0 tx=0).stop 2024-04-22T23:38:11.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.938+0000 7f91f044d700 1 -- 172.21.15.107:0/4135942659 shutdown_connections 2024-04-22T23:38:11.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.938+0000 7f91f044d700 1 --2- 172.21.15.107:0/4135942659 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91e80fcff0 0x7f91e80fd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:11.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.938+0000 7f91f044d700 1 --2- 172.21.15.107:0/4135942659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91e80fbd40 0x7f91e80fc150 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:11.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.938+0000 7f91f044d700 1 -- 172.21.15.107:0/4135942659 >> 172.21.15.107:0/4135942659 conn(0x7f91e80f74c0 msgr2=0x7f91e80f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:38:11.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.938+0000 7f91f044d700 1 -- 172.21.15.107:0/4135942659 shutdown_connections 2024-04-22T23:38:11.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.938+0000 7f91f044d700 1 -- 172.21.15.107:0/4135942659 wait complete. 2024-04-22T23:38:11.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.939+0000 7f91f044d700 1 Processor -- start 2024-04-22T23:38:11.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.939+0000 7f91f044d700 1 -- start start 2024-04-22T23:38:11.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.939+0000 7f91f044d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91e80fbd40 0x7f91e810b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:11.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.939+0000 7f91f044d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91e80fcff0 0x7f91e810bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:11.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.939+0000 7f91f044d700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e810c350 con 0x7f91e80fcff0 2024-04-22T23:38:11.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.939+0000 7f91f044d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e810cba0 con 0x7f91e80fbd40 2024-04-22T23:38:11.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.939+0000 7f91ed9e8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91e80fcff0 0x7f91e810bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:38:11.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.939+0000 7f91ed9e8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91e80fcff0 0x7f91e810bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35828/0 (socket says 172.21.15.107:35828) 2024-04-22T23:38:11.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.939+0000 7f91ed9e8700 1 -- 172.21.15.107:0/2002034017 learned_addr learned my addr 172.21.15.107:0/2002034017 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:38:11.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.939+0000 7f91ee1e9700 1 --2- 172.21.15.107:0/2002034017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91e80fbd40 0x7f91e810b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:38:11.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.939+0000 7f91ee1e9700 1 -- 172.21.15.107:0/2002034017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91e80fbd40 msgr2=0x7f91e810b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-22T23:38:11.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.939+0000 7f91ee1e9700 1 -- 172.21.15.107:0/2002034017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91e80fbd40 msgr2=0x7f91e810b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-22T23:38:11.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.939+0000 7f91ee1e9700 1 --2- 172.21.15.107:0/2002034017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91e80fbd40 0x7f91e810b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-22T23:38:11.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.939+0000 7f91ed9e8700 1 -- 172.21.15.107:0/2002034017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91e80fbd40 msgr2=0x7f91e810b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:11.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.939+0000 7f91ee1e9700 1 --2- 172.21.15.107:0/2002034017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91e80fbd40 0x7f91e810b810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T23:38:11.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.939+0000 7f91ed9e8700 1 --2- 172.21.15.107:0/2002034017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91e80fbd40 0x7f91e810b810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:11.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.939+0000 7f91ed9e8700 1 -- 172.21.15.107:0/2002034017 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d8005040 con 0x7f91e80fcff0 2024-04-22T23:38:11.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.939+0000 7f91ed9e8700 1 --2- 172.21.15.107:0/2002034017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91e80fcff0 0x7f91e810bd50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f91e4004f20 tx=0x7f91e4004b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:38:11.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.940+0000 7f91deffd700 1 -- 172.21.15.107:0/2002034017 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f91e400fc90 con 0x7f91e80fcff0 2024-04-22T23:38:11.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.940+0000 7f91deffd700 1 -- 172.21.15.107:0/2002034017 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f91e4011040 con 0x7f91e80fcff0 2024-04-22T23:38:11.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.940+0000 7f91deffd700 1 -- 172.21.15.107:0/2002034017 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f91e400b410 con 0x7f91e80fcff0 2024-04-22T23:38:11.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.940+0000 7f91f044d700 1 -- 172.21.15.107:0/2002034017 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91e810ce80 con 0x7f91e80fcff0 2024-04-22T23:38:11.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.940+0000 7f91f044d700 1 -- 172.21.15.107:0/2002034017 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91e810d3c0 con 0x7f91e80fcff0 2024-04-22T23:38:11.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.940+0000 7f91f044d700 1 -- 172.21.15.107:0/2002034017 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91e805f5f0 con 0x7f91e80fcff0 2024-04-22T23:38:11.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.941+0000 7f91deffd700 1 -- 172.21.15.107:0/2002034017 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 36) v1 ==== 92849+0+0 (secure 0 0 0) 0x7f91e400b680 con 0x7f91e80fcff0 2024-04-22T23:38:11.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.941+0000 7f91deffd700 1 --2- 172.21.15.107:0/2002034017 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f91d406ed20 0x7f91d40711d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:11.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.942+0000 7f91deffd700 1 -- 172.21.15.107:0/2002034017 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f91e4090b00 con 0x7f91e80fcff0 2024-04-22T23:38:11.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.942+0000 7f91ee1e9700 1 --2- 172.21.15.107:0/2002034017 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f91d406ed20 0x7f91d40711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:38:11.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.942+0000 7f91ee1e9700 1 --2- 172.21.15.107:0/2002034017 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f91d406ed20 0x7f91d40711d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f91d800b3b0 tx=0x7f91d8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:38:11.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:11.946+0000 7f91deffd700 1 -- 172.21.15.107:0/2002034017 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f91e405d1c0 con 0x7f91e80fcff0 2024-04-22T23:38:12.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:11 smithi157 ceph-mon[79403]: pgmap v9: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-22T23:38:12.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.165+0000 7f91f044d700 1 -- 172.21.15.107:0/2002034017 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f91e810d6f0 con 0x7f91d406ed20 2024-04-22T23:38:12.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.166+0000 7f91deffd700 1 -- 172.21.15.107:0/2002034017 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f91e810d6f0 con 0x7f91d406ed20 2024-04-22T23:38:12.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.168+0000 7f91f044d700 1 -- 172.21.15.107:0/2002034017 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f91d406ed20 msgr2=0x7f91d40711d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:12.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.168+0000 7f91f044d700 1 --2- 172.21.15.107:0/2002034017 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f91d406ed20 0x7f91d40711d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f91d800b3b0 tx=0x7f91d8009e30 comp rx=0 tx=0).stop 2024-04-22T23:38:12.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.168+0000 7f91f044d700 1 -- 172.21.15.107:0/2002034017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91e80fcff0 msgr2=0x7f91e810bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:12.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.168+0000 7f91f044d700 1 --2- 172.21.15.107:0/2002034017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91e80fcff0 0x7f91e810bd50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f91e4004f20 tx=0x7f91e4004b90 comp rx=0 tx=0).stop 2024-04-22T23:38:12.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.168+0000 7f91f044d700 1 -- 172.21.15.107:0/2002034017 shutdown_connections 2024-04-22T23:38:12.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.168+0000 7f91f044d700 1 --2- 172.21.15.107:0/2002034017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91e80fbd40 0x7f91e810b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:12.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.169+0000 7f91f044d700 1 --2- 172.21.15.107:0/2002034017 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f91d406ed20 0x7f91d40711d0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:12.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.169+0000 7f91f044d700 1 --2- 172.21.15.107:0/2002034017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91e80fcff0 0x7f91e810bd50 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:12.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.169+0000 7f91f044d700 1 -- 172.21.15.107:0/2002034017 >> 172.21.15.107:0/2002034017 conn(0x7f91e80f74c0 msgr2=0x7f91e80f8b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:38:12.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.169+0000 7f91f044d700 1 -- 172.21.15.107:0/2002034017 shutdown_connections 2024-04-22T23:38:12.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.169+0000 7f91f044d700 1 -- 172.21.15.107:0/2002034017 wait complete. 2024-04-22T23:38:12.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.270+0000 7f07241f7700 1 -- 172.21.15.107:0/2619380643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f071c0fcff0 msgr2=0x7f071c0fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:12.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.270+0000 7f07241f7700 1 --2- 172.21.15.107:0/2619380643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f071c0fcff0 0x7f071c0fd460 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0718004940 tx=0x7f0718009e30 comp rx=0 tx=0).stop 2024-04-22T23:38:12.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.270+0000 7f07241f7700 1 -- 172.21.15.107:0/2619380643 shutdown_connections 2024-04-22T23:38:12.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.270+0000 7f07241f7700 1 --2- 172.21.15.107:0/2619380643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f071c0fcff0 0x7f071c0fd460 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:12.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.270+0000 7f07241f7700 1 --2- 172.21.15.107:0/2619380643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f071c0fbd40 0x7f071c0fc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:12.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.270+0000 7f07241f7700 1 -- 172.21.15.107:0/2619380643 >> 172.21.15.107:0/2619380643 conn(0x7f071c0f74c0 msgr2=0x7f071c0f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:38:12.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.270+0000 7f07241f7700 1 -- 172.21.15.107:0/2619380643 shutdown_connections 2024-04-22T23:38:12.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.270+0000 7f07241f7700 1 -- 172.21.15.107:0/2619380643 wait complete. 2024-04-22T23:38:12.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.270+0000 7f07241f7700 1 Processor -- start 2024-04-22T23:38:12.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.271+0000 7f07241f7700 1 -- start start 2024-04-22T23:38:12.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.271+0000 7f07241f7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f071c0fbd40 0x7f071c0ffe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:12.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.271+0000 7f07241f7700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f071c0fcff0 0x7f071c0fe330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:12.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.271+0000 7f07241f7700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071c0fe8c0 con 0x7f071c0fbd40 2024-04-22T23:38:12.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.271+0000 7f07241f7700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071c0fea00 con 0x7f071c0fcff0 2024-04-22T23:38:12.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.271+0000 7f0721f93700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f071c0fbd40 0x7f071c0ffe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:38:12.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.271+0000 7f0721f93700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f071c0fbd40 0x7f071c0ffe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35854/0 (socket says 172.21.15.107:35854) 2024-04-22T23:38:12.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.271+0000 7f0721f93700 1 -- 172.21.15.107:0/1233405290 learned_addr learned my addr 172.21.15.107:0/1233405290 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:38:12.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.271+0000 7f0721792700 1 --2- 172.21.15.107:0/1233405290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f071c0fcff0 0x7f071c0fe330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:38:12.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.271+0000 7f0721792700 1 -- 172.21.15.107:0/1233405290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f071c0fcff0 msgr2=0x7f071c0fe330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-22T23:38:12.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.271+0000 7f0721792700 1 -- 172.21.15.107:0/1233405290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f071c0fcff0 msgr2=0x7f071c0fe330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-22T23:38:12.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.271+0000 7f0721792700 1 --2- 172.21.15.107:0/1233405290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f071c0fcff0 0x7f071c0fe330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-22T23:38:12.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.271+0000 7f0721792700 1 --2- 172.21.15.107:0/1233405290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f071c0fcff0 0x7f071c0fe330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T23:38:12.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.271+0000 7f0721f93700 1 -- 172.21.15.107:0/1233405290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f071c0fcff0 msgr2=0x7f071c0fe330 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T23:38:12.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.271+0000 7f0721f93700 1 --2- 172.21.15.107:0/1233405290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f071c0fcff0 0x7f071c0fe330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:12.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.271+0000 7f0721f93700 1 -- 172.21.15.107:0/1233405290 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0718005040 con 0x7f071c0fbd40 2024-04-22T23:38:12.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.271+0000 7f0721f93700 1 --2- 172.21.15.107:0/1233405290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f071c0fbd40 0x7f071c0ffe10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f070c002a50 tx=0x7f070c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:38:12.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.271+0000 7f07127fc700 1 -- 172.21.15.107:0/1233405290 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f070c007cc0 con 0x7f071c0fbd40 2024-04-22T23:38:12.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.271+0000 7f07127fc700 1 -- 172.21.15.107:0/1233405290 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f070c00f040 con 0x7f071c0fbd40 2024-04-22T23:38:12.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.271+0000 7f07241f7700 1 -- 172.21.15.107:0/1233405290 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f071c0fece0 con 0x7f071c0fbd40 2024-04-22T23:38:12.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.271+0000 7f07127fc700 1 -- 172.21.15.107:0/1233405290 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f070c009640 con 0x7f071c0fbd40 2024-04-22T23:38:12.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.272+0000 7f07241f7700 1 -- 172.21.15.107:0/1233405290 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f071c115aa0 con 0x7f071c0fbd40 2024-04-22T23:38:12.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.272+0000 7f07241f7700 1 -- 172.21.15.107:0/1233405290 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f071c101b80 con 0x7f071c0fbd40 2024-04-22T23:38:12.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.273+0000 7f07127fc700 1 -- 172.21.15.107:0/1233405290 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 36) v1 ==== 92849+0+0 (secure 0 0 0) 0x7f070c0097a0 con 0x7f071c0fbd40 2024-04-22T23:38:12.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.273+0000 7f07127fc700 1 --2- 172.21.15.107:0/1233405290 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f070806edf0 0x7f07080712a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:12.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.273+0000 7f07127fc700 1 -- 172.21.15.107:0/1233405290 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f070c08e730 con 0x7f071c0fbd40 2024-04-22T23:38:12.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.273+0000 7f0721792700 1 --2- 172.21.15.107:0/1233405290 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f070806edf0 0x7f07080712a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:38:12.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.274+0000 7f0721792700 1 --2- 172.21.15.107:0/1233405290 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f070806edf0 0x7f07080712a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f071c1003f0 tx=0x7f0718009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:38:12.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.278+0000 7f07127fc700 1 -- 172.21.15.107:0/1233405290 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f070c057400 con 0x7f071c0fbd40 2024-04-22T23:38:12.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.487+0000 7f07241f7700 1 -- 172.21.15.107:0/1233405290 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f071c05e400 con 0x7f070806edf0 2024-04-22T23:38:12.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.497+0000 7f07127fc700 1 -- 172.21.15.107:0/1233405290 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f071c05e400 con 0x7f070806edf0 2024-04-22T23:38:12.491 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-22T23:38:12.491 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (90s) 11s ago 11m 16.3M - 0.25.0 c8568f914cd2 ec1a5c6260de 2024-04-22T23:38:12.491 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (11m) 11s ago 11m 6665k - 17.2.7-884-g70d0cb4b cafd8ddac8fa c10611c756e4 2024-04-22T23:38:12.491 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi157 smithi157 running (10m) 13s ago 10m 6669k - 17.2.7-884-g70d0cb4b cafd8ddac8fa 55d2d46294c9 2024-04-22T23:38:12.491 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (85s) 11s ago 10m 80.1M - 9.4.7 954c08fa6188 31449ff442d2 2024-04-22T23:38:12.491 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.dfvvkw smithi107 running (5m) 11s ago 5m 13.0M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 08d3aa8d717d 2024-04-22T23:38:12.491 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.ktuxjl smithi107 running (5m) 11s ago 5m 124M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 762f27fe0f30 2024-04-22T23:38:12.491 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi157.rcaris smithi157 running (5m) 13s ago 5m 129M - 17.2.7-884-g70d0cb4b cafd8ddac8fa d23366ac085d 2024-04-22T23:38:12.492 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi157.ufywse smithi157 running (5m) 13s ago 5m 13.5M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 8b00d61eb355 2024-04-22T23:38:12.492 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.uipxvm smithi107 *:8443,9283,8765 running (72s) 11s ago 12m 474M - 19.0.0-3252-g43be0201 a0434699c1d7 d20f05dfe4f2 2024-04-22T23:38:12.492 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi157.tuylis smithi157 *:8443,9283,8765 running (39s) 13s ago 10m 448M - 19.0.0-3252-g43be0201 a0434699c1d7 957e598c9001 2024-04-22T23:38:12.492 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (25s) 11s ago 12m 30.1M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 bc72bbbe2c39 2024-04-22T23:38:12.492 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi157 smithi157 starting - - - 2048M 2024-04-22T23:38:12.492 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (97s) 11s ago 11m 8044k - 1.5.0 0da6a335fe13 5205cfd9bd51 2024-04-22T23:38:12.492 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi157 smithi157 *:9100 running (77s) 13s ago 10m 17.2M - 1.5.0 0da6a335fe13 168ca191bb8c 2024-04-22T23:38:12.492 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (9m) 11s ago 9m 231M 1557M 17.2.7-884-g70d0cb4b cafd8ddac8fa 18a179b0a618 2024-04-22T23:38:12.492 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (8m) 11s ago 8m 241M 1557M 17.2.7-884-g70d0cb4b cafd8ddac8fa c7272eb84ddb 2024-04-22T23:38:12.492 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (8m) 11s ago 8m 194M 1557M 17.2.7-884-g70d0cb4b cafd8ddac8fa 9db1788096ad 2024-04-22T23:38:12.492 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi157 running (8m) 13s ago 8m 265M 2581M 17.2.7-884-g70d0cb4b cafd8ddac8fa c9fb2d1a13dd 2024-04-22T23:38:12.492 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi157 running (7m) 13s ago 7m 226M 2581M 17.2.7-884-g70d0cb4b cafd8ddac8fa 24a16e19951f 2024-04-22T23:38:12.492 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi157 running (7m) 13s ago 7m 225M 2581M 17.2.7-884-g70d0cb4b cafd8ddac8fa 4c5b7a7ef8bd 2024-04-22T23:38:12.492 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (45s) 11s ago 10m 53.6M - 2.43.0 a07b618ecd1d 99a4ea8594da 2024-04-22T23:38:12.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.499+0000 7f07241f7700 1 -- 172.21.15.107:0/1233405290 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f070806edf0 msgr2=0x7f07080712a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:12.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.499+0000 7f07241f7700 1 --2- 172.21.15.107:0/1233405290 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f070806edf0 0x7f07080712a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f071c1003f0 tx=0x7f0718009e30 comp rx=0 tx=0).stop 2024-04-22T23:38:12.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.499+0000 7f07241f7700 1 -- 172.21.15.107:0/1233405290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f071c0fbd40 msgr2=0x7f071c0ffe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:12.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.499+0000 7f07241f7700 1 --2- 172.21.15.107:0/1233405290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f071c0fbd40 0x7f071c0ffe10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f070c002a50 tx=0x7f070c0046b0 comp rx=0 tx=0).stop 2024-04-22T23:38:12.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.499+0000 7f07241f7700 1 -- 172.21.15.107:0/1233405290 shutdown_connections 2024-04-22T23:38:12.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.499+0000 7f07241f7700 1 --2- 172.21.15.107:0/1233405290 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f070806edf0 0x7f07080712a0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:12.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.499+0000 7f07241f7700 1 --2- 172.21.15.107:0/1233405290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f071c0fbd40 0x7f071c0ffe10 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:12.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.499+0000 7f07241f7700 1 --2- 172.21.15.107:0/1233405290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f071c0fcff0 0x7f071c0fe330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:12.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.499+0000 7f07241f7700 1 -- 172.21.15.107:0/1233405290 >> 172.21.15.107:0/1233405290 conn(0x7f071c0f74c0 msgr2=0x7f071c065c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:38:12.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.499+0000 7f07241f7700 1 -- 172.21.15.107:0/1233405290 shutdown_connections 2024-04-22T23:38:12.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.499+0000 7f07241f7700 1 -- 172.21.15.107:0/1233405290 wait complete. 2024-04-22T23:38:12.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.597+0000 7f5352bca700 1 -- 172.21.15.107:0/3880696522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f534c0fce80 msgr2=0x7f534c0fd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:12.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.597+0000 7f5352bca700 1 --2- 172.21.15.107:0/3880696522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f534c0fce80 0x7f534c0fd290 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5344009ce0 tx=0x7f5344004d10 comp rx=0 tx=0).stop 2024-04-22T23:38:12.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.597+0000 7f5352bca700 1 -- 172.21.15.107:0/3880696522 shutdown_connections 2024-04-22T23:38:12.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.597+0000 7f5352bca700 1 --2- 172.21.15.107:0/3880696522 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f534c0fdff0 0x7f534c0fe460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:12.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.597+0000 7f5352bca700 1 --2- 172.21.15.107:0/3880696522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f534c0fce80 0x7f534c0fd290 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:12.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.597+0000 7f5352bca700 1 -- 172.21.15.107:0/3880696522 >> 172.21.15.107:0/3880696522 conn(0x7f534c0f8580 msgr2=0x7f534c0fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:38:12.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.597+0000 7f5352bca700 1 -- 172.21.15.107:0/3880696522 shutdown_connections 2024-04-22T23:38:12.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.597+0000 7f5352bca700 1 -- 172.21.15.107:0/3880696522 wait complete. 2024-04-22T23:38:12.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.598+0000 7f5352bca700 1 Processor -- start 2024-04-22T23:38:12.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.598+0000 7f5352bca700 1 -- start start 2024-04-22T23:38:12.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.598+0000 7f5352bca700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f534c0fce80 0x7f534c10dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:12.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.598+0000 7f5352bca700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f534c0fdff0 0x7f534c10e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:12.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.598+0000 7f5352bca700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f534c10e690 con 0x7f534c0fdff0 2024-04-22T23:38:12.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.598+0000 7f5352bca700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f534c113030 con 0x7f534c0fce80 2024-04-22T23:38:12.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.598+0000 7f534bfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f534c0fdff0 0x7f534c10e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:38:12.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.598+0000 7f534bfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f534c0fdff0 0x7f534c10e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35882/0 (socket says 172.21.15.107:35882) 2024-04-22T23:38:12.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.598+0000 7f534bfff700 1 -- 172.21.15.107:0/278924692 learned_addr learned my addr 172.21.15.107:0/278924692 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:38:12.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.598+0000 7f5350966700 1 --2- 172.21.15.107:0/278924692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f534c0fce80 0x7f534c10dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:38:12.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.598+0000 7f5350966700 1 -- 172.21.15.107:0/278924692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f534c0fce80 msgr2=0x7f534c10dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-22T23:38:12.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.598+0000 7f5350966700 1 -- 172.21.15.107:0/278924692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f534c0fce80 msgr2=0x7f534c10dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-22T23:38:12.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.598+0000 7f5350966700 1 --2- 172.21.15.107:0/278924692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f534c0fce80 0x7f534c10dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-22T23:38:12.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.599+0000 7f534bfff700 1 -- 172.21.15.107:0/278924692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f534c0fce80 msgr2=0x7f534c10dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:12.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.599+0000 7f5350966700 1 --2- 172.21.15.107:0/278924692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f534c0fce80 0x7f534c10dac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T23:38:12.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.599+0000 7f534bfff700 1 --2- 172.21.15.107:0/278924692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f534c0fce80 0x7f534c10dac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:12.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.599+0000 7f534bfff700 1 -- 172.21.15.107:0/278924692 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5344009940 con 0x7f534c0fdff0 2024-04-22T23:38:12.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.599+0000 7f534bfff700 1 --2- 172.21.15.107:0/278924692 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f534c0fdff0 0x7f534c10e000 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f534000b790 tx=0x7f5340009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:38:12.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.599+0000 7f53497fa700 1 -- 172.21.15.107:0/278924692 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f534000f770 con 0x7f534c0fdff0 2024-04-22T23:38:12.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.599+0000 7f53497fa700 1 -- 172.21.15.107:0/278924692 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f534000fda0 con 0x7f534c0fdff0 2024-04-22T23:38:12.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.599+0000 7f53497fa700 1 -- 172.21.15.107:0/278924692 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f534000d310 con 0x7f534c0fdff0 2024-04-22T23:38:12.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.599+0000 7f5352bca700 1 -- 172.21.15.107:0/278924692 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f534c113290 con 0x7f534c0fdff0 2024-04-22T23:38:12.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.599+0000 7f5352bca700 1 -- 172.21.15.107:0/278924692 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f534c113800 con 0x7f534c0fdff0 2024-04-22T23:38:12.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.600+0000 7f5352bca700 1 -- 172.21.15.107:0/278924692 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f534c04c710 con 0x7f534c0fdff0 2024-04-22T23:38:12.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.600+0000 7f53497fa700 1 -- 172.21.15.107:0/278924692 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 36) v1 ==== 92849+0+0 (secure 0 0 0) 0x7f5340010420 con 0x7f534c0fdff0 2024-04-22T23:38:12.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.601+0000 7f53497fa700 1 --2- 172.21.15.107:0/278924692 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f533406ed30 0x7f53340711e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:12.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.601+0000 7f53497fa700 1 -- 172.21.15.107:0/278924692 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f534008e930 con 0x7f534c0fdff0 2024-04-22T23:38:12.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.601+0000 7f5350966700 1 --2- 172.21.15.107:0/278924692 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f533406ed30 0x7f53340711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:38:12.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.601+0000 7f5350966700 1 --2- 172.21.15.107:0/278924692 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f533406ed30 0x7f53340711e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f53440095a0 tx=0x7f5344004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:38:12.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.606+0000 7f53497fa700 1 -- 172.21.15.107:0/278924692 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f534005b240 con 0x7f534c0fdff0 2024-04-22T23:38:12.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:12 smithi107 ceph-mon[90179]: pgmap v11: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-22T23:38:12.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:12 smithi107 ceph-mon[90179]: 1: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.smithi157 2024-04-22T23:38:12.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:12 smithi107 ceph-mon[90179]: fsmap cephfs:2 {0=cephfs.smithi107.ktuxjl=up:active,1=cephfs.smithi157.rcaris=up:active} 2 up:standby 2024-04-22T23:38:12.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:12 smithi107 ceph-mon[90179]: osdmap e49: 6 total, 6 up, 6 in 2024-04-22T23:38:12.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:12 smithi107 ceph-mon[90179]: mgrmap e36: smithi107.uipxvm(active, since 16s), standbys: smithi157.tuylis 2024-04-22T23:38:12.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:12 smithi107 ceph-mon[90179]: overall HEALTH_OK 2024-04-22T23:38:12.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:12 smithi107 ceph-mon[90179]: mon.smithi107 calling monitor election 2024-04-22T23:38:12.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:12 smithi107 ceph-mon[90179]: mon.smithi157 calling monitor election 2024-04-22T23:38:12.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:12 smithi107 ceph-mon[90179]: mon.smithi107 is new leader, mons smithi107,smithi157 in quorum (ranks 0,1) 2024-04-22T23:38:12.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:12 smithi107 ceph-mon[90179]: monmap epoch 3 2024-04-22T23:38:12.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:12 smithi107 ceph-mon[90179]: fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 2024-04-22T23:38:12.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:12 smithi107 ceph-mon[90179]: last_changed 2024-04-22T23:38:11.723566+0000 2024-04-22T23:38:12.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:12 smithi107 ceph-mon[90179]: created 2024-04-22T23:25:22.712092+0000 2024-04-22T23:38:12.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:12 smithi107 ceph-mon[90179]: min_mon_release 19 (squid) 2024-04-22T23:38:12.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:12 smithi107 ceph-mon[90179]: election_strategy: 1 2024-04-22T23:38:12.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:12 smithi107 ceph-mon[90179]: 0: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.smithi107 2024-04-22T23:38:12.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:12 smithi107 ceph-mon[90179]: 1: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.smithi157 2024-04-22T23:38:12.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:12 smithi107 ceph-mon[90179]: fsmap cephfs:2 {0=cephfs.smithi107.ktuxjl=up:active,1=cephfs.smithi157.rcaris=up:active} 2 up:standby 2024-04-22T23:38:12.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:12 smithi107 ceph-mon[90179]: osdmap e49: 6 total, 6 up, 6 in 2024-04-22T23:38:12.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:12 smithi107 ceph-mon[90179]: mgrmap e36: smithi107.uipxvm(active, since 16s), standbys: smithi157.tuylis 2024-04-22T23:38:12.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:12 smithi107 ceph-mon[90179]: overall HEALTH_OK 2024-04-22T23:38:12.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:12 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:12.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:12 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-04-22T23:38:12.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:12 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mon metadata", "id": "smithi157"}]: dispatch 2024-04-22T23:38:12.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:12 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:12.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:12 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:38:12.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.894+0000 7f5352bca700 1 -- 172.21.15.107:0/278924692 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f534c105960 con 0x7f534c0fdff0 2024-04-22T23:38:12.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.894+0000 7f53497fa700 1 -- 172.21.15.107:0/278924692 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+736 (secure 0 0 0) 0x7f534005add0 con 0x7f534c0fdff0 2024-04-22T23:38:12.888 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-22T23:38:12.888 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-04-22T23:38:12.889 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:38:12.889 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:38:12.889 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-04-22T23:38:12.889 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:38:12.889 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:38:12.889 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-04-22T23:38:12.889 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 6 2024-04-22T23:38:12.889 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:38:12.889 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-04-22T23:38:12.889 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 4 2024-04-22T23:38:12.889 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:38:12.889 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-04-22T23:38:12.889 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 10, 2024-04-22T23:38:12.890 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 4 2024-04-22T23:38:12.890 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T23:38:12.890 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-22T23:38:12.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.896+0000 7f5352bca700 1 -- 172.21.15.107:0/278924692 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f533406ed30 msgr2=0x7f53340711e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:12.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.896+0000 7f5352bca700 1 --2- 172.21.15.107:0/278924692 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f533406ed30 0x7f53340711e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f53440095a0 tx=0x7f5344004d10 comp rx=0 tx=0).stop 2024-04-22T23:38:12.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.896+0000 7f5352bca700 1 -- 172.21.15.107:0/278924692 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f534c0fdff0 msgr2=0x7f534c10e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:12.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.896+0000 7f5352bca700 1 --2- 172.21.15.107:0/278924692 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f534c0fdff0 0x7f534c10e000 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f534000b790 tx=0x7f5340009250 comp rx=0 tx=0).stop 2024-04-22T23:38:12.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.897+0000 7f5352bca700 1 -- 172.21.15.107:0/278924692 shutdown_connections 2024-04-22T23:38:12.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.897+0000 7f5352bca700 1 --2- 172.21.15.107:0/278924692 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f534c0fce80 0x7f534c10dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:12.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.897+0000 7f5352bca700 1 --2- 172.21.15.107:0/278924692 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f533406ed30 0x7f53340711e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:12.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.897+0000 7f5352bca700 1 --2- 172.21.15.107:0/278924692 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f534c0fdff0 0x7f534c10e000 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:12.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.897+0000 7f5352bca700 1 -- 172.21.15.107:0/278924692 >> 172.21.15.107:0/278924692 conn(0x7f534c0f8580 msgr2=0x7f534c0f9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:38:12.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.897+0000 7f5352bca700 1 -- 172.21.15.107:0/278924692 shutdown_connections 2024-04-22T23:38:12.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.897+0000 7f5352bca700 1 -- 172.21.15.107:0/278924692 wait complete. 2024-04-22T23:38:12.897 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:12 smithi157 ceph-mon[79403]: pgmap v11: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-22T23:38:12.897 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:12 smithi157 ceph-mon[79403]: 1: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.smithi157 2024-04-22T23:38:12.897 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:12 smithi157 ceph-mon[79403]: fsmap cephfs:2 {0=cephfs.smithi107.ktuxjl=up:active,1=cephfs.smithi157.rcaris=up:active} 2 up:standby 2024-04-22T23:38:12.897 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:12 smithi157 ceph-mon[79403]: osdmap e49: 6 total, 6 up, 6 in 2024-04-22T23:38:12.897 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:12 smithi157 ceph-mon[79403]: mgrmap e36: smithi107.uipxvm(active, since 16s), standbys: smithi157.tuylis 2024-04-22T23:38:12.897 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:12 smithi157 ceph-mon[79403]: overall HEALTH_OK 2024-04-22T23:38:12.897 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:12 smithi157 ceph-mon[79403]: mon.smithi107 calling monitor election 2024-04-22T23:38:12.898 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:12 smithi157 ceph-mon[79403]: mon.smithi157 calling monitor election 2024-04-22T23:38:12.898 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:12 smithi157 ceph-mon[79403]: mon.smithi107 is new leader, mons smithi107,smithi157 in quorum (ranks 0,1) 2024-04-22T23:38:12.898 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:12 smithi157 ceph-mon[79403]: monmap epoch 3 2024-04-22T23:38:12.898 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:12 smithi157 ceph-mon[79403]: fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 2024-04-22T23:38:12.898 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:12 smithi157 ceph-mon[79403]: last_changed 2024-04-22T23:38:11.723566+0000 2024-04-22T23:38:12.898 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:12 smithi157 ceph-mon[79403]: created 2024-04-22T23:25:22.712092+0000 2024-04-22T23:38:12.898 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:12 smithi157 ceph-mon[79403]: min_mon_release 19 (squid) 2024-04-22T23:38:12.898 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:12 smithi157 ceph-mon[79403]: election_strategy: 1 2024-04-22T23:38:12.898 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:12 smithi157 ceph-mon[79403]: 0: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.smithi107 2024-04-22T23:38:12.898 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:12 smithi157 ceph-mon[79403]: 1: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.smithi157 2024-04-22T23:38:12.898 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:12 smithi157 ceph-mon[79403]: fsmap cephfs:2 {0=cephfs.smithi107.ktuxjl=up:active,1=cephfs.smithi157.rcaris=up:active} 2 up:standby 2024-04-22T23:38:12.898 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:12 smithi157 ceph-mon[79403]: osdmap e49: 6 total, 6 up, 6 in 2024-04-22T23:38:12.898 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:12 smithi157 ceph-mon[79403]: mgrmap e36: smithi107.uipxvm(active, since 16s), standbys: smithi157.tuylis 2024-04-22T23:38:12.899 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:12 smithi157 ceph-mon[79403]: overall HEALTH_OK 2024-04-22T23:38:12.899 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:12 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:12.899 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:12 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mon metadata", "id": "smithi107"}]: dispatch 2024-04-22T23:38:12.899 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:12 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mon metadata", "id": "smithi157"}]: dispatch 2024-04-22T23:38:12.899 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:12 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:12.899 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:12 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:38:12.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.996+0000 7f29ccf11700 1 -- 172.21.15.107:0/1792544270 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29c80fe450 msgr2=0x7f29c80fe860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:12.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.996+0000 7f29ccf11700 1 --2- 172.21.15.107:0/1792544270 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29c80fe450 0x7f29c80fe860 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f29b8004940 tx=0x7f29b8009e30 comp rx=0 tx=0).stop 2024-04-22T23:38:12.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.996+0000 7f29ccf11700 1 -- 172.21.15.107:0/1792544270 shutdown_connections 2024-04-22T23:38:12.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.996+0000 7f29ccf11700 1 --2- 172.21.15.107:0/1792544270 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29c80feda0 0x7f29c80faee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:12.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.996+0000 7f29ccf11700 1 --2- 172.21.15.107:0/1792544270 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29c80fe450 0x7f29c80fe860 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:12.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.996+0000 7f29ccf11700 1 -- 172.21.15.107:0/1792544270 >> 172.21.15.107:0/1792544270 conn(0x7f29c80f6bf0 msgr2=0x7f29c80f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:38:12.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.996+0000 7f29ccf11700 1 -- 172.21.15.107:0/1792544270 shutdown_connections 2024-04-22T23:38:12.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.996+0000 7f29ccf11700 1 -- 172.21.15.107:0/1792544270 wait complete. 2024-04-22T23:38:12.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.997+0000 7f29ccf11700 1 Processor -- start 2024-04-22T23:38:12.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.997+0000 7f29ccf11700 1 -- start start 2024-04-22T23:38:12.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.997+0000 7f29ccf11700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29c80fe450 0x7f29c810d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:12.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.997+0000 7f29ccf11700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29c80feda0 0x7f29c810de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:12.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.997+0000 7f29ccf11700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c810e460 con 0x7f29c80feda0 2024-04-22T23:38:12.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.997+0000 7f29ccf11700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c810e5d0 con 0x7f29c80fe450 2024-04-22T23:38:12.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.997+0000 7f29c5d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29c80feda0 0x7f29c810de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:38:12.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.997+0000 7f29c5d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29c80feda0 0x7f29c810de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35898/0 (socket says 172.21.15.107:35898) 2024-04-22T23:38:12.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.997+0000 7f29c5d9b700 1 -- 172.21.15.107:0/2466553983 learned_addr learned my addr 172.21.15.107:0/2466553983 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:38:12.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.997+0000 7f29c659c700 1 --2- 172.21.15.107:0/2466553983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29c80fe450 0x7f29c810d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:38:12.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.998+0000 7f29c5d9b700 1 -- 172.21.15.107:0/2466553983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29c80fe450 msgr2=0x7f29c810d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:12.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.998+0000 7f29c5d9b700 1 --2- 172.21.15.107:0/2466553983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29c80fe450 0x7f29c810d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:12.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.998+0000 7f29c5d9b700 1 -- 172.21.15.107:0/2466553983 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b8005040 con 0x7f29c80feda0 2024-04-22T23:38:12.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.998+0000 7f29c5d9b700 1 --2- 172.21.15.107:0/2466553983 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29c80feda0 0x7f29c810de60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f29bc00c830 tx=0x7f29bc004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:38:12.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.998+0000 7f29b6ffd700 1 -- 172.21.15.107:0/2466553983 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f29bc00a760 con 0x7f29c80feda0 2024-04-22T23:38:12.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.998+0000 7f29b6ffd700 1 -- 172.21.15.107:0/2466553983 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f29bc00ad90 con 0x7f29c80feda0 2024-04-22T23:38:12.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.998+0000 7f29ccf11700 1 -- 172.21.15.107:0/2466553983 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29c8113030 con 0x7f29c80feda0 2024-04-22T23:38:12.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.998+0000 7f29b6ffd700 1 -- 172.21.15.107:0/2466553983 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f29bc0113a0 con 0x7f29c80feda0 2024-04-22T23:38:12.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.998+0000 7f29ccf11700 1 -- 172.21.15.107:0/2466553983 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29c81135c0 con 0x7f29c80feda0 2024-04-22T23:38:12.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.999+0000 7f29ccf11700 1 -- 172.21.15.107:0/2466553983 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29c804c710 con 0x7f29c80feda0 2024-04-22T23:38:12.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:12.999+0000 7f29b6ffd700 1 -- 172.21.15.107:0/2466553983 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 36) v1 ==== 92849+0+0 (secure 0 0 0) 0x7f29bc012400 con 0x7f29c80feda0 2024-04-22T23:38:12.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.000+0000 7f29b6ffd700 1 --2- 172.21.15.107:0/2466553983 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f29b006ed30 0x7f29b00711e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:12.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.000+0000 7f29b6ffd700 1 -- 172.21.15.107:0/2466553983 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f29bc08e950 con 0x7f29c80feda0 2024-04-22T23:38:12.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.000+0000 7f29c659c700 1 --2- 172.21.15.107:0/2466553983 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f29b006ed30 0x7f29b00711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:38:12.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.000+0000 7f29c659c700 1 --2- 172.21.15.107:0/2466553983 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f29b006ed30 0x7f29b00711e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f29b8004f90 tx=0x7f29b8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:38:12.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.005+0000 7f29b6ffd700 1 -- 172.21.15.107:0/2466553983 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f29bc05b260 con 0x7f29c80feda0 2024-04-22T23:38:13.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.251+0000 7f29ccf11700 1 -- 172.21.15.107:0/2466553983 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f29c8105900 con 0x7f29c80feda0 2024-04-22T23:38:13.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.252+0000 7f29b6ffd700 1 -- 172.21.15.107:0/2466553983 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) v1 ==== 76+0+1928 (secure 0 0 0) 0x7f29bc05adf0 con 0x7f29c80feda0 2024-04-22T23:38:13.246 INFO:teuthology.orchestra.run.smithi107.stdout:e19 2024-04-22T23:38:13.246 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T23:38:13.246 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:38:13.246 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-04-22T23:38:13.246 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:38:13.246 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-04-22T23:38:13.246 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-04-22T23:38:13.246 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 19 2024-04-22T23:38:13.246 INFO:teuthology.orchestra.run.smithi107.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-22T23:38:13.246 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-04-22T23:32:20.146130+0000 2024-04-22T23:38:13.246 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-04-22T23:35:12.338678+0000 2024-04-22T23:38:13.247 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-04-22T23:38:13.247 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-04-22T23:38:13.247 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-04-22T23:38:13.247 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-04-22T23:38:13.247 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-04-22T23:38:13.247 INFO:teuthology.orchestra.run.smithi107.stdout:max_xattr_size 65536 2024-04-22T23:38:13.247 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-04-22T23:38:13.247 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-04-22T23:38:13.247 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 46 2024-04-22T23:38:13.247 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:38:13.247 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 2 2024-04-22T23:38:13.247 INFO:teuthology.orchestra.run.smithi107.stdout:in 0,1 2024-04-22T23:38:13.247 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=14468,1=24265} 2024-04-22T23:38:13.247 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-04-22T23:38:13.247 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-04-22T23:38:13.248 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-04-22T23:38:13.248 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-04-22T23:38:13.248 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-04-22T23:38:13.248 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data disabled 2024-04-22T23:38:13.248 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-04-22T23:38:13.248 INFO:teuthology.orchestra.run.smithi107.stdout:bal_rank_mask -1 2024-04-22T23:38:13.248 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-04-22T23:38:13.248 INFO:teuthology.orchestra.run.smithi107.stdout:qdb_cluster leader: 0 members: 2024-04-22T23:38:13.248 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.ktuxjl{0:14468} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.107:6828/3335313253,v1:172.21.15.107:6829/3335313253] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:38:13.248 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi157.rcaris{1:24265} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.157:6824/2779251893,v1:172.21.15.157:6825/2779251893] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:38:13.248 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:38:13.248 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:38:13.248 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-04-22T23:38:13.248 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:38:13.248 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.dfvvkw{-1:14484} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/3548717247,v1:172.21.15.107:6827/3548717247] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:38:13.249 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi157.ufywse{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.157:6826/4072444199,v1:172.21.15.157:6827/4072444199] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:38:13.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.255+0000 7f29ccf11700 1 -- 172.21.15.107:0/2466553983 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f29b006ed30 msgr2=0x7f29b00711e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:13.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.255+0000 7f29ccf11700 1 --2- 172.21.15.107:0/2466553983 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f29b006ed30 0x7f29b00711e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f29b8004f90 tx=0x7f29b8009e30 comp rx=0 tx=0).stop 2024-04-22T23:38:13.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.255+0000 7f29ccf11700 1 -- 172.21.15.107:0/2466553983 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29c80feda0 msgr2=0x7f29c810de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:13.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.255+0000 7f29ccf11700 1 --2- 172.21.15.107:0/2466553983 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29c80feda0 0x7f29c810de60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f29bc00c830 tx=0x7f29bc004470 comp rx=0 tx=0).stop 2024-04-22T23:38:13.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.255+0000 7f29ccf11700 1 -- 172.21.15.107:0/2466553983 shutdown_connections 2024-04-22T23:38:13.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.255+0000 7f29ccf11700 1 --2- 172.21.15.107:0/2466553983 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29c80fe450 0x7f29c810d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:13.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.255+0000 7f29ccf11700 1 --2- 172.21.15.107:0/2466553983 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f29b006ed30 0x7f29b00711e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:13.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.255+0000 7f29ccf11700 1 --2- 172.21.15.107:0/2466553983 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29c80feda0 0x7f29c810de60 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:13.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.255+0000 7f29ccf11700 1 -- 172.21.15.107:0/2466553983 >> 172.21.15.107:0/2466553983 conn(0x7f29c80f6bf0 msgr2=0x7f29c80f82c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:38:13.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.255+0000 7f29ccf11700 1 -- 172.21.15.107:0/2466553983 shutdown_connections 2024-04-22T23:38:13.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.255+0000 7f29ccf11700 1 -- 172.21.15.107:0/2466553983 wait complete. 2024-04-22T23:38:13.250 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 19 2024-04-22T23:38:13.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.355+0000 7fb9f79bb700 1 -- 172.21.15.107:0/985275862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9f0066230 msgr2=0x7fb9f00ffaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:13.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.355+0000 7fb9f79bb700 1 --2- 172.21.15.107:0/985275862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9f0066230 0x7fb9f00ffaf0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec004940 tx=0x7fb9ec009e30 comp rx=0 tx=0).stop 2024-04-22T23:38:13.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.355+0000 7fb9f79bb700 1 -- 172.21.15.107:0/985275862 shutdown_connections 2024-04-22T23:38:13.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.355+0000 7fb9f79bb700 1 --2- 172.21.15.107:0/985275862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9f0100030 0x7fb9f0102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:13.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.355+0000 7fb9f79bb700 1 --2- 172.21.15.107:0/985275862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9f0066230 0x7fb9f00ffaf0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:13.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.355+0000 7fb9f79bb700 1 -- 172.21.15.107:0/985275862 >> 172.21.15.107:0/985275862 conn(0x7fb9f00f74c0 msgr2=0x7fb9f00f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:38:13.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.356+0000 7fb9f79bb700 1 -- 172.21.15.107:0/985275862 shutdown_connections 2024-04-22T23:38:13.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.356+0000 7fb9f79bb700 1 -- 172.21.15.107:0/985275862 wait complete. 2024-04-22T23:38:13.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.356+0000 7fb9f79bb700 1 Processor -- start 2024-04-22T23:38:13.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.356+0000 7fb9f79bb700 1 -- start start 2024-04-22T23:38:13.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.356+0000 7fb9f79bb700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9f0066230 0x7fb9f010dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:13.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.356+0000 7fb9f79bb700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9f0100030 0x7fb9f010e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:13.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.356+0000 7fb9f79bb700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f010e600 con 0x7fb9f0066230 2024-04-22T23:38:13.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.356+0000 7fb9f79bb700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f010ee50 con 0x7fb9f0100030 2024-04-22T23:38:13.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.356+0000 7fb9f5757700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9f0066230 0x7fb9f010dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:38:13.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.356+0000 7fb9f5757700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9f0066230 0x7fb9f010dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35922/0 (socket says 172.21.15.107:35922) 2024-04-22T23:38:13.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.356+0000 7fb9f5757700 1 -- 172.21.15.107:0/4060211108 learned_addr learned my addr 172.21.15.107:0/4060211108 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:38:13.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.357+0000 7fb9f4f56700 1 --2- 172.21.15.107:0/4060211108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9f0100030 0x7fb9f010e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:38:13.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.357+0000 7fb9f5757700 1 -- 172.21.15.107:0/4060211108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9f0100030 msgr2=0x7fb9f010e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:13.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.357+0000 7fb9f5757700 1 --2- 172.21.15.107:0/4060211108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9f0100030 0x7fb9f010e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:13.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.357+0000 7fb9f5757700 1 -- 172.21.15.107:0/4060211108 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9ec005040 con 0x7fb9f0066230 2024-04-22T23:38:13.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.357+0000 7fb9f4f56700 1 --2- 172.21.15.107:0/4060211108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9f0100030 0x7fb9f010e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T23:38:13.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.357+0000 7fb9f5757700 1 --2- 172.21.15.107:0/4060211108 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9f0066230 0x7fb9f010dac0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec000c00 tx=0x7fb9ec009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:38:13.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.357+0000 7fb9e5ffb700 1 -- 172.21.15.107:0/4060211108 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9ec00e040 con 0x7fb9f0066230 2024-04-22T23:38:13.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.357+0000 7fb9f79bb700 1 -- 172.21.15.107:0/4060211108 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9f010f0d0 con 0x7fb9f0066230 2024-04-22T23:38:13.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.357+0000 7fb9e5ffb700 1 -- 172.21.15.107:0/4060211108 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb9ec00bcc0 con 0x7fb9f0066230 2024-04-22T23:38:13.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.357+0000 7fb9e5ffb700 1 -- 172.21.15.107:0/4060211108 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9ec01c630 con 0x7fb9f0066230 2024-04-22T23:38:13.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.357+0000 7fb9f79bb700 1 -- 172.21.15.107:0/4060211108 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9f010f610 con 0x7fb9f0066230 2024-04-22T23:38:13.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.358+0000 7fb9f79bb700 1 -- 172.21.15.107:0/4060211108 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9f005f5f0 con 0x7fb9f0066230 2024-04-22T23:38:13.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.359+0000 7fb9e5ffb700 1 -- 172.21.15.107:0/4060211108 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 36) v1 ==== 92849+0+0 (secure 0 0 0) 0x7fb9ec013980 con 0x7fb9f0066230 2024-04-22T23:38:13.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.359+0000 7fb9e5ffb700 1 --2- 172.21.15.107:0/4060211108 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fb9dc06ed20 0x7fb9dc0711d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:13.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.359+0000 7fb9e5ffb700 1 -- 172.21.15.107:0/4060211108 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fb9ec08fed0 con 0x7fb9f0066230 2024-04-22T23:38:13.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.359+0000 7fb9f4f56700 1 --2- 172.21.15.107:0/4060211108 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fb9dc06ed20 0x7fb9dc0711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:38:13.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.360+0000 7fb9f4f56700 1 --2- 172.21.15.107:0/4060211108 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fb9dc06ed20 0x7fb9dc0711d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb9e0007fd0 tx=0x7fb9e0009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:38:13.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.364+0000 7fb9e5ffb700 1 -- 172.21.15.107:0/4060211108 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fb9ec05c610 con 0x7fb9f0066230 2024-04-22T23:38:13.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.584+0000 7fb9f79bb700 1 -- 172.21.15.107:0/4060211108 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb9f010f8d0 con 0x7fb9dc06ed20 2024-04-22T23:38:13.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.585+0000 7fb9e5ffb700 1 -- 172.21.15.107:0/4060211108 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fb9f010f8d0 con 0x7fb9dc06ed20 2024-04-22T23:38:13.579 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-22T23:38:13.579 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c", 2024-04-22T23:38:13.579 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-04-22T23:38:13.579 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-22T23:38:13.579 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [ 2024-04-22T23:38:13.580 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr" 2024-04-22T23:38:13.580 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-22T23:38:13.580 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "3/21 daemons upgraded", 2024-04-22T23:38:13.580 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Currently upgrading mon daemons", 2024-04-22T23:38:13.580 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-04-22T23:38:13.580 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-22T23:38:13.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.587+0000 7fb9f79bb700 1 -- 172.21.15.107:0/4060211108 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fb9dc06ed20 msgr2=0x7fb9dc0711d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:13.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.587+0000 7fb9f79bb700 1 --2- 172.21.15.107:0/4060211108 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fb9dc06ed20 0x7fb9dc0711d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb9e0007fd0 tx=0x7fb9e0009040 comp rx=0 tx=0).stop 2024-04-22T23:38:13.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.587+0000 7fb9f79bb700 1 -- 172.21.15.107:0/4060211108 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9f0066230 msgr2=0x7fb9f010dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:13.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.587+0000 7fb9f79bb700 1 --2- 172.21.15.107:0/4060211108 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9f0066230 0x7fb9f010dac0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec000c00 tx=0x7fb9ec009e30 comp rx=0 tx=0).stop 2024-04-22T23:38:13.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.587+0000 7fb9f79bb700 1 -- 172.21.15.107:0/4060211108 shutdown_connections 2024-04-22T23:38:13.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.587+0000 7fb9f79bb700 1 --2- 172.21.15.107:0/4060211108 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fb9dc06ed20 0x7fb9dc0711d0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:13.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.587+0000 7fb9f79bb700 1 --2- 172.21.15.107:0/4060211108 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9f0066230 0x7fb9f010dac0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:13.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.587+0000 7fb9f79bb700 1 --2- 172.21.15.107:0/4060211108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9f0100030 0x7fb9f010e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:13.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.587+0000 7fb9f79bb700 1 -- 172.21.15.107:0/4060211108 >> 172.21.15.107:0/4060211108 conn(0x7fb9f00f74c0 msgr2=0x7fb9f00f7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:38:13.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.588+0000 7fb9f79bb700 1 -- 172.21.15.107:0/4060211108 shutdown_connections 2024-04-22T23:38:13.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.588+0000 7fb9f79bb700 1 -- 172.21.15.107:0/4060211108 wait complete. 2024-04-22T23:38:13.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.689+0000 7ffaf6284700 1 -- 172.21.15.107:0/308251383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffaf0071e20 msgr2=0x7ffaf0070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:13.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.689+0000 7ffaf6284700 1 --2- 172.21.15.107:0/308251383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffaf0071e20 0x7ffaf0070330 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ffad8004940 tx=0x7ffad8009e30 comp rx=0 tx=0).stop 2024-04-22T23:38:13.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.689+0000 7ffaf6284700 1 -- 172.21.15.107:0/308251383 shutdown_connections 2024-04-22T23:38:13.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.689+0000 7ffaf6284700 1 --2- 172.21.15.107:0/308251383 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffaf0070870 0x7ffaf0070ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:13.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.689+0000 7ffaf6284700 1 --2- 172.21.15.107:0/308251383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffaf0071e20 0x7ffaf0070330 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:13.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.689+0000 7ffaf6284700 1 -- 172.21.15.107:0/308251383 >> 172.21.15.107:0/308251383 conn(0x7ffaf00f8580 msgr2=0x7ffaf00fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:38:13.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.689+0000 7ffaf6284700 1 -- 172.21.15.107:0/308251383 shutdown_connections 2024-04-22T23:38:13.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.689+0000 7ffaf6284700 1 -- 172.21.15.107:0/308251383 wait complete. 2024-04-22T23:38:13.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.689+0000 7ffaf6284700 1 Processor -- start 2024-04-22T23:38:13.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.689+0000 7ffaf6284700 1 -- start start 2024-04-22T23:38:13.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.690+0000 7ffaf6284700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffaf0070870 0x7ffaf010fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:13.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.690+0000 7ffaf6284700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffaf0071e20 0x7ffaf01100e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:13.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.690+0000 7ffaf6284700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf01106e0 con 0x7ffaf0071e20 2024-04-22T23:38:13.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.690+0000 7ffaf6284700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf0110850 con 0x7ffaf0070870 2024-04-22T23:38:13.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.690+0000 7ffaef7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffaf0071e20 0x7ffaf01100e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:38:13.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.690+0000 7ffaef7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffaf0071e20 0x7ffaf01100e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35940/0 (socket says 172.21.15.107:35940) 2024-04-22T23:38:13.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.690+0000 7ffaef7fe700 1 -- 172.21.15.107:0/3797463391 learned_addr learned my addr 172.21.15.107:0/3797463391 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:38:13.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.690+0000 7ffaeffff700 1 --2- 172.21.15.107:0/3797463391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffaf0070870 0x7ffaf010fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:38:13.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.690+0000 7ffaeffff700 1 -- 172.21.15.107:0/3797463391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffaf0070870 msgr2=0x7ffaf010fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-22T23:38:13.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.690+0000 7ffaeffff700 1 -- 172.21.15.107:0/3797463391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffaf0070870 msgr2=0x7ffaf010fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-22T23:38:13.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.690+0000 7ffaeffff700 1 --2- 172.21.15.107:0/3797463391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffaf0070870 0x7ffaf010fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-22T23:38:13.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.690+0000 7ffaeffff700 1 --2- 172.21.15.107:0/3797463391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffaf0070870 0x7ffaf010fba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T23:38:13.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.690+0000 7ffaef7fe700 1 -- 172.21.15.107:0/3797463391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffaf0070870 msgr2=0x7ffaf010fba0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T23:38:13.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.690+0000 7ffaef7fe700 1 --2- 172.21.15.107:0/3797463391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffaf0070870 0x7ffaf010fba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:13.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.690+0000 7ffaef7fe700 1 -- 172.21.15.107:0/3797463391 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad8005040 con 0x7ffaf0071e20 2024-04-22T23:38:13.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.690+0000 7ffaef7fe700 1 --2- 172.21.15.107:0/3797463391 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffaf0071e20 0x7ffaf01100e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ffae000c8c0 tx=0x7ffae00044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:38:13.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.690+0000 7ffaecff9700 1 -- 172.21.15.107:0/3797463391 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffae000a980 con 0x7ffaf0071e20 2024-04-22T23:38:13.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.690+0000 7ffaf6284700 1 -- 172.21.15.107:0/3797463391 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaf01152b0 con 0x7ffaf0071e20 2024-04-22T23:38:13.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.690+0000 7ffaecff9700 1 -- 172.21.15.107:0/3797463391 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffae000ce60 con 0x7ffaf0071e20 2024-04-22T23:38:13.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.690+0000 7ffaecff9700 1 -- 172.21.15.107:0/3797463391 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffae00115e0 con 0x7ffaf0071e20 2024-04-22T23:38:13.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.690+0000 7ffaf6284700 1 -- 172.21.15.107:0/3797463391 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaf0115870 con 0x7ffaf0071e20 2024-04-22T23:38:13.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.691+0000 7ffaf6284700 1 -- 172.21.15.107:0/3797463391 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaf004c710 con 0x7ffaf0071e20 2024-04-22T23:38:13.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.692+0000 7ffaecff9700 1 -- 172.21.15.107:0/3797463391 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 36) v1 ==== 92849+0+0 (secure 0 0 0) 0x7ffae0012400 con 0x7ffaf0071e20 2024-04-22T23:38:13.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.692+0000 7ffaecff9700 1 --2- 172.21.15.107:0/3797463391 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7ffadc06ed30 0x7ffadc0711e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:13.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.692+0000 7ffaecff9700 1 -- 172.21.15.107:0/3797463391 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ffae008e9d0 con 0x7ffaf0071e20 2024-04-22T23:38:13.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.692+0000 7ffaeffff700 1 --2- 172.21.15.107:0/3797463391 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7ffadc06ed30 0x7ffadc0711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:38:13.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.693+0000 7ffaeffff700 1 --2- 172.21.15.107:0/3797463391 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7ffadc06ed30 0x7ffadc0711e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ffad800b970 tx=0x7ffad8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:38:13.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.697+0000 7ffaecff9700 1 -- 172.21.15.107:0/3797463391 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7ffae005b2e0 con 0x7ffaf0071e20 2024-04-22T23:38:13.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.981+0000 7ffaf6284700 1 -- 172.21.15.107:0/3797463391 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ffaf0115cb0 con 0x7ffaf0071e20 2024-04-22T23:38:13.975 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:13 smithi107 ceph-mon[90179]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:38:13.975 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:13 smithi107 ceph-mon[90179]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:38:13.975 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:13 smithi107 ceph-mon[90179]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:38:13.975 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:13 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/278924692' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:38:13.975 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:13 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/2466553983' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:38:13.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.981+0000 7ffaecff9700 1 -- 172.21.15.107:0/3797463391 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7ffae00126b0 con 0x7ffaf0071e20 2024-04-22T23:38:13.976 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_OK 2024-04-22T23:38:13.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.984+0000 7ffaf6284700 1 -- 172.21.15.107:0/3797463391 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7ffadc06ed30 msgr2=0x7ffadc0711e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:13.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.984+0000 7ffaf6284700 1 --2- 172.21.15.107:0/3797463391 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7ffadc06ed30 0x7ffadc0711e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ffad800b970 tx=0x7ffad8009e30 comp rx=0 tx=0).stop 2024-04-22T23:38:13.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.984+0000 7ffaf6284700 1 -- 172.21.15.107:0/3797463391 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffaf0071e20 msgr2=0x7ffaf01100e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:13.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.984+0000 7ffaf6284700 1 --2- 172.21.15.107:0/3797463391 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffaf0071e20 0x7ffaf01100e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ffae000c8c0 tx=0x7ffae00044c0 comp rx=0 tx=0).stop 2024-04-22T23:38:13.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.984+0000 7ffaf6284700 1 -- 172.21.15.107:0/3797463391 shutdown_connections 2024-04-22T23:38:13.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.984+0000 7ffaf6284700 1 --2- 172.21.15.107:0/3797463391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffaf0070870 0x7ffaf010fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:13.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.984+0000 7ffaf6284700 1 --2- 172.21.15.107:0/3797463391 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7ffadc06ed30 0x7ffadc0711e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:13.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.984+0000 7ffaf6284700 1 --2- 172.21.15.107:0/3797463391 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffaf0071e20 0x7ffaf01100e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:13.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.984+0000 7ffaf6284700 1 -- 172.21.15.107:0/3797463391 >> 172.21.15.107:0/3797463391 conn(0x7ffaf00f8580 msgr2=0x7ffaf00f9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:38:13.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.984+0000 7ffaf6284700 1 -- 172.21.15.107:0/3797463391 shutdown_connections 2024-04-22T23:38:13.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:13.984+0000 7ffaf6284700 1 -- 172.21.15.107:0/3797463391 wait complete. 2024-04-22T23:38:14.111 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:13 smithi157 ceph-mon[79403]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:38:14.111 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:13 smithi157 ceph-mon[79403]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:38:14.111 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:13 smithi157 ceph-mon[79403]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:38:14.111 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:13 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/278924692' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:38:14.111 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:13 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/2466553983' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:38:14.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:14 smithi107 ceph-mon[90179]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:38:14.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:14 smithi107 ceph-mon[90179]: pgmap v12: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-22T23:38:14.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:14 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/3797463391' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:38:15.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:14 smithi157 ceph-mon[79403]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:38:15.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:14 smithi157 ceph-mon[79403]: pgmap v12: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-22T23:38:15.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:14 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/3797463391' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:38:16.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:15 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:16.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:15 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:16.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:15 smithi157 ceph-mon[79403]: pgmap v13: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:38:16.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:15 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:16.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:15 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:16.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:15 smithi107 ceph-mon[90179]: pgmap v13: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:38:17.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:16 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:17.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:16 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:17.203 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:16 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:17.203 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:16 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:18.114 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:17 smithi157 ceph-mon[79403]: pgmap v14: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:38:18.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:17 smithi107 ceph-mon[90179]: pgmap v14: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:38:20.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:20 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:20.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:20 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:20.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:20 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:20.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:20 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:38:20.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:20 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:20.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:20 smithi157 ceph-mon[79403]: Reconfiguring mon.smithi107 (monmap changed)... 2024-04-22T23:38:20.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:20 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T23:38:20.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:20 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-22T23:38:20.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:20 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:20.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:20 smithi157 ceph-mon[79403]: Reconfiguring daemon mon.smithi107 on smithi107 2024-04-22T23:38:20.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:20 smithi157 ceph-mon[79403]: pgmap v15: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:38:20.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:20 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:20.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:20 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:20.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:20 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:20.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:20 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:38:20.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:20 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:20.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:20 smithi107 ceph-mon[90179]: Reconfiguring mon.smithi107 (monmap changed)... 2024-04-22T23:38:20.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:20 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T23:38:20.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:20 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-22T23:38:20.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:20 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:20.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:20 smithi107 ceph-mon[90179]: Reconfiguring daemon mon.smithi107 on smithi107 2024-04-22T23:38:20.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:20 smithi107 ceph-mon[90179]: pgmap v15: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:38:22.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:22 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:22.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:22 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:22.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:22 smithi107 ceph-mon[90179]: Reconfiguring mgr.smithi107.uipxvm (monmap changed)... 2024-04-22T23:38:22.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:22 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi107.uipxvm", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T23:38:22.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:22 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T23:38:22.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:22 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:22.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:22 smithi107 ceph-mon[90179]: Reconfiguring daemon mgr.smithi107.uipxvm on smithi107 2024-04-22T23:38:22.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:22 smithi107 ceph-mon[90179]: pgmap v16: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:38:22.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:22 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:22.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:22 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:22.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:22 smithi157 ceph-mon[79403]: Reconfiguring mgr.smithi107.uipxvm (monmap changed)... 2024-04-22T23:38:22.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:22 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi107.uipxvm", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T23:38:22.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:22 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T23:38:22.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:22 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:22.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:22 smithi157 ceph-mon[79403]: Reconfiguring daemon mgr.smithi107.uipxvm on smithi107 2024-04-22T23:38:22.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:22 smithi157 ceph-mon[79403]: pgmap v16: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:38:23.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:23 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:23.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:23 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:23.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:23 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi107", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-22T23:38:23.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:23 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:24.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:23 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:24.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:23 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:24.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:23 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi107", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-22T23:38:24.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:23 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:25.035 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:24 smithi107 ceph-mon[90179]: Reconfiguring crash.smithi107 (monmap changed)... 2024-04-22T23:38:25.035 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:24 smithi107 ceph-mon[90179]: Reconfiguring daemon crash.smithi107 on smithi107 2024-04-22T23:38:25.035 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:24 smithi107 ceph-mon[90179]: pgmap v17: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:38:25.035 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:24 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:25.035 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:24 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:25.035 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:24 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-22T23:38:25.035 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:24 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:25.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:24 smithi157 ceph-mon[79403]: Reconfiguring crash.smithi107 (monmap changed)... 2024-04-22T23:38:25.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:24 smithi157 ceph-mon[79403]: Reconfiguring daemon crash.smithi107 on smithi107 2024-04-22T23:38:25.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:24 smithi157 ceph-mon[79403]: pgmap v17: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:38:25.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:24 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:25.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:24 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:25.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:24 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-22T23:38:25.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:24 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:25.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:25 smithi107 ceph-mon[90179]: Reconfiguring osd.0 (monmap changed)... 2024-04-22T23:38:25.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:25 smithi107 ceph-mon[90179]: Reconfiguring daemon osd.0 on smithi107 2024-04-22T23:38:25.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:25 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:38:26.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:25 smithi157 ceph-mon[79403]: Reconfiguring osd.0 (monmap changed)... 2024-04-22T23:38:26.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:25 smithi157 ceph-mon[79403]: Reconfiguring daemon osd.0 on smithi107 2024-04-22T23:38:26.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:25 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:38:27.077 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:26 smithi107 ceph-mon[90179]: pgmap v18: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:38:27.077 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:26 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:27.077 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:26 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:27.077 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:26 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-22T23:38:27.077 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:26 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:27.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:26 smithi157 ceph-mon[79403]: pgmap v18: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:38:27.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:26 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:27.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:26 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:27.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:26 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-22T23:38:27.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:26 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:27.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:27 smithi107 ceph-mon[90179]: Reconfiguring osd.1 (monmap changed)... 2024-04-22T23:38:27.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:27 smithi107 ceph-mon[90179]: Reconfiguring daemon osd.1 on smithi107 2024-04-22T23:38:28.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:27 smithi157 ceph-mon[79403]: Reconfiguring osd.1 (monmap changed)... 2024-04-22T23:38:28.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:27 smithi157 ceph-mon[79403]: Reconfiguring daemon osd.1 on smithi107 2024-04-22T23:38:28.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:28 smithi107 ceph-mon[90179]: pgmap v19: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:38:28.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:28 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:28.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:28 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:28.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:28 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-22T23:38:28.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:28 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:29.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:28 smithi157 ceph-mon[79403]: pgmap v19: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:38:29.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:28 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:29.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:28 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:29.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:28 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-22T23:38:29.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:28 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:30.078 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:29 smithi107 ceph-mon[90179]: Reconfiguring osd.2 (monmap changed)... 2024-04-22T23:38:30.078 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:29 smithi107 ceph-mon[90179]: Reconfiguring daemon osd.2 on smithi107 2024-04-22T23:38:30.079 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:29 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:30.079 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:29 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:30.079 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:29 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi107.dfvvkw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T23:38:30.079 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:29 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:30.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:29 smithi157 ceph-mon[79403]: Reconfiguring osd.2 (monmap changed)... 2024-04-22T23:38:30.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:29 smithi157 ceph-mon[79403]: Reconfiguring daemon osd.2 on smithi107 2024-04-22T23:38:30.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:29 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:30.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:29 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:30.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:29 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi107.dfvvkw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T23:38:30.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:29 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:31.122 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:30 smithi107 ceph-mon[90179]: Reconfiguring mds.cephfs.smithi107.dfvvkw (monmap changed)... 2024-04-22T23:38:31.122 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:30 smithi107 ceph-mon[90179]: Reconfiguring daemon mds.cephfs.smithi107.dfvvkw on smithi107 2024-04-22T23:38:31.122 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:30 smithi107 ceph-mon[90179]: pgmap v20: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:38:31.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:30 smithi157 ceph-mon[79403]: Reconfiguring mds.cephfs.smithi107.dfvvkw (monmap changed)... 2024-04-22T23:38:31.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:30 smithi157 ceph-mon[79403]: Reconfiguring daemon mds.cephfs.smithi107.dfvvkw on smithi107 2024-04-22T23:38:31.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:30 smithi157 ceph-mon[79403]: pgmap v20: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:38:32.583 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:32 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:32.583 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:32 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:32.583 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:32 smithi107 ceph-mon[90179]: Reconfiguring mds.cephfs.smithi107.ktuxjl (monmap changed)... 2024-04-22T23:38:32.583 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:32 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi107.ktuxjl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T23:38:32.584 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:32 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:32.584 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:32 smithi107 ceph-mon[90179]: Reconfiguring daemon mds.cephfs.smithi107.ktuxjl on smithi107 2024-04-22T23:38:32.584 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:32 smithi107 ceph-mon[90179]: pgmap v21: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:38:32.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:32 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:32.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:32 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:32.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:32 smithi157 ceph-mon[79403]: Reconfiguring mds.cephfs.smithi107.ktuxjl (monmap changed)... 2024-04-22T23:38:32.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:32 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi107.ktuxjl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T23:38:32.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:32 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:32.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:32 smithi157 ceph-mon[79403]: Reconfiguring daemon mds.cephfs.smithi107.ktuxjl on smithi107 2024-04-22T23:38:32.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:32 smithi157 ceph-mon[79403]: pgmap v21: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:38:34.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:33 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:34.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:33 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:34.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:33 smithi157 ceph-mon[79403]: Reconfiguring crash.smithi157 (monmap changed)... 2024-04-22T23:38:34.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:33 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi157", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-22T23:38:34.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:33 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:34.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:33 smithi157 ceph-mon[79403]: Reconfiguring daemon crash.smithi157 on smithi157 2024-04-22T23:38:34.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:33 smithi157 ceph-mon[79403]: pgmap v22: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:38:34.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:33 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:34.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:33 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:34.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:33 smithi107 ceph-mon[90179]: Reconfiguring crash.smithi157 (monmap changed)... 2024-04-22T23:38:34.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:33 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi157", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-22T23:38:34.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:33 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:34.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:33 smithi107 ceph-mon[90179]: Reconfiguring daemon crash.smithi157 on smithi157 2024-04-22T23:38:34.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:33 smithi107 ceph-mon[90179]: pgmap v22: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:38:35.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:35 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:35.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:35 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:35.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:35 smithi157 ceph-mon[79403]: Reconfiguring mgr.smithi157.tuylis (monmap changed)... 2024-04-22T23:38:35.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:35 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi157.tuylis", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T23:38:35.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:35 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T23:38:35.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:35 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:35.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:35 smithi157 ceph-mon[79403]: Reconfiguring daemon mgr.smithi157.tuylis on smithi157 2024-04-22T23:38:35.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:35 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:35.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:35 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:35.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:35 smithi107 ceph-mon[90179]: Reconfiguring mgr.smithi157.tuylis (monmap changed)... 2024-04-22T23:38:35.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:35 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi157.tuylis", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T23:38:35.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:35 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T23:38:35.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:35 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:35.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:35 smithi107 ceph-mon[90179]: Reconfiguring daemon mgr.smithi157.tuylis on smithi157 2024-04-22T23:38:36.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:36 smithi107 ceph-mon[90179]: pgmap v23: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:38:36.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:36 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:36.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:36 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:36.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:36 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T23:38:36.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:36 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-22T23:38:36.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:36 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:36.778 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:36 smithi157 ceph-mon[79403]: pgmap v23: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:38:36.778 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:36 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:36.779 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:36 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:36.779 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:36 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T23:38:36.779 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:36 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-22T23:38:36.779 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:36 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:37.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:37 smithi107 ceph-mon[90179]: Reconfiguring mon.smithi157 (monmap changed)... 2024-04-22T23:38:37.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:37 smithi107 ceph-mon[90179]: Reconfiguring daemon mon.smithi157 on smithi157 2024-04-22T23:38:37.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:37 smithi157 ceph-mon[79403]: Reconfiguring mon.smithi157 (monmap changed)... 2024-04-22T23:38:37.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:37 smithi157 ceph-mon[79403]: Reconfiguring daemon mon.smithi157 on smithi157 2024-04-22T23:38:38.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:38 smithi157 ceph-mon[79403]: pgmap v24: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:38:38.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:38 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:38.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:38 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:38.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:38 smithi157 ceph-mon[79403]: Reconfiguring osd.3 (monmap changed)... 2024-04-22T23:38:38.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:38 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-22T23:38:38.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:38 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:38.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:38 smithi157 ceph-mon[79403]: Reconfiguring daemon osd.3 on smithi157 2024-04-22T23:38:38.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:38 smithi107 ceph-mon[90179]: pgmap v24: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:38:38.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:38 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:38.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:38 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:38.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:38 smithi107 ceph-mon[90179]: Reconfiguring osd.3 (monmap changed)... 2024-04-22T23:38:38.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:38 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-22T23:38:38.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:38 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:38.984 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:38 smithi107 ceph-mon[90179]: Reconfiguring daemon osd.3 on smithi157 2024-04-22T23:38:40.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:40 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:40.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:40 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:40.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:40 smithi107 ceph-mon[90179]: Reconfiguring osd.4 (monmap changed)... 2024-04-22T23:38:40.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:40 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-22T23:38:40.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:40 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:40.735 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:40 smithi107 ceph-mon[90179]: Reconfiguring daemon osd.4 on smithi157 2024-04-22T23:38:40.736 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:40 smithi107 ceph-mon[90179]: pgmap v25: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:38:40.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:40 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:40.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:40 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:40.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:40 smithi157 ceph-mon[79403]: Reconfiguring osd.4 (monmap changed)... 2024-04-22T23:38:40.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:40 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-22T23:38:40.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:40 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:40.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:40 smithi157 ceph-mon[79403]: Reconfiguring daemon osd.4 on smithi157 2024-04-22T23:38:40.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:40 smithi157 ceph-mon[79403]: pgmap v25: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:38:41.818 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:41 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:38:41.818 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:41 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:41.818 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:41 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:41.818 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:41 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-22T23:38:41.818 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:41 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:41.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:41 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:38:41.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:41 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:41.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:41 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:41.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:41 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-22T23:38:41.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:41 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:42.798 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:42 smithi157 ceph-mon[79403]: Reconfiguring osd.5 (monmap changed)... 2024-04-22T23:38:42.799 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:42 smithi157 ceph-mon[79403]: Reconfiguring daemon osd.5 on smithi157 2024-04-22T23:38:42.799 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:42 smithi157 ceph-mon[79403]: pgmap v26: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:38:42.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:42 smithi107 ceph-mon[90179]: Reconfiguring osd.5 (monmap changed)... 2024-04-22T23:38:42.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:42 smithi107 ceph-mon[90179]: Reconfiguring daemon osd.5 on smithi157 2024-04-22T23:38:42.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:42 smithi107 ceph-mon[90179]: pgmap v26: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:38:44.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.101+0000 7f2fa625d700 1 -- 172.21.15.107:0/3737378929 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2fa00fe050 msgr2=0x7f2fa00fe4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:44.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.101+0000 7f2fa625d700 1 --2- 172.21.15.107:0/3737378929 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2fa00fe050 0x7f2fa00fe4c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2f88004940 tx=0x7f2f88009e30 comp rx=0 tx=0).stop 2024-04-22T23:38:44.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.101+0000 7f2fa625d700 1 -- 172.21.15.107:0/3737378929 shutdown_connections 2024-04-22T23:38:44.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.101+0000 7f2fa625d700 1 --2- 172.21.15.107:0/3737378929 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2fa00fe050 0x7f2fa00fe4c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:44.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.101+0000 7f2fa625d700 1 --2- 172.21.15.107:0/3737378929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fa00fce50 0x7f2fa00fd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:44.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.101+0000 7f2fa625d700 1 -- 172.21.15.107:0/3737378929 >> 172.21.15.107:0/3737378929 conn(0x7f2fa00f8540 msgr2=0x7f2fa00fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:38:44.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.101+0000 7f2fa625d700 1 -- 172.21.15.107:0/3737378929 shutdown_connections 2024-04-22T23:38:44.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.101+0000 7f2fa625d700 1 -- 172.21.15.107:0/3737378929 wait complete. 2024-04-22T23:38:44.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.102+0000 7f2fa625d700 1 Processor -- start 2024-04-22T23:38:44.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.102+0000 7f2fa625d700 1 -- start start 2024-04-22T23:38:44.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.102+0000 7f2fa625d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2fa00fce50 0x7f2fa0071e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:44.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.102+0000 7f2fa625d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fa00fe050 0x7f2fa0070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:44.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.102+0000 7f2fa625d700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa00708c0 con 0x7f2fa00fce50 2024-04-22T23:38:44.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.102+0000 7f2fa625d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa0070a30 con 0x7f2fa00fe050 2024-04-22T23:38:44.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.102+0000 7f2f9f7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2fa00fce50 0x7f2fa0071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:38:44.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.102+0000 7f2f9f7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2fa00fce50 0x7f2fa0071e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45662/0 (socket says 172.21.15.107:45662) 2024-04-22T23:38:44.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.102+0000 7f2f9f7fe700 1 -- 172.21.15.107:0/3270050572 learned_addr learned my addr 172.21.15.107:0/3270050572 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:38:44.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.102+0000 7f2f96ffd700 1 --2- 172.21.15.107:0/3270050572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fa00fe050 0x7f2fa0070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:38:44.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.102+0000 7f2f9f7fe700 1 -- 172.21.15.107:0/3270050572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fa00fe050 msgr2=0x7f2fa0070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:44.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.102+0000 7f2f9f7fe700 1 --2- 172.21.15.107:0/3270050572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fa00fe050 0x7f2fa0070310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:44.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.102+0000 7f2f9f7fe700 1 -- 172.21.15.107:0/3270050572 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f88005040 con 0x7f2fa00fce50 2024-04-22T23:38:44.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.103+0000 7f2f9f7fe700 1 --2- 172.21.15.107:0/3270050572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2fa00fce50 0x7f2fa0071e40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2f9000c8b0 tx=0x7f2f90004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:38:44.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.103+0000 7f2f9cff9700 1 -- 172.21.15.107:0/3270050572 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f9000a760 con 0x7f2fa00fce50 2024-04-22T23:38:44.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.103+0000 7f2fa625d700 1 -- 172.21.15.107:0/3270050572 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fa0070d10 con 0x7f2fa00fce50 2024-04-22T23:38:44.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.103+0000 7f2f9cff9700 1 -- 172.21.15.107:0/3270050572 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2f9000ad90 con 0x7f2fa00fce50 2024-04-22T23:38:44.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.103+0000 7f2f9cff9700 1 -- 172.21.15.107:0/3270050572 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f900113a0 con 0x7f2fa00fce50 2024-04-22T23:38:44.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.103+0000 7f2fa625d700 1 -- 172.21.15.107:0/3270050572 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fa0115a60 con 0x7f2fa00fce50 2024-04-22T23:38:44.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.104+0000 7f2fa625d700 1 -- 172.21.15.107:0/3270050572 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fa005f5f0 con 0x7f2fa00fce50 2024-04-22T23:38:44.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.104+0000 7f2f9cff9700 1 -- 172.21.15.107:0/3270050572 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 36) v1 ==== 92849+0+0 (secure 0 0 0) 0x7f2f90012400 con 0x7f2fa00fce50 2024-04-22T23:38:44.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.105+0000 7f2f9cff9700 1 --2- 172.21.15.107:0/3270050572 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f2f8c06ed20 0x7f2f8c0711d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:44.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.105+0000 7f2f9cff9700 1 -- 172.21.15.107:0/3270050572 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f2f9008e9f0 con 0x7f2fa00fce50 2024-04-22T23:38:44.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.105+0000 7f2f96ffd700 1 --2- 172.21.15.107:0/3270050572 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f2f8c06ed20 0x7f2f8c0711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:38:44.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.105+0000 7f2f96ffd700 1 --2- 172.21.15.107:0/3270050572 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f2f8c06ed20 0x7f2f8c0711d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2f8800b3b0 tx=0x7f2f88009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:38:44.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.110+0000 7f2f9cff9700 1 -- 172.21.15.107:0/3270050572 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f2f90057710 con 0x7f2fa00fce50 2024-04-22T23:38:44.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:43 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:44.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:43 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:44.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:43 smithi157 ceph-mon[79403]: Reconfiguring mds.cephfs.smithi157.rcaris (monmap changed)... 2024-04-22T23:38:44.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:43 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi157.rcaris", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T23:38:44.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:43 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:44.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:43 smithi157 ceph-mon[79403]: Reconfiguring daemon mds.cephfs.smithi157.rcaris on smithi157 2024-04-22T23:38:44.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:43 smithi157 ceph-mon[79403]: pgmap v27: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:38:44.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:43 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:44.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:43 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:44.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:43 smithi107 ceph-mon[90179]: Reconfiguring mds.cephfs.smithi157.rcaris (monmap changed)... 2024-04-22T23:38:44.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:43 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi157.rcaris", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T23:38:44.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:43 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:44.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:43 smithi107 ceph-mon[90179]: Reconfiguring daemon mds.cephfs.smithi157.rcaris on smithi157 2024-04-22T23:38:44.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:43 smithi107 ceph-mon[90179]: pgmap v27: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:38:44.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.334+0000 7f2fa625d700 1 -- 172.21.15.107:0/3270050572 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2fa0102380 con 0x7f2f8c06ed20 2024-04-22T23:38:44.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.336+0000 7f2f9cff9700 1 -- 172.21.15.107:0/3270050572 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f2fa0102380 con 0x7f2f8c06ed20 2024-04-22T23:38:44.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.338+0000 7f2fa625d700 1 -- 172.21.15.107:0/3270050572 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f2f8c06ed20 msgr2=0x7f2f8c0711d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:44.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.338+0000 7f2fa625d700 1 --2- 172.21.15.107:0/3270050572 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f2f8c06ed20 0x7f2f8c0711d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2f8800b3b0 tx=0x7f2f88009e30 comp rx=0 tx=0).stop 2024-04-22T23:38:44.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.338+0000 7f2fa625d700 1 -- 172.21.15.107:0/3270050572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2fa00fce50 msgr2=0x7f2fa0071e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:44.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.338+0000 7f2fa625d700 1 --2- 172.21.15.107:0/3270050572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2fa00fce50 0x7f2fa0071e40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2f9000c8b0 tx=0x7f2f90004470 comp rx=0 tx=0).stop 2024-04-22T23:38:44.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.338+0000 7f2fa625d700 1 -- 172.21.15.107:0/3270050572 shutdown_connections 2024-04-22T23:38:44.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.338+0000 7f2fa625d700 1 --2- 172.21.15.107:0/3270050572 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f2f8c06ed20 0x7f2f8c0711d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:44.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.338+0000 7f2fa625d700 1 --2- 172.21.15.107:0/3270050572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2fa00fce50 0x7f2fa0071e40 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:44.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.338+0000 7f2fa625d700 1 --2- 172.21.15.107:0/3270050572 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fa00fe050 0x7f2fa0070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:44.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.338+0000 7f2fa625d700 1 -- 172.21.15.107:0/3270050572 >> 172.21.15.107:0/3270050572 conn(0x7f2fa00f8540 msgr2=0x7f2fa00f9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:38:44.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.338+0000 7f2fa625d700 1 -- 172.21.15.107:0/3270050572 shutdown_connections 2024-04-22T23:38:44.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.338+0000 7f2fa625d700 1 -- 172.21.15.107:0/3270050572 wait complete. 2024-04-22T23:38:44.342 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-04-22T23:38:44.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.440+0000 7f7d1f76f700 1 -- 172.21.15.107:0/1946295845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d180fff70 msgr2=0x7f7d18102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:44.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.440+0000 7f7d1f76f700 1 --2- 172.21.15.107:0/1946295845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d180fff70 0x7f7d18102390 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7d08004940 tx=0x7f7d08009e30 comp rx=0 tx=0).stop 2024-04-22T23:38:44.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.440+0000 7f7d1f76f700 1 -- 172.21.15.107:0/1946295845 shutdown_connections 2024-04-22T23:38:44.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.440+0000 7f7d1f76f700 1 --2- 172.21.15.107:0/1946295845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d180fff70 0x7f7d18102390 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:44.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.440+0000 7f7d1f76f700 1 --2- 172.21.15.107:0/1946295845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d180662f0 0x7f7d180ffa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:44.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.440+0000 7f7d1f76f700 1 -- 172.21.15.107:0/1946295845 >> 172.21.15.107:0/1946295845 conn(0x7f7d180f74c0 msgr2=0x7f7d180f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:38:44.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.440+0000 7f7d1f76f700 1 -- 172.21.15.107:0/1946295845 shutdown_connections 2024-04-22T23:38:44.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.440+0000 7f7d1f76f700 1 -- 172.21.15.107:0/1946295845 wait complete. 2024-04-22T23:38:44.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.440+0000 7f7d1f76f700 1 Processor -- start 2024-04-22T23:38:44.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.441+0000 7f7d1f76f700 1 -- start start 2024-04-22T23:38:44.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.441+0000 7f7d1f76f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d180662f0 0x7f7d1810da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:44.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.441+0000 7f7d1f76f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d180fff70 0x7f7d1810dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:44.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.441+0000 7f7d1f76f700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1810e5a0 con 0x7f7d180fff70 2024-04-22T23:38:44.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.441+0000 7f7d1f76f700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1810edf0 con 0x7f7d180662f0 2024-04-22T23:38:44.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.441+0000 7f7d1cd0a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d180fff70 0x7f7d1810dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:38:44.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.441+0000 7f7d1cd0a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d180fff70 0x7f7d1810dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45676/0 (socket says 172.21.15.107:45676) 2024-04-22T23:38:44.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.441+0000 7f7d1cd0a700 1 -- 172.21.15.107:0/1432358489 learned_addr learned my addr 172.21.15.107:0/1432358489 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:38:44.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.441+0000 7f7d1d50b700 1 --2- 172.21.15.107:0/1432358489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d180662f0 0x7f7d1810da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:38:44.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.441+0000 7f7d1cd0a700 1 -- 172.21.15.107:0/1432358489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d180662f0 msgr2=0x7f7d1810da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:44.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.441+0000 7f7d1cd0a700 1 --2- 172.21.15.107:0/1432358489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d180662f0 0x7f7d1810da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:44.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.441+0000 7f7d1cd0a700 1 -- 172.21.15.107:0/1432358489 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d08005040 con 0x7f7d180fff70 2024-04-22T23:38:44.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.441+0000 7f7d1cd0a700 1 --2- 172.21.15.107:0/1432358489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d180fff70 0x7f7d1810dfa0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7d0800b3b0 tx=0x7f7d08009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:38:44.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.441+0000 7f7d0dffb700 1 -- 172.21.15.107:0/1432358489 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d0800e040 con 0x7f7d180fff70 2024-04-22T23:38:44.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.441+0000 7f7d0dffb700 1 -- 172.21.15.107:0/1432358489 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7d0800bbf0 con 0x7f7d180fff70 2024-04-22T23:38:44.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.441+0000 7f7d0dffb700 1 -- 172.21.15.107:0/1432358489 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d0801c630 con 0x7f7d180fff70 2024-04-22T23:38:44.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.441+0000 7f7d1f76f700 1 -- 172.21.15.107:0/1432358489 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d1810f070 con 0x7f7d180fff70 2024-04-22T23:38:44.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.442+0000 7f7d1f76f700 1 -- 172.21.15.107:0/1432358489 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d1810f5b0 con 0x7f7d180fff70 2024-04-22T23:38:44.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.442+0000 7f7d1f76f700 1 -- 172.21.15.107:0/1432358489 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d1805f5f0 con 0x7f7d180fff70 2024-04-22T23:38:44.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.443+0000 7f7d0dffb700 1 -- 172.21.15.107:0/1432358489 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 36) v1 ==== 92849+0+0 (secure 0 0 0) 0x7f7d0800bd60 con 0x7f7d180fff70 2024-04-22T23:38:44.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.443+0000 7f7d0dffb700 1 --2- 172.21.15.107:0/1432358489 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f7d0406ed20 0x7f7d040711d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:44.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.443+0000 7f7d0dffb700 1 -- 172.21.15.107:0/1432358489 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f7d0808f6b0 con 0x7f7d180fff70 2024-04-22T23:38:44.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.444+0000 7f7d1d50b700 1 --2- 172.21.15.107:0/1432358489 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f7d0406ed20 0x7f7d040711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:38:44.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.444+0000 7f7d1d50b700 1 --2- 172.21.15.107:0/1432358489 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f7d0406ed20 0x7f7d040711d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7d18066030 tx=0x7f7d1400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:38:44.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.448+0000 7f7d0dffb700 1 -- 172.21.15.107:0/1432358489 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f7d0805bdf0 con 0x7f7d180fff70 2024-04-22T23:38:44.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.671+0000 7f7d1f76f700 1 -- 172.21.15.107:0/1432358489 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d1810f8e0 con 0x7f7d0406ed20 2024-04-22T23:38:44.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.672+0000 7f7d0dffb700 1 -- 172.21.15.107:0/1432358489 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f7d1810f8e0 con 0x7f7d0406ed20 2024-04-22T23:38:44.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.674+0000 7f7d1f76f700 1 -- 172.21.15.107:0/1432358489 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f7d0406ed20 msgr2=0x7f7d040711d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:44.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.674+0000 7f7d1f76f700 1 --2- 172.21.15.107:0/1432358489 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f7d0406ed20 0x7f7d040711d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7d18066030 tx=0x7f7d1400b040 comp rx=0 tx=0).stop 2024-04-22T23:38:44.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.674+0000 7f7d1f76f700 1 -- 172.21.15.107:0/1432358489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d180fff70 msgr2=0x7f7d1810dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:44.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.674+0000 7f7d1f76f700 1 --2- 172.21.15.107:0/1432358489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d180fff70 0x7f7d1810dfa0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7d0800b3b0 tx=0x7f7d08009e30 comp rx=0 tx=0).stop 2024-04-22T23:38:44.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.675+0000 7f7d1f76f700 1 -- 172.21.15.107:0/1432358489 shutdown_connections 2024-04-22T23:38:44.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.675+0000 7f7d1f76f700 1 --2- 172.21.15.107:0/1432358489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7d180662f0 0x7f7d1810da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:44.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.675+0000 7f7d1f76f700 1 --2- 172.21.15.107:0/1432358489 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f7d0406ed20 0x7f7d040711d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:44.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.675+0000 7f7d1f76f700 1 --2- 172.21.15.107:0/1432358489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d180fff70 0x7f7d1810dfa0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:44.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.675+0000 7f7d1f76f700 1 -- 172.21.15.107:0/1432358489 >> 172.21.15.107:0/1432358489 conn(0x7f7d180f74c0 msgr2=0x7f7d180f7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:38:44.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.675+0000 7f7d1f76f700 1 -- 172.21.15.107:0/1432358489 shutdown_connections 2024-04-22T23:38:44.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.675+0000 7f7d1f76f700 1 -- 172.21.15.107:0/1432358489 wait complete. 2024-04-22T23:38:44.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.776+0000 7f64ee0bb700 1 -- 172.21.15.107:0/3660038477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64e80fe0e0 msgr2=0x7f64e80fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:44.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.776+0000 7f64ee0bb700 1 --2- 172.21.15.107:0/3660038477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64e80fe0e0 0x7f64e80fe550 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f64d8004940 tx=0x7f64d8009e30 comp rx=0 tx=0).stop 2024-04-22T23:38:44.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.776+0000 7f64ee0bb700 1 -- 172.21.15.107:0/3660038477 shutdown_connections 2024-04-22T23:38:44.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.776+0000 7f64ee0bb700 1 --2- 172.21.15.107:0/3660038477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64e80fe0e0 0x7f64e80fe550 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:44.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.776+0000 7f64ee0bb700 1 --2- 172.21.15.107:0/3660038477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64e80fcef0 0x7f64e80fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:44.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.776+0000 7f64ee0bb700 1 -- 172.21.15.107:0/3660038477 >> 172.21.15.107:0/3660038477 conn(0x7f64e80f8530 msgr2=0x7f64e80fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:38:44.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.776+0000 7f64ee0bb700 1 -- 172.21.15.107:0/3660038477 shutdown_connections 2024-04-22T23:38:44.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.776+0000 7f64ee0bb700 1 -- 172.21.15.107:0/3660038477 wait complete. 2024-04-22T23:38:44.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.776+0000 7f64ee0bb700 1 Processor -- start 2024-04-22T23:38:44.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.777+0000 7f64ee0bb700 1 -- start start 2024-04-22T23:38:44.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.777+0000 7f64ee0bb700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64e80fcef0 0x7f64e810d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:44.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.777+0000 7f64ee0bb700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64e80fe0e0 0x7f64e810df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:44.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.777+0000 7f64ee0bb700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e810e500 con 0x7f64e80fe0e0 2024-04-22T23:38:44.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.777+0000 7f64ee0bb700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e810ed50 con 0x7f64e80fcef0 2024-04-22T23:38:44.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.777+0000 7f64e6ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64e80fe0e0 0x7f64e810df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:38:44.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.777+0000 7f64e6ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64e80fe0e0 0x7f64e810df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45690/0 (socket says 172.21.15.107:45690) 2024-04-22T23:38:44.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.777+0000 7f64e6ffd700 1 -- 172.21.15.107:0/3811886703 learned_addr learned my addr 172.21.15.107:0/3811886703 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:38:44.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.777+0000 7f64e77fe700 1 --2- 172.21.15.107:0/3811886703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64e80fcef0 0x7f64e810d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:38:44.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.777+0000 7f64e6ffd700 1 -- 172.21.15.107:0/3811886703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64e80fcef0 msgr2=0x7f64e810d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:44.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.777+0000 7f64e6ffd700 1 --2- 172.21.15.107:0/3811886703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64e80fcef0 0x7f64e810d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:44.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.777+0000 7f64e6ffd700 1 -- 172.21.15.107:0/3811886703 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d8005040 con 0x7f64e80fe0e0 2024-04-22T23:38:44.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.777+0000 7f64e6ffd700 1 --2- 172.21.15.107:0/3811886703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64e80fe0e0 0x7f64e810df00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f64d80043d0 tx=0x7f64d8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:38:44.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.778+0000 7f64cffff700 1 -- 172.21.15.107:0/3811886703 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64d8020070 con 0x7f64e80fe0e0 2024-04-22T23:38:44.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.778+0000 7f64ee0bb700 1 -- 172.21.15.107:0/3811886703 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64e810efd0 con 0x7f64e80fe0e0 2024-04-22T23:38:44.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.778+0000 7f64cffff700 1 -- 172.21.15.107:0/3811886703 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f64d8009430 con 0x7f64e80fe0e0 2024-04-22T23:38:44.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.778+0000 7f64cffff700 1 -- 172.21.15.107:0/3811886703 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64d800e070 con 0x7f64e80fe0e0 2024-04-22T23:38:44.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.778+0000 7f64ee0bb700 1 -- 172.21.15.107:0/3811886703 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64e810f4b0 con 0x7f64e80fe0e0 2024-04-22T23:38:44.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.779+0000 7f64ee0bb700 1 -- 172.21.15.107:0/3811886703 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64e805f5f0 con 0x7f64e80fe0e0 2024-04-22T23:38:44.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.779+0000 7f64cffff700 1 -- 172.21.15.107:0/3811886703 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 36) v1 ==== 92849+0+0 (secure 0 0 0) 0x7f64d801c770 con 0x7f64e80fe0e0 2024-04-22T23:38:44.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.780+0000 7f64cffff700 1 --2- 172.21.15.107:0/3811886703 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f64d006ecc0 0x7f64d0071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:44.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.780+0000 7f64e77fe700 1 --2- 172.21.15.107:0/3811886703 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f64d006ecc0 0x7f64d0071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:38:44.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.780+0000 7f64cffff700 1 -- 172.21.15.107:0/3811886703 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f64d808fd70 con 0x7f64e80fe0e0 2024-04-22T23:38:44.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.780+0000 7f64e77fe700 1 --2- 172.21.15.107:0/3811886703 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f64d006ecc0 0x7f64d0071170 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f64dc009b80 tx=0x7f64dc00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:38:44.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.785+0000 7f64cffff700 1 -- 172.21.15.107:0/3811886703 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f64d805c4b0 con 0x7f64e80fe0e0 2024-04-22T23:38:44.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:44.994+0000 7f64ee0bb700 1 -- 172.21.15.107:0/3811886703 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f64e810f770 con 0x7f64d006ecc0 2024-04-22T23:38:44.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.004+0000 7f64cffff700 1 -- 172.21.15.107:0/3811886703 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f64e810f770 con 0x7f64d006ecc0 2024-04-22T23:38:44.998 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-22T23:38:44.998 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (2m) 43s ago 11m 16.3M - 0.25.0 c8568f914cd2 ec1a5c6260de 2024-04-22T23:38:44.998 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (11m) 43s ago 11m 6665k - 17.2.7-884-g70d0cb4b cafd8ddac8fa c10611c756e4 2024-04-22T23:38:44.998 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi157 smithi157 running (10m) 30s ago 10m 6669k - 17.2.7-884-g70d0cb4b cafd8ddac8fa 55d2d46294c9 2024-04-22T23:38:44.998 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (117s) 43s ago 11m 80.1M - 9.4.7 954c08fa6188 31449ff442d2 2024-04-22T23:38:44.999 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.dfvvkw smithi107 running (6m) 43s ago 6m 13.0M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 08d3aa8d717d 2024-04-22T23:38:44.999 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.ktuxjl smithi107 running (6m) 43s ago 6m 124M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 762f27fe0f30 2024-04-22T23:38:44.999 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi157.rcaris smithi157 running (6m) 30s ago 6m 129M - 17.2.7-884-g70d0cb4b cafd8ddac8fa d23366ac085d 2024-04-22T23:38:44.999 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi157.ufywse smithi157 running (6m) 30s ago 6m 13.9M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 8b00d61eb355 2024-04-22T23:38:44.999 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.uipxvm smithi107 *:8443,9283,8765 running (104s) 43s ago 13m 474M - 19.0.0-3252-g43be0201 a0434699c1d7 d20f05dfe4f2 2024-04-22T23:38:44.999 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi157.tuylis smithi157 *:8443,9283,8765 running (72s) 30s ago 10m 449M - 19.0.0-3252-g43be0201 a0434699c1d7 957e598c9001 2024-04-22T23:38:44.999 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (58s) 43s ago 13m 30.1M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 bc72bbbe2c39 2024-04-22T23:38:44.999 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi157 smithi157 running (34s) 30s ago 10m 19.0M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 ea88de8be4fe 2024-04-22T23:38:44.999 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (2m) 43s ago 11m 8044k - 1.5.0 0da6a335fe13 5205cfd9bd51 2024-04-22T23:38:44.999 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi157 smithi157 *:9100 running (110s) 30s ago 10m 17.2M - 1.5.0 0da6a335fe13 168ca191bb8c 2024-04-22T23:38:44.999 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (10m) 43s ago 10m 231M 1557M 17.2.7-884-g70d0cb4b cafd8ddac8fa 18a179b0a618 2024-04-22T23:38:44.999 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (9m) 43s ago 9m 241M 1557M 17.2.7-884-g70d0cb4b cafd8ddac8fa c7272eb84ddb 2024-04-22T23:38:44.999 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (9m) 43s ago 9m 194M 1557M 17.2.7-884-g70d0cb4b cafd8ddac8fa 9db1788096ad 2024-04-22T23:38:44.999 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi157 running (8m) 30s ago 8m 265M 2581M 17.2.7-884-g70d0cb4b cafd8ddac8fa c9fb2d1a13dd 2024-04-22T23:38:45.000 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi157 running (8m) 30s ago 8m 226M 2581M 17.2.7-884-g70d0cb4b cafd8ddac8fa 24a16e19951f 2024-04-22T23:38:45.000 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi157 running (7m) 30s ago 7m 226M 2581M 17.2.7-884-g70d0cb4b cafd8ddac8fa 4c5b7a7ef8bd 2024-04-22T23:38:45.000 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (77s) 43s ago 11m 53.6M - 2.43.0 a07b618ecd1d 99a4ea8594da 2024-04-22T23:38:45.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.006+0000 7f64ee0bb700 1 -- 172.21.15.107:0/3811886703 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f64d006ecc0 msgr2=0x7f64d0071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:45.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.006+0000 7f64ee0bb700 1 --2- 172.21.15.107:0/3811886703 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f64d006ecc0 0x7f64d0071170 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f64dc009b80 tx=0x7f64dc00b040 comp rx=0 tx=0).stop 2024-04-22T23:38:45.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.006+0000 7f64ee0bb700 1 -- 172.21.15.107:0/3811886703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64e80fe0e0 msgr2=0x7f64e810df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:45.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.006+0000 7f64ee0bb700 1 --2- 172.21.15.107:0/3811886703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64e80fe0e0 0x7f64e810df00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f64d80043d0 tx=0x7f64d8009e30 comp rx=0 tx=0).stop 2024-04-22T23:38:45.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.007+0000 7f64ee0bb700 1 -- 172.21.15.107:0/3811886703 shutdown_connections 2024-04-22T23:38:45.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.007+0000 7f64ee0bb700 1 --2- 172.21.15.107:0/3811886703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64e80fcef0 0x7f64e810d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:45.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.007+0000 7f64ee0bb700 1 --2- 172.21.15.107:0/3811886703 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f64d006ecc0 0x7f64d0071170 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:45.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.007+0000 7f64ee0bb700 1 --2- 172.21.15.107:0/3811886703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64e80fe0e0 0x7f64e810df00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:45.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.007+0000 7f64ee0bb700 1 -- 172.21.15.107:0/3811886703 >> 172.21.15.107:0/3811886703 conn(0x7f64e80f8530 msgr2=0x7f64e80f9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:38:45.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.007+0000 7f64ee0bb700 1 -- 172.21.15.107:0/3811886703 shutdown_connections 2024-04-22T23:38:45.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.007+0000 7f64ee0bb700 1 -- 172.21.15.107:0/3811886703 wait complete. 2024-04-22T23:38:45.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.105+0000 7f6311b5f700 1 -- 172.21.15.107:0/1982599065 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f630c0fbcf0 msgr2=0x7f630c0fc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:45.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.105+0000 7f6311b5f700 1 --2- 172.21.15.107:0/1982599065 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f630c0fbcf0 0x7f630c0fc100 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f62fc004940 tx=0x7f62fc009e30 comp rx=0 tx=0).stop 2024-04-22T23:38:45.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.105+0000 7f6311b5f700 1 -- 172.21.15.107:0/1982599065 shutdown_connections 2024-04-22T23:38:45.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.105+0000 7f6311b5f700 1 --2- 172.21.15.107:0/1982599065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f630c0fcfa0 0x7f630c0fd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:45.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.105+0000 7f6311b5f700 1 --2- 172.21.15.107:0/1982599065 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f630c0fbcf0 0x7f630c0fc100 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:45.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.105+0000 7f6311b5f700 1 -- 172.21.15.107:0/1982599065 >> 172.21.15.107:0/1982599065 conn(0x7f630c0f7450 msgr2=0x7f630c0f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:38:45.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.106+0000 7f6311b5f700 1 -- 172.21.15.107:0/1982599065 shutdown_connections 2024-04-22T23:38:45.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.106+0000 7f6311b5f700 1 -- 172.21.15.107:0/1982599065 wait complete. 2024-04-22T23:38:45.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.106+0000 7f6311b5f700 1 Processor -- start 2024-04-22T23:38:45.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.106+0000 7f6311b5f700 1 -- start start 2024-04-22T23:38:45.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.106+0000 7f6311b5f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f630c0fbcf0 0x7f630c066d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:45.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.106+0000 7f6311b5f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f630c0fcfa0 0x7f630c1129c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:45.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.106+0000 7f6311b5f700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f630c112f00 con 0x7f630c0fbcf0 2024-04-22T23:38:45.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.106+0000 7f6311b5f700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f630c113040 con 0x7f630c0fcfa0 2024-04-22T23:38:45.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.107+0000 7f630b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f630c0fbcf0 0x7f630c066d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:38:45.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.107+0000 7f630b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f630c0fbcf0 0x7f630c066d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45700/0 (socket says 172.21.15.107:45700) 2024-04-22T23:38:45.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.107+0000 7f630b7fe700 1 -- 172.21.15.107:0/1630508509 learned_addr learned my addr 172.21.15.107:0/1630508509 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:38:45.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.107+0000 7f630affd700 1 --2- 172.21.15.107:0/1630508509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f630c0fcfa0 0x7f630c1129c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:38:45.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.107+0000 7f630b7fe700 1 -- 172.21.15.107:0/1630508509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f630c0fcfa0 msgr2=0x7f630c1129c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:45.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.107+0000 7f630b7fe700 1 --2- 172.21.15.107:0/1630508509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f630c0fcfa0 0x7f630c1129c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:45.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.107+0000 7f630b7fe700 1 -- 172.21.15.107:0/1630508509 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62fc005040 con 0x7f630c0fbcf0 2024-04-22T23:38:45.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.107+0000 7f630affd700 1 --2- 172.21.15.107:0/1630508509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f630c0fcfa0 0x7f630c1129c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T23:38:45.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.107+0000 7f630b7fe700 1 --2- 172.21.15.107:0/1630508509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f630c0fbcf0 0x7f630c066d00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f62fc009150 tx=0x7f62fc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:38:45.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.107+0000 7f62f3fff700 1 -- 172.21.15.107:0/1630508509 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62fc00e040 con 0x7f630c0fbcf0 2024-04-22T23:38:45.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.107+0000 7f6311b5f700 1 -- 172.21.15.107:0/1630508509 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f630c113270 con 0x7f630c0fbcf0 2024-04-22T23:38:45.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.107+0000 7f62f3fff700 1 -- 172.21.15.107:0/1630508509 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f62fc00bcc0 con 0x7f630c0fbcf0 2024-04-22T23:38:45.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.107+0000 7f62f3fff700 1 -- 172.21.15.107:0/1630508509 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62fc01c630 con 0x7f630c0fbcf0 2024-04-22T23:38:45.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.107+0000 7f6311b5f700 1 -- 172.21.15.107:0/1630508509 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f630c1137d0 con 0x7f630c0fbcf0 2024-04-22T23:38:45.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.108+0000 7f6311b5f700 1 -- 172.21.15.107:0/1630508509 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f630c04c710 con 0x7f630c0fbcf0 2024-04-22T23:38:45.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.109+0000 7f62f3fff700 1 -- 172.21.15.107:0/1630508509 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 36) v1 ==== 92849+0+0 (secure 0 0 0) 0x7f62fc013420 con 0x7f630c0fbcf0 2024-04-22T23:38:45.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.109+0000 7f62f3fff700 1 --2- 172.21.15.107:0/1630508509 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f62f406ed30 0x7f62f40711e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:45.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.109+0000 7f62f3fff700 1 -- 172.21.15.107:0/1630508509 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f62fc08f810 con 0x7f630c0fbcf0 2024-04-22T23:38:45.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.109+0000 7f630affd700 1 --2- 172.21.15.107:0/1630508509 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f62f406ed30 0x7f62f40711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:38:45.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.110+0000 7f630affd700 1 --2- 172.21.15.107:0/1630508509 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f62f406ed30 0x7f62f40711e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6300007fd0 tx=0x7f6300009f60 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:38:45.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.114+0000 7f62f3fff700 1 -- 172.21.15.107:0/1630508509 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f62fc05bf50 con 0x7f630c0fbcf0 2024-04-22T23:38:45.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.396+0000 7f6311b5f700 1 -- 172.21.15.107:0/1630508509 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f630c1047e0 con 0x7f630c0fbcf0 2024-04-22T23:38:45.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.397+0000 7f62f3fff700 1 -- 172.21.15.107:0/1630508509 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+736 (secure 0 0 0) 0x7f62fc05bae0 con 0x7f630c0fbcf0 2024-04-22T23:38:45.391 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-22T23:38:45.391 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-04-22T23:38:45.391 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:38:45.391 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:38:45.391 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-04-22T23:38:45.391 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:38:45.391 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:38:45.391 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-04-22T23:38:45.391 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 6 2024-04-22T23:38:45.391 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:38:45.391 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-04-22T23:38:45.391 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 4 2024-04-22T23:38:45.392 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:38:45.392 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-04-22T23:38:45.392 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 10, 2024-04-22T23:38:45.392 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 4 2024-04-22T23:38:45.392 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T23:38:45.392 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-22T23:38:45.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.399+0000 7f6311b5f700 1 -- 172.21.15.107:0/1630508509 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f62f406ed30 msgr2=0x7f62f40711e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:45.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.399+0000 7f6311b5f700 1 --2- 172.21.15.107:0/1630508509 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f62f406ed30 0x7f62f40711e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6300007fd0 tx=0x7f6300009f60 comp rx=0 tx=0).stop 2024-04-22T23:38:45.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.399+0000 7f6311b5f700 1 -- 172.21.15.107:0/1630508509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f630c0fbcf0 msgr2=0x7f630c066d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:45.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.399+0000 7f6311b5f700 1 --2- 172.21.15.107:0/1630508509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f630c0fbcf0 0x7f630c066d00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f62fc009150 tx=0x7f62fc009e30 comp rx=0 tx=0).stop 2024-04-22T23:38:45.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.399+0000 7f6311b5f700 1 -- 172.21.15.107:0/1630508509 shutdown_connections 2024-04-22T23:38:45.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.399+0000 7f6311b5f700 1 --2- 172.21.15.107:0/1630508509 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f62f406ed30 0x7f62f40711e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:45.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.399+0000 7f6311b5f700 1 --2- 172.21.15.107:0/1630508509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f630c0fbcf0 0x7f630c066d00 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:45.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.399+0000 7f6311b5f700 1 --2- 172.21.15.107:0/1630508509 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f630c0fcfa0 0x7f630c1129c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:45.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.399+0000 7f6311b5f700 1 -- 172.21.15.107:0/1630508509 >> 172.21.15.107:0/1630508509 conn(0x7f630c0f7450 msgr2=0x7f630c0f8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:38:45.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.399+0000 7f6311b5f700 1 -- 172.21.15.107:0/1630508509 shutdown_connections 2024-04-22T23:38:45.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.399+0000 7f6311b5f700 1 -- 172.21.15.107:0/1630508509 wait complete. 2024-04-22T23:38:45.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.500+0000 7f2c38a40700 1 -- 172.21.15.107:0/1323314639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c340662f0 msgr2=0x7f2c340ffa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:45.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.500+0000 7f2c38a40700 1 --2- 172.21.15.107:0/1323314639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c340662f0 0x7f2c340ffa30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2c24004940 tx=0x7f2c24009e30 comp rx=0 tx=0).stop 2024-04-22T23:38:45.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.500+0000 7f2c38a40700 1 -- 172.21.15.107:0/1323314639 shutdown_connections 2024-04-22T23:38:45.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.500+0000 7f2c38a40700 1 --2- 172.21.15.107:0/1323314639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c340fff70 0x7f2c34102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:45.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.500+0000 7f2c38a40700 1 --2- 172.21.15.107:0/1323314639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c340662f0 0x7f2c340ffa30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:45.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.500+0000 7f2c38a40700 1 -- 172.21.15.107:0/1323314639 >> 172.21.15.107:0/1323314639 conn(0x7f2c340f74c0 msgr2=0x7f2c340f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:38:45.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.500+0000 7f2c38a40700 1 -- 172.21.15.107:0/1323314639 shutdown_connections 2024-04-22T23:38:45.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.500+0000 7f2c38a40700 1 -- 172.21.15.107:0/1323314639 wait complete. 2024-04-22T23:38:45.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.500+0000 7f2c38a40700 1 Processor -- start 2024-04-22T23:38:45.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.500+0000 7f2c38a40700 1 -- start start 2024-04-22T23:38:45.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.501+0000 7f2c38a40700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c340662f0 0x7f2c3410da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:45.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.501+0000 7f2c38a40700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c340fff70 0x7f2c3410df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:45.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.501+0000 7f2c38a40700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3410e580 con 0x7f2c340fff70 2024-04-22T23:38:45.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.501+0000 7f2c38a40700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3410ed80 con 0x7f2c340662f0 2024-04-22T23:38:45.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.501+0000 7f2c31d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c340fff70 0x7f2c3410df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:38:45.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.501+0000 7f2c3259c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c340662f0 0x7f2c3410da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:38:45.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.501+0000 7f2c31d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c340fff70 0x7f2c3410df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45710/0 (socket says 172.21.15.107:45710) 2024-04-22T23:38:45.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.501+0000 7f2c31d9b700 1 -- 172.21.15.107:0/883461375 learned_addr learned my addr 172.21.15.107:0/883461375 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:38:45.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.501+0000 7f2c31d9b700 1 -- 172.21.15.107:0/883461375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c340662f0 msgr2=0x7f2c3410da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:45.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.501+0000 7f2c31d9b700 1 --2- 172.21.15.107:0/883461375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c340662f0 0x7f2c3410da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:45.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.501+0000 7f2c31d9b700 1 -- 172.21.15.107:0/883461375 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c24005040 con 0x7f2c340fff70 2024-04-22T23:38:45.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.501+0000 7f2c31d9b700 1 --2- 172.21.15.107:0/883461375 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c340fff70 0x7f2c3410df80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2c2800eb40 tx=0x7f2c2800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:38:45.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.501+0000 7f2c22ffd700 1 -- 172.21.15.107:0/883461375 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c28007cc0 con 0x7f2c340fff70 2024-04-22T23:38:45.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.501+0000 7f2c22ffd700 1 -- 172.21.15.107:0/883461375 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2c2800f040 con 0x7f2c340fff70 2024-04-22T23:38:45.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.501+0000 7f2c22ffd700 1 -- 172.21.15.107:0/883461375 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c2800ca10 con 0x7f2c340fff70 2024-04-22T23:38:45.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.501+0000 7f2c38a40700 1 -- 172.21.15.107:0/883461375 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c3410f000 con 0x7f2c340fff70 2024-04-22T23:38:45.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.501+0000 7f2c38a40700 1 -- 172.21.15.107:0/883461375 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c3410f540 con 0x7f2c340fff70 2024-04-22T23:38:45.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.502+0000 7f2c38a40700 1 -- 172.21.15.107:0/883461375 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c3405f5f0 con 0x7f2c340fff70 2024-04-22T23:38:45.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.503+0000 7f2c22ffd700 1 -- 172.21.15.107:0/883461375 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 36) v1 ==== 92849+0+0 (secure 0 0 0) 0x7f2c2800cb70 con 0x7f2c340fff70 2024-04-22T23:38:45.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.503+0000 7f2c22ffd700 1 --2- 172.21.15.107:0/883461375 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f2c1c06ed20 0x7f2c1c0711d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:45.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.503+0000 7f2c22ffd700 1 -- 172.21.15.107:0/883461375 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f2c2808e600 con 0x7f2c340fff70 2024-04-22T23:38:45.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.503+0000 7f2c3259c700 1 --2- 172.21.15.107:0/883461375 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f2c1c06ed20 0x7f2c1c0711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:38:45.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.504+0000 7f2c3259c700 1 --2- 172.21.15.107:0/883461375 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f2c1c06ed20 0x7f2c1c0711d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2c24013870 tx=0x7f2c24009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:38:45.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.508+0000 7f2c22ffd700 1 -- 172.21.15.107:0/883461375 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f2c2805adf0 con 0x7f2c340fff70 2024-04-22T23:38:45.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:45 smithi157 ceph-mon[79403]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:38:45.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:45 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:45.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:45 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:45.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:45 smithi157 ceph-mon[79403]: Reconfiguring mds.cephfs.smithi157.ufywse (monmap changed)... 2024-04-22T23:38:45.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:45 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi157.ufywse", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T23:38:45.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:45 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:45.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:45 smithi157 ceph-mon[79403]: Reconfiguring daemon mds.cephfs.smithi157.ufywse on smithi157 2024-04-22T23:38:45.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:45 smithi157 ceph-mon[79403]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:38:45.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:45 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/1630508509' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:38:45.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:45 smithi107 ceph-mon[90179]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:38:45.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:45 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:45.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:45 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:45.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:45 smithi107 ceph-mon[90179]: Reconfiguring mds.cephfs.smithi157.ufywse (monmap changed)... 2024-04-22T23:38:45.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:45 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi157.ufywse", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T23:38:45.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:45 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:45.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:45 smithi107 ceph-mon[90179]: Reconfiguring daemon mds.cephfs.smithi157.ufywse on smithi157 2024-04-22T23:38:45.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:45 smithi107 ceph-mon[90179]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:38:45.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:45 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/1630508509' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:38:45.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.755+0000 7f2c38a40700 1 -- 172.21.15.107:0/883461375 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2c3404bed0 con 0x7f2c340fff70 2024-04-22T23:38:45.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.755+0000 7f2c22ffd700 1 -- 172.21.15.107:0/883461375 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) v1 ==== 76+0+1928 (secure 0 0 0) 0x7f2c2805a980 con 0x7f2c340fff70 2024-04-22T23:38:45.749 INFO:teuthology.orchestra.run.smithi107.stdout:e19 2024-04-22T23:38:45.749 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T23:38:45.749 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:38:45.749 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-04-22T23:38:45.749 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:38:45.749 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-04-22T23:38:45.749 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-04-22T23:38:45.750 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 19 2024-04-22T23:38:45.750 INFO:teuthology.orchestra.run.smithi107.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-22T23:38:45.750 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-04-22T23:32:20.146130+0000 2024-04-22T23:38:45.750 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-04-22T23:35:12.338678+0000 2024-04-22T23:38:45.750 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-04-22T23:38:45.750 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-04-22T23:38:45.750 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-04-22T23:38:45.750 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-04-22T23:38:45.750 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-04-22T23:38:45.750 INFO:teuthology.orchestra.run.smithi107.stdout:max_xattr_size 65536 2024-04-22T23:38:45.750 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-04-22T23:38:45.750 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-04-22T23:38:45.750 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 46 2024-04-22T23:38:45.750 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:38:45.751 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 2 2024-04-22T23:38:45.751 INFO:teuthology.orchestra.run.smithi107.stdout:in 0,1 2024-04-22T23:38:45.751 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=14468,1=24265} 2024-04-22T23:38:45.751 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-04-22T23:38:45.751 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-04-22T23:38:45.751 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-04-22T23:38:45.751 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-04-22T23:38:45.751 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-04-22T23:38:45.751 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data disabled 2024-04-22T23:38:45.751 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-04-22T23:38:45.751 INFO:teuthology.orchestra.run.smithi107.stdout:bal_rank_mask -1 2024-04-22T23:38:45.751 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-04-22T23:38:45.751 INFO:teuthology.orchestra.run.smithi107.stdout:qdb_cluster leader: 0 members: 2024-04-22T23:38:45.751 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.ktuxjl{0:14468} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.107:6828/3335313253,v1:172.21.15.107:6829/3335313253] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:38:45.752 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi157.rcaris{1:24265} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.157:6824/2779251893,v1:172.21.15.157:6825/2779251893] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:38:45.752 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:38:45.752 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:38:45.752 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-04-22T23:38:45.752 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:38:45.752 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.dfvvkw{-1:14484} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/3548717247,v1:172.21.15.107:6827/3548717247] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:38:45.752 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi157.ufywse{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.157:6826/4072444199,v1:172.21.15.157:6827/4072444199] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:38:45.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.758+0000 7f2c38a40700 1 -- 172.21.15.107:0/883461375 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f2c1c06ed20 msgr2=0x7f2c1c0711d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:45.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.758+0000 7f2c38a40700 1 --2- 172.21.15.107:0/883461375 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f2c1c06ed20 0x7f2c1c0711d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2c24013870 tx=0x7f2c24009e30 comp rx=0 tx=0).stop 2024-04-22T23:38:45.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.758+0000 7f2c38a40700 1 -- 172.21.15.107:0/883461375 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c340fff70 msgr2=0x7f2c3410df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:45.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.758+0000 7f2c38a40700 1 --2- 172.21.15.107:0/883461375 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c340fff70 0x7f2c3410df80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2c2800eb40 tx=0x7f2c2800b040 comp rx=0 tx=0).stop 2024-04-22T23:38:45.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.758+0000 7f2c38a40700 1 -- 172.21.15.107:0/883461375 shutdown_connections 2024-04-22T23:38:45.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.758+0000 7f2c38a40700 1 --2- 172.21.15.107:0/883461375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c340662f0 0x7f2c3410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:45.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.758+0000 7f2c38a40700 1 --2- 172.21.15.107:0/883461375 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f2c1c06ed20 0x7f2c1c0711d0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:45.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.758+0000 7f2c38a40700 1 --2- 172.21.15.107:0/883461375 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c340fff70 0x7f2c3410df80 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:45.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.758+0000 7f2c38a40700 1 -- 172.21.15.107:0/883461375 >> 172.21.15.107:0/883461375 conn(0x7f2c340f74c0 msgr2=0x7f2c340f7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:38:45.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.758+0000 7f2c38a40700 1 -- 172.21.15.107:0/883461375 shutdown_connections 2024-04-22T23:38:45.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.758+0000 7f2c38a40700 1 -- 172.21.15.107:0/883461375 wait complete. 2024-04-22T23:38:45.753 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 19 2024-04-22T23:38:45.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.863+0000 7fe2a0daa700 1 -- 172.21.15.107:0/1759733540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe29c0fbd00 msgr2=0x7fe29c0fc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:45.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.863+0000 7fe2a0daa700 1 --2- 172.21.15.107:0/1759733540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe29c0fbd00 0x7fe29c0fc110 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe28c004940 tx=0x7fe28c009e30 comp rx=0 tx=0).stop 2024-04-22T23:38:45.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.863+0000 7fe2a0daa700 1 -- 172.21.15.107:0/1759733540 shutdown_connections 2024-04-22T23:38:45.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.863+0000 7fe2a0daa700 1 --2- 172.21.15.107:0/1759733540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe29c0fcfb0 0x7fe29c0fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:45.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.863+0000 7fe2a0daa700 1 --2- 172.21.15.107:0/1759733540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe29c0fbd00 0x7fe29c0fc110 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:45.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.863+0000 7fe2a0daa700 1 -- 172.21.15.107:0/1759733540 >> 172.21.15.107:0/1759733540 conn(0x7fe29c0f7480 msgr2=0x7fe29c0f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:38:45.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.863+0000 7fe2a0daa700 1 -- 172.21.15.107:0/1759733540 shutdown_connections 2024-04-22T23:38:45.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.863+0000 7fe2a0daa700 1 -- 172.21.15.107:0/1759733540 wait complete. 2024-04-22T23:38:45.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.864+0000 7fe2a0daa700 1 Processor -- start 2024-04-22T23:38:45.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.864+0000 7fe2a0daa700 1 -- start start 2024-04-22T23:38:45.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.864+0000 7fe2a0daa700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe29c0fbd00 0x7fe29c10d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:45.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.864+0000 7fe2a0daa700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe29c0fcfb0 0x7fe29c10def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:45.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.864+0000 7fe2a0daa700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c10e4f0 con 0x7fe29c0fbd00 2024-04-22T23:38:45.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.864+0000 7fe2a0daa700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c10e660 con 0x7fe29c0fcfb0 2024-04-22T23:38:45.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.864+0000 7fe29a59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe29c0fbd00 0x7fe29c10d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:38:45.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.864+0000 7fe29a59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe29c0fbd00 0x7fe29c10d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45726/0 (socket says 172.21.15.107:45726) 2024-04-22T23:38:45.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.864+0000 7fe29a59c700 1 -- 172.21.15.107:0/2338685443 learned_addr learned my addr 172.21.15.107:0/2338685443 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:38:45.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.864+0000 7fe299d9b700 1 --2- 172.21.15.107:0/2338685443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe29c0fcfb0 0x7fe29c10def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:38:45.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.865+0000 7fe29a59c700 1 -- 172.21.15.107:0/2338685443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe29c0fcfb0 msgr2=0x7fe29c10def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:45.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.865+0000 7fe29a59c700 1 --2- 172.21.15.107:0/2338685443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe29c0fcfb0 0x7fe29c10def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:45.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.865+0000 7fe29a59c700 1 -- 172.21.15.107:0/2338685443 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe28c005040 con 0x7fe29c0fbd00 2024-04-22T23:38:45.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.865+0000 7fe29a59c700 1 --2- 172.21.15.107:0/2338685443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe29c0fbd00 0x7fe29c10d9b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe28c009380 tx=0x7fe28c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:38:45.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.865+0000 7fe28affd700 1 -- 172.21.15.107:0/2338685443 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe28c00e040 con 0x7fe29c0fbd00 2024-04-22T23:38:45.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.865+0000 7fe28affd700 1 -- 172.21.15.107:0/2338685443 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe28c00bbf0 con 0x7fe29c0fbd00 2024-04-22T23:38:45.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.865+0000 7fe2a0daa700 1 -- 172.21.15.107:0/2338685443 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe29c113060 con 0x7fe29c0fbd00 2024-04-22T23:38:45.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.865+0000 7fe28affd700 1 -- 172.21.15.107:0/2338685443 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe28c01c3f0 con 0x7fe29c0fbd00 2024-04-22T23:38:45.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.865+0000 7fe2a0daa700 1 -- 172.21.15.107:0/2338685443 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe29c113620 con 0x7fe29c0fbd00 2024-04-22T23:38:45.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.866+0000 7fe2a0daa700 1 -- 172.21.15.107:0/2338685443 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe29c04c710 con 0x7fe29c0fbd00 2024-04-22T23:38:45.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.866+0000 7fe28affd700 1 -- 172.21.15.107:0/2338685443 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 36) v1 ==== 92849+0+0 (secure 0 0 0) 0x7fe28c00bd60 con 0x7fe29c0fbd00 2024-04-22T23:38:45.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.867+0000 7fe28affd700 1 --2- 172.21.15.107:0/2338685443 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fe28406ed30 0x7fe2840711e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:45.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.867+0000 7fe28affd700 1 -- 172.21.15.107:0/2338685443 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fe28c08fe90 con 0x7fe29c0fbd00 2024-04-22T23:38:45.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.867+0000 7fe299d9b700 1 --2- 172.21.15.107:0/2338685443 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fe28406ed30 0x7fe2840711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:38:45.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.867+0000 7fe299d9b700 1 --2- 172.21.15.107:0/2338685443 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fe28406ed30 0x7fe2840711e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe290007fd0 tx=0x7fe290009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:38:45.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:45.871+0000 7fe28affd700 1 -- 172.21.15.107:0/2338685443 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fe28c05c550 con 0x7fe29c0fbd00 2024-04-22T23:38:46.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.096+0000 7fe2a0daa700 1 -- 172.21.15.107:0/2338685443 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe29c101310 con 0x7fe28406ed30 2024-04-22T23:38:46.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.097+0000 7fe28affd700 1 -- 172.21.15.107:0/2338685443 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fe29c101310 con 0x7fe28406ed30 2024-04-22T23:38:46.091 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-22T23:38:46.091 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c", 2024-04-22T23:38:46.091 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-04-22T23:38:46.091 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-22T23:38:46.091 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [ 2024-04-22T23:38:46.091 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr", 2024-04-22T23:38:46.091 INFO:teuthology.orchestra.run.smithi107.stdout: "mon" 2024-04-22T23:38:46.091 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-22T23:38:46.091 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "4/21 daemons upgraded", 2024-04-22T23:38:46.092 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Currently upgrading mon daemons", 2024-04-22T23:38:46.092 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-04-22T23:38:46.092 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-22T23:38:46.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.099+0000 7fe2a0daa700 1 -- 172.21.15.107:0/2338685443 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fe28406ed30 msgr2=0x7fe2840711e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:46.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.099+0000 7fe2a0daa700 1 --2- 172.21.15.107:0/2338685443 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fe28406ed30 0x7fe2840711e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe290007fd0 tx=0x7fe290009040 comp rx=0 tx=0).stop 2024-04-22T23:38:46.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.099+0000 7fe2a0daa700 1 -- 172.21.15.107:0/2338685443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe29c0fbd00 msgr2=0x7fe29c10d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:46.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.099+0000 7fe2a0daa700 1 --2- 172.21.15.107:0/2338685443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe29c0fbd00 0x7fe29c10d9b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe28c009380 tx=0x7fe28c009e30 comp rx=0 tx=0).stop 2024-04-22T23:38:46.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.099+0000 7fe2a0daa700 1 -- 172.21.15.107:0/2338685443 shutdown_connections 2024-04-22T23:38:46.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.099+0000 7fe2a0daa700 1 --2- 172.21.15.107:0/2338685443 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fe28406ed30 0x7fe2840711e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:46.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.099+0000 7fe2a0daa700 1 --2- 172.21.15.107:0/2338685443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe29c0fbd00 0x7fe29c10d9b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:46.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.099+0000 7fe2a0daa700 1 --2- 172.21.15.107:0/2338685443 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe29c0fcfb0 0x7fe29c10def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:46.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.099+0000 7fe2a0daa700 1 -- 172.21.15.107:0/2338685443 >> 172.21.15.107:0/2338685443 conn(0x7fe29c0f7480 msgr2=0x7fe29c0f8bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:38:46.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.099+0000 7fe2a0daa700 1 -- 172.21.15.107:0/2338685443 shutdown_connections 2024-04-22T23:38:46.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.099+0000 7fe2a0daa700 1 -- 172.21.15.107:0/2338685443 wait complete. 2024-04-22T23:38:46.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.214+0000 7fb33f2d4700 1 -- 172.21.15.107:0/959384304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3380fed60 msgr2=0x7fb3380faea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:46.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.214+0000 7fb33f2d4700 1 --2- 172.21.15.107:0/959384304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3380fed60 0x7fb3380faea0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb328004940 tx=0x7fb328009e30 comp rx=0 tx=0).stop 2024-04-22T23:38:46.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.214+0000 7fb33f2d4700 1 -- 172.21.15.107:0/959384304 shutdown_connections 2024-04-22T23:38:46.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.214+0000 7fb33f2d4700 1 --2- 172.21.15.107:0/959384304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3380fed60 0x7fb3380faea0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:46.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.214+0000 7fb33f2d4700 1 --2- 172.21.15.107:0/959384304 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3380fe410 0x7fb3380fe820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:46.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.214+0000 7fb33f2d4700 1 -- 172.21.15.107:0/959384304 >> 172.21.15.107:0/959384304 conn(0x7fb3380f6bd0 msgr2=0x7fb3380f8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:38:46.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.214+0000 7fb33f2d4700 1 -- 172.21.15.107:0/959384304 shutdown_connections 2024-04-22T23:38:46.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.214+0000 7fb33f2d4700 1 -- 172.21.15.107:0/959384304 wait complete. 2024-04-22T23:38:46.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.214+0000 7fb33f2d4700 1 Processor -- start 2024-04-22T23:38:46.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.214+0000 7fb33f2d4700 1 -- start start 2024-04-22T23:38:46.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.214+0000 7fb33f2d4700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3380fe410 0x7fb33810d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:46.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.214+0000 7fb33f2d4700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3380fed60 0x7fb33810ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:46.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.214+0000 7fb33f2d4700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33810e4d0 con 0x7fb3380fed60 2024-04-22T23:38:46.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.214+0000 7fb33f2d4700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33810ed20 con 0x7fb3380fe410 2024-04-22T23:38:46.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.215+0000 7fb33c86f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3380fed60 0x7fb33810ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:38:46.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.215+0000 7fb33c86f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3380fed60 0x7fb33810ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45746/0 (socket says 172.21.15.107:45746) 2024-04-22T23:38:46.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.215+0000 7fb33d070700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3380fe410 0x7fb33810d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:38:46.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.215+0000 7fb33c86f700 1 -- 172.21.15.107:0/3482288748 learned_addr learned my addr 172.21.15.107:0/3482288748 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:38:46.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.215+0000 7fb33c86f700 1 -- 172.21.15.107:0/3482288748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3380fe410 msgr2=0x7fb33810d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:46.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.215+0000 7fb33c86f700 1 --2- 172.21.15.107:0/3482288748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3380fe410 0x7fb33810d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:46.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.215+0000 7fb33c86f700 1 -- 172.21.15.107:0/3482288748 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb328005040 con 0x7fb3380fed60 2024-04-22T23:38:46.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.215+0000 7fb33d070700 1 --2- 172.21.15.107:0/3482288748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3380fe410 0x7fb33810d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T23:38:46.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.215+0000 7fb33c86f700 1 --2- 172.21.15.107:0/3482288748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3380fed60 0x7fb33810ded0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb328013870 tx=0x7fb328009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:38:46.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.215+0000 7fb32dffb700 1 -- 172.21.15.107:0/3482288748 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb328021070 con 0x7fb3380fed60 2024-04-22T23:38:46.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.215+0000 7fb32dffb700 1 -- 172.21.15.107:0/3482288748 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb328009430 con 0x7fb3380fed60 2024-04-22T23:38:46.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.215+0000 7fb32dffb700 1 -- 172.21.15.107:0/3482288748 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb32800e070 con 0x7fb3380fed60 2024-04-22T23:38:46.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.215+0000 7fb33f2d4700 1 -- 172.21.15.107:0/3482288748 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb33810efa0 con 0x7fb3380fed60 2024-04-22T23:38:46.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.216+0000 7fb33f2d4700 1 -- 172.21.15.107:0/3482288748 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb33810f4e0 con 0x7fb3380fed60 2024-04-22T23:38:46.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.216+0000 7fb33f2d4700 1 -- 172.21.15.107:0/3482288748 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb33805f5f0 con 0x7fb3380fed60 2024-04-22T23:38:46.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.217+0000 7fb32dffb700 1 -- 172.21.15.107:0/3482288748 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 36) v1 ==== 92849+0+0 (secure 0 0 0) 0x7fb32801c750 con 0x7fb3380fed60 2024-04-22T23:38:46.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.217+0000 7fb32dffb700 1 --2- 172.21.15.107:0/3482288748 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fb32406ecc0 0x7fb324071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:38:46.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.218+0000 7fb32dffb700 1 -- 172.21.15.107:0/3482288748 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fb32808fcf0 con 0x7fb3380fed60 2024-04-22T23:38:46.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.218+0000 7fb33d070700 1 --2- 172.21.15.107:0/3482288748 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fb32406ecc0 0x7fb324071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:38:46.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.218+0000 7fb33d070700 1 --2- 172.21.15.107:0/3482288748 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fb32406ecc0 0x7fb324071170 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb334009b80 tx=0x7fb33400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:38:46.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.226+0000 7fb32dffb700 1 -- 172.21.15.107:0/3482288748 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fb32805c430 con 0x7fb3380fed60 2024-04-22T23:38:46.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.521+0000 7fb33f2d4700 1 -- 172.21.15.107:0/3482288748 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb3381058c0 con 0x7fb3380fed60 2024-04-22T23:38:46.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.522+0000 7fb32dffb700 1 -- 172.21.15.107:0/3482288748 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fb32805bfc0 con 0x7fb3380fed60 2024-04-22T23:38:46.516 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_OK 2024-04-22T23:38:46.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.524+0000 7fb33f2d4700 1 -- 172.21.15.107:0/3482288748 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fb32406ecc0 msgr2=0x7fb324071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:46.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.524+0000 7fb33f2d4700 1 --2- 172.21.15.107:0/3482288748 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fb32406ecc0 0x7fb324071170 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb334009b80 tx=0x7fb33400b040 comp rx=0 tx=0).stop 2024-04-22T23:38:46.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.524+0000 7fb33f2d4700 1 -- 172.21.15.107:0/3482288748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3380fed60 msgr2=0x7fb33810ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:38:46.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.524+0000 7fb33f2d4700 1 --2- 172.21.15.107:0/3482288748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3380fed60 0x7fb33810ded0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb328013870 tx=0x7fb328009e30 comp rx=0 tx=0).stop 2024-04-22T23:38:46.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.524+0000 7fb33f2d4700 1 -- 172.21.15.107:0/3482288748 shutdown_connections 2024-04-22T23:38:46.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.524+0000 7fb33f2d4700 1 --2- 172.21.15.107:0/3482288748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3380fe410 0x7fb33810d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:46.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.524+0000 7fb33f2d4700 1 --2- 172.21.15.107:0/3482288748 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fb32406ecc0 0x7fb324071170 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:46.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.524+0000 7fb33f2d4700 1 --2- 172.21.15.107:0/3482288748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3380fed60 0x7fb33810ded0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:38:46.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.524+0000 7fb33f2d4700 1 -- 172.21.15.107:0/3482288748 >> 172.21.15.107:0/3482288748 conn(0x7fb3380f6bd0 msgr2=0x7fb3380f8230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:38:46.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.525+0000 7fb33f2d4700 1 -- 172.21.15.107:0/3482288748 shutdown_connections 2024-04-22T23:38:46.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:38:46.525+0000 7fb33f2d4700 1 -- 172.21.15.107:0/3482288748 wait complete. 2024-04-22T23:38:46.750 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:46 smithi107 ceph-mon[90179]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:38:46.750 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:46 smithi107 ceph-mon[90179]: pgmap v28: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:38:46.750 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:46 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/883461375' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:38:46.750 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:46 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:46.750 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:46 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:46.750 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:46 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:38:46.750 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:46 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:38:46.750 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:46 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:38:46.750 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:46 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:46.750 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:46 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi107"}]: dispatch 2024-04-22T23:38:46.750 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:46 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi107"}]': finished 2024-04-22T23:38:46.751 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:46 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi157"}]: dispatch 2024-04-22T23:38:46.751 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:46 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi157"}]': finished 2024-04-22T23:38:46.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:46 smithi157 ceph-mon[79403]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:38:46.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:46 smithi157 ceph-mon[79403]: pgmap v28: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:38:46.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:46 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/883461375' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:38:46.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:46 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:46.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:46 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:46.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:46 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:38:46.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:46 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:38:46.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:46 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:38:46.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:46 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:46.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:46 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi107"}]: dispatch 2024-04-22T23:38:46.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:46 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi107"}]': finished 2024-04-22T23:38:46.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:46 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi157"}]: dispatch 2024-04-22T23:38:46.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:46 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi157"}]': finished 2024-04-22T23:38:47.753 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:47 smithi107 ceph-mon[90179]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:38:47.753 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:47 smithi107 ceph-mon[90179]: Upgrade: Setting container_image for all mon 2024-04-22T23:38:47.753 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:47 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/3482288748' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:38:47.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:47 smithi157 ceph-mon[79403]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:38:47.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:47 smithi157 ceph-mon[79403]: Upgrade: Setting container_image for all mon 2024-04-22T23:38:47.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:47 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/3482288748' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:38:48.781 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:48 smithi107 ceph-mon[90179]: Upgrade: Updating crash.smithi107 (1/2) 2024-04-22T23:38:48.781 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:48 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:48.781 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:48 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi107", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-22T23:38:48.782 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:48 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:48.782 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:48 smithi107 ceph-mon[90179]: Deploying daemon crash.smithi107 on smithi107 2024-04-22T23:38:48.782 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:48 smithi107 ceph-mon[90179]: pgmap v29: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:38:49.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:48 smithi157 ceph-mon[79403]: Upgrade: Updating crash.smithi107 (1/2) 2024-04-22T23:38:49.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:48 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:49.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:48 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi107", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-22T23:38:49.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:48 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:49.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:48 smithi157 ceph-mon[79403]: Deploying daemon crash.smithi107 on smithi107 2024-04-22T23:38:49.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:48 smithi157 ceph-mon[79403]: pgmap v29: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:38:50.913 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:50 smithi107 ceph-mon[90179]: pgmap v30: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:38:51.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:50 smithi157 ceph-mon[79403]: pgmap v30: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:38:52.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:52 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:52.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:52 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:52.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:52 smithi157 ceph-mon[79403]: pgmap v31: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:38:52.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:52 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:52.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:52 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:52.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:52 smithi107 ceph-mon[90179]: pgmap v31: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:38:53.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:53 smithi157 ceph-mon[79403]: Upgrade: Updating crash.smithi157 (2/2) 2024-04-22T23:38:53.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:53 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:53.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:53 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi157", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-22T23:38:53.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:53 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:53.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:53 smithi157 ceph-mon[79403]: Deploying daemon crash.smithi157 on smithi157 2024-04-22T23:38:53.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:53 smithi157 ceph-mon[79403]: pgmap v32: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:38:54.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:53 smithi107 ceph-mon[90179]: Upgrade: Updating crash.smithi157 (2/2) 2024-04-22T23:38:54.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:53 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:54.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:53 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi157", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-22T23:38:54.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:53 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:38:54.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:53 smithi107 ceph-mon[90179]: Deploying daemon crash.smithi157 on smithi157 2024-04-22T23:38:54.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:53 smithi107 ceph-mon[90179]: pgmap v32: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:38:56.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:56 smithi107 ceph-mon[90179]: pgmap v33: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:38:56.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:56 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:38:56.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:56 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:56.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:56 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:56.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:56 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:38:57.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:56 smithi157 ceph-mon[79403]: pgmap v33: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:38:57.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:56 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:38:57.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:56 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:57.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:56 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:38:57.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:56 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:38:58.956 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:38:58 smithi107 ceph-mon[90179]: pgmap v34: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:38:58.970 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:38:58 smithi157 ceph-mon[79403]: pgmap v34: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:39:00.597 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:00 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:00.597 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:00 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:00.597 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:00 smithi107 ceph-mon[90179]: pgmap v35: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:39:00.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:00 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:00.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:00 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:00.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:00 smithi157 ceph-mon[79403]: pgmap v35: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:39:01.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:01 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:01.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:01 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:01.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:01 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:01.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:01 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:02.815 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:02 smithi157 ceph-mon[79403]: pgmap v36: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:39:02.815 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:02 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:02.815 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:02 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:02.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:02 smithi107 ceph-mon[90179]: pgmap v36: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:39:02.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:02 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:02.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:02 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:03.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:03 smithi107 ceph-mon[90179]: fsmap cephfs:2 {0=cephfs.smithi107.ktuxjl=up:active,1=cephfs.smithi157.rcaris=up:active} 2 up:standby 2024-04-22T23:39:03.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:03 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:03.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:03 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:04.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:03 smithi157 ceph-mon[79403]: fsmap cephfs:2 {0=cephfs.smithi107.ktuxjl=up:active,1=cephfs.smithi157.rcaris=up:active} 2 up:standby 2024-04-22T23:39:04.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:03 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:04.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:03 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:04.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:04 smithi107 ceph-mon[90179]: pgmap v37: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:39:04.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:04 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:04.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:04 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:05.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:04 smithi157 ceph-mon[79403]: pgmap v37: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:39:05.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:04 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:05.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:04 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:06.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:06 smithi107 ceph-mon[90179]: pgmap v38: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:39:06.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:06 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:06.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:06 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:06.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:06 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:39:06.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:06 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:39:06.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:06 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:06.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:06 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:39:06.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:06 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:39:06.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:06 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:39:06.984 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:06 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:39:06.984 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:06 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:06.984 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:06 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi107"}]: dispatch 2024-04-22T23:39:06.984 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:06 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi107"}]': finished 2024-04-22T23:39:06.984 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:06 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi157"}]: dispatch 2024-04-22T23:39:06.984 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:06 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi157"}]': finished 2024-04-22T23:39:06.984 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:06 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-22T23:39:07.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:06 smithi157 ceph-mon[79403]: pgmap v38: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:39:07.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:06 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:07.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:06 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:07.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:06 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:39:07.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:06 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:39:07.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:06 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:07.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:06 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:39:07.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:06 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:39:07.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:06 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:39:07.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:06 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:39:07.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:06 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:07.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:06 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi107"}]: dispatch 2024-04-22T23:39:07.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:06 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi107"}]': finished 2024-04-22T23:39:07.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:06 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi157"}]: dispatch 2024-04-22T23:39:07.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:06 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi157"}]': finished 2024-04-22T23:39:07.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:06 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-22T23:39:07.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:07 smithi107 ceph-mon[90179]: Upgrade: Setting container_image for all crash 2024-04-22T23:39:07.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:07 smithi107 ceph-mon[90179]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-22T23:39:07.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:07 smithi107 ceph-mon[90179]: Upgrade: osd.0 is safe to restart 2024-04-22T23:39:07.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:07 smithi107 ceph-mon[90179]: fsmap cephfs:2 {0=cephfs.smithi107.ktuxjl=up:active,1=cephfs.smithi157.rcaris=up:active} 2 up:standby 2024-04-22T23:39:08.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:07 smithi157 ceph-mon[79403]: Upgrade: Setting container_image for all crash 2024-04-22T23:39:08.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:07 smithi157 ceph-mon[79403]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-22T23:39:08.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:07 smithi157 ceph-mon[79403]: Upgrade: osd.0 is safe to restart 2024-04-22T23:39:08.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:07 smithi157 ceph-mon[79403]: fsmap cephfs:2 {0=cephfs.smithi107.ktuxjl=up:active,1=cephfs.smithi157.rcaris=up:active} 2 up:standby 2024-04-22T23:39:08.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:08 smithi107 ceph-mon[90179]: pgmap v39: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:39:08.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:08 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:08.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:08 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-22T23:39:08.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:08 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:39:09.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:08 smithi157 ceph-mon[79403]: pgmap v39: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:39:09.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:08 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:09.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:08 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-22T23:39:09.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:08 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:39:09.904 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:09 smithi107 ceph-mon[90179]: Upgrade: Updating osd.0 2024-04-22T23:39:09.905 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:09 smithi107 ceph-mon[90179]: Deploying daemon osd.0 on smithi107 2024-04-22T23:39:10.034 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:09 smithi157 ceph-mon[79403]: Upgrade: Updating osd.0 2024-04-22T23:39:10.034 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:09 smithi157 ceph-mon[79403]: Deploying daemon osd.0 on smithi107 2024-04-22T23:39:10.927 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:10 smithi107 ceph-mon[90179]: pgmap v40: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:39:10.928 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:10 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:39:11.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:10 smithi157 ceph-mon[79403]: pgmap v40: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:39:11.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:10 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:39:11.574 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:39:11 smithi107 systemd[1]: Stopping Ceph osd.0 for 8252f6f2-00ff-11ef-bc93-c7b262605968... 2024-04-22T23:39:11.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:11 smithi107 ceph-mon[90179]: osd.0 marked itself down and dead 2024-04-22T23:39:11.983 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:39:11 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-0[44710]: 2024-04-22T23:39:11.580+0000 7f5ec8f9a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T23:39:11.983 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:39:11 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-0[44710]: 2024-04-22T23:39:11.580+0000 7f5ec8f9a700 -1 osd.0 49 *** Got signal Terminated *** 2024-04-22T23:39:11.983 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:39:11 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-0[44710]: 2024-04-22T23:39:11.580+0000 7f5ec8f9a700 -1 osd.0 49 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T23:39:12.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:11 smithi157 ceph-mon[79403]: osd.0 marked itself down and dead 2024-04-22T23:39:12.541 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:39:12 smithi107 podman[98147]: 2024-04-22 23:39:12.264553216 +0000 UTC m=+0.807683435 container died 18a179b0a6182bb80bdb61d801358f51d406f9aa7bb71cbbed33f06bfd167e52 (image=quay.ceph.io/ceph-ci/ceph@sha256:3890cd726bd5e73ae623dabdf4ed76dd5df81c936b8bd8d761026babda969466, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=quincy-70d0cb4) 2024-04-22T23:39:12.917 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:39:12 smithi107 podman[98147]: 2024-04-22 23:39:12.622506107 +0000 UTC m=+1.165636313 container cleanup 18a179b0a6182bb80bdb61d801358f51d406f9aa7bb71cbbed33f06bfd167e52 (image=quay.ceph.io/ceph-ci/ceph@sha256:3890cd726bd5e73ae623dabdf4ed76dd5df81c936b8bd8d761026babda969466, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-0, RELEASE=quincy-70d0cb4, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T23:39:12.917 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:39:12 smithi107 bash[98147]: ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-0 2024-04-22T23:39:12.917 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:12 smithi107 ceph-mon[90179]: pgmap v41: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:39:12.917 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:12 smithi107 ceph-mon[90179]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-22T23:39:12.917 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:12 smithi107 ceph-mon[90179]: osdmap e50: 6 total, 5 up, 6 in 2024-04-22T23:39:13.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:12 smithi157 ceph-mon[79403]: pgmap v41: 65 pgs: 65 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:39:13.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:12 smithi157 ceph-mon[79403]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-22T23:39:13.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:12 smithi157 ceph-mon[79403]: osdmap e50: 6 total, 5 up, 6 in 2024-04-22T23:39:13.233 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:39:12 smithi107 podman[98166]: 2024-04-22 23:39:12.923831193 +0000 UTC m=+0.650664930 container remove 18a179b0a6182bb80bdb61d801358f51d406f9aa7bb71cbbed33f06bfd167e52 (image=quay.ceph.io/ceph-ci/ceph@sha256:3890cd726bd5e73ae623dabdf4ed76dd5df81c936b8bd8d761026babda969466, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=quincy-70d0cb4, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-22T23:39:13.499 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:39:13 smithi107 podman[98257]: 2024-04-22 23:39:13.145969324 +0000 UTC m=+0.018275908 image pull a0434699c1d76af4eea1fc95ff0a22ea73d1e04e609464fbf658d6f39bc7f2fc quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c 2024-04-22T23:39:13.499 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:39:13 smithi107 podman[98257]: 2024-04-22 23:39:13.258050288 +0000 UTC m=+0.130356908 container create 96df418bf598a5b1b581d35ae3b0667231352279ff1c18d23522d3d4cb54d07a (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_CLEAN=True, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True) 2024-04-22T23:39:13.499 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:39:13 smithi107 podman[98257]: 2024-04-22 23:39:13.486148847 +0000 UTC m=+0.358455424 container init 96df418bf598a5b1b581d35ae3b0667231352279ff1c18d23522d3d4cb54d07a (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS) 2024-04-22T23:39:13.499 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:39:13 smithi107 podman[98257]: 2024-04-22 23:39:13.489467333 +0000 UTC m=+0.361773916 container start 96df418bf598a5b1b581d35ae3b0667231352279ff1c18d23522d3d4cb54d07a (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416) 2024-04-22T23:39:13.750 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:39:13 smithi107 podman[98257]: 2024-04-22 23:39:13.593709641 +0000 UTC m=+0.466016224 container attach 96df418bf598a5b1b581d35ae3b0667231352279ff1c18d23522d3d4cb54d07a (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416) 2024-04-22T23:39:14.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:13 smithi157 ceph-mon[79403]: osdmap e51: 6 total, 5 up, 6 in 2024-04-22T23:39:14.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:13 smithi107 ceph-mon[90179]: osdmap e51: 6 total, 5 up, 6 in 2024-04-22T23:39:14.233 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:39:13 smithi107 podman[98257]: 2024-04-22 23:39:13.785840785 +0000 UTC m=+0.658147368 container died 96df418bf598a5b1b581d35ae3b0667231352279ff1c18d23522d3d4cb54d07a (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416) 2024-04-22T23:39:14.575 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:39:14 smithi107 podman[98283]: 2024-04-22 23:39:14.347179158 +0000 UTC m=+0.552465107 container remove 96df418bf598a5b1b581d35ae3b0667231352279ff1c18d23522d3d4cb54d07a (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-0-deactivate, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-22T23:39:14.575 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:39:14 smithi107 systemd[1]: ceph-8252f6f2-00ff-11ef-bc93-c7b262605968@osd.0.service: Deactivated successfully. 2024-04-22T23:39:14.575 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:39:14 smithi107 systemd[1]: Stopped Ceph osd.0 for 8252f6f2-00ff-11ef-bc93-c7b262605968. 2024-04-22T23:39:14.575 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:39:14 smithi107 systemd[1]: ceph-8252f6f2-00ff-11ef-bc93-c7b262605968@osd.0.service: Consumed 54.273s CPU time. 2024-04-22T23:39:14.923 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:14 smithi107 ceph-mon[90179]: pgmap v44: 65 pgs: 9 stale+active+clean, 56 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:39:14.924 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:39:14 smithi107 systemd[1]: Starting Ceph osd.0 for 8252f6f2-00ff-11ef-bc93-c7b262605968... 2024-04-22T23:39:14.924 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:39:14 smithi107 podman[98385]: 2024-04-22 23:39:14.701250567 +0000 UTC m=+0.017984220 image pull a0434699c1d76af4eea1fc95ff0a22ea73d1e04e609464fbf658d6f39bc7f2fc quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c 2024-04-22T23:39:14.924 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:39:14 smithi107 podman[98385]: 2024-04-22 23:39:14.806603142 +0000 UTC m=+0.123336795 container create 75f562ea1f34ee298732de1efa6ce87917f50a0038c5e032056dd7dcac6222b2 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-0-activate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-22T23:39:15.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:14 smithi157 ceph-mon[79403]: pgmap v44: 65 pgs: 9 stale+active+clean, 56 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:39:15.232 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:39:15 smithi107 podman[98385]: 2024-04-22 23:39:15.051635773 +0000 UTC m=+0.368369454 container init 75f562ea1f34ee298732de1efa6ce87917f50a0038c5e032056dd7dcac6222b2 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-0-activate, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-22T23:39:15.233 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:39:15 smithi107 podman[98385]: 2024-04-22 23:39:15.055195263 +0000 UTC m=+0.371928921 container start 75f562ea1f34ee298732de1efa6ce87917f50a0038c5e032056dd7dcac6222b2 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-0-activate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_BRANCH=HEAD) 2024-04-22T23:39:15.233 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:39:15 smithi107 podman[98385]: 2024-04-22 23:39:15.142327226 +0000 UTC m=+0.459060883 container attach 75f562ea1f34ee298732de1efa6ce87917f50a0038c5e032056dd7dcac6222b2 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-0-activate, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T23:39:15.881 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:39:15 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-0-activate[98400]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-22T23:39:15.881 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:39:15 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-0-activate[98400]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-22T23:39:15.881 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:39:15 smithi107 bash[98385]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-22T23:39:15.881 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:39:15 smithi107 bash[98385]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-22T23:39:15.881 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:39:15 smithi107 bash[98385]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-22T23:39:15.881 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:39:15 smithi107 bash[98385]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-22T23:39:15.881 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:39:15 smithi107 bash[98385]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-22T23:39:15.881 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:39:15 smithi107 bash[98385]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-22T23:39:15.881 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:39:15 smithi107 bash[98385]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-04-22T23:39:15.882 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:39:15 smithi107 bash[98385]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-22T23:39:15.882 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:39:15 smithi107 bash[98385]: --> ceph-volume raw activate successful for osd ID: 0 2024-04-22T23:39:15.882 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:39:15 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-0-activate[98400]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-22T23:39:15.882 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:39:15 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-0-activate[98400]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-22T23:39:15.882 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:39:15 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-0-activate[98400]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-22T23:39:15.882 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:39:15 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-0-activate[98400]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-22T23:39:15.882 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:39:15 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-0-activate[98400]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-04-22T23:39:15.882 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:39:15 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-0-activate[98400]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-22T23:39:15.882 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:39:15 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-0-activate[98400]: --> ceph-volume raw activate successful for osd ID: 0 2024-04-22T23:39:15.882 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:39:15 smithi107 podman[98385]: 2024-04-22 23:39:15.556068929 +0000 UTC m=+0.872802585 container died 75f562ea1f34ee298732de1efa6ce87917f50a0038c5e032056dd7dcac6222b2 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-0-activate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-22T23:39:16.482 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:39:16 smithi107 podman[98529]: 2024-04-22 23:39:16.179763778 +0000 UTC m=+0.613712466 container remove 75f562ea1f34ee298732de1efa6ce87917f50a0038c5e032056dd7dcac6222b2 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-0-activate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-22T23:39:16.483 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:39:16 smithi107 podman[98595]: 2024-04-22 23:39:16.299052707 +0000 UTC m=+0.018285922 image pull a0434699c1d76af4eea1fc95ff0a22ea73d1e04e609464fbf658d6f39bc7f2fc quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c 2024-04-22T23:39:16.483 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:39:16 smithi107 podman[98595]: 2024-04-22 23:39:16.397179756 +0000 UTC m=+0.116412970 container create fd0ba8ef0c2748d981390c273ce32b055ed4923db55a95004e5f33783271fb18 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T23:39:16.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:16.638+0000 7f93cb88f700 1 -- 172.21.15.107:0/2981430379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93c4066230 msgr2=0x7f93c40ffaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:16.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:16.638+0000 7f93cb88f700 1 --2- 172.21.15.107:0/2981430379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93c4066230 0x7f93c40ffaf0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f93c0004940 tx=0x7f93c0009e30 comp rx=0 tx=0).stop 2024-04-22T23:39:16.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:16.638+0000 7f93cb88f700 1 -- 172.21.15.107:0/2981430379 shutdown_connections 2024-04-22T23:39:16.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:16.638+0000 7f93cb88f700 1 --2- 172.21.15.107:0/2981430379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93c4100030 0x7f93c4102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:16.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:16.638+0000 7f93cb88f700 1 --2- 172.21.15.107:0/2981430379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93c4066230 0x7f93c40ffaf0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:16.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:16.638+0000 7f93cb88f700 1 -- 172.21.15.107:0/2981430379 >> 172.21.15.107:0/2981430379 conn(0x7f93c40f74c0 msgr2=0x7f93c40f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:39:16.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:16.638+0000 7f93cb88f700 1 -- 172.21.15.107:0/2981430379 shutdown_connections 2024-04-22T23:39:16.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:16.638+0000 7f93cb88f700 1 -- 172.21.15.107:0/2981430379 wait complete. 2024-04-22T23:39:16.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:16.638+0000 7f93cb88f700 1 Processor -- start 2024-04-22T23:39:16.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:16.638+0000 7f93cb88f700 1 -- start start 2024-04-22T23:39:16.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:16.639+0000 7f93cb88f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93c4066230 0x7f93c410dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:39:16.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:16.639+0000 7f93cb88f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93c4100030 0x7f93c410e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:39:16.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:16.639+0000 7f93cb88f700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c410e600 con 0x7f93c4100030 2024-04-22T23:39:16.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:16.639+0000 7f93cb88f700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c410ee50 con 0x7f93c4066230 2024-04-22T23:39:16.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:16.639+0000 7f93c8e2a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93c4100030 0x7f93c410e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:39:16.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:16.639+0000 7f93c8e2a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93c4100030 0x7f93c410e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52212/0 (socket says 172.21.15.107:52212) 2024-04-22T23:39:16.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:16.639+0000 7f93c8e2a700 1 -- 172.21.15.107:0/1749547375 learned_addr learned my addr 172.21.15.107:0/1749547375 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:39:16.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:16.639+0000 7f93c962b700 1 --2- 172.21.15.107:0/1749547375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93c4066230 0x7f93c410dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:39:16.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:16.639+0000 7f93c8e2a700 1 -- 172.21.15.107:0/1749547375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93c4066230 msgr2=0x7f93c410dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:16.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:16.639+0000 7f93c8e2a700 1 --2- 172.21.15.107:0/1749547375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93c4066230 0x7f93c410dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:16.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:16.639+0000 7f93c8e2a700 1 -- 172.21.15.107:0/1749547375 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c0005040 con 0x7f93c4100030 2024-04-22T23:39:16.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:16.639+0000 7f93c962b700 1 --2- 172.21.15.107:0/1749547375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93c4066230 0x7f93c410dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T23:39:16.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:16.639+0000 7f93c8e2a700 1 --2- 172.21.15.107:0/1749547375 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93c4100030 0x7f93c410e000 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f93b4002a50 tx=0x7f93b4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:39:16.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:16.639+0000 7f93b9ffb700 1 -- 172.21.15.107:0/1749547375 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93b4007cc0 con 0x7f93c4100030 2024-04-22T23:39:16.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:16.639+0000 7f93b9ffb700 1 -- 172.21.15.107:0/1749547375 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f93b400f040 con 0x7f93c4100030 2024-04-22T23:39:16.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:16.639+0000 7f93b9ffb700 1 -- 172.21.15.107:0/1749547375 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93b400daf0 con 0x7f93c4100030 2024-04-22T23:39:16.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:16.639+0000 7f93cb88f700 1 -- 172.21.15.107:0/1749547375 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93c410f130 con 0x7f93c4100030 2024-04-22T23:39:16.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:16.640+0000 7f93cb88f700 1 -- 172.21.15.107:0/1749547375 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93c410f610 con 0x7f93c4100030 2024-04-22T23:39:16.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:16.641+0000 7f93cb88f700 1 -- 172.21.15.107:0/1749547375 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93c405f5f0 con 0x7f93c4100030 2024-04-22T23:39:16.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:16.641+0000 7f93b9ffb700 1 -- 172.21.15.107:0/1749547375 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 36) v1 ==== 92849+0+0 (secure 0 0 0) 0x7f93b4009810 con 0x7f93c4100030 2024-04-22T23:39:16.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:16.642+0000 7f93b9ffb700 1 --2- 172.21.15.107:0/1749547375 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f93b006ed20 0x7f93b00711d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:39:16.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:16.642+0000 7f93b9ffb700 1 -- 172.21.15.107:0/1749547375 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f93b408e8b0 con 0x7f93c4100030 2024-04-22T23:39:16.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:16.642+0000 7f93c962b700 1 --2- 172.21.15.107:0/1749547375 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f93b006ed20 0x7f93b00711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:39:16.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:16.642+0000 7f93c962b700 1 --2- 172.21.15.107:0/1749547375 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f93b006ed20 0x7f93b00711d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f93c0009fd0 tx=0x7f93c0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:39:16.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:16.647+0000 7f93b9ffb700 1 -- 172.21.15.107:0/1749547375 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f93b405af70 con 0x7f93c4100030 2024-04-22T23:39:16.755 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:39:16 smithi107 podman[98595]: 2024-04-22 23:39:16.608020064 +0000 UTC m=+0.327253281 container init fd0ba8ef0c2748d981390c273ce32b055ed4923db55a95004e5f33783271fb18 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-adk-testing-2024-04-22-1618-43be020) 2024-04-22T23:39:16.755 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:39:16 smithi107 podman[98595]: 2024-04-22 23:39:16.611171995 +0000 UTC m=+0.330405213 container start fd0ba8ef0c2748d981390c273ce32b055ed4923db55a95004e5f33783271fb18 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-adk-testing-2024-04-22-1618-43be020) 2024-04-22T23:39:16.756 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:39:16 smithi107 ceph-osd[98618]: -- 172.21.15.107:0/3024746539 <== mon.0 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5598c0667680 con 0x5598c0631000 2024-04-22T23:39:16.756 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:39:16 smithi107 bash[98595]: fd0ba8ef0c2748d981390c273ce32b055ed4923db55a95004e5f33783271fb18 2024-04-22T23:39:16.756 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:39:16 smithi107 systemd[1]: Started Ceph osd.0 for 8252f6f2-00ff-11ef-bc93-c7b262605968. 2024-04-22T23:39:16.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:16.897+0000 7f93cb88f700 1 -- 172.21.15.107:0/1749547375 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f93c410f9c0 con 0x7f93b006ed20 2024-04-22T23:39:16.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:16.898+0000 7f93b9ffb700 1 -- 172.21.15.107:0/1749547375 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f93c410f9c0 con 0x7f93b006ed20 2024-04-22T23:39:16.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:16.900+0000 7f93cb88f700 1 -- 172.21.15.107:0/1749547375 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f93b006ed20 msgr2=0x7f93b00711d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:16.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:16.900+0000 7f93cb88f700 1 --2- 172.21.15.107:0/1749547375 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f93b006ed20 0x7f93b00711d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f93c0009fd0 tx=0x7f93c0009e30 comp rx=0 tx=0).stop 2024-04-22T23:39:16.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:16.900+0000 7f93cb88f700 1 -- 172.21.15.107:0/1749547375 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93c4100030 msgr2=0x7f93c410e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:16.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:16.900+0000 7f93cb88f700 1 --2- 172.21.15.107:0/1749547375 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93c4100030 0x7f93c410e000 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f93b4002a50 tx=0x7f93b4004600 comp rx=0 tx=0).stop 2024-04-22T23:39:16.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:16.900+0000 7f93cb88f700 1 -- 172.21.15.107:0/1749547375 shutdown_connections 2024-04-22T23:39:16.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:16.900+0000 7f93cb88f700 1 --2- 172.21.15.107:0/1749547375 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93c4066230 0x7f93c410dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:16.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:16.901+0000 7f93cb88f700 1 --2- 172.21.15.107:0/1749547375 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f93b006ed20 0x7f93b00711d0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:16.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:16.901+0000 7f93cb88f700 1 --2- 172.21.15.107:0/1749547375 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93c4100030 0x7f93c410e000 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:16.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:16.901+0000 7f93cb88f700 1 -- 172.21.15.107:0/1749547375 >> 172.21.15.107:0/1749547375 conn(0x7f93c40f74c0 msgr2=0x7f93c40f7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:39:16.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:16.901+0000 7f93cb88f700 1 -- 172.21.15.107:0/1749547375 shutdown_connections 2024-04-22T23:39:16.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:16.901+0000 7f93cb88f700 1 -- 172.21.15.107:0/1749547375 wait complete. 2024-04-22T23:39:16.906 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-04-22T23:39:17.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.009+0000 7f62a476c700 1 -- 172.21.15.107:0/1314797272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f629c0fcf20 msgr2=0x7f629c0fd330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:17.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.009+0000 7f62a476c700 1 --2- 172.21.15.107:0/1314797272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f629c0fcf20 0x7f629c0fd330 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f628c004940 tx=0x7f628c009e30 comp rx=0 tx=0).stop 2024-04-22T23:39:17.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.009+0000 7f62a476c700 1 -- 172.21.15.107:0/1314797272 shutdown_connections 2024-04-22T23:39:17.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.009+0000 7f62a476c700 1 --2- 172.21.15.107:0/1314797272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f629c0fe090 0x7f629c0fe500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:17.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.009+0000 7f62a476c700 1 --2- 172.21.15.107:0/1314797272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f629c0fcf20 0x7f629c0fd330 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:17.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.009+0000 7f62a476c700 1 -- 172.21.15.107:0/1314797272 >> 172.21.15.107:0/1314797272 conn(0x7f629c0f85c0 msgr2=0x7f629c0faa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:39:17.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.009+0000 7f62a476c700 1 -- 172.21.15.107:0/1314797272 shutdown_connections 2024-04-22T23:39:17.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.009+0000 7f62a476c700 1 -- 172.21.15.107:0/1314797272 wait complete. 2024-04-22T23:39:17.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.010+0000 7f62a476c700 1 Processor -- start 2024-04-22T23:39:17.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.010+0000 7f62a476c700 1 -- start start 2024-04-22T23:39:17.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.010+0000 7f62a476c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f629c0fcf20 0x7f629c10dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:39:17.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.010+0000 7f62a476c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f629c0fe090 0x7f629c10dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:39:17.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.010+0000 7f62a476c700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629c10e5f0 con 0x7f629c0fe090 2024-04-22T23:39:17.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.010+0000 7f62a476c700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629c10edf0 con 0x7f629c0fcf20 2024-04-22T23:39:17.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.010+0000 7f62a1d07700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f629c0fe090 0x7f629c10dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:39:17.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.010+0000 7f62a1d07700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f629c0fe090 0x7f629c10dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52218/0 (socket says 172.21.15.107:52218) 2024-04-22T23:39:17.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.010+0000 7f62a1d07700 1 -- 172.21.15.107:0/1759863053 learned_addr learned my addr 172.21.15.107:0/1759863053 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:39:17.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.010+0000 7f62a2508700 1 --2- 172.21.15.107:0/1759863053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f629c0fcf20 0x7f629c10dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:39:17.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.010+0000 7f62a1d07700 1 -- 172.21.15.107:0/1759863053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f629c0fcf20 msgr2=0x7f629c10dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:17.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.010+0000 7f62a1d07700 1 --2- 172.21.15.107:0/1759863053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f629c0fcf20 0x7f629c10dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:17.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.010+0000 7f62a1d07700 1 -- 172.21.15.107:0/1759863053 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f628c005040 con 0x7f629c0fe090 2024-04-22T23:39:17.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.011+0000 7f62a1d07700 1 --2- 172.21.15.107:0/1759863053 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f629c0fe090 0x7f629c10dff0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6298002a50 tx=0x7f62980046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:39:17.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.011+0000 7f6292ffd700 1 -- 172.21.15.107:0/1759863053 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6298007cc0 con 0x7f629c0fe090 2024-04-22T23:39:17.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.011+0000 7f6292ffd700 1 -- 172.21.15.107:0/1759863053 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f629800f040 con 0x7f629c0fe090 2024-04-22T23:39:17.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.011+0000 7f62a476c700 1 -- 172.21.15.107:0/1759863053 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f629c10f0d0 con 0x7f629c0fe090 2024-04-22T23:39:17.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.011+0000 7f6292ffd700 1 -- 172.21.15.107:0/1759863053 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f629800da30 con 0x7f629c0fe090 2024-04-22T23:39:17.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.011+0000 7f62a476c700 1 -- 172.21.15.107:0/1759863053 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f629c10f610 con 0x7f629c0fe090 2024-04-22T23:39:17.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.012+0000 7f62a476c700 1 -- 172.21.15.107:0/1759863053 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f629c05f5f0 con 0x7f629c0fe090 2024-04-22T23:39:17.007 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:16 smithi107 ceph-mon[90179]: pgmap v45: 65 pgs: 9 stale+active+clean, 56 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:39:17.007 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:16 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:17.007 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:16 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:17.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.012+0000 7f6292ffd700 1 -- 172.21.15.107:0/1759863053 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 36) v1 ==== 92849+0+0 (secure 0 0 0) 0x7f629800db90 con 0x7f629c0fe090 2024-04-22T23:39:17.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.013+0000 7f6292ffd700 1 --2- 172.21.15.107:0/1759863053 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f628806ed20 0x7f62880711d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:39:17.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.013+0000 7f6292ffd700 1 -- 172.21.15.107:0/1759863053 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f629808e8b0 con 0x7f629c0fe090 2024-04-22T23:39:17.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.013+0000 7f62a2508700 1 --2- 172.21.15.107:0/1759863053 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f628806ed20 0x7f62880711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:39:17.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.013+0000 7f62a2508700 1 --2- 172.21.15.107:0/1759863053 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f628806ed20 0x7f62880711d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f628c00b970 tx=0x7f628c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:39:17.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.018+0000 7f6292ffd700 1 -- 172.21.15.107:0/1759863053 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f629805af70 con 0x7f629c0fe090 2024-04-22T23:39:17.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:16 smithi157 ceph-mon[79403]: pgmap v45: 65 pgs: 9 stale+active+clean, 56 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:39:17.133 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:16 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:17.133 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:16 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:17.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.249+0000 7f62a476c700 1 -- 172.21.15.107:0/1759863053 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f629c10f940 con 0x7f628806ed20 2024-04-22T23:39:17.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.250+0000 7f6292ffd700 1 -- 172.21.15.107:0/1759863053 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f629c10f940 con 0x7f628806ed20 2024-04-22T23:39:17.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.252+0000 7f62a476c700 1 -- 172.21.15.107:0/1759863053 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f628806ed20 msgr2=0x7f62880711d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:17.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.252+0000 7f62a476c700 1 --2- 172.21.15.107:0/1759863053 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f628806ed20 0x7f62880711d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f628c00b970 tx=0x7f628c009e30 comp rx=0 tx=0).stop 2024-04-22T23:39:17.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.253+0000 7f62a476c700 1 -- 172.21.15.107:0/1759863053 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f629c0fe090 msgr2=0x7f629c10dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:17.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.253+0000 7f62a476c700 1 --2- 172.21.15.107:0/1759863053 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f629c0fe090 0x7f629c10dff0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6298002a50 tx=0x7f62980046b0 comp rx=0 tx=0).stop 2024-04-22T23:39:17.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.253+0000 7f62a476c700 1 -- 172.21.15.107:0/1759863053 shutdown_connections 2024-04-22T23:39:17.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.253+0000 7f62a476c700 1 --2- 172.21.15.107:0/1759863053 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f629c0fcf20 0x7f629c10dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:17.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.253+0000 7f62a476c700 1 --2- 172.21.15.107:0/1759863053 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f628806ed20 0x7f62880711d0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:17.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.253+0000 7f62a476c700 1 --2- 172.21.15.107:0/1759863053 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f629c0fe090 0x7f629c10dff0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:17.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.253+0000 7f62a476c700 1 -- 172.21.15.107:0/1759863053 >> 172.21.15.107:0/1759863053 conn(0x7f629c0f85c0 msgr2=0x7f629c0f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:39:17.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.253+0000 7f62a476c700 1 -- 172.21.15.107:0/1759863053 shutdown_connections 2024-04-22T23:39:17.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.253+0000 7f62a476c700 1 -- 172.21.15.107:0/1759863053 wait complete. 2024-04-22T23:39:17.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.363+0000 7f2f5927f700 1 -- 172.21.15.107:0/2881530917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f540fcff0 msgr2=0x7f2f540fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:17.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.363+0000 7f2f5927f700 1 --2- 172.21.15.107:0/2881530917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f540fcff0 0x7f2f540fd460 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2f48004940 tx=0x7f2f48009e30 comp rx=0 tx=0).stop 2024-04-22T23:39:17.357 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:39:17 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-0[98610]: 2024-04-22T23:39:17.158+0000 7fe1a24907c0 -1 Falling back to public interface 2024-04-22T23:39:17.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.363+0000 7f2f5927f700 1 -- 172.21.15.107:0/2881530917 shutdown_connections 2024-04-22T23:39:17.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.363+0000 7f2f5927f700 1 --2- 172.21.15.107:0/2881530917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f540fcff0 0x7f2f540fd460 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:17.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.363+0000 7f2f5927f700 1 --2- 172.21.15.107:0/2881530917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f540fbd40 0x7f2f540fc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:17.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.363+0000 7f2f5927f700 1 -- 172.21.15.107:0/2881530917 >> 172.21.15.107:0/2881530917 conn(0x7f2f540f74c0 msgr2=0x7f2f540f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:39:17.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.363+0000 7f2f5927f700 1 -- 172.21.15.107:0/2881530917 shutdown_connections 2024-04-22T23:39:17.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.363+0000 7f2f5927f700 1 -- 172.21.15.107:0/2881530917 wait complete. 2024-04-22T23:39:17.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.364+0000 7f2f5927f700 1 Processor -- start 2024-04-22T23:39:17.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.364+0000 7f2f5927f700 1 -- start start 2024-04-22T23:39:17.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.364+0000 7f2f5927f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f540fbd40 0x7f2f5410da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:39:17.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.364+0000 7f2f5927f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f540fcff0 0x7f2f5410dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:39:17.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.364+0000 7f2f5927f700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5410e5b0 con 0x7f2f540fcff0 2024-04-22T23:39:17.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.364+0000 7f2f5927f700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5410edb0 con 0x7f2f540fbd40 2024-04-22T23:39:17.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.364+0000 7f2f527fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f540fcff0 0x7f2f5410dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:39:17.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.364+0000 7f2f527fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f540fcff0 0x7f2f5410dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52234/0 (socket says 172.21.15.107:52234) 2024-04-22T23:39:17.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.364+0000 7f2f52ffd700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f540fbd40 0x7f2f5410da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:39:17.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.364+0000 7f2f527fc700 1 -- 172.21.15.107:0/1332845861 learned_addr learned my addr 172.21.15.107:0/1332845861 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:39:17.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.365+0000 7f2f52ffd700 1 -- 172.21.15.107:0/1332845861 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f540fcff0 msgr2=0x7f2f5410dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:17.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.365+0000 7f2f52ffd700 1 --2- 172.21.15.107:0/1332845861 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f540fcff0 0x7f2f5410dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:17.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.365+0000 7f2f52ffd700 1 -- 172.21.15.107:0/1332845861 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f48005040 con 0x7f2f540fbd40 2024-04-22T23:39:17.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.365+0000 7f2f527fc700 1 --2- 172.21.15.107:0/1332845861 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f540fcff0 0x7f2f5410dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T23:39:17.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.365+0000 7f2f52ffd700 1 --2- 172.21.15.107:0/1332845861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f540fbd40 0x7f2f5410da70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2f44002820 tx=0x7f2f44004570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:39:17.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.365+0000 7f2f3b7fe700 1 -- 172.21.15.107:0/1332845861 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f44007cc0 con 0x7f2f540fbd40 2024-04-22T23:39:17.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.365+0000 7f2f3b7fe700 1 -- 172.21.15.107:0/1332845861 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2f4400f040 con 0x7f2f540fbd40 2024-04-22T23:39:17.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.365+0000 7f2f5927f700 1 -- 172.21.15.107:0/1332845861 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f5410f090 con 0x7f2f540fbd40 2024-04-22T23:39:17.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.365+0000 7f2f3b7fe700 1 -- 172.21.15.107:0/1332845861 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f44013920 con 0x7f2f540fbd40 2024-04-22T23:39:17.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.365+0000 7f2f5927f700 1 -- 172.21.15.107:0/1332845861 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f5410f5d0 con 0x7f2f540fbd40 2024-04-22T23:39:17.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.366+0000 7f2f5927f700 1 -- 172.21.15.107:0/1332845861 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f54101b80 con 0x7f2f540fbd40 2024-04-22T23:39:17.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.368+0000 7f2f3b7fe700 1 -- 172.21.15.107:0/1332845861 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 36) v1 ==== 92849+0+0 (secure 0 0 0) 0x7f2f44013ac0 con 0x7f2f540fbd40 2024-04-22T23:39:17.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.368+0000 7f2f3b7fe700 1 --2- 172.21.15.107:0/1332845861 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f2f3c06ed20 0x7f2f3c0711d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:39:17.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.368+0000 7f2f3b7fe700 1 -- 172.21.15.107:0/1332845861 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f2f4408ea40 con 0x7f2f540fbd40 2024-04-22T23:39:17.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.368+0000 7f2f527fc700 1 --2- 172.21.15.107:0/1332845861 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f2f3c06ed20 0x7f2f3c0711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:39:17.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.369+0000 7f2f527fc700 1 --2- 172.21.15.107:0/1332845861 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f2f3c06ed20 0x7f2f3c0711d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2f48004f90 tx=0x7f2f48009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:39:17.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.373+0000 7f2f3b7fe700 1 -- 172.21.15.107:0/1332845861 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f2f4405b180 con 0x7f2f540fbd40 2024-04-22T23:39:17.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.594+0000 7f2f5927f700 1 -- 172.21.15.107:0/1332845861 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2f5410f840 con 0x7f2f3c06ed20 2024-04-22T23:39:17.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.604+0000 7f2f3b7fe700 1 -- 172.21.15.107:0/1332845861 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f2f5410f840 con 0x7f2f3c06ed20 2024-04-22T23:39:17.598 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-22T23:39:17.598 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (2m) 15s ago 12m 17.2M - 0.25.0 c8568f914cd2 ec1a5c6260de 2024-04-22T23:39:17.598 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (26s) 15s ago 12m 6656k - 19.0.0-3252-g43be0201 a0434699c1d7 e074fd2cbb9c 2024-04-22T23:39:17.598 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi157 smithi157 running (21s) 18s ago 11m 6651k - 19.0.0-3252-g43be0201 a0434699c1d7 abc2649dee22 2024-04-22T23:39:17.598 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (2m) 15s ago 12m 80.2M - 9.4.7 954c08fa6188 31449ff442d2 2024-04-22T23:39:17.598 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.dfvvkw smithi107 running (6m) 15s ago 6m 13.3M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 08d3aa8d717d 2024-04-22T23:39:17.598 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.ktuxjl smithi107 running (6m) 15s ago 6m 124M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 762f27fe0f30 2024-04-22T23:39:17.598 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi157.rcaris smithi157 running (6m) 18s ago 6m 129M - 17.2.7-884-g70d0cb4b cafd8ddac8fa d23366ac085d 2024-04-22T23:39:17.598 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi157.ufywse smithi157 running (6m) 18s ago 6m 14.1M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 8b00d61eb355 2024-04-22T23:39:17.598 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.uipxvm smithi107 *:8443,9283,8765 running (2m) 15s ago 13m 497M - 19.0.0-3252-g43be0201 a0434699c1d7 d20f05dfe4f2 2024-04-22T23:39:17.599 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi157.tuylis smithi157 *:8443,9283,8765 running (104s) 18s ago 11m 449M - 19.0.0-3252-g43be0201 a0434699c1d7 957e598c9001 2024-04-22T23:39:17.599 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (90s) 15s ago 13m 44.0M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 bc72bbbe2c39 2024-04-22T23:39:17.599 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi157 smithi157 running (67s) 18s ago 11m 31.7M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 ea88de8be4fe 2024-04-22T23:39:17.599 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (2m) 15s ago 12m 16.6M - 1.5.0 0da6a335fe13 5205cfd9bd51 2024-04-22T23:39:17.599 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi157 smithi157 *:9100 running (2m) 18s ago 11m 17.0M - 1.5.0 0da6a335fe13 168ca191bb8c 2024-04-22T23:39:17.599 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 starting - - - 1557M 2024-04-22T23:39:17.599 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (10m) 15s ago 10m 242M 1557M 17.2.7-884-g70d0cb4b cafd8ddac8fa c7272eb84ddb 2024-04-22T23:39:17.599 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (9m) 15s ago 9m 195M 1557M 17.2.7-884-g70d0cb4b cafd8ddac8fa 9db1788096ad 2024-04-22T23:39:17.599 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi157 running (9m) 18s ago 9m 266M 2581M 17.2.7-884-g70d0cb4b cafd8ddac8fa c9fb2d1a13dd 2024-04-22T23:39:17.599 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi157 running (8m) 18s ago 8m 227M 2581M 17.2.7-884-g70d0cb4b cafd8ddac8fa 24a16e19951f 2024-04-22T23:39:17.599 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi157 running (8m) 18s ago 8m 226M 2581M 17.2.7-884-g70d0cb4b cafd8ddac8fa 4c5b7a7ef8bd 2024-04-22T23:39:17.599 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (110s) 15s ago 11m 56.2M - 2.43.0 a07b618ecd1d 99a4ea8594da 2024-04-22T23:39:17.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.606+0000 7f2f5927f700 1 -- 172.21.15.107:0/1332845861 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f2f3c06ed20 msgr2=0x7f2f3c0711d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:17.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.606+0000 7f2f5927f700 1 --2- 172.21.15.107:0/1332845861 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f2f3c06ed20 0x7f2f3c0711d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2f48004f90 tx=0x7f2f48009e30 comp rx=0 tx=0).stop 2024-04-22T23:39:17.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.606+0000 7f2f5927f700 1 -- 172.21.15.107:0/1332845861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f540fbd40 msgr2=0x7f2f5410da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:17.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.606+0000 7f2f5927f700 1 --2- 172.21.15.107:0/1332845861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f540fbd40 0x7f2f5410da70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2f44002820 tx=0x7f2f44004570 comp rx=0 tx=0).stop 2024-04-22T23:39:17.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.606+0000 7f2f5927f700 1 -- 172.21.15.107:0/1332845861 shutdown_connections 2024-04-22T23:39:17.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.606+0000 7f2f5927f700 1 --2- 172.21.15.107:0/1332845861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f540fbd40 0x7f2f5410da70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:17.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.606+0000 7f2f5927f700 1 --2- 172.21.15.107:0/1332845861 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f2f3c06ed20 0x7f2f3c0711d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:17.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.606+0000 7f2f5927f700 1 --2- 172.21.15.107:0/1332845861 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f540fcff0 0x7f2f5410dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:17.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.606+0000 7f2f5927f700 1 -- 172.21.15.107:0/1332845861 >> 172.21.15.107:0/1332845861 conn(0x7f2f540f74c0 msgr2=0x7f2f54065b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:39:17.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.606+0000 7f2f5927f700 1 -- 172.21.15.107:0/1332845861 shutdown_connections 2024-04-22T23:39:17.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.607+0000 7f2f5927f700 1 -- 172.21.15.107:0/1332845861 wait complete. 2024-04-22T23:39:17.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.713+0000 7f195349a700 1 -- 172.21.15.107:0/2612463073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f194c102390 msgr2=0x7f194c104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:17.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.713+0000 7f195349a700 1 --2- 172.21.15.107:0/2612463073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f194c102390 0x7f194c104770 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f193c004940 tx=0x7f193c009e30 comp rx=0 tx=0).stop 2024-04-22T23:39:17.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.713+0000 7f195349a700 1 -- 172.21.15.107:0/2612463073 shutdown_connections 2024-04-22T23:39:17.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.713+0000 7f195349a700 1 --2- 172.21.15.107:0/2612463073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f194c102390 0x7f194c104770 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:17.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.713+0000 7f195349a700 1 --2- 172.21.15.107:0/2612463073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f194c066f20 0x7f194c101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:17.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.713+0000 7f195349a700 1 -- 172.21.15.107:0/2612463073 >> 172.21.15.107:0/2612463073 conn(0x7f194c0f7480 msgr2=0x7f194c0f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:39:17.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.713+0000 7f195349a700 1 -- 172.21.15.107:0/2612463073 shutdown_connections 2024-04-22T23:39:17.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.713+0000 7f195349a700 1 -- 172.21.15.107:0/2612463073 wait complete. 2024-04-22T23:39:17.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.713+0000 7f195349a700 1 Processor -- start 2024-04-22T23:39:17.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.713+0000 7f195349a700 1 -- start start 2024-04-22T23:39:17.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.714+0000 7f195349a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f194c066f20 0x7f194c10fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:39:17.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.714+0000 7f195349a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f194c102390 0x7f194c110120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:39:17.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.714+0000 7f195349a700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194c110720 con 0x7f194c102390 2024-04-22T23:39:17.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.714+0000 7f195349a700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194c110890 con 0x7f194c066f20 2024-04-22T23:39:17.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.714+0000 7f1950a35700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f194c102390 0x7f194c110120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:39:17.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.714+0000 7f1950a35700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f194c102390 0x7f194c110120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52266/0 (socket says 172.21.15.107:52266) 2024-04-22T23:39:17.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.714+0000 7f1950a35700 1 -- 172.21.15.107:0/3832774806 learned_addr learned my addr 172.21.15.107:0/3832774806 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:39:17.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.714+0000 7f1951236700 1 --2- 172.21.15.107:0/3832774806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f194c066f20 0x7f194c10fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:39:17.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.714+0000 7f1950a35700 1 -- 172.21.15.107:0/3832774806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f194c066f20 msgr2=0x7f194c10fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:17.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.714+0000 7f1950a35700 1 --2- 172.21.15.107:0/3832774806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f194c066f20 0x7f194c10fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:17.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.714+0000 7f1950a35700 1 -- 172.21.15.107:0/3832774806 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f193c005040 con 0x7f194c102390 2024-04-22T23:39:17.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.714+0000 7f1950a35700 1 --2- 172.21.15.107:0/3832774806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f194c102390 0x7f194c110120 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f193c009fd0 tx=0x7f193c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:39:17.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.714+0000 7f1941ffb700 1 -- 172.21.15.107:0/3832774806 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f193c00e040 con 0x7f194c102390 2024-04-22T23:39:17.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.714+0000 7f1941ffb700 1 -- 172.21.15.107:0/3832774806 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f193c00bcc0 con 0x7f194c102390 2024-04-22T23:39:17.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.714+0000 7f195349a700 1 -- 172.21.15.107:0/3832774806 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f194c115290 con 0x7f194c102390 2024-04-22T23:39:17.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.714+0000 7f1941ffb700 1 -- 172.21.15.107:0/3832774806 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f193c01c630 con 0x7f194c102390 2024-04-22T23:39:17.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.715+0000 7f195349a700 1 -- 172.21.15.107:0/3832774806 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f194c115850 con 0x7f194c102390 2024-04-22T23:39:17.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.716+0000 7f195349a700 1 -- 172.21.15.107:0/3832774806 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f194c04c710 con 0x7f194c102390 2024-04-22T23:39:17.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.716+0000 7f1941ffb700 1 -- 172.21.15.107:0/3832774806 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 36) v1 ==== 92849+0+0 (secure 0 0 0) 0x7f193c013980 con 0x7f194c102390 2024-04-22T23:39:17.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.717+0000 7f1941ffb700 1 --2- 172.21.15.107:0/3832774806 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f193806ed30 0x7f19380711e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:39:17.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.717+0000 7f1941ffb700 1 -- 172.21.15.107:0/3832774806 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f193c08fc10 con 0x7f194c102390 2024-04-22T23:39:17.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.717+0000 7f1951236700 1 --2- 172.21.15.107:0/3832774806 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f193806ed30 0x7f19380711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:39:17.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.717+0000 7f1951236700 1 --2- 172.21.15.107:0/3832774806 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f193806ed30 0x7f19380711e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f194c066c60 tx=0x7f194800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:39:17.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:17.722+0000 7f1941ffb700 1 -- 172.21.15.107:0/3832774806 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f193c05c410 con 0x7f194c102390 2024-04-22T23:39:17.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:17 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:39:18.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.018+0000 7f195349a700 1 -- 172.21.15.107:0/3832774806 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f194c115cd0 con 0x7f194c102390 2024-04-22T23:39:18.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.018+0000 7f1941ffb700 1 -- 172.21.15.107:0/3832774806 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+735 (secure 0 0 0) 0x7f193c05bfa0 con 0x7f194c102390 2024-04-22T23:39:18.012 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-22T23:39:18.012 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-04-22T23:39:18.012 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:39:18.012 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:39:18.012 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-04-22T23:39:18.012 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:39:18.012 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:39:18.012 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-04-22T23:39:18.012 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 5 2024-04-22T23:39:18.013 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:39:18.013 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-04-22T23:39:18.013 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 4 2024-04-22T23:39:18.013 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:39:18.013 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-04-22T23:39:18.013 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 9, 2024-04-22T23:39:18.013 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 4 2024-04-22T23:39:18.013 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T23:39:18.013 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-22T23:39:18.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.020+0000 7f195349a700 1 -- 172.21.15.107:0/3832774806 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f193806ed30 msgr2=0x7f19380711e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:18.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.020+0000 7f195349a700 1 --2- 172.21.15.107:0/3832774806 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f193806ed30 0x7f19380711e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f194c066c60 tx=0x7f194800b040 comp rx=0 tx=0).stop 2024-04-22T23:39:18.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.020+0000 7f195349a700 1 -- 172.21.15.107:0/3832774806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f194c102390 msgr2=0x7f194c110120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:18.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.020+0000 7f195349a700 1 --2- 172.21.15.107:0/3832774806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f194c102390 0x7f194c110120 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f193c009fd0 tx=0x7f193c009e30 comp rx=0 tx=0).stop 2024-04-22T23:39:18.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.020+0000 7f195349a700 1 -- 172.21.15.107:0/3832774806 shutdown_connections 2024-04-22T23:39:18.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.020+0000 7f195349a700 1 --2- 172.21.15.107:0/3832774806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f194c066f20 0x7f194c10fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:18.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.020+0000 7f195349a700 1 --2- 172.21.15.107:0/3832774806 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f193806ed30 0x7f19380711e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:18.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.020+0000 7f195349a700 1 --2- 172.21.15.107:0/3832774806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f194c102390 0x7f194c110120 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:18.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.021+0000 7f195349a700 1 -- 172.21.15.107:0/3832774806 >> 172.21.15.107:0/3832774806 conn(0x7f194c0f7480 msgr2=0x7f194c0f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:39:18.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.021+0000 7f195349a700 1 -- 172.21.15.107:0/3832774806 shutdown_connections 2024-04-22T23:39:18.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.021+0000 7f195349a700 1 -- 172.21.15.107:0/3832774806 wait complete. 2024-04-22T23:39:18.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.129+0000 7f05d256d700 1 -- 172.21.15.107:0/4043868330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05cc066b20 msgr2=0x7f05cc0ffa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:18.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.129+0000 7f05d256d700 1 --2- 172.21.15.107:0/4043868330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05cc066b20 0x7f05cc0ffa40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f05bc004940 tx=0x7f05bc009e30 comp rx=0 tx=0).stop 2024-04-22T23:39:18.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.129+0000 7f05d256d700 1 -- 172.21.15.107:0/4043868330 shutdown_connections 2024-04-22T23:39:18.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.129+0000 7f05d256d700 1 --2- 172.21.15.107:0/4043868330 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05cc0fff80 0x7f05cc102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:18.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.129+0000 7f05d256d700 1 --2- 172.21.15.107:0/4043868330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05cc066b20 0x7f05cc0ffa40 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:18.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.129+0000 7f05d256d700 1 -- 172.21.15.107:0/4043868330 >> 172.21.15.107:0/4043868330 conn(0x7f05cc0f7450 msgr2=0x7f05cc0f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:39:18.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.129+0000 7f05d256d700 1 -- 172.21.15.107:0/4043868330 shutdown_connections 2024-04-22T23:39:18.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.129+0000 7f05d256d700 1 -- 172.21.15.107:0/4043868330 wait complete. 2024-04-22T23:39:18.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.130+0000 7f05d256d700 1 Processor -- start 2024-04-22T23:39:18.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.130+0000 7f05d256d700 1 -- start start 2024-04-22T23:39:18.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.130+0000 7f05d256d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05cc066b20 0x7f05cc10dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:39:18.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.130+0000 7f05d256d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05cc0fff80 0x7f05cc10dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:39:18.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.130+0000 7f05d256d700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05cc10e5f0 con 0x7f05cc066b20 2024-04-22T23:39:18.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.130+0000 7f05d256d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05cc10ee40 con 0x7f05cc0fff80 2024-04-22T23:39:18.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.130+0000 7f05cbfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05cc066b20 0x7f05cc10dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:39:18.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.130+0000 7f05cb7fe700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05cc0fff80 0x7f05cc10dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:39:18.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.130+0000 7f05cbfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05cc066b20 0x7f05cc10dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52294/0 (socket says 172.21.15.107:52294) 2024-04-22T23:39:18.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.130+0000 7f05cbfff700 1 -- 172.21.15.107:0/2441957310 learned_addr learned my addr 172.21.15.107:0/2441957310 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:39:18.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.130+0000 7f05cbfff700 1 -- 172.21.15.107:0/2441957310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05cc0fff80 msgr2=0x7f05cc10dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:18.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.130+0000 7f05cbfff700 1 --2- 172.21.15.107:0/2441957310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05cc0fff80 0x7f05cc10dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:18.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.131+0000 7f05cbfff700 1 -- 172.21.15.107:0/2441957310 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05bc005040 con 0x7f05cc066b20 2024-04-22T23:39:18.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.131+0000 7f05cbfff700 1 --2- 172.21.15.107:0/2441957310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05cc066b20 0x7f05cc10dab0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f05bc00b3b0 tx=0x7f05bc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:39:18.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.131+0000 7f05c8ff9700 1 -- 172.21.15.107:0/2441957310 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05bc00e040 con 0x7f05cc066b20 2024-04-22T23:39:18.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.131+0000 7f05c8ff9700 1 -- 172.21.15.107:0/2441957310 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f05bc00bbf0 con 0x7f05cc066b20 2024-04-22T23:39:18.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.131+0000 7f05d256d700 1 -- 172.21.15.107:0/2441957310 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05cc10f0c0 con 0x7f05cc066b20 2024-04-22T23:39:18.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.131+0000 7f05c8ff9700 1 -- 172.21.15.107:0/2441957310 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05bc01c630 con 0x7f05cc066b20 2024-04-22T23:39:18.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.131+0000 7f05d256d700 1 -- 172.21.15.107:0/2441957310 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05cc10f600 con 0x7f05cc066b20 2024-04-22T23:39:18.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.132+0000 7f05d256d700 1 -- 172.21.15.107:0/2441957310 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05cc05f5f0 con 0x7f05cc066b20 2024-04-22T23:39:18.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.133+0000 7f05c8ff9700 1 -- 172.21.15.107:0/2441957310 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 36) v1 ==== 92849+0+0 (secure 0 0 0) 0x7f05bc013420 con 0x7f05cc066b20 2024-04-22T23:39:18.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.133+0000 7f05c8ff9700 1 --2- 172.21.15.107:0/2441957310 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f05b406ed20 0x7f05b40711d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:39:18.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.133+0000 7f05c8ff9700 1 -- 172.21.15.107:0/2441957310 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f05bc08f580 con 0x7f05cc066b20 2024-04-22T23:39:18.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.133+0000 7f05cb7fe700 1 --2- 172.21.15.107:0/2441957310 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f05b406ed20 0x7f05b40711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:39:18.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.133+0000 7f05cb7fe700 1 --2- 172.21.15.107:0/2441957310 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f05b406ed20 0x7f05b40711d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f05c0004f40 tx=0x7f05c000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:39:18.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:17 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:39:18.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.138+0000 7f05c8ff9700 1 -- 172.21.15.107:0/2441957310 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f05bc05bd80 con 0x7f05cc066b20 2024-04-22T23:39:18.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.397+0000 7f05d256d700 1 -- 172.21.15.107:0/2441957310 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f05cc1058d0 con 0x7f05cc066b20 2024-04-22T23:39:18.391 INFO:teuthology.orchestra.run.smithi107.stdout:e21 2024-04-22T23:39:18.392 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T23:39:18.392 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:39:18.392 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-04-22T23:39:18.392 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:39:18.392 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-04-22T23:39:18.392 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-04-22T23:39:18.392 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 21 2024-04-22T23:39:18.392 INFO:teuthology.orchestra.run.smithi107.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-22T23:39:18.392 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-04-22T23:32:20.146130+0000 2024-04-22T23:39:18.392 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-04-22T23:39:06.006385+0000 2024-04-22T23:39:18.392 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-04-22T23:39:18.392 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-04-22T23:39:18.392 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-04-22T23:39:18.392 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-04-22T23:39:18.392 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-04-22T23:39:18.393 INFO:teuthology.orchestra.run.smithi107.stdout:max_xattr_size 65536 2024-04-22T23:39:18.393 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-04-22T23:39:18.393 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-04-22T23:39:18.393 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 46 2024-04-22T23:39:18.393 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:39:18.393 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 2 2024-04-22T23:39:18.393 INFO:teuthology.orchestra.run.smithi107.stdout:in 0,1 2024-04-22T23:39:18.393 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=14468,1=24265} 2024-04-22T23:39:18.393 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-04-22T23:39:18.393 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-04-22T23:39:18.393 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-04-22T23:39:18.393 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-04-22T23:39:18.393 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-04-22T23:39:18.394 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data disabled 2024-04-22T23:39:18.394 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-04-22T23:39:18.394 INFO:teuthology.orchestra.run.smithi107.stdout:bal_rank_mask -1 2024-04-22T23:39:18.394 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-04-22T23:39:18.394 INFO:teuthology.orchestra.run.smithi107.stdout:qdb_cluster leader: 14468 members: 24265,14468 2024-04-22T23:39:18.394 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.ktuxjl{0:14468} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.107:6828/3335313253,v1:172.21.15.107:6829/3335313253] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:39:18.394 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi157.rcaris{1:24265} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.157:6824/2779251893,v1:172.21.15.157:6825/2779251893] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:39:18.394 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:39:18.394 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:39:18.394 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-04-22T23:39:18.394 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:39:18.394 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.dfvvkw{-1:14484} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/3548717247,v1:172.21.15.107:6827/3548717247] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:39:18.394 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi157.ufywse{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.157:6826/4072444199,v1:172.21.15.157:6827/4072444199] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:39:18.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.398+0000 7f05c8ff9700 1 -- 172.21.15.107:0/2441957310 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1909 (secure 0 0 0) 0x7f05bc05b910 con 0x7f05cc066b20 2024-04-22T23:39:18.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.400+0000 7f05d256d700 1 -- 172.21.15.107:0/2441957310 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f05b406ed20 msgr2=0x7f05b40711d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:18.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.400+0000 7f05d256d700 1 --2- 172.21.15.107:0/2441957310 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f05b406ed20 0x7f05b40711d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f05c0004f40 tx=0x7f05c000b040 comp rx=0 tx=0).stop 2024-04-22T23:39:18.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.400+0000 7f05d256d700 1 -- 172.21.15.107:0/2441957310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05cc066b20 msgr2=0x7f05cc10dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:18.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.400+0000 7f05d256d700 1 --2- 172.21.15.107:0/2441957310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05cc066b20 0x7f05cc10dab0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f05bc00b3b0 tx=0x7f05bc009e30 comp rx=0 tx=0).stop 2024-04-22T23:39:18.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.400+0000 7f05d256d700 1 -- 172.21.15.107:0/2441957310 shutdown_connections 2024-04-22T23:39:18.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.400+0000 7f05d256d700 1 --2- 172.21.15.107:0/2441957310 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f05b406ed20 0x7f05b40711d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:18.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.400+0000 7f05d256d700 1 --2- 172.21.15.107:0/2441957310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05cc066b20 0x7f05cc10dab0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:18.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.400+0000 7f05d256d700 1 --2- 172.21.15.107:0/2441957310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05cc0fff80 0x7f05cc10dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:18.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.400+0000 7f05d256d700 1 -- 172.21.15.107:0/2441957310 >> 172.21.15.107:0/2441957310 conn(0x7f05cc0f7450 msgr2=0x7f05cc0f7870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:39:18.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.400+0000 7f05d256d700 1 -- 172.21.15.107:0/2441957310 shutdown_connections 2024-04-22T23:39:18.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.400+0000 7f05d256d700 1 -- 172.21.15.107:0/2441957310 wait complete. 2024-04-22T23:39:18.396 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 21 2024-04-22T23:39:18.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.506+0000 7f173d8cb700 1 -- 172.21.15.107:0/2190359383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17380fe470 msgr2=0x7f17380fe880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:18.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.506+0000 7f173d8cb700 1 --2- 172.21.15.107:0/2190359383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17380fe470 0x7f17380fe880 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1728004940 tx=0x7f1728009e30 comp rx=0 tx=0).stop 2024-04-22T23:39:18.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.506+0000 7f173d8cb700 1 -- 172.21.15.107:0/2190359383 shutdown_connections 2024-04-22T23:39:18.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.506+0000 7f173d8cb700 1 --2- 172.21.15.107:0/2190359383 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17380fedc0 0x7f17380faf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:18.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.506+0000 7f173d8cb700 1 --2- 172.21.15.107:0/2190359383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17380fe470 0x7f17380fe880 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:18.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.506+0000 7f173d8cb700 1 -- 172.21.15.107:0/2190359383 >> 172.21.15.107:0/2190359383 conn(0x7f17380f6c10 msgr2=0x7f17380f9040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:39:18.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.506+0000 7f173d8cb700 1 -- 172.21.15.107:0/2190359383 shutdown_connections 2024-04-22T23:39:18.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.506+0000 7f173d8cb700 1 -- 172.21.15.107:0/2190359383 wait complete. 2024-04-22T23:39:18.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.507+0000 7f173d8cb700 1 Processor -- start 2024-04-22T23:39:18.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.507+0000 7f173d8cb700 1 -- start start 2024-04-22T23:39:18.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.507+0000 7f173d8cb700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17380fe470 0x7f173810d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:39:18.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.507+0000 7f173d8cb700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17380fedc0 0x7f173810df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:39:18.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.507+0000 7f173d8cb700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173810e510 con 0x7f17380fe470 2024-04-22T23:39:18.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.507+0000 7f173d8cb700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173810e680 con 0x7f17380fedc0 2024-04-22T23:39:18.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.507+0000 7f1736ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17380fe470 0x7f173810d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:39:18.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.507+0000 7f1736ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17380fe470 0x7f173810d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52308/0 (socket says 172.21.15.107:52308) 2024-04-22T23:39:18.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.507+0000 7f1736ffd700 1 -- 172.21.15.107:0/1580880135 learned_addr learned my addr 172.21.15.107:0/1580880135 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:39:18.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.507+0000 7f17367fc700 1 --2- 172.21.15.107:0/1580880135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17380fedc0 0x7f173810df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:39:18.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.508+0000 7f1736ffd700 1 -- 172.21.15.107:0/1580880135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17380fedc0 msgr2=0x7f173810df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:18.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.508+0000 7f1736ffd700 1 --2- 172.21.15.107:0/1580880135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17380fedc0 0x7f173810df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:18.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.508+0000 7f1736ffd700 1 -- 172.21.15.107:0/1580880135 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1728005040 con 0x7f17380fe470 2024-04-22T23:39:18.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.508+0000 7f1736ffd700 1 --2- 172.21.15.107:0/1580880135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17380fe470 0x7f173810d9d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1728009fd0 tx=0x7f1728009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:39:18.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.508+0000 7f171f7fe700 1 -- 172.21.15.107:0/1580880135 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f172800e040 con 0x7f17380fe470 2024-04-22T23:39:18.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.508+0000 7f173d8cb700 1 -- 172.21.15.107:0/1580880135 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1738113080 con 0x7f17380fe470 2024-04-22T23:39:18.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.508+0000 7f171f7fe700 1 -- 172.21.15.107:0/1580880135 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f172800bcc0 con 0x7f17380fe470 2024-04-22T23:39:18.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.508+0000 7f171f7fe700 1 -- 172.21.15.107:0/1580880135 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f172801c630 con 0x7f17380fe470 2024-04-22T23:39:18.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.508+0000 7f173d8cb700 1 -- 172.21.15.107:0/1580880135 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1738113640 con 0x7f17380fe470 2024-04-22T23:39:18.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.509+0000 7f173d8cb700 1 -- 172.21.15.107:0/1580880135 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f173804c710 con 0x7f17380fe470 2024-04-22T23:39:18.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.509+0000 7f171f7fe700 1 -- 172.21.15.107:0/1580880135 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 36) v1 ==== 92849+0+0 (secure 0 0 0) 0x7f1728013980 con 0x7f17380fe470 2024-04-22T23:39:18.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.510+0000 7f171f7fe700 1 --2- 172.21.15.107:0/1580880135 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f172006ed30 0x7f17200711e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:39:18.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.510+0000 7f171f7fe700 1 -- 172.21.15.107:0/1580880135 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f172808fe80 con 0x7f17380fe470 2024-04-22T23:39:18.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.510+0000 7f17367fc700 1 --2- 172.21.15.107:0/1580880135 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f172006ed30 0x7f17200711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:39:18.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.510+0000 7f17367fc700 1 --2- 172.21.15.107:0/1580880135 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f172006ed30 0x7f17200711e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f172c004990 tx=0x7f172c009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:39:18.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.515+0000 7f171f7fe700 1 -- 172.21.15.107:0/1580880135 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f172805c500 con 0x7f17380fe470 2024-04-22T23:39:18.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.740+0000 7f173d8cb700 1 -- 172.21.15.107:0/1580880135 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1738102600 con 0x7f172006ed30 2024-04-22T23:39:18.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.741+0000 7f171f7fe700 1 -- 172.21.15.107:0/1580880135 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f1738102600 con 0x7f172006ed30 2024-04-22T23:39:18.734 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-22T23:39:18.734 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c", 2024-04-22T23:39:18.734 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-04-22T23:39:18.734 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-22T23:39:18.734 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [ 2024-04-22T23:39:18.735 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr", 2024-04-22T23:39:18.735 INFO:teuthology.orchestra.run.smithi107.stdout: "crash", 2024-04-22T23:39:18.735 INFO:teuthology.orchestra.run.smithi107.stdout: "mon" 2024-04-22T23:39:18.735 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-22T23:39:18.735 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "6/21 daemons upgraded", 2024-04-22T23:39:18.735 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Currently upgrading osd daemons", 2024-04-22T23:39:18.735 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-04-22T23:39:18.735 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-22T23:39:18.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.743+0000 7f173d8cb700 1 -- 172.21.15.107:0/1580880135 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f172006ed30 msgr2=0x7f17200711e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:18.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.743+0000 7f173d8cb700 1 --2- 172.21.15.107:0/1580880135 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f172006ed30 0x7f17200711e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f172c004990 tx=0x7f172c009ee0 comp rx=0 tx=0).stop 2024-04-22T23:39:18.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.743+0000 7f173d8cb700 1 -- 172.21.15.107:0/1580880135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17380fe470 msgr2=0x7f173810d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:18.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.743+0000 7f173d8cb700 1 --2- 172.21.15.107:0/1580880135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17380fe470 0x7f173810d9d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1728009fd0 tx=0x7f1728009e30 comp rx=0 tx=0).stop 2024-04-22T23:39:18.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.743+0000 7f173d8cb700 1 -- 172.21.15.107:0/1580880135 shutdown_connections 2024-04-22T23:39:18.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.743+0000 7f173d8cb700 1 --2- 172.21.15.107:0/1580880135 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f172006ed30 0x7f17200711e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:18.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.743+0000 7f173d8cb700 1 --2- 172.21.15.107:0/1580880135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17380fe470 0x7f173810d9d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:18.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.743+0000 7f173d8cb700 1 --2- 172.21.15.107:0/1580880135 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17380fedc0 0x7f173810df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:18.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.743+0000 7f173d8cb700 1 -- 172.21.15.107:0/1580880135 >> 172.21.15.107:0/1580880135 conn(0x7f17380f6c10 msgr2=0x7f17380f8350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:39:18.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.743+0000 7f173d8cb700 1 -- 172.21.15.107:0/1580880135 shutdown_connections 2024-04-22T23:39:18.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.743+0000 7f173d8cb700 1 -- 172.21.15.107:0/1580880135 wait complete. 2024-04-22T23:39:18.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.848+0000 7f42ed163700 1 -- 172.21.15.107:0/3020003746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42e80fbd40 msgr2=0x7f42e80fc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:18.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.848+0000 7f42ed163700 1 --2- 172.21.15.107:0/3020003746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42e80fbd40 0x7f42e80fc150 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f42d8004940 tx=0x7f42d8009e30 comp rx=0 tx=0).stop 2024-04-22T23:39:18.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.848+0000 7f42ed163700 1 -- 172.21.15.107:0/3020003746 shutdown_connections 2024-04-22T23:39:18.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.848+0000 7f42ed163700 1 --2- 172.21.15.107:0/3020003746 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42e80fcff0 0x7f42e80fd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:18.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.848+0000 7f42ed163700 1 --2- 172.21.15.107:0/3020003746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42e80fbd40 0x7f42e80fc150 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:18.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.848+0000 7f42ed163700 1 -- 172.21.15.107:0/3020003746 >> 172.21.15.107:0/3020003746 conn(0x7f42e80f74c0 msgr2=0x7f42e80f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:39:18.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.848+0000 7f42ed163700 1 -- 172.21.15.107:0/3020003746 shutdown_connections 2024-04-22T23:39:18.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.848+0000 7f42ed163700 1 -- 172.21.15.107:0/3020003746 wait complete. 2024-04-22T23:39:18.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.849+0000 7f42ed163700 1 Processor -- start 2024-04-22T23:39:18.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.849+0000 7f42ed163700 1 -- start start 2024-04-22T23:39:18.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.849+0000 7f42ed163700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42e80fbd40 0x7f42e810fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:39:18.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.849+0000 7f42ed163700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42e80fcff0 0x7f42e8110170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:39:18.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.849+0000 7f42ed163700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e8110770 con 0x7f42e80fcff0 2024-04-22T23:39:18.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.849+0000 7f42ed163700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e8110fc0 con 0x7f42e80fbd40 2024-04-22T23:39:18.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.849+0000 7f42e659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42e80fcff0 0x7f42e8110170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:39:18.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.849+0000 7f42e659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42e80fcff0 0x7f42e8110170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52328/0 (socket says 172.21.15.107:52328) 2024-04-22T23:39:18.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.849+0000 7f42e659c700 1 -- 172.21.15.107:0/3085616474 learned_addr learned my addr 172.21.15.107:0/3085616474 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:39:18.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.850+0000 7f42e6d9d700 1 --2- 172.21.15.107:0/3085616474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42e80fbd40 0x7f42e810fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:39:18.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.850+0000 7f42e659c700 1 -- 172.21.15.107:0/3085616474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42e80fbd40 msgr2=0x7f42e810fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:18.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.850+0000 7f42e659c700 1 --2- 172.21.15.107:0/3085616474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42e80fbd40 0x7f42e810fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:18.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.850+0000 7f42e659c700 1 -- 172.21.15.107:0/3085616474 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d8005040 con 0x7f42e80fcff0 2024-04-22T23:39:18.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.850+0000 7f42e6d9d700 1 --2- 172.21.15.107:0/3085616474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42e80fbd40 0x7f42e810fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T23:39:18.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.850+0000 7f42e659c700 1 --2- 172.21.15.107:0/3085616474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42e80fcff0 0x7f42e8110170 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f42dc002a50 tx=0x7f42dc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:39:18.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.850+0000 7f42cf7fe700 1 -- 172.21.15.107:0/3085616474 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42dc007cc0 con 0x7f42e80fcff0 2024-04-22T23:39:18.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.850+0000 7f42ed163700 1 -- 172.21.15.107:0/3085616474 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42e81112a0 con 0x7f42e80fcff0 2024-04-22T23:39:18.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.850+0000 7f42cf7fe700 1 -- 172.21.15.107:0/3085616474 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f42dc00f040 con 0x7f42e80fcff0 2024-04-22T23:39:18.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.850+0000 7f42cf7fe700 1 -- 172.21.15.107:0/3085616474 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42dc009640 con 0x7f42e80fcff0 2024-04-22T23:39:18.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.850+0000 7f42ed163700 1 -- 172.21.15.107:0/3085616474 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42e81117e0 con 0x7f42e80fcff0 2024-04-22T23:39:18.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.851+0000 7f42ed163700 1 -- 172.21.15.107:0/3085616474 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42e8101980 con 0x7f42e80fcff0 2024-04-22T23:39:18.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.851+0000 7f42cf7fe700 1 -- 172.21.15.107:0/3085616474 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 36) v1 ==== 92849+0+0 (secure 0 0 0) 0x7f42dc0097a0 con 0x7f42e80fcff0 2024-04-22T23:39:18.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.852+0000 7f42cf7fe700 1 --2- 172.21.15.107:0/3085616474 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f42d006ed20 0x7f42d00711d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:39:18.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.852+0000 7f42e6d9d700 1 --2- 172.21.15.107:0/3085616474 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f42d006ed20 0x7f42d00711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:39:18.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.852+0000 7f42cf7fe700 1 -- 172.21.15.107:0/3085616474 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f42dc08e910 con 0x7f42e80fcff0 2024-04-22T23:39:18.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.852+0000 7f42e6d9d700 1 --2- 172.21.15.107:0/3085616474 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f42d006ed20 0x7f42d00711d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f42d800b7a0 tx=0x7f42d8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:39:18.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:18.857+0000 7f42cf7fe700 1 -- 172.21.15.107:0/3085616474 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f42dc05afd0 con 0x7f42e80fcff0 2024-04-22T23:39:18.985 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:18 smithi107 ceph-mon[90179]: from='client.34190 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:39:18.986 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:18 smithi107 ceph-mon[90179]: from='client.34194 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:39:18.986 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:18 smithi107 ceph-mon[90179]: from='client.44133 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:39:18.986 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:18 smithi107 ceph-mon[90179]: pgmap v46: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 67/339 objects degraded (19.764%) 2024-04-22T23:39:18.986 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:18 smithi107 ceph-mon[90179]: Health check failed: Degraded data redundancy: 67/339 objects degraded (19.764%), 19 pgs degraded (PG_DEGRADED) 2024-04-22T23:39:18.986 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:18 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/3832774806' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:39:18.986 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:18 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/2441957310' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:39:19.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:18 smithi157 ceph-mon[79403]: from='client.34190 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:39:19.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:18 smithi157 ceph-mon[79403]: from='client.34194 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:39:19.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:18 smithi157 ceph-mon[79403]: from='client.44133 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:39:19.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:18 smithi157 ceph-mon[79403]: pgmap v46: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 67/339 objects degraded (19.764%) 2024-04-22T23:39:19.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:18 smithi157 ceph-mon[79403]: Health check failed: Degraded data redundancy: 67/339 objects degraded (19.764%), 19 pgs degraded (PG_DEGRADED) 2024-04-22T23:39:19.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:18 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/3832774806' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:39:19.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:18 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/2441957310' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:39:19.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:19.149+0000 7f42ed163700 1 -- 172.21.15.107:0/3085616474 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f42e805f5f0 con 0x7f42e80fcff0 2024-04-22T23:39:19.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:19.150+0000 7f42cf7fe700 1 -- 172.21.15.107:0/3085616474 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1328 (secure 0 0 0) 0x7f42dc05ab60 con 0x7f42e80fcff0 2024-04-22T23:39:19.143 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_WARN 1 osds down; Degraded data redundancy: 67/339 objects degraded (19.764%), 19 pgs degraded 2024-04-22T23:39:19.144 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] OSD_DOWN: 1 osds down 2024-04-22T23:39:19.144 INFO:teuthology.orchestra.run.smithi107.stdout: osd.0 (root=default,host=smithi107) is down 2024-04-22T23:39:19.144 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 67/339 objects degraded (19.764%), 19 pgs degraded 2024-04-22T23:39:19.144 INFO:teuthology.orchestra.run.smithi107.stdout: pg 1.0 is active+undersized+degraded, acting [3,1] 2024-04-22T23:39:19.144 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.0 is active+undersized+degraded, acting [3,1] 2024-04-22T23:39:19.144 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.1 is active+undersized+degraded, acting [2,1] 2024-04-22T23:39:19.144 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.2 is active+undersized+degraded, acting [5,1] 2024-04-22T23:39:19.144 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.4 is active+undersized+degraded, acting [1,4] 2024-04-22T23:39:19.144 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.5 is active+undersized+degraded, acting [3,4] 2024-04-22T23:39:19.144 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.8 is active+undersized+degraded, acting [3,5] 2024-04-22T23:39:19.144 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.9 is active+undersized+degraded, acting [1,4] 2024-04-22T23:39:19.144 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.e is active+undersized+degraded, acting [2,3] 2024-04-22T23:39:19.144 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.f is active+undersized+degraded, acting [4,5] 2024-04-22T23:39:19.145 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.10 is active+undersized+degraded, acting [2,1] 2024-04-22T23:39:19.145 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.12 is active+undersized+degraded, acting [3,1] 2024-04-22T23:39:19.145 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.13 is active+undersized+degraded, acting [4,2] 2024-04-22T23:39:19.145 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.15 is active+undersized+degraded, acting [1,3] 2024-04-22T23:39:19.145 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.19 is active+undersized+degraded, acting [4,2] 2024-04-22T23:39:19.145 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.1b is active+undersized+degraded, acting [1,5] 2024-04-22T23:39:19.145 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.1d is active+undersized+degraded, acting [3,5] 2024-04-22T23:39:19.146 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.1e is active+undersized+degraded, acting [2,5] 2024-04-22T23:39:19.146 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.1f is active+undersized+degraded, acting [3,4] 2024-04-22T23:39:19.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:19.152+0000 7f42ed163700 1 -- 172.21.15.107:0/3085616474 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f42d006ed20 msgr2=0x7f42d00711d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:19.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:19.152+0000 7f42ed163700 1 --2- 172.21.15.107:0/3085616474 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f42d006ed20 0x7f42d00711d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f42d800b7a0 tx=0x7f42d8009e30 comp rx=0 tx=0).stop 2024-04-22T23:39:19.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:19.152+0000 7f42ed163700 1 -- 172.21.15.107:0/3085616474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42e80fcff0 msgr2=0x7f42e8110170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:19.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:19.152+0000 7f42ed163700 1 --2- 172.21.15.107:0/3085616474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42e80fcff0 0x7f42e8110170 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f42dc002a50 tx=0x7f42dc0046b0 comp rx=0 tx=0).stop 2024-04-22T23:39:19.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:19.152+0000 7f42ed163700 1 -- 172.21.15.107:0/3085616474 shutdown_connections 2024-04-22T23:39:19.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:19.152+0000 7f42ed163700 1 --2- 172.21.15.107:0/3085616474 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42e80fbd40 0x7f42e810fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:19.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:19.152+0000 7f42ed163700 1 --2- 172.21.15.107:0/3085616474 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f42d006ed20 0x7f42d00711d0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:19.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:19.152+0000 7f42ed163700 1 --2- 172.21.15.107:0/3085616474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42e80fcff0 0x7f42e8110170 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:19.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:19.152+0000 7f42ed163700 1 -- 172.21.15.107:0/3085616474 >> 172.21.15.107:0/3085616474 conn(0x7f42e80f74c0 msgr2=0x7f42e80f8c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:39:19.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:19.152+0000 7f42ed163700 1 -- 172.21.15.107:0/3085616474 shutdown_connections 2024-04-22T23:39:19.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:19.152+0000 7f42ed163700 1 -- 172.21.15.107:0/3085616474 wait complete. 2024-04-22T23:39:19.933 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:19 smithi107 ceph-mon[90179]: from='client.34210 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:39:19.933 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:19 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/3085616474' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:39:20.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:19 smithi157 ceph-mon[79403]: from='client.34210 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:39:20.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:19 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/3085616474' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:39:20.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:20 smithi107 ceph-mon[90179]: pgmap v47: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 67/339 objects degraded (19.764%) 2024-04-22T23:39:21.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:20 smithi157 ceph-mon[79403]: pgmap v47: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 67/339 objects degraded (19.764%) 2024-04-22T23:39:21.955 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:39:21 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-0[98610]: 2024-04-22T23:39:21.772+0000 7fe1a24907c0 -1 osd.0 49 log_to_monitors true 2024-04-22T23:39:23.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:22 smithi157 ceph-mon[79403]: pgmap v48: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 67/339 objects degraded (19.764%) 2024-04-22T23:39:23.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:22 smithi157 ceph-mon[79403]: from='osd.0 [v2:172.21.15.107:6802/149725748,v1:172.21.15.107:6803/149725748]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-22T23:39:23.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:22 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:23.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:22 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:23.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:22 smithi107 ceph-mon[90179]: pgmap v48: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 277 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 67/339 objects degraded (19.764%) 2024-04-22T23:39:23.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:22 smithi107 ceph-mon[90179]: from='osd.0 [v2:172.21.15.107:6802/149725748,v1:172.21.15.107:6803/149725748]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-22T23:39:23.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:22 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:23.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:22 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:23.775 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:39:23 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-0[98610]: 2024-04-22T23:39:23.731+0000 7fe18f8e0700 -1 osd.0 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T23:39:24.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:23 smithi157 ceph-mon[79403]: from='osd.0 [v2:172.21.15.107:6802/149725748,v1:172.21.15.107:6803/149725748]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-22T23:39:24.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:23 smithi157 ceph-mon[79403]: osdmap e52: 6 total, 5 up, 6 in 2024-04-22T23:39:24.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:23 smithi157 ceph-mon[79403]: from='osd.0 [v2:172.21.15.107:6802/149725748,v1:172.21.15.107:6803/149725748]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-04-22T23:39:24.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:23 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:24.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:23 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:24.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:23 smithi157 ceph-mon[79403]: from='osd.0 [v2:172.21.15.107:6802/149725748,v1:172.21.15.107:6803/149725748]' entity='osd.0' 2024-04-22T23:39:24.170 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:23 smithi107 ceph-mon[90179]: from='osd.0 [v2:172.21.15.107:6802/149725748,v1:172.21.15.107:6803/149725748]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-22T23:39:24.170 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:23 smithi107 ceph-mon[90179]: osdmap e52: 6 total, 5 up, 6 in 2024-04-22T23:39:24.170 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:23 smithi107 ceph-mon[90179]: from='osd.0 [v2:172.21.15.107:6802/149725748,v1:172.21.15.107:6803/149725748]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-04-22T23:39:24.170 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:23 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:24.170 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:23 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:24.170 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:23 smithi107 ceph-mon[90179]: from='osd.0 [v2:172.21.15.107:6802/149725748,v1:172.21.15.107:6803/149725748]' entity='osd.0' 2024-04-22T23:39:25.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:24 smithi157 ceph-mon[79403]: pgmap v50: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 277 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 67/339 objects degraded (19.764%) 2024-04-22T23:39:25.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:24 smithi157 ceph-mon[79403]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-22T23:39:25.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:24 smithi157 ceph-mon[79403]: osd.0 [v2:172.21.15.107:6802/149725748,v1:172.21.15.107:6803/149725748] boot 2024-04-22T23:39:25.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:24 smithi157 ceph-mon[79403]: osdmap e53: 6 total, 6 up, 6 in 2024-04-22T23:39:25.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:24 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T23:39:25.171 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:24 smithi107 ceph-mon[90179]: pgmap v50: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 277 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 67/339 objects degraded (19.764%) 2024-04-22T23:39:25.172 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:24 smithi107 ceph-mon[90179]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-22T23:39:25.172 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:24 smithi107 ceph-mon[90179]: osd.0 [v2:172.21.15.107:6802/149725748,v1:172.21.15.107:6803/149725748] boot 2024-04-22T23:39:25.172 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:24 smithi107 ceph-mon[90179]: osdmap e53: 6 total, 6 up, 6 in 2024-04-22T23:39:25.172 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:24 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T23:39:26.970 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:26 smithi107 ceph-mon[90179]: pgmap v52: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 277 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 67/339 objects degraded (19.764%) 2024-04-22T23:39:26.971 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:26 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:26.971 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:26 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:39:26.971 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:26 smithi107 ceph-mon[90179]: osdmap e54: 6 total, 6 up, 6 in 2024-04-22T23:39:27.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:26 smithi157 ceph-mon[79403]: pgmap v52: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 277 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 67/339 objects degraded (19.764%) 2024-04-22T23:39:27.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:26 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:27.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:26 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:39:27.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:26 smithi157 ceph-mon[79403]: osdmap e54: 6 total, 6 up, 6 in 2024-04-22T23:39:28.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:28 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:28.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:28 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:28.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:28 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:39:28.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:28 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:39:28.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:28 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:28.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:28 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:39:28.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:28 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:39:28.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:28 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:39:28.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:28 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:39:28.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:28 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-22T23:39:28.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:28 smithi157 ceph-mon[79403]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-22T23:39:28.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:28 smithi157 ceph-mon[79403]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-04-22T23:39:28.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:28 smithi157 ceph-mon[79403]: pgmap v54: 65 pgs: 3 active+undersized+degraded, 6 active+undersized, 56 active+clean; 277 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s; 12/339 objects degraded (3.540%) 2024-04-22T23:39:28.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:28 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:28.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:28 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:28.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:28 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:39:28.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:28 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:39:28.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:28 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:28.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:28 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:39:28.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:28 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:39:28.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:28 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:39:28.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:28 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:39:28.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:28 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-22T23:39:28.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:28 smithi107 ceph-mon[90179]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-22T23:39:28.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:28 smithi107 ceph-mon[90179]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-04-22T23:39:28.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:28 smithi107 ceph-mon[90179]: pgmap v54: 65 pgs: 3 active+undersized+degraded, 6 active+undersized, 56 active+clean; 277 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s; 12/339 objects degraded (3.540%) 2024-04-22T23:39:29.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:29 smithi157 ceph-mon[79403]: Health check update: Degraded data redundancy: 12/339 objects degraded (3.540%), 3 pgs degraded (PG_DEGRADED) 2024-04-22T23:39:29.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:29 smithi157 ceph-mon[79403]: mgrmap e37: smithi107.uipxvm(active, since 92s), standbys: smithi157.tuylis 2024-04-22T23:39:29.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:29 smithi107 ceph-mon[90179]: Health check update: Degraded data redundancy: 12/339 objects degraded (3.540%), 3 pgs degraded (PG_DEGRADED) 2024-04-22T23:39:29.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:29 smithi107 ceph-mon[90179]: mgrmap e37: smithi107.uipxvm(active, since 92s), standbys: smithi157.tuylis 2024-04-22T23:39:30.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:30 smithi157 ceph-mon[79403]: pgmap v55: 65 pgs: 65 active+clean; 277 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 892 B/s wr, 0 op/s 2024-04-22T23:39:30.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:30 smithi107 ceph-mon[90179]: pgmap v55: 65 pgs: 65 active+clean; 277 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 892 B/s wr, 0 op/s 2024-04-22T23:39:31.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:31 smithi157 ceph-mon[79403]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 12/339 objects degraded (3.540%), 3 pgs degraded) 2024-04-22T23:39:31.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:31 smithi157 ceph-mon[79403]: Cluster is now healthy 2024-04-22T23:39:31.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:31 smithi107 ceph-mon[90179]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 12/339 objects degraded (3.540%), 3 pgs degraded) 2024-04-22T23:39:31.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:31 smithi107 ceph-mon[90179]: Cluster is now healthy 2024-04-22T23:39:32.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:32 smithi157 ceph-mon[79403]: pgmap v56: 65 pgs: 65 active+clean; 277 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-04-22T23:39:32.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:32 smithi107 ceph-mon[90179]: pgmap v56: 65 pgs: 65 active+clean; 277 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-04-22T23:39:34.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:34 smithi107 ceph-mon[90179]: pgmap v57: 65 pgs: 65 active+clean; 277 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 687 B/s wr, 0 op/s 2024-04-22T23:39:35.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:34 smithi157 ceph-mon[79403]: pgmap v57: 65 pgs: 65 active+clean; 277 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 687 B/s wr, 0 op/s 2024-04-22T23:39:36.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:36 smithi107 ceph-mon[90179]: pgmap v58: 65 pgs: 65 active+clean; 277 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 614 B/s wr, 0 op/s 2024-04-22T23:39:36.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:36 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:37.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:36 smithi157 ceph-mon[79403]: pgmap v58: 65 pgs: 65 active+clean; 277 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 614 B/s wr, 0 op/s 2024-04-22T23:39:37.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:36 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:38.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:38 smithi107 ceph-mon[90179]: pgmap v59: 65 pgs: 65 active+clean; 277 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 514 B/s wr, 0 op/s 2024-04-22T23:39:39.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:38 smithi157 ceph-mon[79403]: pgmap v59: 65 pgs: 65 active+clean; 277 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 514 B/s wr, 0 op/s 2024-04-22T23:39:40.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:40 smithi107 ceph-mon[90179]: pgmap v60: 65 pgs: 65 active+clean; 277 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:39:40.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:40 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:40.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:40 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:39:41.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:40 smithi157 ceph-mon[79403]: pgmap v60: 65 pgs: 65 active+clean; 277 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:39:41.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:40 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:41.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:40 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:39:42.980 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:42 smithi107 ceph-mon[90179]: pgmap v61: 65 pgs: 65 active+clean; 277 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:39:42.980 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:42 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-22T23:39:43.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:42 smithi157 ceph-mon[79403]: pgmap v61: 65 pgs: 65 active+clean; 277 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:39:43.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:42 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-22T23:39:43.790 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:43 smithi107 ceph-mon[90179]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-22T23:39:43.790 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:43 smithi107 ceph-mon[90179]: Upgrade: osd.1 is safe to restart 2024-04-22T23:39:43.790 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:43 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:43.790 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:43 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-22T23:39:43.791 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:43 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:39:44.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:43 smithi157 ceph-mon[79403]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-22T23:39:44.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:43 smithi157 ceph-mon[79403]: Upgrade: osd.1 is safe to restart 2024-04-22T23:39:44.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:43 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:44.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:43 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-22T23:39:44.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:43 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:39:44.973 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:44 smithi107 ceph-mon[90179]: pgmap v62: 65 pgs: 65 active+clean; 277 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:39:44.973 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:44 smithi107 ceph-mon[90179]: Upgrade: Updating osd.1 2024-04-22T23:39:44.973 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:44 smithi107 ceph-mon[90179]: Deploying daemon osd.1 on smithi107 2024-04-22T23:39:45.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:44 smithi157 ceph-mon[79403]: pgmap v62: 65 pgs: 65 active+clean; 277 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:39:45.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:44 smithi157 ceph-mon[79403]: Upgrade: Updating osd.1 2024-04-22T23:39:45.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:44 smithi157 ceph-mon[79403]: Deploying daemon osd.1 on smithi107 2024-04-22T23:39:46.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:46 smithi107 ceph-mon[90179]: pgmap v63: 65 pgs: 65 active+clean; 277 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:39:47.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:46 smithi157 ceph-mon[79403]: pgmap v63: 65 pgs: 65 active+clean; 277 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:39:47.296 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:39:47 smithi107 systemd[1]: Stopping Ceph osd.1 for 8252f6f2-00ff-11ef-bc93-c7b262605968... 2024-04-22T23:39:47.732 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:39:47 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-1[52832]: 2024-04-22T23:39:47.302+0000 7fc5113f1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T23:39:47.733 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:39:47 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-1[52832]: 2024-04-22T23:39:47.302+0000 7fc5113f1700 -1 osd.1 54 *** Got signal Terminated *** 2024-04-22T23:39:47.733 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:39:47 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-1[52832]: 2024-04-22T23:39:47.302+0000 7fc5113f1700 -1 osd.1 54 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T23:39:48.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:47 smithi157 ceph-mon[79403]: osd.1 marked itself down and dead 2024-04-22T23:39:48.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:47 smithi107 ceph-mon[90179]: osd.1 marked itself down and dead 2024-04-22T23:39:48.594 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:39:48 smithi107 podman[103677]: 2024-04-22 23:39:48.334212619 +0000 UTC m=+1.160943845 container died c7272eb84ddb002e3a9facf8f43a6242848352baed9c701c40d34dce3e7e7780 (image=quay.ceph.io/ceph-ci/ceph@sha256:3890cd726bd5e73ae623dabdf4ed76dd5df81c936b8bd8d761026babda969466, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-1, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=quincy-70d0cb4, org.label-schema.build-date=20240416, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-22T23:39:48.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:48 smithi107 ceph-mon[90179]: pgmap v64: 65 pgs: 65 active+clean; 277 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:39:48.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:48 smithi107 ceph-mon[90179]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-22T23:39:48.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:48 smithi107 ceph-mon[90179]: osdmap e55: 6 total, 5 up, 6 in 2024-04-22T23:39:48.983 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:39:48 smithi107 podman[103677]: 2024-04-22 23:39:48.716309156 +0000 UTC m=+1.543040379 container cleanup c7272eb84ddb002e3a9facf8f43a6242848352baed9c701c40d34dce3e7e7780 (image=quay.ceph.io/ceph-ci/ceph@sha256:3890cd726bd5e73ae623dabdf4ed76dd5df81c936b8bd8d761026babda969466, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=quincy-70d0cb4, GIT_BRANCH=HEAD) 2024-04-22T23:39:48.983 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:39:48 smithi107 bash[103677]: ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-1 2024-04-22T23:39:49.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:48 smithi157 ceph-mon[79403]: pgmap v64: 65 pgs: 65 active+clean; 277 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:39:49.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:48 smithi157 ceph-mon[79403]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-22T23:39:49.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:48 smithi157 ceph-mon[79403]: osdmap e55: 6 total, 5 up, 6 in 2024-04-22T23:39:49.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.276+0000 7f78622c9700 1 -- 172.21.15.107:0/78609886 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f785c066b80 msgr2=0x7f785c0ff9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:49.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.276+0000 7f78622c9700 1 --2- 172.21.15.107:0/78609886 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f785c066b80 0x7f785c0ff9e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7850004880 tx=0x7f7850009e30 comp rx=0 tx=0).stop 2024-04-22T23:39:49.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.276+0000 7f78622c9700 1 -- 172.21.15.107:0/78609886 shutdown_connections 2024-04-22T23:39:49.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.276+0000 7f78622c9700 1 --2- 172.21.15.107:0/78609886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f785c0fff20 0x7f785c102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:49.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.276+0000 7f78622c9700 1 --2- 172.21.15.107:0/78609886 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f785c066b80 0x7f785c0ff9e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:49.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.276+0000 7f78622c9700 1 -- 172.21.15.107:0/78609886 >> 172.21.15.107:0/78609886 conn(0x7f785c0f7450 msgr2=0x7f785c0f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:39:49.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.276+0000 7f78622c9700 1 -- 172.21.15.107:0/78609886 shutdown_connections 2024-04-22T23:39:49.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.276+0000 7f78622c9700 1 -- 172.21.15.107:0/78609886 wait complete. 2024-04-22T23:39:49.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.276+0000 7f78622c9700 1 Processor -- start 2024-04-22T23:39:49.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.276+0000 7f78622c9700 1 -- start start 2024-04-22T23:39:49.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.277+0000 7f78622c9700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f785c066b80 0x7f785c0fd410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:39:49.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.277+0000 7f78622c9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f785c0fff20 0x7f785c0fb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:39:49.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.277+0000 7f78622c9700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785c0fbe90 con 0x7f785c0fff20 2024-04-22T23:39:49.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.277+0000 7f78622c9700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785c0fc000 con 0x7f785c066b80 2024-04-22T23:39:49.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.277+0000 7f785b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f785c0fff20 0x7f785c0fb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:39:49.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.277+0000 7f785b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f785c0fff20 0x7f785c0fb8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43770/0 (socket says 172.21.15.107:43770) 2024-04-22T23:39:49.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.277+0000 7f785b7fe700 1 -- 172.21.15.107:0/989384846 learned_addr learned my addr 172.21.15.107:0/989384846 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:39:49.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.277+0000 7f785bfff700 1 --2- 172.21.15.107:0/989384846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f785c066b80 0x7f785c0fd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:39:49.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.277+0000 7f785b7fe700 1 -- 172.21.15.107:0/989384846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f785c066b80 msgr2=0x7f785c0fd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:49.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.277+0000 7f785b7fe700 1 --2- 172.21.15.107:0/989384846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f785c066b80 0x7f785c0fd410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:49.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.277+0000 7f785b7fe700 1 -- 172.21.15.107:0/989384846 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7850005040 con 0x7f785c0fff20 2024-04-22T23:39:49.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.277+0000 7f785b7fe700 1 --2- 172.21.15.107:0/989384846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f785c0fff20 0x7f785c0fb8e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f784c0041d0 tx=0x7f784c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:39:49.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.278+0000 7f7858ff9700 1 -- 172.21.15.107:0/989384846 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f784c007cc0 con 0x7f785c0fff20 2024-04-22T23:39:49.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.278+0000 7f7858ff9700 1 -- 172.21.15.107:0/989384846 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f784c00f040 con 0x7f785c0fff20 2024-04-22T23:39:49.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.278+0000 7f7858ff9700 1 -- 172.21.15.107:0/989384846 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f784c00ca50 con 0x7f785c0fff20 2024-04-22T23:39:49.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.278+0000 7f78622c9700 1 -- 172.21.15.107:0/989384846 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f785c0fc280 con 0x7f785c0fff20 2024-04-22T23:39:49.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.278+0000 7f78622c9700 1 -- 172.21.15.107:0/989384846 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f785c115aa0 con 0x7f785c0fff20 2024-04-22T23:39:49.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.279+0000 7f78622c9700 1 -- 172.21.15.107:0/989384846 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f785c05f5f0 con 0x7f785c0fff20 2024-04-22T23:39:49.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.279+0000 7f7858ff9700 1 -- 172.21.15.107:0/989384846 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f784c00cbb0 con 0x7f785c0fff20 2024-04-22T23:39:49.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.280+0000 7f7858ff9700 1 --2- 172.21.15.107:0/989384846 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f784406ec10 0x7f78440710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:39:49.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.280+0000 7f7858ff9700 1 -- 172.21.15.107:0/989384846 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f784c08e200 con 0x7f785c0fff20 2024-04-22T23:39:49.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.280+0000 7f785bfff700 1 --2- 172.21.15.107:0/989384846 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f784406ec10 0x7f78440710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:39:49.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.280+0000 7f785bfff700 1 --2- 172.21.15.107:0/989384846 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f784406ec10 0x7f78440710c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7850000c00 tx=0x7f7850009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:39:49.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.285+0000 7f7858ff9700 1 -- 172.21.15.107:0/989384846 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f784c0570e0 con 0x7f785c0fff20 2024-04-22T23:39:49.420 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:39:49 smithi107 podman[103694]: 2024-04-22 23:39:49.059741806 +0000 UTC m=+0.725891377 container remove c7272eb84ddb002e3a9facf8f43a6242848352baed9c701c40d34dce3e7e7780 (image=quay.ceph.io/ceph-ci/ceph@sha256:3890cd726bd5e73ae623dabdf4ed76dd5df81c936b8bd8d761026babda969466, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-1, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=quincy-70d0cb4, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-22T23:39:49.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.515+0000 7f78622c9700 1 -- 172.21.15.107:0/989384846 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f785c0f8f90 con 0x7f784406ec10 2024-04-22T23:39:49.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.517+0000 7f7858ff9700 1 -- 172.21.15.107:0/989384846 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f785c0f8f90 con 0x7f784406ec10 2024-04-22T23:39:49.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.519+0000 7f78622c9700 1 -- 172.21.15.107:0/989384846 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f784406ec10 msgr2=0x7f78440710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:49.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.519+0000 7f78622c9700 1 --2- 172.21.15.107:0/989384846 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f784406ec10 0x7f78440710c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7850000c00 tx=0x7f7850009e30 comp rx=0 tx=0).stop 2024-04-22T23:39:49.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.519+0000 7f78622c9700 1 -- 172.21.15.107:0/989384846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f785c0fff20 msgr2=0x7f785c0fb8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:49.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.519+0000 7f78622c9700 1 --2- 172.21.15.107:0/989384846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f785c0fff20 0x7f785c0fb8e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f784c0041d0 tx=0x7f784c00b040 comp rx=0 tx=0).stop 2024-04-22T23:39:49.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.519+0000 7f78622c9700 1 -- 172.21.15.107:0/989384846 shutdown_connections 2024-04-22T23:39:49.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.519+0000 7f78622c9700 1 --2- 172.21.15.107:0/989384846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f785c066b80 0x7f785c0fd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:49.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.519+0000 7f78622c9700 1 --2- 172.21.15.107:0/989384846 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f784406ec10 0x7f78440710c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:49.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.519+0000 7f78622c9700 1 --2- 172.21.15.107:0/989384846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f785c0fff20 0x7f785c0fb8e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:49.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.519+0000 7f78622c9700 1 -- 172.21.15.107:0/989384846 >> 172.21.15.107:0/989384846 conn(0x7f785c0f7450 msgr2=0x7f785c0f7870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:39:49.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.519+0000 7f78622c9700 1 -- 172.21.15.107:0/989384846 shutdown_connections 2024-04-22T23:39:49.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.519+0000 7f78622c9700 1 -- 172.21.15.107:0/989384846 wait complete. 2024-04-22T23:39:49.523 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-04-22T23:39:49.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.627+0000 7f59db3a3700 1 -- 172.21.15.107:0/3876571284 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59d40fcff0 msgr2=0x7f59d40fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:49.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.627+0000 7f59db3a3700 1 --2- 172.21.15.107:0/3876571284 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59d40fcff0 0x7f59d40fd460 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f59c4004940 tx=0x7f59c4009e30 comp rx=0 tx=0).stop 2024-04-22T23:39:49.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.628+0000 7f59db3a3700 1 -- 172.21.15.107:0/3876571284 shutdown_connections 2024-04-22T23:39:49.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.628+0000 7f59db3a3700 1 --2- 172.21.15.107:0/3876571284 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59d40fcff0 0x7f59d40fd460 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:49.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.628+0000 7f59db3a3700 1 --2- 172.21.15.107:0/3876571284 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59d40fbd40 0x7f59d40fc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:49.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.628+0000 7f59db3a3700 1 -- 172.21.15.107:0/3876571284 >> 172.21.15.107:0/3876571284 conn(0x7f59d40f74c0 msgr2=0x7f59d40f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:39:49.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.628+0000 7f59db3a3700 1 -- 172.21.15.107:0/3876571284 shutdown_connections 2024-04-22T23:39:49.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.628+0000 7f59db3a3700 1 -- 172.21.15.107:0/3876571284 wait complete. 2024-04-22T23:39:49.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.628+0000 7f59db3a3700 1 Processor -- start 2024-04-22T23:39:49.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.628+0000 7f59db3a3700 1 -- start start 2024-04-22T23:39:49.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.628+0000 7f59db3a3700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59d40fbd40 0x7f59d410c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:39:49.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.628+0000 7f59db3a3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59d40fcff0 0x7f59d410ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:39:49.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.628+0000 7f59db3a3700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d410d460 con 0x7f59d40fcff0 2024-04-22T23:39:49.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.628+0000 7f59db3a3700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d410dcb0 con 0x7f59d40fbd40 2024-04-22T23:39:49.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.628+0000 7f59d893e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59d40fcff0 0x7f59d410ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:39:49.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.628+0000 7f59d893e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59d40fcff0 0x7f59d410ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43794/0 (socket says 172.21.15.107:43794) 2024-04-22T23:39:49.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.628+0000 7f59d893e700 1 -- 172.21.15.107:0/30045626 learned_addr learned my addr 172.21.15.107:0/30045626 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:39:49.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.628+0000 7f59d913f700 1 --2- 172.21.15.107:0/30045626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59d40fbd40 0x7f59d410c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:39:49.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.629+0000 7f59d893e700 1 -- 172.21.15.107:0/30045626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59d40fbd40 msgr2=0x7f59d410c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:49.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.629+0000 7f59d893e700 1 --2- 172.21.15.107:0/30045626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59d40fbd40 0x7f59d410c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:49.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.629+0000 7f59d893e700 1 -- 172.21.15.107:0/30045626 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59c4005040 con 0x7f59d40fcff0 2024-04-22T23:39:49.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.629+0000 7f59d893e700 1 --2- 172.21.15.107:0/30045626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59d40fcff0 0x7f59d410ce60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f59c400b3b0 tx=0x7f59c4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:39:49.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.629+0000 7f59c9ffb700 1 -- 172.21.15.107:0/30045626 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59c400e040 con 0x7f59d40fcff0 2024-04-22T23:39:49.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.629+0000 7f59db3a3700 1 -- 172.21.15.107:0/30045626 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59d410df30 con 0x7f59d40fcff0 2024-04-22T23:39:49.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.629+0000 7f59c9ffb700 1 -- 172.21.15.107:0/30045626 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f59c400bbf0 con 0x7f59d40fcff0 2024-04-22T23:39:49.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.629+0000 7f59c9ffb700 1 -- 172.21.15.107:0/30045626 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59c401c630 con 0x7f59d40fcff0 2024-04-22T23:39:49.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.629+0000 7f59db3a3700 1 -- 172.21.15.107:0/30045626 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59d410e470 con 0x7f59d40fcff0 2024-04-22T23:39:49.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.630+0000 7f59db3a3700 1 -- 172.21.15.107:0/30045626 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59d405f5f0 con 0x7f59d40fcff0 2024-04-22T23:39:49.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.631+0000 7f59c9ffb700 1 -- 172.21.15.107:0/30045626 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f59c400bd60 con 0x7f59d40fcff0 2024-04-22T23:39:49.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.631+0000 7f59c9ffb700 1 --2- 172.21.15.107:0/30045626 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f59c0070e10 0x7f59c00732c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:39:49.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.632+0000 7f59d913f700 1 --2- 172.21.15.107:0/30045626 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f59c0070e10 0x7f59c00732c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:39:49.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.632+0000 7f59c9ffb700 1 -- 172.21.15.107:0/30045626 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f59c408f450 con 0x7f59d40fcff0 2024-04-22T23:39:49.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.632+0000 7f59d913f700 1 --2- 172.21.15.107:0/30045626 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f59c0070e10 0x7f59c00732c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f59d00041d0 tx=0x7f59d00043c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:39:49.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.636+0000 7f59c9ffb700 1 -- 172.21.15.107:0/30045626 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f59c405bdb0 con 0x7f59d40fcff0 2024-04-22T23:39:49.732 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:39:49 smithi107 podman[103810]: 2024-04-22 23:39:49.427033642 +0000 UTC m=+0.113071183 container create 875b22f0a7d75b3126f911f51eb0af320b8f0046da2f374261e0022d631ca9cd (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-1-deactivate, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T23:39:49.733 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:39:49 smithi107 podman[103810]: 2024-04-22 23:39:49.332401202 +0000 UTC m=+0.018438769 image pull a0434699c1d76af4eea1fc95ff0a22ea73d1e04e609464fbf658d6f39bc7f2fc quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c 2024-04-22T23:39:49.733 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:39:49 smithi107 podman[103810]: 2024-04-22 23:39:49.671185107 +0000 UTC m=+0.357222646 container init 875b22f0a7d75b3126f911f51eb0af320b8f0046da2f374261e0022d631ca9cd (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-1-deactivate, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-22T23:39:49.733 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:39:49 smithi107 podman[103810]: 2024-04-22 23:39:49.675855593 +0000 UTC m=+0.361893141 container start 875b22f0a7d75b3126f911f51eb0af320b8f0046da2f374261e0022d631ca9cd (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T23:39:49.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.872+0000 7f59db3a3700 1 -- 172.21.15.107:0/30045626 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f59d410e810 con 0x7f59c0070e10 2024-04-22T23:39:49.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.874+0000 7f59c9ffb700 1 -- 172.21.15.107:0/30045626 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f59d410e810 con 0x7f59c0070e10 2024-04-22T23:39:49.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.876+0000 7f59db3a3700 1 -- 172.21.15.107:0/30045626 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f59c0070e10 msgr2=0x7f59c00732c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:49.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.876+0000 7f59db3a3700 1 --2- 172.21.15.107:0/30045626 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f59c0070e10 0x7f59c00732c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f59d00041d0 tx=0x7f59d00043c0 comp rx=0 tx=0).stop 2024-04-22T23:39:49.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.876+0000 7f59db3a3700 1 -- 172.21.15.107:0/30045626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59d40fcff0 msgr2=0x7f59d410ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:49.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.876+0000 7f59db3a3700 1 --2- 172.21.15.107:0/30045626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59d40fcff0 0x7f59d410ce60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f59c400b3b0 tx=0x7f59c4009e30 comp rx=0 tx=0).stop 2024-04-22T23:39:49.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.876+0000 7f59db3a3700 1 -- 172.21.15.107:0/30045626 shutdown_connections 2024-04-22T23:39:49.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.876+0000 7f59db3a3700 1 --2- 172.21.15.107:0/30045626 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59d40fbd40 0x7f59d410c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:49.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.876+0000 7f59db3a3700 1 --2- 172.21.15.107:0/30045626 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f59c0070e10 0x7f59c00732c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:49.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.876+0000 7f59db3a3700 1 --2- 172.21.15.107:0/30045626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59d40fcff0 0x7f59d410ce60 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:49.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.876+0000 7f59db3a3700 1 -- 172.21.15.107:0/30045626 >> 172.21.15.107:0/30045626 conn(0x7f59d40f74c0 msgr2=0x7f59d40f8bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:39:49.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.876+0000 7f59db3a3700 1 -- 172.21.15.107:0/30045626 shutdown_connections 2024-04-22T23:39:49.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.876+0000 7f59db3a3700 1 -- 172.21.15.107:0/30045626 wait complete. 2024-04-22T23:39:49.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.986+0000 7f276ecb7700 1 -- 172.21.15.107:0/1095676791 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27680fbe20 msgr2=0x7f27680fc230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:49.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.986+0000 7f276ecb7700 1 --2- 172.21.15.107:0/1095676791 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27680fbe20 0x7f27680fc230 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2760004940 tx=0x7f2760009e30 comp rx=0 tx=0).stop 2024-04-22T23:39:49.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.986+0000 7f276ecb7700 1 -- 172.21.15.107:0/1095676791 shutdown_connections 2024-04-22T23:39:49.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.986+0000 7f276ecb7700 1 --2- 172.21.15.107:0/1095676791 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27680fcf90 0x7f27680fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:49.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.986+0000 7f276ecb7700 1 --2- 172.21.15.107:0/1095676791 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27680fbe20 0x7f27680fc230 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:49.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.986+0000 7f276ecb7700 1 -- 172.21.15.107:0/1095676791 >> 172.21.15.107:0/1095676791 conn(0x7f27680f74e0 msgr2=0x7f27680f9910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:39:49.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.986+0000 7f276ecb7700 1 -- 172.21.15.107:0/1095676791 shutdown_connections 2024-04-22T23:39:49.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.986+0000 7f276ecb7700 1 -- 172.21.15.107:0/1095676791 wait complete. 2024-04-22T23:39:49.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.987+0000 7f276ecb7700 1 Processor -- start 2024-04-22T23:39:49.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.987+0000 7f276ecb7700 1 -- start start 2024-04-22T23:39:49.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.987+0000 7f276ecb7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27680fbe20 0x7f276810dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:39:49.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.987+0000 7f276ecb7700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27680fcf90 0x7f276810e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:39:49.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.987+0000 7f276ecb7700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f276810e600 con 0x7f27680fbe20 2024-04-22T23:39:49.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.987+0000 7f276ecb7700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f276810ee00 con 0x7f27680fcf90 2024-04-22T23:39:49.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.987+0000 7f276ca53700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27680fbe20 0x7f276810dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:39:49.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.987+0000 7f276ca53700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27680fbe20 0x7f276810dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43822/0 (socket says 172.21.15.107:43822) 2024-04-22T23:39:49.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.987+0000 7f276ca53700 1 -- 172.21.15.107:0/2528148973 learned_addr learned my addr 172.21.15.107:0/2528148973 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:39:49.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.987+0000 7f2767fff700 1 --2- 172.21.15.107:0/2528148973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27680fcf90 0x7f276810e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:39:49.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.988+0000 7f276ca53700 1 -- 172.21.15.107:0/2528148973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27680fcf90 msgr2=0x7f276810e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:49.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.988+0000 7f276ca53700 1 --2- 172.21.15.107:0/2528148973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27680fcf90 0x7f276810e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:49.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.988+0000 7f276ca53700 1 -- 172.21.15.107:0/2528148973 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2760005040 con 0x7f27680fbe20 2024-04-22T23:39:49.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.988+0000 7f276ca53700 1 --2- 172.21.15.107:0/2528148973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27680fbe20 0x7f276810dac0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2760009150 tx=0x7f2760009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:39:49.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.988+0000 7f27657fa700 1 -- 172.21.15.107:0/2528148973 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f276000e040 con 0x7f27680fbe20 2024-04-22T23:39:49.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.988+0000 7f27657fa700 1 -- 172.21.15.107:0/2528148973 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f276000bbf0 con 0x7f27680fbe20 2024-04-22T23:39:49.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.988+0000 7f276ecb7700 1 -- 172.21.15.107:0/2528148973 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f276810f080 con 0x7f27680fbe20 2024-04-22T23:39:49.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.988+0000 7f27657fa700 1 -- 172.21.15.107:0/2528148973 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f276001c630 con 0x7f27680fbe20 2024-04-22T23:39:49.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.988+0000 7f276ecb7700 1 -- 172.21.15.107:0/2528148973 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f276810f5c0 con 0x7f27680fbe20 2024-04-22T23:39:49.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.989+0000 7f276ecb7700 1 -- 172.21.15.107:0/2528148973 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2768101b20 con 0x7f27680fbe20 2024-04-22T23:39:49.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.990+0000 7f27657fa700 1 -- 172.21.15.107:0/2528148973 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f276001c790 con 0x7f27680fbe20 2024-04-22T23:39:49.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.990+0000 7f27657fa700 1 --2- 172.21.15.107:0/2528148973 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f275406ec10 0x7f27540710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:39:49.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.990+0000 7f27657fa700 1 -- 172.21.15.107:0/2528148973 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f276008fa50 con 0x7f27680fbe20 2024-04-22T23:39:49.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.990+0000 7f2767fff700 1 --2- 172.21.15.107:0/2528148973 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f275406ec10 0x7f27540710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:39:49.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.991+0000 7f2767fff700 1 --2- 172.21.15.107:0/2528148973 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f275406ec10 0x7f27540710c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2758004550 tx=0x7f2758007ed0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:39:49.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:49.995+0000 7f27657fa700 1 -- 172.21.15.107:0/2528148973 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f276005c470 con 0x7f27680fbe20 2024-04-22T23:39:50.003 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:49 smithi107 ceph-mon[90179]: osdmap e56: 6 total, 5 up, 6 in 2024-04-22T23:39:50.004 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:39:49 smithi107 podman[103810]: 2024-04-22 23:39:49.746079157 +0000 UTC m=+0.432116705 container attach 875b22f0a7d75b3126f911f51eb0af320b8f0046da2f374261e0022d631ca9cd (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2) 2024-04-22T23:39:50.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:49 smithi157 ceph-mon[79403]: osdmap e56: 6 total, 5 up, 6 in 2024-04-22T23:39:50.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.214+0000 7f276ecb7700 1 -- 172.21.15.107:0/2528148973 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f276810f830 con 0x7f275406ec10 2024-04-22T23:39:50.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.224+0000 7f27657fa700 1 -- 172.21.15.107:0/2528148973 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f276810f830 con 0x7f275406ec10 2024-04-22T23:39:50.218 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-22T23:39:50.218 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (3m) 27s ago 12m 23.5M - 0.25.0 c8568f914cd2 ec1a5c6260de 2024-04-22T23:39:50.218 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (59s) 27s ago 13m 6656k - 19.0.0-3252-g43be0201 a0434699c1d7 e074fd2cbb9c 2024-04-22T23:39:50.218 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi157 smithi157 running (54s) 50s ago 12m 6651k - 19.0.0-3252-g43be0201 a0434699c1d7 abc2649dee22 2024-04-22T23:39:50.218 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (3m) 27s ago 12m 80.2M - 9.4.7 954c08fa6188 31449ff442d2 2024-04-22T23:39:50.218 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.dfvvkw smithi107 running (7m) 27s ago 7m 13.4M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 08d3aa8d717d 2024-04-22T23:39:50.218 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.ktuxjl smithi107 running (7m) 27s ago 7m 124M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 762f27fe0f30 2024-04-22T23:39:50.218 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi157.rcaris smithi157 running (7m) 50s ago 7m 129M - 17.2.7-884-g70d0cb4b cafd8ddac8fa d23366ac085d 2024-04-22T23:39:50.219 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi157.ufywse smithi157 running (7m) 50s ago 7m 14.1M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 8b00d61eb355 2024-04-22T23:39:50.219 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.uipxvm smithi107 *:8443,9283,8765 running (2m) 27s ago 14m 499M - 19.0.0-3252-g43be0201 a0434699c1d7 d20f05dfe4f2 2024-04-22T23:39:50.219 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi157.tuylis smithi157 *:8443,9283,8765 running (2m) 50s ago 11m 449M - 19.0.0-3252-g43be0201 a0434699c1d7 957e598c9001 2024-04-22T23:39:50.219 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (2m) 27s ago 14m 46.0M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 bc72bbbe2c39 2024-04-22T23:39:50.219 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi157 smithi157 running (100s) 50s ago 11m 31.7M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 ea88de8be4fe 2024-04-22T23:39:50.219 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (3m) 27s ago 12m 17.2M - 1.5.0 0da6a335fe13 5205cfd9bd51 2024-04-22T23:39:50.219 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi157 smithi157 *:9100 running (2m) 50s ago 11m 17.0M - 1.5.0 0da6a335fe13 168ca191bb8c 2024-04-22T23:39:50.219 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (33s) 27s ago 11m 12.4M 1557M 19.0.0-3252-g43be0201 a0434699c1d7 fd0ba8ef0c27 2024-04-22T23:39:50.219 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (10m) 27s ago 10m 241M 1557M 17.2.7-884-g70d0cb4b cafd8ddac8fa c7272eb84ddb 2024-04-22T23:39:50.219 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (10m) 27s ago 10m 195M 1557M 17.2.7-884-g70d0cb4b cafd8ddac8fa 9db1788096ad 2024-04-22T23:39:50.219 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi157 running (9m) 50s ago 9m 266M 2581M 17.2.7-884-g70d0cb4b cafd8ddac8fa c9fb2d1a13dd 2024-04-22T23:39:50.219 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi157 running (9m) 50s ago 9m 227M 2581M 17.2.7-884-g70d0cb4b cafd8ddac8fa 24a16e19951f 2024-04-22T23:39:50.219 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi157 running (8m) 50s ago 8m 226M 2581M 17.2.7-884-g70d0cb4b cafd8ddac8fa 4c5b7a7ef8bd 2024-04-22T23:39:50.219 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (2m) 27s ago 12m 56.5M - 2.43.0 a07b618ecd1d 99a4ea8594da 2024-04-22T23:39:50.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.228+0000 7f276ecb7700 1 -- 172.21.15.107:0/2528148973 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f275406ec10 msgr2=0x7f27540710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:50.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.228+0000 7f276ecb7700 1 --2- 172.21.15.107:0/2528148973 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f275406ec10 0x7f27540710c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2758004550 tx=0x7f2758007ed0 comp rx=0 tx=0).stop 2024-04-22T23:39:50.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.228+0000 7f276ecb7700 1 -- 172.21.15.107:0/2528148973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27680fbe20 msgr2=0x7f276810dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:50.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.228+0000 7f276ecb7700 1 --2- 172.21.15.107:0/2528148973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27680fbe20 0x7f276810dac0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2760009150 tx=0x7f2760009e30 comp rx=0 tx=0).stop 2024-04-22T23:39:50.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.228+0000 7f276ecb7700 1 -- 172.21.15.107:0/2528148973 shutdown_connections 2024-04-22T23:39:50.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.228+0000 7f276ecb7700 1 --2- 172.21.15.107:0/2528148973 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f275406ec10 0x7f27540710c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:50.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.228+0000 7f276ecb7700 1 --2- 172.21.15.107:0/2528148973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27680fbe20 0x7f276810dac0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:50.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.228+0000 7f276ecb7700 1 --2- 172.21.15.107:0/2528148973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27680fcf90 0x7f276810e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:50.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.228+0000 7f276ecb7700 1 -- 172.21.15.107:0/2528148973 >> 172.21.15.107:0/2528148973 conn(0x7f27680f74e0 msgr2=0x7f2768066450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:39:50.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.228+0000 7f276ecb7700 1 -- 172.21.15.107:0/2528148973 shutdown_connections 2024-04-22T23:39:50.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.228+0000 7f276ecb7700 1 -- 172.21.15.107:0/2528148973 wait complete. 2024-04-22T23:39:50.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.332+0000 7f03ebf69700 1 -- 172.21.15.107:0/1602470611 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03e40fce40 msgr2=0x7f03e40fd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:50.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.332+0000 7f03ebf69700 1 --2- 172.21.15.107:0/1602470611 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03e40fce40 0x7f03e40fd250 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f03e0004940 tx=0x7f03e0009e30 comp rx=0 tx=0).stop 2024-04-22T23:39:50.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.332+0000 7f03ebf69700 1 -- 172.21.15.107:0/1602470611 shutdown_connections 2024-04-22T23:39:50.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.332+0000 7f03ebf69700 1 --2- 172.21.15.107:0/1602470611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03e40fe040 0x7f03e40fe4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:50.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.332+0000 7f03ebf69700 1 --2- 172.21.15.107:0/1602470611 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03e40fce40 0x7f03e40fd250 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:50.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.332+0000 7f03ebf69700 1 -- 172.21.15.107:0/1602470611 >> 172.21.15.107:0/1602470611 conn(0x7f03e40f8570 msgr2=0x7f03e40fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:39:50.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.333+0000 7f03ebf69700 1 -- 172.21.15.107:0/1602470611 shutdown_connections 2024-04-22T23:39:50.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.333+0000 7f03ebf69700 1 -- 172.21.15.107:0/1602470611 wait complete. 2024-04-22T23:39:50.327 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:39:50 smithi107 podman[103810]: 2024-04-22 23:39:50.011104937 +0000 UTC m=+0.697142495 container died 875b22f0a7d75b3126f911f51eb0af320b8f0046da2f374261e0022d631ca9cd (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, ceph=True, GIT_CLEAN=True, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-22T23:39:50.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.333+0000 7f03ebf69700 1 Processor -- start 2024-04-22T23:39:50.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.333+0000 7f03ebf69700 1 -- start start 2024-04-22T23:39:50.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.333+0000 7f03ebf69700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03e40fce40 0x7f03e4071e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:39:50.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.333+0000 7f03ebf69700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03e40fe040 0x7f03e4070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:39:50.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.333+0000 7f03ebf69700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e40708c0 con 0x7f03e40fce40 2024-04-22T23:39:50.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.333+0000 7f03ebf69700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e4070a30 con 0x7f03e40fe040 2024-04-22T23:39:50.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.333+0000 7f03e9d05700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03e40fce40 0x7f03e4071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:39:50.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.333+0000 7f03e9d05700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03e40fce40 0x7f03e4071e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43846/0 (socket says 172.21.15.107:43846) 2024-04-22T23:39:50.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.333+0000 7f03e9d05700 1 -- 172.21.15.107:0/1667893722 learned_addr learned my addr 172.21.15.107:0/1667893722 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:39:50.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.334+0000 7f03e9d05700 1 -- 172.21.15.107:0/1667893722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03e40fe040 msgr2=0x7f03e4070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:50.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.334+0000 7f03e9504700 1 --2- 172.21.15.107:0/1667893722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03e40fe040 0x7f03e4070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:39:50.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.334+0000 7f03e9d05700 1 --2- 172.21.15.107:0/1667893722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03e40fe040 0x7f03e4070310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:50.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.334+0000 7f03e9d05700 1 -- 172.21.15.107:0/1667893722 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e0005040 con 0x7f03e40fce40 2024-04-22T23:39:50.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.334+0000 7f03e9d05700 1 --2- 172.21.15.107:0/1667893722 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03e40fce40 0x7f03e4071e40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f03e000b970 tx=0x7f03e0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:39:50.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.334+0000 7f03da7fc700 1 -- 172.21.15.107:0/1667893722 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03e000e040 con 0x7f03e40fce40 2024-04-22T23:39:50.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.334+0000 7f03ebf69700 1 -- 172.21.15.107:0/1667893722 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03e4070cb0 con 0x7f03e40fce40 2024-04-22T23:39:50.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.334+0000 7f03da7fc700 1 -- 172.21.15.107:0/1667893722 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f03e000bbf0 con 0x7f03e40fce40 2024-04-22T23:39:50.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.334+0000 7f03da7fc700 1 -- 172.21.15.107:0/1667893722 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03e001c630 con 0x7f03e40fce40 2024-04-22T23:39:50.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.334+0000 7f03ebf69700 1 -- 172.21.15.107:0/1667893722 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03e4115a90 con 0x7f03e40fce40 2024-04-22T23:39:50.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.335+0000 7f03ebf69700 1 -- 172.21.15.107:0/1667893722 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03e405f5f0 con 0x7f03e40fce40 2024-04-22T23:39:50.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.336+0000 7f03da7fc700 1 -- 172.21.15.107:0/1667893722 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f03e000bd60 con 0x7f03e40fce40 2024-04-22T23:39:50.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.336+0000 7f03da7fc700 1 --2- 172.21.15.107:0/1667893722 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f03d006ec10 0x7f03d00710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:39:50.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.336+0000 7f03da7fc700 1 -- 172.21.15.107:0/1667893722 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f03e008fc30 con 0x7f03e40fce40 2024-04-22T23:39:50.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.336+0000 7f03e9504700 1 --2- 172.21.15.107:0/1667893722 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f03d006ec10 0x7f03d00710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:39:50.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.337+0000 7f03e9504700 1 --2- 172.21.15.107:0/1667893722 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f03d006ec10 0x7f03d00710c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f03d4004e00 tx=0x7f03d400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:39:50.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.341+0000 7f03da7fc700 1 -- 172.21.15.107:0/1667893722 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f03e0058a60 con 0x7f03e40fce40 2024-04-22T23:39:50.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.640+0000 7f03ebf69700 1 -- 172.21.15.107:0/1667893722 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f03e405f5f0 con 0x7f03e40fce40 2024-04-22T23:39:50.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.640+0000 7f03da7fc700 1 -- 172.21.15.107:0/1667893722 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f03e005c120 con 0x7f03e40fce40 2024-04-22T23:39:50.634 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-22T23:39:50.635 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-04-22T23:39:50.635 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:39:50.635 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:39:50.635 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-04-22T23:39:50.635 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:39:50.635 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:39:50.635 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-04-22T23:39:50.635 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 4, 2024-04-22T23:39:50.635 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 1 2024-04-22T23:39:50.635 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:39:50.635 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-04-22T23:39:50.635 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 4 2024-04-22T23:39:50.635 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:39:50.636 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-04-22T23:39:50.636 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 8, 2024-04-22T23:39:50.636 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 5 2024-04-22T23:39:50.636 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T23:39:50.636 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-22T23:39:50.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.643+0000 7f03ebf69700 1 -- 172.21.15.107:0/1667893722 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f03d006ec10 msgr2=0x7f03d00710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:50.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.643+0000 7f03ebf69700 1 --2- 172.21.15.107:0/1667893722 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f03d006ec10 0x7f03d00710c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f03d4004e00 tx=0x7f03d400b040 comp rx=0 tx=0).stop 2024-04-22T23:39:50.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.643+0000 7f03ebf69700 1 -- 172.21.15.107:0/1667893722 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03e40fce40 msgr2=0x7f03e4071e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:50.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.643+0000 7f03ebf69700 1 --2- 172.21.15.107:0/1667893722 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03e40fce40 0x7f03e4071e40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f03e000b970 tx=0x7f03e0009e30 comp rx=0 tx=0).stop 2024-04-22T23:39:50.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.643+0000 7f03ebf69700 1 -- 172.21.15.107:0/1667893722 shutdown_connections 2024-04-22T23:39:50.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.643+0000 7f03ebf69700 1 --2- 172.21.15.107:0/1667893722 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f03d006ec10 0x7f03d00710c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:50.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.643+0000 7f03ebf69700 1 --2- 172.21.15.107:0/1667893722 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03e40fce40 0x7f03e4071e40 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:50.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.643+0000 7f03ebf69700 1 --2- 172.21.15.107:0/1667893722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03e40fe040 0x7f03e4070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:50.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.643+0000 7f03ebf69700 1 -- 172.21.15.107:0/1667893722 >> 172.21.15.107:0/1667893722 conn(0x7f03e40f8570 msgr2=0x7f03e40f9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:39:50.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.643+0000 7f03ebf69700 1 -- 172.21.15.107:0/1667893722 shutdown_connections 2024-04-22T23:39:50.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.643+0000 7f03ebf69700 1 -- 172.21.15.107:0/1667893722 wait complete. 2024-04-22T23:39:50.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.754+0000 7f8d5cde1700 1 -- 172.21.15.107:0/1725070276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d58070890 msgr2=0x7f8d58070d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:50.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.754+0000 7f8d5cde1700 1 --2- 172.21.15.107:0/1725070276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d58070890 0x7f8d58070d00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c004940 tx=0x7f8d4c009e30 comp rx=0 tx=0).stop 2024-04-22T23:39:50.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.754+0000 7f8d5cde1700 1 -- 172.21.15.107:0/1725070276 shutdown_connections 2024-04-22T23:39:50.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.754+0000 7f8d5cde1700 1 --2- 172.21.15.107:0/1725070276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d58070890 0x7f8d58070d00 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:50.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.754+0000 7f8d5cde1700 1 --2- 172.21.15.107:0/1725070276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d58071e40 0x7f8d58070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:50.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.754+0000 7f8d5cde1700 1 -- 172.21.15.107:0/1725070276 >> 172.21.15.107:0/1725070276 conn(0x7f8d580f85c0 msgr2=0x7f8d580faa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:39:50.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.755+0000 7f8d5cde1700 1 -- 172.21.15.107:0/1725070276 shutdown_connections 2024-04-22T23:39:50.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.755+0000 7f8d5cde1700 1 -- 172.21.15.107:0/1725070276 wait complete. 2024-04-22T23:39:50.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.755+0000 7f8d5cde1700 1 Processor -- start 2024-04-22T23:39:50.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.755+0000 7f8d5cde1700 1 -- start start 2024-04-22T23:39:50.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.755+0000 7f8d5cde1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d58070890 0x7f8d5810fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:39:50.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.755+0000 7f8d5cde1700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d58071e40 0x7f8d581101f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:39:50.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.755+0000 7f8d5cde1700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d581107f0 con 0x7f8d58070890 2024-04-22T23:39:50.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.755+0000 7f8d5cde1700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d58110ff0 con 0x7f8d58071e40 2024-04-22T23:39:50.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.756+0000 7f8d5659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d58070890 0x7f8d5810fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:39:50.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.756+0000 7f8d5659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d58070890 0x7f8d5810fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43870/0 (socket says 172.21.15.107:43870) 2024-04-22T23:39:50.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.756+0000 7f8d5659c700 1 -- 172.21.15.107:0/273976794 learned_addr learned my addr 172.21.15.107:0/273976794 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:39:50.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.756+0000 7f8d55d9b700 1 --2- 172.21.15.107:0/273976794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d58071e40 0x7f8d581101f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:39:50.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.756+0000 7f8d5659c700 1 -- 172.21.15.107:0/273976794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d58071e40 msgr2=0x7f8d581101f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:50.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.756+0000 7f8d5659c700 1 --2- 172.21.15.107:0/273976794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d58071e40 0x7f8d581101f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:50.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.756+0000 7f8d5659c700 1 -- 172.21.15.107:0/273976794 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d4c005040 con 0x7f8d58070890 2024-04-22T23:39:50.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.756+0000 7f8d5659c700 1 --2- 172.21.15.107:0/273976794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d58070890 0x7f8d5810fcb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8d4800c800 tx=0x7f8d48004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:39:50.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.756+0000 7f8d46ffd700 1 -- 172.21.15.107:0/273976794 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d4800a760 con 0x7f8d58070890 2024-04-22T23:39:50.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.756+0000 7f8d46ffd700 1 -- 172.21.15.107:0/273976794 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d4800ad90 con 0x7f8d58070890 2024-04-22T23:39:50.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.756+0000 7f8d5cde1700 1 -- 172.21.15.107:0/273976794 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d581112d0 con 0x7f8d58070890 2024-04-22T23:39:50.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.756+0000 7f8d46ffd700 1 -- 172.21.15.107:0/273976794 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d480113a0 con 0x7f8d58070890 2024-04-22T23:39:50.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.756+0000 7f8d5cde1700 1 -- 172.21.15.107:0/273976794 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d58111810 con 0x7f8d58070890 2024-04-22T23:39:50.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.757+0000 7f8d5cde1700 1 -- 172.21.15.107:0/273976794 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d5805f5f0 con 0x7f8d58070890 2024-04-22T23:39:50.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.758+0000 7f8d46ffd700 1 -- 172.21.15.107:0/273976794 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f8d48012400 con 0x7f8d58070890 2024-04-22T23:39:50.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.759+0000 7f8d46ffd700 1 --2- 172.21.15.107:0/273976794 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f8d4006ec10 0x7f8d400710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:39:50.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.759+0000 7f8d55d9b700 1 --2- 172.21.15.107:0/273976794 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f8d4006ec10 0x7f8d400710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:39:50.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.759+0000 7f8d46ffd700 1 -- 172.21.15.107:0/273976794 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f8d4808e790 con 0x7f8d58070890 2024-04-22T23:39:50.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.759+0000 7f8d55d9b700 1 --2- 172.21.15.107:0/273976794 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f8d4006ec10 0x7f8d400710c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c00b3b0 tx=0x7f8d4c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:39:50.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:50.763+0000 7f8d46ffd700 1 -- 172.21.15.107:0/273976794 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f8d4805b1b0 con 0x7f8d58070890 2024-04-22T23:39:50.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:50 smithi107 ceph-mon[90179]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:39:50.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:50 smithi107 ceph-mon[90179]: pgmap v67: 65 pgs: 5 peering, 9 stale+active+clean, 51 active+clean; 277 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:39:50.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:50 smithi107 ceph-mon[90179]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-04-22T23:39:50.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:50 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/1667893722' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:39:50.983 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:39:50 smithi107 podman[103892]: 2024-04-22 23:39:50.600247879 +0000 UTC m=+0.582091696 container remove 875b22f0a7d75b3126f911f51eb0af320b8f0046da2f374261e0022d631ca9cd (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-22T23:39:50.983 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:39:50 smithi107 systemd[1]: ceph-8252f6f2-00ff-11ef-bc93-c7b262605968@osd.1.service: Deactivated successfully. 2024-04-22T23:39:50.983 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:39:50 smithi107 systemd[1]: Stopped Ceph osd.1 for 8252f6f2-00ff-11ef-bc93-c7b262605968. 2024-04-22T23:39:50.983 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:39:50 smithi107 systemd[1]: ceph-8252f6f2-00ff-11ef-bc93-c7b262605968@osd.1.service: Consumed 1min 699ms CPU time. 2024-04-22T23:39:50.983 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:39:50 smithi107 systemd[1]: Starting Ceph osd.1 for 8252f6f2-00ff-11ef-bc93-c7b262605968... 2024-04-22T23:39:51.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.044+0000 7f8d5cde1700 1 -- 172.21.15.107:0/273976794 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8d5804bed0 con 0x7f8d58070890 2024-04-22T23:39:51.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.045+0000 7f8d46ffd700 1 -- 172.21.15.107:0/273976794 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1909 (secure 0 0 0) 0x7f8d4805ad40 con 0x7f8d58070890 2024-04-22T23:39:51.043 INFO:teuthology.orchestra.run.smithi107.stdout:e21 2024-04-22T23:39:51.043 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T23:39:51.043 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:39:51.043 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-04-22T23:39:51.043 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:39:51.043 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-04-22T23:39:51.044 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-04-22T23:39:51.044 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 21 2024-04-22T23:39:51.044 INFO:teuthology.orchestra.run.smithi107.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-22T23:39:51.044 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-04-22T23:32:20.146130+0000 2024-04-22T23:39:51.045 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-04-22T23:39:06.006385+0000 2024-04-22T23:39:51.045 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-04-22T23:39:51.045 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-04-22T23:39:51.045 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-04-22T23:39:51.046 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-04-22T23:39:51.046 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-04-22T23:39:51.046 INFO:teuthology.orchestra.run.smithi107.stdout:max_xattr_size 65536 2024-04-22T23:39:51.046 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-04-22T23:39:51.046 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-04-22T23:39:51.046 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 46 2024-04-22T23:39:51.047 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:39:51.047 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 2 2024-04-22T23:39:51.047 INFO:teuthology.orchestra.run.smithi107.stdout:in 0,1 2024-04-22T23:39:51.047 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=14468,1=24265} 2024-04-22T23:39:51.047 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-04-22T23:39:51.047 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-04-22T23:39:51.047 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-04-22T23:39:51.047 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-04-22T23:39:51.048 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-04-22T23:39:51.048 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data disabled 2024-04-22T23:39:51.048 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-04-22T23:39:51.048 INFO:teuthology.orchestra.run.smithi107.stdout:bal_rank_mask -1 2024-04-22T23:39:51.049 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-04-22T23:39:51.049 INFO:teuthology.orchestra.run.smithi107.stdout:qdb_cluster leader: 14468 members: 24265,14468 2024-04-22T23:39:51.049 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.ktuxjl{0:14468} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.107:6828/3335313253,v1:172.21.15.107:6829/3335313253] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:39:51.049 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi157.rcaris{1:24265} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.157:6824/2779251893,v1:172.21.15.157:6825/2779251893] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:39:51.049 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:39:51.049 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:39:51.049 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-04-22T23:39:51.049 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:39:51.049 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.dfvvkw{-1:14484} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/3548717247,v1:172.21.15.107:6827/3548717247] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:39:51.050 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi157.ufywse{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.157:6826/4072444199,v1:172.21.15.157:6827/4072444199] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:39:51.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.047+0000 7f8d5cde1700 1 -- 172.21.15.107:0/273976794 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f8d4006ec10 msgr2=0x7f8d400710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:51.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.047+0000 7f8d5cde1700 1 --2- 172.21.15.107:0/273976794 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f8d4006ec10 0x7f8d400710c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c00b3b0 tx=0x7f8d4c009e30 comp rx=0 tx=0).stop 2024-04-22T23:39:51.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.047+0000 7f8d5cde1700 1 -- 172.21.15.107:0/273976794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d58070890 msgr2=0x7f8d5810fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:51.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.047+0000 7f8d5cde1700 1 --2- 172.21.15.107:0/273976794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d58070890 0x7f8d5810fcb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8d4800c800 tx=0x7f8d48004470 comp rx=0 tx=0).stop 2024-04-22T23:39:51.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.047+0000 7f8d5cde1700 1 -- 172.21.15.107:0/273976794 shutdown_connections 2024-04-22T23:39:51.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.047+0000 7f8d5cde1700 1 --2- 172.21.15.107:0/273976794 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f8d4006ec10 0x7f8d400710c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:51.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.047+0000 7f8d5cde1700 1 --2- 172.21.15.107:0/273976794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d58070890 0x7f8d5810fcb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:51.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.047+0000 7f8d5cde1700 1 --2- 172.21.15.107:0/273976794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d58071e40 0x7f8d581101f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:51.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.047+0000 7f8d5cde1700 1 -- 172.21.15.107:0/273976794 >> 172.21.15.107:0/273976794 conn(0x7f8d580f85c0 msgr2=0x7f8d580fa050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:39:51.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.047+0000 7f8d5cde1700 1 -- 172.21.15.107:0/273976794 shutdown_connections 2024-04-22T23:39:51.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.047+0000 7f8d5cde1700 1 -- 172.21.15.107:0/273976794 wait complete. 2024-04-22T23:39:51.052 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 21 2024-04-22T23:39:51.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:50 smithi157 ceph-mon[79403]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:39:51.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:50 smithi157 ceph-mon[79403]: pgmap v67: 65 pgs: 5 peering, 9 stale+active+clean, 51 active+clean; 277 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:39:51.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:50 smithi157 ceph-mon[79403]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-04-22T23:39:51.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:50 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/1667893722' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:39:51.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.151+0000 7f9d940f0700 1 -- 172.21.15.107:0/829695876 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d8c0fd710 msgr2=0x7f9d8c0fe070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:51.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.151+0000 7f9d940f0700 1 --2- 172.21.15.107:0/829695876 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d8c0fd710 0x7f9d8c0fe070 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c004940 tx=0x7f9d7c009e30 comp rx=0 tx=0).stop 2024-04-22T23:39:51.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.151+0000 7f9d940f0700 1 -- 172.21.15.107:0/829695876 shutdown_connections 2024-04-22T23:39:51.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.151+0000 7f9d940f0700 1 --2- 172.21.15.107:0/829695876 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d8c0fd710 0x7f9d8c0fe070 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:51.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.151+0000 7f9d940f0700 1 --2- 172.21.15.107:0/829695876 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d8c0fcdc0 0x7f9d8c0fd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:51.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.151+0000 7f9d940f0700 1 -- 172.21.15.107:0/829695876 >> 172.21.15.107:0/829695876 conn(0x7f9d8c0f8540 msgr2=0x7f9d8c0fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:39:51.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.151+0000 7f9d940f0700 1 -- 172.21.15.107:0/829695876 shutdown_connections 2024-04-22T23:39:51.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.151+0000 7f9d940f0700 1 -- 172.21.15.107:0/829695876 wait complete. 2024-04-22T23:39:51.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.151+0000 7f9d940f0700 1 Processor -- start 2024-04-22T23:39:51.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.151+0000 7f9d940f0700 1 -- start start 2024-04-22T23:39:51.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.151+0000 7f9d940f0700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d8c0fcdc0 0x7f9d8c071e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:39:51.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.151+0000 7f9d940f0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d8c0fd710 0x7f9d8c070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:39:51.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.151+0000 7f9d940f0700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8c070950 con 0x7f9d8c0fd710 2024-04-22T23:39:51.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.151+0000 7f9d940f0700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8c070ac0 con 0x7f9d8c0fcdc0 2024-04-22T23:39:51.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.152+0000 7f9d9168b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d8c0fd710 0x7f9d8c070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:39:51.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.152+0000 7f9d9168b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d8c0fd710 0x7f9d8c070310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43900/0 (socket says 172.21.15.107:43900) 2024-04-22T23:39:51.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.152+0000 7f9d9168b700 1 -- 172.21.15.107:0/280022912 learned_addr learned my addr 172.21.15.107:0/280022912 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:39:51.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.152+0000 7f9d9168b700 1 -- 172.21.15.107:0/280022912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d8c0fcdc0 msgr2=0x7f9d8c071e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:51.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.152+0000 7f9d9168b700 1 --2- 172.21.15.107:0/280022912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d8c0fcdc0 0x7f9d8c071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:51.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.152+0000 7f9d9168b700 1 -- 172.21.15.107:0/280022912 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d7c005040 con 0x7f9d8c0fd710 2024-04-22T23:39:51.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.152+0000 7f9d9168b700 1 --2- 172.21.15.107:0/280022912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d8c0fd710 0x7f9d8c070310 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c00b3b0 tx=0x7f9d7c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:39:51.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.152+0000 7f9d827fc700 1 -- 172.21.15.107:0/280022912 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d7c00e040 con 0x7f9d8c0fd710 2024-04-22T23:39:51.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.152+0000 7f9d827fc700 1 -- 172.21.15.107:0/280022912 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9d7c00bcc0 con 0x7f9d8c0fd710 2024-04-22T23:39:51.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.152+0000 7f9d940f0700 1 -- 172.21.15.107:0/280022912 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d8c070d40 con 0x7f9d8c0fd710 2024-04-22T23:39:51.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.152+0000 7f9d827fc700 1 -- 172.21.15.107:0/280022912 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d7c01c630 con 0x7f9d8c0fd710 2024-04-22T23:39:51.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.152+0000 7f9d940f0700 1 -- 172.21.15.107:0/280022912 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d8c115ac0 con 0x7f9d8c0fd710 2024-04-22T23:39:51.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.153+0000 7f9d940f0700 1 -- 172.21.15.107:0/280022912 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d8c05f5f0 con 0x7f9d8c0fd710 2024-04-22T23:39:51.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.154+0000 7f9d827fc700 1 -- 172.21.15.107:0/280022912 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f9d7c0139f0 con 0x7f9d8c0fd710 2024-04-22T23:39:51.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.154+0000 7f9d827fc700 1 --2- 172.21.15.107:0/280022912 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f9d7806ec00 0x7f9d780710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:39:51.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.154+0000 7f9d827fc700 1 -- 172.21.15.107:0/280022912 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f9d7c08f410 con 0x7f9d8c0fd710 2024-04-22T23:39:51.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.154+0000 7f9d91e8c700 1 --2- 172.21.15.107:0/280022912 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f9d7806ec00 0x7f9d780710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:39:51.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.154+0000 7f9d91e8c700 1 --2- 172.21.15.107:0/280022912 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f9d7806ec00 0x7f9d780710b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c071bd0 tx=0x7f9d84009370 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:39:51.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.159+0000 7f9d827fc700 1 -- 172.21.15.107:0/280022912 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f9d7c058300 con 0x7f9d8c0fd710 2024-04-22T23:39:51.297 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:39:51 smithi107 podman[104044]: 2024-04-22 23:39:50.95035949 +0000 UTC m=+0.017381951 image pull a0434699c1d76af4eea1fc95ff0a22ea73d1e04e609464fbf658d6f39bc7f2fc quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c 2024-04-22T23:39:51.297 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:39:51 smithi107 podman[104044]: 2024-04-22 23:39:51.110129637 +0000 UTC m=+0.177152091 container create ee5c23ffc7a47850cce71451ca319a16dce97627078057b580b48bd6dc93c26d (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-1-activate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, maintainer=Guillaume Abrioux ) 2024-04-22T23:39:51.297 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:39:51 smithi107 podman[104044]: 2024-04-22 23:39:51.304469589 +0000 UTC m=+0.371492043 container init ee5c23ffc7a47850cce71451ca319a16dce97627078057b580b48bd6dc93c26d (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-1-activate, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T23:39:51.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.409+0000 7f9d940f0700 1 -- 172.21.15.107:0/280022912 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d8c1023c0 con 0x7f9d7806ec00 2024-04-22T23:39:51.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.410+0000 7f9d827fc700 1 -- 172.21.15.107:0/280022912 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f9d8c1023c0 con 0x7f9d7806ec00 2024-04-22T23:39:51.404 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-22T23:39:51.404 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c", 2024-04-22T23:39:51.404 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-04-22T23:39:51.404 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-22T23:39:51.404 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [ 2024-04-22T23:39:51.404 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr", 2024-04-22T23:39:51.404 INFO:teuthology.orchestra.run.smithi107.stdout: "crash", 2024-04-22T23:39:51.404 INFO:teuthology.orchestra.run.smithi107.stdout: "mon" 2024-04-22T23:39:51.405 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-22T23:39:51.405 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "7/21 daemons upgraded", 2024-04-22T23:39:51.405 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Currently upgrading osd daemons", 2024-04-22T23:39:51.405 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-04-22T23:39:51.405 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-22T23:39:51.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.412+0000 7f9d940f0700 1 -- 172.21.15.107:0/280022912 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f9d7806ec00 msgr2=0x7f9d780710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:51.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.412+0000 7f9d940f0700 1 --2- 172.21.15.107:0/280022912 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f9d7806ec00 0x7f9d780710b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c071bd0 tx=0x7f9d84009370 comp rx=0 tx=0).stop 2024-04-22T23:39:51.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.412+0000 7f9d940f0700 1 -- 172.21.15.107:0/280022912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d8c0fd710 msgr2=0x7f9d8c070310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:51.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.412+0000 7f9d940f0700 1 --2- 172.21.15.107:0/280022912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d8c0fd710 0x7f9d8c070310 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c00b3b0 tx=0x7f9d7c009e30 comp rx=0 tx=0).stop 2024-04-22T23:39:51.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.413+0000 7f9d940f0700 1 -- 172.21.15.107:0/280022912 shutdown_connections 2024-04-22T23:39:51.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.413+0000 7f9d940f0700 1 --2- 172.21.15.107:0/280022912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d8c0fcdc0 0x7f9d8c071e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:51.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.413+0000 7f9d940f0700 1 --2- 172.21.15.107:0/280022912 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f9d7806ec00 0x7f9d780710b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:51.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.413+0000 7f9d940f0700 1 --2- 172.21.15.107:0/280022912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d8c0fd710 0x7f9d8c070310 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:51.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.413+0000 7f9d940f0700 1 -- 172.21.15.107:0/280022912 >> 172.21.15.107:0/280022912 conn(0x7f9d8c0f8540 msgr2=0x7f9d8c0f9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:39:51.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.413+0000 7f9d940f0700 1 -- 172.21.15.107:0/280022912 shutdown_connections 2024-04-22T23:39:51.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.413+0000 7f9d940f0700 1 -- 172.21.15.107:0/280022912 wait complete. 2024-04-22T23:39:51.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.516+0000 7ff05968f700 1 -- 172.21.15.107:0/629187828 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0540fbd00 msgr2=0x7ff0540fc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:51.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.516+0000 7ff05968f700 1 --2- 172.21.15.107:0/629187828 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0540fbd00 0x7ff0540fc110 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff044004940 tx=0x7ff044009e30 comp rx=0 tx=0).stop 2024-04-22T23:39:51.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.516+0000 7ff05968f700 1 -- 172.21.15.107:0/629187828 shutdown_connections 2024-04-22T23:39:51.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.516+0000 7ff05968f700 1 --2- 172.21.15.107:0/629187828 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0540fcfb0 0x7ff0540fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:51.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.516+0000 7ff05968f700 1 --2- 172.21.15.107:0/629187828 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0540fbd00 0x7ff0540fc110 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:51.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.516+0000 7ff05968f700 1 -- 172.21.15.107:0/629187828 >> 172.21.15.107:0/629187828 conn(0x7ff0540f74a0 msgr2=0x7ff0540f98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:39:51.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.516+0000 7ff05968f700 1 -- 172.21.15.107:0/629187828 shutdown_connections 2024-04-22T23:39:51.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.516+0000 7ff05968f700 1 -- 172.21.15.107:0/629187828 wait complete. 2024-04-22T23:39:51.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.517+0000 7ff05968f700 1 Processor -- start 2024-04-22T23:39:51.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.517+0000 7ff05968f700 1 -- start start 2024-04-22T23:39:51.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.517+0000 7ff05968f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0540fbd00 0x7ff0540ffc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:39:51.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.517+0000 7ff05968f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0540fcfb0 0x7ff0540fe2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:39:51.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.517+0000 7ff05968f700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff054100240 con 0x7ff0540fbd00 2024-04-22T23:39:51.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.517+0000 7ff05968f700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0540fe830 con 0x7ff0540fcfb0 2024-04-22T23:39:51.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.517+0000 7ff052ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0540fbd00 0x7ff0540ffc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:39:51.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.517+0000 7ff052ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0540fbd00 0x7ff0540ffc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40448/0 (socket says 172.21.15.107:40448) 2024-04-22T23:39:51.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.517+0000 7ff052ffd700 1 -- 172.21.15.107:0/4168437408 learned_addr learned my addr 172.21.15.107:0/4168437408 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:39:51.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.517+0000 7ff0527fc700 1 --2- 172.21.15.107:0/4168437408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0540fcfb0 0x7ff0540fe2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:39:51.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.518+0000 7ff052ffd700 1 -- 172.21.15.107:0/4168437408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0540fcfb0 msgr2=0x7ff0540fe2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:51.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.518+0000 7ff052ffd700 1 --2- 172.21.15.107:0/4168437408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0540fcfb0 0x7ff0540fe2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:51.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.518+0000 7ff052ffd700 1 -- 172.21.15.107:0/4168437408 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff044005040 con 0x7ff0540fbd00 2024-04-22T23:39:51.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.518+0000 7ff052ffd700 1 --2- 172.21.15.107:0/4168437408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0540fbd00 0x7ff0540ffc40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff044009150 tx=0x7ff044009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:39:51.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.518+0000 7ff03b7fe700 1 -- 172.21.15.107:0/4168437408 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff04400e040 con 0x7ff0540fbd00 2024-04-22T23:39:51.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.518+0000 7ff05968f700 1 -- 172.21.15.107:0/4168437408 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0540fea80 con 0x7ff0540fbd00 2024-04-22T23:39:51.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.518+0000 7ff03b7fe700 1 -- 172.21.15.107:0/4168437408 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff04400bbf0 con 0x7ff0540fbd00 2024-04-22T23:39:51.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.518+0000 7ff03b7fe700 1 -- 172.21.15.107:0/4168437408 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff04401c630 con 0x7ff0540fbd00 2024-04-22T23:39:51.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.518+0000 7ff05968f700 1 -- 172.21.15.107:0/4168437408 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0540ff040 con 0x7ff0540fbd00 2024-04-22T23:39:51.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.519+0000 7ff05968f700 1 -- 172.21.15.107:0/4168437408 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff054101b40 con 0x7ff0540fbd00 2024-04-22T23:39:51.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.520+0000 7ff03b7fe700 1 -- 172.21.15.107:0/4168437408 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7ff04400bd60 con 0x7ff0540fbd00 2024-04-22T23:39:51.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.520+0000 7ff03b7fe700 1 --2- 172.21.15.107:0/4168437408 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7ff03c06ec00 0x7ff03c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:39:51.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.521+0000 7ff0527fc700 1 --2- 172.21.15.107:0/4168437408 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7ff03c06ec00 0x7ff03c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:39:51.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.521+0000 7ff03b7fe700 1 -- 172.21.15.107:0/4168437408 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ff044090680 con 0x7ff0540fbd00 2024-04-22T23:39:51.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.521+0000 7ff0527fc700 1 --2- 172.21.15.107:0/4168437408 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7ff03c06ec00 0x7ff03c0710b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff048004f40 tx=0x7ff04800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:39:51.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.525+0000 7ff03b7fe700 1 -- 172.21.15.107:0/4168437408 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7ff04405d0a0 con 0x7ff0540fbd00 2024-04-22T23:39:51.732 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:39:51 smithi107 podman[104044]: 2024-04-22 23:39:51.308883537 +0000 UTC m=+0.375906001 container start ee5c23ffc7a47850cce71451ca319a16dce97627078057b580b48bd6dc93c26d (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-1-activate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-22T23:39:51.733 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:39:51 smithi107 podman[104044]: 2024-04-22 23:39:51.413029269 +0000 UTC m=+0.480051724 container attach ee5c23ffc7a47850cce71451ca319a16dce97627078057b580b48bd6dc93c26d (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-1-activate, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T23:39:51.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.834+0000 7ff05968f700 1 -- 172.21.15.107:0/4168437408 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff05404c710 con 0x7ff0540fbd00 2024-04-22T23:39:51.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.834+0000 7ff03b7fe700 1 -- 172.21.15.107:0/4168437408 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1387 (secure 0 0 0) 0x7ff044013e60 con 0x7ff0540fbd00 2024-04-22T23:39:51.828 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_WARN 1 osds down; Reduced data availability: 3 pgs peering; Degraded data redundancy: 47/339 objects degraded (13.864%), 14 pgs degraded 2024-04-22T23:39:51.828 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] OSD_DOWN: 1 osds down 2024-04-22T23:39:51.828 INFO:teuthology.orchestra.run.smithi107.stdout: osd.1 (root=default,host=smithi107) is down 2024-04-22T23:39:51.828 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] PG_AVAILABILITY: Reduced data availability: 3 pgs peering 2024-04-22T23:39:51.829 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.4 is stuck peering for 3m, current state peering, last acting [0,4] 2024-04-22T23:39:51.829 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.1b is stuck peering for 5m, current state peering, last acting [0,5] 2024-04-22T23:39:51.829 INFO:teuthology.orchestra.run.smithi107.stdout: pg 3.b is stuck peering for 3m, current state peering, last acting [0,4] 2024-04-22T23:39:51.829 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 47/339 objects degraded (13.864%), 14 pgs degraded 2024-04-22T23:39:51.829 INFO:teuthology.orchestra.run.smithi107.stdout: pg 1.0 is active+undersized+degraded, acting [3,0] 2024-04-22T23:39:51.829 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.0 is active+undersized+degraded, acting [3,0] 2024-04-22T23:39:51.829 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.1 is active+undersized+degraded, acting [2,0] 2024-04-22T23:39:51.829 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.2 is active+undersized+degraded, acting [5,0] 2024-04-22T23:39:51.829 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.3 is active+undersized+degraded, acting [5,2] 2024-04-22T23:39:51.829 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.6 is active+undersized+degraded, acting [3,4] 2024-04-22T23:39:51.829 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.9 is active+undersized+degraded, acting [4,0] 2024-04-22T23:39:51.829 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.a is active+undersized+degraded, acting [4,3] 2024-04-22T23:39:51.829 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.d is active+undersized+degraded, acting [3,2] 2024-04-22T23:39:51.829 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.10 is active+undersized+degraded, acting [2,0] 2024-04-22T23:39:51.829 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.11 is active+undersized+degraded, acting [4,3] 2024-04-22T23:39:51.830 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.12 is active+undersized+degraded, acting [3,0] 2024-04-22T23:39:51.830 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.15 is active+undersized+degraded, acting [3,0] 2024-04-22T23:39:51.830 INFO:teuthology.orchestra.run.smithi107.stdout: pg 2.17 is active+undersized+degraded, acting [5,2] 2024-04-22T23:39:51.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.836+0000 7ff05968f700 1 -- 172.21.15.107:0/4168437408 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7ff03c06ec00 msgr2=0x7ff03c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:51.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.836+0000 7ff05968f700 1 --2- 172.21.15.107:0/4168437408 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7ff03c06ec00 0x7ff03c0710b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff048004f40 tx=0x7ff04800b040 comp rx=0 tx=0).stop 2024-04-22T23:39:51.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.836+0000 7ff05968f700 1 -- 172.21.15.107:0/4168437408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0540fbd00 msgr2=0x7ff0540ffc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:39:51.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.836+0000 7ff05968f700 1 --2- 172.21.15.107:0/4168437408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0540fbd00 0x7ff0540ffc40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff044009150 tx=0x7ff044009e30 comp rx=0 tx=0).stop 2024-04-22T23:39:51.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.836+0000 7ff05968f700 1 -- 172.21.15.107:0/4168437408 shutdown_connections 2024-04-22T23:39:51.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.836+0000 7ff05968f700 1 --2- 172.21.15.107:0/4168437408 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7ff03c06ec00 0x7ff03c0710b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:51.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.836+0000 7ff05968f700 1 --2- 172.21.15.107:0/4168437408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0540fbd00 0x7ff0540ffc40 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:51.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.836+0000 7ff05968f700 1 --2- 172.21.15.107:0/4168437408 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0540fcfb0 0x7ff0540fe2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:39:51.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.836+0000 7ff05968f700 1 -- 172.21.15.107:0/4168437408 >> 172.21.15.107:0/4168437408 conn(0x7ff0540f74a0 msgr2=0x7ff054066820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:39:51.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.837+0000 7ff05968f700 1 -- 172.21.15.107:0/4168437408 shutdown_connections 2024-04-22T23:39:51.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:39:51.837+0000 7ff05968f700 1 -- 172.21.15.107:0/4168437408 wait complete. 2024-04-22T23:39:52.109 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:51 smithi107 ceph-mon[90179]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:39:52.109 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:51 smithi107 ceph-mon[90179]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:39:52.109 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:51 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/273976794' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:39:52.109 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:39:51 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-1-activate[104081]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-22T23:39:52.109 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:39:51 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-1-activate[104081]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-22T23:39:52.109 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:39:51 smithi107 bash[104044]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-22T23:39:52.109 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:39:51 smithi107 bash[104044]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-22T23:39:52.110 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:39:51 smithi107 bash[104044]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-22T23:39:52.110 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:39:51 smithi107 bash[104044]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-22T23:39:52.110 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:39:51 smithi107 bash[104044]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-22T23:39:52.110 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:39:51 smithi107 bash[104044]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-22T23:39:52.110 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:39:51 smithi107 bash[104044]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-04-22T23:39:52.110 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:39:51 smithi107 bash[104044]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-22T23:39:52.110 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:39:51 smithi107 bash[104044]: --> ceph-volume raw activate successful for osd ID: 1 2024-04-22T23:39:52.110 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:39:51 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-1-activate[104081]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-22T23:39:52.110 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:39:51 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-1-activate[104081]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-22T23:39:52.110 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:39:51 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-1-activate[104081]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-22T23:39:52.110 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:39:51 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-1-activate[104081]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-22T23:39:52.110 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:39:51 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-1-activate[104081]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-04-22T23:39:52.110 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:39:51 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-1-activate[104081]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-22T23:39:52.110 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:39:51 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-1-activate[104081]: --> ceph-volume raw activate successful for osd ID: 1 2024-04-22T23:39:52.111 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:39:51 smithi107 podman[104044]: 2024-04-22 23:39:51.817167639 +0000 UTC m=+0.884190093 container died ee5c23ffc7a47850cce71451ca319a16dce97627078057b580b48bd6dc93c26d (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-1-activate, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.vendor=CentOS) 2024-04-22T23:39:52.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:51 smithi157 ceph-mon[79403]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:39:52.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:51 smithi157 ceph-mon[79403]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:39:52.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:51 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/273976794' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:39:52.827 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:52 smithi107 ceph-mon[90179]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:39:52.827 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:52 smithi107 ceph-mon[90179]: pgmap v68: 65 pgs: 14 active+undersized+degraded, 5 peering, 15 active+undersized, 31 active+clean; 277 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 47/339 objects degraded (13.864%) 2024-04-22T23:39:52.827 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:52 smithi107 ceph-mon[90179]: Health check failed: Degraded data redundancy: 47/339 objects degraded (13.864%), 14 pgs degraded (PG_DEGRADED) 2024-04-22T23:39:52.827 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:52 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/4168437408' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:39:52.827 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:39:52 smithi107 podman[104234]: 2024-04-22 23:39:52.583739404 +0000 UTC m=+0.757633254 container remove ee5c23ffc7a47850cce71451ca319a16dce97627078057b580b48bd6dc93c26d (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-1-activate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T23:39:52.828 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:39:52 smithi107 podman[104299]: 2024-04-22 23:39:52.710250293 +0000 UTC m=+0.018625141 image pull a0434699c1d76af4eea1fc95ff0a22ea73d1e04e609464fbf658d6f39bc7f2fc quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c 2024-04-22T23:39:52.828 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:39:52 smithi107 podman[104299]: 2024-04-22 23:39:52.834605453 +0000 UTC m=+0.142980294 container create 77e33b6122e04756b221502e60067adc62348b80981f89757eb58f775365ed1d (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-1, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T23:39:53.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:52 smithi157 ceph-mon[79403]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:39:53.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:52 smithi157 ceph-mon[79403]: pgmap v68: 65 pgs: 14 active+undersized+degraded, 5 peering, 15 active+undersized, 31 active+clean; 277 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 47/339 objects degraded (13.864%) 2024-04-22T23:39:53.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:52 smithi157 ceph-mon[79403]: Health check failed: Degraded data redundancy: 47/339 objects degraded (13.864%), 14 pgs degraded (PG_DEGRADED) 2024-04-22T23:39:53.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:52 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/4168437408' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:39:53.482 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:39:53 smithi107 podman[104299]: 2024-04-22 23:39:53.087157105 +0000 UTC m=+0.395531956 container init 77e33b6122e04756b221502e60067adc62348b80981f89757eb58f775365ed1d (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-1, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS) 2024-04-22T23:39:53.483 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:39:53 smithi107 podman[104299]: 2024-04-22 23:39:53.090512481 +0000 UTC m=+0.398887331 container start 77e33b6122e04756b221502e60067adc62348b80981f89757eb58f775365ed1d (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-1, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS) 2024-04-22T23:39:53.483 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:39:53 smithi107 ceph-osd[104317]: -- 172.21.15.107:0/1460199916 <== mon.0 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e7982ff680 con 0x55e7982c9000 2024-04-22T23:39:53.483 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:39:53 smithi107 bash[104299]: 77e33b6122e04756b221502e60067adc62348b80981f89757eb58f775365ed1d 2024-04-22T23:39:53.483 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:39:53 smithi107 systemd[1]: Started Ceph osd.1 for 8252f6f2-00ff-11ef-bc93-c7b262605968. 2024-04-22T23:39:53.982 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:39:53 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-1[104313]: 2024-04-22T23:39:53.633+0000 7f6e314b47c0 -1 Falling back to public interface 2024-04-22T23:39:54.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:54 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:54.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:54 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:54.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:54 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:39:54.483 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:54 smithi107 ceph-mon[90179]: pgmap v69: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 277 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 54/339 objects degraded (15.929%) 2024-04-22T23:39:54.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:54 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:54.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:54 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:54.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:54 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:39:54.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:54 smithi157 ceph-mon[79403]: pgmap v69: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 277 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 54/339 objects degraded (15.929%) 2024-04-22T23:39:55.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:55 smithi107 ceph-mon[90179]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-04-22T23:39:55.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:55 smithi157 ceph-mon[79403]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-04-22T23:39:56.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:56 smithi107 ceph-mon[90179]: pgmap v70: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 277 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 54/339 objects degraded (15.929%) 2024-04-22T23:39:56.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:56 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:56.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:56 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:39:57.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:56 smithi157 ceph-mon[79403]: pgmap v70: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 277 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 54/339 objects degraded (15.929%) 2024-04-22T23:39:57.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:56 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:57.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:56 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:39:57.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:57 smithi107 ceph-mon[90179]: Health check update: Degraded data redundancy: 54/339 objects degraded (15.929%), 16 pgs degraded (PG_DEGRADED) 2024-04-22T23:39:58.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:57 smithi157 ceph-mon[79403]: Health check update: Degraded data redundancy: 54/339 objects degraded (15.929%), 16 pgs degraded (PG_DEGRADED) 2024-04-22T23:39:58.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:58 smithi107 ceph-mon[90179]: pgmap v71: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 277 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 54/339 objects degraded (15.929%) 2024-04-22T23:39:58.983 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:39:58 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-1[104313]: 2024-04-22T23:39:58.733+0000 7f6e314b47c0 -1 osd.1 54 log_to_monitors true 2024-04-22T23:39:59.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:58 smithi157 ceph-mon[79403]: pgmap v71: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 277 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 54/339 objects degraded (15.929%) 2024-04-22T23:39:59.982 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:39:59 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-1[104313]: 2024-04-22T23:39:59.758+0000 7f6e2290c700 -1 osd.1 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T23:39:59.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:59 smithi107 ceph-mon[90179]: from='osd.1 [v2:172.21.15.107:6810/997953964,v1:172.21.15.107:6811/997953964]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-22T23:39:59.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:59 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:39:59.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:39:59 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:40:00.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:59 smithi157 ceph-mon[79403]: from='osd.1 [v2:172.21.15.107:6810/997953964,v1:172.21.15.107:6811/997953964]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-22T23:40:00.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:59 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:40:00.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:39:59 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:40:00.864 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:00 smithi107 ceph-mon[90179]: pgmap v72: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 277 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 54/339 objects degraded (15.929%) 2024-04-22T23:40:00.864 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:00 smithi107 ceph-mon[90179]: from='osd.1 [v2:172.21.15.107:6810/997953964,v1:172.21.15.107:6811/997953964]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-22T23:40:00.864 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:00 smithi107 ceph-mon[90179]: osdmap e57: 6 total, 5 up, 6 in 2024-04-22T23:40:00.864 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:00 smithi107 ceph-mon[90179]: from='osd.1 [v2:172.21.15.107:6810/997953964,v1:172.21.15.107:6811/997953964]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-04-22T23:40:00.864 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:00 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:40:00.864 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:00 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:40:00.864 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:00 smithi107 ceph-mon[90179]: Health detail: HEALTH_WARN 1 osds down; Degraded data redundancy: 54/339 objects degraded (15.929%), 16 pgs degraded 2024-04-22T23:40:00.864 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:00 smithi107 ceph-mon[90179]: [WRN] OSD_DOWN: 1 osds down 2024-04-22T23:40:00.864 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:00 smithi107 ceph-mon[90179]: osd.1 (root=default,host=smithi107) is down 2024-04-22T23:40:00.865 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:00 smithi107 ceph-mon[90179]: [WRN] PG_DEGRADED: Degraded data redundancy: 54/339 objects degraded (15.929%), 16 pgs degraded 2024-04-22T23:40:00.865 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:00 smithi107 ceph-mon[90179]: pg 1.0 is active+undersized+degraded, acting [3,0] 2024-04-22T23:40:00.865 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:00 smithi107 ceph-mon[90179]: pg 2.0 is active+undersized+degraded, acting [3,0] 2024-04-22T23:40:00.865 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:00 smithi107 ceph-mon[90179]: pg 2.1 is active+undersized+degraded, acting [2,0] 2024-04-22T23:40:00.865 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:00 smithi107 ceph-mon[90179]: pg 2.2 is active+undersized+degraded, acting [5,0] 2024-04-22T23:40:00.865 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:00 smithi107 ceph-mon[90179]: pg 2.3 is active+undersized+degraded, acting [5,2] 2024-04-22T23:40:00.865 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:00 smithi107 ceph-mon[90179]: pg 2.4 is active+undersized+degraded, acting [0,4] 2024-04-22T23:40:00.865 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:00 smithi107 ceph-mon[90179]: pg 2.6 is active+undersized+degraded, acting [3,4] 2024-04-22T23:40:00.865 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:00 smithi107 ceph-mon[90179]: pg 2.9 is active+undersized+degraded, acting [4,0] 2024-04-22T23:40:00.865 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:00 smithi107 ceph-mon[90179]: pg 2.a is active+undersized+degraded, acting [4,3] 2024-04-22T23:40:00.865 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:00 smithi107 ceph-mon[90179]: pg 2.d is active+undersized+degraded, acting [3,2] 2024-04-22T23:40:00.865 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:00 smithi107 ceph-mon[90179]: pg 2.10 is active+undersized+degraded, acting [2,0] 2024-04-22T23:40:00.865 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:00 smithi107 ceph-mon[90179]: pg 2.11 is active+undersized+degraded, acting [4,3] 2024-04-22T23:40:00.865 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:00 smithi107 ceph-mon[90179]: pg 2.12 is active+undersized+degraded, acting [3,0] 2024-04-22T23:40:00.865 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:00 smithi107 ceph-mon[90179]: pg 2.15 is active+undersized+degraded, acting [3,0] 2024-04-22T23:40:00.866 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:00 smithi107 ceph-mon[90179]: pg 2.17 is active+undersized+degraded, acting [5,2] 2024-04-22T23:40:00.866 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:00 smithi107 ceph-mon[90179]: pg 2.1b is active+undersized+degraded, acting [0,5] 2024-04-22T23:40:01.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:00 smithi157 ceph-mon[79403]: pgmap v72: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 277 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 54/339 objects degraded (15.929%) 2024-04-22T23:40:01.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:00 smithi157 ceph-mon[79403]: from='osd.1 [v2:172.21.15.107:6810/997953964,v1:172.21.15.107:6811/997953964]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-22T23:40:01.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:00 smithi157 ceph-mon[79403]: osdmap e57: 6 total, 5 up, 6 in 2024-04-22T23:40:01.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:00 smithi157 ceph-mon[79403]: from='osd.1 [v2:172.21.15.107:6810/997953964,v1:172.21.15.107:6811/997953964]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-04-22T23:40:01.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:00 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:40:01.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:00 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:40:01.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:00 smithi157 ceph-mon[79403]: Health detail: HEALTH_WARN 1 osds down; Degraded data redundancy: 54/339 objects degraded (15.929%), 16 pgs degraded 2024-04-22T23:40:01.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:00 smithi157 ceph-mon[79403]: [WRN] OSD_DOWN: 1 osds down 2024-04-22T23:40:01.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:00 smithi157 ceph-mon[79403]: osd.1 (root=default,host=smithi107) is down 2024-04-22T23:40:01.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:00 smithi157 ceph-mon[79403]: [WRN] PG_DEGRADED: Degraded data redundancy: 54/339 objects degraded (15.929%), 16 pgs degraded 2024-04-22T23:40:01.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:00 smithi157 ceph-mon[79403]: pg 1.0 is active+undersized+degraded, acting [3,0] 2024-04-22T23:40:01.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:00 smithi157 ceph-mon[79403]: pg 2.0 is active+undersized+degraded, acting [3,0] 2024-04-22T23:40:01.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:00 smithi157 ceph-mon[79403]: pg 2.1 is active+undersized+degraded, acting [2,0] 2024-04-22T23:40:01.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:00 smithi157 ceph-mon[79403]: pg 2.2 is active+undersized+degraded, acting [5,0] 2024-04-22T23:40:01.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:00 smithi157 ceph-mon[79403]: pg 2.3 is active+undersized+degraded, acting [5,2] 2024-04-22T23:40:01.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:00 smithi157 ceph-mon[79403]: pg 2.4 is active+undersized+degraded, acting [0,4] 2024-04-22T23:40:01.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:00 smithi157 ceph-mon[79403]: pg 2.6 is active+undersized+degraded, acting [3,4] 2024-04-22T23:40:01.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:00 smithi157 ceph-mon[79403]: pg 2.9 is active+undersized+degraded, acting [4,0] 2024-04-22T23:40:01.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:00 smithi157 ceph-mon[79403]: pg 2.a is active+undersized+degraded, acting [4,3] 2024-04-22T23:40:01.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:00 smithi157 ceph-mon[79403]: pg 2.d is active+undersized+degraded, acting [3,2] 2024-04-22T23:40:01.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:00 smithi157 ceph-mon[79403]: pg 2.10 is active+undersized+degraded, acting [2,0] 2024-04-22T23:40:01.132 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:00 smithi157 ceph-mon[79403]: pg 2.11 is active+undersized+degraded, acting [4,3] 2024-04-22T23:40:01.133 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:00 smithi157 ceph-mon[79403]: pg 2.12 is active+undersized+degraded, acting [3,0] 2024-04-22T23:40:01.133 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:00 smithi157 ceph-mon[79403]: pg 2.15 is active+undersized+degraded, acting [3,0] 2024-04-22T23:40:01.133 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:00 smithi157 ceph-mon[79403]: pg 2.17 is active+undersized+degraded, acting [5,2] 2024-04-22T23:40:01.133 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:00 smithi157 ceph-mon[79403]: pg 2.1b is active+undersized+degraded, acting [0,5] 2024-04-22T23:40:01.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:01 smithi107 ceph-mon[90179]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-22T23:40:01.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:01 smithi107 ceph-mon[90179]: osd.1 [v2:172.21.15.107:6810/997953964,v1:172.21.15.107:6811/997953964] boot 2024-04-22T23:40:01.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:01 smithi107 ceph-mon[90179]: osdmap e58: 6 total, 6 up, 6 in 2024-04-22T23:40:01.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:01 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T23:40:02.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:01 smithi157 ceph-mon[79403]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-22T23:40:02.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:01 smithi157 ceph-mon[79403]: osd.1 [v2:172.21.15.107:6810/997953964,v1:172.21.15.107:6811/997953964] boot 2024-04-22T23:40:02.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:01 smithi157 ceph-mon[79403]: osdmap e58: 6 total, 6 up, 6 in 2024-04-22T23:40:02.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:01 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T23:40:02.862 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:02 smithi107 ceph-mon[90179]: pgmap v75: 65 pgs: 20 peering, 7 active+undersized+degraded, 7 active+undersized, 31 active+clean; 277 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 23/339 objects degraded (6.785%) 2024-04-22T23:40:02.862 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:02 smithi107 ceph-mon[90179]: osdmap e59: 6 total, 6 up, 6 in 2024-04-22T23:40:02.862 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:02 smithi107 ceph-mon[90179]: Health check update: Degraded data redundancy: 23/339 objects degraded (6.785%), 7 pgs degraded (PG_DEGRADED) 2024-04-22T23:40:03.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:02 smithi157 ceph-mon[79403]: pgmap v75: 65 pgs: 20 peering, 7 active+undersized+degraded, 7 active+undersized, 31 active+clean; 277 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 23/339 objects degraded (6.785%) 2024-04-22T23:40:03.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:02 smithi157 ceph-mon[79403]: osdmap e59: 6 total, 6 up, 6 in 2024-04-22T23:40:03.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:02 smithi157 ceph-mon[79403]: Health check update: Degraded data redundancy: 23/339 objects degraded (6.785%), 7 pgs degraded (PG_DEGRADED) 2024-04-22T23:40:04.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:04 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:40:04.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:04 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:40:04.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:04 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:40:04.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:04 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:40:04.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:04 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:40:04.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:04 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:40:04.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:04 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:40:04.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:04 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:40:04.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:04 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:40:04.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:04 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-22T23:40:04.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:04 smithi107 ceph-mon[90179]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-22T23:40:04.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:04 smithi107 ceph-mon[90179]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-04-22T23:40:04.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:04 smithi107 ceph-mon[90179]: pgmap v77: 65 pgs: 20 peering, 7 active+undersized+degraded, 5 active+undersized, 33 active+clean; 277 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 23/339 objects degraded (6.785%) 2024-04-22T23:40:04.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:04 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:40:04.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:04 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:40:04.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:04 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:40:04.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:04 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:40:04.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:04 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:40:04.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:04 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:40:04.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:04 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:40:04.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:04 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:40:04.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:04 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:40:04.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:04 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-22T23:40:04.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:04 smithi157 ceph-mon[79403]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-22T23:40:04.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:04 smithi157 ceph-mon[79403]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-04-22T23:40:04.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:04 smithi157 ceph-mon[79403]: pgmap v77: 65 pgs: 20 peering, 7 active+undersized+degraded, 5 active+undersized, 33 active+clean; 277 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 23/339 objects degraded (6.785%) 2024-04-22T23:40:06.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:05 smithi157 ceph-mon[79403]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 23/339 objects degraded (6.785%), 7 pgs degraded) 2024-04-22T23:40:06.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:05 smithi157 ceph-mon[79403]: Cluster is now healthy 2024-04-22T23:40:06.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:05 smithi107 ceph-mon[90179]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 23/339 objects degraded (6.785%), 7 pgs degraded) 2024-04-22T23:40:06.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:05 smithi107 ceph-mon[90179]: Cluster is now healthy 2024-04-22T23:40:07.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:06 smithi157 ceph-mon[79403]: pgmap v78: 65 pgs: 20 peering, 45 active+clean; 277 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:40:07.203 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:06 smithi107 ceph-mon[90179]: pgmap v78: 65 pgs: 20 peering, 45 active+clean; 277 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:40:08.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:08 smithi107 ceph-mon[90179]: pgmap v79: 65 pgs: 65 active+clean; 277 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:40:09.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:08 smithi157 ceph-mon[79403]: pgmap v79: 65 pgs: 65 active+clean; 277 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:40:11.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:10 smithi157 ceph-mon[79403]: pgmap v80: 65 pgs: 65 active+clean; 277 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:40:11.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:10 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:40:11.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:10 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:40:11.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:10 smithi107 ceph-mon[90179]: pgmap v80: 65 pgs: 65 active+clean; 277 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:40:11.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:10 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:40:11.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:10 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:40:13.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:12 smithi157 ceph-mon[79403]: pgmap v81: 65 pgs: 65 active+clean; 277 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:40:13.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:12 smithi107 ceph-mon[90179]: pgmap v81: 65 pgs: 65 active+clean; 277 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:40:14.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:13 smithi157 ceph-mon[79403]: pgmap v82: 65 pgs: 65 active+clean; 277 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:40:14.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:13 smithi107 ceph-mon[90179]: pgmap v82: 65 pgs: 65 active+clean; 277 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:40:17.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:16 smithi157 ceph-mon[79403]: pgmap v83: 65 pgs: 65 active+clean; 277 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:40:17.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:16 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:40:17.202 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:16 smithi107 ceph-mon[90179]: pgmap v83: 65 pgs: 65 active+clean; 277 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:40:17.203 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:16 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:40:18.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:18 smithi107 ceph-mon[90179]: pgmap v84: 65 pgs: 65 active+clean; 277 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:40:18.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:18 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-22T23:40:19.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:18 smithi157 ceph-mon[79403]: pgmap v84: 65 pgs: 65 active+clean; 277 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:40:19.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:18 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-22T23:40:20.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:19 smithi157 ceph-mon[79403]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-22T23:40:20.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:19 smithi157 ceph-mon[79403]: Upgrade: osd.2 is safe to restart 2024-04-22T23:40:20.135 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:19 smithi107 ceph-mon[90179]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-22T23:40:20.135 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:19 smithi107 ceph-mon[90179]: Upgrade: osd.2 is safe to restart 2024-04-22T23:40:21.081 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:20 smithi107 ceph-mon[90179]: pgmap v85: 65 pgs: 65 active+clean; 277 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:40:21.081 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:20 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:40:21.081 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:20 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-22T23:40:21.081 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:20 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:40:21.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:20 smithi157 ceph-mon[79403]: pgmap v85: 65 pgs: 65 active+clean; 277 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:40:21.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:20 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:40:21.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:20 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-22T23:40:21.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:20 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:40:21.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:21 smithi107 ceph-mon[90179]: Upgrade: Updating osd.2 2024-04-22T23:40:21.851 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:21 smithi107 ceph-mon[90179]: Deploying daemon osd.2 on smithi107 2024-04-22T23:40:21.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:21.946+0000 7f2783cc7700 1 -- 172.21.15.107:0/300732015 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f277c0fbd40 msgr2=0x7f277c0fc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:21.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:21.946+0000 7f2783cc7700 1 --2- 172.21.15.107:0/300732015 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f277c0fbd40 0x7f277c0fc150 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2778004940 tx=0x7f2778009e30 comp rx=0 tx=0).stop 2024-04-22T23:40:21.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:21.946+0000 7f2783cc7700 1 -- 172.21.15.107:0/300732015 shutdown_connections 2024-04-22T23:40:21.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:21.946+0000 7f2783cc7700 1 --2- 172.21.15.107:0/300732015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f277c0fcff0 0x7f277c0fd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:21.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:21.946+0000 7f2783cc7700 1 --2- 172.21.15.107:0/300732015 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f277c0fbd40 0x7f277c0fc150 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:21.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:21.946+0000 7f2783cc7700 1 -- 172.21.15.107:0/300732015 >> 172.21.15.107:0/300732015 conn(0x7f277c0f74c0 msgr2=0x7f277c0f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:40:21.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:21.946+0000 7f2783cc7700 1 -- 172.21.15.107:0/300732015 shutdown_connections 2024-04-22T23:40:21.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:21.946+0000 7f2783cc7700 1 -- 172.21.15.107:0/300732015 wait complete. 2024-04-22T23:40:21.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:21.946+0000 7f2783cc7700 1 Processor -- start 2024-04-22T23:40:21.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:21.947+0000 7f2783cc7700 1 -- start start 2024-04-22T23:40:21.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:21.947+0000 7f2783cc7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f277c0fbd40 0x7f277c0661e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:40:21.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:21.947+0000 7f2783cc7700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f277c0fcff0 0x7f277c112750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:40:21.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:21.947+0000 7f2783cc7700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277c112d80 con 0x7f277c0fbd40 2024-04-22T23:40:21.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:21.947+0000 7f2783cc7700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277c112ef0 con 0x7f277c0fcff0 2024-04-22T23:40:21.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:21.947+0000 7f2781a63700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f277c0fbd40 0x7f277c0661e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:40:21.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:21.947+0000 7f2781a63700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f277c0fbd40 0x7f277c0661e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38688/0 (socket says 172.21.15.107:38688) 2024-04-22T23:40:21.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:21.947+0000 7f2781a63700 1 -- 172.21.15.107:0/2862233311 learned_addr learned my addr 172.21.15.107:0/2862233311 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:40:21.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:21.947+0000 7f2781262700 1 --2- 172.21.15.107:0/2862233311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f277c0fcff0 0x7f277c112750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:40:21.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:21.947+0000 7f2781a63700 1 -- 172.21.15.107:0/2862233311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f277c0fcff0 msgr2=0x7f277c112750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:21.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:21.947+0000 7f2781a63700 1 --2- 172.21.15.107:0/2862233311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f277c0fcff0 0x7f277c112750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:21.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:21.947+0000 7f2781a63700 1 -- 172.21.15.107:0/2862233311 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2778005040 con 0x7f277c0fbd40 2024-04-22T23:40:21.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:21.947+0000 7f2781a63700 1 --2- 172.21.15.107:0/2862233311 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f277c0fbd40 0x7f277c0661e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2778009150 tx=0x7f2778009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:40:21.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:21.948+0000 7f27727fc700 1 -- 172.21.15.107:0/2862233311 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f277800e040 con 0x7f277c0fbd40 2024-04-22T23:40:21.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:21.948+0000 7f2783cc7700 1 -- 172.21.15.107:0/2862233311 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f277c113170 con 0x7f277c0fbd40 2024-04-22T23:40:21.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:21.948+0000 7f2783cc7700 1 -- 172.21.15.107:0/2862233311 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f277c113730 con 0x7f277c0fbd40 2024-04-22T23:40:21.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:21.948+0000 7f27727fc700 1 -- 172.21.15.107:0/2862233311 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f277800bbf0 con 0x7f277c0fbd40 2024-04-22T23:40:21.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:21.948+0000 7f27727fc700 1 -- 172.21.15.107:0/2862233311 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f277801c630 con 0x7f277c0fbd40 2024-04-22T23:40:21.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:21.949+0000 7f2783cc7700 1 -- 172.21.15.107:0/2862233311 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27600052e0 con 0x7f277c0fbd40 2024-04-22T23:40:21.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:21.949+0000 7f27727fc700 1 -- 172.21.15.107:0/2862233311 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f277800bd60 con 0x7f277c0fbd40 2024-04-22T23:40:21.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:21.950+0000 7f27727fc700 1 --2- 172.21.15.107:0/2862233311 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f276806ec10 0x7f27680710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:40:21.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:21.950+0000 7f27727fc700 1 -- 172.21.15.107:0/2862233311 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f277808fa60 con 0x7f277c0fbd40 2024-04-22T23:40:21.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:21.950+0000 7f2781262700 1 --2- 172.21.15.107:0/2862233311 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f276806ec10 0x7f27680710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:40:21.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:21.950+0000 7f2781262700 1 --2- 172.21.15.107:0/2862233311 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f276806ec10 0x7f27680710c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f277c0659d0 tx=0x7f276c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:40:21.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:21.955+0000 7f27727fc700 1 -- 172.21.15.107:0/2862233311 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f2778058950 con 0x7f277c0fbd40 2024-04-22T23:40:22.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:21 smithi157 ceph-mon[79403]: Upgrade: Updating osd.2 2024-04-22T23:40:22.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:21 smithi157 ceph-mon[79403]: Deploying daemon osd.2 on smithi107 2024-04-22T23:40:22.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.186+0000 7f2783cc7700 1 -- 172.21.15.107:0/2862233311 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2760000bf0 con 0x7f276806ec10 2024-04-22T23:40:22.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.187+0000 7f27727fc700 1 -- 172.21.15.107:0/2862233311 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f2760000bf0 con 0x7f276806ec10 2024-04-22T23:40:22.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.190+0000 7f2783cc7700 1 -- 172.21.15.107:0/2862233311 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f276806ec10 msgr2=0x7f27680710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:22.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.190+0000 7f2783cc7700 1 --2- 172.21.15.107:0/2862233311 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f276806ec10 0x7f27680710c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f277c0659d0 tx=0x7f276c00b040 comp rx=0 tx=0).stop 2024-04-22T23:40:22.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.190+0000 7f2783cc7700 1 -- 172.21.15.107:0/2862233311 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f277c0fbd40 msgr2=0x7f277c0661e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:22.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.190+0000 7f2783cc7700 1 --2- 172.21.15.107:0/2862233311 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f277c0fbd40 0x7f277c0661e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2778009150 tx=0x7f2778009e30 comp rx=0 tx=0).stop 2024-04-22T23:40:22.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.190+0000 7f2783cc7700 1 -- 172.21.15.107:0/2862233311 shutdown_connections 2024-04-22T23:40:22.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.190+0000 7f2783cc7700 1 --2- 172.21.15.107:0/2862233311 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f276806ec10 0x7f27680710c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:22.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.190+0000 7f2783cc7700 1 --2- 172.21.15.107:0/2862233311 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f277c0fbd40 0x7f277c0661e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:22.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.190+0000 7f2783cc7700 1 --2- 172.21.15.107:0/2862233311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f277c0fcff0 0x7f277c112750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:22.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.190+0000 7f2783cc7700 1 -- 172.21.15.107:0/2862233311 >> 172.21.15.107:0/2862233311 conn(0x7f277c0f74c0 msgr2=0x7f277c0f8bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:40:22.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.190+0000 7f2783cc7700 1 -- 172.21.15.107:0/2862233311 shutdown_connections 2024-04-22T23:40:22.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.190+0000 7f2783cc7700 1 -- 172.21.15.107:0/2862233311 wait complete. 2024-04-22T23:40:22.195 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-04-22T23:40:22.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.296+0000 7eff060b2700 1 -- 172.21.15.107:0/2658592092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff000fbd40 msgr2=0x7eff000fc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:22.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.296+0000 7eff060b2700 1 --2- 172.21.15.107:0/2658592092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff000fbd40 0x7eff000fc150 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7efef0004940 tx=0x7efef0009e30 comp rx=0 tx=0).stop 2024-04-22T23:40:22.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.296+0000 7eff060b2700 1 -- 172.21.15.107:0/2658592092 shutdown_connections 2024-04-22T23:40:22.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.296+0000 7eff060b2700 1 --2- 172.21.15.107:0/2658592092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff000fcff0 0x7eff000fd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:22.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.296+0000 7eff060b2700 1 --2- 172.21.15.107:0/2658592092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff000fbd40 0x7eff000fc150 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:22.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.296+0000 7eff060b2700 1 -- 172.21.15.107:0/2658592092 >> 172.21.15.107:0/2658592092 conn(0x7eff000f74c0 msgr2=0x7eff000f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:40:22.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.296+0000 7eff060b2700 1 -- 172.21.15.107:0/2658592092 shutdown_connections 2024-04-22T23:40:22.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.296+0000 7eff060b2700 1 -- 172.21.15.107:0/2658592092 wait complete. 2024-04-22T23:40:22.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.296+0000 7eff060b2700 1 Processor -- start 2024-04-22T23:40:22.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.296+0000 7eff060b2700 1 -- start start 2024-04-22T23:40:22.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.297+0000 7eff060b2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff000fbd40 0x7eff0010c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:40:22.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.297+0000 7eff060b2700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff000fcff0 0x7eff0010ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:40:22.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.297+0000 7eff060b2700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0010d460 con 0x7eff000fbd40 2024-04-22T23:40:22.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.297+0000 7eff060b2700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0010dcb0 con 0x7eff000fcff0 2024-04-22T23:40:22.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.297+0000 7efeff7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff000fbd40 0x7eff0010c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:40:22.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.297+0000 7efeff7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff000fbd40 0x7eff0010c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38714/0 (socket says 172.21.15.107:38714) 2024-04-22T23:40:22.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.297+0000 7efeff7fe700 1 -- 172.21.15.107:0/511973268 learned_addr learned my addr 172.21.15.107:0/511973268 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:40:22.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.297+0000 7efefeffd700 1 --2- 172.21.15.107:0/511973268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff000fcff0 0x7eff0010ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:40:22.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.297+0000 7efeff7fe700 1 -- 172.21.15.107:0/511973268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff000fcff0 msgr2=0x7eff0010ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:22.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.297+0000 7efeff7fe700 1 --2- 172.21.15.107:0/511973268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff000fcff0 0x7eff0010ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:22.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.297+0000 7efeff7fe700 1 -- 172.21.15.107:0/511973268 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef0005040 con 0x7eff000fbd40 2024-04-22T23:40:22.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.297+0000 7efeff7fe700 1 --2- 172.21.15.107:0/511973268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff000fbd40 0x7eff0010c920 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7efef0009150 tx=0x7efef0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:40:22.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.297+0000 7efee7fff700 1 -- 172.21.15.107:0/511973268 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efef000e040 con 0x7eff000fbd40 2024-04-22T23:40:22.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.297+0000 7efee7fff700 1 -- 172.21.15.107:0/511973268 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efef0004290 con 0x7eff000fbd40 2024-04-22T23:40:22.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.297+0000 7efee7fff700 1 -- 172.21.15.107:0/511973268 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efef001c630 con 0x7eff000fbd40 2024-04-22T23:40:22.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.297+0000 7eff060b2700 1 -- 172.21.15.107:0/511973268 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff0010df30 con 0x7eff000fbd40 2024-04-22T23:40:22.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.297+0000 7eff060b2700 1 -- 172.21.15.107:0/511973268 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff0010e470 con 0x7eff000fbd40 2024-04-22T23:40:22.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.298+0000 7eff060b2700 1 -- 172.21.15.107:0/511973268 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff0005f5f0 con 0x7eff000fbd40 2024-04-22T23:40:22.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.299+0000 7efee7fff700 1 -- 172.21.15.107:0/511973268 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7efef000bbf0 con 0x7eff000fbd40 2024-04-22T23:40:22.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.299+0000 7efee7fff700 1 --2- 172.21.15.107:0/511973268 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7efee806ec00 0x7efee80710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:40:22.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.299+0000 7efefeffd700 1 --2- 172.21.15.107:0/511973268 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7efee806ec00 0x7efee80710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:40:22.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.300+0000 7efee7fff700 1 -- 172.21.15.107:0/511973268 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6496+0+0 (secure 0 0 0) 0x7efef008f350 con 0x7eff000fbd40 2024-04-22T23:40:22.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.300+0000 7efefeffd700 1 --2- 172.21.15.107:0/511973268 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7efee806ec00 0x7efee80710b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7efef4007fd0 tx=0x7efef4009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:40:22.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.304+0000 7efee7fff700 1 -- 172.21.15.107:0/511973268 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7efef005bd70 con 0x7eff000fbd40 2024-04-22T23:40:22.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.530+0000 7eff060b2700 1 -- 172.21.15.107:0/511973268 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7eff0010e7a0 con 0x7efee806ec00 2024-04-22T23:40:22.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.532+0000 7efee7fff700 1 -- 172.21.15.107:0/511973268 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7eff0010e7a0 con 0x7efee806ec00 2024-04-22T23:40:22.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.534+0000 7eff060b2700 1 -- 172.21.15.107:0/511973268 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7efee806ec00 msgr2=0x7efee80710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:22.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.534+0000 7eff060b2700 1 --2- 172.21.15.107:0/511973268 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7efee806ec00 0x7efee80710b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7efef4007fd0 tx=0x7efef4009040 comp rx=0 tx=0).stop 2024-04-22T23:40:22.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.534+0000 7eff060b2700 1 -- 172.21.15.107:0/511973268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff000fbd40 msgr2=0x7eff0010c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:22.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.534+0000 7eff060b2700 1 --2- 172.21.15.107:0/511973268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff000fbd40 0x7eff0010c920 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7efef0009150 tx=0x7efef0009e30 comp rx=0 tx=0).stop 2024-04-22T23:40:22.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.534+0000 7eff060b2700 1 -- 172.21.15.107:0/511973268 shutdown_connections 2024-04-22T23:40:22.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.534+0000 7eff060b2700 1 --2- 172.21.15.107:0/511973268 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7efee806ec00 0x7efee80710b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:22.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.534+0000 7eff060b2700 1 --2- 172.21.15.107:0/511973268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff000fbd40 0x7eff0010c920 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:22.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.534+0000 7eff060b2700 1 --2- 172.21.15.107:0/511973268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff000fcff0 0x7eff0010ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:22.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.534+0000 7eff060b2700 1 -- 172.21.15.107:0/511973268 >> 172.21.15.107:0/511973268 conn(0x7eff000f74c0 msgr2=0x7eff000f8bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:40:22.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.534+0000 7eff060b2700 1 -- 172.21.15.107:0/511973268 shutdown_connections 2024-04-22T23:40:22.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.534+0000 7eff060b2700 1 -- 172.21.15.107:0/511973268 wait complete. 2024-04-22T23:40:22.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.641+0000 7f1b2dfc5700 1 -- 172.21.15.107:0/1418112890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b280fee20 msgr2=0x7f1b280faf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:22.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.641+0000 7f1b2dfc5700 1 --2- 172.21.15.107:0/1418112890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b280fee20 0x7f1b280faf60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c004940 tx=0x7f1b1c009e30 comp rx=0 tx=0).stop 2024-04-22T23:40:22.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.641+0000 7f1b2dfc5700 1 -- 172.21.15.107:0/1418112890 shutdown_connections 2024-04-22T23:40:22.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.641+0000 7f1b2dfc5700 1 --2- 172.21.15.107:0/1418112890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b280fee20 0x7f1b280faf60 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:22.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.641+0000 7f1b2dfc5700 1 --2- 172.21.15.107:0/1418112890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b280fe4d0 0x7f1b280fe8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:22.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.641+0000 7f1b2dfc5700 1 -- 172.21.15.107:0/1418112890 >> 172.21.15.107:0/1418112890 conn(0x7f1b280f6bd0 msgr2=0x7f1b280f8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:40:22.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.641+0000 7f1b2dfc5700 1 -- 172.21.15.107:0/1418112890 shutdown_connections 2024-04-22T23:40:22.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.641+0000 7f1b2dfc5700 1 -- 172.21.15.107:0/1418112890 wait complete. 2024-04-22T23:40:22.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.641+0000 7f1b2dfc5700 1 Processor -- start 2024-04-22T23:40:22.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.641+0000 7f1b2dfc5700 1 -- start start 2024-04-22T23:40:22.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.642+0000 7f1b2dfc5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b280fe4d0 0x7f1b2810da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:40:22.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.642+0000 7f1b2dfc5700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b280fee20 0x7f1b2810df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:40:22.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.642+0000 7f1b2dfc5700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2810e550 con 0x7f1b280fe4d0 2024-04-22T23:40:22.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.642+0000 7f1b2dfc5700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2810eda0 con 0x7f1b280fee20 2024-04-22T23:40:22.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.642+0000 7f1b277fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b280fe4d0 0x7f1b2810da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:40:22.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.642+0000 7f1b277fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b280fe4d0 0x7f1b2810da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38734/0 (socket says 172.21.15.107:38734) 2024-04-22T23:40:22.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.642+0000 7f1b277fe700 1 -- 172.21.15.107:0/1912471093 learned_addr learned my addr 172.21.15.107:0/1912471093 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:40:22.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.642+0000 7f1b26ffd700 1 --2- 172.21.15.107:0/1912471093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b280fee20 0x7f1b2810df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:40:22.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.642+0000 7f1b277fe700 1 -- 172.21.15.107:0/1912471093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b280fee20 msgr2=0x7f1b2810df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:22.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.642+0000 7f1b277fe700 1 --2- 172.21.15.107:0/1912471093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b280fee20 0x7f1b2810df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:22.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.642+0000 7f1b277fe700 1 -- 172.21.15.107:0/1912471093 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b1c005040 con 0x7f1b280fe4d0 2024-04-22T23:40:22.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.642+0000 7f1b277fe700 1 --2- 172.21.15.107:0/1912471093 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b280fe4d0 0x7f1b2810da10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1b1800c830 tx=0x7f1b18004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:40:22.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.642+0000 7f1b0ffff700 1 -- 172.21.15.107:0/1912471093 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b180106f0 con 0x7f1b280fe4d0 2024-04-22T23:40:22.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.642+0000 7f1b0ffff700 1 -- 172.21.15.107:0/1912471093 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1b18010d20 con 0x7f1b280fe4d0 2024-04-22T23:40:22.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.642+0000 7f1b2dfc5700 1 -- 172.21.15.107:0/1912471093 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b2810f080 con 0x7f1b280fe4d0 2024-04-22T23:40:22.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.642+0000 7f1b0ffff700 1 -- 172.21.15.107:0/1912471093 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b180113a0 con 0x7f1b280fe4d0 2024-04-22T23:40:22.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.642+0000 7f1b2dfc5700 1 -- 172.21.15.107:0/1912471093 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b2810f5c0 con 0x7f1b280fe4d0 2024-04-22T23:40:22.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.643+0000 7f1b2dfc5700 1 -- 172.21.15.107:0/1912471093 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b2805f5f0 con 0x7f1b280fe4d0 2024-04-22T23:40:22.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.644+0000 7f1b0ffff700 1 -- 172.21.15.107:0/1912471093 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f1b18012400 con 0x7f1b280fe4d0 2024-04-22T23:40:22.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.644+0000 7f1b0ffff700 1 --2- 172.21.15.107:0/1912471093 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f1b1006ec00 0x7f1b100710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:40:22.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.644+0000 7f1b0ffff700 1 -- 172.21.15.107:0/1912471093 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f1b1808e840 con 0x7f1b280fe4d0 2024-04-22T23:40:22.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.644+0000 7f1b26ffd700 1 --2- 172.21.15.107:0/1912471093 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f1b1006ec00 0x7f1b100710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:40:22.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.645+0000 7f1b26ffd700 1 --2- 172.21.15.107:0/1912471093 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f1b1006ec00 0x7f1b100710b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c00b3b0 tx=0x7f1b1c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:40:22.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.649+0000 7f1b0ffff700 1 -- 172.21.15.107:0/1912471093 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f1b1805b1a0 con 0x7f1b280fe4d0 2024-04-22T23:40:22.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.864+0000 7f1b2dfc5700 1 -- 172.21.15.107:0/1912471093 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1b2810f940 con 0x7f1b1006ec00 2024-04-22T23:40:22.857 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:22 smithi107 ceph-mon[90179]: pgmap v86: 65 pgs: 65 active+clean; 277 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:40:22.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.874+0000 7f1b0ffff700 1 -- 172.21.15.107:0/1912471093 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f1b2810f940 con 0x7f1b1006ec00 2024-04-22T23:40:22.867 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-22T23:40:22.867 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (3m) 24s ago 13m 25.6M - 0.25.0 c8568f914cd2 ec1a5c6260de 2024-04-22T23:40:22.867 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (92s) 24s ago 13m 6656k - 19.0.0-3252-g43be0201 a0434699c1d7 e074fd2cbb9c 2024-04-22T23:40:22.867 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi157 smithi157 running (86s) 83s ago 12m 6651k - 19.0.0-3252-g43be0201 a0434699c1d7 abc2649dee22 2024-04-22T23:40:22.867 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (3m) 24s ago 13m 79.5M - 9.4.7 954c08fa6188 31449ff442d2 2024-04-22T23:40:22.867 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.dfvvkw smithi107 running (7m) 24s ago 7m 13.5M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 08d3aa8d717d 2024-04-22T23:40:22.867 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.ktuxjl smithi107 running (7m) 24s ago 7m 124M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 762f27fe0f30 2024-04-22T23:40:22.867 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi157.rcaris smithi157 running (7m) 83s ago 7m 129M - 17.2.7-884-g70d0cb4b cafd8ddac8fa d23366ac085d 2024-04-22T23:40:22.867 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi157.ufywse smithi157 running (7m) 83s ago 7m 14.1M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 8b00d61eb355 2024-04-22T23:40:22.867 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.uipxvm smithi107 *:8443,9283,8765 running (3m) 24s ago 14m 500M - 19.0.0-3252-g43be0201 a0434699c1d7 d20f05dfe4f2 2024-04-22T23:40:22.867 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi157.tuylis smithi157 *:8443,9283,8765 running (2m) 83s ago 12m 449M - 19.0.0-3252-g43be0201 a0434699c1d7 957e598c9001 2024-04-22T23:40:22.868 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (2m) 24s ago 14m 47.5M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 bc72bbbe2c39 2024-04-22T23:40:22.868 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi157 smithi157 running (2m) 83s ago 12m 31.7M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 ea88de8be4fe 2024-04-22T23:40:22.868 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (3m) 24s ago 13m 16.2M - 1.5.0 0da6a335fe13 5205cfd9bd51 2024-04-22T23:40:22.868 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi157 smithi157 *:9100 running (3m) 83s ago 12m 17.0M - 1.5.0 0da6a335fe13 168ca191bb8c 2024-04-22T23:40:22.868 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (66s) 24s ago 11m 92.6M 1557M 19.0.0-3252-g43be0201 a0434699c1d7 fd0ba8ef0c27 2024-04-22T23:40:22.868 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (30s) 24s ago 11m 11.7M 1557M 19.0.0-3252-g43be0201 a0434699c1d7 77e33b6122e0 2024-04-22T23:40:22.868 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (10m) 24s ago 10m 194M 1557M 17.2.7-884-g70d0cb4b cafd8ddac8fa 9db1788096ad 2024-04-22T23:40:22.868 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi157 running (10m) 83s ago 10m 266M 2581M 17.2.7-884-g70d0cb4b cafd8ddac8fa c9fb2d1a13dd 2024-04-22T23:40:22.868 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi157 running (9m) 83s ago 9m 227M 2581M 17.2.7-884-g70d0cb4b cafd8ddac8fa 24a16e19951f 2024-04-22T23:40:22.868 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi157 running (9m) 83s ago 9m 226M 2581M 17.2.7-884-g70d0cb4b cafd8ddac8fa 4c5b7a7ef8bd 2024-04-22T23:40:22.868 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (2m) 24s ago 12m 56.6M - 2.43.0 a07b618ecd1d 99a4ea8594da 2024-04-22T23:40:22.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.876+0000 7f1b2dfc5700 1 -- 172.21.15.107:0/1912471093 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f1b1006ec00 msgr2=0x7f1b100710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:22.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.876+0000 7f1b2dfc5700 1 --2- 172.21.15.107:0/1912471093 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f1b1006ec00 0x7f1b100710b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c00b3b0 tx=0x7f1b1c009e30 comp rx=0 tx=0).stop 2024-04-22T23:40:22.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.876+0000 7f1b2dfc5700 1 -- 172.21.15.107:0/1912471093 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b280fe4d0 msgr2=0x7f1b2810da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:22.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.876+0000 7f1b2dfc5700 1 --2- 172.21.15.107:0/1912471093 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b280fe4d0 0x7f1b2810da10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1b1800c830 tx=0x7f1b18004470 comp rx=0 tx=0).stop 2024-04-22T23:40:22.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.876+0000 7f1b2dfc5700 1 -- 172.21.15.107:0/1912471093 shutdown_connections 2024-04-22T23:40:22.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.876+0000 7f1b2dfc5700 1 --2- 172.21.15.107:0/1912471093 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f1b1006ec00 0x7f1b100710b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:22.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.876+0000 7f1b2dfc5700 1 --2- 172.21.15.107:0/1912471093 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b280fe4d0 0x7f1b2810da10 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:22.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.876+0000 7f1b2dfc5700 1 --2- 172.21.15.107:0/1912471093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b280fee20 0x7f1b2810df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:22.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.876+0000 7f1b2dfc5700 1 -- 172.21.15.107:0/1912471093 >> 172.21.15.107:0/1912471093 conn(0x7f1b280f6bd0 msgr2=0x7f1b280f8280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:40:22.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.876+0000 7f1b2dfc5700 1 -- 172.21.15.107:0/1912471093 shutdown_connections 2024-04-22T23:40:22.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.876+0000 7f1b2dfc5700 1 -- 172.21.15.107:0/1912471093 wait complete. 2024-04-22T23:40:22.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.976+0000 7f3aa2d0e700 1 -- 172.21.15.107:0/1067915526 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a9c0fff90 msgr2=0x7f3a9c102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:22.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.976+0000 7f3aa2d0e700 1 --2- 172.21.15.107:0/1067915526 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a9c0fff90 0x7f3a9c102390 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3a88004940 tx=0x7f3a88009e30 comp rx=0 tx=0).stop 2024-04-22T23:40:22.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.976+0000 7f3aa2d0e700 1 -- 172.21.15.107:0/1067915526 shutdown_connections 2024-04-22T23:40:22.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.976+0000 7f3aa2d0e700 1 --2- 172.21.15.107:0/1067915526 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a9c0fff90 0x7f3a9c102390 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:22.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.976+0000 7f3aa2d0e700 1 --2- 172.21.15.107:0/1067915526 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a9c0662d0 0x7f3a9c0ffa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:22.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.976+0000 7f3aa2d0e700 1 -- 172.21.15.107:0/1067915526 >> 172.21.15.107:0/1067915526 conn(0x7f3a9c0f74c0 msgr2=0x7f3a9c0f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:40:22.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.976+0000 7f3aa2d0e700 1 -- 172.21.15.107:0/1067915526 shutdown_connections 2024-04-22T23:40:22.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.976+0000 7f3aa2d0e700 1 -- 172.21.15.107:0/1067915526 wait complete. 2024-04-22T23:40:22.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.976+0000 7f3aa2d0e700 1 Processor -- start 2024-04-22T23:40:22.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.977+0000 7f3aa2d0e700 1 -- start start 2024-04-22T23:40:22.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.977+0000 7f3aa2d0e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a9c0662d0 0x7f3a9c10da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:40:22.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.977+0000 7f3aa2d0e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a9c0fff90 0x7f3a9c10df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:40:22.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.977+0000 7f3aa2d0e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c10e570 con 0x7f3a9c0fff90 2024-04-22T23:40:22.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.977+0000 7f3aa2d0e700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c10edc0 con 0x7f3a9c0662d0 2024-04-22T23:40:22.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.977+0000 7f3a9bfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a9c0fff90 0x7f3a9c10df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:40:22.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.977+0000 7f3a9bfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a9c0fff90 0x7f3a9c10df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38754/0 (socket says 172.21.15.107:38754) 2024-04-22T23:40:22.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.977+0000 7f3a9bfff700 1 -- 172.21.15.107:0/3839471805 learned_addr learned my addr 172.21.15.107:0/3839471805 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:40:22.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.977+0000 7f3aa0aaa700 1 --2- 172.21.15.107:0/3839471805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a9c0662d0 0x7f3a9c10da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:40:22.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.977+0000 7f3a9bfff700 1 -- 172.21.15.107:0/3839471805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a9c0662d0 msgr2=0x7f3a9c10da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:22.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.977+0000 7f3a9bfff700 1 --2- 172.21.15.107:0/3839471805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a9c0662d0 0x7f3a9c10da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:22.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.977+0000 7f3a9bfff700 1 -- 172.21.15.107:0/3839471805 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a88005040 con 0x7f3a9c0fff90 2024-04-22T23:40:22.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.977+0000 7f3a9bfff700 1 --2- 172.21.15.107:0/3839471805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a9c0fff90 0x7f3a9c10df70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3a8800b3b0 tx=0x7f3a88009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:40:22.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.978+0000 7f3a997fa700 1 -- 172.21.15.107:0/3839471805 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a8800e040 con 0x7f3a9c0fff90 2024-04-22T23:40:22.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.978+0000 7f3aa2d0e700 1 -- 172.21.15.107:0/3839471805 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a9c10f040 con 0x7f3a9c0fff90 2024-04-22T23:40:22.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.978+0000 7f3a997fa700 1 -- 172.21.15.107:0/3839471805 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3a8800bbf0 con 0x7f3a9c0fff90 2024-04-22T23:40:22.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.978+0000 7f3a997fa700 1 -- 172.21.15.107:0/3839471805 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a8801c630 con 0x7f3a9c0fff90 2024-04-22T23:40:22.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.978+0000 7f3aa2d0e700 1 -- 172.21.15.107:0/3839471805 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a9c10f580 con 0x7f3a9c0fff90 2024-04-22T23:40:22.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.979+0000 7f3aa2d0e700 1 -- 172.21.15.107:0/3839471805 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a9c05f5f0 con 0x7f3a9c0fff90 2024-04-22T23:40:22.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.979+0000 7f3a997fa700 1 -- 172.21.15.107:0/3839471805 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f3a8800bd60 con 0x7f3a9c0fff90 2024-04-22T23:40:22.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.979+0000 7f3a997fa700 1 --2- 172.21.15.107:0/3839471805 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f3a8c06ec00 0x7f3a8c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:40:22.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.980+0000 7f3a997fa700 1 -- 172.21.15.107:0/3839471805 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f3a8808f420 con 0x7f3a9c0fff90 2024-04-22T23:40:22.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.980+0000 7f3aa0aaa700 1 --2- 172.21.15.107:0/3839471805 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f3a8c06ec00 0x7f3a8c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:40:22.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.980+0000 7f3aa0aaa700 1 --2- 172.21.15.107:0/3839471805 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f3a8c06ec00 0x7f3a8c0710b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c066010 tx=0x7f3a9000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:40:22.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:22.984+0000 7f3a997fa700 1 -- 172.21.15.107:0/3839471805 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f3a8805bd80 con 0x7f3a9c0fff90 2024-04-22T23:40:23.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:22 smithi157 ceph-mon[79403]: pgmap v86: 65 pgs: 65 active+clean; 277 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:40:23.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.272+0000 7f3aa2d0e700 1 -- 172.21.15.107:0/3839471805 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3a9c10f930 con 0x7f3a9c0fff90 2024-04-22T23:40:23.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.273+0000 7f3a997fa700 1 -- 172.21.15.107:0/3839471805 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f3a8805b910 con 0x7f3a9c0fff90 2024-04-22T23:40:23.266 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-22T23:40:23.266 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-04-22T23:40:23.266 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:40:23.266 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:40:23.266 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-04-22T23:40:23.266 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:40:23.267 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:40:23.267 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-04-22T23:40:23.267 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 4, 2024-04-22T23:40:23.267 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:40:23.267 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:40:23.267 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-04-22T23:40:23.267 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 4 2024-04-22T23:40:23.267 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:40:23.267 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-04-22T23:40:23.267 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 8, 2024-04-22T23:40:23.267 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 6 2024-04-22T23:40:23.267 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T23:40:23.267 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-22T23:40:23.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.275+0000 7f3aa2d0e700 1 -- 172.21.15.107:0/3839471805 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f3a8c06ec00 msgr2=0x7f3a8c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:23.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.275+0000 7f3aa2d0e700 1 --2- 172.21.15.107:0/3839471805 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f3a8c06ec00 0x7f3a8c0710b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c066010 tx=0x7f3a9000b040 comp rx=0 tx=0).stop 2024-04-22T23:40:23.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.275+0000 7f3aa2d0e700 1 -- 172.21.15.107:0/3839471805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a9c0fff90 msgr2=0x7f3a9c10df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:23.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.275+0000 7f3aa2d0e700 1 --2- 172.21.15.107:0/3839471805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a9c0fff90 0x7f3a9c10df70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3a8800b3b0 tx=0x7f3a88009e30 comp rx=0 tx=0).stop 2024-04-22T23:40:23.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.275+0000 7f3aa2d0e700 1 -- 172.21.15.107:0/3839471805 shutdown_connections 2024-04-22T23:40:23.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.275+0000 7f3aa2d0e700 1 --2- 172.21.15.107:0/3839471805 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a9c0662d0 0x7f3a9c10da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:23.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.275+0000 7f3aa2d0e700 1 --2- 172.21.15.107:0/3839471805 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f3a8c06ec00 0x7f3a8c0710b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:23.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.275+0000 7f3aa2d0e700 1 --2- 172.21.15.107:0/3839471805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a9c0fff90 0x7f3a9c10df70 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:23.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.275+0000 7f3aa2d0e700 1 -- 172.21.15.107:0/3839471805 >> 172.21.15.107:0/3839471805 conn(0x7f3a9c0f74c0 msgr2=0x7f3a9c0f7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:40:23.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.275+0000 7f3aa2d0e700 1 -- 172.21.15.107:0/3839471805 shutdown_connections 2024-04-22T23:40:23.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.275+0000 7f3aa2d0e700 1 -- 172.21.15.107:0/3839471805 wait complete. 2024-04-22T23:40:23.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.378+0000 7f0d549e3700 1 -- 172.21.15.107:0/514726529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d500fffb0 msgr2=0x7f0d501023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:23.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.378+0000 7f0d549e3700 1 --2- 172.21.15.107:0/514726529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d500fffb0 0x7f0d501023d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0d44004940 tx=0x7f0d44009e30 comp rx=0 tx=0).stop 2024-04-22T23:40:23.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.378+0000 7f0d549e3700 1 -- 172.21.15.107:0/514726529 shutdown_connections 2024-04-22T23:40:23.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.378+0000 7f0d549e3700 1 --2- 172.21.15.107:0/514726529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d500fffb0 0x7f0d501023d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:23.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.378+0000 7f0d549e3700 1 --2- 172.21.15.107:0/514726529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d50066b80 0x7f0d500ffa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:23.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.378+0000 7f0d549e3700 1 -- 172.21.15.107:0/514726529 >> 172.21.15.107:0/514726529 conn(0x7f0d500f74e0 msgr2=0x7f0d500f9910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:40:23.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.378+0000 7f0d549e3700 1 -- 172.21.15.107:0/514726529 shutdown_connections 2024-04-22T23:40:23.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.378+0000 7f0d549e3700 1 -- 172.21.15.107:0/514726529 wait complete. 2024-04-22T23:40:23.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.378+0000 7f0d549e3700 1 Processor -- start 2024-04-22T23:40:23.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.378+0000 7f0d549e3700 1 -- start start 2024-04-22T23:40:23.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.379+0000 7f0d549e3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d50066b80 0x7f0d500fd450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:40:23.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.379+0000 7f0d549e3700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d500fffb0 0x7f0d500fb970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:40:23.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.379+0000 7f0d549e3700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d500fbf90 con 0x7f0d50066b80 2024-04-22T23:40:23.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.379+0000 7f0d549e3700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d500fc0d0 con 0x7f0d500fffb0 2024-04-22T23:40:23.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.379+0000 7f0d4e59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d50066b80 0x7f0d500fd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:40:23.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.379+0000 7f0d4e59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d50066b80 0x7f0d500fd450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38780/0 (socket says 172.21.15.107:38780) 2024-04-22T23:40:23.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.379+0000 7f0d4e59c700 1 -- 172.21.15.107:0/593060840 learned_addr learned my addr 172.21.15.107:0/593060840 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:40:23.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.379+0000 7f0d4dd9b700 1 --2- 172.21.15.107:0/593060840 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d500fffb0 0x7f0d500fb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:40:23.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.379+0000 7f0d4e59c700 1 -- 172.21.15.107:0/593060840 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d500fffb0 msgr2=0x7f0d500fb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:23.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.379+0000 7f0d4e59c700 1 --2- 172.21.15.107:0/593060840 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d500fffb0 0x7f0d500fb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:23.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.379+0000 7f0d4e59c700 1 -- 172.21.15.107:0/593060840 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d44005040 con 0x7f0d50066b80 2024-04-22T23:40:23.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.379+0000 7f0d4e59c700 1 --2- 172.21.15.107:0/593060840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d50066b80 0x7f0d500fd450 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0d4000b790 tx=0x7f0d40009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:40:23.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.379+0000 7f0d3effd700 1 -- 172.21.15.107:0/593060840 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d4000f6f0 con 0x7f0d50066b80 2024-04-22T23:40:23.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.379+0000 7f0d3effd700 1 -- 172.21.15.107:0/593060840 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0d4000fd20 con 0x7f0d50066b80 2024-04-22T23:40:23.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.379+0000 7f0d549e3700 1 -- 172.21.15.107:0/593060840 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d500fc3b0 con 0x7f0d50066b80 2024-04-22T23:40:23.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.379+0000 7f0d3effd700 1 -- 172.21.15.107:0/593060840 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d400103a0 con 0x7f0d50066b80 2024-04-22T23:40:23.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.379+0000 7f0d549e3700 1 -- 172.21.15.107:0/593060840 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d50115b00 con 0x7f0d50066b80 2024-04-22T23:40:23.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.380+0000 7f0d549e3700 1 -- 172.21.15.107:0/593060840 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d5005f5f0 con 0x7f0d50066b80 2024-04-22T23:40:23.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.381+0000 7f0d3effd700 1 -- 172.21.15.107:0/593060840 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f0d40011400 con 0x7f0d50066b80 2024-04-22T23:40:23.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.381+0000 7f0d3effd700 1 --2- 172.21.15.107:0/593060840 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f0d3806ec00 0x7f0d380710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:40:23.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.382+0000 7f0d4dd9b700 1 --2- 172.21.15.107:0/593060840 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f0d3806ec00 0x7f0d380710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:40:23.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.382+0000 7f0d3effd700 1 -- 172.21.15.107:0/593060840 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f0d4008e740 con 0x7f0d50066b80 2024-04-22T23:40:23.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.382+0000 7f0d4dd9b700 1 --2- 172.21.15.107:0/593060840 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f0d3806ec00 0x7f0d380710b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0d500fda30 tx=0x7f0d44009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:40:23.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.386+0000 7f0d3effd700 1 -- 172.21.15.107:0/593060840 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f0d400574f0 con 0x7f0d50066b80 2024-04-22T23:40:23.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.666+0000 7f0d549e3700 1 -- 172.21.15.107:0/593060840 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0d50115e80 con 0x7f0d50066b80 2024-04-22T23:40:23.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.667+0000 7f0d3effd700 1 -- 172.21.15.107:0/593060840 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1909 (secure 0 0 0) 0x7f0d4005abb0 con 0x7f0d50066b80 2024-04-22T23:40:23.660 INFO:teuthology.orchestra.run.smithi107.stdout:e21 2024-04-22T23:40:23.660 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T23:40:23.660 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:40:23.660 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-04-22T23:40:23.660 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:40:23.660 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-04-22T23:40:23.660 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-04-22T23:40:23.660 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 21 2024-04-22T23:40:23.661 INFO:teuthology.orchestra.run.smithi107.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-22T23:40:23.661 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-04-22T23:32:20.146130+0000 2024-04-22T23:40:23.661 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-04-22T23:39:06.006385+0000 2024-04-22T23:40:23.661 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-04-22T23:40:23.661 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-04-22T23:40:23.661 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-04-22T23:40:23.661 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-04-22T23:40:23.661 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-04-22T23:40:23.661 INFO:teuthology.orchestra.run.smithi107.stdout:max_xattr_size 65536 2024-04-22T23:40:23.661 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-04-22T23:40:23.661 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-04-22T23:40:23.661 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 46 2024-04-22T23:40:23.661 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:40:23.661 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 2 2024-04-22T23:40:23.661 INFO:teuthology.orchestra.run.smithi107.stdout:in 0,1 2024-04-22T23:40:23.661 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=14468,1=24265} 2024-04-22T23:40:23.662 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-04-22T23:40:23.662 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-04-22T23:40:23.662 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-04-22T23:40:23.662 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-04-22T23:40:23.662 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-04-22T23:40:23.662 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data disabled 2024-04-22T23:40:23.662 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-04-22T23:40:23.662 INFO:teuthology.orchestra.run.smithi107.stdout:bal_rank_mask -1 2024-04-22T23:40:23.662 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-04-22T23:40:23.662 INFO:teuthology.orchestra.run.smithi107.stdout:qdb_cluster leader: 14468 members: 24265,14468 2024-04-22T23:40:23.662 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.ktuxjl{0:14468} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.107:6828/3335313253,v1:172.21.15.107:6829/3335313253] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:40:23.662 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi157.rcaris{1:24265} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.157:6824/2779251893,v1:172.21.15.157:6825/2779251893] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:40:23.662 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:40:23.662 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:40:23.662 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-04-22T23:40:23.663 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:40:23.663 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.dfvvkw{-1:14484} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/3548717247,v1:172.21.15.107:6827/3548717247] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:40:23.663 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi157.ufywse{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.157:6826/4072444199,v1:172.21.15.157:6827/4072444199] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:40:23.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.669+0000 7f0d549e3700 1 -- 172.21.15.107:0/593060840 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f0d3806ec00 msgr2=0x7f0d380710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:23.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.669+0000 7f0d549e3700 1 --2- 172.21.15.107:0/593060840 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f0d3806ec00 0x7f0d380710b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0d500fda30 tx=0x7f0d44009e30 comp rx=0 tx=0).stop 2024-04-22T23:40:23.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.669+0000 7f0d549e3700 1 -- 172.21.15.107:0/593060840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d50066b80 msgr2=0x7f0d500fd450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:23.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.669+0000 7f0d549e3700 1 --2- 172.21.15.107:0/593060840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d50066b80 0x7f0d500fd450 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0d4000b790 tx=0x7f0d40009220 comp rx=0 tx=0).stop 2024-04-22T23:40:23.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.669+0000 7f0d549e3700 1 -- 172.21.15.107:0/593060840 shutdown_connections 2024-04-22T23:40:23.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.669+0000 7f0d549e3700 1 --2- 172.21.15.107:0/593060840 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f0d3806ec00 0x7f0d380710b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:23.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.669+0000 7f0d549e3700 1 --2- 172.21.15.107:0/593060840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d50066b80 0x7f0d500fd450 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:23.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.669+0000 7f0d549e3700 1 --2- 172.21.15.107:0/593060840 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d500fffb0 0x7f0d500fb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:23.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.669+0000 7f0d549e3700 1 -- 172.21.15.107:0/593060840 >> 172.21.15.107:0/593060840 conn(0x7f0d500f74e0 msgr2=0x7f0d500f7900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:40:23.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.669+0000 7f0d549e3700 1 -- 172.21.15.107:0/593060840 shutdown_connections 2024-04-22T23:40:23.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.669+0000 7f0d549e3700 1 -- 172.21.15.107:0/593060840 wait complete. 2024-04-22T23:40:23.664 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 21 2024-04-22T23:40:23.756 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:40:23 smithi107 systemd[1]: Stopping Ceph osd.2 for 8252f6f2-00ff-11ef-bc93-c7b262605968... 2024-04-22T23:40:23.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.786+0000 7fa5e9318700 1 -- 172.21.15.107:0/3752668232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5e40fffd0 msgr2=0x7fa5e41023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:23.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.786+0000 7fa5e9318700 1 --2- 172.21.15.107:0/3752668232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5e40fffd0 0x7fa5e41023b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4004940 tx=0x7fa5d4009e30 comp rx=0 tx=0).stop 2024-04-22T23:40:23.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.786+0000 7fa5e9318700 1 -- 172.21.15.107:0/3752668232 shutdown_connections 2024-04-22T23:40:23.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.786+0000 7fa5e9318700 1 --2- 172.21.15.107:0/3752668232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5e40fffd0 0x7fa5e41023b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:23.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.786+0000 7fa5e9318700 1 --2- 172.21.15.107:0/3752668232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5e4066290 0x7fa5e40ffa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:23.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.786+0000 7fa5e9318700 1 -- 172.21.15.107:0/3752668232 >> 172.21.15.107:0/3752668232 conn(0x7fa5e40f74c0 msgr2=0x7fa5e40f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:40:23.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.786+0000 7fa5e9318700 1 -- 172.21.15.107:0/3752668232 shutdown_connections 2024-04-22T23:40:23.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.786+0000 7fa5e9318700 1 -- 172.21.15.107:0/3752668232 wait complete. 2024-04-22T23:40:23.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.787+0000 7fa5e9318700 1 Processor -- start 2024-04-22T23:40:23.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.787+0000 7fa5e9318700 1 -- start start 2024-04-22T23:40:23.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.787+0000 7fa5e9318700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5e4066290 0x7fa5e410d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:40:23.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.787+0000 7fa5e9318700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5e40fffd0 0x7fa5e410df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:40:23.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.787+0000 7fa5e9318700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e410e510 con 0x7fa5e40fffd0 2024-04-22T23:40:23.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.787+0000 7fa5e9318700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e410ed10 con 0x7fa5e4066290 2024-04-22T23:40:23.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.787+0000 7fa5e27fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5e40fffd0 0x7fa5e410df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:40:23.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.787+0000 7fa5e27fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5e40fffd0 0x7fa5e410df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38804/0 (socket says 172.21.15.107:38804) 2024-04-22T23:40:23.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.787+0000 7fa5e27fc700 1 -- 172.21.15.107:0/2926533872 learned_addr learned my addr 172.21.15.107:0/2926533872 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:40:23.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.787+0000 7fa5e2ffd700 1 --2- 172.21.15.107:0/2926533872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5e4066290 0x7fa5e410d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:40:23.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.787+0000 7fa5e27fc700 1 -- 172.21.15.107:0/2926533872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5e4066290 msgr2=0x7fa5e410d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:23.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.787+0000 7fa5e27fc700 1 --2- 172.21.15.107:0/2926533872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5e4066290 0x7fa5e410d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:23.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.787+0000 7fa5e27fc700 1 -- 172.21.15.107:0/2926533872 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d4005040 con 0x7fa5e40fffd0 2024-04-22T23:40:23.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.787+0000 7fa5e27fc700 1 --2- 172.21.15.107:0/2926533872 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5e40fffd0 0x7fa5e410df10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4009150 tx=0x7fa5d4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:40:23.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.788+0000 7fa5cb7fe700 1 -- 172.21.15.107:0/2926533872 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5d400e040 con 0x7fa5e40fffd0 2024-04-22T23:40:23.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.788+0000 7fa5cb7fe700 1 -- 172.21.15.107:0/2926533872 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa5d400bcc0 con 0x7fa5e40fffd0 2024-04-22T23:40:23.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.788+0000 7fa5e9318700 1 -- 172.21.15.107:0/2926533872 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5e410ef90 con 0x7fa5e40fffd0 2024-04-22T23:40:23.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.788+0000 7fa5cb7fe700 1 -- 172.21.15.107:0/2926533872 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5d401c630 con 0x7fa5e40fffd0 2024-04-22T23:40:23.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.788+0000 7fa5e9318700 1 -- 172.21.15.107:0/2926533872 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5e410f4d0 con 0x7fa5e40fffd0 2024-04-22T23:40:23.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.789+0000 7fa5e9318700 1 -- 172.21.15.107:0/2926533872 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5e405f5f0 con 0x7fa5e40fffd0 2024-04-22T23:40:23.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.789+0000 7fa5cb7fe700 1 -- 172.21.15.107:0/2926533872 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fa5d4013420 con 0x7fa5e40fffd0 2024-04-22T23:40:23.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.789+0000 7fa5cb7fe700 1 --2- 172.21.15.107:0/2926533872 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fa5cc06ec00 0x7fa5cc0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:40:23.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.790+0000 7fa5cb7fe700 1 -- 172.21.15.107:0/2926533872 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fa5d408f530 con 0x7fa5e40fffd0 2024-04-22T23:40:23.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.790+0000 7fa5e2ffd700 1 --2- 172.21.15.107:0/2926533872 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fa5cc06ec00 0x7fa5cc0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:40:23.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.790+0000 7fa5e2ffd700 1 --2- 172.21.15.107:0/2926533872 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fa5cc06ec00 0x7fa5cc0710b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa5e4065fd0 tx=0x7fa5d8007ea0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:40:23.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:23.794+0000 7fa5cb7fe700 1 -- 172.21.15.107:0/2926533872 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fa5d405be90 con 0x7fa5e40fffd0 2024-04-22T23:40:24.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.020+0000 7fa5e9318700 1 -- 172.21.15.107:0/2926533872 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa5e410f800 con 0x7fa5cc06ec00 2024-04-22T23:40:24.014 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:23 smithi107 ceph-mon[90179]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:40:24.014 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:23 smithi107 ceph-mon[90179]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:40:24.014 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:23 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/3839471805' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:40:24.014 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:23 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/593060840' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:40:24.014 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:40:23 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-2[60154]: 2024-04-22T23:40:23.877+0000 7f439c92a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T23:40:24.014 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:40:23 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-2[60154]: 2024-04-22T23:40:23.877+0000 7f439c92a700 -1 osd.2 59 *** Got signal Terminated *** 2024-04-22T23:40:24.015 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:40:23 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-2[60154]: 2024-04-22T23:40:23.877+0000 7f439c92a700 -1 osd.2 59 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T23:40:24.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.022+0000 7fa5cb7fe700 1 -- 172.21.15.107:0/2926533872 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fa5e410f800 con 0x7fa5cc06ec00 2024-04-22T23:40:24.015 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-22T23:40:24.015 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c", 2024-04-22T23:40:24.015 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-04-22T23:40:24.015 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-22T23:40:24.015 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [ 2024-04-22T23:40:24.015 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr", 2024-04-22T23:40:24.015 INFO:teuthology.orchestra.run.smithi107.stdout: "crash", 2024-04-22T23:40:24.016 INFO:teuthology.orchestra.run.smithi107.stdout: "mon" 2024-04-22T23:40:24.016 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-22T23:40:24.016 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "8/21 daemons upgraded", 2024-04-22T23:40:24.016 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Currently upgrading osd daemons", 2024-04-22T23:40:24.016 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-04-22T23:40:24.016 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-22T23:40:24.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.024+0000 7fa5e9318700 1 -- 172.21.15.107:0/2926533872 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fa5cc06ec00 msgr2=0x7fa5cc0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:24.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.024+0000 7fa5e9318700 1 --2- 172.21.15.107:0/2926533872 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fa5cc06ec00 0x7fa5cc0710b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa5e4065fd0 tx=0x7fa5d8007ea0 comp rx=0 tx=0).stop 2024-04-22T23:40:24.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.024+0000 7fa5e9318700 1 -- 172.21.15.107:0/2926533872 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5e40fffd0 msgr2=0x7fa5e410df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:24.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.024+0000 7fa5e9318700 1 --2- 172.21.15.107:0/2926533872 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5e40fffd0 0x7fa5e410df10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4009150 tx=0x7fa5d4009e30 comp rx=0 tx=0).stop 2024-04-22T23:40:24.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.024+0000 7fa5e9318700 1 -- 172.21.15.107:0/2926533872 shutdown_connections 2024-04-22T23:40:24.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.024+0000 7fa5e9318700 1 --2- 172.21.15.107:0/2926533872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5e4066290 0x7fa5e410d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:24.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.024+0000 7fa5e9318700 1 --2- 172.21.15.107:0/2926533872 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fa5cc06ec00 0x7fa5cc0710b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:24.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.024+0000 7fa5e9318700 1 --2- 172.21.15.107:0/2926533872 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5e40fffd0 0x7fa5e410df10 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:24.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.024+0000 7fa5e9318700 1 -- 172.21.15.107:0/2926533872 >> 172.21.15.107:0/2926533872 conn(0x7fa5e40f74c0 msgr2=0x7fa5e40f7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:40:24.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.024+0000 7fa5e9318700 1 -- 172.21.15.107:0/2926533872 shutdown_connections 2024-04-22T23:40:24.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.024+0000 7fa5e9318700 1 -- 172.21.15.107:0/2926533872 wait complete. 2024-04-22T23:40:24.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.124+0000 7ff211988700 1 -- 172.21.15.107:0/2201531321 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff20c0fcfb0 msgr2=0x7ff20c0fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:24.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.124+0000 7ff211988700 1 --2- 172.21.15.107:0/2201531321 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff20c0fcfb0 0x7ff20c0fd420 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff200004940 tx=0x7ff200009e30 comp rx=0 tx=0).stop 2024-04-22T23:40:24.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.124+0000 7ff211988700 1 -- 172.21.15.107:0/2201531321 shutdown_connections 2024-04-22T23:40:24.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.124+0000 7ff211988700 1 --2- 172.21.15.107:0/2201531321 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff20c0fcfb0 0x7ff20c0fd420 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:24.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.124+0000 7ff211988700 1 --2- 172.21.15.107:0/2201531321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff20c0fbd00 0x7ff20c0fc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:24.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.124+0000 7ff211988700 1 -- 172.21.15.107:0/2201531321 >> 172.21.15.107:0/2201531321 conn(0x7ff20c0f7480 msgr2=0x7ff20c0f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:40:24.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.124+0000 7ff211988700 1 -- 172.21.15.107:0/2201531321 shutdown_connections 2024-04-22T23:40:24.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.124+0000 7ff211988700 1 -- 172.21.15.107:0/2201531321 wait complete. 2024-04-22T23:40:24.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.125+0000 7ff211988700 1 Processor -- start 2024-04-22T23:40:24.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.125+0000 7ff211988700 1 -- start start 2024-04-22T23:40:24.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.125+0000 7ff211988700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff20c0fbd00 0x7ff20c066d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:40:24.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.125+0000 7ff211988700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff20c0fcfb0 0x7ff20c1127e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:40:24.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.125+0000 7ff211988700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff20c112d20 con 0x7ff20c0fbd00 2024-04-22T23:40:24.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.125+0000 7ff211988700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff20c112e60 con 0x7ff20c0fcfb0 2024-04-22T23:40:24.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.125+0000 7ff20affd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff20c0fbd00 0x7ff20c066d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:40:24.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.125+0000 7ff20affd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff20c0fbd00 0x7ff20c066d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38824/0 (socket says 172.21.15.107:38824) 2024-04-22T23:40:24.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.125+0000 7ff20affd700 1 -- 172.21.15.107:0/4080216906 learned_addr learned my addr 172.21.15.107:0/4080216906 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:40:24.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.125+0000 7ff20a7fc700 1 --2- 172.21.15.107:0/4080216906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff20c0fcfb0 0x7ff20c1127e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:40:24.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.125+0000 7ff20affd700 1 -- 172.21.15.107:0/4080216906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff20c0fcfb0 msgr2=0x7ff20c1127e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:24.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.125+0000 7ff20affd700 1 --2- 172.21.15.107:0/4080216906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff20c0fcfb0 0x7ff20c1127e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:24.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.125+0000 7ff20affd700 1 -- 172.21.15.107:0/4080216906 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff200005040 con 0x7ff20c0fbd00 2024-04-22T23:40:24.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.125+0000 7ff20affd700 1 --2- 172.21.15.107:0/4080216906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff20c0fbd00 0x7ff20c066d00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc00c8c0 tx=0x7ff1fc0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:40:24.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.126+0000 7ff1f37fe700 1 -- 172.21.15.107:0/4080216906 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1fc00a980 con 0x7ff20c0fbd00 2024-04-22T23:40:24.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.126+0000 7ff1f37fe700 1 -- 172.21.15.107:0/4080216906 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff1fc00ce60 con 0x7ff20c0fbd00 2024-04-22T23:40:24.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.126+0000 7ff211988700 1 -- 172.21.15.107:0/4080216906 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff20c1130c0 con 0x7ff20c0fbd00 2024-04-22T23:40:24.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.126+0000 7ff1f37fe700 1 -- 172.21.15.107:0/4080216906 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1fc0115e0 con 0x7ff20c0fbd00 2024-04-22T23:40:24.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.126+0000 7ff211988700 1 -- 172.21.15.107:0/4080216906 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff20c113680 con 0x7ff20c0fbd00 2024-04-22T23:40:24.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.127+0000 7ff211988700 1 -- 172.21.15.107:0/4080216906 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff20c04c710 con 0x7ff20c0fbd00 2024-04-22T23:40:24.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.127+0000 7ff1f37fe700 1 -- 172.21.15.107:0/4080216906 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7ff1fc012400 con 0x7ff20c0fbd00 2024-04-22T23:40:24.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.128+0000 7ff1f37fe700 1 --2- 172.21.15.107:0/4080216906 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7ff1f406ec00 0x7ff1f40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:40:24.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.128+0000 7ff1f37fe700 1 -- 172.21.15.107:0/4080216906 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ff1fc08e980 con 0x7ff20c0fbd00 2024-04-22T23:40:24.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.128+0000 7ff20a7fc700 1 --2- 172.21.15.107:0/4080216906 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7ff1f406ec00 0x7ff1f40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:40:24.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.128+0000 7ff20a7fc700 1 --2- 172.21.15.107:0/4080216906 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7ff1f406ec00 0x7ff1f40710b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff200004f90 tx=0x7ff200009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:40:24.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.132+0000 7ff1f37fe700 1 -- 172.21.15.107:0/4080216906 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7ff1fc05b2e0 con 0x7ff20c0fbd00 2024-04-22T23:40:24.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:23 smithi157 ceph-mon[79403]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:40:24.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:23 smithi157 ceph-mon[79403]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:40:24.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:23 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/3839471805' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:40:24.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:23 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/593060840' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:40:24.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.418+0000 7ff211988700 1 -- 172.21.15.107:0/4080216906 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff20c1047f0 con 0x7ff20c0fbd00 2024-04-22T23:40:24.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.418+0000 7ff1f37fe700 1 -- 172.21.15.107:0/4080216906 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7ff1fc0126b0 con 0x7ff20c0fbd00 2024-04-22T23:40:24.413 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_OK 2024-04-22T23:40:24.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.421+0000 7ff211988700 1 -- 172.21.15.107:0/4080216906 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7ff1f406ec00 msgr2=0x7ff1f40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:24.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.421+0000 7ff211988700 1 --2- 172.21.15.107:0/4080216906 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7ff1f406ec00 0x7ff1f40710b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff200004f90 tx=0x7ff200009e30 comp rx=0 tx=0).stop 2024-04-22T23:40:24.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.421+0000 7ff211988700 1 -- 172.21.15.107:0/4080216906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff20c0fbd00 msgr2=0x7ff20c066d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:24.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.421+0000 7ff211988700 1 --2- 172.21.15.107:0/4080216906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff20c0fbd00 0x7ff20c066d00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc00c8c0 tx=0x7ff1fc0044c0 comp rx=0 tx=0).stop 2024-04-22T23:40:24.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.421+0000 7ff211988700 1 -- 172.21.15.107:0/4080216906 shutdown_connections 2024-04-22T23:40:24.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.421+0000 7ff211988700 1 --2- 172.21.15.107:0/4080216906 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7ff1f406ec00 0x7ff1f40710b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:24.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.421+0000 7ff211988700 1 --2- 172.21.15.107:0/4080216906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff20c0fbd00 0x7ff20c066d00 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:24.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.421+0000 7ff211988700 1 --2- 172.21.15.107:0/4080216906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff20c0fcfb0 0x7ff20c1127e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:24.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.421+0000 7ff211988700 1 -- 172.21.15.107:0/4080216906 >> 172.21.15.107:0/4080216906 conn(0x7ff20c0f7480 msgr2=0x7ff20c0f8bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:40:24.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.421+0000 7ff211988700 1 -- 172.21.15.107:0/4080216906 shutdown_connections 2024-04-22T23:40:24.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:24.421+0000 7ff211988700 1 -- 172.21.15.107:0/4080216906 wait complete. 2024-04-22T23:40:25.112 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:24 smithi107 ceph-mon[90179]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:40:25.113 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:24 smithi107 ceph-mon[90179]: pgmap v87: 65 pgs: 65 active+clean; 277 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:40:25.113 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:24 smithi107 ceph-mon[90179]: osd.2 marked itself down and dead 2024-04-22T23:40:25.113 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:24 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/4080216906' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:40:25.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:24 smithi157 ceph-mon[79403]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:40:25.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:24 smithi157 ceph-mon[79403]: pgmap v87: 65 pgs: 65 active+clean; 277 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:40:25.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:24 smithi157 ceph-mon[79403]: osd.2 marked itself down and dead 2024-04-22T23:40:25.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:24 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/4080216906' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:40:25.421 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:40:25 smithi107 podman[109318]: 2024-04-22 23:40:25.120157674 +0000 UTC m=+1.369178055 container died 9db1788096adcc78d36a2de8956922a1389ce2d8f00c35ce05f08b1f70ddbf56 (image=quay.ceph.io/ceph-ci/ceph@sha256:3890cd726bd5e73ae623dabdf4ed76dd5df81c936b8bd8d761026babda969466, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-2, RELEASE=quincy-70d0cb4, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416) 2024-04-22T23:40:25.732 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:40:25 smithi107 podman[109318]: 2024-04-22 23:40:25.518745796 +0000 UTC m=+1.767766175 container cleanup 9db1788096adcc78d36a2de8956922a1389ce2d8f00c35ce05f08b1f70ddbf56 (image=quay.ceph.io/ceph-ci/ceph@sha256:3890cd726bd5e73ae623dabdf4ed76dd5df81c936b8bd8d761026babda969466, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-70d0cb4, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, ceph=True, org.label-schema.schema-version=1.0) 2024-04-22T23:40:25.732 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:40:25 smithi107 bash[109318]: ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-2 2024-04-22T23:40:26.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:25 smithi157 ceph-mon[79403]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:40:26.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:25 smithi157 ceph-mon[79403]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-22T23:40:26.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:25 smithi157 ceph-mon[79403]: osdmap e60: 6 total, 5 up, 6 in 2024-04-22T23:40:26.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:25 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:40:26.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:25 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:40:26.232 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:40:25 smithi107 podman[109381]: 2024-04-22 23:40:25.937355925 +0000 UTC m=+0.820417722 container remove 9db1788096adcc78d36a2de8956922a1389ce2d8f00c35ce05f08b1f70ddbf56 (image=quay.ceph.io/ceph-ci/ceph@sha256:3890cd726bd5e73ae623dabdf4ed76dd5df81c936b8bd8d761026babda969466, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=quincy-70d0cb4, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-22T23:40:26.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:25 smithi107 ceph-mon[90179]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:40:26.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:25 smithi107 ceph-mon[90179]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-22T23:40:26.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:25 smithi107 ceph-mon[90179]: osdmap e60: 6 total, 5 up, 6 in 2024-04-22T23:40:26.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:25 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:40:26.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:25 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:40:26.608 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:40:26 smithi107 podman[109473]: 2024-04-22 23:40:26.192166788 +0000 UTC m=+0.017777623 image pull a0434699c1d76af4eea1fc95ff0a22ea73d1e04e609464fbf658d6f39bc7f2fc quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c 2024-04-22T23:40:26.608 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:40:26 smithi107 podman[109473]: 2024-04-22 23:40:26.304971801 +0000 UTC m=+0.130582681 container create 4cdcdf6427bf0cb30aed485b71c9d09cabcf85dd1cb0265f9987cb22c444a44c (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240416, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-22T23:40:26.608 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:40:26 smithi107 podman[109473]: 2024-04-22 23:40:26.532171166 +0000 UTC m=+0.357782000 container init 4cdcdf6427bf0cb30aed485b71c9d09cabcf85dd1cb0265f9987cb22c444a44c (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-2-deactivate, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-22T23:40:26.609 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:40:26 smithi107 podman[109473]: 2024-04-22 23:40:26.535257643 +0000 UTC m=+0.360868487 container start 4cdcdf6427bf0cb30aed485b71c9d09cabcf85dd1cb0265f9987cb22c444a44c (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-22T23:40:26.609 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:40:26 smithi107 podman[109473]: 2024-04-22 23:40:26.615438346 +0000 UTC m=+0.441049196 container attach 4cdcdf6427bf0cb30aed485b71c9d09cabcf85dd1cb0265f9987cb22c444a44c (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-22T23:40:26.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:26 smithi107 ceph-mon[90179]: pgmap v89: 65 pgs: 9 stale+active+clean, 56 active+clean; 277 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:40:26.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:26 smithi107 ceph-mon[90179]: osdmap e61: 6 total, 5 up, 6 in 2024-04-22T23:40:26.983 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:40:26 smithi107 podman[109473]: 2024-04-22 23:40:26.835559672 +0000 UTC m=+0.661170510 container died 4cdcdf6427bf0cb30aed485b71c9d09cabcf85dd1cb0265f9987cb22c444a44c (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-adk-testing-2024-04-22-1618-43be020) 2024-04-22T23:40:27.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:26 smithi157 ceph-mon[79403]: pgmap v89: 65 pgs: 9 stale+active+clean, 56 active+clean; 277 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-22T23:40:27.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:26 smithi157 ceph-mon[79403]: osdmap e61: 6 total, 5 up, 6 in 2024-04-22T23:40:27.829 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:40:27 smithi107 podman[109499]: 2024-04-22 23:40:27.496214304 +0000 UTC m=+0.655756785 container remove 4cdcdf6427bf0cb30aed485b71c9d09cabcf85dd1cb0265f9987cb22c444a44c (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-22T23:40:27.829 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:40:27 smithi107 systemd[1]: ceph-8252f6f2-00ff-11ef-bc93-c7b262605968@osd.2.service: Deactivated successfully. 2024-04-22T23:40:27.829 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:40:27 smithi107 systemd[1]: Stopped Ceph osd.2 for 8252f6f2-00ff-11ef-bc93-c7b262605968. 2024-04-22T23:40:27.829 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:40:27 smithi107 systemd[1]: ceph-8252f6f2-00ff-11ef-bc93-c7b262605968@osd.2.service: Consumed 48.101s CPU time. 2024-04-22T23:40:27.829 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:40:27 smithi107 systemd[1]: Starting Ceph osd.2 for 8252f6f2-00ff-11ef-bc93-c7b262605968... 2024-04-22T23:40:28.080 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:27 smithi107 ceph-mon[90179]: pgmap v91: 65 pgs: 5 active+undersized+degraded, 7 stale+active+clean, 7 active+undersized, 46 active+clean; 277 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 16/339 objects degraded (4.720%) 2024-04-22T23:40:28.080 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:27 smithi107 ceph-mon[90179]: Health check failed: Degraded data redundancy: 16/339 objects degraded (4.720%), 5 pgs degraded (PG_DEGRADED) 2024-04-22T23:40:28.080 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:40:27 smithi107 podman[109602]: 2024-04-22 23:40:27.854447365 +0000 UTC m=+0.018141372 image pull a0434699c1d76af4eea1fc95ff0a22ea73d1e04e609464fbf658d6f39bc7f2fc quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c 2024-04-22T23:40:28.080 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:40:27 smithi107 podman[109602]: 2024-04-22 23:40:27.955595507 +0000 UTC m=+0.119289513 container create e4339bfaf7f7f0fff0151fd96ba4743b62f3bd489aa8a6c882fb95beec05c7c6 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-2-activate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-adk-testing-2024-04-22-1618-43be020) 2024-04-22T23:40:28.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:27 smithi157 ceph-mon[79403]: pgmap v91: 65 pgs: 5 active+undersized+degraded, 7 stale+active+clean, 7 active+undersized, 46 active+clean; 277 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 16/339 objects degraded (4.720%) 2024-04-22T23:40:28.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:27 smithi157 ceph-mon[79403]: Health check failed: Degraded data redundancy: 16/339 objects degraded (4.720%), 5 pgs degraded (PG_DEGRADED) 2024-04-22T23:40:28.482 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:40:28 smithi107 podman[109602]: 2024-04-22 23:40:28.216863925 +0000 UTC m=+0.380557935 container init e4339bfaf7f7f0fff0151fd96ba4743b62f3bd489aa8a6c882fb95beec05c7c6 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-2-activate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_BRANCH=HEAD) 2024-04-22T23:40:28.483 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:40:28 smithi107 podman[109602]: 2024-04-22 23:40:28.219945906 +0000 UTC m=+0.383639917 container start e4339bfaf7f7f0fff0151fd96ba4743b62f3bd489aa8a6c882fb95beec05c7c6 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-2-activate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-22T23:40:28.483 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:40:28 smithi107 podman[109602]: 2024-04-22 23:40:28.316670504 +0000 UTC m=+0.480364523 container attach e4339bfaf7f7f0fff0151fd96ba4743b62f3bd489aa8a6c882fb95beec05c7c6 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-2-activate, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, ceph=True) 2024-04-22T23:40:28.983 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:40:28 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-2-activate[109617]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-22T23:40:28.983 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:40:28 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-2-activate[109617]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-22T23:40:28.983 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:40:28 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-2-activate[109617]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-22T23:40:28.983 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:40:28 smithi107 bash[109602]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-22T23:40:28.983 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:40:28 smithi107 bash[109602]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-22T23:40:28.983 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:40:28 smithi107 bash[109602]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-22T23:40:28.983 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:40:28 smithi107 bash[109602]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-22T23:40:28.983 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:40:28 smithi107 bash[109602]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-22T23:40:28.983 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:40:28 smithi107 bash[109602]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-22T23:40:28.984 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:40:28 smithi107 bash[109602]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-04-22T23:40:28.984 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:40:28 smithi107 bash[109602]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-22T23:40:28.984 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:40:28 smithi107 bash[109602]: --> ceph-volume raw activate successful for osd ID: 2 2024-04-22T23:40:28.984 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:40:28 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-2-activate[109617]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-22T23:40:28.984 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:40:28 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-2-activate[109617]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-22T23:40:28.984 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:40:28 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-2-activate[109617]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-22T23:40:28.984 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:40:28 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-2-activate[109617]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-04-22T23:40:28.984 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:40:28 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-2-activate[109617]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-22T23:40:28.984 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:40:28 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-2-activate[109617]: --> ceph-volume raw activate successful for osd ID: 2 2024-04-22T23:40:28.984 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:40:28 smithi107 podman[109602]: 2024-04-22 23:40:28.718949458 +0000 UTC m=+0.882643469 container died e4339bfaf7f7f0fff0151fd96ba4743b62f3bd489aa8a6c882fb95beec05c7c6 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-2-activate, CEPH_POINT_RELEASE=, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T23:40:29.732 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:40:29 smithi107 podman[109747]: 2024-04-22 23:40:29.45373543 +0000 UTC m=+0.728990907 container remove e4339bfaf7f7f0fff0151fd96ba4743b62f3bd489aa8a6c882fb95beec05c7c6 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-2-activate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-adk-testing-2024-04-22-1618-43be020) 2024-04-22T23:40:29.732 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:40:29 smithi107 podman[109814]: 2024-04-22 23:40:29.579156196 +0000 UTC m=+0.019048846 image pull a0434699c1d76af4eea1fc95ff0a22ea73d1e04e609464fbf658d6f39bc7f2fc quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c 2024-04-22T23:40:29.732 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:40:29 smithi107 podman[109814]: 2024-04-22 23:40:29.67985601 +0000 UTC m=+0.119748655 container create c307bd8cdcfcf2460761891e53a002ded7355fac92d94fe6311f36fd97444d85 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux ) 2024-04-22T23:40:30.232 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:40:29 smithi107 podman[109814]: 2024-04-22 23:40:29.924019525 +0000 UTC m=+0.363912169 container init c307bd8cdcfcf2460761891e53a002ded7355fac92d94fe6311f36fd97444d85 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-2, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-22T23:40:30.232 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:40:29 smithi107 podman[109814]: 2024-04-22 23:40:29.927020332 +0000 UTC m=+0.366912985 container start c307bd8cdcfcf2460761891e53a002ded7355fac92d94fe6311f36fd97444d85 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-2, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-22T23:40:30.232 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:40:29 smithi107 ceph-osd[109833]: -- 172.21.15.107:0/271593465 <== mon.0 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5584d6c0b680 con 0x5584d6bd5000 2024-04-22T23:40:30.232 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:40:30 smithi107 bash[109814]: c307bd8cdcfcf2460761891e53a002ded7355fac92d94fe6311f36fd97444d85 2024-04-22T23:40:30.232 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:40:30 smithi107 systemd[1]: Started Ceph osd.2 for 8252f6f2-00ff-11ef-bc93-c7b262605968. 2024-04-22T23:40:30.725 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:40:30 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-2[109829]: 2024-04-22T23:40:30.474+0000 7fa80c7317c0 -1 Falling back to public interface 2024-04-22T23:40:30.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:30 smithi107 ceph-mon[90179]: pgmap v92: 65 pgs: 9 active+undersized+degraded, 4 stale+active+clean, 11 active+undersized, 41 active+clean; 277 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 27/339 objects degraded (7.965%) 2024-04-22T23:40:30.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:30 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:40:30.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:30 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:40:30.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:30 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:40:31.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:30 smithi157 ceph-mon[79403]: pgmap v92: 65 pgs: 9 active+undersized+degraded, 4 stale+active+clean, 11 active+undersized, 41 active+clean; 277 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 27/339 objects degraded (7.965%) 2024-04-22T23:40:31.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:30 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:40:31.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:30 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:40:31.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:30 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:40:32.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:32 smithi107 ceph-mon[90179]: pgmap v93: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 277 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 40/339 objects degraded (11.799%) 2024-04-22T23:40:33.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:32 smithi157 ceph-mon[79403]: pgmap v93: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 277 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 40/339 objects degraded (11.799%) 2024-04-22T23:40:34.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:34 smithi107 ceph-mon[90179]: pgmap v94: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 277 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 40/339 objects degraded (11.799%) 2024-04-22T23:40:34.982 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:40:34 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-2[109829]: 2024-04-22T23:40:34.791+0000 7fa80c7317c0 -1 osd.2 59 log_to_monitors true 2024-04-22T23:40:35.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:34 smithi157 ceph-mon[79403]: pgmap v94: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 277 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 40/339 objects degraded (11.799%) 2024-04-22T23:40:35.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:35 smithi107 ceph-mon[90179]: from='osd.2 [v2:172.21.15.107:6818/1823995026,v1:172.21.15.107:6819/1823995026]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-22T23:40:35.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:35 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:40:35.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:35 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:40:36.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:35 smithi157 ceph-mon[79403]: from='osd.2 [v2:172.21.15.107:6818/1823995026,v1:172.21.15.107:6819/1823995026]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-22T23:40:36.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:35 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:40:36.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:35 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:40:36.982 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:40:36 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-2[109829]: 2024-04-22T23:40:36.675+0000 7fa7f9b81700 -1 osd.2 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T23:40:36.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:36 smithi107 ceph-mon[90179]: pgmap v95: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 277 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 40/339 objects degraded (11.799%) 2024-04-22T23:40:36.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:36 smithi107 ceph-mon[90179]: from='osd.2 [v2:172.21.15.107:6818/1823995026,v1:172.21.15.107:6819/1823995026]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-22T23:40:36.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:36 smithi107 ceph-mon[90179]: osdmap e62: 6 total, 5 up, 6 in 2024-04-22T23:40:36.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:36 smithi107 ceph-mon[90179]: from='osd.2 [v2:172.21.15.107:6818/1823995026,v1:172.21.15.107:6819/1823995026]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-04-22T23:40:36.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:36 smithi107 ceph-mon[90179]: from='osd.2 [v2:172.21.15.107:6818/1823995026,v1:172.21.15.107:6819/1823995026]' entity='osd.2' 2024-04-22T23:40:37.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:36 smithi157 ceph-mon[79403]: pgmap v95: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 277 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 40/339 objects degraded (11.799%) 2024-04-22T23:40:37.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:36 smithi157 ceph-mon[79403]: from='osd.2 [v2:172.21.15.107:6818/1823995026,v1:172.21.15.107:6819/1823995026]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-22T23:40:37.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:36 smithi157 ceph-mon[79403]: osdmap e62: 6 total, 5 up, 6 in 2024-04-22T23:40:37.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:36 smithi157 ceph-mon[79403]: from='osd.2 [v2:172.21.15.107:6818/1823995026,v1:172.21.15.107:6819/1823995026]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-04-22T23:40:37.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:36 smithi157 ceph-mon[79403]: from='osd.2 [v2:172.21.15.107:6818/1823995026,v1:172.21.15.107:6819/1823995026]' entity='osd.2' 2024-04-22T23:40:38.036 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:37 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:40:38.036 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:37 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:40:38.036 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:37 smithi107 ceph-mon[90179]: Health check update: Degraded data redundancy: 40/339 objects degraded (11.799%), 12 pgs degraded (PG_DEGRADED) 2024-04-22T23:40:38.036 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:37 smithi107 ceph-mon[90179]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-22T23:40:38.036 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:37 smithi107 ceph-mon[90179]: osd.2 [v2:172.21.15.107:6818/1823995026,v1:172.21.15.107:6819/1823995026] boot 2024-04-22T23:40:38.036 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:37 smithi107 ceph-mon[90179]: osdmap e63: 6 total, 6 up, 6 in 2024-04-22T23:40:38.036 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:37 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T23:40:38.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:37 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:40:38.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:37 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:40:38.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:37 smithi157 ceph-mon[79403]: Health check update: Degraded data redundancy: 40/339 objects degraded (11.799%), 12 pgs degraded (PG_DEGRADED) 2024-04-22T23:40:38.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:37 smithi157 ceph-mon[79403]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-22T23:40:38.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:37 smithi157 ceph-mon[79403]: osd.2 [v2:172.21.15.107:6818/1823995026,v1:172.21.15.107:6819/1823995026] boot 2024-04-22T23:40:38.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:37 smithi157 ceph-mon[79403]: osdmap e63: 6 total, 6 up, 6 in 2024-04-22T23:40:38.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:37 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T23:40:38.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:38 smithi107 ceph-mon[90179]: pgmap v98: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 277 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 40/339 objects degraded (11.799%) 2024-04-22T23:40:38.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:38 smithi107 ceph-mon[90179]: osdmap e64: 6 total, 6 up, 6 in 2024-04-22T23:40:39.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:38 smithi157 ceph-mon[79403]: pgmap v98: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 277 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 40/339 objects degraded (11.799%) 2024-04-22T23:40:39.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:38 smithi157 ceph-mon[79403]: osdmap e64: 6 total, 6 up, 6 in 2024-04-22T23:40:40.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:40 smithi107 ceph-mon[90179]: pgmap v100: 65 pgs: 5 peering, 10 active+undersized+degraded, 12 active+undersized, 38 active+clean; 277 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 32/339 objects degraded (9.440%) 2024-04-22T23:40:40.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:40 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:40:40.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:40 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:40:40.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:40 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:40:40.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:40 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:40:40.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:40 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:40:40.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:40 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:40:40.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:40 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:40:40.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:40 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:40:40.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:40 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:40:40.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:40 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-22T23:40:40.984 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:40 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:40:40.984 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:40 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:40:41.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:40 smithi157 ceph-mon[79403]: pgmap v100: 65 pgs: 5 peering, 10 active+undersized+degraded, 12 active+undersized, 38 active+clean; 277 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 32/339 objects degraded (9.440%) 2024-04-22T23:40:41.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:40 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:40:41.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:40 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:40:41.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:40 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:40:41.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:40 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:40:41.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:40 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:40:41.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:40 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:40:41.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:40 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:40:41.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:40 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:40:41.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:40 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:40:41.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:40 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-22T23:40:41.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:40 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:40:41.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:40 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:40:42.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:41 smithi157 ceph-mon[79403]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-22T23:40:42.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:41 smithi157 ceph-mon[79403]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-04-22T23:40:42.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:41 smithi107 ceph-mon[90179]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-22T23:40:42.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:41 smithi107 ceph-mon[90179]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-04-22T23:40:43.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:42 smithi157 ceph-mon[79403]: pgmap v101: 65 pgs: 5 peering, 60 active+clean; 277 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:40:43.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:42 smithi157 ceph-mon[79403]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 32/339 objects degraded (9.440%), 10 pgs degraded) 2024-04-22T23:40:43.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:42 smithi157 ceph-mon[79403]: Cluster is now healthy 2024-04-22T23:40:43.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:42 smithi107 ceph-mon[90179]: pgmap v101: 65 pgs: 5 peering, 60 active+clean; 277 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:40:43.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:42 smithi107 ceph-mon[90179]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 32/339 objects degraded (9.440%), 10 pgs degraded) 2024-04-22T23:40:43.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:42 smithi107 ceph-mon[90179]: Cluster is now healthy 2024-04-22T23:40:45.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:44 smithi157 ceph-mon[79403]: pgmap v102: 65 pgs: 65 active+clean; 277 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:40:45.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:44 smithi107 ceph-mon[90179]: pgmap v102: 65 pgs: 65 active+clean; 277 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:40:47.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:46 smithi157 ceph-mon[79403]: pgmap v103: 65 pgs: 65 active+clean; 277 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:40:47.202 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:46 smithi107 ceph-mon[90179]: pgmap v103: 65 pgs: 65 active+clean; 277 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:40:48.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:47 smithi157 ceph-mon[79403]: pgmap v104: 65 pgs: 65 active+clean; 277 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:40:48.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:47 smithi107 ceph-mon[90179]: pgmap v104: 65 pgs: 65 active+clean; 277 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:40:50.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:50 smithi107 ceph-mon[90179]: pgmap v105: 65 pgs: 65 active+clean; 277 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:40:51.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:50 smithi157 ceph-mon[79403]: pgmap v105: 65 pgs: 65 active+clean; 277 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:40:52.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:51 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:40:52.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:51 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:40:53.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:52 smithi157 ceph-mon[79403]: pgmap v106: 65 pgs: 65 active+clean; 277 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:40:53.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:52 smithi107 ceph-mon[90179]: pgmap v106: 65 pgs: 65 active+clean; 277 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:40:54.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.540+0000 7f7dd009a700 1 -- 172.21.15.107:0/2655655196 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7dc80fcfb0 msgr2=0x7f7dc80fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:54.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.540+0000 7f7dd009a700 1 --2- 172.21.15.107:0/2655655196 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7dc80fcfb0 0x7f7dc80fd420 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7db8004940 tx=0x7f7db8009e30 comp rx=0 tx=0).stop 2024-04-22T23:40:54.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.540+0000 7f7dd009a700 1 -- 172.21.15.107:0/2655655196 shutdown_connections 2024-04-22T23:40:54.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.540+0000 7f7dd009a700 1 --2- 172.21.15.107:0/2655655196 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7dc80fcfb0 0x7f7dc80fd420 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:54.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.540+0000 7f7dd009a700 1 --2- 172.21.15.107:0/2655655196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7dc80fbd00 0x7f7dc80fc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:54.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.540+0000 7f7dd009a700 1 -- 172.21.15.107:0/2655655196 >> 172.21.15.107:0/2655655196 conn(0x7f7dc80f74a0 msgr2=0x7f7dc80f98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:40:54.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.540+0000 7f7dd009a700 1 -- 172.21.15.107:0/2655655196 shutdown_connections 2024-04-22T23:40:54.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.540+0000 7f7dd009a700 1 -- 172.21.15.107:0/2655655196 wait complete. 2024-04-22T23:40:54.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.540+0000 7f7dd009a700 1 Processor -- start 2024-04-22T23:40:54.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.540+0000 7f7dd009a700 1 -- start start 2024-04-22T23:40:54.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.541+0000 7f7dd009a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7dc80fbd00 0x7f7dc8066d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:40:54.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.541+0000 7f7dd009a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7dc80fcfb0 0x7f7dc8112730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:40:54.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.541+0000 7f7dd009a700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc8112c70 con 0x7f7dc80fcfb0 2024-04-22T23:40:54.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.541+0000 7f7dd009a700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc8112db0 con 0x7f7dc80fbd00 2024-04-22T23:40:54.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.541+0000 7f7dcd635700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7dc80fcfb0 0x7f7dc8112730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:40:54.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.541+0000 7f7dcd635700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7dc80fcfb0 0x7f7dc8112730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54196/0 (socket says 172.21.15.107:54196) 2024-04-22T23:40:54.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.541+0000 7f7dcd635700 1 -- 172.21.15.107:0/1456883590 learned_addr learned my addr 172.21.15.107:0/1456883590 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:40:54.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.541+0000 7f7dcde36700 1 --2- 172.21.15.107:0/1456883590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7dc80fbd00 0x7f7dc8066d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:40:54.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.541+0000 7f7dcd635700 1 -- 172.21.15.107:0/1456883590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7dc80fbd00 msgr2=0x7f7dc8066d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:54.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.541+0000 7f7dcd635700 1 --2- 172.21.15.107:0/1456883590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7dc80fbd00 0x7f7dc8066d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:54.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.541+0000 7f7dcd635700 1 -- 172.21.15.107:0/1456883590 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db8005040 con 0x7f7dc80fcfb0 2024-04-22T23:40:54.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.541+0000 7f7dcd635700 1 --2- 172.21.15.107:0/1456883590 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7dc80fcfb0 0x7f7dc8112730 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7db800b3b0 tx=0x7f7db8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:40:54.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.541+0000 7f7dbe7fc700 1 -- 172.21.15.107:0/1456883590 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7db800e040 con 0x7f7dc80fcfb0 2024-04-22T23:40:54.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.541+0000 7f7dbe7fc700 1 -- 172.21.15.107:0/1456883590 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7db800bbf0 con 0x7f7dc80fcfb0 2024-04-22T23:40:54.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.541+0000 7f7dd009a700 1 -- 172.21.15.107:0/1456883590 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dc8112fb0 con 0x7f7dc80fcfb0 2024-04-22T23:40:54.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.541+0000 7f7dbe7fc700 1 -- 172.21.15.107:0/1456883590 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7db801c630 con 0x7f7dc80fcfb0 2024-04-22T23:40:54.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.542+0000 7f7dd009a700 1 -- 172.21.15.107:0/1456883590 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dc8113540 con 0x7f7dc80fcfb0 2024-04-22T23:40:54.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.543+0000 7f7dd009a700 1 -- 172.21.15.107:0/1456883590 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dc804c710 con 0x7f7dc80fcfb0 2024-04-22T23:40:54.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.543+0000 7f7dbe7fc700 1 -- 172.21.15.107:0/1456883590 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f7db800bd60 con 0x7f7dc80fcfb0 2024-04-22T23:40:54.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.543+0000 7f7dbe7fc700 1 --2- 172.21.15.107:0/1456883590 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f7db406ec00 0x7f7db40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:40:54.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.544+0000 7f7dbe7fc700 1 -- 172.21.15.107:0/1456883590 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f7db808f410 con 0x7f7dc80fcfb0 2024-04-22T23:40:54.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.544+0000 7f7dcde36700 1 --2- 172.21.15.107:0/1456883590 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f7db406ec00 0x7f7db40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:40:54.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.544+0000 7f7dcde36700 1 --2- 172.21.15.107:0/1456883590 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f7db406ec00 0x7f7db40710b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4004f40 tx=0x7f7dc400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:40:54.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.548+0000 7f7dbe7fc700 1 -- 172.21.15.107:0/1456883590 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f7db805bd70 con 0x7f7dc80fcfb0 2024-04-22T23:40:54.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.770+0000 7f7dd009a700 1 -- 172.21.15.107:0/1456883590 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7dc8113730 con 0x7f7db406ec00 2024-04-22T23:40:54.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.771+0000 7f7dbe7fc700 1 -- 172.21.15.107:0/1456883590 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f7dc8113730 con 0x7f7db406ec00 2024-04-22T23:40:54.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.773+0000 7f7dd009a700 1 -- 172.21.15.107:0/1456883590 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f7db406ec00 msgr2=0x7f7db40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:54.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.773+0000 7f7dd009a700 1 --2- 172.21.15.107:0/1456883590 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f7db406ec00 0x7f7db40710b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4004f40 tx=0x7f7dc400b040 comp rx=0 tx=0).stop 2024-04-22T23:40:54.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.773+0000 7f7dd009a700 1 -- 172.21.15.107:0/1456883590 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7dc80fcfb0 msgr2=0x7f7dc8112730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:54.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.773+0000 7f7dd009a700 1 --2- 172.21.15.107:0/1456883590 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7dc80fcfb0 0x7f7dc8112730 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7db800b3b0 tx=0x7f7db8009e30 comp rx=0 tx=0).stop 2024-04-22T23:40:54.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.773+0000 7f7dd009a700 1 -- 172.21.15.107:0/1456883590 shutdown_connections 2024-04-22T23:40:54.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.773+0000 7f7dd009a700 1 --2- 172.21.15.107:0/1456883590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7dc80fbd00 0x7f7dc8066d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:54.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.774+0000 7f7dd009a700 1 --2- 172.21.15.107:0/1456883590 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f7db406ec00 0x7f7db40710b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:54.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.774+0000 7f7dd009a700 1 --2- 172.21.15.107:0/1456883590 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7dc80fcfb0 0x7f7dc8112730 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:54.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.774+0000 7f7dd009a700 1 -- 172.21.15.107:0/1456883590 >> 172.21.15.107:0/1456883590 conn(0x7f7dc80f74a0 msgr2=0x7f7dc80f8b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:40:54.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.774+0000 7f7dd009a700 1 -- 172.21.15.107:0/1456883590 shutdown_connections 2024-04-22T23:40:54.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.774+0000 7f7dd009a700 1 -- 172.21.15.107:0/1456883590 wait complete. 2024-04-22T23:40:54.777 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-04-22T23:40:54.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.879+0000 7f764bbb2700 1 -- 172.21.15.107:0/2562904452 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76440fdff0 msgr2=0x7f76440fe460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:54.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.879+0000 7f764bbb2700 1 --2- 172.21.15.107:0/2562904452 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76440fdff0 0x7f76440fe460 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7634004940 tx=0x7f7634009e30 comp rx=0 tx=0).stop 2024-04-22T23:40:54.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.879+0000 7f764bbb2700 1 -- 172.21.15.107:0/2562904452 shutdown_connections 2024-04-22T23:40:54.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.879+0000 7f764bbb2700 1 --2- 172.21.15.107:0/2562904452 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76440fdff0 0x7f76440fe460 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:54.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.879+0000 7f764bbb2700 1 --2- 172.21.15.107:0/2562904452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76440fce80 0x7f76440fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:54.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.879+0000 7f764bbb2700 1 -- 172.21.15.107:0/2562904452 >> 172.21.15.107:0/2562904452 conn(0x7f76440f8580 msgr2=0x7f76440fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:40:54.873 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:54 smithi107 ceph-mon[90179]: pgmap v107: 65 pgs: 65 active+clean; 277 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:40:54.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.880+0000 7f764bbb2700 1 -- 172.21.15.107:0/2562904452 shutdown_connections 2024-04-22T23:40:54.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.880+0000 7f764bbb2700 1 -- 172.21.15.107:0/2562904452 wait complete. 2024-04-22T23:40:54.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.880+0000 7f764bbb2700 1 Processor -- start 2024-04-22T23:40:54.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.880+0000 7f764bbb2700 1 -- start start 2024-04-22T23:40:54.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.880+0000 7f764bbb2700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76440fce80 0x7f764410d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:40:54.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.880+0000 7f764bbb2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76440fdff0 0x7f764410de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:40:54.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.880+0000 7f764bbb2700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764410e460 con 0x7f76440fdff0 2024-04-22T23:40:54.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.880+0000 7f764bbb2700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764410e5d0 con 0x7f76440fce80 2024-04-22T23:40:54.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.880+0000 7f764914d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76440fdff0 0x7f764410de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:40:54.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.880+0000 7f764914d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76440fdff0 0x7f764410de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54220/0 (socket says 172.21.15.107:54220) 2024-04-22T23:40:54.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.881+0000 7f764914d700 1 -- 172.21.15.107:0/1353066224 learned_addr learned my addr 172.21.15.107:0/1353066224 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:40:54.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.881+0000 7f764914d700 1 -- 172.21.15.107:0/1353066224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76440fce80 msgr2=0x7f764410d920 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-22T23:40:54.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.881+0000 7f764914d700 1 --2- 172.21.15.107:0/1353066224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76440fce80 0x7f764410d920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:54.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.881+0000 7f764914d700 1 -- 172.21.15.107:0/1353066224 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7634005040 con 0x7f76440fdff0 2024-04-22T23:40:54.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.881+0000 7f764914d700 1 --2- 172.21.15.107:0/1353066224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76440fdff0 0x7f764410de60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7634009150 tx=0x7f7634009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:40:54.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.881+0000 7f763a7fc700 1 -- 172.21.15.107:0/1353066224 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f763400e040 con 0x7f76440fdff0 2024-04-22T23:40:54.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.881+0000 7f764bbb2700 1 -- 172.21.15.107:0/1353066224 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7644112fd0 con 0x7f76440fdff0 2024-04-22T23:40:54.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.881+0000 7f763a7fc700 1 -- 172.21.15.107:0/1353066224 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f763400bcc0 con 0x7f76440fdff0 2024-04-22T23:40:54.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.881+0000 7f763a7fc700 1 -- 172.21.15.107:0/1353066224 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f763401c630 con 0x7f76440fdff0 2024-04-22T23:40:54.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.881+0000 7f764bbb2700 1 -- 172.21.15.107:0/1353066224 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7644113560 con 0x7f76440fdff0 2024-04-22T23:40:54.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.882+0000 7f764bbb2700 1 -- 172.21.15.107:0/1353066224 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f764404c710 con 0x7f76440fdff0 2024-04-22T23:40:54.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.883+0000 7f763a7fc700 1 -- 172.21.15.107:0/1353066224 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f7634013420 con 0x7f76440fdff0 2024-04-22T23:40:54.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.883+0000 7f763a7fc700 1 --2- 172.21.15.107:0/1353066224 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f763006ec00 0x7f76300710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:40:54.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.883+0000 7f763a7fc700 1 -- 172.21.15.107:0/1353066224 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f763408f530 con 0x7f76440fdff0 2024-04-22T23:40:54.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.883+0000 7f764994e700 1 --2- 172.21.15.107:0/1353066224 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f763006ec00 0x7f76300710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:40:54.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.884+0000 7f764994e700 1 --2- 172.21.15.107:0/1353066224 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f763006ec00 0x7f76300710b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7640004ad0 tx=0x7f7640007d40 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:40:54.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:54.888+0000 7f763a7fc700 1 -- 172.21.15.107:0/1353066224 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f763405be90 con 0x7f76440fdff0 2024-04-22T23:40:55.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.107+0000 7f764bbb2700 1 -- 172.21.15.107:0/1353066224 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7644102310 con 0x7f763006ec00 2024-04-22T23:40:55.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.108+0000 7f763a7fc700 1 -- 172.21.15.107:0/1353066224 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f7644102310 con 0x7f763006ec00 2024-04-22T23:40:55.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.110+0000 7f764bbb2700 1 -- 172.21.15.107:0/1353066224 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f763006ec00 msgr2=0x7f76300710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:55.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.110+0000 7f764bbb2700 1 --2- 172.21.15.107:0/1353066224 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f763006ec00 0x7f76300710b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7640004ad0 tx=0x7f7640007d40 comp rx=0 tx=0).stop 2024-04-22T23:40:55.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.110+0000 7f764bbb2700 1 -- 172.21.15.107:0/1353066224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76440fdff0 msgr2=0x7f764410de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:55.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.110+0000 7f764bbb2700 1 --2- 172.21.15.107:0/1353066224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76440fdff0 0x7f764410de60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7634009150 tx=0x7f7634009e30 comp rx=0 tx=0).stop 2024-04-22T23:40:55.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.111+0000 7f764bbb2700 1 -- 172.21.15.107:0/1353066224 shutdown_connections 2024-04-22T23:40:55.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.111+0000 7f764bbb2700 1 --2- 172.21.15.107:0/1353066224 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f76440fce80 0x7f764410d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:55.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.111+0000 7f764bbb2700 1 --2- 172.21.15.107:0/1353066224 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f763006ec00 0x7f76300710b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:55.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.111+0000 7f764bbb2700 1 --2- 172.21.15.107:0/1353066224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76440fdff0 0x7f764410de60 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:55.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.111+0000 7f764bbb2700 1 -- 172.21.15.107:0/1353066224 >> 172.21.15.107:0/1353066224 conn(0x7f76440f8580 msgr2=0x7f76440f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:40:55.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.111+0000 7f764bbb2700 1 -- 172.21.15.107:0/1353066224 shutdown_connections 2024-04-22T23:40:55.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.111+0000 7f764bbb2700 1 -- 172.21.15.107:0/1353066224 wait complete. 2024-04-22T23:40:55.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:54 smithi157 ceph-mon[79403]: pgmap v107: 65 pgs: 65 active+clean; 277 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:40:55.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.211+0000 7f6f1cb26700 1 -- 172.21.15.107:0/202323044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f18066f40 msgr2=0x7f6f180ffa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:55.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.211+0000 7f6f1cb26700 1 --2- 172.21.15.107:0/202323044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f18066f40 0x7f6f180ffa50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6f08004940 tx=0x7f6f08009e30 comp rx=0 tx=0).stop 2024-04-22T23:40:55.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.211+0000 7f6f1cb26700 1 -- 172.21.15.107:0/202323044 shutdown_connections 2024-04-22T23:40:55.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.211+0000 7f6f1cb26700 1 --2- 172.21.15.107:0/202323044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f180fff90 0x7f6f18102370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:55.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.211+0000 7f6f1cb26700 1 --2- 172.21.15.107:0/202323044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f18066f40 0x7f6f180ffa50 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:55.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.211+0000 7f6f1cb26700 1 -- 172.21.15.107:0/202323044 >> 172.21.15.107:0/202323044 conn(0x7f6f180f7480 msgr2=0x7f6f180f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:40:55.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.211+0000 7f6f1cb26700 1 -- 172.21.15.107:0/202323044 shutdown_connections 2024-04-22T23:40:55.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.211+0000 7f6f1cb26700 1 -- 172.21.15.107:0/202323044 wait complete. 2024-04-22T23:40:55.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.211+0000 7f6f1cb26700 1 Processor -- start 2024-04-22T23:40:55.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.211+0000 7f6f1cb26700 1 -- start start 2024-04-22T23:40:55.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.212+0000 7f6f1cb26700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f18066f40 0x7f6f1810d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:40:55.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.212+0000 7f6f1cb26700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f180fff90 0x7f6f1810de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:40:55.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.212+0000 7f6f1cb26700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1810e490 con 0x7f6f180fff90 2024-04-22T23:40:55.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.212+0000 7f6f1cb26700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1810e600 con 0x7f6f18066f40 2024-04-22T23:40:55.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.212+0000 7f6f15d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f180fff90 0x7f6f1810de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:40:55.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.212+0000 7f6f15d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f180fff90 0x7f6f1810de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54238/0 (socket says 172.21.15.107:54238) 2024-04-22T23:40:55.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.212+0000 7f6f15d9b700 1 -- 172.21.15.107:0/2348261705 learned_addr learned my addr 172.21.15.107:0/2348261705 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:40:55.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.212+0000 7f6f1659c700 1 --2- 172.21.15.107:0/2348261705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f18066f40 0x7f6f1810d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:40:55.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.212+0000 7f6f15d9b700 1 -- 172.21.15.107:0/2348261705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f18066f40 msgr2=0x7f6f1810d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:55.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.212+0000 7f6f15d9b700 1 --2- 172.21.15.107:0/2348261705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f18066f40 0x7f6f1810d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:55.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.212+0000 7f6f15d9b700 1 -- 172.21.15.107:0/2348261705 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f08005040 con 0x7f6f180fff90 2024-04-22T23:40:55.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.212+0000 7f6f15d9b700 1 --2- 172.21.15.107:0/2348261705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f180fff90 0x7f6f1810de90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c002a50 tx=0x7f6f0c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:40:55.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.212+0000 7f6f06ffd700 1 -- 172.21.15.107:0/2348261705 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f0c007cc0 con 0x7f6f180fff90 2024-04-22T23:40:55.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.212+0000 7f6f1cb26700 1 -- 172.21.15.107:0/2348261705 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f18113060 con 0x7f6f180fff90 2024-04-22T23:40:55.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.212+0000 7f6f06ffd700 1 -- 172.21.15.107:0/2348261705 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6f0c00f040 con 0x7f6f180fff90 2024-04-22T23:40:55.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.212+0000 7f6f06ffd700 1 -- 172.21.15.107:0/2348261705 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f0c009640 con 0x7f6f180fff90 2024-04-22T23:40:55.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.213+0000 7f6f1cb26700 1 -- 172.21.15.107:0/2348261705 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f18113620 con 0x7f6f180fff90 2024-04-22T23:40:55.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.214+0000 7f6f1cb26700 1 -- 172.21.15.107:0/2348261705 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f1804c710 con 0x7f6f180fff90 2024-04-22T23:40:55.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.214+0000 7f6f06ffd700 1 -- 172.21.15.107:0/2348261705 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f6f0c0097a0 con 0x7f6f180fff90 2024-04-22T23:40:55.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.214+0000 7f6f06ffd700 1 --2- 172.21.15.107:0/2348261705 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f6f0006ec00 0x7f6f000710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:40:55.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.214+0000 7f6f06ffd700 1 -- 172.21.15.107:0/2348261705 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f6f0c08e5b0 con 0x7f6f180fff90 2024-04-22T23:40:55.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.214+0000 7f6f1659c700 1 --2- 172.21.15.107:0/2348261705 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f6f0006ec00 0x7f6f000710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:40:55.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.215+0000 7f6f1659c700 1 --2- 172.21.15.107:0/2348261705 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f6f0006ec00 0x7f6f000710b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6f08009fd0 tx=0x7f6f08009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:40:55.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.219+0000 7f6f06ffd700 1 -- 172.21.15.107:0/2348261705 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f6f0c05af10 con 0x7f6f180fff90 2024-04-22T23:40:55.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.429+0000 7f6f1cb26700 1 -- 172.21.15.107:0/2348261705 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6f180f8f70 con 0x7f6f0006ec00 2024-04-22T23:40:55.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.439+0000 7f6f06ffd700 1 -- 172.21.15.107:0/2348261705 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f6f180f8f70 con 0x7f6f0006ec00 2024-04-22T23:40:55.432 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-22T23:40:55.432 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (4m) 19s ago 13m 25.6M - 0.25.0 c8568f914cd2 ec1a5c6260de 2024-04-22T23:40:55.432 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (2m) 19s ago 14m 6656k - 19.0.0-3252-g43be0201 a0434699c1d7 e074fd2cbb9c 2024-04-22T23:40:55.432 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi157 smithi157 running (119s) 115s ago 13m 6651k - 19.0.0-3252-g43be0201 a0434699c1d7 abc2649dee22 2024-04-22T23:40:55.432 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (4m) 19s ago 13m 79.9M - 9.4.7 954c08fa6188 31449ff442d2 2024-04-22T23:40:55.432 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.dfvvkw smithi107 running (8m) 19s ago 8m 13.7M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 08d3aa8d717d 2024-04-22T23:40:55.432 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.ktuxjl smithi107 running (8m) 19s ago 8m 124M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 762f27fe0f30 2024-04-22T23:40:55.432 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi157.rcaris smithi157 running (8m) 115s ago 8m 129M - 17.2.7-884-g70d0cb4b cafd8ddac8fa d23366ac085d 2024-04-22T23:40:55.433 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi157.ufywse smithi157 running (8m) 115s ago 8m 14.1M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 8b00d61eb355 2024-04-22T23:40:55.433 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.uipxvm smithi107 *:8443,9283,8765 running (3m) 19s ago 15m 502M - 19.0.0-3252-g43be0201 a0434699c1d7 d20f05dfe4f2 2024-04-22T23:40:55.433 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi157.tuylis smithi157 *:8443,9283,8765 running (3m) 115s ago 13m 449M - 19.0.0-3252-g43be0201 a0434699c1d7 957e598c9001 2024-04-22T23:40:55.433 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (3m) 19s ago 15m 49.4M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 bc72bbbe2c39 2024-04-22T23:40:55.433 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi157 smithi157 running (2m) 115s ago 12m 31.7M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 ea88de8be4fe 2024-04-22T23:40:55.433 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (4m) 19s ago 14m 16.2M - 1.5.0 0da6a335fe13 5205cfd9bd51 2024-04-22T23:40:55.433 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi157 smithi157 *:9100 running (4m) 115s ago 13m 17.0M - 1.5.0 0da6a335fe13 168ca191bb8c 2024-04-22T23:40:55.433 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (99s) 19s ago 12m 94.2M 1557M 19.0.0-3252-g43be0201 a0434699c1d7 fd0ba8ef0c27 2024-04-22T23:40:55.433 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (62s) 19s ago 11m 91.2M 1557M 19.0.0-3252-g43be0201 a0434699c1d7 77e33b6122e0 2024-04-22T23:40:55.433 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (25s) 19s ago 11m 15.0M 1557M 19.0.0-3252-g43be0201 a0434699c1d7 c307bd8cdcfc 2024-04-22T23:40:55.433 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi157 running (10m) 115s ago 10m 266M 2581M 17.2.7-884-g70d0cb4b cafd8ddac8fa c9fb2d1a13dd 2024-04-22T23:40:55.433 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi157 running (10m) 115s ago 10m 227M 2581M 17.2.7-884-g70d0cb4b cafd8ddac8fa 24a16e19951f 2024-04-22T23:40:55.433 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi157 running (9m) 115s ago 9m 226M 2581M 17.2.7-884-g70d0cb4b cafd8ddac8fa 4c5b7a7ef8bd 2024-04-22T23:40:55.433 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (3m) 19s ago 13m 58.8M - 2.43.0 a07b618ecd1d 99a4ea8594da 2024-04-22T23:40:55.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.441+0000 7f6f1cb26700 1 -- 172.21.15.107:0/2348261705 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f6f0006ec00 msgr2=0x7f6f000710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:55.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.441+0000 7f6f1cb26700 1 --2- 172.21.15.107:0/2348261705 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f6f0006ec00 0x7f6f000710b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6f08009fd0 tx=0x7f6f08009e30 comp rx=0 tx=0).stop 2024-04-22T23:40:55.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.441+0000 7f6f1cb26700 1 -- 172.21.15.107:0/2348261705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f180fff90 msgr2=0x7f6f1810de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:55.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.441+0000 7f6f1cb26700 1 --2- 172.21.15.107:0/2348261705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f180fff90 0x7f6f1810de90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c002a50 tx=0x7f6f0c0046b0 comp rx=0 tx=0).stop 2024-04-22T23:40:55.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.441+0000 7f6f1cb26700 1 -- 172.21.15.107:0/2348261705 shutdown_connections 2024-04-22T23:40:55.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.441+0000 7f6f1cb26700 1 --2- 172.21.15.107:0/2348261705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f18066f40 0x7f6f1810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:55.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.441+0000 7f6f1cb26700 1 --2- 172.21.15.107:0/2348261705 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f6f0006ec00 0x7f6f000710b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:55.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.441+0000 7f6f1cb26700 1 --2- 172.21.15.107:0/2348261705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f180fff90 0x7f6f1810de90 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:55.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.441+0000 7f6f1cb26700 1 -- 172.21.15.107:0/2348261705 >> 172.21.15.107:0/2348261705 conn(0x7f6f180f7480 msgr2=0x7f6f180f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:40:55.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.441+0000 7f6f1cb26700 1 -- 172.21.15.107:0/2348261705 shutdown_connections 2024-04-22T23:40:55.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.441+0000 7f6f1cb26700 1 -- 172.21.15.107:0/2348261705 wait complete. 2024-04-22T23:40:55.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.540+0000 7f18fccf2700 1 -- 172.21.15.107:0/3015510338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18f80fce80 msgr2=0x7f18f80fd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:55.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.540+0000 7f18fccf2700 1 --2- 172.21.15.107:0/3015510338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18f80fce80 0x7f18f80fd290 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f18ec004940 tx=0x7f18ec009e30 comp rx=0 tx=0).stop 2024-04-22T23:40:55.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.540+0000 7f18fccf2700 1 -- 172.21.15.107:0/3015510338 shutdown_connections 2024-04-22T23:40:55.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.540+0000 7f18fccf2700 1 --2- 172.21.15.107:0/3015510338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18f80fdff0 0x7f18f80fe460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:55.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.540+0000 7f18fccf2700 1 --2- 172.21.15.107:0/3015510338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18f80fce80 0x7f18f80fd290 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:55.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.540+0000 7f18fccf2700 1 -- 172.21.15.107:0/3015510338 >> 172.21.15.107:0/3015510338 conn(0x7f18f80f8580 msgr2=0x7f18f80fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:40:55.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.540+0000 7f18fccf2700 1 -- 172.21.15.107:0/3015510338 shutdown_connections 2024-04-22T23:40:55.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.540+0000 7f18fccf2700 1 -- 172.21.15.107:0/3015510338 wait complete. 2024-04-22T23:40:55.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.541+0000 7f18fccf2700 1 Processor -- start 2024-04-22T23:40:55.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.541+0000 7f18fccf2700 1 -- start start 2024-04-22T23:40:55.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.541+0000 7f18fccf2700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18f80fce80 0x7f18f810d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:40:55.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.541+0000 7f18fccf2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18f80fdff0 0x7f18f810dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:40:55.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.541+0000 7f18fccf2700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f810e4c0 con 0x7f18f80fdff0 2024-04-22T23:40:55.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.541+0000 7f18fccf2700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f810e630 con 0x7f18f80fce80 2024-04-22T23:40:55.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.541+0000 7f18f5d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18f80fdff0 0x7f18f810dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:40:55.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.541+0000 7f18f5d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18f80fdff0 0x7f18f810dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54258/0 (socket says 172.21.15.107:54258) 2024-04-22T23:40:55.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.541+0000 7f18f5d9b700 1 -- 172.21.15.107:0/987154603 learned_addr learned my addr 172.21.15.107:0/987154603 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:40:55.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.541+0000 7f18f659c700 1 --2- 172.21.15.107:0/987154603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18f80fce80 0x7f18f810d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:40:55.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.541+0000 7f18f5d9b700 1 -- 172.21.15.107:0/987154603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18f80fce80 msgr2=0x7f18f810d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:55.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.541+0000 7f18f5d9b700 1 --2- 172.21.15.107:0/987154603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18f80fce80 0x7f18f810d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:55.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.541+0000 7f18f5d9b700 1 -- 172.21.15.107:0/987154603 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18ec005040 con 0x7f18f80fdff0 2024-04-22T23:40:55.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.542+0000 7f18f5d9b700 1 --2- 172.21.15.107:0/987154603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18f80fdff0 0x7f18f810dec0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f18e8002a50 tx=0x7f18e80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:40:55.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.542+0000 7f18e6ffd700 1 -- 172.21.15.107:0/987154603 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18e8007cc0 con 0x7f18f80fdff0 2024-04-22T23:40:55.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.542+0000 7f18e6ffd700 1 -- 172.21.15.107:0/987154603 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f18e800f040 con 0x7f18f80fdff0 2024-04-22T23:40:55.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.542+0000 7f18e6ffd700 1 -- 172.21.15.107:0/987154603 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18e800da70 con 0x7f18f80fdff0 2024-04-22T23:40:55.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.542+0000 7f18fccf2700 1 -- 172.21.15.107:0/987154603 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18f8113090 con 0x7f18f80fdff0 2024-04-22T23:40:55.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.542+0000 7f18fccf2700 1 -- 172.21.15.107:0/987154603 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18f8113650 con 0x7f18f80fdff0 2024-04-22T23:40:55.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.543+0000 7f18fccf2700 1 -- 172.21.15.107:0/987154603 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18f804c710 con 0x7f18f80fdff0 2024-04-22T23:40:55.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.543+0000 7f18e6ffd700 1 -- 172.21.15.107:0/987154603 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f18e800dc10 con 0x7f18f80fdff0 2024-04-22T23:40:55.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.544+0000 7f18e6ffd700 1 --2- 172.21.15.107:0/987154603 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f18e006ec00 0x7f18e00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:40:55.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.544+0000 7f18f659c700 1 --2- 172.21.15.107:0/987154603 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f18e006ec00 0x7f18e00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:40:55.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.544+0000 7f18e6ffd700 1 -- 172.21.15.107:0/987154603 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f18e808def0 con 0x7f18f80fdff0 2024-04-22T23:40:55.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.544+0000 7f18f659c700 1 --2- 172.21.15.107:0/987154603 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f18e006ec00 0x7f18e00710b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f18ec00b970 tx=0x7f18ec009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:40:55.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.548+0000 7f18e6ffd700 1 -- 172.21.15.107:0/987154603 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f18e805a7d0 con 0x7f18f80fdff0 2024-04-22T23:40:55.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.843+0000 7f18fccf2700 1 -- 172.21.15.107:0/987154603 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f18f8113a50 con 0x7f18f80fdff0 2024-04-22T23:40:55.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:55 smithi107 ceph-mon[90179]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:40:55.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:55 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-22T23:40:55.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:55 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:40:55.837 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:55 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:40:55.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.844+0000 7f18e6ffd700 1 -- 172.21.15.107:0/987154603 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f18e805a360 con 0x7f18f80fdff0 2024-04-22T23:40:55.837 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-22T23:40:55.837 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-04-22T23:40:55.838 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:40:55.838 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:40:55.838 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-04-22T23:40:55.838 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:40:55.838 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:40:55.838 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-04-22T23:40:55.838 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 3, 2024-04-22T23:40:55.838 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 3 2024-04-22T23:40:55.838 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:40:55.838 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-04-22T23:40:55.838 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 4 2024-04-22T23:40:55.838 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:40:55.838 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-04-22T23:40:55.838 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 7, 2024-04-22T23:40:55.839 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 7 2024-04-22T23:40:55.839 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T23:40:55.839 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-22T23:40:55.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.846+0000 7f18fccf2700 1 -- 172.21.15.107:0/987154603 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f18e006ec00 msgr2=0x7f18e00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:55.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.846+0000 7f18fccf2700 1 --2- 172.21.15.107:0/987154603 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f18e006ec00 0x7f18e00710b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f18ec00b970 tx=0x7f18ec009e30 comp rx=0 tx=0).stop 2024-04-22T23:40:55.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.846+0000 7f18fccf2700 1 -- 172.21.15.107:0/987154603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18f80fdff0 msgr2=0x7f18f810dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:55.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.846+0000 7f18fccf2700 1 --2- 172.21.15.107:0/987154603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18f80fdff0 0x7f18f810dec0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f18e8002a50 tx=0x7f18e80046b0 comp rx=0 tx=0).stop 2024-04-22T23:40:55.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.846+0000 7f18fccf2700 1 -- 172.21.15.107:0/987154603 shutdown_connections 2024-04-22T23:40:55.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.846+0000 7f18fccf2700 1 --2- 172.21.15.107:0/987154603 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18f80fce80 0x7f18f810d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:55.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.846+0000 7f18fccf2700 1 --2- 172.21.15.107:0/987154603 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f18e006ec00 0x7f18e00710b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:55.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.846+0000 7f18fccf2700 1 --2- 172.21.15.107:0/987154603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18f80fdff0 0x7f18f810dec0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:55.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.846+0000 7f18fccf2700 1 -- 172.21.15.107:0/987154603 >> 172.21.15.107:0/987154603 conn(0x7f18f80f8580 msgr2=0x7f18f80f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:40:55.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.846+0000 7f18fccf2700 1 -- 172.21.15.107:0/987154603 shutdown_connections 2024-04-22T23:40:55.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.846+0000 7f18fccf2700 1 -- 172.21.15.107:0/987154603 wait complete. 2024-04-22T23:40:55.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.947+0000 7f79c8715700 1 -- 172.21.15.107:0/3808898806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79c00fcad0 msgr2=0x7f79c00faee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:55.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.947+0000 7f79c8715700 1 --2- 172.21.15.107:0/3808898806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79c00fcad0 0x7f79c00faee0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f79b0004940 tx=0x7f79b0009e30 comp rx=0 tx=0).stop 2024-04-22T23:40:55.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.947+0000 7f79c8715700 1 -- 172.21.15.107:0/3808898806 shutdown_connections 2024-04-22T23:40:55.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.947+0000 7f79c8715700 1 --2- 172.21.15.107:0/3808898806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79c00fcad0 0x7f79c00faee0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:55.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.947+0000 7f79c8715700 1 --2- 172.21.15.107:0/3808898806 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79c00fc180 0x7f79c00fc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:55.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.947+0000 7f79c8715700 1 -- 172.21.15.107:0/3808898806 >> 172.21.15.107:0/3808898806 conn(0x7f79c00f6bf0 msgr2=0x7f79c00f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:40:55.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.947+0000 7f79c8715700 1 -- 172.21.15.107:0/3808898806 shutdown_connections 2024-04-22T23:40:55.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.947+0000 7f79c8715700 1 -- 172.21.15.107:0/3808898806 wait complete. 2024-04-22T23:40:55.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.947+0000 7f79c8715700 1 Processor -- start 2024-04-22T23:40:55.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.948+0000 7f79c8715700 1 -- start start 2024-04-22T23:40:55.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.948+0000 7f79c8715700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79c00fc180 0x7f79c0113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:40:55.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.948+0000 7f79c8715700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79c00fcad0 0x7f79c0114400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:40:55.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.948+0000 7f79c8715700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c0114a00 con 0x7f79c00fcad0 2024-04-22T23:40:55.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.948+0000 7f79c8715700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c0114b70 con 0x7f79c00fc180 2024-04-22T23:40:55.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.948+0000 7f79c5cb0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79c00fcad0 0x7f79c0114400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:40:55.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.948+0000 7f79c5cb0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79c00fcad0 0x7f79c0114400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54274/0 (socket says 172.21.15.107:54274) 2024-04-22T23:40:55.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.948+0000 7f79c5cb0700 1 -- 172.21.15.107:0/2634254001 learned_addr learned my addr 172.21.15.107:0/2634254001 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:40:55.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.948+0000 7f79c64b1700 1 --2- 172.21.15.107:0/2634254001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79c00fc180 0x7f79c0113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:40:55.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.948+0000 7f79c5cb0700 1 -- 172.21.15.107:0/2634254001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79c00fc180 msgr2=0x7f79c0113ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:55.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.948+0000 7f79c5cb0700 1 --2- 172.21.15.107:0/2634254001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79c00fc180 0x7f79c0113ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:55.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.948+0000 7f79c5cb0700 1 -- 172.21.15.107:0/2634254001 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b0005040 con 0x7f79c00fcad0 2024-04-22T23:40:55.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.948+0000 7f79c5cb0700 1 --2- 172.21.15.107:0/2634254001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79c00fcad0 0x7f79c0114400 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f79b0013870 tx=0x7f79b0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:40:55.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.948+0000 7f79b6ffd700 1 -- 172.21.15.107:0/2634254001 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79b000e070 con 0x7f79c00fcad0 2024-04-22T23:40:55.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.948+0000 7f79b6ffd700 1 -- 172.21.15.107:0/2634254001 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f79b0013de0 con 0x7f79c00fcad0 2024-04-22T23:40:55.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.948+0000 7f79c8715700 1 -- 172.21.15.107:0/2634254001 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79c0119570 con 0x7f79c00fcad0 2024-04-22T23:40:55.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.948+0000 7f79b6ffd700 1 -- 172.21.15.107:0/2634254001 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79b001cea0 con 0x7f79c00fcad0 2024-04-22T23:40:55.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.948+0000 7f79c8715700 1 -- 172.21.15.107:0/2634254001 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79c0119a50 con 0x7f79c00fcad0 2024-04-22T23:40:55.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.949+0000 7f79c8715700 1 -- 172.21.15.107:0/2634254001 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79c004c710 con 0x7f79c00fcad0 2024-04-22T23:40:55.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.950+0000 7f79b6ffd700 1 -- 172.21.15.107:0/2634254001 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f79b001c690 con 0x7f79c00fcad0 2024-04-22T23:40:55.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.950+0000 7f79b6ffd700 1 --2- 172.21.15.107:0/2634254001 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f79ac06ec00 0x7f79ac0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:40:55.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.950+0000 7f79b6ffd700 1 -- 172.21.15.107:0/2634254001 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f79b008ff90 con 0x7f79c00fcad0 2024-04-22T23:40:55.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.950+0000 7f79c64b1700 1 --2- 172.21.15.107:0/2634254001 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f79ac06ec00 0x7f79ac0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:40:55.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.951+0000 7f79c64b1700 1 --2- 172.21.15.107:0/2634254001 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f79ac06ec00 0x7f79ac0710b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f79bc009b80 tx=0x7f79bc00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:40:55.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:55.955+0000 7f79b6ffd700 1 -- 172.21.15.107:0/2634254001 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f79b00c5150 con 0x7f79c00fcad0 2024-04-22T23:40:56.055 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:55 smithi157 ceph-mon[79403]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:40:56.056 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:55 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-22T23:40:56.056 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:55 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:40:56.056 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:55 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:40:56.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.196+0000 7f79c8715700 1 -- 172.21.15.107:0/2634254001 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f79c0119e70 con 0x7f79c00fcad0 2024-04-22T23:40:56.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.196+0000 7f79b6ffd700 1 -- 172.21.15.107:0/2634254001 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1909 (secure 0 0 0) 0x7f79b005c8f0 con 0x7f79c00fcad0 2024-04-22T23:40:56.189 INFO:teuthology.orchestra.run.smithi107.stdout:e21 2024-04-22T23:40:56.189 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T23:40:56.189 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:40:56.190 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-04-22T23:40:56.190 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:40:56.190 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-04-22T23:40:56.190 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-04-22T23:40:56.190 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 21 2024-04-22T23:40:56.190 INFO:teuthology.orchestra.run.smithi107.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-22T23:40:56.190 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-04-22T23:32:20.146130+0000 2024-04-22T23:40:56.190 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-04-22T23:39:06.006385+0000 2024-04-22T23:40:56.190 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-04-22T23:40:56.190 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-04-22T23:40:56.190 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-04-22T23:40:56.190 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-04-22T23:40:56.190 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-04-22T23:40:56.191 INFO:teuthology.orchestra.run.smithi107.stdout:max_xattr_size 65536 2024-04-22T23:40:56.191 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-04-22T23:40:56.191 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-04-22T23:40:56.191 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 46 2024-04-22T23:40:56.191 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:40:56.191 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 2 2024-04-22T23:40:56.191 INFO:teuthology.orchestra.run.smithi107.stdout:in 0,1 2024-04-22T23:40:56.191 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=14468,1=24265} 2024-04-22T23:40:56.191 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-04-22T23:40:56.191 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-04-22T23:40:56.191 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-04-22T23:40:56.191 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-04-22T23:40:56.191 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-04-22T23:40:56.191 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data disabled 2024-04-22T23:40:56.191 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-04-22T23:40:56.192 INFO:teuthology.orchestra.run.smithi107.stdout:bal_rank_mask -1 2024-04-22T23:40:56.192 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-04-22T23:40:56.192 INFO:teuthology.orchestra.run.smithi107.stdout:qdb_cluster leader: 14468 members: 24265,14468 2024-04-22T23:40:56.192 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.ktuxjl{0:14468} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.107:6828/3335313253,v1:172.21.15.107:6829/3335313253] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:40:56.192 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi157.rcaris{1:24265} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.157:6824/2779251893,v1:172.21.15.157:6825/2779251893] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:40:56.192 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:40:56.192 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:40:56.192 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-04-22T23:40:56.192 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:40:56.192 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.dfvvkw{-1:14484} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/3548717247,v1:172.21.15.107:6827/3548717247] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:40:56.192 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi157.ufywse{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.157:6826/4072444199,v1:172.21.15.157:6827/4072444199] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:40:56.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.199+0000 7f79c8715700 1 -- 172.21.15.107:0/2634254001 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f79ac06ec00 msgr2=0x7f79ac0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:56.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.199+0000 7f79c8715700 1 --2- 172.21.15.107:0/2634254001 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f79ac06ec00 0x7f79ac0710b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f79bc009b80 tx=0x7f79bc00b040 comp rx=0 tx=0).stop 2024-04-22T23:40:56.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.199+0000 7f79c8715700 1 -- 172.21.15.107:0/2634254001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79c00fcad0 msgr2=0x7f79c0114400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:56.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.199+0000 7f79c8715700 1 --2- 172.21.15.107:0/2634254001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79c00fcad0 0x7f79c0114400 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f79b0013870 tx=0x7f79b0009e30 comp rx=0 tx=0).stop 2024-04-22T23:40:56.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.199+0000 7f79c8715700 1 -- 172.21.15.107:0/2634254001 shutdown_connections 2024-04-22T23:40:56.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.199+0000 7f79c8715700 1 --2- 172.21.15.107:0/2634254001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79c00fc180 0x7f79c0113ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:56.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.199+0000 7f79c8715700 1 --2- 172.21.15.107:0/2634254001 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f79ac06ec00 0x7f79ac0710b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:56.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.199+0000 7f79c8715700 1 --2- 172.21.15.107:0/2634254001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79c00fcad0 0x7f79c0114400 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:56.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.199+0000 7f79c8715700 1 -- 172.21.15.107:0/2634254001 >> 172.21.15.107:0/2634254001 conn(0x7f79c00f6bf0 msgr2=0x7f79c00f8280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:40:56.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.199+0000 7f79c8715700 1 -- 172.21.15.107:0/2634254001 shutdown_connections 2024-04-22T23:40:56.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.199+0000 7f79c8715700 1 -- 172.21.15.107:0/2634254001 wait complete. 2024-04-22T23:40:56.194 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 21 2024-04-22T23:40:56.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.301+0000 7f8e363d0700 1 -- 172.21.15.107:0/1426701894 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e300fbd00 msgr2=0x7f8e300fc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:56.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.301+0000 7f8e363d0700 1 --2- 172.21.15.107:0/1426701894 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e300fbd00 0x7f8e300fc110 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8e20004940 tx=0x7f8e20009e30 comp rx=0 tx=0).stop 2024-04-22T23:40:56.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.301+0000 7f8e363d0700 1 -- 172.21.15.107:0/1426701894 shutdown_connections 2024-04-22T23:40:56.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.301+0000 7f8e363d0700 1 --2- 172.21.15.107:0/1426701894 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e300fcfb0 0x7f8e300fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:56.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.301+0000 7f8e363d0700 1 --2- 172.21.15.107:0/1426701894 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e300fbd00 0x7f8e300fc110 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:56.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.301+0000 7f8e363d0700 1 -- 172.21.15.107:0/1426701894 >> 172.21.15.107:0/1426701894 conn(0x7f8e300f7480 msgr2=0x7f8e300f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:40:56.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.301+0000 7f8e363d0700 1 -- 172.21.15.107:0/1426701894 shutdown_connections 2024-04-22T23:40:56.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.301+0000 7f8e363d0700 1 -- 172.21.15.107:0/1426701894 wait complete. 2024-04-22T23:40:56.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.302+0000 7f8e363d0700 1 Processor -- start 2024-04-22T23:40:56.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.302+0000 7f8e363d0700 1 -- start start 2024-04-22T23:40:56.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.302+0000 7f8e363d0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e300fbd00 0x7f8e3010c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:40:56.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.302+0000 7f8e363d0700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e300fcfb0 0x7f8e3010cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:40:56.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.302+0000 7f8e363d0700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3010d370 con 0x7f8e300fbd00 2024-04-22T23:40:56.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.302+0000 7f8e363d0700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3010d4e0 con 0x7f8e300fcfb0 2024-04-22T23:40:56.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.302+0000 7f8e2ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e300fbd00 0x7f8e3010c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:40:56.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.302+0000 7f8e2ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e300fbd00 0x7f8e3010c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54294/0 (socket says 172.21.15.107:54294) 2024-04-22T23:40:56.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.302+0000 7f8e2ffff700 1 -- 172.21.15.107:0/2201854031 learned_addr learned my addr 172.21.15.107:0/2201854031 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:40:56.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.303+0000 7f8e2f7fe700 1 --2- 172.21.15.107:0/2201854031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e300fcfb0 0x7f8e3010cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:40:56.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.303+0000 7f8e2ffff700 1 -- 172.21.15.107:0/2201854031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e300fcfb0 msgr2=0x7f8e3010cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:56.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.303+0000 7f8e2ffff700 1 --2- 172.21.15.107:0/2201854031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e300fcfb0 0x7f8e3010cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:56.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.303+0000 7f8e2ffff700 1 -- 172.21.15.107:0/2201854031 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e20005040 con 0x7f8e300fbd00 2024-04-22T23:40:56.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.303+0000 7f8e2ffff700 1 --2- 172.21.15.107:0/2201854031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e300fbd00 0x7f8e3010c830 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8e20009380 tx=0x7f8e20009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:40:56.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.303+0000 7f8e2cff9700 1 -- 172.21.15.107:0/2201854031 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e2000e040 con 0x7f8e300fbd00 2024-04-22T23:40:56.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.303+0000 7f8e2cff9700 1 -- 172.21.15.107:0/2201854031 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8e2000bbf0 con 0x7f8e300fbd00 2024-04-22T23:40:56.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.303+0000 7f8e363d0700 1 -- 172.21.15.107:0/2201854031 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e30111ee0 con 0x7f8e300fbd00 2024-04-22T23:40:56.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.303+0000 7f8e2cff9700 1 -- 172.21.15.107:0/2201854031 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e2001c3f0 con 0x7f8e300fbd00 2024-04-22T23:40:56.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.303+0000 7f8e363d0700 1 -- 172.21.15.107:0/2201854031 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e30112410 con 0x7f8e300fbd00 2024-04-22T23:40:56.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.304+0000 7f8e363d0700 1 -- 172.21.15.107:0/2201854031 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e3004c710 con 0x7f8e300fbd00 2024-04-22T23:40:56.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.305+0000 7f8e2cff9700 1 -- 172.21.15.107:0/2201854031 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f8e2000bd60 con 0x7f8e300fbd00 2024-04-22T23:40:56.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.305+0000 7f8e2cff9700 1 --2- 172.21.15.107:0/2201854031 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f8e1806ec00 0x7f8e180710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:40:56.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.305+0000 7f8e2cff9700 1 -- 172.21.15.107:0/2201854031 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f8e2008fc20 con 0x7f8e300fbd00 2024-04-22T23:40:56.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.305+0000 7f8e2f7fe700 1 --2- 172.21.15.107:0/2201854031 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f8e1806ec00 0x7f8e180710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:40:56.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.305+0000 7f8e2f7fe700 1 --2- 172.21.15.107:0/2201854031 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f8e1806ec00 0x7f8e180710b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8e24009fd0 tx=0x7f8e2400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:40:56.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.310+0000 7f8e2cff9700 1 -- 172.21.15.107:0/2201854031 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f8e2005c500 con 0x7f8e300fbd00 2024-04-22T23:40:56.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.532+0000 7f8e363d0700 1 -- 172.21.15.107:0/2201854031 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e301012c0 con 0x7f8e1806ec00 2024-04-22T23:40:56.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.533+0000 7f8e2cff9700 1 -- 172.21.15.107:0/2201854031 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f8e301012c0 con 0x7f8e1806ec00 2024-04-22T23:40:56.527 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-22T23:40:56.527 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c", 2024-04-22T23:40:56.527 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-04-22T23:40:56.527 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-22T23:40:56.527 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [ 2024-04-22T23:40:56.527 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr", 2024-04-22T23:40:56.527 INFO:teuthology.orchestra.run.smithi107.stdout: "crash", 2024-04-22T23:40:56.527 INFO:teuthology.orchestra.run.smithi107.stdout: "mon" 2024-04-22T23:40:56.527 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-22T23:40:56.527 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "9/21 daemons upgraded", 2024-04-22T23:40:56.527 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Currently upgrading osd daemons", 2024-04-22T23:40:56.527 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-04-22T23:40:56.527 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-22T23:40:56.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.535+0000 7f8e363d0700 1 -- 172.21.15.107:0/2201854031 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f8e1806ec00 msgr2=0x7f8e180710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:56.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.535+0000 7f8e363d0700 1 --2- 172.21.15.107:0/2201854031 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f8e1806ec00 0x7f8e180710b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8e24009fd0 tx=0x7f8e2400b040 comp rx=0 tx=0).stop 2024-04-22T23:40:56.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.535+0000 7f8e363d0700 1 -- 172.21.15.107:0/2201854031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e300fbd00 msgr2=0x7f8e3010c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:56.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.535+0000 7f8e363d0700 1 --2- 172.21.15.107:0/2201854031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e300fbd00 0x7f8e3010c830 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8e20009380 tx=0x7f8e20009e30 comp rx=0 tx=0).stop 2024-04-22T23:40:56.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.536+0000 7f8e363d0700 1 -- 172.21.15.107:0/2201854031 shutdown_connections 2024-04-22T23:40:56.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.536+0000 7f8e363d0700 1 --2- 172.21.15.107:0/2201854031 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f8e1806ec00 0x7f8e180710b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:56.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.536+0000 7f8e363d0700 1 --2- 172.21.15.107:0/2201854031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e300fbd00 0x7f8e3010c830 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:56.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.536+0000 7f8e363d0700 1 --2- 172.21.15.107:0/2201854031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e300fcfb0 0x7f8e3010cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:56.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.536+0000 7f8e363d0700 1 -- 172.21.15.107:0/2201854031 >> 172.21.15.107:0/2201854031 conn(0x7f8e300f7480 msgr2=0x7f8e300f8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:40:56.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.536+0000 7f8e363d0700 1 -- 172.21.15.107:0/2201854031 shutdown_connections 2024-04-22T23:40:56.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.536+0000 7f8e363d0700 1 -- 172.21.15.107:0/2201854031 wait complete. 2024-04-22T23:40:56.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.635+0000 7f2bffbb6700 1 -- 172.21.15.107:0/1224877934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bf80fce80 msgr2=0x7f2bf80fd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:56.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.635+0000 7f2bffbb6700 1 --2- 172.21.15.107:0/1224877934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bf80fce80 0x7f2bf80fd290 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0004940 tx=0x7f2bf0009e30 comp rx=0 tx=0).stop 2024-04-22T23:40:56.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.635+0000 7f2bffbb6700 1 -- 172.21.15.107:0/1224877934 shutdown_connections 2024-04-22T23:40:56.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.635+0000 7f2bffbb6700 1 --2- 172.21.15.107:0/1224877934 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bf80fdff0 0x7f2bf80fe460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:56.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.635+0000 7f2bffbb6700 1 --2- 172.21.15.107:0/1224877934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bf80fce80 0x7f2bf80fd290 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:56.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.635+0000 7f2bffbb6700 1 -- 172.21.15.107:0/1224877934 >> 172.21.15.107:0/1224877934 conn(0x7f2bf80f8580 msgr2=0x7f2bf80fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:40:56.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.635+0000 7f2bffbb6700 1 -- 172.21.15.107:0/1224877934 shutdown_connections 2024-04-22T23:40:56.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.635+0000 7f2bffbb6700 1 -- 172.21.15.107:0/1224877934 wait complete. 2024-04-22T23:40:56.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.636+0000 7f2bffbb6700 1 Processor -- start 2024-04-22T23:40:56.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.636+0000 7f2bffbb6700 1 -- start start 2024-04-22T23:40:56.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.636+0000 7f2bffbb6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bf80fce80 0x7f2bf810d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:40:56.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.636+0000 7f2bffbb6700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bf80fdff0 0x7f2bf810dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:40:56.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.636+0000 7f2bffbb6700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf810e4a0 con 0x7f2bf80fdff0 2024-04-22T23:40:56.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.636+0000 7f2bffbb6700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf810e610 con 0x7f2bf80fce80 2024-04-22T23:40:56.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.636+0000 7f2bfd151700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bf80fdff0 0x7f2bf810dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:40:56.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.636+0000 7f2bfd151700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bf80fdff0 0x7f2bf810dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54318/0 (socket says 172.21.15.107:54318) 2024-04-22T23:40:56.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.636+0000 7f2bfd151700 1 -- 172.21.15.107:0/2317992970 learned_addr learned my addr 172.21.15.107:0/2317992970 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:40:56.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.636+0000 7f2bfd952700 1 --2- 172.21.15.107:0/2317992970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bf80fce80 0x7f2bf810d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:40:56.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.637+0000 7f2bfd151700 1 -- 172.21.15.107:0/2317992970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bf80fce80 msgr2=0x7f2bf810d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:56.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.637+0000 7f2bfd151700 1 --2- 172.21.15.107:0/2317992970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bf80fce80 0x7f2bf810d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:56.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.637+0000 7f2bfd151700 1 -- 172.21.15.107:0/2317992970 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bf0005040 con 0x7f2bf80fdff0 2024-04-22T23:40:56.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.637+0000 7f2bfd151700 1 --2- 172.21.15.107:0/2317992970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bf80fdff0 0x7f2bf810dea0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2be8002a50 tx=0x7f2be80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:40:56.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.637+0000 7f2bee7fc700 1 -- 172.21.15.107:0/2317992970 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2be8007cc0 con 0x7f2bf80fdff0 2024-04-22T23:40:56.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.637+0000 7f2bee7fc700 1 -- 172.21.15.107:0/2317992970 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2be800f040 con 0x7f2bf80fdff0 2024-04-22T23:40:56.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.637+0000 7f2bffbb6700 1 -- 172.21.15.107:0/2317992970 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bf8113070 con 0x7f2bf80fdff0 2024-04-22T23:40:56.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.637+0000 7f2bee7fc700 1 -- 172.21.15.107:0/2317992970 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2be800da30 con 0x7f2bf80fdff0 2024-04-22T23:40:56.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.637+0000 7f2bffbb6700 1 -- 172.21.15.107:0/2317992970 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bf8113630 con 0x7f2bf80fdff0 2024-04-22T23:40:56.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.638+0000 7f2bffbb6700 1 -- 172.21.15.107:0/2317992970 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bf804c710 con 0x7f2bf80fdff0 2024-04-22T23:40:56.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.638+0000 7f2bee7fc700 1 -- 172.21.15.107:0/2317992970 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f2be800db90 con 0x7f2bf80fdff0 2024-04-22T23:40:56.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.639+0000 7f2bee7fc700 1 --2- 172.21.15.107:0/2317992970 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f2be406ec00 0x7f2be40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:40:56.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.639+0000 7f2bee7fc700 1 -- 172.21.15.107:0/2317992970 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f2be808e620 con 0x7f2bf80fdff0 2024-04-22T23:40:56.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.639+0000 7f2bfd952700 1 --2- 172.21.15.107:0/2317992970 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f2be406ec00 0x7f2be40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:40:56.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.639+0000 7f2bfd952700 1 --2- 172.21.15.107:0/2317992970 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f2be406ec00 0x7f2be40710b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2bf000b970 tx=0x7f2bf0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:40:56.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.643+0000 7f2bee7fc700 1 -- 172.21.15.107:0/2317992970 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f2be805af00 con 0x7f2bf80fdff0 2024-04-22T23:40:56.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.932+0000 7f2bffbb6700 1 -- 172.21.15.107:0/2317992970 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f2bf8113a60 con 0x7f2bf80fdff0 2024-04-22T23:40:56.925 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:56 smithi107 ceph-mon[90179]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:40:56.925 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:56 smithi107 ceph-mon[90179]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-22T23:40:56.926 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:56 smithi107 ceph-mon[90179]: Upgrade: osd.3 is safe to restart 2024-04-22T23:40:56.926 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:56 smithi107 ceph-mon[90179]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:40:56.926 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:56 smithi107 ceph-mon[90179]: pgmap v108: 65 pgs: 65 active+clean; 277 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:40:56.926 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:56 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/987154603' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:40:56.926 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:56 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/2634254001' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:40:56.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.932+0000 7f2bee7fc700 1 -- 172.21.15.107:0/2317992970 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f2be805aa90 con 0x7f2bf80fdff0 2024-04-22T23:40:56.926 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_OK 2024-04-22T23:40:56.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.934+0000 7f2bffbb6700 1 -- 172.21.15.107:0/2317992970 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f2be406ec00 msgr2=0x7f2be40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:56.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.934+0000 7f2bffbb6700 1 --2- 172.21.15.107:0/2317992970 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f2be406ec00 0x7f2be40710b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2bf000b970 tx=0x7f2bf0009e30 comp rx=0 tx=0).stop 2024-04-22T23:40:56.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.934+0000 7f2bffbb6700 1 -- 172.21.15.107:0/2317992970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bf80fdff0 msgr2=0x7f2bf810dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:40:56.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.934+0000 7f2bffbb6700 1 --2- 172.21.15.107:0/2317992970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bf80fdff0 0x7f2bf810dea0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2be8002a50 tx=0x7f2be80046b0 comp rx=0 tx=0).stop 2024-04-22T23:40:56.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.934+0000 7f2bffbb6700 1 -- 172.21.15.107:0/2317992970 shutdown_connections 2024-04-22T23:40:56.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.934+0000 7f2bffbb6700 1 --2- 172.21.15.107:0/2317992970 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2bf80fce80 0x7f2bf810d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:56.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.934+0000 7f2bffbb6700 1 --2- 172.21.15.107:0/2317992970 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f2be406ec00 0x7f2be40710b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:56.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.934+0000 7f2bffbb6700 1 --2- 172.21.15.107:0/2317992970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bf80fdff0 0x7f2bf810dea0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:40:56.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.935+0000 7f2bffbb6700 1 -- 172.21.15.107:0/2317992970 >> 172.21.15.107:0/2317992970 conn(0x7f2bf80f8580 msgr2=0x7f2bf80f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:40:56.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.935+0000 7f2bffbb6700 1 -- 172.21.15.107:0/2317992970 shutdown_connections 2024-04-22T23:40:56.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:40:56.935+0000 7f2bffbb6700 1 -- 172.21.15.107:0/2317992970 wait complete. 2024-04-22T23:40:57.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:56 smithi157 ceph-mon[79403]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:40:57.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:56 smithi157 ceph-mon[79403]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-22T23:40:57.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:56 smithi157 ceph-mon[79403]: Upgrade: osd.3 is safe to restart 2024-04-22T23:40:57.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:56 smithi157 ceph-mon[79403]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:40:57.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:56 smithi157 ceph-mon[79403]: pgmap v108: 65 pgs: 65 active+clean; 277 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:40:57.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:56 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/987154603' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:40:57.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:56 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/2634254001' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:40:58.116 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:57 smithi157 ceph-mon[79403]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:40:58.117 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:57 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:40:58.117 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:57 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-22T23:40:58.117 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:57 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:40:58.117 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:57 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/2317992970' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:40:58.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:57 smithi107 ceph-mon[90179]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:40:58.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:57 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:40:58.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:57 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-22T23:40:58.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:57 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:40:58.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:57 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/2317992970' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:40:58.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:58 smithi107 ceph-mon[90179]: Upgrade: Updating osd.3 2024-04-22T23:40:58.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:58 smithi107 ceph-mon[90179]: Deploying daemon osd.3 on smithi157 2024-04-22T23:40:58.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:40:58 smithi107 ceph-mon[90179]: pgmap v109: 65 pgs: 65 active+clean; 277 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:40:59.105 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:58 smithi157 ceph-mon[79403]: Upgrade: Updating osd.3 2024-04-22T23:40:59.106 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:58 smithi157 ceph-mon[79403]: Deploying daemon osd.3 on smithi157 2024-04-22T23:40:59.106 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:40:58 smithi157 ceph-mon[79403]: pgmap v109: 65 pgs: 65 active+clean; 277 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:41:00.381 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:41:00 smithi157 systemd[1]: Stopping Ceph osd.3 for 8252f6f2-00ff-11ef-bc93-c7b262605968... 2024-04-22T23:41:00.381 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:41:00 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-3[39457]: 2024-04-22T23:41:00.360+0000 7fd0d6ea4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T23:41:00.381 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:41:00 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-3[39457]: 2024-04-22T23:41:00.360+0000 7fd0d6ea4700 -1 osd.3 64 *** Got signal Terminated *** 2024-04-22T23:41:00.381 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:41:00 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-3[39457]: 2024-04-22T23:41:00.360+0000 7fd0d6ea4700 -1 osd.3 64 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T23:41:01.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:00 smithi157 ceph-mon[79403]: pgmap v110: 65 pgs: 65 active+clean; 277 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:41:01.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:00 smithi157 ceph-mon[79403]: osd.3 marked itself down and dead 2024-04-22T23:41:01.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:00 smithi107 ceph-mon[90179]: pgmap v110: 65 pgs: 65 active+clean; 277 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:41:01.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:00 smithi107 ceph-mon[90179]: osd.3 marked itself down and dead 2024-04-22T23:41:01.529 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:41:01 smithi157 podman[88352]: 2024-04-22 23:41:01.21947299 +0000 UTC m=+0.958538181 container died c9fb2d1a13dd8437e5fbeb9901cf15441f240a12413e25e38b5a9731eb159d9f (image=quay.ceph.io/ceph-ci/ceph@sha256:3890cd726bd5e73ae623dabdf4ed76dd5df81c936b8bd8d761026babda969466, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-3, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=quincy-70d0cb4) 2024-04-22T23:41:01.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:01 smithi157 ceph-mon[79403]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-22T23:41:01.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:01 smithi157 ceph-mon[79403]: osdmap e65: 6 total, 5 up, 6 in 2024-04-22T23:41:01.880 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:41:01 smithi157 podman[88352]: 2024-04-22 23:41:01.627221903 +0000 UTC m=+1.366287093 container cleanup c9fb2d1a13dd8437e5fbeb9901cf15441f240a12413e25e38b5a9731eb159d9f (image=quay.ceph.io/ceph-ci/ceph@sha256:3890cd726bd5e73ae623dabdf4ed76dd5df81c936b8bd8d761026babda969466, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-3, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=quincy-70d0cb4, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True) 2024-04-22T23:41:01.880 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:41:01 smithi157 bash[88352]: ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-3 2024-04-22T23:41:02.130 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:41:01 smithi157 podman[88372]: 2024-04-22 23:41:01.887360022 +0000 UTC m=+0.666770301 container remove c9fb2d1a13dd8437e5fbeb9901cf15441f240a12413e25e38b5a9731eb159d9f (image=quay.ceph.io/ceph-ci/ceph@sha256:3890cd726bd5e73ae623dabdf4ed76dd5df81c936b8bd8d761026babda969466, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, RELEASE=quincy-70d0cb4, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240416) 2024-04-22T23:41:02.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:01 smithi107 ceph-mon[90179]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-22T23:41:02.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:01 smithi107 ceph-mon[90179]: osdmap e65: 6 total, 5 up, 6 in 2024-04-22T23:41:02.558 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:41:02 smithi157 podman[88466]: 2024-04-22 23:41:02.145517961 +0000 UTC m=+0.017935781 image pull a0434699c1d76af4eea1fc95ff0a22ea73d1e04e609464fbf658d6f39bc7f2fc quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c 2024-04-22T23:41:02.558 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:41:02 smithi157 podman[88466]: 2024-04-22 23:41:02.280051086 +0000 UTC m=+0.152468935 container create 7079ecba66b5dfa81a67a5b1bfa7a75441fa3002642f5713b6861e7d4e23ef8b (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True) 2024-04-22T23:41:02.558 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:41:02 smithi157 podman[88466]: 2024-04-22 23:41:02.565866959 +0000 UTC m=+0.438284764 container init 7079ecba66b5dfa81a67a5b1bfa7a75441fa3002642f5713b6861e7d4e23ef8b (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-3-deactivate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T23:41:02.854 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:02 smithi157 ceph-mon[79403]: pgmap v112: 65 pgs: 18 peering, 6 stale+active+clean, 41 active+clean; 277 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:41:02.854 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:02 smithi157 ceph-mon[79403]: Health check failed: Reduced data availability: 10 pgs peering (PG_AVAILABILITY) 2024-04-22T23:41:02.855 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:02 smithi157 ceph-mon[79403]: osdmap e66: 6 total, 5 up, 6 in 2024-04-22T23:41:02.855 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:41:02 smithi157 podman[88466]: 2024-04-22 23:41:02.569124784 +0000 UTC m=+0.441542600 container start 7079ecba66b5dfa81a67a5b1bfa7a75441fa3002642f5713b6861e7d4e23ef8b (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.build-date=20240416, ceph=True) 2024-04-22T23:41:02.855 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:41:02 smithi157 podman[88466]: 2024-04-22 23:41:02.649104783 +0000 UTC m=+0.521522599 container attach 7079ecba66b5dfa81a67a5b1bfa7a75441fa3002642f5713b6861e7d4e23ef8b (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, ceph=True) 2024-04-22T23:41:03.130 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:41:02 smithi157 podman[88466]: 2024-04-22 23:41:02.859603625 +0000 UTC m=+0.732021449 container died 7079ecba66b5dfa81a67a5b1bfa7a75441fa3002642f5713b6861e7d4e23ef8b (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-22T23:41:03.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:02 smithi107 ceph-mon[90179]: pgmap v112: 65 pgs: 18 peering, 6 stale+active+clean, 41 active+clean; 277 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:41:03.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:02 smithi107 ceph-mon[90179]: Health check failed: Reduced data availability: 10 pgs peering (PG_AVAILABILITY) 2024-04-22T23:41:03.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:02 smithi107 ceph-mon[90179]: osdmap e66: 6 total, 5 up, 6 in 2024-04-22T23:41:03.822 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:41:03 smithi157 podman[88493]: 2024-04-22 23:41:03.402513569 +0000 UTC m=+0.537680604 container remove 7079ecba66b5dfa81a67a5b1bfa7a75441fa3002642f5713b6861e7d4e23ef8b (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-3-deactivate, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True) 2024-04-22T23:41:03.823 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:41:03 smithi157 systemd[1]: ceph-8252f6f2-00ff-11ef-bc93-c7b262605968@osd.3.service: Deactivated successfully. 2024-04-22T23:41:03.823 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:41:03 smithi157 systemd[1]: Stopped Ceph osd.3 for 8252f6f2-00ff-11ef-bc93-c7b262605968. 2024-04-22T23:41:03.823 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:41:03 smithi157 systemd[1]: ceph-8252f6f2-00ff-11ef-bc93-c7b262605968@osd.3.service: Consumed 1min 7.266s CPU time. 2024-04-22T23:41:03.823 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:41:03 smithi157 systemd[1]: Starting Ceph osd.3 for 8252f6f2-00ff-11ef-bc93-c7b262605968... 2024-04-22T23:41:04.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:03 smithi157 ceph-mon[79403]: pgmap v114: 65 pgs: 3 active+undersized+degraded, 18 peering, 3 stale+active+clean, 4 active+undersized, 37 active+clean; 277 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 9/339 objects degraded (2.655%) 2024-04-22T23:41:04.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:03 smithi157 ceph-mon[79403]: Health check failed: Degraded data redundancy: 9/339 objects degraded (2.655%), 3 pgs degraded (PG_DEGRADED) 2024-04-22T23:41:04.131 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:41:03 smithi157 podman[88595]: 2024-04-22 23:41:03.736358505 +0000 UTC m=+0.018409089 image pull a0434699c1d76af4eea1fc95ff0a22ea73d1e04e609464fbf658d6f39bc7f2fc quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c 2024-04-22T23:41:04.131 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:41:03 smithi157 podman[88595]: 2024-04-22 23:41:03.870419697 +0000 UTC m=+0.152470272 container create 4d07d06c3ca82c2565437c69858518e3c37368d5468b87fb9c656168da8d1311 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-3-activate, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240416, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-adk-testing-2024-04-22-1618-43be020, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-22T23:41:04.131 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:41:04 smithi157 podman[88595]: 2024-04-22 23:41:04.064329998 +0000 UTC m=+0.346380573 container init 4d07d06c3ca82c2565437c69858518e3c37368d5468b87fb9c656168da8d1311 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-3-activate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-22T23:41:04.131 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:41:04 smithi157 podman[88595]: 2024-04-22 23:41:04.067399506 +0000 UTC m=+0.349450082 container start 4d07d06c3ca82c2565437c69858518e3c37368d5468b87fb9c656168da8d1311 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-3-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=) 2024-04-22T23:41:04.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:03 smithi107 ceph-mon[90179]: pgmap v114: 65 pgs: 3 active+undersized+degraded, 18 peering, 3 stale+active+clean, 4 active+undersized, 37 active+clean; 277 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 9/339 objects degraded (2.655%) 2024-04-22T23:41:04.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:03 smithi107 ceph-mon[90179]: Health check failed: Degraded data redundancy: 9/339 objects degraded (2.655%), 3 pgs degraded (PG_DEGRADED) 2024-04-22T23:41:04.539 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:41:04 smithi157 podman[88595]: 2024-04-22 23:41:04.147770973 +0000 UTC m=+0.429821554 container attach 4d07d06c3ca82c2565437c69858518e3c37368d5468b87fb9c656168da8d1311 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-3-activate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, ceph=True) 2024-04-22T23:41:04.881 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:41:04 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-3-activate[88610]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-22T23:41:04.881 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:41:04 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-3-activate[88610]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-22T23:41:04.881 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:41:04 smithi157 bash[88595]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-22T23:41:04.881 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:41:04 smithi157 bash[88595]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-22T23:41:04.881 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:41:04 smithi157 bash[88595]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-22T23:41:04.881 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:41:04 smithi157 bash[88595]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-22T23:41:04.881 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:41:04 smithi157 bash[88595]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-22T23:41:04.881 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:41:04 smithi157 bash[88595]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-22T23:41:04.882 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:41:04 smithi157 bash[88595]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-04-22T23:41:04.882 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:41:04 smithi157 bash[88595]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-22T23:41:04.882 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:41:04 smithi157 bash[88595]: --> ceph-volume raw activate successful for osd ID: 3 2024-04-22T23:41:04.882 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:41:04 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-3-activate[88610]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-22T23:41:04.882 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:41:04 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-3-activate[88610]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-22T23:41:04.882 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:41:04 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-3-activate[88610]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-22T23:41:04.882 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:41:04 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-3-activate[88610]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-22T23:41:04.882 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:41:04 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-3-activate[88610]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-04-22T23:41:04.882 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:41:04 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-3-activate[88610]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-22T23:41:04.882 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:41:04 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-3-activate[88610]: --> ceph-volume raw activate successful for osd ID: 3 2024-04-22T23:41:04.882 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:41:04 smithi157 podman[88595]: 2024-04-22 23:41:04.571226539 +0000 UTC m=+0.853277108 container died 4d07d06c3ca82c2565437c69858518e3c37368d5468b87fb9c656168da8d1311 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-3-activate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-22T23:41:05.806 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:41:05 smithi157 podman[88751]: 2024-04-22 23:41:05.418427783 +0000 UTC m=+0.842185792 container remove 4d07d06c3ca82c2565437c69858518e3c37368d5468b87fb9c656168da8d1311 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-3-activate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-adk-testing-2024-04-22-1618-43be020) 2024-04-22T23:41:05.806 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:41:05 smithi157 podman[88818]: 2024-04-22 23:41:05.536422398 +0000 UTC m=+0.017852216 image pull a0434699c1d76af4eea1fc95ff0a22ea73d1e04e609464fbf658d6f39bc7f2fc quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c 2024-04-22T23:41:05.806 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:41:05 smithi157 podman[88818]: 2024-04-22 23:41:05.652702764 +0000 UTC m=+0.134132558 container create e81ca519c8318f203dcc7937b3566041fdddfd735147df2d9a41f7a22b19b590 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-3, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T23:41:06.130 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:41:05 smithi157 podman[88818]: 2024-04-22 23:41:05.955505664 +0000 UTC m=+0.436935465 container init e81ca519c8318f203dcc7937b3566041fdddfd735147df2d9a41f7a22b19b590 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-3, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-adk-testing-2024-04-22-1618-43be020) 2024-04-22T23:41:06.130 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:41:05 smithi157 podman[88818]: 2024-04-22 23:41:05.958506293 +0000 UTC m=+0.439936087 container start e81ca519c8318f203dcc7937b3566041fdddfd735147df2d9a41f7a22b19b590 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-3, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-22T23:41:06.130 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:41:05 smithi157 ceph-osd[88837]: -- 172.21.15.157:0/680901279 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5654285f9680 con 0x5654285c2c00 2024-04-22T23:41:06.130 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:41:06 smithi157 bash[88818]: e81ca519c8318f203dcc7937b3566041fdddfd735147df2d9a41f7a22b19b590 2024-04-22T23:41:06.131 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:41:06 smithi157 systemd[1]: Started Ceph osd.3 for 8252f6f2-00ff-11ef-bc93-c7b262605968. 2024-04-22T23:41:06.880 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:41:06 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-3[88833]: 2024-04-22T23:41:06.505+0000 7f227e7fd7c0 -1 Falling back to public interface 2024-04-22T23:41:06.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:06 smithi157 ceph-mon[79403]: pgmap v115: 65 pgs: 9 active+undersized+degraded, 18 peering, 1 stale+active+clean, 7 active+undersized, 30 active+clean; 277 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 29/339 objects degraded (8.555%) 2024-04-22T23:41:06.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:06 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:41:06.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:06 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:41:06.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:06 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:41:07.202 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:06 smithi107 ceph-mon[90179]: pgmap v115: 65 pgs: 9 active+undersized+degraded, 18 peering, 1 stale+active+clean, 7 active+undersized, 30 active+clean; 277 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 29/339 objects degraded (8.555%) 2024-04-22T23:41:07.202 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:06 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:41:07.202 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:06 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:41:07.202 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:06 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:41:07.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:07 smithi157 ceph-mon[79403]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 10 pgs peering) 2024-04-22T23:41:08.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:07 smithi107 ceph-mon[90179]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 10 pgs peering) 2024-04-22T23:41:08.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:08 smithi107 ceph-mon[90179]: pgmap v116: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 277 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 70/339 objects degraded (20.649%) 2024-04-22T23:41:09.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:08 smithi157 ceph-mon[79403]: pgmap v116: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 277 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 70/339 objects degraded (20.649%) 2024-04-22T23:41:10.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:10 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:41:10.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:10 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:41:10.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:10 smithi157 ceph-mon[79403]: pgmap v117: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 277 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 70/339 objects degraded (20.649%) 2024-04-22T23:41:10.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:10 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:41:10.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:10 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:41:10.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:10 smithi107 ceph-mon[90179]: pgmap v117: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 277 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 70/339 objects degraded (20.649%) 2024-04-22T23:41:11.380 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:11 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:41:11.380 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:11 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:41:11.380 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:11 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:41:11.380 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:11 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:41:11.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:11 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:41:11.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:11 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:41:11.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:11 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:41:11.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:11 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:41:11.983 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:41:11 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-3[88833]: 2024-04-22T23:41:11.721+0000 7f227e7fd7c0 -1 osd.3 64 log_to_monitors true 2024-04-22T23:41:12.293 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:41:12 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-3[88833]: 2024-04-22T23:41:12.199+0000 7f226fc55700 -1 osd.3 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T23:41:12.294 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:12 smithi157 ceph-mon[79403]: pgmap v118: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 277 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 70/339 objects degraded (20.649%) 2024-04-22T23:41:12.294 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:12 smithi157 ceph-mon[79403]: from='osd.3 [v2:172.21.15.157:6800/1553340859,v1:172.21.15.157:6801/1553340859]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-22T23:41:12.294 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:12 smithi157 ceph-mon[79403]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-22T23:41:12.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:12 smithi107 ceph-mon[90179]: pgmap v118: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 277 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 70/339 objects degraded (20.649%) 2024-04-22T23:41:12.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:12 smithi107 ceph-mon[90179]: from='osd.3 [v2:172.21.15.157:6800/1553340859,v1:172.21.15.157:6801/1553340859]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-22T23:41:12.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:12 smithi107 ceph-mon[90179]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-22T23:41:13.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:13 smithi157 ceph-mon[79403]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-22T23:41:13.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:13 smithi157 ceph-mon[79403]: from='osd.3 [v2:172.21.15.157:6800/1553340859,v1:172.21.15.157:6801/1553340859]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-22T23:41:13.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:13 smithi157 ceph-mon[79403]: osdmap e67: 6 total, 5 up, 6 in 2024-04-22T23:41:13.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:13 smithi157 ceph-mon[79403]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-22T23:41:13.382 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:13 smithi157 ceph-mon[79403]: Health check update: Degraded data redundancy: 70/339 objects degraded (20.649%), 19 pgs degraded (PG_DEGRADED) 2024-04-22T23:41:13.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:13 smithi107 ceph-mon[90179]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-22T23:41:13.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:13 smithi107 ceph-mon[90179]: from='osd.3 [v2:172.21.15.157:6800/1553340859,v1:172.21.15.157:6801/1553340859]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-22T23:41:13.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:13 smithi107 ceph-mon[90179]: osdmap e67: 6 total, 5 up, 6 in 2024-04-22T23:41:13.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:13 smithi107 ceph-mon[90179]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-22T23:41:13.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:13 smithi107 ceph-mon[90179]: Health check update: Degraded data redundancy: 70/339 objects degraded (20.649%), 19 pgs degraded (PG_DEGRADED) 2024-04-22T23:41:14.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:14 smithi107 ceph-mon[90179]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-22T23:41:14.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:14 smithi107 ceph-mon[90179]: osd.3 [v2:172.21.15.157:6800/1553340859,v1:172.21.15.157:6801/1553340859] boot 2024-04-22T23:41:14.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:14 smithi107 ceph-mon[90179]: osdmap e68: 6 total, 6 up, 6 in 2024-04-22T23:41:14.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:14 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T23:41:14.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:14 smithi107 ceph-mon[90179]: pgmap v121: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 277 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 70/339 objects degraded (20.649%) 2024-04-22T23:41:14.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:14 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:41:14.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:14 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:41:14.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:14 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:41:14.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:14 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:41:14.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:14 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:41:14.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:14 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:41:14.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:14 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:41:14.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:14 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:41:14.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:14 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:41:14.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:14 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-22T23:41:14.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:14 smithi107 ceph-mon[90179]: osdmap e69: 6 total, 6 up, 6 in 2024-04-22T23:41:14.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:14 smithi157 ceph-mon[79403]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-22T23:41:14.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:14 smithi157 ceph-mon[79403]: osd.3 [v2:172.21.15.157:6800/1553340859,v1:172.21.15.157:6801/1553340859] boot 2024-04-22T23:41:14.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:14 smithi157 ceph-mon[79403]: osdmap e68: 6 total, 6 up, 6 in 2024-04-22T23:41:14.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:14 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T23:41:14.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:14 smithi157 ceph-mon[79403]: pgmap v121: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 277 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 70/339 objects degraded (20.649%) 2024-04-22T23:41:14.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:14 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:41:14.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:14 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:41:14.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:14 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:41:14.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:14 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:41:14.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:14 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:41:14.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:14 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:41:14.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:14 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:41:14.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:14 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:41:14.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:14 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:41:14.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:14 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-22T23:41:14.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:14 smithi157 ceph-mon[79403]: osdmap e69: 6 total, 6 up, 6 in 2024-04-22T23:41:15.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:15 smithi107 ceph-mon[90179]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-22T23:41:15.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:15 smithi107 ceph-mon[90179]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-04-22T23:41:15.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:15 smithi157 ceph-mon[79403]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-22T23:41:15.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:15 smithi157 ceph-mon[79403]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-04-22T23:41:16.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:16 smithi107 ceph-mon[90179]: pgmap v123: 65 pgs: 15 active+undersized+degraded, 16 active+undersized, 34 active+clean; 277 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 58/339 objects degraded (17.109%) 2024-04-22T23:41:16.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:16 smithi157 ceph-mon[79403]: pgmap v123: 65 pgs: 15 active+undersized+degraded, 16 active+undersized, 34 active+clean; 277 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 58/339 objects degraded (17.109%) 2024-04-22T23:41:17.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:17 smithi107 ceph-mon[90179]: Health check update: Degraded data redundancy: 58/339 objects degraded (17.109%), 15 pgs degraded (PG_DEGRADED) 2024-04-22T23:41:17.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:17 smithi157 ceph-mon[79403]: Health check update: Degraded data redundancy: 58/339 objects degraded (17.109%), 15 pgs degraded (PG_DEGRADED) 2024-04-22T23:41:18.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:18 smithi157 ceph-mon[79403]: pgmap v124: 65 pgs: 1 active+undersized+degraded, 3 active+undersized, 61 active+clean; 277 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 4/339 objects degraded (1.180%) 2024-04-22T23:41:18.651 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:18 smithi107 ceph-mon[90179]: pgmap v124: 65 pgs: 1 active+undersized+degraded, 3 active+undersized, 61 active+clean; 277 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 4/339 objects degraded (1.180%) 2024-04-22T23:41:20.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:19 smithi157 ceph-mon[79403]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 4/339 objects degraded (1.180%), 1 pg degraded) 2024-04-22T23:41:20.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:19 smithi157 ceph-mon[79403]: Cluster is now healthy 2024-04-22T23:41:20.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:19 smithi107 ceph-mon[90179]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 4/339 objects degraded (1.180%), 1 pg degraded) 2024-04-22T23:41:20.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:19 smithi107 ceph-mon[90179]: Cluster is now healthy 2024-04-22T23:41:21.053 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:20 smithi157 ceph-mon[79403]: pgmap v125: 65 pgs: 65 active+clean; 277 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:41:21.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:20 smithi107 ceph-mon[90179]: pgmap v125: 65 pgs: 65 active+clean; 277 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:41:23.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:22 smithi157 ceph-mon[79403]: pgmap v126: 65 pgs: 65 active+clean; 277 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:41:23.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:22 smithi107 ceph-mon[90179]: pgmap v126: 65 pgs: 65 active+clean; 277 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:41:24.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:23 smithi157 ceph-mon[79403]: pgmap v127: 65 pgs: 65 active+clean; 277 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:41:24.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:23 smithi107 ceph-mon[90179]: pgmap v127: 65 pgs: 65 active+clean; 277 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:41:26.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:26 smithi107 ceph-mon[90179]: pgmap v128: 65 pgs: 65 active+clean; 277 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:41:26.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:26 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:41:26.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:26 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:41:26.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:26 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:41:27.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.048+0000 7f6999a32700 1 -- 172.21.15.107:0/2433678366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69940fdff0 msgr2=0x7f69940fe460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:27.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.048+0000 7f6999a32700 1 --2- 172.21.15.107:0/2433678366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69940fdff0 0x7f69940fe460 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f697c004940 tx=0x7f697c009e30 comp rx=0 tx=0).stop 2024-04-22T23:41:27.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.048+0000 7f6999a32700 1 -- 172.21.15.107:0/2433678366 shutdown_connections 2024-04-22T23:41:27.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.048+0000 7f6999a32700 1 --2- 172.21.15.107:0/2433678366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69940fdff0 0x7f69940fe460 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:27.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.048+0000 7f6999a32700 1 --2- 172.21.15.107:0/2433678366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69940fce80 0x7f69940fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:27.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.048+0000 7f6999a32700 1 -- 172.21.15.107:0/2433678366 >> 172.21.15.107:0/2433678366 conn(0x7f69940f8580 msgr2=0x7f69940fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:27.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.048+0000 7f6999a32700 1 -- 172.21.15.107:0/2433678366 shutdown_connections 2024-04-22T23:41:27.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.048+0000 7f6999a32700 1 -- 172.21.15.107:0/2433678366 wait complete. 2024-04-22T23:41:27.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.049+0000 7f6999a32700 1 Processor -- start 2024-04-22T23:41:27.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.049+0000 7f6999a32700 1 -- start start 2024-04-22T23:41:27.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.049+0000 7f6999a32700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69940fce80 0x7f6994071c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:27.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.049+0000 7f6999a32700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69940fdff0 0x7f6994070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:27.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.049+0000 7f6999a32700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6994072260 con 0x7f69940fdff0 2024-04-22T23:41:27.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.049+0000 7f6999a32700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6994070850 con 0x7f69940fce80 2024-04-22T23:41:27.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.050+0000 7f69927fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69940fdff0 0x7f6994070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:27.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.050+0000 7f69927fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69940fdff0 0x7f6994070310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33194/0 (socket says 172.21.15.107:33194) 2024-04-22T23:41:27.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.050+0000 7f69927fc700 1 -- 172.21.15.107:0/2753154554 learned_addr learned my addr 172.21.15.107:0/2753154554 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:41:27.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.050+0000 7f6992ffd700 1 --2- 172.21.15.107:0/2753154554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69940fce80 0x7f6994071c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:27.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.050+0000 7f69927fc700 1 -- 172.21.15.107:0/2753154554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69940fce80 msgr2=0x7f6994071c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:27.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.050+0000 7f69927fc700 1 --2- 172.21.15.107:0/2753154554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69940fce80 0x7f6994071c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:27.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.050+0000 7f69927fc700 1 -- 172.21.15.107:0/2753154554 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f697c005040 con 0x7f69940fdff0 2024-04-22T23:41:27.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.050+0000 7f69927fc700 1 --2- 172.21.15.107:0/2753154554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69940fdff0 0x7f6994070310 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f697c00b3b0 tx=0x7f697c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:27.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.050+0000 7f698b7fe700 1 -- 172.21.15.107:0/2753154554 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f697c00e040 con 0x7f69940fdff0 2024-04-22T23:41:27.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.050+0000 7f698b7fe700 1 -- 172.21.15.107:0/2753154554 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f697c00bbf0 con 0x7f69940fdff0 2024-04-22T23:41:27.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.050+0000 7f6999a32700 1 -- 172.21.15.107:0/2753154554 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6994070aa0 con 0x7f69940fdff0 2024-04-22T23:41:27.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.050+0000 7f698b7fe700 1 -- 172.21.15.107:0/2753154554 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f697c01c630 con 0x7f69940fdff0 2024-04-22T23:41:27.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.050+0000 7f6999a32700 1 -- 172.21.15.107:0/2753154554 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6994071060 con 0x7f69940fdff0 2024-04-22T23:41:27.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.052+0000 7f6999a32700 1 -- 172.21.15.107:0/2753154554 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f699404c710 con 0x7f69940fdff0 2024-04-22T23:41:27.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.052+0000 7f698b7fe700 1 -- 172.21.15.107:0/2753154554 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f697c013420 con 0x7f69940fdff0 2024-04-22T23:41:27.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.052+0000 7f698b7fe700 1 --2- 172.21.15.107:0/2753154554 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f698006ec00 0x7f69800710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:27.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.052+0000 7f698b7fe700 1 -- 172.21.15.107:0/2753154554 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f697c08f440 con 0x7f69940fdff0 2024-04-22T23:41:27.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.052+0000 7f6992ffd700 1 --2- 172.21.15.107:0/2753154554 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f698006ec00 0x7f69800710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:27.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.053+0000 7f6992ffd700 1 --2- 172.21.15.107:0/2753154554 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f698006ec00 0x7f69800710b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f698c004550 tx=0x7f698c007ed0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:27.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.057+0000 7f698b7fe700 1 -- 172.21.15.107:0/2753154554 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f697c05bda0 con 0x7f69940fdff0 2024-04-22T23:41:27.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:26 smithi157 ceph-mon[79403]: pgmap v128: 65 pgs: 65 active+clean; 277 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:41:27.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:26 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:41:27.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:26 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:41:27.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:26 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:41:27.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.276+0000 7f6999a32700 1 -- 172.21.15.107:0/2753154554 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f69941023b0 con 0x7f698006ec00 2024-04-22T23:41:27.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.277+0000 7f698b7fe700 1 -- 172.21.15.107:0/2753154554 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f69941023b0 con 0x7f698006ec00 2024-04-22T23:41:27.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.279+0000 7f6999a32700 1 -- 172.21.15.107:0/2753154554 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f698006ec00 msgr2=0x7f69800710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:27.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.279+0000 7f6999a32700 1 --2- 172.21.15.107:0/2753154554 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f698006ec00 0x7f69800710b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f698c004550 tx=0x7f698c007ed0 comp rx=0 tx=0).stop 2024-04-22T23:41:27.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.279+0000 7f6999a32700 1 -- 172.21.15.107:0/2753154554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69940fdff0 msgr2=0x7f6994070310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:27.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.279+0000 7f6999a32700 1 --2- 172.21.15.107:0/2753154554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69940fdff0 0x7f6994070310 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f697c00b3b0 tx=0x7f697c009e30 comp rx=0 tx=0).stop 2024-04-22T23:41:27.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.280+0000 7f6999a32700 1 -- 172.21.15.107:0/2753154554 shutdown_connections 2024-04-22T23:41:27.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.280+0000 7f6999a32700 1 --2- 172.21.15.107:0/2753154554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69940fce80 0x7f6994071c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:27.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.280+0000 7f6999a32700 1 --2- 172.21.15.107:0/2753154554 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f698006ec00 0x7f69800710b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:27.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.280+0000 7f6999a32700 1 --2- 172.21.15.107:0/2753154554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69940fdff0 0x7f6994070310 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:27.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.280+0000 7f6999a32700 1 -- 172.21.15.107:0/2753154554 >> 172.21.15.107:0/2753154554 conn(0x7f69940f8580 msgr2=0x7f69940f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:27.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.280+0000 7f6999a32700 1 -- 172.21.15.107:0/2753154554 shutdown_connections 2024-04-22T23:41:27.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.280+0000 7f6999a32700 1 -- 172.21.15.107:0/2753154554 wait complete. 2024-04-22T23:41:27.283 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-04-22T23:41:27.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.382+0000 7f1e2f24a700 1 -- 172.21.15.107:0/2451566469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e280fffd0 msgr2=0x7f1e281023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:27.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.382+0000 7f1e2f24a700 1 --2- 172.21.15.107:0/2451566469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e280fffd0 0x7f1e281023b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1e14004940 tx=0x7f1e14009e30 comp rx=0 tx=0).stop 2024-04-22T23:41:27.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.382+0000 7f1e2f24a700 1 -- 172.21.15.107:0/2451566469 shutdown_connections 2024-04-22T23:41:27.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.382+0000 7f1e2f24a700 1 --2- 172.21.15.107:0/2451566469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e280fffd0 0x7f1e281023b0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:27.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.382+0000 7f1e2f24a700 1 --2- 172.21.15.107:0/2451566469 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e28066ee0 0x7f1e280ffa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:27.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.382+0000 7f1e2f24a700 1 -- 172.21.15.107:0/2451566469 >> 172.21.15.107:0/2451566469 conn(0x7f1e280f7480 msgr2=0x7f1e280f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:27.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.382+0000 7f1e2f24a700 1 -- 172.21.15.107:0/2451566469 shutdown_connections 2024-04-22T23:41:27.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.382+0000 7f1e2f24a700 1 -- 172.21.15.107:0/2451566469 wait complete. 2024-04-22T23:41:27.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.382+0000 7f1e2f24a700 1 Processor -- start 2024-04-22T23:41:27.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.382+0000 7f1e2f24a700 1 -- start start 2024-04-22T23:41:27.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.383+0000 7f1e2f24a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e28066ee0 0x7f1e2810d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:27.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.383+0000 7f1e2f24a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e280fffd0 0x7f1e2810dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:27.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.383+0000 7f1e2f24a700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2810e4e0 con 0x7f1e28066ee0 2024-04-22T23:41:27.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.383+0000 7f1e2f24a700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2810e650 con 0x7f1e280fffd0 2024-04-22T23:41:27.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.383+0000 7f1e2cfe6700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e28066ee0 0x7f1e2810d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:27.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.383+0000 7f1e2cfe6700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e28066ee0 0x7f1e2810d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33216/0 (socket says 172.21.15.107:33216) 2024-04-22T23:41:27.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.383+0000 7f1e2cfe6700 1 -- 172.21.15.107:0/3637330549 learned_addr learned my addr 172.21.15.107:0/3637330549 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:41:27.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.383+0000 7f1e27fff700 1 --2- 172.21.15.107:0/3637330549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e280fffd0 0x7f1e2810dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:27.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.383+0000 7f1e2cfe6700 1 -- 172.21.15.107:0/3637330549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e280fffd0 msgr2=0x7f1e2810dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:27.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.383+0000 7f1e2cfe6700 1 --2- 172.21.15.107:0/3637330549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e280fffd0 0x7f1e2810dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:27.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.383+0000 7f1e2cfe6700 1 -- 172.21.15.107:0/3637330549 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e14005040 con 0x7f1e28066ee0 2024-04-22T23:41:27.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.383+0000 7f1e2cfe6700 1 --2- 172.21.15.107:0/3637330549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e28066ee0 0x7f1e2810d9a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c002a80 tx=0x7f1e1c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:27.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.384+0000 7f1e257fa700 1 -- 172.21.15.107:0/3637330549 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e1c007cc0 con 0x7f1e28066ee0 2024-04-22T23:41:27.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.384+0000 7f1e2f24a700 1 -- 172.21.15.107:0/3637330549 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e281130b0 con 0x7f1e28066ee0 2024-04-22T23:41:27.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.384+0000 7f1e257fa700 1 -- 172.21.15.107:0/3637330549 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1e1c00f040 con 0x7f1e28066ee0 2024-04-22T23:41:27.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.384+0000 7f1e257fa700 1 -- 172.21.15.107:0/3637330549 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e1c00da30 con 0x7f1e28066ee0 2024-04-22T23:41:27.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.384+0000 7f1e2f24a700 1 -- 172.21.15.107:0/3637330549 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e28113670 con 0x7f1e28066ee0 2024-04-22T23:41:27.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.385+0000 7f1e2f24a700 1 -- 172.21.15.107:0/3637330549 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e2804c710 con 0x7f1e28066ee0 2024-04-22T23:41:27.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.385+0000 7f1e257fa700 1 -- 172.21.15.107:0/3637330549 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f1e1c00db90 con 0x7f1e28066ee0 2024-04-22T23:41:27.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.386+0000 7f1e257fa700 1 --2- 172.21.15.107:0/3637330549 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f1e1806ec00 0x7f1e180710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:27.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.386+0000 7f1e257fa700 1 -- 172.21.15.107:0/3637330549 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f1e1c08e600 con 0x7f1e28066ee0 2024-04-22T23:41:27.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.386+0000 7f1e27fff700 1 --2- 172.21.15.107:0/3637330549 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f1e1806ec00 0x7f1e180710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:27.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.386+0000 7f1e27fff700 1 --2- 172.21.15.107:0/3637330549 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f1e1806ec00 0x7f1e180710b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1e1400b3b0 tx=0x7f1e14009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:27.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.391+0000 7f1e257fa700 1 -- 172.21.15.107:0/3637330549 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f1e1c05aee0 con 0x7f1e28066ee0 2024-04-22T23:41:27.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.610+0000 7f1e2f24a700 1 -- 172.21.15.107:0/3637330549 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1e280f8f70 con 0x7f1e1806ec00 2024-04-22T23:41:27.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.611+0000 7f1e257fa700 1 -- 172.21.15.107:0/3637330549 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f1e280f8f70 con 0x7f1e1806ec00 2024-04-22T23:41:27.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.613+0000 7f1e2f24a700 1 -- 172.21.15.107:0/3637330549 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f1e1806ec00 msgr2=0x7f1e180710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:27.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.613+0000 7f1e2f24a700 1 --2- 172.21.15.107:0/3637330549 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f1e1806ec00 0x7f1e180710b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1e1400b3b0 tx=0x7f1e14009e30 comp rx=0 tx=0).stop 2024-04-22T23:41:27.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.613+0000 7f1e2f24a700 1 -- 172.21.15.107:0/3637330549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e28066ee0 msgr2=0x7f1e2810d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:27.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.613+0000 7f1e2f24a700 1 --2- 172.21.15.107:0/3637330549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e28066ee0 0x7f1e2810d9a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c002a80 tx=0x7f1e1c004600 comp rx=0 tx=0).stop 2024-04-22T23:41:27.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.613+0000 7f1e2f24a700 1 -- 172.21.15.107:0/3637330549 shutdown_connections 2024-04-22T23:41:27.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.613+0000 7f1e2f24a700 1 --2- 172.21.15.107:0/3637330549 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f1e1806ec00 0x7f1e180710b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:27.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.613+0000 7f1e2f24a700 1 --2- 172.21.15.107:0/3637330549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e28066ee0 0x7f1e2810d9a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:27.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.613+0000 7f1e2f24a700 1 --2- 172.21.15.107:0/3637330549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1e280fffd0 0x7f1e2810dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:27.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.613+0000 7f1e2f24a700 1 -- 172.21.15.107:0/3637330549 >> 172.21.15.107:0/3637330549 conn(0x7f1e280f7480 msgr2=0x7f1e280f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:27.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.613+0000 7f1e2f24a700 1 -- 172.21.15.107:0/3637330549 shutdown_connections 2024-04-22T23:41:27.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.613+0000 7f1e2f24a700 1 -- 172.21.15.107:0/3637330549 wait complete. 2024-04-22T23:41:27.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.714+0000 7f6ecfe8a700 1 -- 172.21.15.107:0/2691530160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ec80fbcf0 msgr2=0x7f6ec80fc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:27.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.714+0000 7f6ecfe8a700 1 --2- 172.21.15.107:0/2691530160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ec80fbcf0 0x7f6ec80fc100 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8004940 tx=0x7f6eb8009e30 comp rx=0 tx=0).stop 2024-04-22T23:41:27.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.714+0000 7f6ecfe8a700 1 -- 172.21.15.107:0/2691530160 shutdown_connections 2024-04-22T23:41:27.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.714+0000 7f6ecfe8a700 1 --2- 172.21.15.107:0/2691530160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ec80fcfa0 0x7f6ec80fd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:27.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.714+0000 7f6ecfe8a700 1 --2- 172.21.15.107:0/2691530160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ec80fbcf0 0x7f6ec80fc100 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:27.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.714+0000 7f6ecfe8a700 1 -- 172.21.15.107:0/2691530160 >> 172.21.15.107:0/2691530160 conn(0x7f6ec80f7450 msgr2=0x7f6ec80f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:27.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.714+0000 7f6ecfe8a700 1 -- 172.21.15.107:0/2691530160 shutdown_connections 2024-04-22T23:41:27.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.714+0000 7f6ecfe8a700 1 -- 172.21.15.107:0/2691530160 wait complete. 2024-04-22T23:41:27.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.715+0000 7f6ecfe8a700 1 Processor -- start 2024-04-22T23:41:27.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.715+0000 7f6ecfe8a700 1 -- start start 2024-04-22T23:41:27.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.715+0000 7f6ecfe8a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ec80fbcf0 0x7f6ec8066d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:27.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.715+0000 7f6ecfe8a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ec80fcfa0 0x7f6ec8112a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:27.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.715+0000 7f6ecfe8a700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec8112fd0 con 0x7f6ec80fbcf0 2024-04-22T23:41:27.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.715+0000 7f6ecfe8a700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec8113110 con 0x7f6ec80fcfa0 2024-04-22T23:41:27.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.715+0000 7f6ecdc26700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ec80fbcf0 0x7f6ec8066d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:27.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.715+0000 7f6ecdc26700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ec80fbcf0 0x7f6ec8066d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33232/0 (socket says 172.21.15.107:33232) 2024-04-22T23:41:27.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.715+0000 7f6ecdc26700 1 -- 172.21.15.107:0/2819270664 learned_addr learned my addr 172.21.15.107:0/2819270664 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:41:27.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.715+0000 7f6ecd425700 1 --2- 172.21.15.107:0/2819270664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ec80fcfa0 0x7f6ec8112a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:27.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.715+0000 7f6ecdc26700 1 -- 172.21.15.107:0/2819270664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ec80fcfa0 msgr2=0x7f6ec8112a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:27.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.715+0000 7f6ecdc26700 1 --2- 172.21.15.107:0/2819270664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ec80fcfa0 0x7f6ec8112a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:27.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.715+0000 7f6ecdc26700 1 -- 172.21.15.107:0/2819270664 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb8005040 con 0x7f6ec80fbcf0 2024-04-22T23:41:27.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.715+0000 7f6ecdc26700 1 --2- 172.21.15.107:0/2819270664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ec80fbcf0 0x7f6ec8066d40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6eb800b3b0 tx=0x7f6eb8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:27.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.716+0000 7f6ebe7fc700 1 -- 172.21.15.107:0/2819270664 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6eb800e040 con 0x7f6ec80fbcf0 2024-04-22T23:41:27.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.716+0000 7f6ebe7fc700 1 -- 172.21.15.107:0/2819270664 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6eb800bcc0 con 0x7f6ec80fbcf0 2024-04-22T23:41:27.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.716+0000 7f6ebe7fc700 1 -- 172.21.15.107:0/2819270664 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6eb801c630 con 0x7f6ec80fbcf0 2024-04-22T23:41:27.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.716+0000 7f6ecfe8a700 1 -- 172.21.15.107:0/2819270664 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ec8113390 con 0x7f6ec80fbcf0 2024-04-22T23:41:27.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.716+0000 7f6ecfe8a700 1 -- 172.21.15.107:0/2819270664 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ec8113950 con 0x7f6ec80fbcf0 2024-04-22T23:41:27.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.717+0000 7f6ecfe8a700 1 -- 172.21.15.107:0/2819270664 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ec804c710 con 0x7f6ec80fbcf0 2024-04-22T23:41:27.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.717+0000 7f6ebe7fc700 1 -- 172.21.15.107:0/2819270664 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f6eb8013980 con 0x7f6ec80fbcf0 2024-04-22T23:41:27.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.718+0000 7f6ebe7fc700 1 --2- 172.21.15.107:0/2819270664 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f6eb406ec00 0x7f6eb40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:27.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.718+0000 7f6ebe7fc700 1 -- 172.21.15.107:0/2819270664 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f6eb808f4f0 con 0x7f6ec80fbcf0 2024-04-22T23:41:27.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.718+0000 7f6ecd425700 1 --2- 172.21.15.107:0/2819270664 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f6eb406ec00 0x7f6eb40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:27.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.718+0000 7f6ecd425700 1 --2- 172.21.15.107:0/2819270664 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f6eb406ec00 0x7f6eb40710b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4006fd0 tx=0x7f6ec4008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:27.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.723+0000 7f6ebe7fc700 1 -- 172.21.15.107:0/2819270664 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f6eb805be50 con 0x7f6ec80fbcf0 2024-04-22T23:41:27.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.941+0000 7f6ecfe8a700 1 -- 172.21.15.107:0/2819270664 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ec8101240 con 0x7f6eb406ec00 2024-04-22T23:41:27.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.951+0000 7f6ebe7fc700 1 -- 172.21.15.107:0/2819270664 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f6ec8101240 con 0x7f6eb406ec00 2024-04-22T23:41:27.944 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-22T23:41:27.944 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (4m) 52s ago 14m 25.6M - 0.25.0 c8568f914cd2 ec1a5c6260de 2024-04-22T23:41:27.944 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (2m) 52s ago 14m 6656k - 19.0.0-3252-g43be0201 a0434699c1d7 e074fd2cbb9c 2024-04-22T23:41:27.944 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi157 smithi157 running (2m) 18s ago 13m 6651k - 19.0.0-3252-g43be0201 a0434699c1d7 abc2649dee22 2024-04-22T23:41:27.944 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (4m) 52s ago 14m 79.9M - 9.4.7 954c08fa6188 31449ff442d2 2024-04-22T23:41:27.944 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.dfvvkw smithi107 running (9m) 52s ago 9m 13.7M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 08d3aa8d717d 2024-04-22T23:41:27.945 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.ktuxjl smithi107 running (8m) 52s ago 8m 124M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 762f27fe0f30 2024-04-22T23:41:27.945 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi157.rcaris smithi157 running (9m) 18s ago 9m 129M - 17.2.7-884-g70d0cb4b cafd8ddac8fa d23366ac085d 2024-04-22T23:41:27.945 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi157.ufywse smithi157 running (8m) 18s ago 8m 14.6M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 8b00d61eb355 2024-04-22T23:41:27.945 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.uipxvm smithi107 *:8443,9283,8765 running (4m) 52s ago 15m 502M - 19.0.0-3252-g43be0201 a0434699c1d7 d20f05dfe4f2 2024-04-22T23:41:27.945 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi157.tuylis smithi157 *:8443,9283,8765 running (3m) 18s ago 13m 451M - 19.0.0-3252-g43be0201 a0434699c1d7 957e598c9001 2024-04-22T23:41:27.945 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (3m) 52s ago 16m 49.4M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 bc72bbbe2c39 2024-04-22T23:41:27.945 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi157 smithi157 running (3m) 18s ago 13m 37.3M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 ea88de8be4fe 2024-04-22T23:41:27.945 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (4m) 52s ago 14m 16.2M - 1.5.0 0da6a335fe13 5205cfd9bd51 2024-04-22T23:41:27.945 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi157 smithi157 *:9100 running (4m) 18s ago 13m 16.9M - 1.5.0 0da6a335fe13 168ca191bb8c 2024-04-22T23:41:27.945 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (2m) 52s ago 12m 94.2M 1557M 19.0.0-3252-g43be0201 a0434699c1d7 fd0ba8ef0c27 2024-04-22T23:41:27.945 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (95s) 52s ago 12m 91.2M 1557M 19.0.0-3252-g43be0201 a0434699c1d7 77e33b6122e0 2024-04-22T23:41:27.945 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (58s) 52s ago 11m 15.0M 1557M 19.0.0-3252-g43be0201 a0434699c1d7 c307bd8cdcfc 2024-04-22T23:41:27.945 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi157 running (22s) 18s ago 11m 15.0M 2581M 19.0.0-3252-g43be0201 a0434699c1d7 e81ca519c831 2024-04-22T23:41:27.945 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi157 running (10m) 18s ago 10m 228M 2581M 17.2.7-884-g70d0cb4b cafd8ddac8fa 24a16e19951f 2024-04-22T23:41:27.945 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi157 running (10m) 18s ago 10m 225M 2581M 17.2.7-884-g70d0cb4b cafd8ddac8fa 4c5b7a7ef8bd 2024-04-22T23:41:27.946 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (4m) 52s ago 14m 58.8M - 2.43.0 a07b618ecd1d 99a4ea8594da 2024-04-22T23:41:27.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.954+0000 7f6ecfe8a700 1 -- 172.21.15.107:0/2819270664 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f6eb406ec00 msgr2=0x7f6eb40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:27.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.954+0000 7f6ecfe8a700 1 --2- 172.21.15.107:0/2819270664 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f6eb406ec00 0x7f6eb40710b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4006fd0 tx=0x7f6ec4008040 comp rx=0 tx=0).stop 2024-04-22T23:41:27.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.954+0000 7f6ecfe8a700 1 -- 172.21.15.107:0/2819270664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ec80fbcf0 msgr2=0x7f6ec8066d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:27.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.954+0000 7f6ecfe8a700 1 --2- 172.21.15.107:0/2819270664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ec80fbcf0 0x7f6ec8066d40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6eb800b3b0 tx=0x7f6eb8009e30 comp rx=0 tx=0).stop 2024-04-22T23:41:27.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.954+0000 7f6ecfe8a700 1 -- 172.21.15.107:0/2819270664 shutdown_connections 2024-04-22T23:41:27.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.954+0000 7f6ecfe8a700 1 --2- 172.21.15.107:0/2819270664 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f6eb406ec00 0x7f6eb40710b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:27.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.954+0000 7f6ecfe8a700 1 --2- 172.21.15.107:0/2819270664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ec80fbcf0 0x7f6ec8066d40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:27.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.954+0000 7f6ecfe8a700 1 --2- 172.21.15.107:0/2819270664 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ec80fcfa0 0x7f6ec8112a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:27.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.954+0000 7f6ecfe8a700 1 -- 172.21.15.107:0/2819270664 >> 172.21.15.107:0/2819270664 conn(0x7f6ec80f7450 msgr2=0x7f6ec80f8d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:27.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.954+0000 7f6ecfe8a700 1 -- 172.21.15.107:0/2819270664 shutdown_connections 2024-04-22T23:41:27.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:27.954+0000 7f6ecfe8a700 1 -- 172.21.15.107:0/2819270664 wait complete. 2024-04-22T23:41:28.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.055+0000 7f27f9cba700 1 -- 172.21.15.107:0/2943074859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27f40fff70 msgr2=0x7f27f4102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:28.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.055+0000 7f27f9cba700 1 --2- 172.21.15.107:0/2943074859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27f40fff70 0x7f27f4102390 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f27e8004940 tx=0x7f27e8009e30 comp rx=0 tx=0).stop 2024-04-22T23:41:28.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.055+0000 7f27f9cba700 1 -- 172.21.15.107:0/2943074859 shutdown_connections 2024-04-22T23:41:28.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.055+0000 7f27f9cba700 1 --2- 172.21.15.107:0/2943074859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27f40fff70 0x7f27f4102390 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:28.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.055+0000 7f27f9cba700 1 --2- 172.21.15.107:0/2943074859 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27f40662f0 0x7f27f40ffa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:28.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.055+0000 7f27f9cba700 1 -- 172.21.15.107:0/2943074859 >> 172.21.15.107:0/2943074859 conn(0x7f27f40f74c0 msgr2=0x7f27f40f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:28.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.055+0000 7f27f9cba700 1 -- 172.21.15.107:0/2943074859 shutdown_connections 2024-04-22T23:41:28.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.055+0000 7f27f9cba700 1 -- 172.21.15.107:0/2943074859 wait complete. 2024-04-22T23:41:28.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.055+0000 7f27f9cba700 1 Processor -- start 2024-04-22T23:41:28.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.055+0000 7f27f9cba700 1 -- start start 2024-04-22T23:41:28.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.055+0000 7f27f9cba700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27f40662f0 0x7f27f410dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:28.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.055+0000 7f27f9cba700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27f40fff70 0x7f27f410e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:28.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.055+0000 7f27f9cba700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f410e6b0 con 0x7f27f40fff70 2024-04-22T23:41:28.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.056+0000 7f27f9cba700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f410ee00 con 0x7f27f40662f0 2024-04-22T23:41:28.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.056+0000 7f27f2ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27f40fff70 0x7f27f410e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:28.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.056+0000 7f27f2ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27f40fff70 0x7f27f410e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33256/0 (socket says 172.21.15.107:33256) 2024-04-22T23:41:28.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.056+0000 7f27f2ffd700 1 -- 172.21.15.107:0/3794900917 learned_addr learned my addr 172.21.15.107:0/3794900917 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:41:28.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.056+0000 7f27f37fe700 1 --2- 172.21.15.107:0/3794900917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27f40662f0 0x7f27f410dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:28.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.056+0000 7f27f2ffd700 1 -- 172.21.15.107:0/3794900917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27f40662f0 msgr2=0x7f27f410dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:28.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.056+0000 7f27f2ffd700 1 --2- 172.21.15.107:0/3794900917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27f40662f0 0x7f27f410dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:28.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.056+0000 7f27f2ffd700 1 -- 172.21.15.107:0/3794900917 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e8005040 con 0x7f27f40fff70 2024-04-22T23:41:28.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.056+0000 7f27f2ffd700 1 --2- 172.21.15.107:0/3794900917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27f40fff70 0x7f27f410e020 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f27e8009150 tx=0x7f27e8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:28.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.056+0000 7f27dbfff700 1 -- 172.21.15.107:0/3794900917 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27e800e040 con 0x7f27f40fff70 2024-04-22T23:41:28.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.056+0000 7f27f9cba700 1 -- 172.21.15.107:0/3794900917 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27f410f080 con 0x7f27f40fff70 2024-04-22T23:41:28.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.056+0000 7f27dbfff700 1 -- 172.21.15.107:0/3794900917 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f27e800bbf0 con 0x7f27f40fff70 2024-04-22T23:41:28.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.056+0000 7f27dbfff700 1 -- 172.21.15.107:0/3794900917 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27e801c630 con 0x7f27f40fff70 2024-04-22T23:41:28.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.056+0000 7f27f9cba700 1 -- 172.21.15.107:0/3794900917 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27f410f640 con 0x7f27f40fff70 2024-04-22T23:41:28.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.057+0000 7f27f9cba700 1 -- 172.21.15.107:0/3794900917 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27f405f5f0 con 0x7f27f40fff70 2024-04-22T23:41:28.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.058+0000 7f27dbfff700 1 -- 172.21.15.107:0/3794900917 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f27e800bd60 con 0x7f27f40fff70 2024-04-22T23:41:28.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.058+0000 7f27dbfff700 1 --2- 172.21.15.107:0/3794900917 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f27dc06ec10 0x7f27dc0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:28.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.058+0000 7f27dbfff700 1 -- 172.21.15.107:0/3794900917 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f27e808fb50 con 0x7f27f40fff70 2024-04-22T23:41:28.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.058+0000 7f27f37fe700 1 --2- 172.21.15.107:0/3794900917 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f27dc06ec10 0x7f27dc0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:28.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.059+0000 7f27f37fe700 1 --2- 172.21.15.107:0/3794900917 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f27dc06ec10 0x7f27dc0710c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f27e4006fd0 tx=0x7f27e4008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:28.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.063+0000 7f27dbfff700 1 -- 172.21.15.107:0/3794900917 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f27e805c570 con 0x7f27f40fff70 2024-04-22T23:41:28.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.348+0000 7f27f9cba700 1 -- 172.21.15.107:0/3794900917 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f27f410f970 con 0x7f27f40fff70 2024-04-22T23:41:28.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.348+0000 7f27dbfff700 1 -- 172.21.15.107:0/3794900917 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f27e805c100 con 0x7f27f40fff70 2024-04-22T23:41:28.341 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-22T23:41:28.342 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-04-22T23:41:28.342 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:41:28.342 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:41:28.342 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-04-22T23:41:28.342 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:41:28.342 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:41:28.342 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-04-22T23:41:28.342 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 2, 2024-04-22T23:41:28.342 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 4 2024-04-22T23:41:28.342 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:41:28.342 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-04-22T23:41:28.342 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 4 2024-04-22T23:41:28.343 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:41:28.343 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-04-22T23:41:28.343 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 6, 2024-04-22T23:41:28.343 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 8 2024-04-22T23:41:28.343 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T23:41:28.343 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-22T23:41:28.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.350+0000 7f27f9cba700 1 -- 172.21.15.107:0/3794900917 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f27dc06ec10 msgr2=0x7f27dc0710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:28.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.351+0000 7f27f9cba700 1 --2- 172.21.15.107:0/3794900917 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f27dc06ec10 0x7f27dc0710c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f27e4006fd0 tx=0x7f27e4008040 comp rx=0 tx=0).stop 2024-04-22T23:41:28.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.351+0000 7f27f9cba700 1 -- 172.21.15.107:0/3794900917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27f40fff70 msgr2=0x7f27f410e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:28.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.351+0000 7f27f9cba700 1 --2- 172.21.15.107:0/3794900917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27f40fff70 0x7f27f410e020 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f27e8009150 tx=0x7f27e8009e30 comp rx=0 tx=0).stop 2024-04-22T23:41:28.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.351+0000 7f27f9cba700 1 -- 172.21.15.107:0/3794900917 shutdown_connections 2024-04-22T23:41:28.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.351+0000 7f27f9cba700 1 --2- 172.21.15.107:0/3794900917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27f40662f0 0x7f27f410dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:28.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.351+0000 7f27f9cba700 1 --2- 172.21.15.107:0/3794900917 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f27dc06ec10 0x7f27dc0710c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:28.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.351+0000 7f27f9cba700 1 --2- 172.21.15.107:0/3794900917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27f40fff70 0x7f27f410e020 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:28.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.351+0000 7f27f9cba700 1 -- 172.21.15.107:0/3794900917 >> 172.21.15.107:0/3794900917 conn(0x7f27f40f74c0 msgr2=0x7f27f40f7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:28.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.351+0000 7f27f9cba700 1 -- 172.21.15.107:0/3794900917 shutdown_connections 2024-04-22T23:41:28.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.351+0000 7f27f9cba700 1 -- 172.21.15.107:0/3794900917 wait complete. 2024-04-22T23:41:28.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.451+0000 7f7c3c425700 1 -- 172.21.15.107:0/2666318464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c34066fa0 msgr2=0x7f7c340ff9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:28.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.451+0000 7f7c3c425700 1 --2- 172.21.15.107:0/2666318464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c34066fa0 0x7f7c340ff9f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7c30004940 tx=0x7f7c30009e30 comp rx=0 tx=0).stop 2024-04-22T23:41:28.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.452+0000 7f7c3c425700 1 -- 172.21.15.107:0/2666318464 shutdown_connections 2024-04-22T23:41:28.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.452+0000 7f7c3c425700 1 --2- 172.21.15.107:0/2666318464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c340fff30 0x7f7c34102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:28.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.452+0000 7f7c3c425700 1 --2- 172.21.15.107:0/2666318464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c34066fa0 0x7f7c340ff9f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:28.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.452+0000 7f7c3c425700 1 -- 172.21.15.107:0/2666318464 >> 172.21.15.107:0/2666318464 conn(0x7f7c340f7480 msgr2=0x7f7c340f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:28.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.452+0000 7f7c3c425700 1 -- 172.21.15.107:0/2666318464 shutdown_connections 2024-04-22T23:41:28.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.452+0000 7f7c3c425700 1 -- 172.21.15.107:0/2666318464 wait complete. 2024-04-22T23:41:28.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.452+0000 7f7c3c425700 1 Processor -- start 2024-04-22T23:41:28.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.452+0000 7f7c3c425700 1 -- start start 2024-04-22T23:41:28.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.452+0000 7f7c3c425700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c34066fa0 0x7f7c340fd240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:28.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.452+0000 7f7c3c425700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c340fff30 0x7f7c340fb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:28.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.452+0000 7f7c3c425700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c340fd840 con 0x7f7c34066fa0 2024-04-22T23:41:28.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.452+0000 7f7c3c425700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c340fbe30 con 0x7f7c340fff30 2024-04-22T23:41:28.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.452+0000 7f7c3a1c1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c34066fa0 0x7f7c340fd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:28.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.452+0000 7f7c3a1c1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c34066fa0 0x7f7c340fd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33278/0 (socket says 172.21.15.107:33278) 2024-04-22T23:41:28.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.452+0000 7f7c3a1c1700 1 -- 172.21.15.107:0/1093569290 learned_addr learned my addr 172.21.15.107:0/1093569290 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:41:28.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.453+0000 7f7c399c0700 1 --2- 172.21.15.107:0/1093569290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c340fff30 0x7f7c340fb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:28.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.453+0000 7f7c3a1c1700 1 -- 172.21.15.107:0/1093569290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c340fff30 msgr2=0x7f7c340fb8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:28.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.453+0000 7f7c3a1c1700 1 --2- 172.21.15.107:0/1093569290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c340fff30 0x7f7c340fb8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:28.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.453+0000 7f7c3a1c1700 1 -- 172.21.15.107:0/1093569290 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c30005040 con 0x7f7c34066fa0 2024-04-22T23:41:28.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.453+0000 7f7c399c0700 1 --2- 172.21.15.107:0/1093569290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c340fff30 0x7f7c340fb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T23:41:28.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.453+0000 7f7c3a1c1700 1 --2- 172.21.15.107:0/1093569290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c34066fa0 0x7f7c340fd240 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7c3000b3b0 tx=0x7f7c30009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:28.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.453+0000 7f7c2affd700 1 -- 172.21.15.107:0/1093569290 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c3000e040 con 0x7f7c34066fa0 2024-04-22T23:41:28.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.453+0000 7f7c3c425700 1 -- 172.21.15.107:0/1093569290 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c340fc080 con 0x7f7c34066fa0 2024-04-22T23:41:28.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.453+0000 7f7c2affd700 1 -- 172.21.15.107:0/1093569290 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7c3000bcc0 con 0x7f7c34066fa0 2024-04-22T23:41:28.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.453+0000 7f7c2affd700 1 -- 172.21.15.107:0/1093569290 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c3001c630 con 0x7f7c34066fa0 2024-04-22T23:41:28.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.453+0000 7f7c3c425700 1 -- 172.21.15.107:0/1093569290 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c340fc640 con 0x7f7c34066fa0 2024-04-22T23:41:28.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.454+0000 7f7c3c425700 1 -- 172.21.15.107:0/1093569290 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c3404c710 con 0x7f7c34066fa0 2024-04-22T23:41:28.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.454+0000 7f7c2affd700 1 -- 172.21.15.107:0/1093569290 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f7c30013420 con 0x7f7c34066fa0 2024-04-22T23:41:28.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.455+0000 7f7c2affd700 1 --2- 172.21.15.107:0/1093569290 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f7c2006ec00 0x7f7c200710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:28.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.455+0000 7f7c2affd700 1 -- 172.21.15.107:0/1093569290 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f7c3008f520 con 0x7f7c34066fa0 2024-04-22T23:41:28.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.455+0000 7f7c399c0700 1 --2- 172.21.15.107:0/1093569290 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f7c2006ec00 0x7f7c200710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:28.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.455+0000 7f7c399c0700 1 --2- 172.21.15.107:0/1093569290 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f7c2006ec00 0x7f7c200710b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7c24009fd0 tx=0x7f7c2400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:28.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.460+0000 7f7c2affd700 1 -- 172.21.15.107:0/1093569290 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f7c3005be80 con 0x7f7c34066fa0 2024-04-22T23:41:28.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.701+0000 7f7c3c425700 1 -- 172.21.15.107:0/1093569290 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7c341058e0 con 0x7f7c34066fa0 2024-04-22T23:41:28.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.702+0000 7f7c2affd700 1 -- 172.21.15.107:0/1093569290 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1909 (secure 0 0 0) 0x7f7c3005ba10 con 0x7f7c34066fa0 2024-04-22T23:41:28.695 INFO:teuthology.orchestra.run.smithi107.stdout:e21 2024-04-22T23:41:28.695 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T23:41:28.695 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:41:28.695 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-04-22T23:41:28.695 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:41:28.695 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-04-22T23:41:28.695 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-04-22T23:41:28.695 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 21 2024-04-22T23:41:28.695 INFO:teuthology.orchestra.run.smithi107.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-22T23:41:28.695 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-04-22T23:32:20.146130+0000 2024-04-22T23:41:28.695 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-04-22T23:39:06.006385+0000 2024-04-22T23:41:28.696 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-04-22T23:41:28.696 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-04-22T23:41:28.696 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-04-22T23:41:28.696 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-04-22T23:41:28.696 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-04-22T23:41:28.696 INFO:teuthology.orchestra.run.smithi107.stdout:max_xattr_size 65536 2024-04-22T23:41:28.696 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-04-22T23:41:28.696 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-04-22T23:41:28.696 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 46 2024-04-22T23:41:28.696 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:41:28.696 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 2 2024-04-22T23:41:28.696 INFO:teuthology.orchestra.run.smithi107.stdout:in 0,1 2024-04-22T23:41:28.696 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=14468,1=24265} 2024-04-22T23:41:28.696 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-04-22T23:41:28.696 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-04-22T23:41:28.697 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-04-22T23:41:28.697 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-04-22T23:41:28.697 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-04-22T23:41:28.697 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data disabled 2024-04-22T23:41:28.697 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-04-22T23:41:28.697 INFO:teuthology.orchestra.run.smithi107.stdout:bal_rank_mask -1 2024-04-22T23:41:28.697 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-04-22T23:41:28.697 INFO:teuthology.orchestra.run.smithi107.stdout:qdb_cluster leader: 14468 members: 24265,14468 2024-04-22T23:41:28.697 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.ktuxjl{0:14468} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.107:6828/3335313253,v1:172.21.15.107:6829/3335313253] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:41:28.697 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi157.rcaris{1:24265} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.157:6824/2779251893,v1:172.21.15.157:6825/2779251893] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:41:28.697 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:41:28.697 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:41:28.697 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-04-22T23:41:28.697 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:41:28.697 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.dfvvkw{-1:14484} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/3548717247,v1:172.21.15.107:6827/3548717247] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:41:28.697 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi157.ufywse{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.157:6826/4072444199,v1:172.21.15.157:6827/4072444199] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:41:28.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.704+0000 7f7c3c425700 1 -- 172.21.15.107:0/1093569290 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f7c2006ec00 msgr2=0x7f7c200710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:28.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.704+0000 7f7c3c425700 1 --2- 172.21.15.107:0/1093569290 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f7c2006ec00 0x7f7c200710b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7c24009fd0 tx=0x7f7c2400b040 comp rx=0 tx=0).stop 2024-04-22T23:41:28.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.704+0000 7f7c3c425700 1 -- 172.21.15.107:0/1093569290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c34066fa0 msgr2=0x7f7c340fd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:28.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.704+0000 7f7c3c425700 1 --2- 172.21.15.107:0/1093569290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c34066fa0 0x7f7c340fd240 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7c3000b3b0 tx=0x7f7c30009e30 comp rx=0 tx=0).stop 2024-04-22T23:41:28.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.704+0000 7f7c3c425700 1 -- 172.21.15.107:0/1093569290 shutdown_connections 2024-04-22T23:41:28.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.704+0000 7f7c3c425700 1 --2- 172.21.15.107:0/1093569290 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f7c2006ec00 0x7f7c200710b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:28.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.704+0000 7f7c3c425700 1 --2- 172.21.15.107:0/1093569290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c34066fa0 0x7f7c340fd240 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:28.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.704+0000 7f7c3c425700 1 --2- 172.21.15.107:0/1093569290 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c340fff30 0x7f7c340fb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:28.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.704+0000 7f7c3c425700 1 -- 172.21.15.107:0/1093569290 >> 172.21.15.107:0/1093569290 conn(0x7f7c340f7480 msgr2=0x7f7c340f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:28.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.704+0000 7f7c3c425700 1 -- 172.21.15.107:0/1093569290 shutdown_connections 2024-04-22T23:41:28.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.704+0000 7f7c3c425700 1 -- 172.21.15.107:0/1093569290 wait complete. 2024-04-22T23:41:28.699 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 21 2024-04-22T23:41:28.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.807+0000 7ff23cb93700 1 -- 172.21.15.107:0/1601954917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2380fcff0 msgr2=0x7ff2380fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:28.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.807+0000 7ff23cb93700 1 --2- 172.21.15.107:0/1601954917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2380fcff0 0x7ff2380fd460 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff228004940 tx=0x7ff228009e30 comp rx=0 tx=0).stop 2024-04-22T23:41:28.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.807+0000 7ff23cb93700 1 -- 172.21.15.107:0/1601954917 shutdown_connections 2024-04-22T23:41:28.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.807+0000 7ff23cb93700 1 --2- 172.21.15.107:0/1601954917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2380fcff0 0x7ff2380fd460 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:28.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.807+0000 7ff23cb93700 1 --2- 172.21.15.107:0/1601954917 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2380fbd40 0x7ff2380fc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:28.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.807+0000 7ff23cb93700 1 -- 172.21.15.107:0/1601954917 >> 172.21.15.107:0/1601954917 conn(0x7ff2380f74c0 msgr2=0x7ff2380f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:28.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.807+0000 7ff23cb93700 1 -- 172.21.15.107:0/1601954917 shutdown_connections 2024-04-22T23:41:28.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.807+0000 7ff23cb93700 1 -- 172.21.15.107:0/1601954917 wait complete. 2024-04-22T23:41:28.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.807+0000 7ff23cb93700 1 Processor -- start 2024-04-22T23:41:28.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.808+0000 7ff23cb93700 1 -- start start 2024-04-22T23:41:28.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.808+0000 7ff23cb93700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2380fbd40 0x7ff23810da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:28.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.808+0000 7ff23cb93700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2380fcff0 0x7ff23810df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:28.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.808+0000 7ff23cb93700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff23810e610 con 0x7ff2380fcff0 2024-04-22T23:41:28.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.808+0000 7ff23cb93700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff23810ee00 con 0x7ff2380fbd40 2024-04-22T23:41:28.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.808+0000 7ff235d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2380fcff0 0x7ff23810df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:28.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.808+0000 7ff235d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2380fcff0 0x7ff23810df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33290/0 (socket says 172.21.15.107:33290) 2024-04-22T23:41:28.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.808+0000 7ff235d9b700 1 -- 172.21.15.107:0/1386521722 learned_addr learned my addr 172.21.15.107:0/1386521722 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:41:28.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.808+0000 7ff23659c700 1 --2- 172.21.15.107:0/1386521722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2380fbd40 0x7ff23810da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:28.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.808+0000 7ff235d9b700 1 -- 172.21.15.107:0/1386521722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2380fbd40 msgr2=0x7ff23810da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:28.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.808+0000 7ff235d9b700 1 --2- 172.21.15.107:0/1386521722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2380fbd40 0x7ff23810da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:28.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.808+0000 7ff235d9b700 1 -- 172.21.15.107:0/1386521722 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff228005040 con 0x7ff2380fcff0 2024-04-22T23:41:28.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.808+0000 7ff235d9b700 1 --2- 172.21.15.107:0/1386521722 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2380fcff0 0x7ff23810df80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff22800b3b0 tx=0x7ff228009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:28.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.808+0000 7ff226ffd700 1 -- 172.21.15.107:0/1386521722 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff22800e040 con 0x7ff2380fcff0 2024-04-22T23:41:28.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.808+0000 7ff23cb93700 1 -- 172.21.15.107:0/1386521722 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff23810f080 con 0x7ff2380fcff0 2024-04-22T23:41:28.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.809+0000 7ff226ffd700 1 -- 172.21.15.107:0/1386521722 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff22800bbf0 con 0x7ff2380fcff0 2024-04-22T23:41:28.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.809+0000 7ff226ffd700 1 -- 172.21.15.107:0/1386521722 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff22801c630 con 0x7ff2380fcff0 2024-04-22T23:41:28.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.809+0000 7ff23cb93700 1 -- 172.21.15.107:0/1386521722 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff23810f640 con 0x7ff2380fcff0 2024-04-22T23:41:28.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.810+0000 7ff23cb93700 1 -- 172.21.15.107:0/1386521722 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff238101b80 con 0x7ff2380fcff0 2024-04-22T23:41:28.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.810+0000 7ff226ffd700 1 -- 172.21.15.107:0/1386521722 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7ff22800bd60 con 0x7ff2380fcff0 2024-04-22T23:41:28.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.810+0000 7ff226ffd700 1 --2- 172.21.15.107:0/1386521722 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7ff22006ec00 0x7ff2200710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:28.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.811+0000 7ff226ffd700 1 -- 172.21.15.107:0/1386521722 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6496+0+0 (secure 0 0 0) 0x7ff22808f360 con 0x7ff2380fcff0 2024-04-22T23:41:28.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.811+0000 7ff23659c700 1 --2- 172.21.15.107:0/1386521722 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7ff22006ec00 0x7ff2200710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:28.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.811+0000 7ff23659c700 1 --2- 172.21.15.107:0/1386521722 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7ff22006ec00 0x7ff2200710b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff22c009bf0 tx=0x7ff22c0091a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:28.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:28.815+0000 7ff226ffd700 1 -- 172.21.15.107:0/1386521722 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7ff22805bd80 con 0x7ff2380fcff0 2024-04-22T23:41:28.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:28 smithi107 ceph-mon[90179]: from='client.34314 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:41:28.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:28 smithi107 ceph-mon[90179]: from='client.34318 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:41:28.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:28 smithi107 ceph-mon[90179]: pgmap v129: 65 pgs: 65 active+clean; 277 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:41:28.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:28 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/3794900917' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:41:28.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:28 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/1093569290' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:41:29.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.036+0000 7ff23cb93700 1 -- 172.21.15.107:0/1386521722 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff2381049a0 con 0x7ff22006ec00 2024-04-22T23:41:29.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.037+0000 7ff226ffd700 1 -- 172.21.15.107:0/1386521722 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7ff2381049a0 con 0x7ff22006ec00 2024-04-22T23:41:29.030 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-22T23:41:29.030 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c", 2024-04-22T23:41:29.030 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-04-22T23:41:29.030 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-22T23:41:29.031 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [ 2024-04-22T23:41:29.031 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr", 2024-04-22T23:41:29.031 INFO:teuthology.orchestra.run.smithi107.stdout: "crash", 2024-04-22T23:41:29.031 INFO:teuthology.orchestra.run.smithi107.stdout: "mon" 2024-04-22T23:41:29.031 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-22T23:41:29.031 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "10/21 daemons upgraded", 2024-04-22T23:41:29.031 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Currently upgrading osd daemons", 2024-04-22T23:41:29.031 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-04-22T23:41:29.031 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-22T23:41:29.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.039+0000 7ff23cb93700 1 -- 172.21.15.107:0/1386521722 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7ff22006ec00 msgr2=0x7ff2200710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:29.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.039+0000 7ff23cb93700 1 --2- 172.21.15.107:0/1386521722 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7ff22006ec00 0x7ff2200710b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff22c009bf0 tx=0x7ff22c0091a0 comp rx=0 tx=0).stop 2024-04-22T23:41:29.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.039+0000 7ff23cb93700 1 -- 172.21.15.107:0/1386521722 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2380fcff0 msgr2=0x7ff23810df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:29.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.039+0000 7ff23cb93700 1 --2- 172.21.15.107:0/1386521722 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2380fcff0 0x7ff23810df80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff22800b3b0 tx=0x7ff228009e30 comp rx=0 tx=0).stop 2024-04-22T23:41:29.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.039+0000 7ff23cb93700 1 -- 172.21.15.107:0/1386521722 shutdown_connections 2024-04-22T23:41:29.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.039+0000 7ff23cb93700 1 --2- 172.21.15.107:0/1386521722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff2380fbd40 0x7ff23810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:29.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.039+0000 7ff23cb93700 1 --2- 172.21.15.107:0/1386521722 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7ff22006ec00 0x7ff2200710b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:29.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.039+0000 7ff23cb93700 1 --2- 172.21.15.107:0/1386521722 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2380fcff0 0x7ff23810df80 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:29.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.039+0000 7ff23cb93700 1 -- 172.21.15.107:0/1386521722 >> 172.21.15.107:0/1386521722 conn(0x7ff2380f74c0 msgr2=0x7ff238065b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:29.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.039+0000 7ff23cb93700 1 -- 172.21.15.107:0/1386521722 shutdown_connections 2024-04-22T23:41:29.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.039+0000 7ff23cb93700 1 -- 172.21.15.107:0/1386521722 wait complete. 2024-04-22T23:41:29.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:28 smithi157 ceph-mon[79403]: from='client.34314 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:41:29.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:28 smithi157 ceph-mon[79403]: from='client.34318 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:41:29.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:28 smithi157 ceph-mon[79403]: pgmap v129: 65 pgs: 65 active+clean; 277 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:41:29.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:28 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/3794900917' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:41:29.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:28 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/1093569290' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:41:29.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.139+0000 7fc0f0976700 1 -- 172.21.15.107:0/1160449675 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0ec0fbd40 msgr2=0x7fc0ec0fc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:29.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.139+0000 7fc0f0976700 1 --2- 172.21.15.107:0/1160449675 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0ec0fbd40 0x7fc0ec0fc150 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4009ce0 tx=0x7fc0e4004d10 comp rx=0 tx=0).stop 2024-04-22T23:41:29.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.139+0000 7fc0f0976700 1 -- 172.21.15.107:0/1160449675 shutdown_connections 2024-04-22T23:41:29.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.139+0000 7fc0f0976700 1 --2- 172.21.15.107:0/1160449675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0ec0fcff0 0x7fc0ec0fd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:29.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.139+0000 7fc0f0976700 1 --2- 172.21.15.107:0/1160449675 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0ec0fbd40 0x7fc0ec0fc150 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:29.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.139+0000 7fc0f0976700 1 -- 172.21.15.107:0/1160449675 >> 172.21.15.107:0/1160449675 conn(0x7fc0ec0f74c0 msgr2=0x7fc0ec0f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:29.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.139+0000 7fc0f0976700 1 -- 172.21.15.107:0/1160449675 shutdown_connections 2024-04-22T23:41:29.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.139+0000 7fc0f0976700 1 -- 172.21.15.107:0/1160449675 wait complete. 2024-04-22T23:41:29.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.139+0000 7fc0f0976700 1 Processor -- start 2024-04-22T23:41:29.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.139+0000 7fc0f0976700 1 -- start start 2024-04-22T23:41:29.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.139+0000 7fc0f0976700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0ec0fbd40 0x7fc0ec110880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:29.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.139+0000 7fc0f0976700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0ec0fcff0 0x7fc0ec112dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:29.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.140+0000 7fc0f0976700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ec066360 con 0x7fc0ec0fbd40 2024-04-22T23:41:29.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.140+0000 7fc0f0976700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ec0664a0 con 0x7fc0ec0fcff0 2024-04-22T23:41:29.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.140+0000 7fc0ea59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0ec0fbd40 0x7fc0ec110880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:29.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.140+0000 7fc0ea59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0ec0fbd40 0x7fc0ec110880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33316/0 (socket says 172.21.15.107:33316) 2024-04-22T23:41:29.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.140+0000 7fc0ea59c700 1 -- 172.21.15.107:0/2176353074 learned_addr learned my addr 172.21.15.107:0/2176353074 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:41:29.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.140+0000 7fc0e9d9b700 1 --2- 172.21.15.107:0/2176353074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0ec0fcff0 0x7fc0ec112dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:29.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.140+0000 7fc0ea59c700 1 -- 172.21.15.107:0/2176353074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0ec0fcff0 msgr2=0x7fc0ec112dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:29.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.140+0000 7fc0ea59c700 1 --2- 172.21.15.107:0/2176353074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0ec0fcff0 0x7fc0ec112dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:29.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.140+0000 7fc0ea59c700 1 -- 172.21.15.107:0/2176353074 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0e4009940 con 0x7fc0ec0fbd40 2024-04-22T23:41:29.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.140+0000 7fc0e9d9b700 1 --2- 172.21.15.107:0/2176353074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0ec0fcff0 0x7fc0ec112dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T23:41:29.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.140+0000 7fc0ea59c700 1 --2- 172.21.15.107:0/2176353074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0ec0fbd40 0x7fc0ec110880 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4005010 tx=0x7fc0e4004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:29.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.140+0000 7fc0daffd700 1 -- 172.21.15.107:0/2176353074 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0e400e040 con 0x7fc0ec0fbd40 2024-04-22T23:41:29.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.140+0000 7fc0f0976700 1 -- 172.21.15.107:0/2176353074 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0ec113310 con 0x7fc0ec0fbd40 2024-04-22T23:41:29.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.140+0000 7fc0daffd700 1 -- 172.21.15.107:0/2176353074 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc0e4008ed0 con 0x7fc0ec0fbd40 2024-04-22T23:41:29.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.140+0000 7fc0daffd700 1 -- 172.21.15.107:0/2176353074 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0e401c870 con 0x7fc0ec0fbd40 2024-04-22T23:41:29.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.140+0000 7fc0f0976700 1 -- 172.21.15.107:0/2176353074 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0ec1138d0 con 0x7fc0ec0fbd40 2024-04-22T23:41:29.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.141+0000 7fc0f0976700 1 -- 172.21.15.107:0/2176353074 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0ec05f5f0 con 0x7fc0ec0fbd40 2024-04-22T23:41:29.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.142+0000 7fc0daffd700 1 -- 172.21.15.107:0/2176353074 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fc0e4012c70 con 0x7fc0ec0fbd40 2024-04-22T23:41:29.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.142+0000 7fc0daffd700 1 --2- 172.21.15.107:0/2176353074 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fc0d406ec10 0x7fc0d40710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:29.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.142+0000 7fc0daffd700 1 -- 172.21.15.107:0/2176353074 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fc0e4090390 con 0x7fc0ec0fbd40 2024-04-22T23:41:29.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.142+0000 7fc0e9d9b700 1 --2- 172.21.15.107:0/2176353074 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fc0d406ec10 0x7fc0d40710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:29.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.143+0000 7fc0e9d9b700 1 --2- 172.21.15.107:0/2176353074 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fc0d406ec10 0x7fc0d40710c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc007fd0 tx=0x7fc0dc009f60 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:29.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.147+0000 7fc0daffd700 1 -- 172.21.15.107:0/2176353074 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fc0e40591c0 con 0x7fc0ec0fbd40 2024-04-22T23:41:29.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.429+0000 7fc0f0976700 1 -- 172.21.15.107:0/2176353074 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc0ec04bed0 con 0x7fc0ec0fbd40 2024-04-22T23:41:29.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.429+0000 7fc0daffd700 1 -- 172.21.15.107:0/2176353074 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fc0e4004e30 con 0x7fc0ec0fbd40 2024-04-22T23:41:29.423 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_OK 2024-04-22T23:41:29.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.432+0000 7fc0f0976700 1 -- 172.21.15.107:0/2176353074 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fc0d406ec10 msgr2=0x7fc0d40710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:29.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.432+0000 7fc0f0976700 1 --2- 172.21.15.107:0/2176353074 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fc0d406ec10 0x7fc0d40710c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc007fd0 tx=0x7fc0dc009f60 comp rx=0 tx=0).stop 2024-04-22T23:41:29.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.432+0000 7fc0f0976700 1 -- 172.21.15.107:0/2176353074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0ec0fbd40 msgr2=0x7fc0ec110880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:29.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.432+0000 7fc0f0976700 1 --2- 172.21.15.107:0/2176353074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0ec0fbd40 0x7fc0ec110880 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4005010 tx=0x7fc0e4004d10 comp rx=0 tx=0).stop 2024-04-22T23:41:29.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.432+0000 7fc0f0976700 1 -- 172.21.15.107:0/2176353074 shutdown_connections 2024-04-22T23:41:29.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.432+0000 7fc0f0976700 1 --2- 172.21.15.107:0/2176353074 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fc0d406ec10 0x7fc0d40710c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:29.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.432+0000 7fc0f0976700 1 --2- 172.21.15.107:0/2176353074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0ec0fbd40 0x7fc0ec110880 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:29.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.432+0000 7fc0f0976700 1 --2- 172.21.15.107:0/2176353074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc0ec0fcff0 0x7fc0ec112dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:29.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.432+0000 7fc0f0976700 1 -- 172.21.15.107:0/2176353074 >> 172.21.15.107:0/2176353074 conn(0x7fc0ec0f74c0 msgr2=0x7fc0ec0f8bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:29.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.432+0000 7fc0f0976700 1 -- 172.21.15.107:0/2176353074 shutdown_connections 2024-04-22T23:41:29.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:29.432+0000 7fc0f0976700 1 -- 172.21.15.107:0/2176353074 wait complete. 2024-04-22T23:41:29.867 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:29 smithi157 ceph-mon[79403]: from='client.34322 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:41:29.868 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:29 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-22T23:41:29.868 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:29 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/2176353074' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:41:29.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:29 smithi107 ceph-mon[90179]: from='client.34322 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:41:29.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:29 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-22T23:41:29.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:29 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/2176353074' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:41:30.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:30 smithi157 ceph-mon[79403]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-22T23:41:30.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:30 smithi157 ceph-mon[79403]: Upgrade: osd.4 is safe to restart 2024-04-22T23:41:30.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:30 smithi157 ceph-mon[79403]: from='client.34334 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:41:30.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:30 smithi157 ceph-mon[79403]: pgmap v130: 65 pgs: 65 active+clean; 277 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:41:30.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:30 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:41:30.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:30 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-22T23:41:30.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:30 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:41:30.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:30 smithi107 ceph-mon[90179]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-22T23:41:30.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:30 smithi107 ceph-mon[90179]: Upgrade: osd.4 is safe to restart 2024-04-22T23:41:30.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:30 smithi107 ceph-mon[90179]: from='client.34334 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:41:30.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:30 smithi107 ceph-mon[90179]: pgmap v130: 65 pgs: 65 active+clean; 277 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:41:30.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:30 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:41:30.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:30 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-22T23:41:30.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:30 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:41:31.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:31 smithi107 ceph-mon[90179]: Upgrade: Updating osd.4 2024-04-22T23:41:31.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:31 smithi107 ceph-mon[90179]: Deploying daemon osd.4 on smithi157 2024-04-22T23:41:32.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:31 smithi157 ceph-mon[79403]: Upgrade: Updating osd.4 2024-04-22T23:41:32.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:31 smithi157 ceph-mon[79403]: Deploying daemon osd.4 on smithi157 2024-04-22T23:41:32.867 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:32 smithi157 ceph-mon[79403]: pgmap v131: 65 pgs: 65 active+clean; 277 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:41:33.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:32 smithi107 ceph-mon[90179]: pgmap v131: 65 pgs: 65 active+clean; 277 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:41:34.075 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:41:33 smithi157 systemd[1]: Stopping Ceph osd.4 for 8252f6f2-00ff-11ef-bc93-c7b262605968... 2024-04-22T23:41:34.380 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:41:34 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4[46184]: 2024-04-22T23:41:34.097+0000 7f1dc5ae2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T23:41:34.380 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:41:34 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4[46184]: 2024-04-22T23:41:34.097+0000 7f1dc5ae2700 -1 osd.4 69 *** Got signal Terminated *** 2024-04-22T23:41:34.380 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:41:34 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4[46184]: 2024-04-22T23:41:34.097+0000 7f1dc5ae2700 -1 osd.4 69 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T23:41:35.025 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:34 smithi157 ceph-mon[79403]: pgmap v132: 65 pgs: 65 active+clean; 277 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:41:35.025 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:34 smithi157 ceph-mon[79403]: osd.4 marked itself down and dead 2024-04-22T23:41:35.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:34 smithi107 ceph-mon[90179]: pgmap v132: 65 pgs: 65 active+clean; 277 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:41:35.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:34 smithi107 ceph-mon[90179]: osd.4 marked itself down and dead 2024-04-22T23:41:35.630 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:41:35 smithi157 podman[93517]: 2024-04-22 23:41:35.289962348 +0000 UTC m=+1.313119956 container died 24a16e19951f662e6c8f870fa7e4c169a0814d86300976ff9cc936e84e83e90a (image=quay.ceph.io/ceph-ci/ceph@sha256:3890cd726bd5e73ae623dabdf4ed76dd5df81c936b8bd8d761026babda969466, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=quincy-70d0cb4, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-22T23:41:36.108 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:35 smithi157 ceph-mon[79403]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-22T23:41:36.108 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:35 smithi157 ceph-mon[79403]: osdmap e70: 6 total, 5 up, 6 in 2024-04-22T23:41:36.108 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:41:35 smithi157 podman[93517]: 2024-04-22 23:41:35.856535928 +0000 UTC m=+1.879693537 container cleanup 24a16e19951f662e6c8f870fa7e4c169a0814d86300976ff9cc936e84e83e90a (image=quay.ceph.io/ceph-ci/ceph@sha256:3890cd726bd5e73ae623dabdf4ed76dd5df81c936b8bd8d761026babda969466, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=quincy-70d0cb4, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416) 2024-04-22T23:41:36.109 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:41:35 smithi157 bash[93517]: ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4 2024-04-22T23:41:36.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:35 smithi107 ceph-mon[90179]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-22T23:41:36.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:35 smithi107 ceph-mon[90179]: osdmap e70: 6 total, 5 up, 6 in 2024-04-22T23:41:36.380 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:41:36 smithi157 podman[93549]: 2024-04-22 23:41:36.115961005 +0000 UTC m=+0.829139978 container remove 24a16e19951f662e6c8f870fa7e4c169a0814d86300976ff9cc936e84e83e90a (image=quay.ceph.io/ceph-ci/ceph@sha256:3890cd726bd5e73ae623dabdf4ed76dd5df81c936b8bd8d761026babda969466, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=quincy-70d0cb4, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T23:41:36.748 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:41:36 smithi157 podman[93640]: 2024-04-22 23:41:36.377857628 +0000 UTC m=+0.017906902 image pull a0434699c1d76af4eea1fc95ff0a22ea73d1e04e609464fbf658d6f39bc7f2fc quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c 2024-04-22T23:41:36.748 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:41:36 smithi157 podman[93640]: 2024-04-22 23:41:36.50033664 +0000 UTC m=+0.140385899 container create 93bcf863bb35d0ae1614d90606de7068fa76e63cad41a3f45302358966ebcf68 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True) 2024-04-22T23:41:36.748 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:41:36 smithi157 podman[93640]: 2024-04-22 23:41:36.70290205 +0000 UTC m=+0.342951320 container init 93bcf863bb35d0ae1614d90606de7068fa76e63cad41a3f45302358966ebcf68 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416) 2024-04-22T23:41:36.749 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:41:36 smithi157 podman[93640]: 2024-04-22 23:41:36.706512554 +0000 UTC m=+0.346561820 container start 93bcf863bb35d0ae1614d90606de7068fa76e63cad41a3f45302358966ebcf68 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T23:41:37.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:36 smithi157 ceph-mon[79403]: pgmap v134: 65 pgs: 6 stale+active+clean, 59 active+clean; 277 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:41:37.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:36 smithi157 ceph-mon[79403]: osdmap e71: 6 total, 5 up, 6 in 2024-04-22T23:41:37.131 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:41:36 smithi157 podman[93640]: 2024-04-22 23:41:36.797495565 +0000 UTC m=+0.437544834 container attach 93bcf863bb35d0ae1614d90606de7068fa76e63cad41a3f45302358966ebcf68 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T23:41:37.131 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:41:37 smithi157 podman[93640]: 2024-04-22 23:41:37.002517018 +0000 UTC m=+0.642566285 container died 93bcf863bb35d0ae1614d90606de7068fa76e63cad41a3f45302358966ebcf68 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_BRANCH=HEAD) 2024-04-22T23:41:37.201 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:36 smithi107 ceph-mon[90179]: pgmap v134: 65 pgs: 6 stale+active+clean, 59 active+clean; 277 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:41:37.201 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:36 smithi107 ceph-mon[90179]: osdmap e71: 6 total, 5 up, 6 in 2024-04-22T23:41:37.800 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:41:37 smithi157 podman[93666]: 2024-04-22 23:41:37.589785783 +0000 UTC m=+0.577812685 container remove 93bcf863bb35d0ae1614d90606de7068fa76e63cad41a3f45302358966ebcf68 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4-deactivate, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T23:41:37.800 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:41:37 smithi157 systemd[1]: ceph-8252f6f2-00ff-11ef-bc93-c7b262605968@osd.4.service: Deactivated successfully. 2024-04-22T23:41:37.800 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:41:37 smithi157 systemd[1]: Stopped Ceph osd.4 for 8252f6f2-00ff-11ef-bc93-c7b262605968. 2024-04-22T23:41:37.800 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:41:37 smithi157 systemd[1]: ceph-8252f6f2-00ff-11ef-bc93-c7b262605968@osd.4.service: Consumed 55.069s CPU time. 2024-04-22T23:41:38.058 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:37 smithi157 ceph-mon[79403]: pgmap v136: 65 pgs: 8 active+undersized+degraded, 3 stale+active+clean, 10 active+undersized, 44 active+clean; 277 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 31/339 objects degraded (9.145%) 2024-04-22T23:41:38.058 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:37 smithi157 ceph-mon[79403]: Health check failed: Degraded data redundancy: 31/339 objects degraded (9.145%), 8 pgs degraded (PG_DEGRADED) 2024-04-22T23:41:38.058 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:41:37 smithi157 systemd[1]: Starting Ceph osd.4 for 8252f6f2-00ff-11ef-bc93-c7b262605968... 2024-04-22T23:41:38.058 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:41:38 smithi157 podman[93771]: 2024-04-22 23:41:37.939549096 +0000 UTC m=+0.017995167 image pull a0434699c1d76af4eea1fc95ff0a22ea73d1e04e609464fbf658d6f39bc7f2fc quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c 2024-04-22T23:41:38.058 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:41:38 smithi157 podman[93771]: 2024-04-22 23:41:38.065731098 +0000 UTC m=+0.144177184 container create 0185cb1c426725a74e5651e6f147fc8cf853f561530c90485bae939817792b20 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4-activate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-22T23:41:38.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:37 smithi107 ceph-mon[90179]: pgmap v136: 65 pgs: 8 active+undersized+degraded, 3 stale+active+clean, 10 active+undersized, 44 active+clean; 277 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 31/339 objects degraded (9.145%) 2024-04-22T23:41:38.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:37 smithi107 ceph-mon[90179]: Health check failed: Degraded data redundancy: 31/339 objects degraded (9.145%), 8 pgs degraded (PG_DEGRADED) 2024-04-22T23:41:38.377 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:41:38 smithi157 podman[93771]: 2024-04-22 23:41:38.301810855 +0000 UTC m=+0.380256908 container init 0185cb1c426725a74e5651e6f147fc8cf853f561530c90485bae939817792b20 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4-activate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-adk-testing-2024-04-22-1618-43be020) 2024-04-22T23:41:38.377 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:41:38 smithi157 podman[93771]: 2024-04-22 23:41:38.305087003 +0000 UTC m=+0.383533053 container start 0185cb1c426725a74e5651e6f147fc8cf853f561530c90485bae939817792b20 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4-activate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-22T23:41:38.378 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:41:38 smithi157 podman[93771]: 2024-04-22 23:41:38.385045905 +0000 UTC m=+0.463491964 container attach 0185cb1c426725a74e5651e6f147fc8cf853f561530c90485bae939817792b20 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4-activate, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-adk-testing-2024-04-22-1618-43be020) 2024-04-22T23:41:39.097 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:41:38 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4-activate[93786]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-22T23:41:39.097 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:41:38 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4-activate[93786]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-22T23:41:39.097 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:41:38 smithi157 bash[93771]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-22T23:41:39.098 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:41:38 smithi157 bash[93771]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-22T23:41:39.098 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:41:38 smithi157 bash[93771]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-22T23:41:39.098 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:41:38 smithi157 bash[93771]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-22T23:41:39.098 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:41:38 smithi157 bash[93771]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-22T23:41:39.098 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:41:38 smithi157 bash[93771]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-22T23:41:39.098 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:41:38 smithi157 bash[93771]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-04-22T23:41:39.098 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:41:38 smithi157 bash[93771]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-22T23:41:39.098 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:41:38 smithi157 bash[93771]: --> ceph-volume raw activate successful for osd ID: 4 2024-04-22T23:41:39.098 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:41:38 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4-activate[93786]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-22T23:41:39.098 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:41:38 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4-activate[93786]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-22T23:41:39.098 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:41:38 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4-activate[93786]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-22T23:41:39.098 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:41:38 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4-activate[93786]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-22T23:41:39.098 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:41:38 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4-activate[93786]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-04-22T23:41:39.099 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:41:38 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4-activate[93786]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-22T23:41:39.099 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:41:38 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4-activate[93786]: --> ceph-volume raw activate successful for osd ID: 4 2024-04-22T23:41:39.099 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:41:38 smithi157 podman[93771]: 2024-04-22 23:41:38.81159836 +0000 UTC m=+0.890044412 container died 0185cb1c426725a74e5651e6f147fc8cf853f561530c90485bae939817792b20 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4-activate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-adk-testing-2024-04-22-1618-43be020) 2024-04-22T23:41:39.682 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:41:39 smithi157 podman[93927]: 2024-04-22 23:41:39.430472602 +0000 UTC m=+0.612395730 container remove 0185cb1c426725a74e5651e6f147fc8cf853f561530c90485bae939817792b20 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4-activate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-22T23:41:39.683 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:41:39 smithi157 podman[93991]: 2024-04-22 23:41:39.547036883 +0000 UTC m=+0.017981983 image pull a0434699c1d76af4eea1fc95ff0a22ea73d1e04e609464fbf658d6f39bc7f2fc quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c 2024-04-22T23:41:39.683 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:41:39 smithi157 podman[93991]: 2024-04-22 23:41:39.6905285 +0000 UTC m=+0.161473571 container create d5b5a2e4781af2f41eab518af21506bfa1e3fb42f5f040299d734b29679c4e5b (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T23:41:40.001 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:41:39 smithi157 podman[93991]: 2024-04-22 23:41:39.909890935 +0000 UTC m=+0.380836008 container init d5b5a2e4781af2f41eab518af21506bfa1e3fb42f5f040299d734b29679c4e5b (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-22T23:41:40.001 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:41:39 smithi157 podman[93991]: 2024-04-22 23:41:39.912828818 +0000 UTC m=+0.383773891 container start d5b5a2e4781af2f41eab518af21506bfa1e3fb42f5f040299d734b29679c4e5b (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-22T23:41:40.001 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:41:39 smithi157 ceph-osd[94011]: -- 172.21.15.157:0/3183229736 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e8792fd680 con 0x55e8792c7000 2024-04-22T23:41:40.380 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:41:40 smithi157 bash[93991]: d5b5a2e4781af2f41eab518af21506bfa1e3fb42f5f040299d734b29679c4e5b 2024-04-22T23:41:40.380 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:41:40 smithi157 systemd[1]: Started Ceph osd.4 for 8252f6f2-00ff-11ef-bc93-c7b262605968. 2024-04-22T23:41:40.745 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:41:40 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4[94007]: 2024-04-22T23:41:40.454+0000 7f30e6e657c0 -1 Falling back to public interface 2024-04-22T23:41:41.056 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:40 smithi157 ceph-mon[79403]: pgmap v137: 65 pgs: 12 active+undersized+degraded, 1 stale+active+clean, 14 active+undersized, 38 active+clean; 277 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 45/339 objects degraded (13.274%) 2024-04-22T23:41:41.056 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:40 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:41:41.056 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:40 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:41:41.056 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:40 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:41:41.056 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:40 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:41:41.056 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:40 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:41:41.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:40 smithi107 ceph-mon[90179]: pgmap v137: 65 pgs: 12 active+undersized+degraded, 1 stale+active+clean, 14 active+undersized, 38 active+clean; 277 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 45/339 objects degraded (13.274%) 2024-04-22T23:41:41.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:40 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:41:41.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:40 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:41:41.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:40 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:41:41.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:40 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:41:41.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:40 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:41:42.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:42 smithi157 ceph-mon[79403]: pgmap v138: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 277 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 57/339 objects degraded (16.814%) 2024-04-22T23:41:43.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:42 smithi107 ceph-mon[90179]: pgmap v138: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 277 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 57/339 objects degraded (16.814%) 2024-04-22T23:41:44.380 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:41:44.380 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:41:44.380 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:44 smithi157 ceph-mon[79403]: pgmap v139: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 277 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 57/339 objects degraded (16.814%) 2024-04-22T23:41:44.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:41:44.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:41:44.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:44 smithi107 ceph-mon[90179]: pgmap v139: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 277 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 57/339 objects degraded (16.814%) 2024-04-22T23:41:45.380 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:45 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:41:45.380 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:45 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:41:45.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:45 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:41:45.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:45 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:41:45.777 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:41:45 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4[94007]: 2024-04-22T23:41:45.523+0000 7f30e6e657c0 -1 osd.4 69 log_to_monitors true 2024-04-22T23:41:46.380 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:46 smithi157 ceph-mon[79403]: from='osd.4 [v2:172.21.15.157:6808/2579998472,v1:172.21.15.157:6809/2579998472]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-22T23:41:46.380 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:46 smithi157 ceph-mon[79403]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-22T23:41:46.380 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:46 smithi157 ceph-mon[79403]: pgmap v140: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 277 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 57/339 objects degraded (16.814%) 2024-04-22T23:41:46.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:46 smithi107 ceph-mon[90179]: from='osd.4 [v2:172.21.15.157:6808/2579998472,v1:172.21.15.157:6809/2579998472]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-22T23:41:46.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:46 smithi107 ceph-mon[90179]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-22T23:41:46.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:46 smithi107 ceph-mon[90179]: pgmap v140: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 277 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 57/339 objects degraded (16.814%) 2024-04-22T23:41:47.130 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:41:46 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4[94007]: 2024-04-22T23:41:46.867+0000 7f30d42b5700 -1 osd.4 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T23:41:47.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:47 smithi107 ceph-mon[90179]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-22T23:41:47.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:47 smithi107 ceph-mon[90179]: osdmap e72: 6 total, 5 up, 6 in 2024-04-22T23:41:47.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:47 smithi107 ceph-mon[90179]: from='osd.4 [v2:172.21.15.157:6808/2579998472,v1:172.21.15.157:6809/2579998472]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-22T23:41:47.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:47 smithi107 ceph-mon[90179]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-22T23:41:47.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:47 smithi107 ceph-mon[90179]: from='osd.4 ' entity='osd.4' 2024-04-22T23:41:47.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:47 smithi157 ceph-mon[79403]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-22T23:41:47.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:47 smithi157 ceph-mon[79403]: osdmap e72: 6 total, 5 up, 6 in 2024-04-22T23:41:47.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:47 smithi157 ceph-mon[79403]: from='osd.4 [v2:172.21.15.157:6808/2579998472,v1:172.21.15.157:6809/2579998472]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-22T23:41:47.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:47 smithi157 ceph-mon[79403]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-22T23:41:47.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:47 smithi157 ceph-mon[79403]: from='osd.4 ' entity='osd.4' 2024-04-22T23:41:48.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:48 smithi107 ceph-mon[90179]: Health check update: Degraded data redundancy: 57/339 objects degraded (16.814%), 16 pgs degraded (PG_DEGRADED) 2024-04-22T23:41:48.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:48 smithi107 ceph-mon[90179]: pgmap v142: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 277 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 57/339 objects degraded (16.814%) 2024-04-22T23:41:48.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:48 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:41:48.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:48 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:41:48.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:48 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:41:48.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:48 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:41:48.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:48 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:41:48.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:48 smithi107 ceph-mon[90179]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-22T23:41:48.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:48 smithi107 ceph-mon[90179]: osd.4 [v2:172.21.15.157:6808/2579998472,v1:172.21.15.157:6809/2579998472] boot 2024-04-22T23:41:48.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:48 smithi107 ceph-mon[90179]: osdmap e73: 6 total, 6 up, 6 in 2024-04-22T23:41:48.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:48 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T23:41:48.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:48 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:41:48.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:48 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:41:48.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:48 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:41:48.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:48 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:41:48.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:48 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-22T23:41:48.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:48 smithi157 ceph-mon[79403]: Health check update: Degraded data redundancy: 57/339 objects degraded (16.814%), 16 pgs degraded (PG_DEGRADED) 2024-04-22T23:41:48.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:48 smithi157 ceph-mon[79403]: pgmap v142: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 277 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 57/339 objects degraded (16.814%) 2024-04-22T23:41:48.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:48 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:41:48.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:48 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:41:48.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:48 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:41:48.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:48 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:41:48.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:48 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:41:48.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:48 smithi157 ceph-mon[79403]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-22T23:41:48.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:48 smithi157 ceph-mon[79403]: osd.4 [v2:172.21.15.157:6808/2579998472,v1:172.21.15.157:6809/2579998472] boot 2024-04-22T23:41:48.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:48 smithi157 ceph-mon[79403]: osdmap e73: 6 total, 6 up, 6 in 2024-04-22T23:41:48.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:48 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T23:41:48.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:48 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:41:48.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:48 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:41:48.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:48 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:41:48.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:48 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:41:48.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:48 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-22T23:41:49.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:49 smithi107 ceph-mon[90179]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-22T23:41:49.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:49 smithi107 ceph-mon[90179]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-04-22T23:41:49.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:49 smithi107 ceph-mon[90179]: osdmap e74: 6 total, 6 up, 6 in 2024-04-22T23:41:49.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:49 smithi157 ceph-mon[79403]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-22T23:41:49.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:49 smithi157 ceph-mon[79403]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-04-22T23:41:49.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:49 smithi157 ceph-mon[79403]: osdmap e74: 6 total, 6 up, 6 in 2024-04-22T23:41:50.380 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:50 smithi157 ceph-mon[79403]: pgmap v145: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 277 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 57/339 objects degraded (16.814%) 2024-04-22T23:41:50.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:50 smithi107 ceph-mon[90179]: pgmap v145: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 277 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 57/339 objects degraded (16.814%) 2024-04-22T23:41:53.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:52 smithi157 ceph-mon[79403]: pgmap v146: 65 pgs: 8 active+undersized+degraded, 8 active+undersized, 49 active+clean; 277 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 36/339 objects degraded (10.619%) 2024-04-22T23:41:53.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:52 smithi157 ceph-mon[79403]: Health check update: Degraded data redundancy: 36/339 objects degraded (10.619%), 8 pgs degraded (PG_DEGRADED) 2024-04-22T23:41:53.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:52 smithi107 ceph-mon[90179]: pgmap v146: 65 pgs: 8 active+undersized+degraded, 8 active+undersized, 49 active+clean; 277 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 36/339 objects degraded (10.619%) 2024-04-22T23:41:53.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:52 smithi107 ceph-mon[90179]: Health check update: Degraded data redundancy: 36/339 objects degraded (10.619%), 8 pgs degraded (PG_DEGRADED) 2024-04-22T23:41:54.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:53 smithi157 ceph-mon[79403]: pgmap v147: 65 pgs: 65 active+clean; 277 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:41:54.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:53 smithi157 ceph-mon[79403]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 36/339 objects degraded (10.619%), 8 pgs degraded) 2024-04-22T23:41:54.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:53 smithi157 ceph-mon[79403]: Cluster is now healthy 2024-04-22T23:41:54.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:53 smithi107 ceph-mon[90179]: pgmap v147: 65 pgs: 65 active+clean; 277 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:41:54.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:53 smithi107 ceph-mon[90179]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 36/339 objects degraded (10.619%), 8 pgs degraded) 2024-04-22T23:41:54.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:53 smithi107 ceph-mon[90179]: Cluster is now healthy 2024-04-22T23:41:56.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:56 smithi107 ceph-mon[90179]: pgmap v148: 65 pgs: 65 active+clean; 277 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:41:56.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:56 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:41:56.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:56 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:41:57.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:56 smithi157 ceph-mon[79403]: pgmap v148: 65 pgs: 65 active+clean; 277 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:41:57.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:56 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:41:57.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:56 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:41:58.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:41:58 smithi107 ceph-mon[90179]: pgmap v149: 65 pgs: 65 active+clean; 277 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:41:59.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:41:58 smithi157 ceph-mon[79403]: pgmap v149: 65 pgs: 65 active+clean; 277 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:41:59.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.547+0000 7f710e02b700 1 -- 172.21.15.107:0/1151769659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71080fce50 msgr2=0x7f71080fd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:59.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.547+0000 7f710e02b700 1 --2- 172.21.15.107:0/1151769659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71080fce50 0x7f71080fd260 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f70f8004940 tx=0x7f70f8009e30 comp rx=0 tx=0).stop 2024-04-22T23:41:59.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.547+0000 7f710e02b700 1 -- 172.21.15.107:0/1151769659 shutdown_connections 2024-04-22T23:41:59.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.547+0000 7f710e02b700 1 --2- 172.21.15.107:0/1151769659 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71080fe050 0x7f71080fe4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:59.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.547+0000 7f710e02b700 1 --2- 172.21.15.107:0/1151769659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71080fce50 0x7f71080fd260 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:59.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.547+0000 7f710e02b700 1 -- 172.21.15.107:0/1151769659 >> 172.21.15.107:0/1151769659 conn(0x7f71080f8580 msgr2=0x7f71080fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:59.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.547+0000 7f710e02b700 1 -- 172.21.15.107:0/1151769659 shutdown_connections 2024-04-22T23:41:59.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.547+0000 7f710e02b700 1 -- 172.21.15.107:0/1151769659 wait complete. 2024-04-22T23:41:59.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.548+0000 7f710e02b700 1 Processor -- start 2024-04-22T23:41:59.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.548+0000 7f710e02b700 1 -- start start 2024-04-22T23:41:59.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.548+0000 7f710e02b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71080fce50 0x7f710810d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:59.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.548+0000 7f710e02b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71080fe050 0x7f710810deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:59.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.548+0000 7f710e02b700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f710810e4b0 con 0x7f71080fce50 2024-04-22T23:41:59.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.548+0000 7f710e02b700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f710810e620 con 0x7f71080fe050 2024-04-22T23:41:59.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.548+0000 7f71077fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71080fce50 0x7f710810d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:59.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.548+0000 7f71077fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71080fce50 0x7f710810d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49308/0 (socket says 172.21.15.107:49308) 2024-04-22T23:41:59.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.549+0000 7f71077fe700 1 -- 172.21.15.107:0/3280395590 learned_addr learned my addr 172.21.15.107:0/3280395590 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:41:59.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.549+0000 7f70feffd700 1 --2- 172.21.15.107:0/3280395590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71080fe050 0x7f710810deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:59.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.549+0000 7f71077fe700 1 -- 172.21.15.107:0/3280395590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71080fe050 msgr2=0x7f710810deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:59.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.549+0000 7f71077fe700 1 --2- 172.21.15.107:0/3280395590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71080fe050 0x7f710810deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:59.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.549+0000 7f71077fe700 1 -- 172.21.15.107:0/3280395590 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f8005040 con 0x7f71080fce50 2024-04-22T23:41:59.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.549+0000 7f71077fe700 1 --2- 172.21.15.107:0/3280395590 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71080fce50 0x7f710810d970 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f70f800b970 tx=0x7f70f8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:59.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.549+0000 7f7104ff9700 1 -- 172.21.15.107:0/3280395590 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70f800e040 con 0x7f71080fce50 2024-04-22T23:41:59.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.549+0000 7f710e02b700 1 -- 172.21.15.107:0/3280395590 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7108113020 con 0x7f71080fce50 2024-04-22T23:41:59.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.549+0000 7f7104ff9700 1 -- 172.21.15.107:0/3280395590 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f70f8004290 con 0x7f71080fce50 2024-04-22T23:41:59.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.549+0000 7f7104ff9700 1 -- 172.21.15.107:0/3280395590 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70f801c3f0 con 0x7f71080fce50 2024-04-22T23:41:59.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.549+0000 7f710e02b700 1 -- 172.21.15.107:0/3280395590 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71081135e0 con 0x7f71080fce50 2024-04-22T23:41:59.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.550+0000 7f710e02b700 1 -- 172.21.15.107:0/3280395590 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f710804c710 con 0x7f71080fce50 2024-04-22T23:41:59.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.551+0000 7f7104ff9700 1 -- 172.21.15.107:0/3280395590 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f70f8013420 con 0x7f71080fce50 2024-04-22T23:41:59.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.551+0000 7f7104ff9700 1 --2- 172.21.15.107:0/3280395590 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f70f406ec00 0x7f70f40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:59.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.551+0000 7f7104ff9700 1 -- 172.21.15.107:0/3280395590 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f70f808fa60 con 0x7f71080fce50 2024-04-22T23:41:59.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.551+0000 7f70feffd700 1 --2- 172.21.15.107:0/3280395590 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f70f406ec00 0x7f70f40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:59.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.552+0000 7f70feffd700 1 --2- 172.21.15.107:0/3280395590 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f70f406ec00 0x7f70f40710b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f70f0004f40 tx=0x7f70f000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:59.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.556+0000 7f7104ff9700 1 -- 172.21.15.107:0/3280395590 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f70f805c480 con 0x7f71080fce50 2024-04-22T23:41:59.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.780+0000 7f710e02b700 1 -- 172.21.15.107:0/3280395590 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7108102460 con 0x7f70f406ec00 2024-04-22T23:41:59.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.782+0000 7f7104ff9700 1 -- 172.21.15.107:0/3280395590 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f7108102460 con 0x7f70f406ec00 2024-04-22T23:41:59.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.784+0000 7f710e02b700 1 -- 172.21.15.107:0/3280395590 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f70f406ec00 msgr2=0x7f70f40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:59.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.784+0000 7f710e02b700 1 --2- 172.21.15.107:0/3280395590 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f70f406ec00 0x7f70f40710b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f70f0004f40 tx=0x7f70f000b040 comp rx=0 tx=0).stop 2024-04-22T23:41:59.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.784+0000 7f710e02b700 1 -- 172.21.15.107:0/3280395590 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71080fce50 msgr2=0x7f710810d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:59.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.784+0000 7f710e02b700 1 --2- 172.21.15.107:0/3280395590 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71080fce50 0x7f710810d970 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f70f800b970 tx=0x7f70f8009e30 comp rx=0 tx=0).stop 2024-04-22T23:41:59.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.784+0000 7f710e02b700 1 -- 172.21.15.107:0/3280395590 shutdown_connections 2024-04-22T23:41:59.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.784+0000 7f710e02b700 1 --2- 172.21.15.107:0/3280395590 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f70f406ec00 0x7f70f40710b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:59.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.784+0000 7f710e02b700 1 --2- 172.21.15.107:0/3280395590 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71080fce50 0x7f710810d970 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:59.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.784+0000 7f710e02b700 1 --2- 172.21.15.107:0/3280395590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f71080fe050 0x7f710810deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:59.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.784+0000 7f710e02b700 1 -- 172.21.15.107:0/3280395590 >> 172.21.15.107:0/3280395590 conn(0x7f71080f8580 msgr2=0x7f71080f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:59.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.784+0000 7f710e02b700 1 -- 172.21.15.107:0/3280395590 shutdown_connections 2024-04-22T23:41:59.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.784+0000 7f710e02b700 1 -- 172.21.15.107:0/3280395590 wait complete. 2024-04-22T23:41:59.787 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-04-22T23:41:59.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.886+0000 7f94cc582700 1 -- 172.21.15.107:0/3863301432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94c40fc320 msgr2=0x7f94c40fc730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:59.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.886+0000 7f94cc582700 1 --2- 172.21.15.107:0/3863301432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94c40fc320 0x7f94c40fc730 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f94c0004940 tx=0x7f94c0009e30 comp rx=0 tx=0).stop 2024-04-22T23:41:59.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.887+0000 7f94cc582700 1 -- 172.21.15.107:0/3863301432 shutdown_connections 2024-04-22T23:41:59.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.887+0000 7f94cc582700 1 --2- 172.21.15.107:0/3863301432 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94c40fcc70 0x7f94c40fd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:59.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.887+0000 7f94cc582700 1 --2- 172.21.15.107:0/3863301432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94c40fc320 0x7f94c40fc730 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:59.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.887+0000 7f94cc582700 1 -- 172.21.15.107:0/3863301432 >> 172.21.15.107:0/3863301432 conn(0x7f94c40f6c10 msgr2=0x7f94c40f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:41:59.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.887+0000 7f94cc582700 1 -- 172.21.15.107:0/3863301432 shutdown_connections 2024-04-22T23:41:59.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.887+0000 7f94cc582700 1 -- 172.21.15.107:0/3863301432 wait complete. 2024-04-22T23:41:59.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.887+0000 7f94cc582700 1 Processor -- start 2024-04-22T23:41:59.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.887+0000 7f94cc582700 1 -- start start 2024-04-22T23:41:59.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.887+0000 7f94cc582700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94c40fc320 0x7f94c410c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:59.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.887+0000 7f94cc582700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94c40fcc70 0x7f94c410cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:59.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.887+0000 7f94cc582700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c410d390 con 0x7f94c40fcc70 2024-04-22T23:41:59.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.887+0000 7f94cc582700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c410d500 con 0x7f94c40fc320 2024-04-22T23:41:59.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.887+0000 7f94c9b1d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94c40fcc70 0x7f94c410cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:59.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.888+0000 7f94c9b1d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94c40fcc70 0x7f94c410cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49328/0 (socket says 172.21.15.107:49328) 2024-04-22T23:41:59.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.888+0000 7f94c9b1d700 1 -- 172.21.15.107:0/4142095275 learned_addr learned my addr 172.21.15.107:0/4142095275 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:41:59.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.888+0000 7f94ca31e700 1 --2- 172.21.15.107:0/4142095275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94c40fc320 0x7f94c410c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:59.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.888+0000 7f94c9b1d700 1 -- 172.21.15.107:0/4142095275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94c40fc320 msgr2=0x7f94c410c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:41:59.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.888+0000 7f94c9b1d700 1 --2- 172.21.15.107:0/4142095275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94c40fc320 0x7f94c410c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:41:59.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.888+0000 7f94c9b1d700 1 -- 172.21.15.107:0/4142095275 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c0005040 con 0x7f94c40fcc70 2024-04-22T23:41:59.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.888+0000 7f94c9b1d700 1 --2- 172.21.15.107:0/4142095275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94c40fcc70 0x7f94c410cd90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f94b400c8c0 tx=0x7f94b40044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:59.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.888+0000 7f94baffd700 1 -- 172.21.15.107:0/4142095275 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94b400a980 con 0x7f94c40fcc70 2024-04-22T23:41:59.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.888+0000 7f94baffd700 1 -- 172.21.15.107:0/4142095275 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f94b400ce60 con 0x7f94c40fcc70 2024-04-22T23:41:59.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.888+0000 7f94cc582700 1 -- 172.21.15.107:0/4142095275 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94c4111f60 con 0x7f94c40fcc70 2024-04-22T23:41:59.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.888+0000 7f94baffd700 1 -- 172.21.15.107:0/4142095275 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94b40115e0 con 0x7f94c40fcc70 2024-04-22T23:41:59.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.888+0000 7f94cc582700 1 -- 172.21.15.107:0/4142095275 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94c41124f0 con 0x7f94c40fcc70 2024-04-22T23:41:59.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.889+0000 7f94cc582700 1 -- 172.21.15.107:0/4142095275 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94c404c710 con 0x7f94c40fcc70 2024-04-22T23:41:59.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.890+0000 7f94baffd700 1 -- 172.21.15.107:0/4142095275 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f94b4012400 con 0x7f94c40fcc70 2024-04-22T23:41:59.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.890+0000 7f94baffd700 1 --2- 172.21.15.107:0/4142095275 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f94b006ec00 0x7f94b00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:41:59.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.890+0000 7f94ca31e700 1 --2- 172.21.15.107:0/4142095275 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f94b006ec00 0x7f94b00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:41:59.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.890+0000 7f94baffd700 1 -- 172.21.15.107:0/4142095275 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f94b408e980 con 0x7f94c40fcc70 2024-04-22T23:41:59.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.891+0000 7f94ca31e700 1 --2- 172.21.15.107:0/4142095275 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f94b006ec00 0x7f94b00710b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f94c000b3b0 tx=0x7f94c0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:41:59.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:41:59.895+0000 7f94baffd700 1 -- 172.21.15.107:0/4142095275 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f94b405b2e0 con 0x7f94c40fcc70 2024-04-22T23:42:00.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.114+0000 7f94cc582700 1 -- 172.21.15.107:0/4142095275 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f94c41014c0 con 0x7f94b006ec00 2024-04-22T23:42:00.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.115+0000 7f94baffd700 1 -- 172.21.15.107:0/4142095275 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f94c41014c0 con 0x7f94b006ec00 2024-04-22T23:42:00.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.117+0000 7f94cc582700 1 -- 172.21.15.107:0/4142095275 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f94b006ec00 msgr2=0x7f94b00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:00.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.117+0000 7f94cc582700 1 --2- 172.21.15.107:0/4142095275 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f94b006ec00 0x7f94b00710b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f94c000b3b0 tx=0x7f94c0009e30 comp rx=0 tx=0).stop 2024-04-22T23:42:00.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.117+0000 7f94cc582700 1 -- 172.21.15.107:0/4142095275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94c40fcc70 msgr2=0x7f94c410cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:00.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.117+0000 7f94cc582700 1 --2- 172.21.15.107:0/4142095275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94c40fcc70 0x7f94c410cd90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f94b400c8c0 tx=0x7f94b40044c0 comp rx=0 tx=0).stop 2024-04-22T23:42:00.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.117+0000 7f94cc582700 1 -- 172.21.15.107:0/4142095275 shutdown_connections 2024-04-22T23:42:00.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.117+0000 7f94cc582700 1 --2- 172.21.15.107:0/4142095275 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94c40fc320 0x7f94c410c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:00.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.117+0000 7f94cc582700 1 --2- 172.21.15.107:0/4142095275 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f94b006ec00 0x7f94b00710b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:00.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.117+0000 7f94cc582700 1 --2- 172.21.15.107:0/4142095275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94c40fcc70 0x7f94c410cd90 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:00.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.117+0000 7f94cc582700 1 -- 172.21.15.107:0/4142095275 >> 172.21.15.107:0/4142095275 conn(0x7f94c40f6c10 msgr2=0x7f94c40f82e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:00.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.117+0000 7f94cc582700 1 -- 172.21.15.107:0/4142095275 shutdown_connections 2024-04-22T23:42:00.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.117+0000 7f94cc582700 1 -- 172.21.15.107:0/4142095275 wait complete. 2024-04-22T23:42:00.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.218+0000 7f4299db9700 1 -- 172.21.15.107:0/2004200751 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4294071ea0 msgr2=0x7f42940703b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:00.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.218+0000 7f4299db9700 1 --2- 172.21.15.107:0/2004200751 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4294071ea0 0x7f42940703b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4284004940 tx=0x7f4284009e30 comp rx=0 tx=0).stop 2024-04-22T23:42:00.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.218+0000 7f4299db9700 1 -- 172.21.15.107:0/2004200751 shutdown_connections 2024-04-22T23:42:00.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.218+0000 7f4299db9700 1 --2- 172.21.15.107:0/2004200751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f42940708f0 0x7f4294070d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:00.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.218+0000 7f4299db9700 1 --2- 172.21.15.107:0/2004200751 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4294071ea0 0x7f42940703b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:00.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.218+0000 7f4299db9700 1 -- 172.21.15.107:0/2004200751 >> 172.21.15.107:0/2004200751 conn(0x7f42940f85a0 msgr2=0x7f42940fa9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:00.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.218+0000 7f4299db9700 1 -- 172.21.15.107:0/2004200751 shutdown_connections 2024-04-22T23:42:00.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.218+0000 7f4299db9700 1 -- 172.21.15.107:0/2004200751 wait complete. 2024-04-22T23:42:00.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.218+0000 7f4299db9700 1 Processor -- start 2024-04-22T23:42:00.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.218+0000 7f4299db9700 1 -- start start 2024-04-22T23:42:00.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.219+0000 7f4299db9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42940708f0 0x7f429410fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:00.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.219+0000 7f4299db9700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4294071ea0 0x7f4294110040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:00.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.219+0000 7f4299db9700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4294110640 con 0x7f42940708f0 2024-04-22T23:42:00.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.219+0000 7f4299db9700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42941107b0 con 0x7f4294071ea0 2024-04-22T23:42:00.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.219+0000 7f42937fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42940708f0 0x7f429410fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:00.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.219+0000 7f42937fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42940708f0 0x7f429410fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49352/0 (socket says 172.21.15.107:49352) 2024-04-22T23:42:00.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.219+0000 7f42937fe700 1 -- 172.21.15.107:0/4109522013 learned_addr learned my addr 172.21.15.107:0/4109522013 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:42:00.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.219+0000 7f428affd700 1 --2- 172.21.15.107:0/4109522013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4294071ea0 0x7f4294110040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:00.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.219+0000 7f42937fe700 1 -- 172.21.15.107:0/4109522013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4294071ea0 msgr2=0x7f4294110040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:00.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.219+0000 7f42937fe700 1 --2- 172.21.15.107:0/4109522013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4294071ea0 0x7f4294110040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:00.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.219+0000 7f42937fe700 1 -- 172.21.15.107:0/4109522013 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4284005040 con 0x7f42940708f0 2024-04-22T23:42:00.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.219+0000 7f42937fe700 1 --2- 172.21.15.107:0/4109522013 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42940708f0 0x7f429410fb00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4284000c00 tx=0x7f4284009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:00.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.219+0000 7f4290ff9700 1 -- 172.21.15.107:0/4109522013 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f428400e040 con 0x7f42940708f0 2024-04-22T23:42:00.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.219+0000 7f4299db9700 1 -- 172.21.15.107:0/4109522013 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42941151b0 con 0x7f42940708f0 2024-04-22T23:42:00.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.219+0000 7f4290ff9700 1 -- 172.21.15.107:0/4109522013 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f428400bbf0 con 0x7f42940708f0 2024-04-22T23:42:00.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.219+0000 7f4290ff9700 1 -- 172.21.15.107:0/4109522013 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f428401c630 con 0x7f42940708f0 2024-04-22T23:42:00.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.220+0000 7f4299db9700 1 -- 172.21.15.107:0/4109522013 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4294115770 con 0x7f42940708f0 2024-04-22T23:42:00.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.220+0000 7f4299db9700 1 -- 172.21.15.107:0/4109522013 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f429404c710 con 0x7f42940708f0 2024-04-22T23:42:00.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.221+0000 7f4290ff9700 1 -- 172.21.15.107:0/4109522013 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f428400bd60 con 0x7f42940708f0 2024-04-22T23:42:00.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.222+0000 7f4290ff9700 1 --2- 172.21.15.107:0/4109522013 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f428006ec00 0x7f42800710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:00.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.222+0000 7f4290ff9700 1 -- 172.21.15.107:0/4109522013 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f428408fa40 con 0x7f42940708f0 2024-04-22T23:42:00.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.222+0000 7f428affd700 1 --2- 172.21.15.107:0/4109522013 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f428006ec00 0x7f42800710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:00.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.222+0000 7f428affd700 1 --2- 172.21.15.107:0/4109522013 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f428006ec00 0x7f42800710b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f427c009fd0 tx=0x7f427c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:00.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.226+0000 7f4290ff9700 1 -- 172.21.15.107:0/4109522013 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f428405c460 con 0x7f42940708f0 2024-04-22T23:42:00.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.437+0000 7f4299db9700 1 -- 172.21.15.107:0/4109522013 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4294103e40 con 0x7f428006ec00 2024-04-22T23:42:00.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.447+0000 7f4290ff9700 1 -- 172.21.15.107:0/4109522013 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f4294103e40 con 0x7f428006ec00 2024-04-22T23:42:00.440 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-22T23:42:00.440 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (5m) 84s ago 14m 25.6M - 0.25.0 c8568f914cd2 ec1a5c6260de 2024-04-22T23:42:00.440 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (3m) 84s ago 15m 6656k - 19.0.0-3252-g43be0201 a0434699c1d7 e074fd2cbb9c 2024-04-22T23:42:00.440 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi157 smithi157 running (3m) 17s ago 14m 6651k - 19.0.0-3252-g43be0201 a0434699c1d7 abc2649dee22 2024-04-22T23:42:00.440 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (5m) 84s ago 14m 79.9M - 9.4.7 954c08fa6188 31449ff442d2 2024-04-22T23:42:00.440 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.dfvvkw smithi107 running (9m) 84s ago 9m 13.7M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 08d3aa8d717d 2024-04-22T23:42:00.440 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.ktuxjl smithi107 running (9m) 84s ago 9m 124M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 762f27fe0f30 2024-04-22T23:42:00.440 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi157.rcaris smithi157 running (9m) 17s ago 9m 129M - 17.2.7-884-g70d0cb4b cafd8ddac8fa d23366ac085d 2024-04-22T23:42:00.441 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi157.ufywse smithi157 running (9m) 17s ago 9m 14.7M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 8b00d61eb355 2024-04-22T23:42:00.441 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.uipxvm smithi107 *:8443,9283,8765 running (5m) 84s ago 16m 502M - 19.0.0-3252-g43be0201 a0434699c1d7 d20f05dfe4f2 2024-04-22T23:42:00.441 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi157.tuylis smithi157 *:8443,9283,8765 running (4m) 17s ago 14m 451M - 19.0.0-3252-g43be0201 a0434699c1d7 957e598c9001 2024-04-22T23:42:00.441 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (4m) 84s ago 16m 49.4M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 bc72bbbe2c39 2024-04-22T23:42:00.441 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi157 smithi157 running (3m) 17s ago 14m 38.4M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 ea88de8be4fe 2024-04-22T23:42:00.441 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (5m) 84s ago 15m 16.2M - 1.5.0 0da6a335fe13 5205cfd9bd51 2024-04-22T23:42:00.441 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi157 smithi157 *:9100 running (5m) 17s ago 14m 17.1M - 1.5.0 0da6a335fe13 168ca191bb8c 2024-04-22T23:42:00.441 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (2m) 84s ago 13m 94.2M 1557M 19.0.0-3252-g43be0201 a0434699c1d7 fd0ba8ef0c27 2024-04-22T23:42:00.441 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (2m) 84s ago 12m 91.2M 1557M 19.0.0-3252-g43be0201 a0434699c1d7 77e33b6122e0 2024-04-22T23:42:00.441 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (90s) 84s ago 12m 15.0M 1557M 19.0.0-3252-g43be0201 a0434699c1d7 c307bd8cdcfc 2024-04-22T23:42:00.441 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi157 running (54s) 17s ago 11m 98.2M 2581M 19.0.0-3252-g43be0201 a0434699c1d7 e81ca519c831 2024-04-22T23:42:00.441 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi157 running (20s) 17s ago 11m 11.8M 2581M 19.0.0-3252-g43be0201 a0434699c1d7 d5b5a2e4781a 2024-04-22T23:42:00.441 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi157 running (11m) 17s ago 11m 225M 2581M 17.2.7-884-g70d0cb4b cafd8ddac8fa 4c5b7a7ef8bd 2024-04-22T23:42:00.441 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (4m) 84s ago 14m 58.8M - 2.43.0 a07b618ecd1d 99a4ea8594da 2024-04-22T23:42:00.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.449+0000 7f4299db9700 1 -- 172.21.15.107:0/4109522013 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f428006ec00 msgr2=0x7f42800710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:00.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.449+0000 7f4299db9700 1 --2- 172.21.15.107:0/4109522013 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f428006ec00 0x7f42800710b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f427c009fd0 tx=0x7f427c00b040 comp rx=0 tx=0).stop 2024-04-22T23:42:00.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.449+0000 7f4299db9700 1 -- 172.21.15.107:0/4109522013 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42940708f0 msgr2=0x7f429410fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:00.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.449+0000 7f4299db9700 1 --2- 172.21.15.107:0/4109522013 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42940708f0 0x7f429410fb00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4284000c00 tx=0x7f4284009e30 comp rx=0 tx=0).stop 2024-04-22T23:42:00.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.449+0000 7f4299db9700 1 -- 172.21.15.107:0/4109522013 shutdown_connections 2024-04-22T23:42:00.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.449+0000 7f4299db9700 1 --2- 172.21.15.107:0/4109522013 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f428006ec00 0x7f42800710b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:00.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.449+0000 7f4299db9700 1 --2- 172.21.15.107:0/4109522013 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42940708f0 0x7f429410fb00 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:00.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.449+0000 7f4299db9700 1 --2- 172.21.15.107:0/4109522013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4294071ea0 0x7f4294110040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:00.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.449+0000 7f4299db9700 1 -- 172.21.15.107:0/4109522013 >> 172.21.15.107:0/4109522013 conn(0x7f42940f85a0 msgr2=0x7f42940f9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:00.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.450+0000 7f4299db9700 1 -- 172.21.15.107:0/4109522013 shutdown_connections 2024-04-22T23:42:00.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.450+0000 7f4299db9700 1 -- 172.21.15.107:0/4109522013 wait complete. 2024-04-22T23:42:00.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.550+0000 7fc619700700 1 -- 172.21.15.107:0/4099570685 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6140fbe20 msgr2=0x7fc6140fc230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:00.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.550+0000 7fc619700700 1 --2- 172.21.15.107:0/4099570685 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6140fbe20 0x7fc6140fc230 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc608004940 tx=0x7fc608009e30 comp rx=0 tx=0).stop 2024-04-22T23:42:00.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.550+0000 7fc619700700 1 -- 172.21.15.107:0/4099570685 shutdown_connections 2024-04-22T23:42:00.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.550+0000 7fc619700700 1 --2- 172.21.15.107:0/4099570685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6140fcf90 0x7fc6140fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:00.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.550+0000 7fc619700700 1 --2- 172.21.15.107:0/4099570685 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6140fbe20 0x7fc6140fc230 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:00.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.550+0000 7fc619700700 1 -- 172.21.15.107:0/4099570685 >> 172.21.15.107:0/4099570685 conn(0x7fc6140f74e0 msgr2=0x7fc6140f9910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:00.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.550+0000 7fc619700700 1 -- 172.21.15.107:0/4099570685 shutdown_connections 2024-04-22T23:42:00.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.550+0000 7fc619700700 1 -- 172.21.15.107:0/4099570685 wait complete. 2024-04-22T23:42:00.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.551+0000 7fc619700700 1 Processor -- start 2024-04-22T23:42:00.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.551+0000 7fc619700700 1 -- start start 2024-04-22T23:42:00.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.551+0000 7fc619700700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6140fbe20 0x7fc61410da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:00.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.551+0000 7fc619700700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6140fcf90 0x7fc61410dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:00.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.551+0000 7fc619700700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc61410e5a0 con 0x7fc6140fbe20 2024-04-22T23:42:00.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.551+0000 7fc619700700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc61410edf0 con 0x7fc6140fcf90 2024-04-22T23:42:00.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.551+0000 7fc612ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6140fbe20 0x7fc61410da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:00.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.551+0000 7fc612ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6140fbe20 0x7fc61410da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49372/0 (socket says 172.21.15.107:49372) 2024-04-22T23:42:00.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.551+0000 7fc612ffd700 1 -- 172.21.15.107:0/2220330817 learned_addr learned my addr 172.21.15.107:0/2220330817 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:42:00.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.551+0000 7fc6127fc700 1 --2- 172.21.15.107:0/2220330817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6140fcf90 0x7fc61410dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:00.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.551+0000 7fc612ffd700 1 -- 172.21.15.107:0/2220330817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6140fcf90 msgr2=0x7fc61410dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:00.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.551+0000 7fc612ffd700 1 --2- 172.21.15.107:0/2220330817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6140fcf90 0x7fc61410dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:00.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.551+0000 7fc612ffd700 1 -- 172.21.15.107:0/2220330817 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc608005040 con 0x7fc6140fbe20 2024-04-22T23:42:00.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.551+0000 7fc6127fc700 1 --2- 172.21.15.107:0/2220330817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6140fcf90 0x7fc61410dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T23:42:00.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.552+0000 7fc612ffd700 1 --2- 172.21.15.107:0/2220330817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6140fbe20 0x7fc61410da60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc60800b3b0 tx=0x7fc608009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:00.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.552+0000 7fc5fb7fe700 1 -- 172.21.15.107:0/2220330817 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc60800e040 con 0x7fc6140fbe20 2024-04-22T23:42:00.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.552+0000 7fc619700700 1 -- 172.21.15.107:0/2220330817 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc61410f070 con 0x7fc6140fbe20 2024-04-22T23:42:00.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.552+0000 7fc5fb7fe700 1 -- 172.21.15.107:0/2220330817 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc60800bcc0 con 0x7fc6140fbe20 2024-04-22T23:42:00.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.552+0000 7fc5fb7fe700 1 -- 172.21.15.107:0/2220330817 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc60801c630 con 0x7fc6140fbe20 2024-04-22T23:42:00.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.552+0000 7fc619700700 1 -- 172.21.15.107:0/2220330817 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc61410f5b0 con 0x7fc6140fbe20 2024-04-22T23:42:00.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.553+0000 7fc619700700 1 -- 172.21.15.107:0/2220330817 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc61405f5f0 con 0x7fc6140fbe20 2024-04-22T23:42:00.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.553+0000 7fc5fb7fe700 1 -- 172.21.15.107:0/2220330817 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fc608013980 con 0x7fc6140fbe20 2024-04-22T23:42:00.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.554+0000 7fc5fb7fe700 1 --2- 172.21.15.107:0/2220330817 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fc5fc06ec00 0x7fc5fc0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:00.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.554+0000 7fc5fb7fe700 1 -- 172.21.15.107:0/2220330817 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fc60808f580 con 0x7fc6140fbe20 2024-04-22T23:42:00.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.554+0000 7fc6127fc700 1 --2- 172.21.15.107:0/2220330817 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fc5fc06ec00 0x7fc5fc0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:00.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.554+0000 7fc6127fc700 1 --2- 172.21.15.107:0/2220330817 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fc5fc06ec00 0x7fc5fc0710b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc604009fd0 tx=0x7fc60400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:00.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.558+0000 7fc5fb7fe700 1 -- 172.21.15.107:0/2220330817 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fc60805be60 con 0x7fc6140fbe20 2024-04-22T23:42:00.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.847+0000 7fc619700700 1 -- 172.21.15.107:0/2220330817 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc61410f890 con 0x7fc6140fbe20 2024-04-22T23:42:00.840 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:00 smithi107 ceph-mon[90179]: pgmap v150: 65 pgs: 65 active+clean; 277 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:42:00.840 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:00 smithi107 ceph-mon[90179]: from='client.34344 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:42:00.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.847+0000 7fc5fb7fe700 1 -- 172.21.15.107:0/2220330817 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7fc60805b9f0 con 0x7fc6140fbe20 2024-04-22T23:42:00.841 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-22T23:42:00.841 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-04-22T23:42:00.841 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:42:00.841 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:42:00.841 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-04-22T23:42:00.841 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:42:00.841 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:42:00.841 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-04-22T23:42:00.841 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 1, 2024-04-22T23:42:00.841 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 5 2024-04-22T23:42:00.841 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:42:00.841 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-04-22T23:42:00.842 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 4 2024-04-22T23:42:00.842 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:42:00.842 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-04-22T23:42:00.842 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 5, 2024-04-22T23:42:00.842 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 9 2024-04-22T23:42:00.842 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T23:42:00.842 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-22T23:42:00.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.849+0000 7fc619700700 1 -- 172.21.15.107:0/2220330817 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fc5fc06ec00 msgr2=0x7fc5fc0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:00.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.849+0000 7fc619700700 1 --2- 172.21.15.107:0/2220330817 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fc5fc06ec00 0x7fc5fc0710b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc604009fd0 tx=0x7fc60400b040 comp rx=0 tx=0).stop 2024-04-22T23:42:00.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.849+0000 7fc619700700 1 -- 172.21.15.107:0/2220330817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6140fbe20 msgr2=0x7fc61410da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:00.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.849+0000 7fc619700700 1 --2- 172.21.15.107:0/2220330817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6140fbe20 0x7fc61410da60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc60800b3b0 tx=0x7fc608009e30 comp rx=0 tx=0).stop 2024-04-22T23:42:00.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.850+0000 7fc619700700 1 -- 172.21.15.107:0/2220330817 shutdown_connections 2024-04-22T23:42:00.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.850+0000 7fc619700700 1 --2- 172.21.15.107:0/2220330817 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fc5fc06ec00 0x7fc5fc0710b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:00.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.850+0000 7fc619700700 1 --2- 172.21.15.107:0/2220330817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6140fbe20 0x7fc61410da60 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:00.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.850+0000 7fc619700700 1 --2- 172.21.15.107:0/2220330817 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6140fcf90 0x7fc61410dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:00.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.850+0000 7fc619700700 1 -- 172.21.15.107:0/2220330817 >> 172.21.15.107:0/2220330817 conn(0x7fc6140f74e0 msgr2=0x7fc6140f8bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:00.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.850+0000 7fc619700700 1 -- 172.21.15.107:0/2220330817 shutdown_connections 2024-04-22T23:42:00.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.850+0000 7fc619700700 1 -- 172.21.15.107:0/2220330817 wait complete. 2024-04-22T23:42:00.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.950+0000 7f7a88b39700 1 -- 172.21.15.107:0/3827021759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a840fe070 msgr2=0x7f7a840fe4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:00.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.950+0000 7f7a88b39700 1 --2- 172.21.15.107:0/3827021759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a840fe070 0x7f7a840fe4e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7a78004940 tx=0x7f7a78009e30 comp rx=0 tx=0).stop 2024-04-22T23:42:00.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.950+0000 7f7a88b39700 1 -- 172.21.15.107:0/3827021759 shutdown_connections 2024-04-22T23:42:00.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.950+0000 7f7a88b39700 1 --2- 172.21.15.107:0/3827021759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a840fe070 0x7f7a840fe4e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:00.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.950+0000 7f7a88b39700 1 --2- 172.21.15.107:0/3827021759 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a840fcf00 0x7f7a840fd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:00.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.950+0000 7f7a88b39700 1 -- 172.21.15.107:0/3827021759 >> 172.21.15.107:0/3827021759 conn(0x7f7a840f85e0 msgr2=0x7f7a840fa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:00.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.950+0000 7f7a88b39700 1 -- 172.21.15.107:0/3827021759 shutdown_connections 2024-04-22T23:42:00.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.950+0000 7f7a88b39700 1 -- 172.21.15.107:0/3827021759 wait complete. 2024-04-22T23:42:00.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.950+0000 7f7a88b39700 1 Processor -- start 2024-04-22T23:42:00.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.950+0000 7f7a88b39700 1 -- start start 2024-04-22T23:42:00.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.951+0000 7f7a88b39700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a840fcf00 0x7f7a8410da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:00.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.951+0000 7f7a88b39700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a840fe070 0x7f7a8410df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:00.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.951+0000 7f7a88b39700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8410e560 con 0x7f7a840fe070 2024-04-22T23:42:00.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.951+0000 7f7a88b39700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8410edb0 con 0x7f7a840fcf00 2024-04-22T23:42:00.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.951+0000 7f7a81d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a840fe070 0x7f7a8410df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:00.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.951+0000 7f7a81d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a840fe070 0x7f7a8410df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49388/0 (socket says 172.21.15.107:49388) 2024-04-22T23:42:00.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.951+0000 7f7a81d9b700 1 -- 172.21.15.107:0/4176725263 learned_addr learned my addr 172.21.15.107:0/4176725263 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:42:00.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.951+0000 7f7a8259c700 1 --2- 172.21.15.107:0/4176725263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a840fcf00 0x7f7a8410da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:00.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.951+0000 7f7a81d9b700 1 -- 172.21.15.107:0/4176725263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a840fcf00 msgr2=0x7f7a8410da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:00.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.951+0000 7f7a81d9b700 1 --2- 172.21.15.107:0/4176725263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a840fcf00 0x7f7a8410da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:00.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.951+0000 7f7a81d9b700 1 -- 172.21.15.107:0/4176725263 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a78005040 con 0x7f7a840fe070 2024-04-22T23:42:00.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.951+0000 7f7a81d9b700 1 --2- 172.21.15.107:0/4176725263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a840fe070 0x7f7a8410df60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7a7800b3b0 tx=0x7f7a78009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:00.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.951+0000 7f7a72ffd700 1 -- 172.21.15.107:0/4176725263 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a7800e040 con 0x7f7a840fe070 2024-04-22T23:42:00.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.951+0000 7f7a72ffd700 1 -- 172.21.15.107:0/4176725263 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7a7800bbf0 con 0x7f7a840fe070 2024-04-22T23:42:00.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.951+0000 7f7a72ffd700 1 -- 172.21.15.107:0/4176725263 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a7801c630 con 0x7f7a840fe070 2024-04-22T23:42:00.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.951+0000 7f7a88b39700 1 -- 172.21.15.107:0/4176725263 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a8410f030 con 0x7f7a840fe070 2024-04-22T23:42:00.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.951+0000 7f7a88b39700 1 -- 172.21.15.107:0/4176725263 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a8410f570 con 0x7f7a840fe070 2024-04-22T23:42:00.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.952+0000 7f7a88b39700 1 -- 172.21.15.107:0/4176725263 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a8405f5f0 con 0x7f7a840fe070 2024-04-22T23:42:00.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.953+0000 7f7a72ffd700 1 -- 172.21.15.107:0/4176725263 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f7a7800bd60 con 0x7f7a840fe070 2024-04-22T23:42:00.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.953+0000 7f7a72ffd700 1 --2- 172.21.15.107:0/4176725263 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f7a6c06ec00 0x7f7a6c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:00.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.953+0000 7f7a72ffd700 1 -- 172.21.15.107:0/4176725263 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f7a7808f450 con 0x7f7a840fe070 2024-04-22T23:42:00.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.953+0000 7f7a8259c700 1 --2- 172.21.15.107:0/4176725263 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f7a6c06ec00 0x7f7a6c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:00.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.954+0000 7f7a8259c700 1 --2- 172.21.15.107:0/4176725263 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f7a6c06ec00 0x7f7a6c0710b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c007a40 tx=0x7f7a7c009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:00.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:00.958+0000 7f7a72ffd700 1 -- 172.21.15.107:0/4176725263 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f7a7805bdb0 con 0x7f7a840fe070 2024-04-22T23:42:01.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:00 smithi157 ceph-mon[79403]: pgmap v150: 65 pgs: 65 active+clean; 277 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:42:01.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:00 smithi157 ceph-mon[79403]: from='client.34344 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:42:01.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.202+0000 7f7a88b39700 1 -- 172.21.15.107:0/4176725263 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7a8404bed0 con 0x7f7a840fe070 2024-04-22T23:42:01.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.202+0000 7f7a72ffd700 1 -- 172.21.15.107:0/4176725263 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1909 (secure 0 0 0) 0x7f7a7805b940 con 0x7f7a840fe070 2024-04-22T23:42:01.195 INFO:teuthology.orchestra.run.smithi107.stdout:e21 2024-04-22T23:42:01.195 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T23:42:01.195 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:42:01.195 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-04-22T23:42:01.195 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:42:01.195 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-04-22T23:42:01.195 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-04-22T23:42:01.196 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 21 2024-04-22T23:42:01.196 INFO:teuthology.orchestra.run.smithi107.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-22T23:42:01.196 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-04-22T23:32:20.146130+0000 2024-04-22T23:42:01.196 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-04-22T23:39:06.006385+0000 2024-04-22T23:42:01.196 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-04-22T23:42:01.196 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-04-22T23:42:01.196 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-04-22T23:42:01.196 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-04-22T23:42:01.196 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-04-22T23:42:01.196 INFO:teuthology.orchestra.run.smithi107.stdout:max_xattr_size 65536 2024-04-22T23:42:01.196 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-04-22T23:42:01.196 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-04-22T23:42:01.196 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 46 2024-04-22T23:42:01.196 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:42:01.197 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 2 2024-04-22T23:42:01.197 INFO:teuthology.orchestra.run.smithi107.stdout:in 0,1 2024-04-22T23:42:01.197 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=14468,1=24265} 2024-04-22T23:42:01.197 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-04-22T23:42:01.197 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-04-22T23:42:01.197 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-04-22T23:42:01.197 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-04-22T23:42:01.197 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-04-22T23:42:01.197 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data disabled 2024-04-22T23:42:01.197 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-04-22T23:42:01.197 INFO:teuthology.orchestra.run.smithi107.stdout:bal_rank_mask -1 2024-04-22T23:42:01.197 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-04-22T23:42:01.197 INFO:teuthology.orchestra.run.smithi107.stdout:qdb_cluster leader: 14468 members: 24265,14468 2024-04-22T23:42:01.197 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.ktuxjl{0:14468} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.107:6828/3335313253,v1:172.21.15.107:6829/3335313253] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:42:01.198 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi157.rcaris{1:24265} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.157:6824/2779251893,v1:172.21.15.157:6825/2779251893] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:42:01.198 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:42:01.198 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:42:01.198 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-04-22T23:42:01.198 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:42:01.198 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.dfvvkw{-1:14484} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/3548717247,v1:172.21.15.107:6827/3548717247] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:42:01.198 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi157.ufywse{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.157:6826/4072444199,v1:172.21.15.157:6827/4072444199] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:42:01.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.205+0000 7f7a88b39700 1 -- 172.21.15.107:0/4176725263 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f7a6c06ec00 msgr2=0x7f7a6c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:01.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.205+0000 7f7a88b39700 1 --2- 172.21.15.107:0/4176725263 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f7a6c06ec00 0x7f7a6c0710b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c007a40 tx=0x7f7a7c009da0 comp rx=0 tx=0).stop 2024-04-22T23:42:01.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.205+0000 7f7a88b39700 1 -- 172.21.15.107:0/4176725263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a840fe070 msgr2=0x7f7a8410df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:01.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.205+0000 7f7a88b39700 1 --2- 172.21.15.107:0/4176725263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a840fe070 0x7f7a8410df60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7a7800b3b0 tx=0x7f7a78009e30 comp rx=0 tx=0).stop 2024-04-22T23:42:01.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.205+0000 7f7a88b39700 1 -- 172.21.15.107:0/4176725263 shutdown_connections 2024-04-22T23:42:01.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.205+0000 7f7a88b39700 1 --2- 172.21.15.107:0/4176725263 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7a840fcf00 0x7f7a8410da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:01.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.205+0000 7f7a88b39700 1 --2- 172.21.15.107:0/4176725263 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f7a6c06ec00 0x7f7a6c0710b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:01.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.205+0000 7f7a88b39700 1 --2- 172.21.15.107:0/4176725263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a840fe070 0x7f7a8410df60 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:01.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.205+0000 7f7a88b39700 1 -- 172.21.15.107:0/4176725263 >> 172.21.15.107:0/4176725263 conn(0x7f7a840f85e0 msgr2=0x7f7a840f9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:01.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.205+0000 7f7a88b39700 1 -- 172.21.15.107:0/4176725263 shutdown_connections 2024-04-22T23:42:01.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.205+0000 7f7a88b39700 1 -- 172.21.15.107:0/4176725263 wait complete. 2024-04-22T23:42:01.199 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 21 2024-04-22T23:42:01.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.305+0000 7f16678e2700 1 -- 172.21.15.107:0/570044247 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16600fbd00 msgr2=0x7f16600fc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:01.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.305+0000 7f16678e2700 1 --2- 172.21.15.107:0/570044247 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16600fbd00 0x7f16600fc110 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f165c004940 tx=0x7f165c009e30 comp rx=0 tx=0).stop 2024-04-22T23:42:01.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.306+0000 7f16678e2700 1 -- 172.21.15.107:0/570044247 shutdown_connections 2024-04-22T23:42:01.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.306+0000 7f16678e2700 1 --2- 172.21.15.107:0/570044247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16600fcfb0 0x7f16600fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:01.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.306+0000 7f16678e2700 1 --2- 172.21.15.107:0/570044247 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16600fbd00 0x7f16600fc110 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:01.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.306+0000 7f16678e2700 1 -- 172.21.15.107:0/570044247 >> 172.21.15.107:0/570044247 conn(0x7f16600f7480 msgr2=0x7f16600f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:01.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.306+0000 7f16678e2700 1 -- 172.21.15.107:0/570044247 shutdown_connections 2024-04-22T23:42:01.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.306+0000 7f16678e2700 1 -- 172.21.15.107:0/570044247 wait complete. 2024-04-22T23:42:01.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.306+0000 7f16678e2700 1 Processor -- start 2024-04-22T23:42:01.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.306+0000 7f16678e2700 1 -- start start 2024-04-22T23:42:01.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.306+0000 7f16678e2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16600fbd00 0x7f166010da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:01.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.306+0000 7f16678e2700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16600fcfb0 0x7f166010df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:01.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.306+0000 7f16678e2700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f166010e550 con 0x7f16600fbd00 2024-04-22T23:42:01.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.306+0000 7f16678e2700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f166010e6c0 con 0x7f16600fcfb0 2024-04-22T23:42:01.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.306+0000 7f166567e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16600fbd00 0x7f166010da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:01.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.306+0000 7f166567e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16600fbd00 0x7f166010da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49406/0 (socket says 172.21.15.107:49406) 2024-04-22T23:42:01.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.306+0000 7f166567e700 1 -- 172.21.15.107:0/4041532219 learned_addr learned my addr 172.21.15.107:0/4041532219 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:42:01.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.307+0000 7f1664e7d700 1 --2- 172.21.15.107:0/4041532219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16600fcfb0 0x7f166010df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:01.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.307+0000 7f166567e700 1 -- 172.21.15.107:0/4041532219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16600fcfb0 msgr2=0x7f166010df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:01.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.307+0000 7f166567e700 1 --2- 172.21.15.107:0/4041532219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16600fcfb0 0x7f166010df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:01.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.307+0000 7f166567e700 1 -- 172.21.15.107:0/4041532219 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f165c005040 con 0x7f16600fbd00 2024-04-22T23:42:01.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.307+0000 7f166567e700 1 --2- 172.21.15.107:0/4041532219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16600fbd00 0x7f166010da10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f165c009fd0 tx=0x7f165c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:01.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.307+0000 7f1655ffb700 1 -- 172.21.15.107:0/4041532219 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f165c00e040 con 0x7f16600fbd00 2024-04-22T23:42:01.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.307+0000 7f1655ffb700 1 -- 172.21.15.107:0/4041532219 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f165c00bcc0 con 0x7f16600fbd00 2024-04-22T23:42:01.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.307+0000 7f16678e2700 1 -- 172.21.15.107:0/4041532219 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1660113070 con 0x7f16600fbd00 2024-04-22T23:42:01.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.307+0000 7f1655ffb700 1 -- 172.21.15.107:0/4041532219 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f165c01c630 con 0x7f16600fbd00 2024-04-22T23:42:01.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.307+0000 7f16678e2700 1 -- 172.21.15.107:0/4041532219 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1660113630 con 0x7f16600fbd00 2024-04-22T23:42:01.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.308+0000 7f16678e2700 1 -- 172.21.15.107:0/4041532219 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1660101b40 con 0x7f16600fbd00 2024-04-22T23:42:01.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.309+0000 7f1655ffb700 1 -- 172.21.15.107:0/4041532219 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f165c013980 con 0x7f16600fbd00 2024-04-22T23:42:01.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.309+0000 7f1655ffb700 1 --2- 172.21.15.107:0/4041532219 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f164c06ec00 0x7f164c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:01.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.310+0000 7f1664e7d700 1 --2- 172.21.15.107:0/4041532219 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f164c06ec00 0x7f164c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:01.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.310+0000 7f1655ffb700 1 -- 172.21.15.107:0/4041532219 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f165c08faa0 con 0x7f16600fbd00 2024-04-22T23:42:01.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.310+0000 7f1664e7d700 1 --2- 172.21.15.107:0/4041532219 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f164c06ec00 0x7f164c0710b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1650009fd0 tx=0x7f1650009f20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:01.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.314+0000 7f1655ffb700 1 -- 172.21.15.107:0/4041532219 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f165c05c400 con 0x7f16600fbd00 2024-04-22T23:42:01.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.531+0000 7f16678e2700 1 -- 172.21.15.107:0/4041532219 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f166002b340 con 0x7f164c06ec00 2024-04-22T23:42:01.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.533+0000 7f1655ffb700 1 -- 172.21.15.107:0/4041532219 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f166002b340 con 0x7f164c06ec00 2024-04-22T23:42:01.526 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-22T23:42:01.526 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c", 2024-04-22T23:42:01.526 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-04-22T23:42:01.526 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-22T23:42:01.526 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [ 2024-04-22T23:42:01.526 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr", 2024-04-22T23:42:01.526 INFO:teuthology.orchestra.run.smithi107.stdout: "crash", 2024-04-22T23:42:01.526 INFO:teuthology.orchestra.run.smithi107.stdout: "mon" 2024-04-22T23:42:01.526 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-22T23:42:01.526 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "11/21 daemons upgraded", 2024-04-22T23:42:01.526 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Currently upgrading osd daemons", 2024-04-22T23:42:01.526 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-04-22T23:42:01.526 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-22T23:42:01.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.535+0000 7f16678e2700 1 -- 172.21.15.107:0/4041532219 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f164c06ec00 msgr2=0x7f164c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:01.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.535+0000 7f16678e2700 1 --2- 172.21.15.107:0/4041532219 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f164c06ec00 0x7f164c0710b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1650009fd0 tx=0x7f1650009f20 comp rx=0 tx=0).stop 2024-04-22T23:42:01.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.535+0000 7f16678e2700 1 -- 172.21.15.107:0/4041532219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16600fbd00 msgr2=0x7f166010da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:01.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.535+0000 7f16678e2700 1 --2- 172.21.15.107:0/4041532219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16600fbd00 0x7f166010da10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f165c009fd0 tx=0x7f165c009e30 comp rx=0 tx=0).stop 2024-04-22T23:42:01.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.535+0000 7f16678e2700 1 -- 172.21.15.107:0/4041532219 shutdown_connections 2024-04-22T23:42:01.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.535+0000 7f16678e2700 1 --2- 172.21.15.107:0/4041532219 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f164c06ec00 0x7f164c0710b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:01.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.535+0000 7f16678e2700 1 --2- 172.21.15.107:0/4041532219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16600fbd00 0x7f166010da10 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:01.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.535+0000 7f16678e2700 1 --2- 172.21.15.107:0/4041532219 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16600fcfb0 0x7f166010df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:01.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.535+0000 7f16678e2700 1 -- 172.21.15.107:0/4041532219 >> 172.21.15.107:0/4041532219 conn(0x7f16600f7480 msgr2=0x7f1660066930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:01.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.535+0000 7f16678e2700 1 -- 172.21.15.107:0/4041532219 shutdown_connections 2024-04-22T23:42:01.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.535+0000 7f16678e2700 1 -- 172.21.15.107:0/4041532219 wait complete. 2024-04-22T23:42:01.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.635+0000 7f5c96f35700 1 -- 172.21.15.107:0/2262340137 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c900fce80 msgr2=0x7f5c900fd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:01.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.635+0000 7f5c96f35700 1 --2- 172.21.15.107:0/2262340137 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c900fce80 0x7f5c900fd290 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5c84004940 tx=0x7f5c84009e30 comp rx=0 tx=0).stop 2024-04-22T23:42:01.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.635+0000 7f5c96f35700 1 -- 172.21.15.107:0/2262340137 shutdown_connections 2024-04-22T23:42:01.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.635+0000 7f5c96f35700 1 --2- 172.21.15.107:0/2262340137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c900fdff0 0x7f5c900fe460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:01.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.635+0000 7f5c96f35700 1 --2- 172.21.15.107:0/2262340137 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c900fce80 0x7f5c900fd290 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:01.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.635+0000 7f5c96f35700 1 -- 172.21.15.107:0/2262340137 >> 172.21.15.107:0/2262340137 conn(0x7f5c900f8580 msgr2=0x7f5c900fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:01.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.635+0000 7f5c96f35700 1 -- 172.21.15.107:0/2262340137 shutdown_connections 2024-04-22T23:42:01.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.635+0000 7f5c96f35700 1 -- 172.21.15.107:0/2262340137 wait complete. 2024-04-22T23:42:01.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.635+0000 7f5c96f35700 1 Processor -- start 2024-04-22T23:42:01.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.635+0000 7f5c96f35700 1 -- start start 2024-04-22T23:42:01.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.635+0000 7f5c96f35700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c900fce80 0x7f5c9010d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:01.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.635+0000 7f5c96f35700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c900fdff0 0x7f5c9010dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:01.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.636+0000 7f5c96f35700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9010e4a0 con 0x7f5c900fce80 2024-04-22T23:42:01.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.636+0000 7f5c96f35700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9010e610 con 0x7f5c900fdff0 2024-04-22T23:42:01.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.636+0000 7f5c94cd1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c900fce80 0x7f5c9010d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:01.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.636+0000 7f5c94cd1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c900fce80 0x7f5c9010d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56010/0 (socket says 172.21.15.107:56010) 2024-04-22T23:42:01.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.636+0000 7f5c94cd1700 1 -- 172.21.15.107:0/3337546294 learned_addr learned my addr 172.21.15.107:0/3337546294 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:42:01.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.636+0000 7f5c8ffff700 1 --2- 172.21.15.107:0/3337546294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c900fdff0 0x7f5c9010dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:01.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.636+0000 7f5c94cd1700 1 -- 172.21.15.107:0/3337546294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c900fdff0 msgr2=0x7f5c9010dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:01.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.636+0000 7f5c94cd1700 1 --2- 172.21.15.107:0/3337546294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c900fdff0 0x7f5c9010dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:01.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.636+0000 7f5c94cd1700 1 -- 172.21.15.107:0/3337546294 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c84005040 con 0x7f5c900fce80 2024-04-22T23:42:01.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.636+0000 7f5c94cd1700 1 --2- 172.21.15.107:0/3337546294 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c900fce80 0x7f5c9010d960 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5c8400b3b0 tx=0x7f5c84009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:01.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.636+0000 7f5c8d7fa700 1 -- 172.21.15.107:0/3337546294 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c8400e040 con 0x7f5c900fce80 2024-04-22T23:42:01.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.636+0000 7f5c8d7fa700 1 -- 172.21.15.107:0/3337546294 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5c8400bcc0 con 0x7f5c900fce80 2024-04-22T23:42:01.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.636+0000 7f5c8d7fa700 1 -- 172.21.15.107:0/3337546294 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c8401c630 con 0x7f5c900fce80 2024-04-22T23:42:01.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.636+0000 7f5c96f35700 1 -- 172.21.15.107:0/3337546294 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c90113010 con 0x7f5c900fce80 2024-04-22T23:42:01.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.636+0000 7f5c96f35700 1 -- 172.21.15.107:0/3337546294 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c90113580 con 0x7f5c900fce80 2024-04-22T23:42:01.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.637+0000 7f5c96f35700 1 -- 172.21.15.107:0/3337546294 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c9004c710 con 0x7f5c900fce80 2024-04-22T23:42:01.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.638+0000 7f5c8d7fa700 1 -- 172.21.15.107:0/3337546294 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f5c84013980 con 0x7f5c900fce80 2024-04-22T23:42:01.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.638+0000 7f5c8d7fa700 1 --2- 172.21.15.107:0/3337546294 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f5c8006ec00 0x7f5c800710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:01.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.638+0000 7f5c8ffff700 1 --2- 172.21.15.107:0/3337546294 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f5c8006ec00 0x7f5c800710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:01.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.638+0000 7f5c8d7fa700 1 -- 172.21.15.107:0/3337546294 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f5c8408f5a0 con 0x7f5c900fce80 2024-04-22T23:42:01.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.639+0000 7f5c8ffff700 1 --2- 172.21.15.107:0/3337546294 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f5c8006ec00 0x7f5c800710b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c0041d0 tx=0x7f5c7c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:01.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.643+0000 7f5c8d7fa700 1 -- 172.21.15.107:0/3337546294 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f5c8405be80 con 0x7f5c900fce80 2024-04-22T23:42:01.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.933+0000 7f5c96f35700 1 -- 172.21.15.107:0/3337546294 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5c901058e0 con 0x7f5c900fce80 2024-04-22T23:42:01.927 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:01 smithi107 ceph-mon[90179]: from='client.34348 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:42:01.927 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:01 smithi107 ceph-mon[90179]: from='client.34352 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:42:01.927 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:01 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:42:01.927 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:01 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/2220330817' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:42:01.927 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:01 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/4176725263' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:42:01.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.934+0000 7f5c8d7fa700 1 -- 172.21.15.107:0/3337546294 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f5c8405ba10 con 0x7f5c900fce80 2024-04-22T23:42:01.927 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_OK 2024-04-22T23:42:01.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.936+0000 7f5c96f35700 1 -- 172.21.15.107:0/3337546294 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f5c8006ec00 msgr2=0x7f5c800710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:01.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.936+0000 7f5c96f35700 1 --2- 172.21.15.107:0/3337546294 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f5c8006ec00 0x7f5c800710b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c0041d0 tx=0x7f5c7c009040 comp rx=0 tx=0).stop 2024-04-22T23:42:01.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.936+0000 7f5c96f35700 1 -- 172.21.15.107:0/3337546294 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c900fce80 msgr2=0x7f5c9010d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:01.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.936+0000 7f5c96f35700 1 --2- 172.21.15.107:0/3337546294 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c900fce80 0x7f5c9010d960 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5c8400b3b0 tx=0x7f5c84009e30 comp rx=0 tx=0).stop 2024-04-22T23:42:01.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.936+0000 7f5c96f35700 1 -- 172.21.15.107:0/3337546294 shutdown_connections 2024-04-22T23:42:01.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.936+0000 7f5c96f35700 1 --2- 172.21.15.107:0/3337546294 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f5c8006ec00 0x7f5c800710b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:01.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.936+0000 7f5c96f35700 1 --2- 172.21.15.107:0/3337546294 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c900fce80 0x7f5c9010d960 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:01.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.936+0000 7f5c96f35700 1 --2- 172.21.15.107:0/3337546294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c900fdff0 0x7f5c9010dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:01.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.936+0000 7f5c96f35700 1 -- 172.21.15.107:0/3337546294 >> 172.21.15.107:0/3337546294 conn(0x7f5c900f8580 msgr2=0x7f5c900f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:01.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.936+0000 7f5c96f35700 1 -- 172.21.15.107:0/3337546294 shutdown_connections 2024-04-22T23:42:01.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:01.936+0000 7f5c96f35700 1 -- 172.21.15.107:0/3337546294 wait complete. 2024-04-22T23:42:02.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:01 smithi157 ceph-mon[79403]: from='client.34348 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:42:02.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:01 smithi157 ceph-mon[79403]: from='client.34352 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:42:02.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:01 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:42:02.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:01 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/2220330817' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:42:02.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:01 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/4176725263' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:42:03.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:02 smithi157 ceph-mon[79403]: from='client.34364 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:42:03.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:02 smithi157 ceph-mon[79403]: pgmap v151: 65 pgs: 65 active+clean; 277 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:42:03.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:02 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/3337546294' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:42:03.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:02 smithi107 ceph-mon[90179]: from='client.34364 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:42:03.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:02 smithi107 ceph-mon[90179]: pgmap v151: 65 pgs: 65 active+clean; 277 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:42:03.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:02 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/3337546294' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:42:03.864 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:03 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-22T23:42:04.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:03 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-22T23:42:05.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:04 smithi157 ceph-mon[79403]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-22T23:42:05.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:04 smithi157 ceph-mon[79403]: Upgrade: osd.5 is safe to restart 2024-04-22T23:42:05.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:04 smithi157 ceph-mon[79403]: pgmap v152: 65 pgs: 65 active+clean; 277 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:42:05.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:04 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:42:05.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:04 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-22T23:42:05.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:04 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:42:05.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:04 smithi107 ceph-mon[90179]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-22T23:42:05.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:04 smithi107 ceph-mon[90179]: Upgrade: osd.5 is safe to restart 2024-04-22T23:42:05.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:04 smithi107 ceph-mon[90179]: pgmap v152: 65 pgs: 65 active+clean; 277 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:42:05.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:04 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:42:05.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:04 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-22T23:42:05.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:04 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:42:06.123 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:05 smithi157 ceph-mon[79403]: Upgrade: Updating osd.5 2024-04-22T23:42:06.123 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:05 smithi157 ceph-mon[79403]: Deploying daemon osd.5 on smithi157 2024-04-22T23:42:06.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:05 smithi107 ceph-mon[90179]: Upgrade: Updating osd.5 2024-04-22T23:42:06.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:05 smithi107 ceph-mon[90179]: Deploying daemon osd.5 on smithi157 2024-04-22T23:42:07.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:06 smithi157 ceph-mon[79403]: pgmap v153: 65 pgs: 65 active+clean; 277 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:42:07.202 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:06 smithi107 ceph-mon[90179]: pgmap v153: 65 pgs: 65 active+clean; 277 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:42:08.144 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:42:07 smithi157 systemd[1]: Stopping Ceph osd.5 for 8252f6f2-00ff-11ef-bc93-c7b262605968... 2024-04-22T23:42:08.630 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:42:08 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5[52684]: 2024-04-22T23:42:08.151+0000 7f96e022d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T23:42:08.630 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:42:08 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5[52684]: 2024-04-22T23:42:08.151+0000 7f96e022d700 -1 osd.5 74 *** Got signal Terminated *** 2024-04-22T23:42:08.630 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:42:08 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5[52684]: 2024-04-22T23:42:08.151+0000 7f96e022d700 -1 osd.5 74 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T23:42:09.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:08 smithi157 ceph-mon[79403]: pgmap v154: 65 pgs: 65 active+clean; 277 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:42:09.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:08 smithi157 ceph-mon[79403]: osd.5 marked itself down and dead 2024-04-22T23:42:09.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:08 smithi107 ceph-mon[90179]: pgmap v154: 65 pgs: 65 active+clean; 277 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:42:09.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:08 smithi107 ceph-mon[90179]: osd.5 marked itself down and dead 2024-04-22T23:42:09.520 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:42:09 smithi157 podman[98702]: 2024-04-22 23:42:09.256071506 +0000 UTC m=+1.212045982 container died 4c5b7a7ef8bdfcb529e35c8729f5da7b55e7fd0e641f284cbcd2dae4607b31b1 (image=quay.ceph.io/ceph-ci/ceph@sha256:3890cd726bd5e73ae623dabdf4ed76dd5df81c936b8bd8d761026babda969466, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5, RELEASE=quincy-70d0cb4, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T23:42:09.794 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:42:09 smithi157 podman[98702]: 2024-04-22 23:42:09.601026822 +0000 UTC m=+1.557001341 container cleanup 4c5b7a7ef8bdfcb529e35c8729f5da7b55e7fd0e641f284cbcd2dae4607b31b1 (image=quay.ceph.io/ceph-ci/ceph@sha256:3890cd726bd5e73ae623dabdf4ed76dd5df81c936b8bd8d761026babda969466, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-70d0cb4, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-22T23:42:09.794 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:42:09 smithi157 bash[98702]: ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5 2024-04-22T23:42:10.130 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:42:09 smithi157 podman[98722]: 2024-04-22 23:42:09.802197787 +0000 UTC m=+0.540295972 container remove 4c5b7a7ef8bdfcb529e35c8729f5da7b55e7fd0e641f284cbcd2dae4607b31b1 (image=quay.ceph.io/ceph-ci/ceph@sha256:3890cd726bd5e73ae623dabdf4ed76dd5df81c936b8bd8d761026babda969466, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=quincy-70d0cb4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=) 2024-04-22T23:42:10.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:09 smithi157 ceph-mon[79403]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-22T23:42:10.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:09 smithi157 ceph-mon[79403]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-04-22T23:42:10.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:09 smithi157 ceph-mon[79403]: osdmap e75: 6 total, 5 up, 6 in 2024-04-22T23:42:10.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:09 smithi107 ceph-mon[90179]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-22T23:42:10.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:09 smithi107 ceph-mon[90179]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-04-22T23:42:10.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:09 smithi107 ceph-mon[90179]: osdmap e75: 6 total, 5 up, 6 in 2024-04-22T23:42:10.385 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:42:10 smithi157 podman[98812]: 2024-04-22 23:42:10.129666466 +0000 UTC m=+0.018299832 image pull a0434699c1d76af4eea1fc95ff0a22ea73d1e04e609464fbf658d6f39bc7f2fc quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c 2024-04-22T23:42:10.385 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:42:10 smithi157 podman[98812]: 2024-04-22 23:42:10.244776005 +0000 UTC m=+0.133409370 container create 2c1439113ac8390e53e50feafa687f6a2362a07dd8c9eaf71ea47b9ab8766dcc (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True) 2024-04-22T23:42:10.784 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:42:10 smithi157 podman[98812]: 2024-04-22 23:42:10.497726163 +0000 UTC m=+0.386359527 container init 2c1439113ac8390e53e50feafa687f6a2362a07dd8c9eaf71ea47b9ab8766dcc (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, CEPH_POINT_RELEASE=) 2024-04-22T23:42:10.784 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:42:10 smithi157 podman[98812]: 2024-04-22 23:42:10.500843896 +0000 UTC m=+0.389477268 container start 2c1439113ac8390e53e50feafa687f6a2362a07dd8c9eaf71ea47b9ab8766dcc (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, io.buildah.version=1.33.5) 2024-04-22T23:42:10.784 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:42:10 smithi157 podman[98812]: 2024-04-22 23:42:10.597680665 +0000 UTC m=+0.486314039 container attach 2c1439113ac8390e53e50feafa687f6a2362a07dd8c9eaf71ea47b9ab8766dcc (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416) 2024-04-22T23:42:11.101 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:10 smithi157 ceph-mon[79403]: pgmap v156: 65 pgs: 13 stale+active+clean, 52 active+clean; 277 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:42:11.101 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:10 smithi157 ceph-mon[79403]: osdmap e76: 6 total, 5 up, 6 in 2024-04-22T23:42:11.101 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:10 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:42:11.101 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:10 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:42:11.102 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:42:10 smithi157 podman[98812]: 2024-04-22 23:42:10.792026491 +0000 UTC m=+0.680659859 container died 2c1439113ac8390e53e50feafa687f6a2362a07dd8c9eaf71ea47b9ab8766dcc (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-22T23:42:11.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:10 smithi107 ceph-mon[90179]: pgmap v156: 65 pgs: 13 stale+active+clean, 52 active+clean; 277 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:42:11.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:10 smithi107 ceph-mon[90179]: osdmap e76: 6 total, 5 up, 6 in 2024-04-22T23:42:11.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:10 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:42:11.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:10 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:42:11.630 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:42:11 smithi157 podman[98838]: 2024-04-22 23:42:11.376297957 +0000 UTC m=+0.578424001 container remove 2c1439113ac8390e53e50feafa687f6a2362a07dd8c9eaf71ea47b9ab8766dcc (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-22T23:42:11.630 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:42:11 smithi157 systemd[1]: ceph-8252f6f2-00ff-11ef-bc93-c7b262605968@osd.5.service: Deactivated successfully. 2024-04-22T23:42:11.630 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:42:11 smithi157 systemd[1]: Stopped Ceph osd.5 for 8252f6f2-00ff-11ef-bc93-c7b262605968. 2024-04-22T23:42:11.630 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:42:11 smithi157 systemd[1]: ceph-8252f6f2-00ff-11ef-bc93-c7b262605968@osd.5.service: Consumed 59.076s CPU time. 2024-04-22T23:42:11.630 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:42:11 smithi157 systemd[1]: Starting Ceph osd.5 for 8252f6f2-00ff-11ef-bc93-c7b262605968... 2024-04-22T23:42:12.072 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:11 smithi157 ceph-mon[79403]: pgmap v158: 65 pgs: 6 active+undersized+degraded, 3 peering, 5 stale+active+clean, 6 active+undersized, 45 active+clean; 277 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 14/339 objects degraded (4.130%) 2024-04-22T23:42:12.072 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:11 smithi157 ceph-mon[79403]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-22T23:42:12.072 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:11 smithi157 ceph-mon[79403]: Health check failed: Degraded data redundancy: 14/339 objects degraded (4.130%), 6 pgs degraded (PG_DEGRADED) 2024-04-22T23:42:12.072 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:42:11 smithi157 podman[98940]: 2024-04-22 23:42:11.722769848 +0000 UTC m=+0.018878471 image pull a0434699c1d76af4eea1fc95ff0a22ea73d1e04e609464fbf658d6f39bc7f2fc quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c 2024-04-22T23:42:12.072 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:42:11 smithi157 podman[98940]: 2024-04-22 23:42:11.827218903 +0000 UTC m=+0.123327527 container create 4494a32ee23673d2bf546531565f30b69e5a10e1064befe1d9cb1eea3b1e70a9 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5-activate, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-22T23:42:12.072 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:42:12 smithi157 podman[98940]: 2024-04-22 23:42:12.079660588 +0000 UTC m=+0.375769202 container init 4494a32ee23673d2bf546531565f30b69e5a10e1064befe1d9cb1eea3b1e70a9 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5-activate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T23:42:12.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:11 smithi107 ceph-mon[90179]: pgmap v158: 65 pgs: 6 active+undersized+degraded, 3 peering, 5 stale+active+clean, 6 active+undersized, 45 active+clean; 277 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 14/339 objects degraded (4.130%) 2024-04-22T23:42:12.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:11 smithi107 ceph-mon[90179]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-22T23:42:12.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:11 smithi107 ceph-mon[90179]: Health check failed: Degraded data redundancy: 14/339 objects degraded (4.130%), 6 pgs degraded (PG_DEGRADED) 2024-04-22T23:42:12.380 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:42:12 smithi157 podman[98940]: 2024-04-22 23:42:12.082929755 +0000 UTC m=+0.379038371 container start 4494a32ee23673d2bf546531565f30b69e5a10e1064befe1d9cb1eea3b1e70a9 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5-activate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T23:42:12.380 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:42:12 smithi157 podman[98940]: 2024-04-22 23:42:12.162933297 +0000 UTC m=+0.459041922 container attach 4494a32ee23673d2bf546531565f30b69e5a10e1064befe1d9cb1eea3b1e70a9 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5-activate, io.buildah.version=1.33.5, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-22T23:42:12.880 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:42:12 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5-activate[98955]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-22T23:42:12.880 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:42:12 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5-activate[98955]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-22T23:42:12.881 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:42:12 smithi157 bash[98940]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-22T23:42:12.881 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:42:12 smithi157 bash[98940]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-22T23:42:12.881 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:42:12 smithi157 bash[98940]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-22T23:42:12.881 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:42:12 smithi157 bash[98940]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-22T23:42:12.881 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:42:12 smithi157 bash[98940]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-22T23:42:12.881 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:42:12 smithi157 bash[98940]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-22T23:42:12.881 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:42:12 smithi157 bash[98940]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-04-22T23:42:12.881 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:42:12 smithi157 bash[98940]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-22T23:42:12.881 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:42:12 smithi157 bash[98940]: --> ceph-volume raw activate successful for osd ID: 5 2024-04-22T23:42:12.881 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:42:12 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5-activate[98955]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-22T23:42:12.881 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:42:12 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5-activate[98955]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-22T23:42:12.881 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:42:12 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5-activate[98955]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-22T23:42:12.881 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:42:12 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5-activate[98955]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-22T23:42:12.881 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:42:12 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5-activate[98955]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-04-22T23:42:12.881 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:42:12 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5-activate[98955]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-22T23:42:12.881 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:42:12 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5-activate[98955]: --> ceph-volume raw activate successful for osd ID: 5 2024-04-22T23:42:12.882 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:42:12 smithi157 podman[98940]: 2024-04-22 23:42:12.595116502 +0000 UTC m=+0.891225117 container died 4494a32ee23673d2bf546531565f30b69e5a10e1064befe1d9cb1eea3b1e70a9 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5-activate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-22T23:42:13.479 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:42:13 smithi157 podman[99096]: 2024-04-22 23:42:13.16696058 +0000 UTC m=+0.566408674 container remove 4494a32ee23673d2bf546531565f30b69e5a10e1064befe1d9cb1eea3b1e70a9 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, ceph=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-22T23:42:13.479 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:42:13 smithi157 podman[99161]: 2024-04-22 23:42:13.376111166 +0000 UTC m=+0.112588122 container create 0ee423135fdb3187335ab2befe55b4aee05754903fe6b26051f148ffea5a0588 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T23:42:13.479 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:42:13 smithi157 podman[99161]: 2024-04-22 23:42:13.28196508 +0000 UTC m=+0.018442046 image pull a0434699c1d76af4eea1fc95ff0a22ea73d1e04e609464fbf658d6f39bc7f2fc quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c 2024-04-22T23:42:13.880 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:42:13 smithi157 podman[99161]: 2024-04-22 23:42:13.603448464 +0000 UTC m=+0.339925421 container init 0ee423135fdb3187335ab2befe55b4aee05754903fe6b26051f148ffea5a0588 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-22T23:42:13.880 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:42:13 smithi157 podman[99161]: 2024-04-22 23:42:13.606436701 +0000 UTC m=+0.342913660 container start 0ee423135fdb3187335ab2befe55b4aee05754903fe6b26051f148ffea5a0588 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, ceph=True, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T23:42:13.881 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:42:13 smithi157 ceph-osd[99181]: -- 172.21.15.157:0/2273281371 <== mon.1 v2:172.21.15.157:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5622d1d2d680 con 0x5622d1cf6c00 2024-04-22T23:42:13.881 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:42:13 smithi157 bash[99161]: 0ee423135fdb3187335ab2befe55b4aee05754903fe6b26051f148ffea5a0588 2024-04-22T23:42:13.881 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:42:13 smithi157 systemd[1]: Started Ceph osd.5 for 8252f6f2-00ff-11ef-bc93-c7b262605968. 2024-04-22T23:42:14.585 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:42:14 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5[99177]: 2024-04-22T23:42:14.151+0000 7f5d54b3f7c0 -1 Falling back to public interface 2024-04-22T23:42:14.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:14 smithi157 ceph-mon[79403]: pgmap v159: 65 pgs: 12 active+undersized+degraded, 3 peering, 13 active+undersized, 37 active+clean; 277 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 44/339 objects degraded (12.979%) 2024-04-22T23:42:14.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:14 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:42:14.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:14 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:42:14.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:14 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:42:15.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:14 smithi107 ceph-mon[90179]: pgmap v159: 65 pgs: 12 active+undersized+degraded, 3 peering, 13 active+undersized, 37 active+clean; 277 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 44/339 objects degraded (12.979%) 2024-04-22T23:42:15.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:14 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:42:15.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:14 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:42:15.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:14 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:42:16.120 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:15 smithi157 ceph-mon[79403]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-22T23:42:16.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:15 smithi107 ceph-mon[90179]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-22T23:42:17.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:16 smithi157 ceph-mon[79403]: pgmap v160: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 277 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 51/339 objects degraded (15.044%) 2024-04-22T23:42:17.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:16 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:42:17.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:16 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:42:17.202 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:16 smithi107 ceph-mon[90179]: pgmap v160: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 277 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 51/339 objects degraded (15.044%) 2024-04-22T23:42:17.202 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:16 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:42:17.202 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:16 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:42:18.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:17 smithi157 ceph-mon[79403]: Health check update: Degraded data redundancy: 51/339 objects degraded (15.044%), 14 pgs degraded (PG_DEGRADED) 2024-04-22T23:42:18.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:17 smithi107 ceph-mon[90179]: Health check update: Degraded data redundancy: 51/339 objects degraded (15.044%), 14 pgs degraded (PG_DEGRADED) 2024-04-22T23:42:18.806 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:42:18 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5[99177]: 2024-04-22T23:42:18.522+0000 7f5d54b3f7c0 -1 osd.5 74 log_to_monitors true 2024-04-22T23:42:19.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:18 smithi157 ceph-mon[79403]: pgmap v161: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 277 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 51/339 objects degraded (15.044%) 2024-04-22T23:42:19.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:18 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:42:19.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:18 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:42:19.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:18 smithi157 ceph-mon[79403]: from='osd.5 [v2:172.21.15.157:6816/3093110961,v1:172.21.15.157:6817/3093110961]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-22T23:42:19.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:18 smithi157 ceph-mon[79403]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-22T23:42:19.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:18 smithi107 ceph-mon[90179]: pgmap v161: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 277 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 51/339 objects degraded (15.044%) 2024-04-22T23:42:19.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:18 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:42:19.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:18 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:42:19.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:18 smithi107 ceph-mon[90179]: from='osd.5 [v2:172.21.15.157:6816/3093110961,v1:172.21.15.157:6817/3093110961]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-22T23:42:19.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:18 smithi107 ceph-mon[90179]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-22T23:42:19.735 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:42:19 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5[99177]: 2024-04-22T23:42:19.535+0000 7f5d41f8f700 -1 osd.5 74 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T23:42:20.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:19 smithi157 ceph-mon[79403]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-22T23:42:20.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:19 smithi157 ceph-mon[79403]: osdmap e77: 6 total, 5 up, 6 in 2024-04-22T23:42:20.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:19 smithi157 ceph-mon[79403]: from='osd.5 [v2:172.21.15.157:6816/3093110961,v1:172.21.15.157:6817/3093110961]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-22T23:42:20.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:19 smithi157 ceph-mon[79403]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-22T23:42:20.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:19 smithi157 ceph-mon[79403]: from='osd.5 ' entity='osd.5' 2024-04-22T23:42:20.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:19 smithi157 ceph-mon[79403]: pgmap v163: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 277 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 51/339 objects degraded (15.044%) 2024-04-22T23:42:20.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:19 smithi107 ceph-mon[90179]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-22T23:42:20.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:19 smithi107 ceph-mon[90179]: osdmap e77: 6 total, 5 up, 6 in 2024-04-22T23:42:20.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:19 smithi107 ceph-mon[90179]: from='osd.5 [v2:172.21.15.157:6816/3093110961,v1:172.21.15.157:6817/3093110961]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-22T23:42:20.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:19 smithi107 ceph-mon[90179]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi157", "root=default"]}]: dispatch 2024-04-22T23:42:20.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:19 smithi107 ceph-mon[90179]: from='osd.5 ' entity='osd.5' 2024-04-22T23:42:20.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:19 smithi107 ceph-mon[90179]: pgmap v163: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 277 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 51/339 objects degraded (15.044%) 2024-04-22T23:42:21.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:20 smithi157 ceph-mon[79403]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-22T23:42:21.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:20 smithi157 ceph-mon[79403]: osd.5 [v2:172.21.15.157:6816/3093110961,v1:172.21.15.157:6817/3093110961] boot 2024-04-22T23:42:21.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:20 smithi157 ceph-mon[79403]: osdmap e78: 6 total, 6 up, 6 in 2024-04-22T23:42:21.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:20 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T23:42:21.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:20 smithi107 ceph-mon[90179]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-22T23:42:21.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:20 smithi107 ceph-mon[90179]: osd.5 [v2:172.21.15.157:6816/3093110961,v1:172.21.15.157:6817/3093110961] boot 2024-04-22T23:42:21.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:20 smithi107 ceph-mon[90179]: osdmap e78: 6 total, 6 up, 6 in 2024-04-22T23:42:21.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:20 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T23:42:22.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:22 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:42:22.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:22 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:42:22.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:22 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:42:22.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:22 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:42:22.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:22 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:42:22.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:22 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:42:22.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:22 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:42:22.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:22 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:42:22.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:22 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:42:22.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:22 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:42:22.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:22 smithi157 ceph-mon[79403]: Upgrade: Setting container_image for all osd 2024-04-22T23:42:22.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:22 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:42:22.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:22 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-04-22T23:42:22.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:22 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-04-22T23:42:22.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:22 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-04-22T23:42:22.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:22 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-04-22T23:42:22.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:22 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-04-22T23:42:22.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:22 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-04-22T23:42:22.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:22 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-04-22T23:42:22.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:22 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-04-22T23:42:22.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:22 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-04-22T23:42:22.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:22 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-04-22T23:42:22.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:22 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-04-22T23:42:22.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:22 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-04-22T23:42:22.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:22 smithi157 ceph-mon[79403]: Upgrade: Setting require_osd_release to 19 squid 2024-04-22T23:42:22.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:22 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-04-22T23:42:22.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:22 smithi157 ceph-mon[79403]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-04-22T23:42:22.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:22 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-04-22T23:42:22.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:22 smithi157 ceph-mon[79403]: osdmap e79: 6 total, 6 up, 6 in 2024-04-22T23:42:22.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:22 smithi157 ceph-mon[79403]: Upgrade: Scaling down filesystem cephfs 2024-04-22T23:42:22.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:22 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:42:22.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:22 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-04-22T23:42:22.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:22 smithi157 ceph-mon[79403]: pgmap v166: 65 pgs: 5 peering, 11 active+undersized+degraded, 12 active+undersized, 37 active+clean; 277 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 46/339 objects degraded (13.569%) 2024-04-22T23:42:22.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:22 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:42:22.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:22 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:42:22.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:22 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:42:22.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:22 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:42:22.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:22 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:42:22.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:22 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:42:22.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:22 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:42:22.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:22 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:42:22.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:22 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:42:22.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:22 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:42:22.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:22 smithi107 ceph-mon[90179]: Upgrade: Setting container_image for all osd 2024-04-22T23:42:22.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:22 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:42:22.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:22 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-04-22T23:42:22.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:22 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-04-22T23:42:22.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:22 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-04-22T23:42:22.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:22 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-04-22T23:42:22.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:22 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-04-22T23:42:22.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:22 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-04-22T23:42:22.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:22 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-04-22T23:42:22.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:22 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-04-22T23:42:22.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:22 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-04-22T23:42:22.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:22 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-04-22T23:42:22.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:22 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-04-22T23:42:22.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:22 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-04-22T23:42:22.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:22 smithi107 ceph-mon[90179]: Upgrade: Setting require_osd_release to 19 squid 2024-04-22T23:42:22.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:22 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-04-22T23:42:22.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:22 smithi107 ceph-mon[90179]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-04-22T23:42:22.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:22 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-04-22T23:42:22.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:22 smithi107 ceph-mon[90179]: osdmap e79: 6 total, 6 up, 6 in 2024-04-22T23:42:22.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:22 smithi107 ceph-mon[90179]: Upgrade: Scaling down filesystem cephfs 2024-04-22T23:42:22.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:22 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:42:22.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:22 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-04-22T23:42:22.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:22 smithi107 ceph-mon[90179]: pgmap v166: 65 pgs: 5 peering, 11 active+undersized+degraded, 12 active+undersized, 37 active+clean; 277 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 46/339 objects degraded (13.569%) 2024-04-22T23:42:23.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:23 smithi157 ceph-mon[79403]: stopping daemon mds.cephfs.smithi157.rcaris 2024-04-22T23:42:23.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:23 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-04-22T23:42:23.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:23 smithi157 ceph-mon[79403]: fsmap cephfs:2 {0=cephfs.smithi107.ktuxjl=up:active,1=cephfs.smithi157.rcaris=up:stopping} 2 up:standby 2024-04-22T23:42:23.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:23 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:42:23.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:23 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:42:23.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:23 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:42:23.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:23 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:42:23.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:23 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:42:23.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:23 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:42:23.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:23 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:42:23.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:23 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:42:23.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:23 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:42:23.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:23 smithi157 ceph-mon[79403]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2024-04-22T23:42:23.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:23 smithi157 ceph-mon[79403]: Health check update: Degraded data redundancy: 46/339 objects degraded (13.569%), 11 pgs degraded (PG_DEGRADED) 2024-04-22T23:42:23.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:23 smithi107 ceph-mon[90179]: stopping daemon mds.cephfs.smithi157.rcaris 2024-04-22T23:42:23.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:23 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-04-22T23:42:23.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:23 smithi107 ceph-mon[90179]: fsmap cephfs:2 {0=cephfs.smithi107.ktuxjl=up:active,1=cephfs.smithi157.rcaris=up:stopping} 2 up:standby 2024-04-22T23:42:23.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:23 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:42:23.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:23 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:42:23.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:23 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:42:23.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:23 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:42:23.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:23 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:42:23.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:23 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:42:23.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:23 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:42:23.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:23 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:42:23.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:23 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:42:23.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:23 smithi107 ceph-mon[90179]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2024-04-22T23:42:23.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:23 smithi107 ceph-mon[90179]: Health check update: Degraded data redundancy: 46/339 objects degraded (13.569%), 11 pgs degraded (PG_DEGRADED) 2024-04-22T23:42:24.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:24 smithi157 ceph-mon[79403]: pgmap v167: 65 pgs: 5 peering, 6 active+undersized+degraded, 9 active+undersized, 45 active+clean; 277 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 20/339 objects degraded (5.900%) 2024-04-22T23:42:24.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:24 smithi107 ceph-mon[90179]: pgmap v167: 65 pgs: 5 peering, 6 active+undersized+degraded, 9 active+undersized, 45 active+clean; 277 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 20/339 objects degraded (5.900%) 2024-04-22T23:42:26.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:26 smithi107 ceph-mon[90179]: pgmap v168: 65 pgs: 5 peering, 60 active+clean; 277 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:42:26.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:26 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:42:26.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:26 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:42:26.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:26 smithi107 ceph-mon[90179]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 20/339 objects degraded (5.900%), 6 pgs degraded) 2024-04-22T23:42:26.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:26 smithi107 ceph-mon[90179]: Cluster is now healthy 2024-04-22T23:42:27.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:26 smithi157 ceph-mon[79403]: pgmap v168: 65 pgs: 5 peering, 60 active+clean; 277 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-22T23:42:27.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:26 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:42:27.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:26 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:42:27.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:26 smithi157 ceph-mon[79403]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 20/339 objects degraded (5.900%), 6 pgs degraded) 2024-04-22T23:42:27.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:26 smithi157 ceph-mon[79403]: Cluster is now healthy 2024-04-22T23:42:28.981 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:28 smithi107 ceph-mon[90179]: pgmap v169: 65 pgs: 65 active+clean; 277 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-22T23:42:29.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:28 smithi157 ceph-mon[79403]: pgmap v169: 65 pgs: 65 active+clean; 277 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-22T23:42:30.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:30 smithi107 ceph-mon[90179]: pgmap v170: 65 pgs: 65 active+clean; 277 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s wr, 0 op/s 2024-04-22T23:42:31.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:30 smithi157 ceph-mon[79403]: pgmap v170: 65 pgs: 65 active+clean; 277 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s wr, 0 op/s 2024-04-22T23:42:32.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.050+0000 7f796a2ef700 1 -- 172.21.15.107:0/3418992348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79640fce80 msgr2=0x7f79640fd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:32.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.050+0000 7f796a2ef700 1 --2- 172.21.15.107:0/3418992348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79640fce80 0x7f79640fd290 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7954004940 tx=0x7f7954009e30 comp rx=0 tx=0).stop 2024-04-22T23:42:32.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.050+0000 7f796a2ef700 1 -- 172.21.15.107:0/3418992348 shutdown_connections 2024-04-22T23:42:32.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.050+0000 7f796a2ef700 1 --2- 172.21.15.107:0/3418992348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79640fdff0 0x7f79640fe460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:32.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.050+0000 7f796a2ef700 1 --2- 172.21.15.107:0/3418992348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79640fce80 0x7f79640fd290 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:32.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.050+0000 7f796a2ef700 1 -- 172.21.15.107:0/3418992348 >> 172.21.15.107:0/3418992348 conn(0x7f79640f8580 msgr2=0x7f79640fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:32.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.050+0000 7f796a2ef700 1 -- 172.21.15.107:0/3418992348 shutdown_connections 2024-04-22T23:42:32.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.050+0000 7f796a2ef700 1 -- 172.21.15.107:0/3418992348 wait complete. 2024-04-22T23:42:32.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.051+0000 7f796a2ef700 1 Processor -- start 2024-04-22T23:42:32.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.051+0000 7f796a2ef700 1 -- start start 2024-04-22T23:42:32.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.051+0000 7f796a2ef700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79640fce80 0x7f796410fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:32.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.051+0000 7f796a2ef700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79640fdff0 0x7f79641100b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:32.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.051+0000 7f796a2ef700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7964110740 con 0x7f79640fdff0 2024-04-22T23:42:32.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.051+0000 7f796a2ef700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79641150e0 con 0x7f79640fce80 2024-04-22T23:42:32.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.051+0000 7f79637fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79640fdff0 0x7f79641100b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:32.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.051+0000 7f79637fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79640fdff0 0x7f79641100b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49050/0 (socket says 172.21.15.107:49050) 2024-04-22T23:42:32.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.051+0000 7f79637fe700 1 -- 172.21.15.107:0/2966178777 learned_addr learned my addr 172.21.15.107:0/2966178777 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:42:32.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.051+0000 7f7963fff700 1 --2- 172.21.15.107:0/2966178777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79640fce80 0x7f796410fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:32.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.052+0000 7f79637fe700 1 -- 172.21.15.107:0/2966178777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79640fce80 msgr2=0x7f796410fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:32.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.052+0000 7f79637fe700 1 --2- 172.21.15.107:0/2966178777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79640fce80 0x7f796410fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:32.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.052+0000 7f79637fe700 1 -- 172.21.15.107:0/2966178777 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7954005040 con 0x7f79640fdff0 2024-04-22T23:42:32.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.052+0000 7f79637fe700 1 --2- 172.21.15.107:0/2966178777 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79640fdff0 0x7f79641100b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f795800b790 tx=0x7f7958009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:32.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.052+0000 7f7960ff9700 1 -- 172.21.15.107:0/2966178777 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f795800f770 con 0x7f79640fdff0 2024-04-22T23:42:32.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.052+0000 7f7960ff9700 1 -- 172.21.15.107:0/2966178777 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f795800fda0 con 0x7f79640fdff0 2024-04-22T23:42:32.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.052+0000 7f796a2ef700 1 -- 172.21.15.107:0/2966178777 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7964115340 con 0x7f79640fdff0 2024-04-22T23:42:32.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.052+0000 7f7960ff9700 1 -- 172.21.15.107:0/2966178777 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f795800d310 con 0x7f79640fdff0 2024-04-22T23:42:32.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.052+0000 7f796a2ef700 1 -- 172.21.15.107:0/2966178777 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79641158b0 con 0x7f79640fdff0 2024-04-22T23:42:32.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.053+0000 7f796a2ef700 1 -- 172.21.15.107:0/2966178777 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7964101980 con 0x7f79640fdff0 2024-04-22T23:42:32.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.054+0000 7f7960ff9700 1 -- 172.21.15.107:0/2966178777 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f7958010420 con 0x7f79640fdff0 2024-04-22T23:42:32.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.054+0000 7f7960ff9700 1 --2- 172.21.15.107:0/2966178777 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f794c06ec00 0x7f794c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:32.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.054+0000 7f7960ff9700 1 -- 172.21.15.107:0/2966178777 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f795808e8d0 con 0x7f79640fdff0 2024-04-22T23:42:32.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.054+0000 7f7963fff700 1 --2- 172.21.15.107:0/2966178777 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f794c06ec00 0x7f794c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:32.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.054+0000 7f7963fff700 1 --2- 172.21.15.107:0/2966178777 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f794c06ec00 0x7f794c0710b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f795400b3b0 tx=0x7f7954009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:32.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.059+0000 7f7960ff9700 1 -- 172.21.15.107:0/2966178777 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f795805b230 con 0x7f79640fdff0 2024-04-22T23:42:32.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.278+0000 7f796a2ef700 1 -- 172.21.15.107:0/2966178777 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f796402b340 con 0x7f794c06ec00 2024-04-22T23:42:32.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.279+0000 7f7960ff9700 1 -- 172.21.15.107:0/2966178777 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f796402b340 con 0x7f794c06ec00 2024-04-22T23:42:32.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.281+0000 7f796a2ef700 1 -- 172.21.15.107:0/2966178777 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f794c06ec00 msgr2=0x7f794c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:32.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.281+0000 7f796a2ef700 1 --2- 172.21.15.107:0/2966178777 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f794c06ec00 0x7f794c0710b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f795400b3b0 tx=0x7f7954009e30 comp rx=0 tx=0).stop 2024-04-22T23:42:32.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.282+0000 7f796a2ef700 1 -- 172.21.15.107:0/2966178777 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79640fdff0 msgr2=0x7f79641100b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:32.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.282+0000 7f796a2ef700 1 --2- 172.21.15.107:0/2966178777 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79640fdff0 0x7f79641100b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f795800b790 tx=0x7f7958009250 comp rx=0 tx=0).stop 2024-04-22T23:42:32.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.282+0000 7f796a2ef700 1 -- 172.21.15.107:0/2966178777 shutdown_connections 2024-04-22T23:42:32.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.282+0000 7f796a2ef700 1 --2- 172.21.15.107:0/2966178777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f79640fce80 0x7f796410fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:32.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.282+0000 7f796a2ef700 1 --2- 172.21.15.107:0/2966178777 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f794c06ec00 0x7f794c0710b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:32.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.282+0000 7f796a2ef700 1 --2- 172.21.15.107:0/2966178777 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79640fdff0 0x7f79641100b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:32.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.282+0000 7f796a2ef700 1 -- 172.21.15.107:0/2966178777 >> 172.21.15.107:0/2966178777 conn(0x7f79640f8580 msgr2=0x7f79640f9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:32.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.282+0000 7f796a2ef700 1 -- 172.21.15.107:0/2966178777 shutdown_connections 2024-04-22T23:42:32.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.282+0000 7f796a2ef700 1 -- 172.21.15.107:0/2966178777 wait complete. 2024-04-22T23:42:32.285 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-04-22T23:42:32.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.383+0000 7f9f57ca6700 1 -- 172.21.15.107:0/4188680811 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f500fdff0 msgr2=0x7f9f500fe460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:32.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.384+0000 7f9f57ca6700 1 --2- 172.21.15.107:0/4188680811 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f500fdff0 0x7f9f500fe460 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9f40004940 tx=0x7f9f40009e30 comp rx=0 tx=0).stop 2024-04-22T23:42:32.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.384+0000 7f9f57ca6700 1 -- 172.21.15.107:0/4188680811 shutdown_connections 2024-04-22T23:42:32.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.384+0000 7f9f57ca6700 1 --2- 172.21.15.107:0/4188680811 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f500fdff0 0x7f9f500fe460 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:32.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.384+0000 7f9f57ca6700 1 --2- 172.21.15.107:0/4188680811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f500fce80 0x7f9f500fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:32.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.384+0000 7f9f57ca6700 1 -- 172.21.15.107:0/4188680811 >> 172.21.15.107:0/4188680811 conn(0x7f9f500f8580 msgr2=0x7f9f500fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:32.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.384+0000 7f9f57ca6700 1 -- 172.21.15.107:0/4188680811 shutdown_connections 2024-04-22T23:42:32.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.384+0000 7f9f57ca6700 1 -- 172.21.15.107:0/4188680811 wait complete. 2024-04-22T23:42:32.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.384+0000 7f9f57ca6700 1 Processor -- start 2024-04-22T23:42:32.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.384+0000 7f9f57ca6700 1 -- start start 2024-04-22T23:42:32.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.384+0000 7f9f57ca6700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f500fce80 0x7f9f5010fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:32.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.384+0000 7f9f57ca6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f500fdff0 0x7f9f50110080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:32.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.384+0000 7f9f57ca6700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f50110680 con 0x7f9f500fce80 2024-04-22T23:42:32.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.384+0000 7f9f57ca6700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f501107f0 con 0x7f9f500fdff0 2024-04-22T23:42:32.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.385+0000 7f9f55a42700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f500fce80 0x7f9f5010fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:32.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.385+0000 7f9f55a42700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f500fce80 0x7f9f5010fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49072/0 (socket says 172.21.15.107:49072) 2024-04-22T23:42:32.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.385+0000 7f9f55a42700 1 -- 172.21.15.107:0/1504407241 learned_addr learned my addr 172.21.15.107:0/1504407241 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:42:32.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.385+0000 7f9f55241700 1 --2- 172.21.15.107:0/1504407241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f500fdff0 0x7f9f50110080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:32.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.385+0000 7f9f55a42700 1 -- 172.21.15.107:0/1504407241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f500fdff0 msgr2=0x7f9f50110080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:32.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.385+0000 7f9f55a42700 1 --2- 172.21.15.107:0/1504407241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f500fdff0 0x7f9f50110080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:32.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.385+0000 7f9f55a42700 1 -- 172.21.15.107:0/1504407241 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f40005040 con 0x7f9f500fce80 2024-04-22T23:42:32.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.385+0000 7f9f55a42700 1 --2- 172.21.15.107:0/1504407241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f500fce80 0x7f9f5010fb40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c002a50 tx=0x7f9f4c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:32.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.385+0000 7f9f467fc700 1 -- 172.21.15.107:0/1504407241 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f4c007cc0 con 0x7f9f500fce80 2024-04-22T23:42:32.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.385+0000 7f9f467fc700 1 -- 172.21.15.107:0/1504407241 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9f4c00f040 con 0x7f9f500fce80 2024-04-22T23:42:32.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.385+0000 7f9f57ca6700 1 -- 172.21.15.107:0/1504407241 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f50115250 con 0x7f9f500fce80 2024-04-22T23:42:32.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.385+0000 7f9f467fc700 1 -- 172.21.15.107:0/1504407241 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f4c00da30 con 0x7f9f500fce80 2024-04-22T23:42:32.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.385+0000 7f9f57ca6700 1 -- 172.21.15.107:0/1504407241 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f50115810 con 0x7f9f500fce80 2024-04-22T23:42:32.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.387+0000 7f9f57ca6700 1 -- 172.21.15.107:0/1504407241 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f50101980 con 0x7f9f500fce80 2024-04-22T23:42:32.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.387+0000 7f9f467fc700 1 -- 172.21.15.107:0/1504407241 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f9f4c00dbb0 con 0x7f9f500fce80 2024-04-22T23:42:32.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.387+0000 7f9f467fc700 1 --2- 172.21.15.107:0/1504407241 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f9f3c06ec00 0x7f9f3c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:32.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.387+0000 7f9f467fc700 1 -- 172.21.15.107:0/1504407241 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f9f4c08e620 con 0x7f9f500fce80 2024-04-22T23:42:32.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.387+0000 7f9f55241700 1 --2- 172.21.15.107:0/1504407241 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f9f3c06ec00 0x7f9f3c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:32.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.388+0000 7f9f55241700 1 --2- 172.21.15.107:0/1504407241 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f9f3c06ec00 0x7f9f3c0710b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9f4000b970 tx=0x7f9f40009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:32.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.392+0000 7f9f467fc700 1 -- 172.21.15.107:0/1504407241 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f9f4c05af00 con 0x7f9f500fce80 2024-04-22T23:42:32.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.627+0000 7f9f57ca6700 1 -- 172.21.15.107:0/1504407241 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f5002b310 con 0x7f9f3c06ec00 2024-04-22T23:42:32.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.628+0000 7f9f467fc700 1 -- 172.21.15.107:0/1504407241 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f9f5002b310 con 0x7f9f3c06ec00 2024-04-22T23:42:32.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.630+0000 7f9f57ca6700 1 -- 172.21.15.107:0/1504407241 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f9f3c06ec00 msgr2=0x7f9f3c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:32.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.630+0000 7f9f57ca6700 1 --2- 172.21.15.107:0/1504407241 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f9f3c06ec00 0x7f9f3c0710b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9f4000b970 tx=0x7f9f40009e30 comp rx=0 tx=0).stop 2024-04-22T23:42:32.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.630+0000 7f9f57ca6700 1 -- 172.21.15.107:0/1504407241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f500fce80 msgr2=0x7f9f5010fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:32.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.630+0000 7f9f57ca6700 1 --2- 172.21.15.107:0/1504407241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f500fce80 0x7f9f5010fb40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c002a50 tx=0x7f9f4c0046b0 comp rx=0 tx=0).stop 2024-04-22T23:42:32.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.630+0000 7f9f57ca6700 1 -- 172.21.15.107:0/1504407241 shutdown_connections 2024-04-22T23:42:32.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.630+0000 7f9f57ca6700 1 --2- 172.21.15.107:0/1504407241 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f9f3c06ec00 0x7f9f3c0710b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:32.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.630+0000 7f9f57ca6700 1 --2- 172.21.15.107:0/1504407241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f500fce80 0x7f9f5010fb40 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:32.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.630+0000 7f9f57ca6700 1 --2- 172.21.15.107:0/1504407241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f500fdff0 0x7f9f50110080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:32.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.630+0000 7f9f57ca6700 1 -- 172.21.15.107:0/1504407241 >> 172.21.15.107:0/1504407241 conn(0x7f9f500f8580 msgr2=0x7f9f500f9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:32.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.630+0000 7f9f57ca6700 1 -- 172.21.15.107:0/1504407241 shutdown_connections 2024-04-22T23:42:32.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.630+0000 7f9f57ca6700 1 -- 172.21.15.107:0/1504407241 wait complete. 2024-04-22T23:42:32.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.732+0000 7f0e01164700 1 -- 172.21.15.107:0/1418674955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0dfc0fedc0 msgr2=0x7f0dfc0faf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:32.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.732+0000 7f0e01164700 1 --2- 172.21.15.107:0/1418674955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0dfc0fedc0 0x7f0dfc0faf00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0dec004940 tx=0x7f0dec009e30 comp rx=0 tx=0).stop 2024-04-22T23:42:32.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.732+0000 7f0e01164700 1 -- 172.21.15.107:0/1418674955 shutdown_connections 2024-04-22T23:42:32.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.732+0000 7f0e01164700 1 --2- 172.21.15.107:0/1418674955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0dfc0fedc0 0x7f0dfc0faf00 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:32.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.732+0000 7f0e01164700 1 --2- 172.21.15.107:0/1418674955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dfc0fe470 0x7f0dfc0fe880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:32.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.732+0000 7f0e01164700 1 -- 172.21.15.107:0/1418674955 >> 172.21.15.107:0/1418674955 conn(0x7f0dfc0f6c10 msgr2=0x7f0dfc0f9040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:32.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.732+0000 7f0e01164700 1 -- 172.21.15.107:0/1418674955 shutdown_connections 2024-04-22T23:42:32.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.732+0000 7f0e01164700 1 -- 172.21.15.107:0/1418674955 wait complete. 2024-04-22T23:42:32.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.733+0000 7f0e01164700 1 Processor -- start 2024-04-22T23:42:32.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.733+0000 7f0e01164700 1 -- start start 2024-04-22T23:42:32.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.733+0000 7f0e01164700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0dfc0fe470 0x7f0dfc10d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:32.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.733+0000 7f0e01164700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dfc0fedc0 0x7f0dfc10def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:32.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.733+0000 7f0e01164700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dfc10e4f0 con 0x7f0dfc0fe470 2024-04-22T23:42:32.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.733+0000 7f0e01164700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dfc10e660 con 0x7f0dfc0fedc0 2024-04-22T23:42:32.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.733+0000 7f0dfad9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0dfc0fe470 0x7f0dfc10d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:32.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.733+0000 7f0dfad9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0dfc0fe470 0x7f0dfc10d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49094/0 (socket says 172.21.15.107:49094) 2024-04-22T23:42:32.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.733+0000 7f0dfad9d700 1 -- 172.21.15.107:0/3211881807 learned_addr learned my addr 172.21.15.107:0/3211881807 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:42:32.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.733+0000 7f0dfa59c700 1 --2- 172.21.15.107:0/3211881807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dfc0fedc0 0x7f0dfc10def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:32.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.733+0000 7f0dfad9d700 1 -- 172.21.15.107:0/3211881807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dfc0fedc0 msgr2=0x7f0dfc10def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:32.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.733+0000 7f0dfad9d700 1 --2- 172.21.15.107:0/3211881807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dfc0fedc0 0x7f0dfc10def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:32.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.733+0000 7f0dfad9d700 1 -- 172.21.15.107:0/3211881807 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dec005040 con 0x7f0dfc0fe470 2024-04-22T23:42:32.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.734+0000 7f0dfad9d700 1 --2- 172.21.15.107:0/3211881807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0dfc0fe470 0x7f0dfc10d9b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0df000c830 tx=0x7f0df0004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:32.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.734+0000 7f0de37fe700 1 -- 172.21.15.107:0/3211881807 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0df000a760 con 0x7f0dfc0fe470 2024-04-22T23:42:32.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.734+0000 7f0e01164700 1 -- 172.21.15.107:0/3211881807 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dfc1130c0 con 0x7f0dfc0fe470 2024-04-22T23:42:32.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.734+0000 7f0e01164700 1 -- 172.21.15.107:0/3211881807 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dfc113680 con 0x7f0dfc0fe470 2024-04-22T23:42:32.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.734+0000 7f0de37fe700 1 -- 172.21.15.107:0/3211881807 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0df000ad90 con 0x7f0dfc0fe470 2024-04-22T23:42:32.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.734+0000 7f0de37fe700 1 -- 172.21.15.107:0/3211881807 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0df00113a0 con 0x7f0dfc0fe470 2024-04-22T23:42:32.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.735+0000 7f0e01164700 1 -- 172.21.15.107:0/3211881807 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0de80052e0 con 0x7f0dfc0fe470 2024-04-22T23:42:32.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.736+0000 7f0de37fe700 1 -- 172.21.15.107:0/3211881807 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f0df0012400 con 0x7f0dfc0fe470 2024-04-22T23:42:32.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.736+0000 7f0de37fe700 1 --2- 172.21.15.107:0/3211881807 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f0de406ec00 0x7f0de40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:32.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.736+0000 7f0de37fe700 1 -- 172.21.15.107:0/3211881807 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f0df008e7b0 con 0x7f0dfc0fe470 2024-04-22T23:42:32.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.736+0000 7f0dfa59c700 1 --2- 172.21.15.107:0/3211881807 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f0de406ec00 0x7f0de40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:32.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.737+0000 7f0dfa59c700 1 --2- 172.21.15.107:0/3211881807 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f0de406ec00 0x7f0de40710b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0dec004f90 tx=0x7f0dec009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:32.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.743+0000 7f0de37fe700 1 -- 172.21.15.107:0/3211881807 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f0df005b1d0 con 0x7f0dfc0fe470 2024-04-22T23:42:32.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.950+0000 7f0e01164700 1 -- 172.21.15.107:0/3211881807 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0de8000bf0 con 0x7f0de406ec00 2024-04-22T23:42:32.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.960+0000 7f0de37fe700 1 -- 172.21.15.107:0/3211881807 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f0de8000bf0 con 0x7f0de406ec00 2024-04-22T23:42:32.952 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-22T23:42:32.952 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (5m) 117s ago 15m 25.6M - 0.25.0 c8568f914cd2 ec1a5c6260de 2024-04-22T23:42:32.952 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (3m) 117s ago 15m 6656k - 19.0.0-3252-g43be0201 a0434699c1d7 e074fd2cbb9c 2024-04-22T23:42:32.953 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi157 smithi157 running (3m) 16s ago 14m 6651k - 19.0.0-3252-g43be0201 a0434699c1d7 abc2649dee22 2024-04-22T23:42:32.953 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (5m) 117s ago 15m 79.9M - 9.4.7 954c08fa6188 31449ff442d2 2024-04-22T23:42:32.953 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.dfvvkw smithi107 running (10m) 117s ago 10m 13.7M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 08d3aa8d717d 2024-04-22T23:42:32.953 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.ktuxjl smithi107 running (10m) 117s ago 10m 124M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 762f27fe0f30 2024-04-22T23:42:32.953 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi157.rcaris smithi157 running (10m) 16s ago 10m 129M - 17.2.7-884-g70d0cb4b cafd8ddac8fa d23366ac085d 2024-04-22T23:42:32.953 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi157.ufywse smithi157 running (9m) 16s ago 9m 14.9M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 8b00d61eb355 2024-04-22T23:42:32.953 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.uipxvm smithi107 *:8443,9283,8765 running (5m) 117s ago 16m 502M - 19.0.0-3252-g43be0201 a0434699c1d7 d20f05dfe4f2 2024-04-22T23:42:32.953 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi157.tuylis smithi157 *:8443,9283,8765 running (5m) 16s ago 14m 451M - 19.0.0-3252-g43be0201 a0434699c1d7 957e598c9001 2024-04-22T23:42:32.953 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (4m) 117s ago 17m 49.4M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 bc72bbbe2c39 2024-04-22T23:42:32.953 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi157 smithi157 running (4m) 16s ago 14m 39.5M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 ea88de8be4fe 2024-04-22T23:42:32.953 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (5m) 117s ago 15m 16.2M - 1.5.0 0da6a335fe13 5205cfd9bd51 2024-04-22T23:42:32.953 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi157 smithi157 *:9100 running (5m) 16s ago 14m 17.2M - 1.5.0 0da6a335fe13 168ca191bb8c 2024-04-22T23:42:32.953 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (3m) 117s ago 13m 94.2M 1557M 19.0.0-3252-g43be0201 a0434699c1d7 fd0ba8ef0c27 2024-04-22T23:42:32.953 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (2m) 117s ago 13m 91.2M 1557M 19.0.0-3252-g43be0201 a0434699c1d7 77e33b6122e0 2024-04-22T23:42:32.953 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (2m) 117s ago 12m 15.0M 1557M 19.0.0-3252-g43be0201 a0434699c1d7 c307bd8cdcfc 2024-04-22T23:42:32.954 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi157 running (87s) 16s ago 12m 100M 2581M 19.0.0-3252-g43be0201 a0434699c1d7 e81ca519c831 2024-04-22T23:42:32.954 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi157 running (53s) 16s ago 12m 98.3M 2581M 19.0.0-3252-g43be0201 a0434699c1d7 d5b5a2e4781a 2024-04-22T23:42:32.954 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi157 running (19s) 16s ago 11m 11.6M 2581M 19.0.0-3252-g43be0201 a0434699c1d7 0ee423135fdb 2024-04-22T23:42:32.954 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (5m) 117s ago 15m 58.8M - 2.43.0 a07b618ecd1d 99a4ea8594da 2024-04-22T23:42:32.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.962+0000 7f0e01164700 1 -- 172.21.15.107:0/3211881807 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f0de406ec00 msgr2=0x7f0de40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:32.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.962+0000 7f0e01164700 1 --2- 172.21.15.107:0/3211881807 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f0de406ec00 0x7f0de40710b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0dec004f90 tx=0x7f0dec009e30 comp rx=0 tx=0).stop 2024-04-22T23:42:32.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.962+0000 7f0e01164700 1 -- 172.21.15.107:0/3211881807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0dfc0fe470 msgr2=0x7f0dfc10d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:32.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.962+0000 7f0e01164700 1 --2- 172.21.15.107:0/3211881807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0dfc0fe470 0x7f0dfc10d9b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0df000c830 tx=0x7f0df0004470 comp rx=0 tx=0).stop 2024-04-22T23:42:32.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.962+0000 7f0e01164700 1 -- 172.21.15.107:0/3211881807 shutdown_connections 2024-04-22T23:42:32.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.962+0000 7f0e01164700 1 --2- 172.21.15.107:0/3211881807 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f0de406ec00 0x7f0de40710b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:32.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.962+0000 7f0e01164700 1 --2- 172.21.15.107:0/3211881807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0dfc0fe470 0x7f0dfc10d9b0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:32.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.962+0000 7f0e01164700 1 --2- 172.21.15.107:0/3211881807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dfc0fedc0 0x7f0dfc10def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:32.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.962+0000 7f0e01164700 1 -- 172.21.15.107:0/3211881807 >> 172.21.15.107:0/3211881807 conn(0x7f0dfc0f6c10 msgr2=0x7f0dfc0f8350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:32.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.962+0000 7f0e01164700 1 -- 172.21.15.107:0/3211881807 shutdown_connections 2024-04-22T23:42:32.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:32.962+0000 7f0e01164700 1 -- 172.21.15.107:0/3211881807 wait complete. 2024-04-22T23:42:32.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:32 smithi107 ceph-mon[90179]: pgmap v171: 65 pgs: 65 active+clean; 277 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s wr, 0 op/s 2024-04-22T23:42:32.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:32 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:42:32.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:32 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:42:32.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:32 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:42:32.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:32 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:42:32.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:32 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:42:32.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:32 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:42:32.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:32 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:42:32.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:32 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:42:32.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:32 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:42:33.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.062+0000 7f5bfe112700 1 -- 172.21.15.107:0/2018060069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bf80fee40 msgr2=0x7f5bf80faf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:33.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.062+0000 7f5bfe112700 1 --2- 172.21.15.107:0/2018060069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bf80fee40 0x7f5bf80faf80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5bec004940 tx=0x7f5bec009e30 comp rx=0 tx=0).stop 2024-04-22T23:42:33.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.062+0000 7f5bfe112700 1 -- 172.21.15.107:0/2018060069 shutdown_connections 2024-04-22T23:42:33.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.062+0000 7f5bfe112700 1 --2- 172.21.15.107:0/2018060069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bf80fee40 0x7f5bf80faf80 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:33.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.062+0000 7f5bfe112700 1 --2- 172.21.15.107:0/2018060069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bf80fe4f0 0x7f5bf80fe900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:33.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.062+0000 7f5bfe112700 1 -- 172.21.15.107:0/2018060069 >> 172.21.15.107:0/2018060069 conn(0x7f5bf80f6bd0 msgr2=0x7f5bf80f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:33.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.063+0000 7f5bfe112700 1 -- 172.21.15.107:0/2018060069 shutdown_connections 2024-04-22T23:42:33.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.063+0000 7f5bfe112700 1 -- 172.21.15.107:0/2018060069 wait complete. 2024-04-22T23:42:33.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.063+0000 7f5bfe112700 1 Processor -- start 2024-04-22T23:42:33.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.063+0000 7f5bfe112700 1 -- start start 2024-04-22T23:42:33.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.063+0000 7f5bfe112700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bf80fe4f0 0x7f5bf810d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:33.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.063+0000 7f5bfe112700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bf80fee40 0x7f5bf810deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:33.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.063+0000 7f5bfe112700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf810e4b0 con 0x7f5bf80fee40 2024-04-22T23:42:33.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.063+0000 7f5bfe112700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf810e620 con 0x7f5bf80fe4f0 2024-04-22T23:42:33.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.063+0000 7f5bf6ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bf80fee40 0x7f5bf810deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:33.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.064+0000 7f5bf6ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bf80fee40 0x7f5bf810deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49118/0 (socket says 172.21.15.107:49118) 2024-04-22T23:42:33.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.064+0000 7f5bf6ffd700 1 -- 172.21.15.107:0/144314070 learned_addr learned my addr 172.21.15.107:0/144314070 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:42:33.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.064+0000 7f5bf77fe700 1 --2- 172.21.15.107:0/144314070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bf80fe4f0 0x7f5bf810d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:33.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.064+0000 7f5bf6ffd700 1 -- 172.21.15.107:0/144314070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bf80fe4f0 msgr2=0x7f5bf810d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:33.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.064+0000 7f5bf6ffd700 1 --2- 172.21.15.107:0/144314070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bf80fe4f0 0x7f5bf810d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:33.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.064+0000 7f5bf6ffd700 1 -- 172.21.15.107:0/144314070 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bec005040 con 0x7f5bf80fee40 2024-04-22T23:42:33.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.064+0000 7f5bf77fe700 1 --2- 172.21.15.107:0/144314070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bf80fe4f0 0x7f5bf810d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T23:42:33.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.064+0000 7f5bf6ffd700 1 --2- 172.21.15.107:0/144314070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bf80fee40 0x7f5bf810deb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5bec00b3b0 tx=0x7f5bec009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:33.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.064+0000 7f5bdffff700 1 -- 172.21.15.107:0/144314070 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5bec00e040 con 0x7f5bf80fee40 2024-04-22T23:42:33.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.064+0000 7f5bfe112700 1 -- 172.21.15.107:0/144314070 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bf8113020 con 0x7f5bf80fee40 2024-04-22T23:42:33.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.064+0000 7f5bdffff700 1 -- 172.21.15.107:0/144314070 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5bec00bcc0 con 0x7f5bf80fee40 2024-04-22T23:42:33.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.064+0000 7f5bdffff700 1 -- 172.21.15.107:0/144314070 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5bec01c630 con 0x7f5bf80fee40 2024-04-22T23:42:33.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.064+0000 7f5bfe112700 1 -- 172.21.15.107:0/144314070 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bf81135e0 con 0x7f5bf80fee40 2024-04-22T23:42:33.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.065+0000 7f5bfe112700 1 -- 172.21.15.107:0/144314070 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bf804c710 con 0x7f5bf80fee40 2024-04-22T23:42:33.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.066+0000 7f5bdffff700 1 -- 172.21.15.107:0/144314070 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f5bec013980 con 0x7f5bf80fee40 2024-04-22T23:42:33.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.066+0000 7f5bdffff700 1 --2- 172.21.15.107:0/144314070 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f5be006ec00 0x7f5be00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:33.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.066+0000 7f5bdffff700 1 -- 172.21.15.107:0/144314070 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f5bec08f4e0 con 0x7f5bf80fee40 2024-04-22T23:42:33.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.066+0000 7f5bf77fe700 1 --2- 172.21.15.107:0/144314070 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f5be006ec00 0x7f5be00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:33.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.067+0000 7f5bf77fe700 1 --2- 172.21.15.107:0/144314070 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f5be006ec00 0x7f5be00710b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5be8004550 tx=0x7f5be8009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:33.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.071+0000 7f5bdffff700 1 -- 172.21.15.107:0/144314070 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f5bec05be40 con 0x7f5bf80fee40 2024-04-22T23:42:33.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:32 smithi157 ceph-mon[79403]: pgmap v171: 65 pgs: 65 active+clean; 277 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s wr, 0 op/s 2024-04-22T23:42:33.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:32 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:42:33.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:32 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:42:33.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:32 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:42:33.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:32 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:42:33.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:32 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:42:33.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:32 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:42:33.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:32 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:42:33.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:32 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:42:33.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:32 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:42:33.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.357+0000 7f5bfe112700 1 -- 172.21.15.107:0/144314070 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5bf8113a30 con 0x7f5bf80fee40 2024-04-22T23:42:33.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.357+0000 7f5bdffff700 1 -- 172.21.15.107:0/144314070 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+733 (secure 0 0 0) 0x7f5bec05b9d0 con 0x7f5bf80fee40 2024-04-22T23:42:33.350 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-22T23:42:33.350 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-04-22T23:42:33.350 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:42:33.351 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:42:33.351 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-04-22T23:42:33.351 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:42:33.351 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:42:33.351 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-04-22T23:42:33.351 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 6 2024-04-22T23:42:33.351 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:42:33.351 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-04-22T23:42:33.351 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 4 2024-04-22T23:42:33.351 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:42:33.351 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-04-22T23:42:33.351 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 4, 2024-04-22T23:42:33.351 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 10 2024-04-22T23:42:33.351 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T23:42:33.352 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-22T23:42:33.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.360+0000 7f5bfe112700 1 -- 172.21.15.107:0/144314070 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f5be006ec00 msgr2=0x7f5be00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:33.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.360+0000 7f5bfe112700 1 --2- 172.21.15.107:0/144314070 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f5be006ec00 0x7f5be00710b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5be8004550 tx=0x7f5be8009040 comp rx=0 tx=0).stop 2024-04-22T23:42:33.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.360+0000 7f5bfe112700 1 -- 172.21.15.107:0/144314070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bf80fee40 msgr2=0x7f5bf810deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:33.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.360+0000 7f5bfe112700 1 --2- 172.21.15.107:0/144314070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bf80fee40 0x7f5bf810deb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5bec00b3b0 tx=0x7f5bec009e30 comp rx=0 tx=0).stop 2024-04-22T23:42:33.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.360+0000 7f5bfe112700 1 -- 172.21.15.107:0/144314070 shutdown_connections 2024-04-22T23:42:33.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.360+0000 7f5bfe112700 1 --2- 172.21.15.107:0/144314070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bf80fe4f0 0x7f5bf810d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:33.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.360+0000 7f5bfe112700 1 --2- 172.21.15.107:0/144314070 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f5be006ec00 0x7f5be00710b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:33.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.360+0000 7f5bfe112700 1 --2- 172.21.15.107:0/144314070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bf80fee40 0x7f5bf810deb0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:33.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.360+0000 7f5bfe112700 1 -- 172.21.15.107:0/144314070 >> 172.21.15.107:0/144314070 conn(0x7f5bf80f6bd0 msgr2=0x7f5bf80f8290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:33.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.360+0000 7f5bfe112700 1 -- 172.21.15.107:0/144314070 shutdown_connections 2024-04-22T23:42:33.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.360+0000 7f5bfe112700 1 -- 172.21.15.107:0/144314070 wait complete. 2024-04-22T23:42:33.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.459+0000 7f0dacec7700 1 -- 172.21.15.107:0/2145719420 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0da80662f0 msgr2=0x7f0da80ffa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:33.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.459+0000 7f0dacec7700 1 --2- 172.21.15.107:0/2145719420 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0da80662f0 0x7f0da80ffa30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0da0009ce0 tx=0x7f0da0004d10 comp rx=0 tx=0).stop 2024-04-22T23:42:33.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.459+0000 7f0dacec7700 1 -- 172.21.15.107:0/2145719420 shutdown_connections 2024-04-22T23:42:33.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.459+0000 7f0dacec7700 1 --2- 172.21.15.107:0/2145719420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0da80fff70 0x7f0da8102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:33.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.459+0000 7f0dacec7700 1 --2- 172.21.15.107:0/2145719420 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0da80662f0 0x7f0da80ffa30 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:33.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.459+0000 7f0dacec7700 1 -- 172.21.15.107:0/2145719420 >> 172.21.15.107:0/2145719420 conn(0x7f0da80f74c0 msgr2=0x7f0da80f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:33.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.459+0000 7f0dacec7700 1 -- 172.21.15.107:0/2145719420 shutdown_connections 2024-04-22T23:42:33.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.459+0000 7f0dacec7700 1 -- 172.21.15.107:0/2145719420 wait complete. 2024-04-22T23:42:33.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.459+0000 7f0dacec7700 1 Processor -- start 2024-04-22T23:42:33.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.459+0000 7f0dacec7700 1 -- start start 2024-04-22T23:42:33.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.460+0000 7f0dacec7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0da80662f0 0x7f0da810dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:33.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.460+0000 7f0dacec7700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0da80fff70 0x7f0da810e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:33.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.460+0000 7f0dacec7700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da810e7d0 con 0x7f0da80662f0 2024-04-22T23:42:33.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.460+0000 7f0dacec7700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da810ef70 con 0x7f0da80fff70 2024-04-22T23:42:33.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.460+0000 7f0da659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0da80662f0 0x7f0da810dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:33.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.460+0000 7f0da659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0da80662f0 0x7f0da810dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49126/0 (socket says 172.21.15.107:49126) 2024-04-22T23:42:33.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.460+0000 7f0da659c700 1 -- 172.21.15.107:0/2385350355 learned_addr learned my addr 172.21.15.107:0/2385350355 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:42:33.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.460+0000 7f0da659c700 1 -- 172.21.15.107:0/2385350355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0da80fff70 msgr2=0x7f0da810e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:33.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.460+0000 7f0da659c700 1 --2- 172.21.15.107:0/2385350355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0da80fff70 0x7f0da810e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:33.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.460+0000 7f0da659c700 1 -- 172.21.15.107:0/2385350355 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da0009940 con 0x7f0da80662f0 2024-04-22T23:42:33.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.460+0000 7f0da659c700 1 --2- 172.21.15.107:0/2385350355 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0da80662f0 0x7f0da810dc00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0da0009380 tx=0x7f0da0004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:33.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.460+0000 7f0d9effd700 1 -- 172.21.15.107:0/2385350355 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0da000e040 con 0x7f0da80662f0 2024-04-22T23:42:33.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.460+0000 7f0dacec7700 1 -- 172.21.15.107:0/2385350355 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0da810f1f0 con 0x7f0da80662f0 2024-04-22T23:42:33.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.460+0000 7f0d9effd700 1 -- 172.21.15.107:0/2385350355 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0da00128e0 con 0x7f0da80662f0 2024-04-22T23:42:33.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.460+0000 7f0d9effd700 1 -- 172.21.15.107:0/2385350355 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0da001c9f0 con 0x7f0da80662f0 2024-04-22T23:42:33.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.460+0000 7f0dacec7700 1 -- 172.21.15.107:0/2385350355 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0da810f7b0 con 0x7f0da80662f0 2024-04-22T23:42:33.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.461+0000 7f0dacec7700 1 -- 172.21.15.107:0/2385350355 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0da805f5f0 con 0x7f0da80662f0 2024-04-22T23:42:33.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.462+0000 7f0d9effd700 1 -- 172.21.15.107:0/2385350355 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f0da0012a50 con 0x7f0da80662f0 2024-04-22T23:42:33.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.462+0000 7f0d9effd700 1 --2- 172.21.15.107:0/2385350355 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f0d9406ec00 0x7f0d940710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:33.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.462+0000 7f0d9effd700 1 -- 172.21.15.107:0/2385350355 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f0da0090540 con 0x7f0da80662f0 2024-04-22T23:42:33.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.462+0000 7f0da5d9b700 1 --2- 172.21.15.107:0/2385350355 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f0d9406ec00 0x7f0d940710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:33.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.463+0000 7f0da5d9b700 1 --2- 172.21.15.107:0/2385350355 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f0d9406ec00 0x7f0d940710b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0d90006fd0 tx=0x7f0d90006db0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:33.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.467+0000 7f0d9effd700 1 -- 172.21.15.107:0/2385350355 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f0da005ce20 con 0x7f0da80662f0 2024-04-22T23:42:33.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.711+0000 7f0dacec7700 1 -- 172.21.15.107:0/2385350355 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0da810fad0 con 0x7f0da80662f0 2024-04-22T23:42:33.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.711+0000 7f0d9effd700 1 -- 172.21.15.107:0/2385350355 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+1922 (secure 0 0 0) 0x7f0da005c9b0 con 0x7f0da80662f0 2024-04-22T23:42:33.705 INFO:teuthology.orchestra.run.smithi107.stdout:e23 2024-04-22T23:42:33.705 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T23:42:33.705 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:42:33.705 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-04-22T23:42:33.706 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:42:33.706 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-04-22T23:42:33.706 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-04-22T23:42:33.706 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 23 2024-04-22T23:42:33.706 INFO:teuthology.orchestra.run.smithi107.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-22T23:42:33.706 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-04-22T23:32:20.146130+0000 2024-04-22T23:42:33.706 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-04-22T23:42:31.924457+0000 2024-04-22T23:42:33.706 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-04-22T23:42:33.706 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-04-22T23:42:33.706 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-04-22T23:42:33.706 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-04-22T23:42:33.706 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-04-22T23:42:33.706 INFO:teuthology.orchestra.run.smithi107.stdout:max_xattr_size 65536 2024-04-22T23:42:33.707 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-04-22T23:42:33.707 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-04-22T23:42:33.707 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 46 2024-04-22T23:42:33.707 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:42:33.707 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 1 2024-04-22T23:42:33.707 INFO:teuthology.orchestra.run.smithi107.stdout:in 0,1 2024-04-22T23:42:33.707 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=14468,1=24265} 2024-04-22T23:42:33.707 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-04-22T23:42:33.707 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-04-22T23:42:33.707 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-04-22T23:42:33.707 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-04-22T23:42:33.707 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-04-22T23:42:33.707 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data disabled 2024-04-22T23:42:33.707 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-04-22T23:42:33.707 INFO:teuthology.orchestra.run.smithi107.stdout:bal_rank_mask -1 2024-04-22T23:42:33.708 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-04-22T23:42:33.708 INFO:teuthology.orchestra.run.smithi107.stdout:qdb_cluster leader: 14468 members: 14468 2024-04-22T23:42:33.708 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.ktuxjl{0:14468} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.107:6828/3335313253,v1:172.21.15.107:6829/3335313253] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:42:33.708 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi157.rcaris{1:24265} state up:stopping seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.157:6824/2779251893,v1:172.21.15.157:6825/2779251893] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:42:33.708 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:42:33.708 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:42:33.708 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-04-22T23:42:33.708 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:42:33.708 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.dfvvkw{-1:14484} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/3548717247,v1:172.21.15.107:6827/3548717247] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:42:33.708 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi157.ufywse{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.157:6826/4072444199,v1:172.21.15.157:6827/4072444199] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:42:33.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.713+0000 7f0dacec7700 1 -- 172.21.15.107:0/2385350355 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f0d9406ec00 msgr2=0x7f0d940710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:33.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.713+0000 7f0dacec7700 1 --2- 172.21.15.107:0/2385350355 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f0d9406ec00 0x7f0d940710b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0d90006fd0 tx=0x7f0d90006db0 comp rx=0 tx=0).stop 2024-04-22T23:42:33.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.713+0000 7f0dacec7700 1 -- 172.21.15.107:0/2385350355 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0da80662f0 msgr2=0x7f0da810dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:33.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.713+0000 7f0dacec7700 1 --2- 172.21.15.107:0/2385350355 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0da80662f0 0x7f0da810dc00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0da0009380 tx=0x7f0da0004d10 comp rx=0 tx=0).stop 2024-04-22T23:42:33.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.714+0000 7f0dacec7700 1 -- 172.21.15.107:0/2385350355 shutdown_connections 2024-04-22T23:42:33.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.714+0000 7f0dacec7700 1 --2- 172.21.15.107:0/2385350355 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f0d9406ec00 0x7f0d940710b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:33.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.714+0000 7f0dacec7700 1 --2- 172.21.15.107:0/2385350355 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0da80662f0 0x7f0da810dc00 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:33.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.714+0000 7f0dacec7700 1 --2- 172.21.15.107:0/2385350355 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0da80fff70 0x7f0da810e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:33.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.714+0000 7f0dacec7700 1 -- 172.21.15.107:0/2385350355 >> 172.21.15.107:0/2385350355 conn(0x7f0da80f74c0 msgr2=0x7f0da80f7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:33.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.714+0000 7f0dacec7700 1 -- 172.21.15.107:0/2385350355 shutdown_connections 2024-04-22T23:42:33.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.714+0000 7f0dacec7700 1 -- 172.21.15.107:0/2385350355 wait complete. 2024-04-22T23:42:33.710 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 23 2024-04-22T23:42:33.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.818+0000 7fc52edd2700 1 -- 172.21.15.107:0/970211168 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5280fbd40 msgr2=0x7fc5280fc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:33.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.818+0000 7fc52edd2700 1 --2- 172.21.15.107:0/970211168 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5280fbd40 0x7fc5280fc150 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc51c004940 tx=0x7fc51c009e30 comp rx=0 tx=0).stop 2024-04-22T23:42:33.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.818+0000 7fc52edd2700 1 -- 172.21.15.107:0/970211168 shutdown_connections 2024-04-22T23:42:33.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.818+0000 7fc52edd2700 1 --2- 172.21.15.107:0/970211168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5280fcff0 0x7fc5280fd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:33.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.818+0000 7fc52edd2700 1 --2- 172.21.15.107:0/970211168 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5280fbd40 0x7fc5280fc150 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:33.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.818+0000 7fc52edd2700 1 -- 172.21.15.107:0/970211168 >> 172.21.15.107:0/970211168 conn(0x7fc5280f74c0 msgr2=0x7fc5280f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:33.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.818+0000 7fc52edd2700 1 -- 172.21.15.107:0/970211168 shutdown_connections 2024-04-22T23:42:33.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.818+0000 7fc52edd2700 1 -- 172.21.15.107:0/970211168 wait complete. 2024-04-22T23:42:33.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.818+0000 7fc52edd2700 1 Processor -- start 2024-04-22T23:42:33.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.818+0000 7fc52edd2700 1 -- start start 2024-04-22T23:42:33.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.819+0000 7fc52edd2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5280fbd40 0x7fc52810da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:33.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.819+0000 7fc52edd2700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5280fcff0 0x7fc52810df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:33.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.819+0000 7fc52edd2700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52810e5e0 con 0x7fc5280fbd40 2024-04-22T23:42:33.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.819+0000 7fc52edd2700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52810edd0 con 0x7fc5280fcff0 2024-04-22T23:42:33.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.819+0000 7fc52cb6e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5280fbd40 0x7fc52810da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:33.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.819+0000 7fc52cb6e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5280fbd40 0x7fc52810da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49132/0 (socket says 172.21.15.107:49132) 2024-04-22T23:42:33.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.819+0000 7fc52cb6e700 1 -- 172.21.15.107:0/1321529239 learned_addr learned my addr 172.21.15.107:0/1321529239 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:42:33.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.819+0000 7fc527fff700 1 --2- 172.21.15.107:0/1321529239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5280fcff0 0x7fc52810df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:33.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.819+0000 7fc52cb6e700 1 -- 172.21.15.107:0/1321529239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5280fcff0 msgr2=0x7fc52810df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:33.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.819+0000 7fc52cb6e700 1 --2- 172.21.15.107:0/1321529239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5280fcff0 0x7fc52810df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:33.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.819+0000 7fc52cb6e700 1 -- 172.21.15.107:0/1321529239 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc51c005040 con 0x7fc5280fbd40 2024-04-22T23:42:33.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.819+0000 7fc52cb6e700 1 --2- 172.21.15.107:0/1321529239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5280fbd40 0x7fc52810da10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc51c00b3b0 tx=0x7fc51c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:33.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.819+0000 7fc5257fa700 1 -- 172.21.15.107:0/1321529239 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc51c00e040 con 0x7fc5280fbd40 2024-04-22T23:42:33.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.820+0000 7fc5257fa700 1 -- 172.21.15.107:0/1321529239 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc51c00bcc0 con 0x7fc5280fbd40 2024-04-22T23:42:33.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.820+0000 7fc52edd2700 1 -- 172.21.15.107:0/1321529239 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc52810f050 con 0x7fc5280fbd40 2024-04-22T23:42:33.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.820+0000 7fc5257fa700 1 -- 172.21.15.107:0/1321529239 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc51c01c630 con 0x7fc5280fbd40 2024-04-22T23:42:33.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.820+0000 7fc52edd2700 1 -- 172.21.15.107:0/1321529239 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc52810f610 con 0x7fc5280fbd40 2024-04-22T23:42:33.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.821+0000 7fc52edd2700 1 -- 172.21.15.107:0/1321529239 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc528101b80 con 0x7fc5280fbd40 2024-04-22T23:42:33.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.821+0000 7fc5257fa700 1 -- 172.21.15.107:0/1321529239 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fc51c013980 con 0x7fc5280fbd40 2024-04-22T23:42:33.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.821+0000 7fc5257fa700 1 --2- 172.21.15.107:0/1321529239 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fc51806ec00 0x7fc5180710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:33.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.821+0000 7fc5257fa700 1 -- 172.21.15.107:0/1321529239 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fc51c08f5a0 con 0x7fc5280fbd40 2024-04-22T23:42:33.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.821+0000 7fc527fff700 1 --2- 172.21.15.107:0/1321529239 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fc51806ec00 0x7fc5180710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:33.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.822+0000 7fc527fff700 1 --2- 172.21.15.107:0/1321529239 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fc51806ec00 0x7fc5180710b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc514006fd0 tx=0x7fc514008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:33.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:33.826+0000 7fc5257fa700 1 -- 172.21.15.107:0/1321529239 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fc51c05be80 con 0x7fc5280fbd40 2024-04-22T23:42:33.981 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:33 smithi107 ceph-mon[90179]: from='client.34374 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:42:33.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:33 smithi107 ceph-mon[90179]: from='client.34378 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:42:33.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:33 smithi107 ceph-mon[90179]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2024-04-22T23:42:33.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:33 smithi107 ceph-mon[90179]: fsmap cephfs:2 {0=cephfs.smithi107.ktuxjl=up:active,1=cephfs.smithi157.rcaris=up:stopping} 2 up:standby 2024-04-22T23:42:33.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:33 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/144314070' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:42:33.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:33 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/2385350355' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:42:34.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.046+0000 7fc52edd2700 1 -- 172.21.15.107:0/1321529239 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc528104a00 con 0x7fc51806ec00 2024-04-22T23:42:34.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.048+0000 7fc5257fa700 1 -- 172.21.15.107:0/1321529239 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fc528104a00 con 0x7fc51806ec00 2024-04-22T23:42:34.041 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-22T23:42:34.041 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c", 2024-04-22T23:42:34.042 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-04-22T23:42:34.042 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-22T23:42:34.042 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [ 2024-04-22T23:42:34.042 INFO:teuthology.orchestra.run.smithi107.stdout: "osd", 2024-04-22T23:42:34.042 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr", 2024-04-22T23:42:34.042 INFO:teuthology.orchestra.run.smithi107.stdout: "crash", 2024-04-22T23:42:34.042 INFO:teuthology.orchestra.run.smithi107.stdout: "mon" 2024-04-22T23:42:34.042 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-22T23:42:34.042 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "12/21 daemons upgraded", 2024-04-22T23:42:34.042 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Currently upgrading osd daemons", 2024-04-22T23:42:34.042 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-04-22T23:42:34.042 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-22T23:42:34.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.050+0000 7fc52edd2700 1 -- 172.21.15.107:0/1321529239 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fc51806ec00 msgr2=0x7fc5180710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:34.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.050+0000 7fc52edd2700 1 --2- 172.21.15.107:0/1321529239 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fc51806ec00 0x7fc5180710b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc514006fd0 tx=0x7fc514008040 comp rx=0 tx=0).stop 2024-04-22T23:42:34.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.050+0000 7fc52edd2700 1 -- 172.21.15.107:0/1321529239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5280fbd40 msgr2=0x7fc52810da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:34.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.050+0000 7fc52edd2700 1 --2- 172.21.15.107:0/1321529239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5280fbd40 0x7fc52810da10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc51c00b3b0 tx=0x7fc51c009e30 comp rx=0 tx=0).stop 2024-04-22T23:42:34.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.050+0000 7fc52edd2700 1 -- 172.21.15.107:0/1321529239 shutdown_connections 2024-04-22T23:42:34.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.050+0000 7fc52edd2700 1 --2- 172.21.15.107:0/1321529239 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fc51806ec00 0x7fc5180710b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:34.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.050+0000 7fc52edd2700 1 --2- 172.21.15.107:0/1321529239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5280fbd40 0x7fc52810da10 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:34.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.050+0000 7fc52edd2700 1 --2- 172.21.15.107:0/1321529239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5280fcff0 0x7fc52810df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:34.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.050+0000 7fc52edd2700 1 -- 172.21.15.107:0/1321529239 >> 172.21.15.107:0/1321529239 conn(0x7fc5280f74c0 msgr2=0x7fc5280f8b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:34.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.050+0000 7fc52edd2700 1 -- 172.21.15.107:0/1321529239 shutdown_connections 2024-04-22T23:42:34.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.050+0000 7fc52edd2700 1 -- 172.21.15.107:0/1321529239 wait complete. 2024-04-22T23:42:34.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:33 smithi157 ceph-mon[79403]: from='client.34374 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:42:34.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:33 smithi157 ceph-mon[79403]: from='client.34378 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:42:34.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:33 smithi157 ceph-mon[79403]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2024-04-22T23:42:34.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:33 smithi157 ceph-mon[79403]: fsmap cephfs:2 {0=cephfs.smithi107.ktuxjl=up:active,1=cephfs.smithi157.rcaris=up:stopping} 2 up:standby 2024-04-22T23:42:34.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:33 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/144314070' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:42:34.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:33 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/2385350355' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:42:34.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.149+0000 7fb975a73700 1 -- 172.21.15.107:0/3901015149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9700feda0 msgr2=0x7fb9700faee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:34.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.149+0000 7fb975a73700 1 --2- 172.21.15.107:0/3901015149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9700feda0 0x7fb9700faee0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb964004940 tx=0x7fb964009e30 comp rx=0 tx=0).stop 2024-04-22T23:42:34.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.149+0000 7fb975a73700 1 -- 172.21.15.107:0/3901015149 shutdown_connections 2024-04-22T23:42:34.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.149+0000 7fb975a73700 1 --2- 172.21.15.107:0/3901015149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9700feda0 0x7fb9700faee0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:34.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.149+0000 7fb975a73700 1 --2- 172.21.15.107:0/3901015149 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9700fe450 0x7fb9700fe860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:34.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.149+0000 7fb975a73700 1 -- 172.21.15.107:0/3901015149 >> 172.21.15.107:0/3901015149 conn(0x7fb9700f6bf0 msgr2=0x7fb9700f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:34.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.149+0000 7fb975a73700 1 -- 172.21.15.107:0/3901015149 shutdown_connections 2024-04-22T23:42:34.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.150+0000 7fb975a73700 1 -- 172.21.15.107:0/3901015149 wait complete. 2024-04-22T23:42:34.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.150+0000 7fb975a73700 1 Processor -- start 2024-04-22T23:42:34.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.150+0000 7fb975a73700 1 -- start start 2024-04-22T23:42:34.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.150+0000 7fb975a73700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9700fe450 0x7fb97010d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:34.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.150+0000 7fb975a73700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9700feda0 0x7fb97010de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:34.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.150+0000 7fb975a73700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97010e460 con 0x7fb9700fe450 2024-04-22T23:42:34.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.150+0000 7fb975a73700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97010e5d0 con 0x7fb9700feda0 2024-04-22T23:42:34.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.150+0000 7fb96f7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9700fe450 0x7fb97010d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:34.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.150+0000 7fb96f7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9700fe450 0x7fb97010d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49154/0 (socket says 172.21.15.107:49154) 2024-04-22T23:42:34.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.150+0000 7fb96f7fe700 1 -- 172.21.15.107:0/1032052051 learned_addr learned my addr 172.21.15.107:0/1032052051 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:42:34.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.151+0000 7fb96effd700 1 --2- 172.21.15.107:0/1032052051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9700feda0 0x7fb97010de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:34.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.151+0000 7fb96f7fe700 1 -- 172.21.15.107:0/1032052051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9700feda0 msgr2=0x7fb97010de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:34.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.151+0000 7fb96f7fe700 1 --2- 172.21.15.107:0/1032052051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9700feda0 0x7fb97010de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:34.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.151+0000 7fb96f7fe700 1 -- 172.21.15.107:0/1032052051 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb964005040 con 0x7fb9700fe450 2024-04-22T23:42:34.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.151+0000 7fb96f7fe700 1 --2- 172.21.15.107:0/1032052051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9700fe450 0x7fb97010d920 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb960002a50 tx=0x7fb9600046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:34.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.151+0000 7fb957fff700 1 -- 172.21.15.107:0/1032052051 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb960007cc0 con 0x7fb9700fe450 2024-04-22T23:42:34.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.151+0000 7fb957fff700 1 -- 172.21.15.107:0/1032052051 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb96000f040 con 0x7fb9700fe450 2024-04-22T23:42:34.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.151+0000 7fb975a73700 1 -- 172.21.15.107:0/1032052051 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb970112fd0 con 0x7fb9700fe450 2024-04-22T23:42:34.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.151+0000 7fb957fff700 1 -- 172.21.15.107:0/1032052051 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb96000da30 con 0x7fb9700fe450 2024-04-22T23:42:34.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.151+0000 7fb975a73700 1 -- 172.21.15.107:0/1032052051 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb970113560 con 0x7fb9700fe450 2024-04-22T23:42:34.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.152+0000 7fb975a73700 1 -- 172.21.15.107:0/1032052051 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb97004c710 con 0x7fb9700fe450 2024-04-22T23:42:34.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.152+0000 7fb957fff700 1 -- 172.21.15.107:0/1032052051 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fb96000db90 con 0x7fb9700fe450 2024-04-22T23:42:34.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.153+0000 7fb957fff700 1 --2- 172.21.15.107:0/1032052051 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fb95806ec00 0x7fb9580710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:42:34.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.153+0000 7fb96effd700 1 --2- 172.21.15.107:0/1032052051 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fb95806ec00 0x7fb9580710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:42:34.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.153+0000 7fb957fff700 1 -- 172.21.15.107:0/1032052051 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fb96008e680 con 0x7fb9700fe450 2024-04-22T23:42:34.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.154+0000 7fb96effd700 1 --2- 172.21.15.107:0/1032052051 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fb95806ec00 0x7fb9580710b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb96400b970 tx=0x7fb964009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:42:34.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.158+0000 7fb957fff700 1 -- 172.21.15.107:0/1032052051 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fb96005afd0 con 0x7fb9700fe450 2024-04-22T23:42:34.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.439+0000 7fb975a73700 1 -- 172.21.15.107:0/1032052051 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb970113750 con 0x7fb9700fe450 2024-04-22T23:42:34.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.439+0000 7fb957fff700 1 -- 172.21.15.107:0/1032052051 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fb96005ab60 con 0x7fb9700fe450 2024-04-22T23:42:34.433 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_OK 2024-04-22T23:42:34.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.441+0000 7fb975a73700 1 -- 172.21.15.107:0/1032052051 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fb95806ec00 msgr2=0x7fb9580710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:34.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.441+0000 7fb975a73700 1 --2- 172.21.15.107:0/1032052051 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fb95806ec00 0x7fb9580710b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb96400b970 tx=0x7fb964009e30 comp rx=0 tx=0).stop 2024-04-22T23:42:34.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.441+0000 7fb975a73700 1 -- 172.21.15.107:0/1032052051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9700fe450 msgr2=0x7fb97010d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:42:34.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.441+0000 7fb975a73700 1 --2- 172.21.15.107:0/1032052051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9700fe450 0x7fb97010d920 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb960002a50 tx=0x7fb9600046b0 comp rx=0 tx=0).stop 2024-04-22T23:42:34.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.441+0000 7fb975a73700 1 -- 172.21.15.107:0/1032052051 shutdown_connections 2024-04-22T23:42:34.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.441+0000 7fb975a73700 1 --2- 172.21.15.107:0/1032052051 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fb95806ec00 0x7fb9580710b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:34.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.441+0000 7fb975a73700 1 --2- 172.21.15.107:0/1032052051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9700fe450 0x7fb97010d920 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:34.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.441+0000 7fb975a73700 1 --2- 172.21.15.107:0/1032052051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9700feda0 0x7fb97010de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:42:34.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.442+0000 7fb975a73700 1 -- 172.21.15.107:0/1032052051 >> 172.21.15.107:0/1032052051 conn(0x7fb9700f6bf0 msgr2=0x7fb9700f82b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:42:34.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.442+0000 7fb975a73700 1 -- 172.21.15.107:0/1032052051 shutdown_connections 2024-04-22T23:42:34.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:42:34.442+0000 7fb975a73700 1 -- 172.21.15.107:0/1032052051 wait complete. 2024-04-22T23:42:35.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:34 smithi157 ceph-mon[79403]: from='client.34382 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:42:35.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:34 smithi157 ceph-mon[79403]: pgmap v172: 65 pgs: 65 active+clean; 277 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-04-22T23:42:35.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:34 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/1032052051' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:42:35.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:34 smithi107 ceph-mon[90179]: from='client.34382 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:42:35.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:34 smithi107 ceph-mon[90179]: pgmap v172: 65 pgs: 65 active+clean; 277 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-04-22T23:42:35.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:34 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/1032052051' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:42:36.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:35 smithi157 ceph-mon[79403]: from='client.34394 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:42:36.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:35 smithi107 ceph-mon[90179]: from='client.34394 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:42:37.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:36 smithi157 ceph-mon[79403]: pgmap v173: 65 pgs: 65 active+clean; 277 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s wr, 0 op/s 2024-04-22T23:42:37.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:36 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:42:37.201 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:36 smithi107 ceph-mon[90179]: pgmap v173: 65 pgs: 65 active+clean; 277 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s wr, 0 op/s 2024-04-22T23:42:37.202 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:36 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:42:39.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:38 smithi157 ceph-mon[79403]: pgmap v174: 65 pgs: 65 active+clean; 277 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 85 B/s rd, 1.6 KiB/s wr, 0 op/s 2024-04-22T23:42:39.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:38 smithi107 ceph-mon[90179]: pgmap v174: 65 pgs: 65 active+clean; 277 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 85 B/s rd, 1.6 KiB/s wr, 0 op/s 2024-04-22T23:42:41.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:40 smithi157 ceph-mon[79403]: pgmap v175: 65 pgs: 65 active+clean; 277 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 85 B/s rd, 1.5 KiB/s wr, 1 op/s 2024-04-22T23:42:41.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:40 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:42:41.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:40 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:42:41.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:40 smithi107 ceph-mon[90179]: pgmap v175: 65 pgs: 65 active+clean; 277 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 85 B/s rd, 1.5 KiB/s wr, 1 op/s 2024-04-22T23:42:41.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:40 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:42:41.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:40 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:42:43.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:42 smithi157 ceph-mon[79403]: pgmap v176: 65 pgs: 65 active+clean; 277 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 85 B/s rd, 938 B/s wr, 0 op/s 2024-04-22T23:42:43.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:42 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:42:43.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:42 smithi107 ceph-mon[90179]: pgmap v176: 65 pgs: 65 active+clean; 277 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 85 B/s rd, 938 B/s wr, 0 op/s 2024-04-22T23:42:43.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:42 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:42:44.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:43 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:42:44.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:43 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:42:44.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:43 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:42:44.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:43 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:42:44.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:43 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:42:44.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:43 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:42:44.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:43 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:42:44.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:43 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:42:44.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:43 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:42:44.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:43 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:42:44.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:43 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:42:44.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:43 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:42:44.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:43 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:42:44.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:43 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:42:44.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:43 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:42:44.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:43 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:42:45.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:44 smithi157 ceph-mon[79403]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2024-04-22T23:42:45.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:44 smithi157 ceph-mon[79403]: pgmap v177: 65 pgs: 65 active+clean; 277 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 85 B/s rd, 938 B/s wr, 3 op/s 2024-04-22T23:42:45.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:44 smithi107 ceph-mon[90179]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2024-04-22T23:42:45.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:44 smithi107 ceph-mon[90179]: pgmap v177: 65 pgs: 65 active+clean; 277 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 85 B/s rd, 938 B/s wr, 3 op/s 2024-04-22T23:42:47.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:46 smithi157 ceph-mon[79403]: pgmap v178: 65 pgs: 65 active+clean; 277 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 85 B/s rd, 1.4 KiB/s wr, 5 op/s 2024-04-22T23:42:47.201 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:46 smithi107 ceph-mon[90179]: pgmap v178: 65 pgs: 65 active+clean; 277 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 85 B/s rd, 1.4 KiB/s wr, 5 op/s 2024-04-22T23:42:49.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:48 smithi157 ceph-mon[79403]: pgmap v179: 65 pgs: 65 active+clean; 241 MiB data, 996 MiB used, 535 GiB / 536 GiB avail; 85 B/s rd, 1.1 KiB/s wr, 6 op/s 2024-04-22T23:42:49.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:48 smithi107 ceph-mon[90179]: pgmap v179: 65 pgs: 65 active+clean; 241 MiB data, 996 MiB used, 535 GiB / 536 GiB avail; 85 B/s rd, 1.1 KiB/s wr, 6 op/s 2024-04-22T23:42:51.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:50 smithi157 ceph-mon[79403]: pgmap v180: 65 pgs: 65 active+clean; 221 MiB data, 884 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 938 B/s wr, 6 op/s 2024-04-22T23:42:51.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:50 smithi107 ceph-mon[90179]: pgmap v180: 65 pgs: 65 active+clean; 221 MiB data, 884 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 938 B/s wr, 6 op/s 2024-04-22T23:42:52.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:52 smithi157 ceph-mon[79403]: pgmap v181: 65 pgs: 65 active+clean; 173 MiB data, 732 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 767 B/s wr, 7 op/s 2024-04-22T23:42:53.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:52 smithi107 ceph-mon[90179]: pgmap v181: 65 pgs: 65 active+clean; 173 MiB data, 732 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 767 B/s wr, 7 op/s 2024-04-22T23:42:54.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:53 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:42:54.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:53 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:42:54.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:53 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:42:54.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:53 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:42:54.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:53 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:42:54.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:53 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:42:54.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:53 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:42:54.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:53 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:42:54.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:53 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:42:54.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:53 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:42:54.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:53 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:42:54.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:53 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:42:54.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:53 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:42:54.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:53 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:42:54.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:53 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:42:54.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:53 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:42:54.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:53 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:42:54.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:53 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:42:55.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:54 smithi157 ceph-mon[79403]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2024-04-22T23:42:55.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:54 smithi157 ceph-mon[79403]: pgmap v182: 65 pgs: 65 active+clean; 165 MiB data, 724 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 767 B/s wr, 7 op/s 2024-04-22T23:42:55.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:54 smithi107 ceph-mon[90179]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2024-04-22T23:42:55.234 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:54 smithi107 ceph-mon[90179]: pgmap v182: 65 pgs: 65 active+clean; 165 MiB data, 724 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 767 B/s wr, 7 op/s 2024-04-22T23:42:56.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:55 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:42:56.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:55 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:42:57.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:56 smithi157 ceph-mon[79403]: pgmap v183: 65 pgs: 65 active+clean; 165 MiB data, 724 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1023 B/s wr, 5 op/s 2024-04-22T23:42:57.202 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:56 smithi107 ceph-mon[90179]: pgmap v183: 65 pgs: 65 active+clean; 165 MiB data, 724 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1023 B/s wr, 5 op/s 2024-04-22T23:42:58.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:57 smithi157 ceph-mon[79403]: daemon mds.cephfs.smithi157.rcaris finished stopping rank 1 in filesystem cephfs (now has 1 ranks) 2024-04-22T23:42:58.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:57 smithi107 ceph-mon[90179]: daemon mds.cephfs.smithi157.rcaris finished stopping rank 1 in filesystem cephfs (now has 1 ranks) 2024-04-22T23:42:59.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:58 smithi157 ceph-mon[79403]: pgmap v184: 65 pgs: 65 active+clean; 165 MiB data, 724 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 426 B/s wr, 3 op/s 2024-04-22T23:42:59.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:58 smithi157 ceph-mon[79403]: mds.1 [v2:172.21.15.157:6824/2779251893,v1:172.21.15.157:6825/2779251893] down:stopped 2024-04-22T23:42:59.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:58 smithi157 ceph-mon[79403]: fsmap cephfs:1 {0=cephfs.smithi107.ktuxjl=up:active} 2 up:standby 2024-04-22T23:42:59.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:58 smithi107 ceph-mon[90179]: pgmap v184: 65 pgs: 65 active+clean; 165 MiB data, 724 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 426 B/s wr, 3 op/s 2024-04-22T23:42:59.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:58 smithi107 ceph-mon[90179]: mds.1 [v2:172.21.15.157:6824/2779251893,v1:172.21.15.157:6825/2779251893] down:stopped 2024-04-22T23:42:59.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:58 smithi107 ceph-mon[90179]: fsmap cephfs:1 {0=cephfs.smithi107.ktuxjl=up:active} 2 up:standby 2024-04-22T23:43:00.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:59 smithi157 ceph-mon[79403]: mds.? [v2:172.21.15.157:6824/1722945322,v1:172.21.15.157:6825/1722945322] up:boot 2024-04-22T23:43:00.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:59 smithi157 ceph-mon[79403]: fsmap cephfs:1 {0=cephfs.smithi107.ktuxjl=up:active} 3 up:standby 2024-04-22T23:43:00.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:59 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi157.rcaris"}]: dispatch 2024-04-22T23:43:00.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:42:59 smithi157 ceph-mon[79403]: pgmap v185: 65 pgs: 65 active+clean; 165 MiB data, 724 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 1 op/s 2024-04-22T23:43:00.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:59 smithi107 ceph-mon[90179]: mds.? [v2:172.21.15.157:6824/1722945322,v1:172.21.15.157:6825/1722945322] up:boot 2024-04-22T23:43:00.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:59 smithi107 ceph-mon[90179]: fsmap cephfs:1 {0=cephfs.smithi107.ktuxjl=up:active} 3 up:standby 2024-04-22T23:43:00.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:59 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi157.rcaris"}]: dispatch 2024-04-22T23:43:00.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:42:59 smithi107 ceph-mon[90179]: pgmap v185: 65 pgs: 65 active+clean; 165 MiB data, 724 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 1 op/s 2024-04-22T23:43:03.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:02 smithi157 ceph-mon[79403]: pgmap v186: 65 pgs: 65 active+clean; 165 MiB data, 724 MiB used, 536 GiB / 536 GiB avail; 341 B/s wr, 1 op/s 2024-04-22T23:43:03.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:02 smithi107 ceph-mon[90179]: pgmap v186: 65 pgs: 65 active+clean; 165 MiB data, 724 MiB used, 536 GiB / 536 GiB avail; 341 B/s wr, 1 op/s 2024-04-22T23:43:04.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:03 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:43:04.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:03 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:43:04.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:03 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:43:04.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:03 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:43:04.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:03 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:43:04.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:03 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:43:04.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:03 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:43:04.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:03 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:43:04.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:03 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:43:04.131 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:03 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi107.dfvvkw"]}]: dispatch 2024-04-22T23:43:04.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:03 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:43:04.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:03 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:43:04.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:03 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:43:04.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:03 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:43:04.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:03 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:43:04.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:03 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:43:04.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:03 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:43:04.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:03 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:43:04.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:03 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:43:04.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:03 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi107.dfvvkw"]}]: dispatch 2024-04-22T23:43:04.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.549+0000 7f82ca2fc700 1 -- 172.21.15.107:0/2120373918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82c40fff70 msgr2=0x7f82c4102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:04.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.549+0000 7f82ca2fc700 1 --2- 172.21.15.107:0/2120373918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82c40fff70 0x7f82c4102390 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f82b8004940 tx=0x7f82b8009e30 comp rx=0 tx=0).stop 2024-04-22T23:43:04.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.549+0000 7f82ca2fc700 1 -- 172.21.15.107:0/2120373918 shutdown_connections 2024-04-22T23:43:04.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.549+0000 7f82ca2fc700 1 --2- 172.21.15.107:0/2120373918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82c40fff70 0x7f82c4102390 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:04.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.549+0000 7f82ca2fc700 1 --2- 172.21.15.107:0/2120373918 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82c40662f0 0x7f82c40ffa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:04.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.549+0000 7f82ca2fc700 1 -- 172.21.15.107:0/2120373918 >> 172.21.15.107:0/2120373918 conn(0x7f82c40f74c0 msgr2=0x7f82c40f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:43:04.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.550+0000 7f82ca2fc700 1 -- 172.21.15.107:0/2120373918 shutdown_connections 2024-04-22T23:43:04.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.550+0000 7f82ca2fc700 1 -- 172.21.15.107:0/2120373918 wait complete. 2024-04-22T23:43:04.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.550+0000 7f82ca2fc700 1 Processor -- start 2024-04-22T23:43:04.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.550+0000 7f82ca2fc700 1 -- start start 2024-04-22T23:43:04.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.551+0000 7f82ca2fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82c40662f0 0x7f82c410da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:04.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.551+0000 7f82ca2fc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82c40fff70 0x7f82c410dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:04.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.551+0000 7f82ca2fc700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c410e5a0 con 0x7f82c40662f0 2024-04-22T23:43:04.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.551+0000 7f82ca2fc700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c410edf0 con 0x7f82c40fff70 2024-04-22T23:43:04.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.551+0000 7f82c3fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82c40662f0 0x7f82c410da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:04.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.551+0000 7f82c3fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82c40662f0 0x7f82c410da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43392/0 (socket says 172.21.15.107:43392) 2024-04-22T23:43:04.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.551+0000 7f82c3fff700 1 -- 172.21.15.107:0/1945320520 learned_addr learned my addr 172.21.15.107:0/1945320520 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:43:04.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.551+0000 7f82c37fe700 1 --2- 172.21.15.107:0/1945320520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82c40fff70 0x7f82c410dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:04.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.551+0000 7f82c3fff700 1 -- 172.21.15.107:0/1945320520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82c40fff70 msgr2=0x7f82c410dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:04.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.551+0000 7f82c3fff700 1 --2- 172.21.15.107:0/1945320520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82c40fff70 0x7f82c410dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:04.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.551+0000 7f82c3fff700 1 -- 172.21.15.107:0/1945320520 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b8005040 con 0x7f82c40662f0 2024-04-22T23:43:04.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.551+0000 7f82c3fff700 1 --2- 172.21.15.107:0/1945320520 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82c40662f0 0x7f82c410da60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f82b4002a50 tx=0x7f82b4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:43:04.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.551+0000 7f82c0ff9700 1 -- 172.21.15.107:0/1945320520 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82b4007cc0 con 0x7f82c40662f0 2024-04-22T23:43:04.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.552+0000 7f82ca2fc700 1 -- 172.21.15.107:0/1945320520 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82c410f0d0 con 0x7f82c40662f0 2024-04-22T23:43:04.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.552+0000 7f82c0ff9700 1 -- 172.21.15.107:0/1945320520 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f82b400f040 con 0x7f82c40662f0 2024-04-22T23:43:04.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.552+0000 7f82c0ff9700 1 -- 172.21.15.107:0/1945320520 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82b400d9c0 con 0x7f82c40662f0 2024-04-22T23:43:04.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.552+0000 7f82ca2fc700 1 -- 172.21.15.107:0/1945320520 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82c410f610 con 0x7f82c40662f0 2024-04-22T23:43:04.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.553+0000 7f82c0ff9700 1 -- 172.21.15.107:0/1945320520 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f82b400dc20 con 0x7f82c40662f0 2024-04-22T23:43:04.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.553+0000 7f82ca2fc700 1 -- 172.21.15.107:0/1945320520 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82c405f5f0 con 0x7f82c40662f0 2024-04-22T23:43:04.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.553+0000 7f82c0ff9700 1 --2- 172.21.15.107:0/1945320520 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f82ac06ec00 0x7f82ac0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:04.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.554+0000 7f82c0ff9700 1 -- 172.21.15.107:0/1945320520 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f82b4092170 con 0x7f82c40662f0 2024-04-22T23:43:04.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.554+0000 7f82c37fe700 1 --2- 172.21.15.107:0/1945320520 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f82ac06ec00 0x7f82ac0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:04.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.554+0000 7f82c37fe700 1 --2- 172.21.15.107:0/1945320520 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f82ac06ec00 0x7f82ac0710b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f82b800b970 tx=0x7f82b8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:43:04.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.559+0000 7f82c0ff9700 1 -- 172.21.15.107:0/1945320520 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f82b4093060 con 0x7f82c40662f0 2024-04-22T23:43:04.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.786+0000 7f82ca2fc700 1 -- 172.21.15.107:0/1945320520 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f82c410f9b0 con 0x7f82ac06ec00 2024-04-22T23:43:04.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.787+0000 7f82c0ff9700 1 -- 172.21.15.107:0/1945320520 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f82c410f9b0 con 0x7f82ac06ec00 2024-04-22T23:43:04.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.789+0000 7f82ca2fc700 1 -- 172.21.15.107:0/1945320520 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f82ac06ec00 msgr2=0x7f82ac0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:04.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.789+0000 7f82ca2fc700 1 --2- 172.21.15.107:0/1945320520 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f82ac06ec00 0x7f82ac0710b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f82b800b970 tx=0x7f82b8009e30 comp rx=0 tx=0).stop 2024-04-22T23:43:04.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.789+0000 7f82ca2fc700 1 -- 172.21.15.107:0/1945320520 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82c40662f0 msgr2=0x7f82c410da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:04.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.789+0000 7f82ca2fc700 1 --2- 172.21.15.107:0/1945320520 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82c40662f0 0x7f82c410da60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f82b4002a50 tx=0x7f82b4004600 comp rx=0 tx=0).stop 2024-04-22T23:43:04.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.789+0000 7f82ca2fc700 1 -- 172.21.15.107:0/1945320520 shutdown_connections 2024-04-22T23:43:04.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.789+0000 7f82ca2fc700 1 --2- 172.21.15.107:0/1945320520 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f82ac06ec00 0x7f82ac0710b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:04.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.789+0000 7f82ca2fc700 1 --2- 172.21.15.107:0/1945320520 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82c40662f0 0x7f82c410da60 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:04.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.789+0000 7f82ca2fc700 1 --2- 172.21.15.107:0/1945320520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82c40fff70 0x7f82c410dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:04.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.789+0000 7f82ca2fc700 1 -- 172.21.15.107:0/1945320520 >> 172.21.15.107:0/1945320520 conn(0x7f82c40f74c0 msgr2=0x7f82c40f7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:43:04.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.790+0000 7f82ca2fc700 1 -- 172.21.15.107:0/1945320520 shutdown_connections 2024-04-22T23:43:04.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.790+0000 7f82ca2fc700 1 -- 172.21.15.107:0/1945320520 wait complete. 2024-04-22T23:43:04.792 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-04-22T23:43:04.792 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:04 smithi107 ceph-mon[90179]: Upgrade: It appears safe to stop mds.cephfs.smithi107.dfvvkw 2024-04-22T23:43:04.793 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:04 smithi107 ceph-mon[90179]: pgmap v187: 65 pgs: 65 active+clean; 165 MiB data, 724 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-04-22T23:43:04.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.892+0000 7fc24edad700 1 -- 172.21.15.107:0/713258346 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2480fcad0 msgr2=0x7fc2480faee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:04.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.892+0000 7fc24edad700 1 --2- 172.21.15.107:0/713258346 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2480fcad0 0x7fc2480faee0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc234004940 tx=0x7fc234009e30 comp rx=0 tx=0).stop 2024-04-22T23:43:04.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.892+0000 7fc24edad700 1 -- 172.21.15.107:0/713258346 shutdown_connections 2024-04-22T23:43:04.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.892+0000 7fc24edad700 1 --2- 172.21.15.107:0/713258346 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2480fcad0 0x7fc2480faee0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:04.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.892+0000 7fc24edad700 1 --2- 172.21.15.107:0/713258346 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2480fc180 0x7fc2480fc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:04.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.892+0000 7fc24edad700 1 -- 172.21.15.107:0/713258346 >> 172.21.15.107:0/713258346 conn(0x7fc2480f6bf0 msgr2=0x7fc2480f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:43:04.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.892+0000 7fc24edad700 1 -- 172.21.15.107:0/713258346 shutdown_connections 2024-04-22T23:43:04.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.892+0000 7fc24edad700 1 -- 172.21.15.107:0/713258346 wait complete. 2024-04-22T23:43:04.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.893+0000 7fc24edad700 1 Processor -- start 2024-04-22T23:43:04.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.893+0000 7fc24edad700 1 -- start start 2024-04-22T23:43:04.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.893+0000 7fc24edad700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2480fc180 0x7fc248113ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:04.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.893+0000 7fc24edad700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2480fcad0 0x7fc248114430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:04.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.893+0000 7fc24edad700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc248114ac0 con 0x7fc2480fc180 2024-04-22T23:43:04.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.893+0000 7fc24edad700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc248119460 con 0x7fc2480fcad0 2024-04-22T23:43:04.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.893+0000 7fc24cb49700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2480fc180 0x7fc248113ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:04.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.893+0000 7fc24cb49700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2480fc180 0x7fc248113ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43406/0 (socket says 172.21.15.107:43406) 2024-04-22T23:43:04.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.893+0000 7fc24cb49700 1 -- 172.21.15.107:0/1201731597 learned_addr learned my addr 172.21.15.107:0/1201731597 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:43:04.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.894+0000 7fc247fff700 1 --2- 172.21.15.107:0/1201731597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2480fcad0 0x7fc248114430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:04.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.894+0000 7fc24cb49700 1 -- 172.21.15.107:0/1201731597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2480fcad0 msgr2=0x7fc248114430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:04.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.894+0000 7fc24cb49700 1 --2- 172.21.15.107:0/1201731597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2480fcad0 0x7fc248114430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:04.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.894+0000 7fc24cb49700 1 -- 172.21.15.107:0/1201731597 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc234005040 con 0x7fc2480fc180 2024-04-22T23:43:04.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.894+0000 7fc24cb49700 1 --2- 172.21.15.107:0/1201731597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2480fc180 0x7fc248113ef0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc23c00b790 tx=0x7fc23c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:43:04.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.894+0000 7fc2457fa700 1 -- 172.21.15.107:0/1201731597 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc23c00f770 con 0x7fc2480fc180 2024-04-22T23:43:04.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.894+0000 7fc2457fa700 1 -- 172.21.15.107:0/1201731597 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc23c00fda0 con 0x7fc2480fc180 2024-04-22T23:43:04.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.894+0000 7fc2457fa700 1 -- 172.21.15.107:0/1201731597 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc23c00d310 con 0x7fc2480fc180 2024-04-22T23:43:04.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.894+0000 7fc24edad700 1 -- 172.21.15.107:0/1201731597 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2481196c0 con 0x7fc2480fc180 2024-04-22T23:43:04.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.894+0000 7fc24edad700 1 -- 172.21.15.107:0/1201731597 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc248119c30 con 0x7fc2480fc180 2024-04-22T23:43:04.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.895+0000 7fc24edad700 1 -- 172.21.15.107:0/1201731597 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc24804c710 con 0x7fc2480fc180 2024-04-22T23:43:04.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.895+0000 7fc2457fa700 1 -- 172.21.15.107:0/1201731597 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fc23c010420 con 0x7fc2480fc180 2024-04-22T23:43:04.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.896+0000 7fc2457fa700 1 --2- 172.21.15.107:0/1201731597 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fc23806ec00 0x7fc2380710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:04.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.896+0000 7fc2457fa700 1 -- 172.21.15.107:0/1201731597 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fc23c08e7b0 con 0x7fc2480fc180 2024-04-22T23:43:04.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.896+0000 7fc247fff700 1 --2- 172.21.15.107:0/1201731597 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fc23806ec00 0x7fc2380710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:04.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.896+0000 7fc247fff700 1 --2- 172.21.15.107:0/1201731597 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fc23806ec00 0x7fc2380710b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc23400b3b0 tx=0x7fc234009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:43:04.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:04.901+0000 7fc2457fa700 1 -- 172.21.15.107:0/1201731597 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fc23c05b1d0 con 0x7fc2480fc180 2024-04-22T23:43:05.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.122+0000 7fc24edad700 1 -- 172.21.15.107:0/1201731597 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc248100190 con 0x7fc23806ec00 2024-04-22T23:43:05.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.124+0000 7fc2457fa700 1 -- 172.21.15.107:0/1201731597 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fc248100190 con 0x7fc23806ec00 2024-04-22T23:43:05.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.126+0000 7fc24edad700 1 -- 172.21.15.107:0/1201731597 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fc23806ec00 msgr2=0x7fc2380710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:05.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.126+0000 7fc24edad700 1 --2- 172.21.15.107:0/1201731597 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fc23806ec00 0x7fc2380710b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc23400b3b0 tx=0x7fc234009e30 comp rx=0 tx=0).stop 2024-04-22T23:43:05.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.126+0000 7fc24edad700 1 -- 172.21.15.107:0/1201731597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2480fc180 msgr2=0x7fc248113ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:05.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.126+0000 7fc24edad700 1 --2- 172.21.15.107:0/1201731597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2480fc180 0x7fc248113ef0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc23c00b790 tx=0x7fc23c009250 comp rx=0 tx=0).stop 2024-04-22T23:43:05.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.126+0000 7fc24edad700 1 -- 172.21.15.107:0/1201731597 shutdown_connections 2024-04-22T23:43:05.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.126+0000 7fc24edad700 1 --2- 172.21.15.107:0/1201731597 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fc23806ec00 0x7fc2380710b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:05.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.126+0000 7fc24edad700 1 --2- 172.21.15.107:0/1201731597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2480fc180 0x7fc248113ef0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:05.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.126+0000 7fc24edad700 1 --2- 172.21.15.107:0/1201731597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2480fcad0 0x7fc248114430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:05.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.126+0000 7fc24edad700 1 -- 172.21.15.107:0/1201731597 >> 172.21.15.107:0/1201731597 conn(0x7fc2480f6bf0 msgr2=0x7fc2480f8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:43:05.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.126+0000 7fc24edad700 1 -- 172.21.15.107:0/1201731597 shutdown_connections 2024-04-22T23:43:05.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.126+0000 7fc24edad700 1 -- 172.21.15.107:0/1201731597 wait complete. 2024-04-22T23:43:05.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:04 smithi157 ceph-mon[79403]: Upgrade: It appears safe to stop mds.cephfs.smithi107.dfvvkw 2024-04-22T23:43:05.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:04 smithi157 ceph-mon[79403]: pgmap v187: 65 pgs: 65 active+clean; 165 MiB data, 724 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-04-22T23:43:05.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.228+0000 7fb569d73700 1 -- 172.21.15.107:0/713883310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5640fbcf0 msgr2=0x7fb5640fc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:05.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.228+0000 7fb569d73700 1 --2- 172.21.15.107:0/713883310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5640fbcf0 0x7fb5640fc100 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb554009ce0 tx=0x7fb554004d10 comp rx=0 tx=0).stop 2024-04-22T23:43:05.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.228+0000 7fb569d73700 1 -- 172.21.15.107:0/713883310 shutdown_connections 2024-04-22T23:43:05.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.228+0000 7fb569d73700 1 --2- 172.21.15.107:0/713883310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5640fcfa0 0x7fb5640fd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:05.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.228+0000 7fb569d73700 1 --2- 172.21.15.107:0/713883310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5640fbcf0 0x7fb5640fc100 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:05.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.228+0000 7fb569d73700 1 -- 172.21.15.107:0/713883310 >> 172.21.15.107:0/713883310 conn(0x7fb5640f7450 msgr2=0x7fb5640f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:43:05.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.228+0000 7fb569d73700 1 -- 172.21.15.107:0/713883310 shutdown_connections 2024-04-22T23:43:05.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.228+0000 7fb569d73700 1 -- 172.21.15.107:0/713883310 wait complete. 2024-04-22T23:43:05.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.228+0000 7fb569d73700 1 Processor -- start 2024-04-22T23:43:05.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.229+0000 7fb569d73700 1 -- start start 2024-04-22T23:43:05.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.229+0000 7fb569d73700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5640fbcf0 0x7fb564066d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:05.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.229+0000 7fb569d73700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5640fcfa0 0x7fb564114d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:05.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.229+0000 7fb569d73700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb56410e750 con 0x7fb5640fbcf0 2024-04-22T23:43:05.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.229+0000 7fb569d73700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5641152b0 con 0x7fb5640fcfa0 2024-04-22T23:43:05.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.229+0000 7fb5637fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5640fbcf0 0x7fb564066d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:05.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.229+0000 7fb5637fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5640fbcf0 0x7fb564066d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43430/0 (socket says 172.21.15.107:43430) 2024-04-22T23:43:05.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.229+0000 7fb5637fe700 1 -- 172.21.15.107:0/2422634187 learned_addr learned my addr 172.21.15.107:0/2422634187 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:43:05.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.229+0000 7fb562ffd700 1 --2- 172.21.15.107:0/2422634187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5640fcfa0 0x7fb564114d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:05.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.229+0000 7fb5637fe700 1 -- 172.21.15.107:0/2422634187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5640fcfa0 msgr2=0x7fb564114d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:05.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.229+0000 7fb5637fe700 1 --2- 172.21.15.107:0/2422634187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5640fcfa0 0x7fb564114d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:05.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.229+0000 7fb5637fe700 1 -- 172.21.15.107:0/2422634187 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb554009940 con 0x7fb5640fbcf0 2024-04-22T23:43:05.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.229+0000 7fb5637fe700 1 --2- 172.21.15.107:0/2422634187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5640fbcf0 0x7fb564066d60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb554004b70 tx=0x7fb554004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:43:05.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.229+0000 7fb54bfff700 1 -- 172.21.15.107:0/2422634187 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb55400e040 con 0x7fb5640fbcf0 2024-04-22T23:43:05.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.229+0000 7fb54bfff700 1 -- 172.21.15.107:0/2422634187 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb554008f00 con 0x7fb5640fbcf0 2024-04-22T23:43:05.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.229+0000 7fb54bfff700 1 -- 172.21.15.107:0/2422634187 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb55401c800 con 0x7fb5640fbcf0 2024-04-22T23:43:05.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.229+0000 7fb569d73700 1 -- 172.21.15.107:0/2422634187 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb564115450 con 0x7fb5640fbcf0 2024-04-22T23:43:05.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.230+0000 7fb569d73700 1 -- 172.21.15.107:0/2422634187 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5641159c0 con 0x7fb5640fbcf0 2024-04-22T23:43:05.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.230+0000 7fb569d73700 1 -- 172.21.15.107:0/2422634187 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb56404c710 con 0x7fb5640fbcf0 2024-04-22T23:43:05.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.231+0000 7fb54bfff700 1 -- 172.21.15.107:0/2422634187 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fb554012410 con 0x7fb5640fbcf0 2024-04-22T23:43:05.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.231+0000 7fb54bfff700 1 --2- 172.21.15.107:0/2422634187 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fb54c06ec00 0x7fb54c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:05.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.231+0000 7fb54bfff700 1 -- 172.21.15.107:0/2422634187 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fb554090580 con 0x7fb5640fbcf0 2024-04-22T23:43:05.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.232+0000 7fb562ffd700 1 --2- 172.21.15.107:0/2422634187 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fb54c06ec00 0x7fb54c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:05.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.232+0000 7fb562ffd700 1 --2- 172.21.15.107:0/2422634187 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fb54c06ec00 0x7fb54c0710b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb558009fd0 tx=0x7fb558009f20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:43:05.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.236+0000 7fb54bfff700 1 -- 172.21.15.107:0/2422634187 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fb55405ce60 con 0x7fb5640fbcf0 2024-04-22T23:43:05.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.456+0000 7fb569d73700 1 -- 172.21.15.107:0/2422634187 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb564101380 con 0x7fb54c06ec00 2024-04-22T23:43:05.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.467+0000 7fb54bfff700 1 -- 172.21.15.107:0/2422634187 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fb564101380 con 0x7fb54c06ec00 2024-04-22T23:43:05.460 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-22T23:43:05.460 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (6m) 2m ago 16m 25.6M - 0.25.0 c8568f914cd2 ec1a5c6260de 2024-04-22T23:43:05.460 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (4m) 2m ago 16m 6656k - 19.0.0-3252-g43be0201 a0434699c1d7 e074fd2cbb9c 2024-04-22T23:43:05.460 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi157 smithi157 running (4m) 48s ago 15m 6651k - 19.0.0-3252-g43be0201 a0434699c1d7 abc2649dee22 2024-04-22T23:43:05.460 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (6m) 2m ago 15m 79.9M - 9.4.7 954c08fa6188 31449ff442d2 2024-04-22T23:43:05.460 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.dfvvkw smithi107 running (10m) 2m ago 10m 13.7M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 08d3aa8d717d 2024-04-22T23:43:05.460 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.ktuxjl smithi107 running (10m) 2m ago 10m 124M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 762f27fe0f30 2024-04-22T23:43:05.460 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi157.rcaris smithi157 running (10m) 48s ago 10m 129M - 17.2.7-884-g70d0cb4b cafd8ddac8fa d23366ac085d 2024-04-22T23:43:05.460 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi157.ufywse smithi157 running (10m) 48s ago 10m 14.9M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 8b00d61eb355 2024-04-22T23:43:05.460 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.uipxvm smithi107 *:8443,9283,8765 running (6m) 2m ago 17m 502M - 19.0.0-3252-g43be0201 a0434699c1d7 d20f05dfe4f2 2024-04-22T23:43:05.460 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi157.tuylis smithi157 *:8443,9283,8765 running (5m) 48s ago 15m 451M - 19.0.0-3252-g43be0201 a0434699c1d7 957e598c9001 2024-04-22T23:43:05.460 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (5m) 2m ago 17m 49.4M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 bc72bbbe2c39 2024-04-22T23:43:05.460 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi157 smithi157 running (4m) 48s ago 15m 39.5M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 ea88de8be4fe 2024-04-22T23:43:05.461 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (6m) 2m ago 16m 16.2M - 1.5.0 0da6a335fe13 5205cfd9bd51 2024-04-22T23:43:05.461 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi157 smithi157 *:9100 running (6m) 48s ago 15m 17.2M - 1.5.0 0da6a335fe13 168ca191bb8c 2024-04-22T23:43:05.461 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (3m) 2m ago 14m 94.2M 1557M 19.0.0-3252-g43be0201 a0434699c1d7 fd0ba8ef0c27 2024-04-22T23:43:05.461 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (3m) 2m ago 13m 91.2M 1557M 19.0.0-3252-g43be0201 a0434699c1d7 77e33b6122e0 2024-04-22T23:43:05.461 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (2m) 2m ago 13m 15.0M 1557M 19.0.0-3252-g43be0201 a0434699c1d7 c307bd8cdcfc 2024-04-22T23:43:05.461 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi157 running (119s) 48s ago 13m 100M 2581M 19.0.0-3252-g43be0201 a0434699c1d7 e81ca519c831 2024-04-22T23:43:05.461 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi157 running (85s) 48s ago 12m 98.3M 2581M 19.0.0-3252-g43be0201 a0434699c1d7 d5b5a2e4781a 2024-04-22T23:43:05.461 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi157 running (52s) 48s ago 12m 11.6M 2581M 19.0.0-3252-g43be0201 a0434699c1d7 0ee423135fdb 2024-04-22T23:43:05.461 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (5m) 2m ago 15m 58.8M - 2.43.0 a07b618ecd1d 99a4ea8594da 2024-04-22T23:43:05.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.469+0000 7fb569d73700 1 -- 172.21.15.107:0/2422634187 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fb54c06ec00 msgr2=0x7fb54c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:05.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.469+0000 7fb569d73700 1 --2- 172.21.15.107:0/2422634187 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fb54c06ec00 0x7fb54c0710b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb558009fd0 tx=0x7fb558009f20 comp rx=0 tx=0).stop 2024-04-22T23:43:05.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.469+0000 7fb569d73700 1 -- 172.21.15.107:0/2422634187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5640fbcf0 msgr2=0x7fb564066d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:05.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.469+0000 7fb569d73700 1 --2- 172.21.15.107:0/2422634187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5640fbcf0 0x7fb564066d60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb554004b70 tx=0x7fb554004d10 comp rx=0 tx=0).stop 2024-04-22T23:43:05.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.469+0000 7fb569d73700 1 -- 172.21.15.107:0/2422634187 shutdown_connections 2024-04-22T23:43:05.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.469+0000 7fb569d73700 1 --2- 172.21.15.107:0/2422634187 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fb54c06ec00 0x7fb54c0710b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:05.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.469+0000 7fb569d73700 1 --2- 172.21.15.107:0/2422634187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5640fbcf0 0x7fb564066d60 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:05.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.469+0000 7fb569d73700 1 --2- 172.21.15.107:0/2422634187 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5640fcfa0 0x7fb564114d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:05.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.469+0000 7fb569d73700 1 -- 172.21.15.107:0/2422634187 >> 172.21.15.107:0/2422634187 conn(0x7fb5640f7450 msgr2=0x7fb5640f8be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:43:05.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.469+0000 7fb569d73700 1 -- 172.21.15.107:0/2422634187 shutdown_connections 2024-04-22T23:43:05.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.469+0000 7fb569d73700 1 -- 172.21.15.107:0/2422634187 wait complete. 2024-04-22T23:43:05.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.572+0000 7f1527ddb700 1 -- 172.21.15.107:0/1175955910 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15200662f0 msgr2=0x7f15200ffa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:05.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.572+0000 7f1527ddb700 1 --2- 172.21.15.107:0/1175955910 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15200662f0 0x7f15200ffa30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f151c004940 tx=0x7f151c009e30 comp rx=0 tx=0).stop 2024-04-22T23:43:05.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.572+0000 7f1527ddb700 1 -- 172.21.15.107:0/1175955910 shutdown_connections 2024-04-22T23:43:05.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.572+0000 7f1527ddb700 1 --2- 172.21.15.107:0/1175955910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15200fff70 0x7f1520102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:05.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.572+0000 7f1527ddb700 1 --2- 172.21.15.107:0/1175955910 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15200662f0 0x7f15200ffa30 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:05.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.572+0000 7f1527ddb700 1 -- 172.21.15.107:0/1175955910 >> 172.21.15.107:0/1175955910 conn(0x7f15200f74c0 msgr2=0x7f15200f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:43:05.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.572+0000 7f1527ddb700 1 -- 172.21.15.107:0/1175955910 shutdown_connections 2024-04-22T23:43:05.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.572+0000 7f1527ddb700 1 -- 172.21.15.107:0/1175955910 wait complete. 2024-04-22T23:43:05.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.573+0000 7f1527ddb700 1 Processor -- start 2024-04-22T23:43:05.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.573+0000 7f1527ddb700 1 -- start start 2024-04-22T23:43:05.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.573+0000 7f1527ddb700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15200662f0 0x7f152010da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:05.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.573+0000 7f1527ddb700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15200fff70 0x7f152010dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:05.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.573+0000 7f1527ddb700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152010e5a0 con 0x7f15200662f0 2024-04-22T23:43:05.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.573+0000 7f1527ddb700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152010edf0 con 0x7f15200fff70 2024-04-22T23:43:05.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.573+0000 7f1525b77700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15200662f0 0x7f152010da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:05.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.573+0000 7f1525b77700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15200662f0 0x7f152010da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43454/0 (socket says 172.21.15.107:43454) 2024-04-22T23:43:05.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.573+0000 7f1525b77700 1 -- 172.21.15.107:0/1088985038 learned_addr learned my addr 172.21.15.107:0/1088985038 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:43:05.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.574+0000 7f1525376700 1 --2- 172.21.15.107:0/1088985038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15200fff70 0x7f152010dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:05.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.574+0000 7f1525b77700 1 -- 172.21.15.107:0/1088985038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15200fff70 msgr2=0x7f152010dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:05.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.574+0000 7f1525b77700 1 --2- 172.21.15.107:0/1088985038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15200fff70 0x7f152010dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:05.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.574+0000 7f1525b77700 1 -- 172.21.15.107:0/1088985038 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f151c005040 con 0x7f15200662f0 2024-04-22T23:43:05.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.574+0000 7f1525376700 1 --2- 172.21.15.107:0/1088985038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15200fff70 0x7f152010dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T23:43:05.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.574+0000 7f1525b77700 1 --2- 172.21.15.107:0/1088985038 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15200662f0 0x7f152010da60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f151c00b970 tx=0x7f151c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:43:05.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.574+0000 7f15167fc700 1 -- 172.21.15.107:0/1088985038 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f151c00e040 con 0x7f15200662f0 2024-04-22T23:43:05.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.574+0000 7f15167fc700 1 -- 172.21.15.107:0/1088985038 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f151c00bcc0 con 0x7f15200662f0 2024-04-22T23:43:05.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.574+0000 7f15167fc700 1 -- 172.21.15.107:0/1088985038 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f151c013660 con 0x7f15200662f0 2024-04-22T23:43:05.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.574+0000 7f1527ddb700 1 -- 172.21.15.107:0/1088985038 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f152010f070 con 0x7f15200662f0 2024-04-22T23:43:05.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.574+0000 7f1527ddb700 1 -- 172.21.15.107:0/1088985038 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f152010f560 con 0x7f15200662f0 2024-04-22T23:43:05.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.575+0000 7f1527ddb700 1 -- 172.21.15.107:0/1088985038 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f152005f5f0 con 0x7f15200662f0 2024-04-22T23:43:05.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.576+0000 7f15167fc700 1 -- 172.21.15.107:0/1088985038 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f151c01ca00 con 0x7f15200662f0 2024-04-22T23:43:05.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.576+0000 7f15167fc700 1 --2- 172.21.15.107:0/1088985038 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f150c06ec00 0x7f150c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:05.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.576+0000 7f15167fc700 1 -- 172.21.15.107:0/1088985038 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f151c08f540 con 0x7f15200662f0 2024-04-22T23:43:05.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.576+0000 7f1525376700 1 --2- 172.21.15.107:0/1088985038 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f150c06ec00 0x7f150c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:05.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.577+0000 7f1525376700 1 --2- 172.21.15.107:0/1088985038 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f150c06ec00 0x7f150c0710b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f15100041d0 tx=0x7f1510009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:43:05.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.585+0000 7f15167fc700 1 -- 172.21.15.107:0/1088985038 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f151c05bf50 con 0x7f15200662f0 2024-04-22T23:43:05.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.883+0000 7f1527ddb700 1 -- 172.21.15.107:0/1088985038 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f152010f930 con 0x7f15200662f0 2024-04-22T23:43:05.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.883+0000 7f15167fc700 1 -- 172.21.15.107:0/1088985038 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+733 (secure 0 0 0) 0x7f151c05bae0 con 0x7f15200662f0 2024-04-22T23:43:05.876 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-22T23:43:05.876 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-04-22T23:43:05.876 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:43:05.876 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:43:05.877 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-04-22T23:43:05.877 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:43:05.877 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:43:05.877 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-04-22T23:43:05.877 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 6 2024-04-22T23:43:05.877 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:43:05.877 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-04-22T23:43:05.877 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 4 2024-04-22T23:43:05.877 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:43:05.877 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-04-22T23:43:05.877 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 4, 2024-04-22T23:43:05.877 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 10 2024-04-22T23:43:05.877 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T23:43:05.877 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-22T23:43:05.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.886+0000 7f1527ddb700 1 -- 172.21.15.107:0/1088985038 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f150c06ec00 msgr2=0x7f150c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:05.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.886+0000 7f1527ddb700 1 --2- 172.21.15.107:0/1088985038 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f150c06ec00 0x7f150c0710b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f15100041d0 tx=0x7f1510009040 comp rx=0 tx=0).stop 2024-04-22T23:43:05.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.886+0000 7f1527ddb700 1 -- 172.21.15.107:0/1088985038 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15200662f0 msgr2=0x7f152010da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:05.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.886+0000 7f1527ddb700 1 --2- 172.21.15.107:0/1088985038 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15200662f0 0x7f152010da60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f151c00b970 tx=0x7f151c009e30 comp rx=0 tx=0).stop 2024-04-22T23:43:05.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.886+0000 7f1527ddb700 1 -- 172.21.15.107:0/1088985038 shutdown_connections 2024-04-22T23:43:05.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.886+0000 7f1527ddb700 1 --2- 172.21.15.107:0/1088985038 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f150c06ec00 0x7f150c0710b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:05.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.886+0000 7f1527ddb700 1 --2- 172.21.15.107:0/1088985038 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15200662f0 0x7f152010da60 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:05.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.886+0000 7f1527ddb700 1 --2- 172.21.15.107:0/1088985038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15200fff70 0x7f152010dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:05.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.886+0000 7f1527ddb700 1 -- 172.21.15.107:0/1088985038 >> 172.21.15.107:0/1088985038 conn(0x7f15200f74c0 msgr2=0x7f15200f7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:43:05.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.886+0000 7f1527ddb700 1 -- 172.21.15.107:0/1088985038 shutdown_connections 2024-04-22T23:43:05.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.886+0000 7f1527ddb700 1 -- 172.21.15.107:0/1088985038 wait complete. 2024-04-22T23:43:05.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.988+0000 7f180c5e6700 1 -- 172.21.15.107:0/3983243587 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18040fdff0 msgr2=0x7f18040fe460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:05.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.988+0000 7f180c5e6700 1 --2- 172.21.15.107:0/3983243587 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18040fdff0 0x7f18040fe460 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f17f4004940 tx=0x7f17f4009e30 comp rx=0 tx=0).stop 2024-04-22T23:43:05.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.988+0000 7f180c5e6700 1 -- 172.21.15.107:0/3983243587 shutdown_connections 2024-04-22T23:43:05.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.988+0000 7f180c5e6700 1 --2- 172.21.15.107:0/3983243587 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18040fdff0 0x7f18040fe460 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:05.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.988+0000 7f180c5e6700 1 --2- 172.21.15.107:0/3983243587 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18040fce80 0x7f18040fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:05.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.988+0000 7f180c5e6700 1 -- 172.21.15.107:0/3983243587 >> 172.21.15.107:0/3983243587 conn(0x7f18040f8580 msgr2=0x7f18040fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:43:05.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.988+0000 7f180c5e6700 1 -- 172.21.15.107:0/3983243587 shutdown_connections 2024-04-22T23:43:05.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.988+0000 7f180c5e6700 1 -- 172.21.15.107:0/3983243587 wait complete. 2024-04-22T23:43:05.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.988+0000 7f180c5e6700 1 Processor -- start 2024-04-22T23:43:05.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.988+0000 7f180c5e6700 1 -- start start 2024-04-22T23:43:05.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.988+0000 7f180c5e6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18040fce80 0x7f180410d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:05.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.988+0000 7f180c5e6700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18040fdff0 0x7f180410de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:05.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.988+0000 7f180c5e6700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180410e470 con 0x7f18040fdff0 2024-04-22T23:43:05.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.988+0000 7f180c5e6700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180410e5e0 con 0x7f18040fce80 2024-04-22T23:43:05.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.989+0000 7f1809b81700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18040fdff0 0x7f180410de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:05.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.989+0000 7f1809b81700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18040fdff0 0x7f180410de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43478/0 (socket says 172.21.15.107:43478) 2024-04-22T23:43:05.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.989+0000 7f1809b81700 1 -- 172.21.15.107:0/1437995646 learned_addr learned my addr 172.21.15.107:0/1437995646 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:43:05.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.989+0000 7f180a382700 1 --2- 172.21.15.107:0/1437995646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18040fce80 0x7f180410d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:05.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.989+0000 7f1809b81700 1 -- 172.21.15.107:0/1437995646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18040fce80 msgr2=0x7f180410d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:05.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.989+0000 7f1809b81700 1 --2- 172.21.15.107:0/1437995646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18040fce80 0x7f180410d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:05.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.989+0000 7f1809b81700 1 -- 172.21.15.107:0/1437995646 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17f4005040 con 0x7f18040fdff0 2024-04-22T23:43:05.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.989+0000 7f180a382700 1 --2- 172.21.15.107:0/1437995646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18040fce80 0x7f180410d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T23:43:05.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.989+0000 7f1809b81700 1 --2- 172.21.15.107:0/1437995646 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18040fdff0 0x7f180410de70 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f17f4009150 tx=0x7f17f4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:43:05.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.989+0000 7f17faffd700 1 -- 172.21.15.107:0/1437995646 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17f400e040 con 0x7f18040fdff0 2024-04-22T23:43:05.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.989+0000 7f17faffd700 1 -- 172.21.15.107:0/1437995646 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f17f400bcc0 con 0x7f18040fdff0 2024-04-22T23:43:05.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.989+0000 7f180c5e6700 1 -- 172.21.15.107:0/1437995646 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1804112fe0 con 0x7f18040fdff0 2024-04-22T23:43:05.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.989+0000 7f17faffd700 1 -- 172.21.15.107:0/1437995646 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17f401c630 con 0x7f18040fdff0 2024-04-22T23:43:05.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.989+0000 7f180c5e6700 1 -- 172.21.15.107:0/1437995646 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1804113570 con 0x7f18040fdff0 2024-04-22T23:43:05.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.990+0000 7f180c5e6700 1 -- 172.21.15.107:0/1437995646 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f180404c710 con 0x7f18040fdff0 2024-04-22T23:43:05.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.991+0000 7f17faffd700 1 -- 172.21.15.107:0/1437995646 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f17f4013420 con 0x7f18040fdff0 2024-04-22T23:43:05.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.991+0000 7f17faffd700 1 --2- 172.21.15.107:0/1437995646 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f17f006ec00 0x7f17f00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:05.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.991+0000 7f17faffd700 1 -- 172.21.15.107:0/1437995646 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f17f408f400 con 0x7f18040fdff0 2024-04-22T23:43:05.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.991+0000 7f180a382700 1 --2- 172.21.15.107:0/1437995646 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f17f006ec00 0x7f17f00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:05.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.992+0000 7f180a382700 1 --2- 172.21.15.107:0/1437995646 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f17f006ec00 0x7f17f00710b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1800007fd0 tx=0x7f1800009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:43:05.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:05.996+0000 7f17faffd700 1 -- 172.21.15.107:0/1437995646 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f17f405be20 con 0x7f18040fdff0 2024-04-22T23:43:06.023 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:05 smithi107 ceph-mon[90179]: from='client.34406 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:43:06.023 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:05 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:43:06.023 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:05 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi107.dfvvkw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T23:43:06.023 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:05 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:43:06.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:05 smithi157 ceph-mon[79403]: from='client.34406 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:43:06.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:05 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:43:06.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:05 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi107.dfvvkw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T23:43:06.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:05 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:43:06.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.261+0000 7f180c5e6700 1 -- 172.21.15.107:0/1437995646 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f18041139c0 con 0x7f18040fdff0 2024-04-22T23:43:06.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.261+0000 7f17faffd700 1 -- 172.21.15.107:0/1437995646 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) v1 ==== 76+0+1896 (secure 0 0 0) 0x7f17f405b9b0 con 0x7f18040fdff0 2024-04-22T23:43:06.254 INFO:teuthology.orchestra.run.smithi107.stdout:e25 2024-04-22T23:43:06.254 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T23:43:06.254 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:43:06.254 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-04-22T23:43:06.254 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:43:06.254 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-04-22T23:43:06.254 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-04-22T23:43:06.254 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 24 2024-04-22T23:43:06.254 INFO:teuthology.orchestra.run.smithi107.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-22T23:43:06.254 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-04-22T23:32:20.146130+0000 2024-04-22T23:43:06.255 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-04-22T23:42:56.925708+0000 2024-04-22T23:43:06.255 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-04-22T23:43:06.255 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-04-22T23:43:06.255 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-04-22T23:43:06.255 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-04-22T23:43:06.255 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-04-22T23:43:06.255 INFO:teuthology.orchestra.run.smithi107.stdout:max_xattr_size 65536 2024-04-22T23:43:06.255 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-04-22T23:43:06.255 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-04-22T23:43:06.255 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 46 2024-04-22T23:43:06.255 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:43:06.255 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 1 2024-04-22T23:43:06.255 INFO:teuthology.orchestra.run.smithi107.stdout:in 0 2024-04-22T23:43:06.255 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=14468} 2024-04-22T23:43:06.256 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-04-22T23:43:06.256 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-04-22T23:43:06.256 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 1 2024-04-22T23:43:06.256 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-04-22T23:43:06.256 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-04-22T23:43:06.256 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data disabled 2024-04-22T23:43:06.256 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-04-22T23:43:06.256 INFO:teuthology.orchestra.run.smithi107.stdout:bal_rank_mask -1 2024-04-22T23:43:06.256 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-04-22T23:43:06.256 INFO:teuthology.orchestra.run.smithi107.stdout:qdb_cluster leader: 14468 members: 14468 2024-04-22T23:43:06.256 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.ktuxjl{0:14468} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.107:6828/3335313253,v1:172.21.15.107:6829/3335313253] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:43:06.256 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:43:06.256 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:43:06.256 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-04-22T23:43:06.257 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:43:06.257 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.dfvvkw{-1:14484} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/3548717247,v1:172.21.15.107:6827/3548717247] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:43:06.257 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi157.ufywse{-1:24277} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.157:6826/4072444199,v1:172.21.15.157:6827/4072444199] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:43:06.257 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi157.rcaris{-1:44279} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.157:6824/1722945322,v1:172.21.15.157:6825/1722945322] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:43:06.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.264+0000 7f180c5e6700 1 -- 172.21.15.107:0/1437995646 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f17f006ec00 msgr2=0x7f17f00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:06.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.264+0000 7f180c5e6700 1 --2- 172.21.15.107:0/1437995646 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f17f006ec00 0x7f17f00710b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1800007fd0 tx=0x7f1800009040 comp rx=0 tx=0).stop 2024-04-22T23:43:06.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.264+0000 7f180c5e6700 1 -- 172.21.15.107:0/1437995646 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18040fdff0 msgr2=0x7f180410de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:06.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.264+0000 7f180c5e6700 1 --2- 172.21.15.107:0/1437995646 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18040fdff0 0x7f180410de70 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f17f4009150 tx=0x7f17f4009e30 comp rx=0 tx=0).stop 2024-04-22T23:43:06.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.264+0000 7f180c5e6700 1 -- 172.21.15.107:0/1437995646 shutdown_connections 2024-04-22T23:43:06.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.264+0000 7f180c5e6700 1 --2- 172.21.15.107:0/1437995646 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f18040fce80 0x7f180410d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:06.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.264+0000 7f180c5e6700 1 --2- 172.21.15.107:0/1437995646 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f17f006ec00 0x7f17f00710b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:06.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.264+0000 7f180c5e6700 1 --2- 172.21.15.107:0/1437995646 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18040fdff0 0x7f180410de70 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:06.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.264+0000 7f180c5e6700 1 -- 172.21.15.107:0/1437995646 >> 172.21.15.107:0/1437995646 conn(0x7f18040f8580 msgr2=0x7f18040f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:43:06.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.264+0000 7f180c5e6700 1 -- 172.21.15.107:0/1437995646 shutdown_connections 2024-04-22T23:43:06.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.264+0000 7f180c5e6700 1 -- 172.21.15.107:0/1437995646 wait complete. 2024-04-22T23:43:06.259 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 25 2024-04-22T23:43:06.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.366+0000 7fc9ac159700 1 -- 172.21.15.107:0/602113367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9a40fe050 msgr2=0x7fc9a40fe4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:06.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.366+0000 7fc9ac159700 1 --2- 172.21.15.107:0/602113367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9a40fe050 0x7fc9a40fe4c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc994004940 tx=0x7fc994009e30 comp rx=0 tx=0).stop 2024-04-22T23:43:06.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.366+0000 7fc9ac159700 1 -- 172.21.15.107:0/602113367 shutdown_connections 2024-04-22T23:43:06.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.366+0000 7fc9ac159700 1 --2- 172.21.15.107:0/602113367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9a40fe050 0x7fc9a40fe4c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:06.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.366+0000 7fc9ac159700 1 --2- 172.21.15.107:0/602113367 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9a40fce50 0x7fc9a40fd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:06.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.366+0000 7fc9ac159700 1 -- 172.21.15.107:0/602113367 >> 172.21.15.107:0/602113367 conn(0x7fc9a40f8580 msgr2=0x7fc9a40fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:43:06.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.366+0000 7fc9ac159700 1 -- 172.21.15.107:0/602113367 shutdown_connections 2024-04-22T23:43:06.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.366+0000 7fc9ac159700 1 -- 172.21.15.107:0/602113367 wait complete. 2024-04-22T23:43:06.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.367+0000 7fc9ac159700 1 Processor -- start 2024-04-22T23:43:06.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.367+0000 7fc9ac159700 1 -- start start 2024-04-22T23:43:06.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.367+0000 7fc9ac159700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9a40fce50 0x7fc9a4071e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:06.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.367+0000 7fc9ac159700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9a40fe050 0x7fc9a4070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:06.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.367+0000 7fc9ac159700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a40708c0 con 0x7fc9a40fe050 2024-04-22T23:43:06.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.367+0000 7fc9ac159700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a4070a30 con 0x7fc9a40fce50 2024-04-22T23:43:06.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.367+0000 7fc9a96f4700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9a40fe050 0x7fc9a4070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:06.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.367+0000 7fc9a96f4700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9a40fe050 0x7fc9a4070310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43500/0 (socket says 172.21.15.107:43500) 2024-04-22T23:43:06.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.367+0000 7fc9a96f4700 1 -- 172.21.15.107:0/4264600460 learned_addr learned my addr 172.21.15.107:0/4264600460 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:43:06.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.367+0000 7fc9a9ef5700 1 --2- 172.21.15.107:0/4264600460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9a40fce50 0x7fc9a4071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:06.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.367+0000 7fc9a96f4700 1 -- 172.21.15.107:0/4264600460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9a40fce50 msgr2=0x7fc9a4071e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:06.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.367+0000 7fc9a96f4700 1 --2- 172.21.15.107:0/4264600460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9a40fce50 0x7fc9a4071e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:06.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.367+0000 7fc9a96f4700 1 -- 172.21.15.107:0/4264600460 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc994005040 con 0x7fc9a40fe050 2024-04-22T23:43:06.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.368+0000 7fc9a96f4700 1 --2- 172.21.15.107:0/4264600460 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9a40fe050 0x7fc9a4070310 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc99400b3b0 tx=0x7fc994009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:43:06.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.368+0000 7fc99a7fc700 1 -- 172.21.15.107:0/4264600460 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc99400e040 con 0x7fc9a40fe050 2024-04-22T23:43:06.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.368+0000 7fc99a7fc700 1 -- 172.21.15.107:0/4264600460 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc99400bbf0 con 0x7fc9a40fe050 2024-04-22T23:43:06.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.368+0000 7fc9ac159700 1 -- 172.21.15.107:0/4264600460 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9a4070cb0 con 0x7fc9a40fe050 2024-04-22T23:43:06.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.368+0000 7fc99a7fc700 1 -- 172.21.15.107:0/4264600460 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc99401c630 con 0x7fc9a40fe050 2024-04-22T23:43:06.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.368+0000 7fc9ac159700 1 -- 172.21.15.107:0/4264600460 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9a4115a90 con 0x7fc9a40fe050 2024-04-22T23:43:06.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.369+0000 7fc9ac159700 1 -- 172.21.15.107:0/4264600460 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9a405f5f0 con 0x7fc9a40fe050 2024-04-22T23:43:06.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.369+0000 7fc99a7fc700 1 -- 172.21.15.107:0/4264600460 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fc99400bd60 con 0x7fc9a40fe050 2024-04-22T23:43:06.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.370+0000 7fc99a7fc700 1 --2- 172.21.15.107:0/4264600460 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fc99006ec00 0x7fc9900710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:06.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.370+0000 7fc99a7fc700 1 -- 172.21.15.107:0/4264600460 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6496+0+0 (secure 0 0 0) 0x7fc99408f450 con 0x7fc9a40fe050 2024-04-22T23:43:06.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.370+0000 7fc9a9ef5700 1 --2- 172.21.15.107:0/4264600460 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fc99006ec00 0x7fc9900710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:06.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.370+0000 7fc9a9ef5700 1 --2- 172.21.15.107:0/4264600460 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fc99006ec00 0x7fc9900710b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4071bd0 tx=0x7fc9a0009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:43:06.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.374+0000 7fc99a7fc700 1 -- 172.21.15.107:0/4264600460 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fc994058280 con 0x7fc9a40fe050 2024-04-22T23:43:06.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.599+0000 7fc9ac159700 1 -- 172.21.15.107:0/4264600460 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc9a4102360 con 0x7fc99006ec00 2024-04-22T23:43:06.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.600+0000 7fc99a7fc700 1 -- 172.21.15.107:0/4264600460 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fc9a4102360 con 0x7fc99006ec00 2024-04-22T23:43:06.593 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-22T23:43:06.593 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c", 2024-04-22T23:43:06.593 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-04-22T23:43:06.593 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-22T23:43:06.593 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [ 2024-04-22T23:43:06.593 INFO:teuthology.orchestra.run.smithi107.stdout: "osd", 2024-04-22T23:43:06.593 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr", 2024-04-22T23:43:06.593 INFO:teuthology.orchestra.run.smithi107.stdout: "crash", 2024-04-22T23:43:06.593 INFO:teuthology.orchestra.run.smithi107.stdout: "mon" 2024-04-22T23:43:06.593 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-22T23:43:06.593 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "12/21 daemons upgraded", 2024-04-22T23:43:06.593 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Currently upgrading mds daemons", 2024-04-22T23:43:06.593 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-04-22T23:43:06.594 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-22T23:43:06.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.602+0000 7fc9ac159700 1 -- 172.21.15.107:0/4264600460 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fc99006ec00 msgr2=0x7fc9900710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:06.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.602+0000 7fc9ac159700 1 --2- 172.21.15.107:0/4264600460 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fc99006ec00 0x7fc9900710b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4071bd0 tx=0x7fc9a0009040 comp rx=0 tx=0).stop 2024-04-22T23:43:06.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.602+0000 7fc9ac159700 1 -- 172.21.15.107:0/4264600460 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9a40fe050 msgr2=0x7fc9a4070310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:06.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.602+0000 7fc9ac159700 1 --2- 172.21.15.107:0/4264600460 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9a40fe050 0x7fc9a4070310 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc99400b3b0 tx=0x7fc994009e30 comp rx=0 tx=0).stop 2024-04-22T23:43:06.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.602+0000 7fc9ac159700 1 -- 172.21.15.107:0/4264600460 shutdown_connections 2024-04-22T23:43:06.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.602+0000 7fc9ac159700 1 --2- 172.21.15.107:0/4264600460 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc9a40fce50 0x7fc9a4071e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:06.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.602+0000 7fc9ac159700 1 --2- 172.21.15.107:0/4264600460 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fc99006ec00 0x7fc9900710b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:06.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.602+0000 7fc9ac159700 1 --2- 172.21.15.107:0/4264600460 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9a40fe050 0x7fc9a4070310 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:06.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.602+0000 7fc9ac159700 1 -- 172.21.15.107:0/4264600460 >> 172.21.15.107:0/4264600460 conn(0x7fc9a40f8580 msgr2=0x7fc9a40f9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:43:06.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.602+0000 7fc9ac159700 1 -- 172.21.15.107:0/4264600460 shutdown_connections 2024-04-22T23:43:06.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.602+0000 7fc9ac159700 1 -- 172.21.15.107:0/4264600460 wait complete. 2024-04-22T23:43:06.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.703+0000 7f94de29e700 1 -- 172.21.15.107:0/2854090946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94d80fbd00 msgr2=0x7f94d80fc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:06.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.703+0000 7f94de29e700 1 --2- 172.21.15.107:0/2854090946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94d80fbd00 0x7f94d80fc110 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f94c8004940 tx=0x7f94c8009e30 comp rx=0 tx=0).stop 2024-04-22T23:43:06.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.703+0000 7f94de29e700 1 -- 172.21.15.107:0/2854090946 shutdown_connections 2024-04-22T23:43:06.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.703+0000 7f94de29e700 1 --2- 172.21.15.107:0/2854090946 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94d80fcfb0 0x7f94d80fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:06.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.703+0000 7f94de29e700 1 --2- 172.21.15.107:0/2854090946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94d80fbd00 0x7f94d80fc110 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:06.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.703+0000 7f94de29e700 1 -- 172.21.15.107:0/2854090946 >> 172.21.15.107:0/2854090946 conn(0x7f94d80f74a0 msgr2=0x7f94d80f98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:43:06.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.703+0000 7f94de29e700 1 -- 172.21.15.107:0/2854090946 shutdown_connections 2024-04-22T23:43:06.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.703+0000 7f94de29e700 1 -- 172.21.15.107:0/2854090946 wait complete. 2024-04-22T23:43:06.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.704+0000 7f94de29e700 1 Processor -- start 2024-04-22T23:43:06.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.704+0000 7f94de29e700 1 -- start start 2024-04-22T23:43:06.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.704+0000 7f94de29e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94d80fbd00 0x7f94d810c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:06.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.704+0000 7f94de29e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94d80fcfb0 0x7f94d810cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:06.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.704+0000 7f94de29e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d810d360 con 0x7f94d80fbd00 2024-04-22T23:43:06.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.704+0000 7f94de29e700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d810d4d0 con 0x7f94d80fcfb0 2024-04-22T23:43:06.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.704+0000 7f94d7fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94d80fbd00 0x7f94d810c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:06.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.704+0000 7f94d7fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94d80fbd00 0x7f94d810c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43510/0 (socket says 172.21.15.107:43510) 2024-04-22T23:43:06.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.704+0000 7f94d7fff700 1 -- 172.21.15.107:0/2679592888 learned_addr learned my addr 172.21.15.107:0/2679592888 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:43:06.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.704+0000 7f94d77fe700 1 --2- 172.21.15.107:0/2679592888 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94d80fcfb0 0x7f94d810cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:06.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.704+0000 7f94d7fff700 1 -- 172.21.15.107:0/2679592888 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94d80fcfb0 msgr2=0x7f94d810cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:06.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.704+0000 7f94d7fff700 1 --2- 172.21.15.107:0/2679592888 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94d80fcfb0 0x7f94d810cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:06.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.704+0000 7f94d7fff700 1 -- 172.21.15.107:0/2679592888 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c8005040 con 0x7f94d80fbd00 2024-04-22T23:43:06.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.704+0000 7f94d77fe700 1 --2- 172.21.15.107:0/2679592888 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94d80fcfb0 0x7f94d810cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T23:43:06.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.705+0000 7f94d7fff700 1 --2- 172.21.15.107:0/2679592888 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94d80fbd00 0x7f94d810c820 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f94c800b3b0 tx=0x7f94c8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:43:06.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.705+0000 7f94d4ff9700 1 -- 172.21.15.107:0/2679592888 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94c800e040 con 0x7f94d80fbd00 2024-04-22T23:43:06.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.705+0000 7f94d4ff9700 1 -- 172.21.15.107:0/2679592888 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f94c800bcc0 con 0x7f94d80fbd00 2024-04-22T23:43:06.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.705+0000 7f94de29e700 1 -- 172.21.15.107:0/2679592888 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94d8111ed0 con 0x7f94d80fbd00 2024-04-22T23:43:06.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.705+0000 7f94d4ff9700 1 -- 172.21.15.107:0/2679592888 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94c801c630 con 0x7f94d80fbd00 2024-04-22T23:43:06.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.705+0000 7f94de29e700 1 -- 172.21.15.107:0/2679592888 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94d8112400 con 0x7f94d80fbd00 2024-04-22T23:43:06.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.706+0000 7f94de29e700 1 -- 172.21.15.107:0/2679592888 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94d804c710 con 0x7f94d80fbd00 2024-04-22T23:43:06.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.706+0000 7f94d4ff9700 1 -- 172.21.15.107:0/2679592888 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f94c8013980 con 0x7f94d80fbd00 2024-04-22T23:43:06.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.707+0000 7f94d4ff9700 1 --2- 172.21.15.107:0/2679592888 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f94c006ec00 0x7f94c00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:06.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.707+0000 7f94d4ff9700 1 -- 172.21.15.107:0/2679592888 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6496+0+0 (secure 0 0 0) 0x7f94c808f4e0 con 0x7f94d80fbd00 2024-04-22T23:43:06.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.707+0000 7f94d77fe700 1 --2- 172.21.15.107:0/2679592888 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f94c006ec00 0x7f94c00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:06.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.707+0000 7f94d77fe700 1 --2- 172.21.15.107:0/2679592888 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f94c006ec00 0x7f94c00710b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f94cc009fd0 tx=0x7f94cc009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:43:06.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:06.712+0000 7f94d4ff9700 1 -- 172.21.15.107:0/2679592888 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f94c805be40 con 0x7f94d80fbd00 2024-04-22T23:43:06.981 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:06 smithi107 ceph-mon[90179]: from='client.34410 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:43:06.981 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:06 smithi107 ceph-mon[90179]: Upgrade: Updating mds.cephfs.smithi107.dfvvkw 2024-04-22T23:43:06.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:06 smithi107 ceph-mon[90179]: Deploying daemon mds.cephfs.smithi107.dfvvkw on smithi107 2024-04-22T23:43:06.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:06 smithi107 ceph-mon[90179]: from='client.34414 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:43:06.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:06 smithi107 ceph-mon[90179]: pgmap v188: 65 pgs: 65 active+clean; 165 MiB data, 724 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-04-22T23:43:06.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:06 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/1088985038' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:43:06.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:06 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/1437995646' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:43:07.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:07.008+0000 7f94de29e700 1 -- 172.21.15.107:0/2679592888 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f94d81047f0 con 0x7f94d80fbd00 2024-04-22T23:43:07.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:07.009+0000 7f94d4ff9700 1 -- 172.21.15.107:0/2679592888 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f94c805b9d0 con 0x7f94d80fbd00 2024-04-22T23:43:07.002 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_OK 2024-04-22T23:43:07.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:07.011+0000 7f94de29e700 1 -- 172.21.15.107:0/2679592888 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f94c006ec00 msgr2=0x7f94c00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:07.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:07.011+0000 7f94de29e700 1 --2- 172.21.15.107:0/2679592888 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f94c006ec00 0x7f94c00710b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f94cc009fd0 tx=0x7f94cc009ee0 comp rx=0 tx=0).stop 2024-04-22T23:43:07.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:07.011+0000 7f94de29e700 1 -- 172.21.15.107:0/2679592888 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94d80fbd00 msgr2=0x7f94d810c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:07.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:07.011+0000 7f94de29e700 1 --2- 172.21.15.107:0/2679592888 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94d80fbd00 0x7f94d810c820 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f94c800b3b0 tx=0x7f94c8009e30 comp rx=0 tx=0).stop 2024-04-22T23:43:07.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:07.011+0000 7f94de29e700 1 -- 172.21.15.107:0/2679592888 shutdown_connections 2024-04-22T23:43:07.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:07.011+0000 7f94de29e700 1 --2- 172.21.15.107:0/2679592888 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f94c006ec00 0x7f94c00710b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:07.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:07.011+0000 7f94de29e700 1 --2- 172.21.15.107:0/2679592888 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94d80fbd00 0x7f94d810c820 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:07.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:07.011+0000 7f94de29e700 1 --2- 172.21.15.107:0/2679592888 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94d80fcfb0 0x7f94d810cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:07.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:07.011+0000 7f94de29e700 1 -- 172.21.15.107:0/2679592888 >> 172.21.15.107:0/2679592888 conn(0x7f94d80f74a0 msgr2=0x7f94d80f8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:43:07.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:07.011+0000 7f94de29e700 1 -- 172.21.15.107:0/2679592888 shutdown_connections 2024-04-22T23:43:07.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:07.011+0000 7f94de29e700 1 -- 172.21.15.107:0/2679592888 wait complete. 2024-04-22T23:43:07.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:06 smithi157 ceph-mon[79403]: from='client.34410 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:43:07.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:06 smithi157 ceph-mon[79403]: Upgrade: Updating mds.cephfs.smithi107.dfvvkw 2024-04-22T23:43:07.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:06 smithi157 ceph-mon[79403]: Deploying daemon mds.cephfs.smithi107.dfvvkw on smithi107 2024-04-22T23:43:07.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:06 smithi157 ceph-mon[79403]: from='client.34414 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:43:07.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:06 smithi157 ceph-mon[79403]: pgmap v188: 65 pgs: 65 active+clean; 165 MiB data, 724 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-04-22T23:43:07.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:06 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/1088985038' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:43:07.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:06 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/1437995646' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:43:08.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:07 smithi157 ceph-mon[79403]: from='client.34426 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:43:08.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:07 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/2679592888' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:43:08.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:07 smithi107 ceph-mon[90179]: from='client.34426 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:43:08.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:07 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/2679592888' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:43:09.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:08 smithi157 ceph-mon[79403]: pgmap v189: 65 pgs: 65 active+clean; 165 MiB data, 724 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:43:09.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:08 smithi157 ceph-mon[79403]: osdmap e80: 6 total, 6 up, 6 in 2024-04-22T23:43:09.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:08 smithi157 ceph-mon[79403]: fsmap cephfs:1 {0=cephfs.smithi107.ktuxjl=up:active} 2 up:standby 2024-04-22T23:43:09.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:08 smithi107 ceph-mon[90179]: pgmap v189: 65 pgs: 65 active+clean; 165 MiB data, 724 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:43:09.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:08 smithi107 ceph-mon[90179]: osdmap e80: 6 total, 6 up, 6 in 2024-04-22T23:43:09.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:08 smithi107 ceph-mon[90179]: fsmap cephfs:1 {0=cephfs.smithi107.ktuxjl=up:active} 2 up:standby 2024-04-22T23:43:11.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:10 smithi157 ceph-mon[79403]: pgmap v191: 65 pgs: 65 active+clean; 165 MiB data, 724 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:43:11.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:10 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:43:11.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:10 smithi107 ceph-mon[90179]: pgmap v191: 65 pgs: 65 active+clean; 165 MiB data, 724 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:43:11.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:10 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:43:13.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:12 smithi157 ceph-mon[79403]: pgmap v192: 65 pgs: 65 active+clean; 165 MiB data, 724 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:43:13.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:12 smithi107 ceph-mon[90179]: pgmap v192: 65 pgs: 65 active+clean; 165 MiB data, 724 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:43:14.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:14 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:43:14.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:14 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:43:14.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:14 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:43:14.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:14 smithi107 ceph-mon[90179]: pgmap v193: 65 pgs: 65 active+clean; 165 MiB data, 724 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:43:14.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:14 smithi107 ceph-mon[90179]: mds.? [v2:172.21.15.107:6826/418145840,v1:172.21.15.107:6827/418145840] up:boot 2024-04-22T23:43:14.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:14 smithi107 ceph-mon[90179]: fsmap cephfs:1 {0=cephfs.smithi107.ktuxjl=up:active} 3 up:standby 2024-04-22T23:43:14.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:14 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.dfvvkw"}]: dispatch 2024-04-22T23:43:14.380 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:14 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:43:14.380 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:14 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:43:14.380 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:14 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:43:14.380 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:14 smithi157 ceph-mon[79403]: pgmap v193: 65 pgs: 65 active+clean; 165 MiB data, 724 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:43:14.380 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:14 smithi157 ceph-mon[79403]: mds.? [v2:172.21.15.107:6826/418145840,v1:172.21.15.107:6827/418145840] up:boot 2024-04-22T23:43:14.380 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:14 smithi157 ceph-mon[79403]: fsmap cephfs:1 {0=cephfs.smithi107.ktuxjl=up:active} 3 up:standby 2024-04-22T23:43:14.380 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:14 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.dfvvkw"}]: dispatch 2024-04-22T23:43:17.037 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:16 smithi107 ceph-mon[90179]: pgmap v194: 65 pgs: 65 active+clean; 165 MiB data, 724 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:43:17.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:16 smithi157 ceph-mon[79403]: pgmap v194: 65 pgs: 65 active+clean; 165 MiB data, 724 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:43:19.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:18 smithi157 ceph-mon[79403]: pgmap v195: 65 pgs: 65 active+clean; 165 MiB data, 724 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:43:19.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:18 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:43:19.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:18 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:43:19.132 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:18 smithi107 ceph-mon[90179]: pgmap v195: 65 pgs: 65 active+clean; 165 MiB data, 724 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:43:19.133 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:18 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:43:19.133 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:18 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:43:20.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:20 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:43:20.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:20 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:43:20.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:20 smithi157 ceph-mon[79403]: pgmap v196: 65 pgs: 65 active+clean; 165 MiB data, 724 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:43:20.731 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:20 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:43:20.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:20 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:43:20.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:20 smithi107 ceph-mon[90179]: pgmap v196: 65 pgs: 65 active+clean; 165 MiB data, 724 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:43:23.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:22 smithi157 ceph-mon[79403]: pgmap v197: 65 pgs: 65 active+clean; 165 MiB data, 724 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:43:23.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:22 smithi107 ceph-mon[90179]: pgmap v197: 65 pgs: 65 active+clean; 165 MiB data, 724 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:43:24.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:23 smithi107 ceph-mon[90179]: Detected new or changed devices on smithi107 2024-04-22T23:43:24.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:23 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:43:24.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:23 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:43:24.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:23 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:43:24.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:23 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:43:24.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:23 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:43:24.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:23 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:43:24.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:23 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:43:24.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:23 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:43:24.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:23 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:43:24.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:23 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:43:24.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:23 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi107.ktuxjl"]}]: dispatch 2024-04-22T23:43:24.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:23 smithi107 ceph-mon[90179]: Upgrade: It appears safe to stop mds.cephfs.smithi107.ktuxjl 2024-04-22T23:43:24.233 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:23 smithi107 ceph-mon[90179]: pgmap v198: 65 pgs: 65 active+clean; 165 MiB data, 724 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:43:24.380 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:23 smithi157 ceph-mon[79403]: Detected new or changed devices on smithi107 2024-04-22T23:43:24.380 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:23 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:43:24.380 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:23 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:43:24.380 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:23 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:43:24.380 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:23 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:43:24.380 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:23 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:43:24.380 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:23 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:43:24.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:23 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:43:24.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:23 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:43:24.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:23 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:43:24.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:23 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:43:24.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:23 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi107.ktuxjl"]}]: dispatch 2024-04-22T23:43:24.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:23 smithi157 ceph-mon[79403]: Upgrade: It appears safe to stop mds.cephfs.smithi107.ktuxjl 2024-04-22T23:43:24.381 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:23 smithi157 ceph-mon[79403]: pgmap v198: 65 pgs: 65 active+clean; 165 MiB data, 724 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:43:25.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:25 smithi107 ceph-mon[90179]: Upgrade: Updating mds.cephfs.smithi107.ktuxjl 2024-04-22T23:43:25.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:25 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:43:25.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:25 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi107.ktuxjl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T23:43:25.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:25 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:43:25.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:25 smithi107 ceph-mon[90179]: Deploying daemon mds.cephfs.smithi107.ktuxjl on smithi107 2024-04-22T23:43:25.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:25 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:43:26.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:25 smithi157 ceph-mon[79403]: Upgrade: Updating mds.cephfs.smithi107.ktuxjl 2024-04-22T23:43:26.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:25 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:43:26.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:25 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi107.ktuxjl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T23:43:26.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:25 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:43:26.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:25 smithi157 ceph-mon[79403]: Deploying daemon mds.cephfs.smithi107.ktuxjl on smithi107 2024-04-22T23:43:26.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:25 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:43:26.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:26 smithi107 ceph-mon[90179]: pgmap v199: 65 pgs: 65 active+clean; 165 MiB data, 724 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:43:26.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:26 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-mon-smithi107[90175]: 2024-04-22T23:43:26.926+0000 7f293c91b700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-22T23:43:27.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:26 smithi157 ceph-mon[79403]: pgmap v199: 65 pgs: 65 active+clean; 165 MiB data, 724 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:43:28.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:27 smithi157 ceph-mon[79403]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-22T23:43:28.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:27 smithi157 ceph-mon[79403]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-22T23:43:28.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:27 smithi157 ceph-mon[79403]: osdmap e81: 6 total, 6 up, 6 in 2024-04-22T23:43:28.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:27 smithi157 ceph-mon[79403]: Standby daemon mds.cephfs.smithi157.ufywse assigned to filesystem cephfs as rank 0 2024-04-22T23:43:28.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:27 smithi157 ceph-mon[79403]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-22T23:43:28.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:27 smithi157 ceph-mon[79403]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-22T23:43:28.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:27 smithi157 ceph-mon[79403]: fsmap cephfs:1/1 {0=cephfs.smithi157.ufywse=up:replay} 2 up:standby 2024-04-22T23:43:28.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:27 smithi107 ceph-mon[90179]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-22T23:43:28.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:27 smithi107 ceph-mon[90179]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-22T23:43:28.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:27 smithi107 ceph-mon[90179]: osdmap e81: 6 total, 6 up, 6 in 2024-04-22T23:43:28.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:27 smithi107 ceph-mon[90179]: Standby daemon mds.cephfs.smithi157.ufywse assigned to filesystem cephfs as rank 0 2024-04-22T23:43:28.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:27 smithi107 ceph-mon[90179]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-22T23:43:28.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:27 smithi107 ceph-mon[90179]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-22T23:43:28.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:27 smithi107 ceph-mon[90179]: fsmap cephfs:1/1 {0=cephfs.smithi157.ufywse=up:replay} 2 up:standby 2024-04-22T23:43:29.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:28 smithi157 ceph-mon[79403]: pgmap v201: 65 pgs: 65 active+clean; 165 MiB data, 724 MiB used, 536 GiB / 536 GiB avail; 2.8 MiB/s rd, 1 op/s 2024-04-22T23:43:29.184 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:28 smithi107 ceph-mon[90179]: pgmap v201: 65 pgs: 65 active+clean; 165 MiB data, 724 MiB used, 536 GiB / 536 GiB avail; 2.8 MiB/s rd, 1 op/s 2024-04-22T23:43:31.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:30 smithi157 ceph-mon[79403]: pgmap v202: 65 pgs: 65 active+clean; 165 MiB data, 724 MiB used, 536 GiB / 536 GiB avail; 5.6 MiB/s rd, 2 op/s 2024-04-22T23:43:31.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:30 smithi107 ceph-mon[90179]: pgmap v202: 65 pgs: 65 active+clean; 165 MiB data, 724 MiB used, 536 GiB / 536 GiB avail; 5.6 MiB/s rd, 2 op/s 2024-04-22T23:43:33.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:32 smithi157 ceph-mon[79403]: pgmap v203: 65 pgs: 65 active+clean; 165 MiB data, 724 MiB used, 536 GiB / 536 GiB avail; 9.2 MiB/s rd, 3 op/s 2024-04-22T23:43:33.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:32 smithi107 ceph-mon[90179]: pgmap v203: 65 pgs: 65 active+clean; 165 MiB data, 724 MiB used, 536 GiB / 536 GiB avail; 9.2 MiB/s rd, 3 op/s 2024-04-22T23:43:35.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:34 smithi157 ceph-mon[79403]: pgmap v204: 65 pgs: 65 active+clean; 165 MiB data, 724 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 4 op/s 2024-04-22T23:43:35.201 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:34 smithi107 ceph-mon[90179]: pgmap v204: 65 pgs: 65 active+clean; 165 MiB data, 724 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 4 op/s 2024-04-22T23:43:35.852 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:35 smithi107 ceph-mon[90179]: reconnect by client.14522 v1:192.168.0.1:0/3769680709 after 0 2024-04-22T23:43:35.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:35 smithi107 ceph-mon[90179]: reconnect by client.14528 v1:172.21.15.157:0/2901108260 after 0 2024-04-22T23:43:35.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:35 smithi107 ceph-mon[90179]: mds.? [v2:172.21.15.157:6826/4072444199,v1:172.21.15.157:6827/4072444199] up:reconnect 2024-04-22T23:43:35.853 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:35 smithi107 ceph-mon[90179]: fsmap cephfs:1/1 {0=cephfs.smithi157.ufywse=up:reconnect} 2 up:standby 2024-04-22T23:43:36.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:35 smithi157 ceph-mon[79403]: reconnect by client.14522 v1:192.168.0.1:0/3769680709 after 0 2024-04-22T23:43:36.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:35 smithi157 ceph-mon[79403]: reconnect by client.14528 v1:172.21.15.157:0/2901108260 after 0 2024-04-22T23:43:36.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:35 smithi157 ceph-mon[79403]: mds.? [v2:172.21.15.157:6826/4072444199,v1:172.21.15.157:6827/4072444199] up:reconnect 2024-04-22T23:43:36.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:35 smithi157 ceph-mon[79403]: fsmap cephfs:1/1 {0=cephfs.smithi157.ufywse=up:reconnect} 2 up:standby 2024-04-22T23:43:37.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.121+0000 7fb2cf3e6700 1 -- 172.21.15.107:0/1375700230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2c80fc700 msgr2=0x7fb2c80fcb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:37.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.121+0000 7fb2cf3e6700 1 --2- 172.21.15.107:0/1375700230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2c80fc700 0x7fb2c80fcb70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8004940 tx=0x7fb2b8009e30 comp rx=0 tx=0).stop 2024-04-22T23:43:37.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.121+0000 7fb2cf3e6700 1 -- 172.21.15.107:0/1375700230 shutdown_connections 2024-04-22T23:43:37.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.121+0000 7fb2cf3e6700 1 --2- 172.21.15.107:0/1375700230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2c80fc700 0x7fb2c80fcb70 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:37.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.121+0000 7fb2cf3e6700 1 --2- 172.21.15.107:0/1375700230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2c80fb450 0x7fb2c80fb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:37.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.121+0000 7fb2cf3e6700 1 -- 172.21.15.107:0/1375700230 >> 172.21.15.107:0/1375700230 conn(0x7fb2c80f6bd0 msgr2=0x7fb2c80f8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:43:37.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.121+0000 7fb2cf3e6700 1 -- 172.21.15.107:0/1375700230 shutdown_connections 2024-04-22T23:43:37.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.121+0000 7fb2cf3e6700 1 -- 172.21.15.107:0/1375700230 wait complete. 2024-04-22T23:43:37.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.121+0000 7fb2cf3e6700 1 Processor -- start 2024-04-22T23:43:37.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.122+0000 7fb2cf3e6700 1 -- start start 2024-04-22T23:43:37.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.122+0000 7fb2cf3e6700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2c80fb450 0x7fb2c810da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:37.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.122+0000 7fb2cf3e6700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2c80fc700 0x7fb2c810dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:37.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.122+0000 7fb2cf3e6700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c810e5a0 con 0x7fb2c80fb450 2024-04-22T23:43:37.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.122+0000 7fb2cf3e6700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c810eda0 con 0x7fb2c80fc700 2024-04-22T23:43:37.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.122+0000 7fb2cd182700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2c80fb450 0x7fb2c810da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:37.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.122+0000 7fb2cd182700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2c80fb450 0x7fb2c810da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48228/0 (socket says 172.21.15.107:48228) 2024-04-22T23:43:37.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.122+0000 7fb2cd182700 1 -- 172.21.15.107:0/2049680386 learned_addr learned my addr 172.21.15.107:0/2049680386 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:43:37.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.122+0000 7fb2cc981700 1 --2- 172.21.15.107:0/2049680386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2c80fc700 0x7fb2c810dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:37.117 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:36 smithi107 ceph-mon[90179]: pgmap v205: 65 pgs: 65 active+clean; 165 MiB data, 724 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-04-22T23:43:37.118 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:36 smithi107 ceph-mon[90179]: mds.? [v2:172.21.15.157:6826/4072444199,v1:172.21.15.157:6827/4072444199] up:rejoin 2024-04-22T23:43:37.118 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:36 smithi107 ceph-mon[90179]: fsmap cephfs:1/1 {0=cephfs.smithi157.ufywse=up:rejoin} 2 up:standby 2024-04-22T23:43:37.118 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:36 smithi107 ceph-mon[90179]: daemon mds.cephfs.smithi157.ufywse is now active in filesystem cephfs as rank 0 2024-04-22T23:43:37.118 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:36 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:43:37.118 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:36 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:43:37.118 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:36 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:43:37.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.122+0000 7fb2cd182700 1 -- 172.21.15.107:0/2049680386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2c80fc700 msgr2=0x7fb2c810dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:37.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.122+0000 7fb2cd182700 1 --2- 172.21.15.107:0/2049680386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2c80fc700 0x7fb2c810dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:37.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.122+0000 7fb2cd182700 1 -- 172.21.15.107:0/2049680386 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b8005040 con 0x7fb2c80fb450 2024-04-22T23:43:37.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.122+0000 7fb2cd182700 1 --2- 172.21.15.107:0/2049680386 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2c80fb450 0x7fb2c810da60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb2c400a6e0 tx=0x7fb2c400c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:43:37.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.123+0000 7fb2bdffb700 1 -- 172.21.15.107:0/2049680386 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2c400f040 con 0x7fb2c80fb450 2024-04-22T23:43:37.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.123+0000 7fb2bdffb700 1 -- 172.21.15.107:0/2049680386 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb2c400de70 con 0x7fb2c80fb450 2024-04-22T23:43:37.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.123+0000 7fb2cf3e6700 1 -- 172.21.15.107:0/2049680386 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2c810f080 con 0x7fb2c80fb450 2024-04-22T23:43:37.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.123+0000 7fb2bdffb700 1 -- 172.21.15.107:0/2049680386 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2c40133e0 con 0x7fb2c80fb450 2024-04-22T23:43:37.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.123+0000 7fb2cf3e6700 1 -- 172.21.15.107:0/2049680386 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2c810f5c0 con 0x7fb2c80fb450 2024-04-22T23:43:37.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.124+0000 7fb2cf3e6700 1 -- 172.21.15.107:0/2049680386 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2c805f5f0 con 0x7fb2c80fb450 2024-04-22T23:43:37.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.124+0000 7fb2bdffb700 1 -- 172.21.15.107:0/2049680386 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fb2c400d9a0 con 0x7fb2c80fb450 2024-04-22T23:43:37.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.125+0000 7fb2bdffb700 1 --2- 172.21.15.107:0/2049680386 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fb2b406ecc0 0x7fb2b4071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:37.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.125+0000 7fb2bdffb700 1 -- 172.21.15.107:0/2049680386 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fb2c408e420 con 0x7fb2c80fb450 2024-04-22T23:43:37.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.125+0000 7fb2cc981700 1 --2- 172.21.15.107:0/2049680386 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fb2b406ecc0 0x7fb2b4071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:37.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.125+0000 7fb2cc981700 1 --2- 172.21.15.107:0/2049680386 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fb2b406ecc0 0x7fb2b4071170 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8009fd0 tx=0x7fb2b8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:43:37.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.130+0000 7fb2bdffb700 1 -- 172.21.15.107:0/2049680386 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fb2c405ae70 con 0x7fb2c80fb450 2024-04-22T23:43:37.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:36 smithi157 ceph-mon[79403]: pgmap v205: 65 pgs: 65 active+clean; 165 MiB data, 724 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-04-22T23:43:37.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:36 smithi157 ceph-mon[79403]: mds.? [v2:172.21.15.157:6826/4072444199,v1:172.21.15.157:6827/4072444199] up:rejoin 2024-04-22T23:43:37.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:36 smithi157 ceph-mon[79403]: fsmap cephfs:1/1 {0=cephfs.smithi157.ufywse=up:rejoin} 2 up:standby 2024-04-22T23:43:37.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:36 smithi157 ceph-mon[79403]: daemon mds.cephfs.smithi157.ufywse is now active in filesystem cephfs as rank 0 2024-04-22T23:43:37.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:36 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:43:37.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:36 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:43:37.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:36 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:43:37.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.370+0000 7fb2cf3e6700 1 -- 172.21.15.107:0/2049680386 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb2c810f930 con 0x7fb2b406ecc0 2024-04-22T23:43:37.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.371+0000 7fb2bdffb700 1 -- 172.21.15.107:0/2049680386 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fb2c810f930 con 0x7fb2b406ecc0 2024-04-22T23:43:37.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.373+0000 7fb2cf3e6700 1 -- 172.21.15.107:0/2049680386 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fb2b406ecc0 msgr2=0x7fb2b4071170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:37.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.373+0000 7fb2cf3e6700 1 --2- 172.21.15.107:0/2049680386 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fb2b406ecc0 0x7fb2b4071170 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8009fd0 tx=0x7fb2b8009e30 comp rx=0 tx=0).stop 2024-04-22T23:43:37.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.373+0000 7fb2cf3e6700 1 -- 172.21.15.107:0/2049680386 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2c80fb450 msgr2=0x7fb2c810da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:37.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.373+0000 7fb2cf3e6700 1 --2- 172.21.15.107:0/2049680386 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2c80fb450 0x7fb2c810da60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb2c400a6e0 tx=0x7fb2c400c040 comp rx=0 tx=0).stop 2024-04-22T23:43:37.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.373+0000 7fb2cf3e6700 1 -- 172.21.15.107:0/2049680386 shutdown_connections 2024-04-22T23:43:37.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.373+0000 7fb2cf3e6700 1 --2- 172.21.15.107:0/2049680386 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fb2b406ecc0 0x7fb2b4071170 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:37.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.373+0000 7fb2cf3e6700 1 --2- 172.21.15.107:0/2049680386 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2c80fb450 0x7fb2c810da60 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:37.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.373+0000 7fb2cf3e6700 1 --2- 172.21.15.107:0/2049680386 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2c80fc700 0x7fb2c810dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:37.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.373+0000 7fb2cf3e6700 1 -- 172.21.15.107:0/2049680386 >> 172.21.15.107:0/2049680386 conn(0x7fb2c80f6bd0 msgr2=0x7fb2c80f82b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:43:37.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.374+0000 7fb2cf3e6700 1 -- 172.21.15.107:0/2049680386 shutdown_connections 2024-04-22T23:43:37.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.374+0000 7fb2cf3e6700 1 -- 172.21.15.107:0/2049680386 wait complete. 2024-04-22T23:43:37.378 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-04-22T23:43:37.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.477+0000 7fd736f1c700 1 -- 172.21.15.107:0/332159986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7300fff20 msgr2=0x7fd730102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:37.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.477+0000 7fd736f1c700 1 --2- 172.21.15.107:0/332159986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7300fff20 0x7fd730102340 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd71c004940 tx=0x7fd71c009e30 comp rx=0 tx=0).stop 2024-04-22T23:43:37.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.477+0000 7fd736f1c700 1 -- 172.21.15.107:0/332159986 shutdown_connections 2024-04-22T23:43:37.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.477+0000 7fd736f1c700 1 --2- 172.21.15.107:0/332159986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7300fff20 0x7fd730102340 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:37.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.477+0000 7fd736f1c700 1 --2- 172.21.15.107:0/332159986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd730066b80 0x7fd7300ff9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:37.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.477+0000 7fd736f1c700 1 -- 172.21.15.107:0/332159986 >> 172.21.15.107:0/332159986 conn(0x7fd7300f7450 msgr2=0x7fd7300f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:43:37.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.477+0000 7fd736f1c700 1 -- 172.21.15.107:0/332159986 shutdown_connections 2024-04-22T23:43:37.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.477+0000 7fd736f1c700 1 -- 172.21.15.107:0/332159986 wait complete. 2024-04-22T23:43:37.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.478+0000 7fd736f1c700 1 Processor -- start 2024-04-22T23:43:37.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.478+0000 7fd736f1c700 1 -- start start 2024-04-22T23:43:37.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.478+0000 7fd736f1c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd730066b80 0x7fd73010da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:37.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.478+0000 7fd736f1c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7300fff20 0x7fd73010dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:37.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.478+0000 7fd736f1c700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd73010e5d0 con 0x7fd7300fff20 2024-04-22T23:43:37.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.478+0000 7fd736f1c700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd73010edd0 con 0x7fd730066b80 2024-04-22T23:43:37.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.478+0000 7fd72ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7300fff20 0x7fd73010dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:37.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.478+0000 7fd72ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7300fff20 0x7fd73010dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48242/0 (socket says 172.21.15.107:48242) 2024-04-22T23:43:37.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.478+0000 7fd72ffff700 1 -- 172.21.15.107:0/1568988241 learned_addr learned my addr 172.21.15.107:0/1568988241 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:43:37.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.479+0000 7fd734cb8700 1 --2- 172.21.15.107:0/1568988241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd730066b80 0x7fd73010da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:37.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.479+0000 7fd72ffff700 1 -- 172.21.15.107:0/1568988241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd730066b80 msgr2=0x7fd73010da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:37.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.479+0000 7fd72ffff700 1 --2- 172.21.15.107:0/1568988241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd730066b80 0x7fd73010da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:37.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.479+0000 7fd72ffff700 1 -- 172.21.15.107:0/1568988241 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd71c005040 con 0x7fd7300fff20 2024-04-22T23:43:37.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.479+0000 7fd72ffff700 1 --2- 172.21.15.107:0/1568988241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7300fff20 0x7fd73010dfd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd71c00b3b0 tx=0x7fd71c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:43:37.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.479+0000 7fd72d7fa700 1 -- 172.21.15.107:0/1568988241 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd71c00e040 con 0x7fd7300fff20 2024-04-22T23:43:37.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.479+0000 7fd736f1c700 1 -- 172.21.15.107:0/1568988241 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd73010f050 con 0x7fd7300fff20 2024-04-22T23:43:37.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.479+0000 7fd72d7fa700 1 -- 172.21.15.107:0/1568988241 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd71c00bbf0 con 0x7fd7300fff20 2024-04-22T23:43:37.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.479+0000 7fd72d7fa700 1 -- 172.21.15.107:0/1568988241 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd71c01c630 con 0x7fd7300fff20 2024-04-22T23:43:37.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.479+0000 7fd736f1c700 1 -- 172.21.15.107:0/1568988241 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd73010f590 con 0x7fd7300fff20 2024-04-22T23:43:37.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.480+0000 7fd736f1c700 1 -- 172.21.15.107:0/1568988241 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd73005f5f0 con 0x7fd7300fff20 2024-04-22T23:43:37.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.481+0000 7fd72d7fa700 1 -- 172.21.15.107:0/1568988241 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fd71c00bd60 con 0x7fd7300fff20 2024-04-22T23:43:37.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.481+0000 7fd72d7fa700 1 --2- 172.21.15.107:0/1568988241 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fd72006ec00 0x7fd7200710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:37.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.481+0000 7fd72d7fa700 1 -- 172.21.15.107:0/1568988241 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fd71c08f480 con 0x7fd7300fff20 2024-04-22T23:43:37.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.481+0000 7fd734cb8700 1 --2- 172.21.15.107:0/1568988241 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fd72006ec00 0x7fd7200710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:37.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.482+0000 7fd734cb8700 1 --2- 172.21.15.107:0/1568988241 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fd72006ec00 0x7fd7200710b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd7300668c0 tx=0x7fd72400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:43:37.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.486+0000 7fd72d7fa700 1 -- 172.21.15.107:0/1568988241 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fd71c05bd80 con 0x7fd7300fff20 2024-04-22T23:43:37.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.713+0000 7fd736f1c700 1 -- 172.21.15.107:0/1568988241 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd73010f8c0 con 0x7fd72006ec00 2024-04-22T23:43:37.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.714+0000 7fd72d7fa700 1 -- 172.21.15.107:0/1568988241 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd73010f8c0 con 0x7fd72006ec00 2024-04-22T23:43:37.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.716+0000 7fd736f1c700 1 -- 172.21.15.107:0/1568988241 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fd72006ec00 msgr2=0x7fd7200710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:37.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.716+0000 7fd736f1c700 1 --2- 172.21.15.107:0/1568988241 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fd72006ec00 0x7fd7200710b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd7300668c0 tx=0x7fd72400b040 comp rx=0 tx=0).stop 2024-04-22T23:43:37.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.716+0000 7fd736f1c700 1 -- 172.21.15.107:0/1568988241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7300fff20 msgr2=0x7fd73010dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:37.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.716+0000 7fd736f1c700 1 --2- 172.21.15.107:0/1568988241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7300fff20 0x7fd73010dfd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd71c00b3b0 tx=0x7fd71c009e30 comp rx=0 tx=0).stop 2024-04-22T23:43:37.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.716+0000 7fd736f1c700 1 -- 172.21.15.107:0/1568988241 shutdown_connections 2024-04-22T23:43:37.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.716+0000 7fd736f1c700 1 --2- 172.21.15.107:0/1568988241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd730066b80 0x7fd73010da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:37.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.716+0000 7fd736f1c700 1 --2- 172.21.15.107:0/1568988241 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fd72006ec00 0x7fd7200710b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:37.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.716+0000 7fd736f1c700 1 --2- 172.21.15.107:0/1568988241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7300fff20 0x7fd73010dfd0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:37.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.716+0000 7fd736f1c700 1 -- 172.21.15.107:0/1568988241 >> 172.21.15.107:0/1568988241 conn(0x7fd7300f7450 msgr2=0x7fd7300f7870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:43:37.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.716+0000 7fd736f1c700 1 -- 172.21.15.107:0/1568988241 shutdown_connections 2024-04-22T23:43:37.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.716+0000 7fd736f1c700 1 -- 172.21.15.107:0/1568988241 wait complete. 2024-04-22T23:43:37.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.819+0000 7f46cda2a700 1 -- 172.21.15.107:0/2701503981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46c80fe0e0 msgr2=0x7f46c80fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:37.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.819+0000 7f46cda2a700 1 --2- 172.21.15.107:0/2701503981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46c80fe0e0 0x7f46c80fe550 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f46bc004940 tx=0x7f46bc009e30 comp rx=0 tx=0).stop 2024-04-22T23:43:37.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.819+0000 7f46cda2a700 1 -- 172.21.15.107:0/2701503981 shutdown_connections 2024-04-22T23:43:37.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.819+0000 7f46cda2a700 1 --2- 172.21.15.107:0/2701503981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46c80fe0e0 0x7f46c80fe550 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:37.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.819+0000 7f46cda2a700 1 --2- 172.21.15.107:0/2701503981 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46c80fcef0 0x7f46c80fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:37.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.819+0000 7f46cda2a700 1 -- 172.21.15.107:0/2701503981 >> 172.21.15.107:0/2701503981 conn(0x7f46c80f8530 msgr2=0x7f46c80fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:43:37.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.819+0000 7f46cda2a700 1 -- 172.21.15.107:0/2701503981 shutdown_connections 2024-04-22T23:43:37.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.819+0000 7f46cda2a700 1 -- 172.21.15.107:0/2701503981 wait complete. 2024-04-22T23:43:37.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.820+0000 7f46cda2a700 1 Processor -- start 2024-04-22T23:43:37.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.820+0000 7f46cda2a700 1 -- start start 2024-04-22T23:43:37.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.820+0000 7f46cda2a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46c80fcef0 0x7f46c810da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:37.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.820+0000 7f46cda2a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46c80fe0e0 0x7f46c810dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:37.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.820+0000 7f46cda2a700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c810e650 con 0x7f46c80fe0e0 2024-04-22T23:43:37.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.820+0000 7f46cda2a700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c810ee40 con 0x7f46c80fcef0 2024-04-22T23:43:37.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.820+0000 7f46c67fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46c80fe0e0 0x7f46c810dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:37.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.820+0000 7f46c67fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46c80fe0e0 0x7f46c810dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48264/0 (socket says 172.21.15.107:48264) 2024-04-22T23:43:37.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.820+0000 7f46c67fc700 1 -- 172.21.15.107:0/1423806105 learned_addr learned my addr 172.21.15.107:0/1423806105 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:43:37.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.821+0000 7f46c67fc700 1 -- 172.21.15.107:0/1423806105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46c80fcef0 msgr2=0x7f46c810da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:37.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.821+0000 7f46c67fc700 1 --2- 172.21.15.107:0/1423806105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46c80fcef0 0x7f46c810da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:37.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.821+0000 7f46c67fc700 1 -- 172.21.15.107:0/1423806105 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46bc005040 con 0x7f46c80fe0e0 2024-04-22T23:43:37.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.821+0000 7f46c67fc700 1 --2- 172.21.15.107:0/1423806105 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46c80fe0e0 0x7f46c810dfc0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f46bc00b3b0 tx=0x7f46bc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:43:37.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.821+0000 7f46af7fe700 1 -- 172.21.15.107:0/1423806105 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46bc00e040 con 0x7f46c80fe0e0 2024-04-22T23:43:37.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.821+0000 7f46af7fe700 1 -- 172.21.15.107:0/1423806105 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f46bc00bcf0 con 0x7f46c80fe0e0 2024-04-22T23:43:37.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.821+0000 7f46cda2a700 1 -- 172.21.15.107:0/1423806105 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46c810f0c0 con 0x7f46c80fe0e0 2024-04-22T23:43:37.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.821+0000 7f46af7fe700 1 -- 172.21.15.107:0/1423806105 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46bc013640 con 0x7f46c80fe0e0 2024-04-22T23:43:37.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.821+0000 7f46cda2a700 1 -- 172.21.15.107:0/1423806105 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46c810f680 con 0x7f46c80fe0e0 2024-04-22T23:43:37.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.822+0000 7f46cda2a700 1 -- 172.21.15.107:0/1423806105 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46c805f5f0 con 0x7f46c80fe0e0 2024-04-22T23:43:37.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.823+0000 7f46af7fe700 1 -- 172.21.15.107:0/1423806105 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f46bc01b580 con 0x7f46c80fe0e0 2024-04-22T23:43:37.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.823+0000 7f46af7fe700 1 --2- 172.21.15.107:0/1423806105 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f46b006ec00 0x7f46b00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:37.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.823+0000 7f46af7fe700 1 -- 172.21.15.107:0/1423806105 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f46bc0937b0 con 0x7f46c80fe0e0 2024-04-22T23:43:37.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.823+0000 7f46c6ffd700 1 --2- 172.21.15.107:0/1423806105 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f46b006ec00 0x7f46b00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:37.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.823+0000 7f46c6ffd700 1 --2- 172.21.15.107:0/1423806105 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f46b006ec00 0x7f46b00710b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f46b8006fd0 tx=0x7f46b8006db0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:43:37.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:37.828+0000 7f46af7fe700 1 -- 172.21.15.107:0/1423806105 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f46bc060030 con 0x7f46c80fe0e0 2024-04-22T23:43:37.981 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:37 smithi107 ceph-mon[90179]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-22T23:43:37.981 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:37 smithi107 ceph-mon[90179]: Cluster is now healthy 2024-04-22T23:43:37.981 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:37 smithi107 ceph-mon[90179]: mds.? [v2:172.21.15.157:6826/4072444199,v1:172.21.15.157:6827/4072444199] up:active 2024-04-22T23:43:37.981 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:37 smithi107 ceph-mon[90179]: mds.? [v2:172.21.15.107:6828/1256497807,v1:172.21.15.107:6829/1256497807] up:boot 2024-04-22T23:43:37.981 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:37 smithi107 ceph-mon[90179]: fsmap cephfs:1 {0=cephfs.smithi157.ufywse=up:active} 3 up:standby 2024-04-22T23:43:37.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:37 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.ktuxjl"}]: dispatch 2024-04-22T23:43:38.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.043+0000 7f46cda2a700 1 -- 172.21.15.107:0/1423806105 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f46c810fa10 con 0x7f46b006ec00 2024-04-22T23:43:38.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.053+0000 7f46af7fe700 1 -- 172.21.15.107:0/1423806105 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f46c810fa10 con 0x7f46b006ec00 2024-04-22T23:43:38.047 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-22T23:43:38.047 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (6m) 19s ago 16m 24.6M - 0.25.0 c8568f914cd2 ec1a5c6260de 2024-04-22T23:43:38.047 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (4m) 19s ago 16m 6656k - 19.0.0-3252-g43be0201 a0434699c1d7 e074fd2cbb9c 2024-04-22T23:43:38.047 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi157 smithi157 running (4m) 81s ago 15m 6651k - 19.0.0-3252-g43be0201 a0434699c1d7 abc2649dee22 2024-04-22T23:43:38.047 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (6m) 19s ago 16m 86.7M - 9.4.7 954c08fa6188 31449ff442d2 2024-04-22T23:43:38.047 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.dfvvkw smithi107 running (25s) 19s ago 11m 12.7M - 19.0.0-3252-g43be0201 a0434699c1d7 7566b5cc3d01 2024-04-22T23:43:38.047 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.ktuxjl smithi107 starting - - - - 2024-04-22T23:43:38.047 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi157.rcaris smithi157 running (11m) 81s ago 11m 129M - 17.2.7-884-g70d0cb4b cafd8ddac8fa d23366ac085d 2024-04-22T23:43:38.048 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi157.ufywse smithi157 running (11m) 81s ago 11m 14.9M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 8b00d61eb355 2024-04-22T23:43:38.048 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.uipxvm smithi107 *:8443,9283,8765 running (6m) 19s ago 17m 510M - 19.0.0-3252-g43be0201 a0434699c1d7 d20f05dfe4f2 2024-04-22T23:43:38.048 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi157.tuylis smithi157 *:8443,9283,8765 running (6m) 81s ago 15m 451M - 19.0.0-3252-g43be0201 a0434699c1d7 957e598c9001 2024-04-22T23:43:38.048 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (5m) 19s ago 18m 53.3M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 bc72bbbe2c39 2024-04-22T23:43:38.048 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi157 smithi157 running (5m) 81s ago 15m 39.5M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 ea88de8be4fe 2024-04-22T23:43:38.048 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (7m) 19s ago 16m 16.4M - 1.5.0 0da6a335fe13 5205cfd9bd51 2024-04-22T23:43:38.048 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi157 smithi157 *:9100 running (6m) 81s ago 15m 17.2M - 1.5.0 0da6a335fe13 168ca191bb8c 2024-04-22T23:43:38.048 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (4m) 19s ago 14m 99.8M 1557M 19.0.0-3252-g43be0201 a0434699c1d7 fd0ba8ef0c27 2024-04-22T23:43:38.048 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (3m) 19s ago 14m 95.9M 1557M 19.0.0-3252-g43be0201 a0434699c1d7 77e33b6122e0 2024-04-22T23:43:38.048 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (3m) 19s ago 13m 93.0M 1557M 19.0.0-3252-g43be0201 a0434699c1d7 c307bd8cdcfc 2024-04-22T23:43:38.048 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi157 running (2m) 81s ago 13m 100M 2581M 19.0.0-3252-g43be0201 a0434699c1d7 e81ca519c831 2024-04-22T23:43:38.048 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi157 running (118s) 81s ago 13m 98.3M 2581M 19.0.0-3252-g43be0201 a0434699c1d7 d5b5a2e4781a 2024-04-22T23:43:38.048 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi157 running (84s) 81s ago 12m 11.6M 2581M 19.0.0-3252-g43be0201 a0434699c1d7 0ee423135fdb 2024-04-22T23:43:38.048 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (6m) 19s ago 16m 66.1M - 2.43.0 a07b618ecd1d 99a4ea8594da 2024-04-22T23:43:38.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.056+0000 7f46cda2a700 1 -- 172.21.15.107:0/1423806105 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f46b006ec00 msgr2=0x7f46b00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:38.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.056+0000 7f46cda2a700 1 --2- 172.21.15.107:0/1423806105 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f46b006ec00 0x7f46b00710b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f46b8006fd0 tx=0x7f46b8006db0 comp rx=0 tx=0).stop 2024-04-22T23:43:38.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.056+0000 7f46cda2a700 1 -- 172.21.15.107:0/1423806105 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46c80fe0e0 msgr2=0x7f46c810dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:38.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.056+0000 7f46cda2a700 1 --2- 172.21.15.107:0/1423806105 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46c80fe0e0 0x7f46c810dfc0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f46bc00b3b0 tx=0x7f46bc009e30 comp rx=0 tx=0).stop 2024-04-22T23:43:38.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.056+0000 7f46cda2a700 1 -- 172.21.15.107:0/1423806105 shutdown_connections 2024-04-22T23:43:38.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.056+0000 7f46cda2a700 1 --2- 172.21.15.107:0/1423806105 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46c80fcef0 0x7f46c810da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:38.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.056+0000 7f46cda2a700 1 --2- 172.21.15.107:0/1423806105 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f46b006ec00 0x7f46b00710b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:38.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.056+0000 7f46cda2a700 1 --2- 172.21.15.107:0/1423806105 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46c80fe0e0 0x7f46c810dfc0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:38.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.056+0000 7f46cda2a700 1 -- 172.21.15.107:0/1423806105 >> 172.21.15.107:0/1423806105 conn(0x7f46c80f8530 msgr2=0x7f46c80f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:43:38.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.056+0000 7f46cda2a700 1 -- 172.21.15.107:0/1423806105 shutdown_connections 2024-04-22T23:43:38.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.056+0000 7f46cda2a700 1 -- 172.21.15.107:0/1423806105 wait complete. 2024-04-22T23:43:38.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:37 smithi157 ceph-mon[79403]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-22T23:43:38.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:37 smithi157 ceph-mon[79403]: Cluster is now healthy 2024-04-22T23:43:38.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:37 smithi157 ceph-mon[79403]: mds.? [v2:172.21.15.157:6826/4072444199,v1:172.21.15.157:6827/4072444199] up:active 2024-04-22T23:43:38.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:37 smithi157 ceph-mon[79403]: mds.? [v2:172.21.15.107:6828/1256497807,v1:172.21.15.107:6829/1256497807] up:boot 2024-04-22T23:43:38.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:37 smithi157 ceph-mon[79403]: fsmap cephfs:1 {0=cephfs.smithi157.ufywse=up:active} 3 up:standby 2024-04-22T23:43:38.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:37 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi107.ktuxjl"}]: dispatch 2024-04-22T23:43:38.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.175+0000 7f9b5d35b700 1 -- 172.21.15.107:0/1425331503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b580fcff0 msgr2=0x7f9b580fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:38.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.175+0000 7f9b5d35b700 1 --2- 172.21.15.107:0/1425331503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b580fcff0 0x7f9b580fd460 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9b40004940 tx=0x7f9b40009e30 comp rx=0 tx=0).stop 2024-04-22T23:43:38.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.175+0000 7f9b5d35b700 1 -- 172.21.15.107:0/1425331503 shutdown_connections 2024-04-22T23:43:38.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.175+0000 7f9b5d35b700 1 --2- 172.21.15.107:0/1425331503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b580fcff0 0x7f9b580fd460 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:38.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.175+0000 7f9b5d35b700 1 --2- 172.21.15.107:0/1425331503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b580fbd40 0x7f9b580fc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:38.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.175+0000 7f9b5d35b700 1 -- 172.21.15.107:0/1425331503 >> 172.21.15.107:0/1425331503 conn(0x7f9b580f74c0 msgr2=0x7f9b580f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:43:38.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.175+0000 7f9b5d35b700 1 -- 172.21.15.107:0/1425331503 shutdown_connections 2024-04-22T23:43:38.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.175+0000 7f9b5d35b700 1 -- 172.21.15.107:0/1425331503 wait complete. 2024-04-22T23:43:38.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.175+0000 7f9b5d35b700 1 Processor -- start 2024-04-22T23:43:38.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.175+0000 7f9b5d35b700 1 -- start start 2024-04-22T23:43:38.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.176+0000 7f9b5d35b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b580fbd40 0x7f9b581106e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:38.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.176+0000 7f9b5d35b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b580fcff0 0x7f9b58112c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:38.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.176+0000 7f9b5d35b700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b580663c0 con 0x7f9b580fcff0 2024-04-22T23:43:38.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.176+0000 7f9b5d35b700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b58066500 con 0x7f9b580fbd40 2024-04-22T23:43:38.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.176+0000 7f9b567fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b580fcff0 0x7f9b58112c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:38.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.176+0000 7f9b567fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b580fcff0 0x7f9b58112c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48280/0 (socket says 172.21.15.107:48280) 2024-04-22T23:43:38.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.176+0000 7f9b567fc700 1 -- 172.21.15.107:0/2202059880 learned_addr learned my addr 172.21.15.107:0/2202059880 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:43:38.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.176+0000 7f9b56ffd700 1 --2- 172.21.15.107:0/2202059880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b580fbd40 0x7f9b581106e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:38.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.176+0000 7f9b567fc700 1 -- 172.21.15.107:0/2202059880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b580fbd40 msgr2=0x7f9b581106e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:38.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.176+0000 7f9b567fc700 1 --2- 172.21.15.107:0/2202059880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b580fbd40 0x7f9b581106e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:38.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.176+0000 7f9b567fc700 1 -- 172.21.15.107:0/2202059880 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b40005040 con 0x7f9b580fcff0 2024-04-22T23:43:38.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.176+0000 7f9b567fc700 1 --2- 172.21.15.107:0/2202059880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b580fcff0 0x7f9b58112c30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9b40009150 tx=0x7f9b40009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:43:38.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.176+0000 7f9b4f7fe700 1 -- 172.21.15.107:0/2202059880 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b4000e040 con 0x7f9b580fcff0 2024-04-22T23:43:38.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.176+0000 7f9b4f7fe700 1 -- 172.21.15.107:0/2202059880 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9b40004290 con 0x7f9b580fcff0 2024-04-22T23:43:38.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.176+0000 7f9b5d35b700 1 -- 172.21.15.107:0/2202059880 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b581131d0 con 0x7f9b580fcff0 2024-04-22T23:43:38.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.176+0000 7f9b4f7fe700 1 -- 172.21.15.107:0/2202059880 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b4001c630 con 0x7f9b580fcff0 2024-04-22T23:43:38.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.176+0000 7f9b5d35b700 1 -- 172.21.15.107:0/2202059880 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b58113790 con 0x7f9b580fcff0 2024-04-22T23:43:38.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.177+0000 7f9b5d35b700 1 -- 172.21.15.107:0/2202059880 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b5805f5f0 con 0x7f9b580fcff0 2024-04-22T23:43:38.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.178+0000 7f9b4f7fe700 1 -- 172.21.15.107:0/2202059880 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f9b4000bbf0 con 0x7f9b580fcff0 2024-04-22T23:43:38.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.178+0000 7f9b4f7fe700 1 --2- 172.21.15.107:0/2202059880 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f9b4406ec00 0x7f9b440710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:38.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.178+0000 7f9b4f7fe700 1 -- 172.21.15.107:0/2202059880 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f9b4008f490 con 0x7f9b580fcff0 2024-04-22T23:43:38.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.178+0000 7f9b56ffd700 1 --2- 172.21.15.107:0/2202059880 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f9b4406ec00 0x7f9b440710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:38.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.179+0000 7f9b56ffd700 1 --2- 172.21.15.107:0/2202059880 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f9b4406ec00 0x7f9b440710b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9b480041d0 tx=0x7f9b480043c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:43:38.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.183+0000 7f9b4f7fe700 1 -- 172.21.15.107:0/2202059880 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f9b40058260 con 0x7f9b580fcff0 2024-04-22T23:43:38.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.475+0000 7f9b5d35b700 1 -- 172.21.15.107:0/2202059880 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9b5804bed0 con 0x7f9b580fcff0 2024-04-22T23:43:38.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.475+0000 7f9b4f7fe700 1 -- 172.21.15.107:0/2202059880 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f9b4005b920 con 0x7f9b580fcff0 2024-04-22T23:43:38.469 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-22T23:43:38.469 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-04-22T23:43:38.469 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:43:38.469 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:43:38.469 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-04-22T23:43:38.469 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:43:38.469 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:43:38.469 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-04-22T23:43:38.470 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 6 2024-04-22T23:43:38.470 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:43:38.470 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-04-22T23:43:38.470 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 2, 2024-04-22T23:43:38.470 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:43:38.470 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:43:38.470 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-04-22T23:43:38.470 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 17.2.7-884-g70d0cb4b (70d0cb4b9d576828afb0a05dcf8732894d343ece) quincy (stable)": 2, 2024-04-22T23:43:38.470 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 12 2024-04-22T23:43:38.470 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T23:43:38.470 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-22T23:43:38.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.478+0000 7f9b5d35b700 1 -- 172.21.15.107:0/2202059880 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f9b4406ec00 msgr2=0x7f9b440710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:38.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.478+0000 7f9b5d35b700 1 --2- 172.21.15.107:0/2202059880 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f9b4406ec00 0x7f9b440710b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9b480041d0 tx=0x7f9b480043c0 comp rx=0 tx=0).stop 2024-04-22T23:43:38.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.478+0000 7f9b5d35b700 1 -- 172.21.15.107:0/2202059880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b580fcff0 msgr2=0x7f9b58112c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:38.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.478+0000 7f9b5d35b700 1 --2- 172.21.15.107:0/2202059880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b580fcff0 0x7f9b58112c30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9b40009150 tx=0x7f9b40009e30 comp rx=0 tx=0).stop 2024-04-22T23:43:38.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.478+0000 7f9b5d35b700 1 -- 172.21.15.107:0/2202059880 shutdown_connections 2024-04-22T23:43:38.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.478+0000 7f9b5d35b700 1 --2- 172.21.15.107:0/2202059880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9b580fbd40 0x7f9b581106e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:38.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.478+0000 7f9b5d35b700 1 --2- 172.21.15.107:0/2202059880 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f9b4406ec00 0x7f9b440710b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:38.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.478+0000 7f9b5d35b700 1 --2- 172.21.15.107:0/2202059880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b580fcff0 0x7f9b58112c30 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:38.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.478+0000 7f9b5d35b700 1 -- 172.21.15.107:0/2202059880 >> 172.21.15.107:0/2202059880 conn(0x7f9b580f74c0 msgr2=0x7f9b580f8bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:43:38.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.478+0000 7f9b5d35b700 1 -- 172.21.15.107:0/2202059880 shutdown_connections 2024-04-22T23:43:38.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.478+0000 7f9b5d35b700 1 -- 172.21.15.107:0/2202059880 wait complete. 2024-04-22T23:43:38.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.582+0000 7fdfacdc9700 1 -- 172.21.15.107:0/1522355826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfa80fcf90 msgr2=0x7fdfa80fd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:38.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.582+0000 7fdfacdc9700 1 --2- 172.21.15.107:0/1522355826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfa80fcf90 0x7fdfa80fd400 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c004940 tx=0x7fdf9c009e30 comp rx=0 tx=0).stop 2024-04-22T23:43:38.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.583+0000 7fdfacdc9700 1 -- 172.21.15.107:0/1522355826 shutdown_connections 2024-04-22T23:43:38.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.583+0000 7fdfacdc9700 1 --2- 172.21.15.107:0/1522355826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfa80fcf90 0x7fdfa80fd400 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:38.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.583+0000 7fdfacdc9700 1 --2- 172.21.15.107:0/1522355826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfa80fbe20 0x7fdfa80fc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:38.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.583+0000 7fdfacdc9700 1 -- 172.21.15.107:0/1522355826 >> 172.21.15.107:0/1522355826 conn(0x7fdfa80f74e0 msgr2=0x7fdfa80f9910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:43:38.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.583+0000 7fdfacdc9700 1 -- 172.21.15.107:0/1522355826 shutdown_connections 2024-04-22T23:43:38.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.583+0000 7fdfacdc9700 1 -- 172.21.15.107:0/1522355826 wait complete. 2024-04-22T23:43:38.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.583+0000 7fdfacdc9700 1 Processor -- start 2024-04-22T23:43:38.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.583+0000 7fdfacdc9700 1 -- start start 2024-04-22T23:43:38.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.583+0000 7fdfacdc9700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfa80fbe20 0x7fdfa810c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:38.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.583+0000 7fdfacdc9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfa80fcf90 0x7fdfa810ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:38.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.583+0000 7fdfacdc9700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa810d510 con 0x7fdfa80fcf90 2024-04-22T23:43:38.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.583+0000 7fdfacdc9700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa810dd00 con 0x7fdfa80fbe20 2024-04-22T23:43:38.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.584+0000 7fdfa5d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfa80fcf90 0x7fdfa810ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:38.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.584+0000 7fdfa5d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfa80fcf90 0x7fdfa810ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48284/0 (socket says 172.21.15.107:48284) 2024-04-22T23:43:38.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.584+0000 7fdfa5d9b700 1 -- 172.21.15.107:0/1812825647 learned_addr learned my addr 172.21.15.107:0/1812825647 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:43:38.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.584+0000 7fdfa659c700 1 --2- 172.21.15.107:0/1812825647 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfa80fbe20 0x7fdfa810c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:38.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.584+0000 7fdfa5d9b700 1 -- 172.21.15.107:0/1812825647 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfa80fbe20 msgr2=0x7fdfa810c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:38.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.584+0000 7fdfa5d9b700 1 --2- 172.21.15.107:0/1812825647 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfa80fbe20 0x7fdfa810c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:38.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.584+0000 7fdfa5d9b700 1 -- 172.21.15.107:0/1812825647 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf9c005040 con 0x7fdfa80fcf90 2024-04-22T23:43:38.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.584+0000 7fdfa5d9b700 1 --2- 172.21.15.107:0/1812825647 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfa80fcf90 0x7fdfa810ce80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c009150 tx=0x7fdf9c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:43:38.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.584+0000 7fdf96ffd700 1 -- 172.21.15.107:0/1812825647 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf9c00e040 con 0x7fdfa80fcf90 2024-04-22T23:43:38.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.584+0000 7fdf96ffd700 1 -- 172.21.15.107:0/1812825647 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdf9c00bcc0 con 0x7fdfa80fcf90 2024-04-22T23:43:38.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.584+0000 7fdfacdc9700 1 -- 172.21.15.107:0/1812825647 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfa810df80 con 0x7fdfa80fcf90 2024-04-22T23:43:38.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.584+0000 7fdf96ffd700 1 -- 172.21.15.107:0/1812825647 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf9c01c630 con 0x7fdfa80fcf90 2024-04-22T23:43:38.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.584+0000 7fdfacdc9700 1 -- 172.21.15.107:0/1812825647 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfa810e540 con 0x7fdfa80fcf90 2024-04-22T23:43:38.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.585+0000 7fdfacdc9700 1 -- 172.21.15.107:0/1812825647 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfa805f5f0 con 0x7fdfa80fcf90 2024-04-22T23:43:38.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.586+0000 7fdf96ffd700 1 -- 172.21.15.107:0/1812825647 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fdf9c013420 con 0x7fdfa80fcf90 2024-04-22T23:43:38.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.586+0000 7fdf96ffd700 1 --2- 172.21.15.107:0/1812825647 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fdf9006ec00 0x7fdf900710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:38.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.587+0000 7fdf96ffd700 1 -- 172.21.15.107:0/1812825647 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fdf9c08f590 con 0x7fdfa80fcf90 2024-04-22T23:43:38.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.587+0000 7fdfa659c700 1 --2- 172.21.15.107:0/1812825647 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fdf9006ec00 0x7fdf900710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:38.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.587+0000 7fdfa659c700 1 --2- 172.21.15.107:0/1812825647 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fdf9006ec00 0x7fdf900710b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdf98006fd0 tx=0x7fdf98008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:43:38.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.591+0000 7fdf96ffd700 1 -- 172.21.15.107:0/1812825647 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fdf9c05be90 con 0x7fdfa80fcf90 2024-04-22T23:43:38.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.838+0000 7fdfacdc9700 1 -- 172.21.15.107:0/1812825647 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fdfa810e760 con 0x7fdfa80fcf90 2024-04-22T23:43:38.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.838+0000 7fdf96ffd700 1 -- 172.21.15.107:0/1812825647 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1898 (secure 0 0 0) 0x7fdf9c05ba20 con 0x7fdfa80fcf90 2024-04-22T23:43:38.832 INFO:teuthology.orchestra.run.smithi107.stdout:e32 2024-04-22T23:43:38.832 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T23:43:38.832 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:43:38.832 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-04-22T23:43:38.832 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:43:38.832 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-04-22T23:43:38.832 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-04-22T23:43:38.833 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 32 2024-04-22T23:43:38.833 INFO:teuthology.orchestra.run.smithi107.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-22T23:43:38.833 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-04-22T23:32:20.146130+0000 2024-04-22T23:43:38.833 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-04-22T23:43:36.801717+0000 2024-04-22T23:43:38.833 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-04-22T23:43:38.833 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-04-22T23:43:38.833 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-04-22T23:43:38.833 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-04-22T23:43:38.833 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-04-22T23:43:38.833 INFO:teuthology.orchestra.run.smithi107.stdout:max_xattr_size 65536 2024-04-22T23:43:38.833 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-04-22T23:43:38.833 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-04-22T23:43:38.833 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 81 2024-04-22T23:43:38.833 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:43:38.834 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 1 2024-04-22T23:43:38.834 INFO:teuthology.orchestra.run.smithi107.stdout:in 0 2024-04-22T23:43:38.834 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=24277} 2024-04-22T23:43:38.834 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-04-22T23:43:38.834 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-04-22T23:43:38.834 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 1 2024-04-22T23:43:38.834 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-04-22T23:43:38.834 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-04-22T23:43:38.834 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data disabled 2024-04-22T23:43:38.834 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-04-22T23:43:38.834 INFO:teuthology.orchestra.run.smithi107.stdout:bal_rank_mask -1 2024-04-22T23:43:38.834 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-04-22T23:43:38.834 INFO:teuthology.orchestra.run.smithi107.stdout:qdb_cluster leader: 24277 members: 24277 2024-04-22T23:43:38.835 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi157.ufywse{0:24277} state up:active seq 169 join_fscid=1 addr [v2:172.21.15.157:6826/4072444199,v1:172.21.15.157:6827/4072444199] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:43:38.835 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:43:38.835 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:43:38.835 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-04-22T23:43:38.835 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:43:38.835 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.dfvvkw{-1:34434} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/418145840,v1:172.21.15.107:6827/418145840] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T23:43:38.835 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.ktuxjl{-1:34438} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6828/1256497807,v1:172.21.15.107:6829/1256497807] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T23:43:38.835 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi157.rcaris{-1:44279} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.157:6824/1722945322,v1:172.21.15.157:6825/1722945322] compat {c=[1],r=[1],i=[7ff]}] 2024-04-22T23:43:38.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.841+0000 7fdfacdc9700 1 -- 172.21.15.107:0/1812825647 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fdf9006ec00 msgr2=0x7fdf900710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:38.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.841+0000 7fdfacdc9700 1 --2- 172.21.15.107:0/1812825647 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fdf9006ec00 0x7fdf900710b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdf98006fd0 tx=0x7fdf98008040 comp rx=0 tx=0).stop 2024-04-22T23:43:38.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.841+0000 7fdfacdc9700 1 -- 172.21.15.107:0/1812825647 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfa80fcf90 msgr2=0x7fdfa810ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:38.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.841+0000 7fdfacdc9700 1 --2- 172.21.15.107:0/1812825647 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfa80fcf90 0x7fdfa810ce80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c009150 tx=0x7fdf9c009e30 comp rx=0 tx=0).stop 2024-04-22T23:43:38.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.841+0000 7fdfacdc9700 1 -- 172.21.15.107:0/1812825647 shutdown_connections 2024-04-22T23:43:38.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.841+0000 7fdfacdc9700 1 --2- 172.21.15.107:0/1812825647 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfa80fbe20 0x7fdfa810c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:38.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.841+0000 7fdfacdc9700 1 --2- 172.21.15.107:0/1812825647 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fdf9006ec00 0x7fdf900710b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:38.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.841+0000 7fdfacdc9700 1 --2- 172.21.15.107:0/1812825647 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfa80fcf90 0x7fdfa810ce80 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:38.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.841+0000 7fdfacdc9700 1 -- 172.21.15.107:0/1812825647 >> 172.21.15.107:0/1812825647 conn(0x7fdfa80f74e0 msgr2=0x7fdfa80f8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:43:38.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.842+0000 7fdfacdc9700 1 -- 172.21.15.107:0/1812825647 shutdown_connections 2024-04-22T23:43:38.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.842+0000 7fdfacdc9700 1 -- 172.21.15.107:0/1812825647 wait complete. 2024-04-22T23:43:38.837 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 32 2024-04-22T23:43:38.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.945+0000 7f009fb9a700 1 -- 172.21.15.107:0/2780454092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00980fcff0 msgr2=0x7f00980fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:38.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.945+0000 7f009fb9a700 1 --2- 172.21.15.107:0/2780454092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00980fcff0 0x7f00980fd460 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0088004940 tx=0x7f0088009e30 comp rx=0 tx=0).stop 2024-04-22T23:43:38.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.945+0000 7f009fb9a700 1 -- 172.21.15.107:0/2780454092 shutdown_connections 2024-04-22T23:43:38.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.945+0000 7f009fb9a700 1 --2- 172.21.15.107:0/2780454092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00980fcff0 0x7f00980fd460 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:38.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.945+0000 7f009fb9a700 1 --2- 172.21.15.107:0/2780454092 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00980fbd40 0x7f00980fc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:38.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.945+0000 7f009fb9a700 1 -- 172.21.15.107:0/2780454092 >> 172.21.15.107:0/2780454092 conn(0x7f00980f74c0 msgr2=0x7f00980f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:43:38.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.945+0000 7f009fb9a700 1 -- 172.21.15.107:0/2780454092 shutdown_connections 2024-04-22T23:43:38.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.945+0000 7f009fb9a700 1 -- 172.21.15.107:0/2780454092 wait complete. 2024-04-22T23:43:38.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.946+0000 7f009fb9a700 1 Processor -- start 2024-04-22T23:43:38.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.946+0000 7f009fb9a700 1 -- start start 2024-04-22T23:43:38.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.946+0000 7f009fb9a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00980fbd40 0x7f00980661e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:38.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.946+0000 7f009fb9a700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00980fcff0 0x7f0098112750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:38.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.946+0000 7f009fb9a700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0098112d80 con 0x7f00980fbd40 2024-04-22T23:43:38.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.946+0000 7f009fb9a700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0098112ef0 con 0x7f00980fcff0 2024-04-22T23:43:38.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.946+0000 7f009d936700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00980fbd40 0x7f00980661e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:38.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.946+0000 7f009d936700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00980fbd40 0x7f00980661e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48300/0 (socket says 172.21.15.107:48300) 2024-04-22T23:43:38.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.946+0000 7f009d936700 1 -- 172.21.15.107:0/4053289663 learned_addr learned my addr 172.21.15.107:0/4053289663 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:43:38.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.946+0000 7f009d135700 1 --2- 172.21.15.107:0/4053289663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00980fcff0 0x7f0098112750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:38.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.946+0000 7f009d936700 1 -- 172.21.15.107:0/4053289663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00980fcff0 msgr2=0x7f0098112750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:38.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.946+0000 7f009d936700 1 --2- 172.21.15.107:0/4053289663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00980fcff0 0x7f0098112750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:38.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.946+0000 7f009d936700 1 -- 172.21.15.107:0/4053289663 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0088005040 con 0x7f00980fbd40 2024-04-22T23:43:38.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.947+0000 7f009d936700 1 --2- 172.21.15.107:0/4053289663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00980fbd40 0x7f00980661e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0094002a50 tx=0x7f00940046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:43:38.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.947+0000 7f008e7fc700 1 -- 172.21.15.107:0/4053289663 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0094007cc0 con 0x7f00980fbd40 2024-04-22T23:43:38.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.947+0000 7f008e7fc700 1 -- 172.21.15.107:0/4053289663 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f009400f040 con 0x7f00980fbd40 2024-04-22T23:43:38.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.947+0000 7f009fb9a700 1 -- 172.21.15.107:0/4053289663 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00981131d0 con 0x7f00980fbd40 2024-04-22T23:43:38.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.947+0000 7f008e7fc700 1 -- 172.21.15.107:0/4053289663 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f009400da30 con 0x7f00980fbd40 2024-04-22T23:43:38.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.947+0000 7f009fb9a700 1 -- 172.21.15.107:0/4053289663 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0098113790 con 0x7f00980fbd40 2024-04-22T23:43:38.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.948+0000 7f009fb9a700 1 -- 172.21.15.107:0/4053289663 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f009805f5f0 con 0x7f00980fbd40 2024-04-22T23:43:38.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.948+0000 7f008e7fc700 1 -- 172.21.15.107:0/4053289663 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f009400db90 con 0x7f00980fbd40 2024-04-22T23:43:38.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.949+0000 7f008e7fc700 1 --2- 172.21.15.107:0/4053289663 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f008406ec10 0x7f00840710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:38.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.949+0000 7f008e7fc700 1 -- 172.21.15.107:0/4053289663 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f009408e680 con 0x7f00980fbd40 2024-04-22T23:43:38.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.949+0000 7f009d135700 1 --2- 172.21.15.107:0/4053289663 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f008406ec10 0x7f00840710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:38.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.949+0000 7f009d135700 1 --2- 172.21.15.107:0/4053289663 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f008406ec10 0x7f00840710c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0088000c00 tx=0x7f0088009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:43:38.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:38.954+0000 7f008e7fc700 1 -- 172.21.15.107:0/4053289663 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f00940573d0 con 0x7f00980fbd40 2024-04-22T23:43:39.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:38 smithi157 ceph-mon[79403]: from='client.34442 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:43:39.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:38 smithi157 ceph-mon[79403]: from='client.34446 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:43:39.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:38 smithi157 ceph-mon[79403]: pgmap v206: 65 pgs: 65 active+clean; 165 MiB data, 724 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 189 B/s wr, 6 op/s 2024-04-22T23:43:39.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:38 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/2202059880' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:43:39.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.184+0000 7f009fb9a700 1 -- 172.21.15.107:0/4053289663 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0098101350 con 0x7f008406ec10 2024-04-22T23:43:39.177 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:38 smithi107 ceph-mon[90179]: from='client.34442 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:43:39.177 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:38 smithi107 ceph-mon[90179]: from='client.34446 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:43:39.177 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:38 smithi107 ceph-mon[90179]: pgmap v206: 65 pgs: 65 active+clean; 165 MiB data, 724 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 189 B/s wr, 6 op/s 2024-04-22T23:43:39.177 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:38 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/2202059880' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:43:39.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.185+0000 7f008e7fc700 1 -- 172.21.15.107:0/4053289663 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f0098101350 con 0x7f008406ec10 2024-04-22T23:43:39.178 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-22T23:43:39.178 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c", 2024-04-22T23:43:39.178 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-04-22T23:43:39.178 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-22T23:43:39.179 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [ 2024-04-22T23:43:39.179 INFO:teuthology.orchestra.run.smithi107.stdout: "osd", 2024-04-22T23:43:39.179 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr", 2024-04-22T23:43:39.179 INFO:teuthology.orchestra.run.smithi107.stdout: "crash", 2024-04-22T23:43:39.179 INFO:teuthology.orchestra.run.smithi107.stdout: "mon" 2024-04-22T23:43:39.179 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-22T23:43:39.179 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "13/21 daemons upgraded", 2024-04-22T23:43:39.179 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Currently upgrading mds daemons", 2024-04-22T23:43:39.179 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-04-22T23:43:39.179 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-22T23:43:39.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.187+0000 7f009fb9a700 1 -- 172.21.15.107:0/4053289663 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f008406ec10 msgr2=0x7f00840710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:39.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.187+0000 7f009fb9a700 1 --2- 172.21.15.107:0/4053289663 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f008406ec10 0x7f00840710c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0088000c00 tx=0x7f0088009e30 comp rx=0 tx=0).stop 2024-04-22T23:43:39.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.187+0000 7f009fb9a700 1 -- 172.21.15.107:0/4053289663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00980fbd40 msgr2=0x7f00980661e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:39.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.187+0000 7f009fb9a700 1 --2- 172.21.15.107:0/4053289663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00980fbd40 0x7f00980661e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0094002a50 tx=0x7f00940046b0 comp rx=0 tx=0).stop 2024-04-22T23:43:39.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.187+0000 7f009fb9a700 1 -- 172.21.15.107:0/4053289663 shutdown_connections 2024-04-22T23:43:39.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.187+0000 7f009fb9a700 1 --2- 172.21.15.107:0/4053289663 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f008406ec10 0x7f00840710c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:39.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.187+0000 7f009fb9a700 1 --2- 172.21.15.107:0/4053289663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00980fbd40 0x7f00980661e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:39.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.187+0000 7f009fb9a700 1 --2- 172.21.15.107:0/4053289663 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00980fcff0 0x7f0098112750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:39.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.187+0000 7f009fb9a700 1 -- 172.21.15.107:0/4053289663 >> 172.21.15.107:0/4053289663 conn(0x7f00980f74c0 msgr2=0x7f00980f8bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:43:39.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.187+0000 7f009fb9a700 1 -- 172.21.15.107:0/4053289663 shutdown_connections 2024-04-22T23:43:39.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.187+0000 7f009fb9a700 1 -- 172.21.15.107:0/4053289663 wait complete. 2024-04-22T23:43:39.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.290+0000 7f339974f700 1 -- 172.21.15.107:0/354003442 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3394102380 msgr2=0x7f3394104760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:39.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.290+0000 7f339974f700 1 --2- 172.21.15.107:0/354003442 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3394102380 0x7f3394104760 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3388004940 tx=0x7f3388009e30 comp rx=0 tx=0).stop 2024-04-22T23:43:39.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.290+0000 7f339974f700 1 -- 172.21.15.107:0/354003442 shutdown_connections 2024-04-22T23:43:39.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.290+0000 7f339974f700 1 --2- 172.21.15.107:0/354003442 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3394102380 0x7f3394104760 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:39.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.290+0000 7f339974f700 1 --2- 172.21.15.107:0/354003442 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3394066b20 0x7f3394101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:39.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.290+0000 7f339974f700 1 -- 172.21.15.107:0/354003442 >> 172.21.15.107:0/354003442 conn(0x7f33940f7450 msgr2=0x7f33940f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:43:39.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.291+0000 7f339974f700 1 -- 172.21.15.107:0/354003442 shutdown_connections 2024-04-22T23:43:39.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.291+0000 7f339974f700 1 -- 172.21.15.107:0/354003442 wait complete. 2024-04-22T23:43:39.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.291+0000 7f339974f700 1 Processor -- start 2024-04-22T23:43:39.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.291+0000 7f339974f700 1 -- start start 2024-04-22T23:43:39.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.291+0000 7f339974f700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3394066b20 0x7f339410fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:39.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.291+0000 7f339974f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3394102380 0x7f33941101a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:39.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.291+0000 7f339974f700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33941107a0 con 0x7f3394102380 2024-04-22T23:43:39.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.291+0000 7f339974f700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3394110fa0 con 0x7f3394066b20 2024-04-22T23:43:39.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.292+0000 7f33927fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3394102380 0x7f33941101a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:39.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.292+0000 7f33927fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3394102380 0x7f33941101a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48330/0 (socket says 172.21.15.107:48330) 2024-04-22T23:43:39.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.292+0000 7f33927fc700 1 -- 172.21.15.107:0/2731784415 learned_addr learned my addr 172.21.15.107:0/2731784415 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:43:39.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.292+0000 7f3392ffd700 1 --2- 172.21.15.107:0/2731784415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3394066b20 0x7f339410fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:39.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.292+0000 7f33927fc700 1 -- 172.21.15.107:0/2731784415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3394066b20 msgr2=0x7f339410fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:39.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.292+0000 7f33927fc700 1 --2- 172.21.15.107:0/2731784415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3394066b20 0x7f339410fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:39.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.292+0000 7f33927fc700 1 -- 172.21.15.107:0/2731784415 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3388005040 con 0x7f3394102380 2024-04-22T23:43:39.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.292+0000 7f33927fc700 1 --2- 172.21.15.107:0/2731784415 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3394102380 0x7f33941101a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f338800b3b0 tx=0x7f3388009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:43:39.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.292+0000 7f337b7fe700 1 -- 172.21.15.107:0/2731784415 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f338800e040 con 0x7f3394102380 2024-04-22T23:43:39.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.292+0000 7f339974f700 1 -- 172.21.15.107:0/2731784415 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3394111220 con 0x7f3394102380 2024-04-22T23:43:39.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.292+0000 7f339974f700 1 -- 172.21.15.107:0/2731784415 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3394111760 con 0x7f3394102380 2024-04-22T23:43:39.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.292+0000 7f337b7fe700 1 -- 172.21.15.107:0/2731784415 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f338800bcc0 con 0x7f3394102380 2024-04-22T23:43:39.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.293+0000 7f337b7fe700 1 -- 172.21.15.107:0/2731784415 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f338801c630 con 0x7f3394102380 2024-04-22T23:43:39.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.293+0000 7f339974f700 1 -- 172.21.15.107:0/2731784415 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f339405f5f0 con 0x7f3394102380 2024-04-22T23:43:39.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.294+0000 7f337b7fe700 1 -- 172.21.15.107:0/2731784415 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f3388026440 con 0x7f3394102380 2024-04-22T23:43:39.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.294+0000 7f337b7fe700 1 --2- 172.21.15.107:0/2731784415 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f337c06ec00 0x7f337c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:43:39.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.294+0000 7f337b7fe700 1 -- 172.21.15.107:0/2731784415 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f338808f520 con 0x7f3394102380 2024-04-22T23:43:39.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.294+0000 7f3392ffd700 1 --2- 172.21.15.107:0/2731784415 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f337c06ec00 0x7f337c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:43:39.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.295+0000 7f3392ffd700 1 --2- 172.21.15.107:0/2731784415 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f337c06ec00 0x7f337c0710b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3384009fd0 tx=0x7f3384009f00 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:43:39.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.299+0000 7f337b7fe700 1 -- 172.21.15.107:0/2731784415 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f338805be20 con 0x7f3394102380 2024-04-22T23:43:39.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.587+0000 7f339974f700 1 -- 172.21.15.107:0/2731784415 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f339404bed0 con 0x7f3394102380 2024-04-22T23:43:39.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.587+0000 7f337b7fe700 1 -- 172.21.15.107:0/2731784415 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f338805b9b0 con 0x7f3394102380 2024-04-22T23:43:39.581 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_OK 2024-04-22T23:43:39.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.589+0000 7f339974f700 1 -- 172.21.15.107:0/2731784415 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f337c06ec00 msgr2=0x7f337c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:39.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.589+0000 7f339974f700 1 --2- 172.21.15.107:0/2731784415 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f337c06ec00 0x7f337c0710b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3384009fd0 tx=0x7f3384009f00 comp rx=0 tx=0).stop 2024-04-22T23:43:39.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.589+0000 7f339974f700 1 -- 172.21.15.107:0/2731784415 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3394102380 msgr2=0x7f33941101a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:43:39.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.589+0000 7f339974f700 1 --2- 172.21.15.107:0/2731784415 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3394102380 0x7f33941101a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f338800b3b0 tx=0x7f3388009e30 comp rx=0 tx=0).stop 2024-04-22T23:43:39.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.589+0000 7f339974f700 1 -- 172.21.15.107:0/2731784415 shutdown_connections 2024-04-22T23:43:39.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.589+0000 7f339974f700 1 --2- 172.21.15.107:0/2731784415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3394066b20 0x7f339410fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:39.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.589+0000 7f339974f700 1 --2- 172.21.15.107:0/2731784415 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f337c06ec00 0x7f337c0710b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:39.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.589+0000 7f339974f700 1 --2- 172.21.15.107:0/2731784415 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3394102380 0x7f33941101a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:43:39.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.589+0000 7f339974f700 1 -- 172.21.15.107:0/2731784415 >> 172.21.15.107:0/2731784415 conn(0x7f33940f7450 msgr2=0x7f33940f7870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:43:39.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.589+0000 7f339974f700 1 -- 172.21.15.107:0/2731784415 shutdown_connections 2024-04-22T23:43:39.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:43:39.590+0000 7f339974f700 1 -- 172.21.15.107:0/2731784415 wait complete. 2024-04-22T23:43:40.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:39 smithi157 ceph-mon[79403]: from='client.34450 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:43:40.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:39 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/1812825647' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:43:40.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:39 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/2731784415' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:43:40.147 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:39 smithi107 ceph-mon[90179]: from='client.34450 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:43:40.147 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:39 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/1812825647' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:43:40.147 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:39 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/2731784415' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:43:40.981 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:40 smithi107 ceph-mon[90179]: from='client.34462 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:43:40.981 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:40 smithi107 ceph-mon[90179]: pgmap v207: 65 pgs: 65 active+clean; 165 MiB data, 724 MiB used, 536 GiB / 536 GiB avail; 11 MiB/s rd, 170 B/s wr, 4 op/s 2024-04-22T23:43:40.981 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:40 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:43:41.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:40 smithi157 ceph-mon[79403]: from='client.34462 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:43:41.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:40 smithi157 ceph-mon[79403]: pgmap v207: 65 pgs: 65 active+clean; 165 MiB data, 724 MiB used, 536 GiB / 536 GiB avail; 11 MiB/s rd, 170 B/s wr, 4 op/s 2024-04-22T23:43:41.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:40 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:43:42.629 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:42 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:43:42.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:42 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:43:42.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:42 smithi157 ceph-mon[79403]: pgmap v208: 65 pgs: 65 active+clean; 165 MiB data, 728 MiB used, 536 GiB / 536 GiB avail; 8.7 MiB/s rd, 170 B/s wr, 6 op/s 2024-04-22T23:43:42.731 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:42 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:43:42.731 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:42 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:43:42.731 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:42 smithi107 ceph-mon[90179]: pgmap v208: 65 pgs: 65 active+clean; 165 MiB data, 728 MiB used, 536 GiB / 536 GiB avail; 8.7 MiB/s rd, 170 B/s wr, 6 op/s 2024-04-22T23:43:43.731 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:43 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:43:43.731 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:43 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:43:43.879 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:43 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:43:43.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:43 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:43:44.626 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:44 smithi107 ceph-mon[90179]: pgmap v209: 65 pgs: 65 active+clean; 165 MiB data, 728 MiB used, 536 GiB / 536 GiB avail; 5.7 MiB/s rd, 4.3 KiB/s wr, 6 op/s 2024-04-22T23:43:44.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:44 smithi157 ceph-mon[79403]: pgmap v209: 65 pgs: 65 active+clean; 165 MiB data, 728 MiB used, 536 GiB / 536 GiB avail; 5.7 MiB/s rd, 4.3 KiB/s wr, 6 op/s 2024-04-22T23:43:47.086 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:46 smithi157 ceph-mon[79403]: pgmap v210: 65 pgs: 65 active+clean; 165 MiB data, 728 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-04-22T23:43:47.086 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:46 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:43:47.086 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:46 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:43:47.086 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:46 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:43:47.086 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:46 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:43:47.086 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:46 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:43:47.087 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:46 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:43:47.087 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:46 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:43:47.087 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:46 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:43:47.087 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:46 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:43:47.087 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:46 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:43:47.087 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:46 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi157.rcaris"]}]: dispatch 2024-04-22T23:43:47.203 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:46 smithi107 ceph-mon[90179]: pgmap v210: 65 pgs: 65 active+clean; 165 MiB data, 728 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-04-22T23:43:47.203 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:46 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:43:47.203 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:46 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:43:47.203 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:46 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:43:47.203 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:46 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:43:47.203 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:46 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:43:47.203 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:46 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:43:47.203 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:46 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:43:47.203 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:46 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:43:47.203 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:46 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:43:47.204 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:46 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:43:47.204 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:46 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi157.rcaris"]}]: dispatch 2024-04-22T23:43:48.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:47 smithi157 ceph-mon[79403]: Upgrade: It appears safe to stop mds.cephfs.smithi157.rcaris 2024-04-22T23:43:48.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:47 smithi107 ceph-mon[90179]: Upgrade: It appears safe to stop mds.cephfs.smithi157.rcaris 2024-04-22T23:43:49.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:48 smithi157 ceph-mon[79403]: pgmap v211: 65 pgs: 65 active+clean; 165 MiB data, 728 MiB used, 536 GiB / 536 GiB avail; 3.2 KiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-04-22T23:43:49.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:48 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:43:49.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:48 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi157.rcaris", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T23:43:49.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:48 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:43:49.132 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:48 smithi107 ceph-mon[90179]: pgmap v211: 65 pgs: 65 active+clean; 165 MiB data, 728 MiB used, 536 GiB / 536 GiB avail; 3.2 KiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-04-22T23:43:49.132 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:48 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:43:49.133 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:48 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi157.rcaris", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T23:43:49.133 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:48 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:43:50.053 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:49 smithi157 ceph-mon[79403]: Upgrade: Updating mds.cephfs.smithi157.rcaris 2024-04-22T23:43:50.053 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:49 smithi157 ceph-mon[79403]: Deploying daemon mds.cephfs.smithi157.rcaris on smithi157 2024-04-22T23:43:50.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:49 smithi107 ceph-mon[90179]: Upgrade: Updating mds.cephfs.smithi157.rcaris 2024-04-22T23:43:50.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:49 smithi107 ceph-mon[90179]: Deploying daemon mds.cephfs.smithi157.rcaris on smithi157 2024-04-22T23:43:51.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:50 smithi157 ceph-mon[79403]: pgmap v212: 65 pgs: 65 active+clean; 165 MiB data, 728 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 4 op/s 2024-04-22T23:43:51.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:50 smithi107 ceph-mon[90179]: pgmap v212: 65 pgs: 65 active+clean; 165 MiB data, 728 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 4 op/s 2024-04-22T23:43:52.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:51 smithi157 ceph-mon[79403]: osdmap e82: 6 total, 6 up, 6 in 2024-04-22T23:43:52.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:51 smithi157 ceph-mon[79403]: fsmap cephfs:1 {0=cephfs.smithi157.ufywse=up:active} 2 up:standby 2024-04-22T23:43:52.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:51 smithi107 ceph-mon[90179]: osdmap e82: 6 total, 6 up, 6 in 2024-04-22T23:43:52.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:51 smithi107 ceph-mon[90179]: fsmap cephfs:1 {0=cephfs.smithi157.ufywse=up:active} 2 up:standby 2024-04-22T23:43:53.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:52 smithi157 ceph-mon[79403]: pgmap v214: 65 pgs: 65 active+clean; 165 MiB data, 728 MiB used, 536 GiB / 536 GiB avail; 5.0 KiB/s wr, 2 op/s 2024-04-22T23:43:53.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:52 smithi107 ceph-mon[90179]: pgmap v214: 65 pgs: 65 active+clean; 165 MiB data, 728 MiB used, 536 GiB / 536 GiB avail; 5.0 KiB/s wr, 2 op/s 2024-04-22T23:43:54.879 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:54 smithi157 ceph-mon[79403]: pgmap v215: 65 pgs: 65 active+clean; 165 MiB data, 728 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:43:54.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:54 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi157.rcaris"}]: dispatch 2024-04-22T23:43:55.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:54 smithi107 ceph-mon[90179]: pgmap v215: 65 pgs: 65 active+clean; 165 MiB data, 728 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:43:55.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:54 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi157.rcaris"}]: dispatch 2024-04-22T23:43:56.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:56 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:43:56.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:56 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:43:56.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:56 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:43:56.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:56 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:43:56.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:56 smithi157 ceph-mon[79403]: pgmap v216: 65 pgs: 65 active+clean; 165 MiB data, 728 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:43:56.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:56 smithi157 ceph-mon[79403]: mds.? [v2:172.21.15.157:6824/3923812398,v1:172.21.15.157:6825/3923812398] up:boot 2024-04-22T23:43:56.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:56 smithi157 ceph-mon[79403]: fsmap cephfs:1 {0=cephfs.smithi157.ufywse=up:active} 3 up:standby 2024-04-22T23:43:56.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:56 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi157.rcaris"}]: dispatch 2024-04-22T23:43:56.981 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:56 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:43:56.981 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:56 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:43:56.981 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:56 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:43:56.981 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:56 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:43:56.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:56 smithi107 ceph-mon[90179]: pgmap v216: 65 pgs: 65 active+clean; 165 MiB data, 728 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:43:56.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:56 smithi107 ceph-mon[90179]: mds.? [v2:172.21.15.157:6824/3923812398,v1:172.21.15.157:6825/3923812398] up:boot 2024-04-22T23:43:56.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:56 smithi107 ceph-mon[90179]: fsmap cephfs:1 {0=cephfs.smithi157.ufywse=up:active} 3 up:standby 2024-04-22T23:43:56.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:56 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi157.rcaris"}]: dispatch 2024-04-22T23:43:58.629 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:58 smithi157 ceph-mon[79403]: pgmap v217: 65 pgs: 65 active+clean; 165 MiB data, 728 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:43:58.731 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:58 smithi107 ceph-mon[90179]: pgmap v217: 65 pgs: 65 active+clean; 165 MiB data, 728 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:43:59.879 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:59 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:43:59.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:43:59 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:43:59.981 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:59 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:43:59.981 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:43:59 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:00.879 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:00 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:00.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:00 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:00.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:00 smithi157 ceph-mon[79403]: pgmap v218: 65 pgs: 65 active+clean; 165 MiB data, 728 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:44:00.981 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:00 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:00.981 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:00 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:00.981 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:00 smithi107 ceph-mon[90179]: pgmap v218: 65 pgs: 65 active+clean; 165 MiB data, 728 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:44:03.059 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:02 smithi157 ceph-mon[79403]: pgmap v219: 65 pgs: 65 active+clean; 165 MiB data, 728 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:44:03.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:02 smithi107 ceph-mon[90179]: pgmap v219: 65 pgs: 65 active+clean; 165 MiB data, 728 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:44:04.481 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:04 smithi107 ceph-mon[90179]: Detected new or changed devices on smithi157 2024-04-22T23:44:04.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:04 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:04.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:04 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:04.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:04 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:44:04.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:04 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:44:04.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:04 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:04.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:04 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:44:04.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:04 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:04.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:04 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:04.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:04 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:04.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:04 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:04.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:04 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi157.ufywse"]}]: dispatch 2024-04-22T23:44:04.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:04 smithi107 ceph-mon[90179]: Upgrade: It appears safe to stop mds.cephfs.smithi157.ufywse 2024-04-22T23:44:04.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:04 smithi107 ceph-mon[90179]: pgmap v220: 65 pgs: 65 active+clean; 165 MiB data, 728 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:44:04.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:04 smithi157 ceph-mon[79403]: Detected new or changed devices on smithi157 2024-04-22T23:44:04.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:04 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:04.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:04 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:04.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:04 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:44:04.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:04 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:44:04.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:04 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:04.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:04 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:44:04.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:04 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:04.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:04 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:04.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:04 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:04.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:04 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:04.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:04 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi157.ufywse"]}]: dispatch 2024-04-22T23:44:04.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:04 smithi157 ceph-mon[79403]: Upgrade: It appears safe to stop mds.cephfs.smithi157.ufywse 2024-04-22T23:44:04.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:04 smithi157 ceph-mon[79403]: pgmap v220: 65 pgs: 65 active+clean; 165 MiB data, 728 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:44:06.289 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:06 smithi157 ceph-mon[79403]: Upgrade: Updating mds.cephfs.smithi157.ufywse 2024-04-22T23:44:06.289 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:06 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:06.289 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:06 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi157.ufywse", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T23:44:06.290 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:06 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:44:06.290 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:06 smithi157 ceph-mon[79403]: Deploying daemon mds.cephfs.smithi157.ufywse on smithi157 2024-04-22T23:44:06.290 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:06 smithi157 ceph-mon[79403]: pgmap v221: 65 pgs: 65 active+clean; 165 MiB data, 728 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:44:06.481 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:06 smithi107 ceph-mon[90179]: Upgrade: Updating mds.cephfs.smithi157.ufywse 2024-04-22T23:44:06.481 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:06 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:06.481 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:06 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi157.ufywse", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T23:44:06.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:06 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:44:06.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:06 smithi107 ceph-mon[90179]: Deploying daemon mds.cephfs.smithi157.ufywse on smithi157 2024-04-22T23:44:06.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:06 smithi107 ceph-mon[90179]: pgmap v221: 65 pgs: 65 active+clean; 165 MiB data, 728 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:44:07.481 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:07 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-mon-smithi107[90175]: 2024-04-22T23:44:07.204+0000 7f293c91b700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-22T23:44:07.481 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:07 smithi107 ceph-mon[90179]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-22T23:44:07.481 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:07 smithi107 ceph-mon[90179]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-22T23:44:07.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:07 smithi107 ceph-mon[90179]: osdmap e83: 6 total, 6 up, 6 in 2024-04-22T23:44:07.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:07 smithi107 ceph-mon[90179]: Standby daemon mds.cephfs.smithi107.dfvvkw assigned to filesystem cephfs as rank 0 2024-04-22T23:44:07.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:07 smithi107 ceph-mon[90179]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-22T23:44:07.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:07 smithi107 ceph-mon[90179]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-22T23:44:07.482 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:07 smithi107 ceph-mon[90179]: fsmap cephfs:1/1 {0=cephfs.smithi107.dfvvkw=up:replay} 2 up:standby 2024-04-22T23:44:07.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:07 smithi157 ceph-mon[79403]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-22T23:44:07.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:07 smithi157 ceph-mon[79403]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-22T23:44:07.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:07 smithi157 ceph-mon[79403]: osdmap e83: 6 total, 6 up, 6 in 2024-04-22T23:44:07.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:07 smithi157 ceph-mon[79403]: Standby daemon mds.cephfs.smithi107.dfvvkw assigned to filesystem cephfs as rank 0 2024-04-22T23:44:07.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:07 smithi157 ceph-mon[79403]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-22T23:44:07.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:07 smithi157 ceph-mon[79403]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-22T23:44:07.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:07 smithi157 ceph-mon[79403]: fsmap cephfs:1/1 {0=cephfs.smithi107.dfvvkw=up:replay} 2 up:standby 2024-04-22T23:44:08.629 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:08 smithi157 ceph-mon[79403]: pgmap v223: 65 pgs: 65 active+clean; 165 MiB data, 728 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:44:08.731 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:08 smithi107 ceph-mon[90179]: pgmap v223: 65 pgs: 65 active+clean; 165 MiB data, 728 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:44:09.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:09.697+0000 7fb6ef56c700 1 -- 172.21.15.107:0/2291863005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6e80662f0 msgr2=0x7fb6e80ffa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:09.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:09.697+0000 7fb6ef56c700 1 --2- 172.21.15.107:0/2291863005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6e80662f0 0x7fb6e80ffa30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4004940 tx=0x7fb6e4009e30 comp rx=0 tx=0).stop 2024-04-22T23:44:09.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:09.697+0000 7fb6ef56c700 1 -- 172.21.15.107:0/2291863005 shutdown_connections 2024-04-22T23:44:09.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:09.697+0000 7fb6ef56c700 1 --2- 172.21.15.107:0/2291863005 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6e80fff70 0x7fb6e8102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:09.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:09.697+0000 7fb6ef56c700 1 --2- 172.21.15.107:0/2291863005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6e80662f0 0x7fb6e80ffa30 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:09.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:09.697+0000 7fb6ef56c700 1 -- 172.21.15.107:0/2291863005 >> 172.21.15.107:0/2291863005 conn(0x7fb6e80f74c0 msgr2=0x7fb6e80f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:44:09.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:09.698+0000 7fb6ef56c700 1 -- 172.21.15.107:0/2291863005 shutdown_connections 2024-04-22T23:44:09.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:09.698+0000 7fb6ef56c700 1 -- 172.21.15.107:0/2291863005 wait complete. 2024-04-22T23:44:09.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:09.698+0000 7fb6ef56c700 1 Processor -- start 2024-04-22T23:44:09.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:09.698+0000 7fb6ef56c700 1 -- start start 2024-04-22T23:44:09.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:09.698+0000 7fb6ef56c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6e80662f0 0x7fb6e810da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:09.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:09.698+0000 7fb6ef56c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6e80fff70 0x7fb6e810dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:09.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:09.698+0000 7fb6ef56c700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e810e660 con 0x7fb6e80fff70 2024-04-22T23:44:09.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:09.698+0000 7fb6ef56c700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e810ee00 con 0x7fb6e80662f0 2024-04-22T23:44:09.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:09.698+0000 7fb6ecb07700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6e80fff70 0x7fb6e810dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:09.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:09.698+0000 7fb6ecb07700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6e80fff70 0x7fb6e810dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35474/0 (socket says 172.21.15.107:35474) 2024-04-22T23:44:09.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:09.698+0000 7fb6ecb07700 1 -- 172.21.15.107:0/4264520744 learned_addr learned my addr 172.21.15.107:0/4264520744 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:44:09.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:09.699+0000 7fb6ed308700 1 --2- 172.21.15.107:0/4264520744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6e80662f0 0x7fb6e810da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:09.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:09.699+0000 7fb6ecb07700 1 -- 172.21.15.107:0/4264520744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6e80662f0 msgr2=0x7fb6e810da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:09.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:09.699+0000 7fb6ecb07700 1 --2- 172.21.15.107:0/4264520744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6e80662f0 0x7fb6e810da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:09.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:09.699+0000 7fb6ecb07700 1 -- 172.21.15.107:0/4264520744 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6e4005040 con 0x7fb6e80fff70 2024-04-22T23:44:09.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:09.699+0000 7fb6ecb07700 1 --2- 172.21.15.107:0/4264520744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6e80fff70 0x7fb6e810dfd0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb6d800b790 tx=0x7fb6d8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:44:09.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:09.699+0000 7fb6ddffb700 1 -- 172.21.15.107:0/4264520744 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb6d800f770 con 0x7fb6e80fff70 2024-04-22T23:44:09.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:09.699+0000 7fb6ef56c700 1 -- 172.21.15.107:0/4264520744 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6e810f0e0 con 0x7fb6e80fff70 2024-04-22T23:44:09.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:09.699+0000 7fb6ddffb700 1 -- 172.21.15.107:0/4264520744 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb6d800fda0 con 0x7fb6e80fff70 2024-04-22T23:44:09.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:09.699+0000 7fb6ddffb700 1 -- 172.21.15.107:0/4264520744 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb6d800d310 con 0x7fb6e80fff70 2024-04-22T23:44:09.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:09.699+0000 7fb6ef56c700 1 -- 172.21.15.107:0/4264520744 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6e810f6a0 con 0x7fb6e80fff70 2024-04-22T23:44:09.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:09.701+0000 7fb6ef56c700 1 -- 172.21.15.107:0/4264520744 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6e805f5f0 con 0x7fb6e80fff70 2024-04-22T23:44:09.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:09.701+0000 7fb6ddffb700 1 -- 172.21.15.107:0/4264520744 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fb6d800f8d0 con 0x7fb6e80fff70 2024-04-22T23:44:09.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:09.701+0000 7fb6ddffb700 1 --2- 172.21.15.107:0/4264520744 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fb6d406ec00 0x7fb6d40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:09.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:09.701+0000 7fb6ddffb700 1 -- 172.21.15.107:0/4264520744 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fb6d808e880 con 0x7fb6e80fff70 2024-04-22T23:44:09.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:09.702+0000 7fb6ed308700 1 --2- 172.21.15.107:0/4264520744 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fb6d406ec00 0x7fb6d40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:09.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:09.702+0000 7fb6ed308700 1 --2- 172.21.15.107:0/4264520744 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fb6d406ec00 0x7fb6d40710b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb6e400b970 tx=0x7fb6e4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:44:09.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:09.706+0000 7fb6ddffb700 1 -- 172.21.15.107:0/4264520744 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fb6d805b1d0 con 0x7fb6e80fff70 2024-04-22T23:44:09.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:09.954+0000 7fb6ef56c700 1 -- 172.21.15.107:0/4264520744 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb6e810f9e0 con 0x7fb6d406ec00 2024-04-22T23:44:09.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:09.956+0000 7fb6ddffb700 1 -- 172.21.15.107:0/4264520744 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fb6e810f9e0 con 0x7fb6d406ec00 2024-04-22T23:44:09.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:09.958+0000 7fb6ef56c700 1 -- 172.21.15.107:0/4264520744 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fb6d406ec00 msgr2=0x7fb6d40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:09.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:09.959+0000 7fb6ef56c700 1 --2- 172.21.15.107:0/4264520744 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fb6d406ec00 0x7fb6d40710b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb6e400b970 tx=0x7fb6e4009e30 comp rx=0 tx=0).stop 2024-04-22T23:44:09.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:09.959+0000 7fb6ef56c700 1 -- 172.21.15.107:0/4264520744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6e80fff70 msgr2=0x7fb6e810dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:09.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:09.959+0000 7fb6ef56c700 1 --2- 172.21.15.107:0/4264520744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6e80fff70 0x7fb6e810dfd0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb6d800b790 tx=0x7fb6d8009250 comp rx=0 tx=0).stop 2024-04-22T23:44:09.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:09.959+0000 7fb6ef56c700 1 -- 172.21.15.107:0/4264520744 shutdown_connections 2024-04-22T23:44:09.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:09.959+0000 7fb6ef56c700 1 --2- 172.21.15.107:0/4264520744 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6e80662f0 0x7fb6e810da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:09.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:09.959+0000 7fb6ef56c700 1 --2- 172.21.15.107:0/4264520744 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fb6d406ec00 0x7fb6d40710b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:09.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:09.959+0000 7fb6ef56c700 1 --2- 172.21.15.107:0/4264520744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6e80fff70 0x7fb6e810dfd0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:09.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:09.959+0000 7fb6ef56c700 1 -- 172.21.15.107:0/4264520744 >> 172.21.15.107:0/4264520744 conn(0x7fb6e80f74c0 msgr2=0x7fb6e80f7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:44:09.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:09.959+0000 7fb6ef56c700 1 -- 172.21.15.107:0/4264520744 shutdown_connections 2024-04-22T23:44:09.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:09.959+0000 7fb6ef56c700 1 -- 172.21.15.107:0/4264520744 wait complete. 2024-04-22T23:44:09.961 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-04-22T23:44:10.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.064+0000 7f94c0039700 1 -- 172.21.15.107:0/2682076093 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94b8066b80 msgr2=0x7f94b80ff9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:10.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.064+0000 7f94c0039700 1 --2- 172.21.15.107:0/2682076093 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94b8066b80 0x7f94b80ff9e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f94b4004940 tx=0x7f94b4009e30 comp rx=0 tx=0).stop 2024-04-22T23:44:10.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.064+0000 7f94c0039700 1 -- 172.21.15.107:0/2682076093 shutdown_connections 2024-04-22T23:44:10.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.064+0000 7f94c0039700 1 --2- 172.21.15.107:0/2682076093 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94b80fff20 0x7f94b8102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:10.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.064+0000 7f94c0039700 1 --2- 172.21.15.107:0/2682076093 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94b8066b80 0x7f94b80ff9e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:10.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.064+0000 7f94c0039700 1 -- 172.21.15.107:0/2682076093 >> 172.21.15.107:0/2682076093 conn(0x7f94b80f7450 msgr2=0x7f94b80f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:44:10.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.064+0000 7f94c0039700 1 -- 172.21.15.107:0/2682076093 shutdown_connections 2024-04-22T23:44:10.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.064+0000 7f94c0039700 1 -- 172.21.15.107:0/2682076093 wait complete. 2024-04-22T23:44:10.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.065+0000 7f94c0039700 1 Processor -- start 2024-04-22T23:44:10.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.065+0000 7f94c0039700 1 -- start start 2024-04-22T23:44:10.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.065+0000 7f94c0039700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94b8066b80 0x7f94b810fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:10.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.065+0000 7f94c0039700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94b80fff20 0x7f94b8110170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:10.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.065+0000 7f94c0039700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b8110800 con 0x7f94b80fff20 2024-04-22T23:44:10.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.065+0000 7f94c0039700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b8110ff0 con 0x7f94b8066b80 2024-04-22T23:44:10.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.065+0000 7f94bd5d4700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94b80fff20 0x7f94b8110170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:10.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.065+0000 7f94bd5d4700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94b80fff20 0x7f94b8110170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35496/0 (socket says 172.21.15.107:35496) 2024-04-22T23:44:10.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.065+0000 7f94bd5d4700 1 -- 172.21.15.107:0/2295189882 learned_addr learned my addr 172.21.15.107:0/2295189882 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:44:10.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.065+0000 7f94bddd5700 1 --2- 172.21.15.107:0/2295189882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94b8066b80 0x7f94b810fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:10.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.066+0000 7f94bd5d4700 1 -- 172.21.15.107:0/2295189882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94b8066b80 msgr2=0x7f94b810fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:10.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.066+0000 7f94bd5d4700 1 --2- 172.21.15.107:0/2295189882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94b8066b80 0x7f94b810fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:10.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.066+0000 7f94bd5d4700 1 -- 172.21.15.107:0/2295189882 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b4005040 con 0x7f94b80fff20 2024-04-22T23:44:10.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.066+0000 7f94bd5d4700 1 --2- 172.21.15.107:0/2295189882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94b80fff20 0x7f94b8110170 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f94a800b790 tx=0x7f94a8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:44:10.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.066+0000 7f94ae7fc700 1 -- 172.21.15.107:0/2295189882 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94a800f6f0 con 0x7f94b80fff20 2024-04-22T23:44:10.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.066+0000 7f94c0039700 1 -- 172.21.15.107:0/2295189882 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94b81112d0 con 0x7f94b80fff20 2024-04-22T23:44:10.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.066+0000 7f94ae7fc700 1 -- 172.21.15.107:0/2295189882 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f94a800fd20 con 0x7f94b80fff20 2024-04-22T23:44:10.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.066+0000 7f94ae7fc700 1 -- 172.21.15.107:0/2295189882 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94a80103a0 con 0x7f94b80fff20 2024-04-22T23:44:10.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.066+0000 7f94c0039700 1 -- 172.21.15.107:0/2295189882 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94b8111890 con 0x7f94b80fff20 2024-04-22T23:44:10.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.067+0000 7f94c0039700 1 -- 172.21.15.107:0/2295189882 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94b805f5f0 con 0x7f94b80fff20 2024-04-22T23:44:10.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.067+0000 7f94ae7fc700 1 -- 172.21.15.107:0/2295189882 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f94a8010500 con 0x7f94b80fff20 2024-04-22T23:44:10.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.068+0000 7f94ae7fc700 1 --2- 172.21.15.107:0/2295189882 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f94a406ec10 0x7f94a40710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:10.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.068+0000 7f94ae7fc700 1 -- 172.21.15.107:0/2295189882 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f94a808e640 con 0x7f94b80fff20 2024-04-22T23:44:10.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.068+0000 7f94bddd5700 1 --2- 172.21.15.107:0/2295189882 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f94a406ec10 0x7f94a40710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:10.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.069+0000 7f94bddd5700 1 --2- 172.21.15.107:0/2295189882 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f94a406ec10 0x7f94a40710c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f94b4000c00 tx=0x7f94b4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:44:10.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.073+0000 7f94ae7fc700 1 -- 172.21.15.107:0/2295189882 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f94a805b000 con 0x7f94b80fff20 2024-04-22T23:44:10.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.343+0000 7f94c0039700 1 -- 172.21.15.107:0/2295189882 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f94b8111bd0 con 0x7f94a406ec10 2024-04-22T23:44:10.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.345+0000 7f94ae7fc700 1 -- 172.21.15.107:0/2295189882 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f94b8111bd0 con 0x7f94a406ec10 2024-04-22T23:44:10.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.347+0000 7f94c0039700 1 -- 172.21.15.107:0/2295189882 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f94a406ec10 msgr2=0x7f94a40710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:10.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.347+0000 7f94c0039700 1 --2- 172.21.15.107:0/2295189882 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f94a406ec10 0x7f94a40710c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f94b4000c00 tx=0x7f94b4009e30 comp rx=0 tx=0).stop 2024-04-22T23:44:10.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.347+0000 7f94c0039700 1 -- 172.21.15.107:0/2295189882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94b80fff20 msgr2=0x7f94b8110170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:10.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.347+0000 7f94c0039700 1 --2- 172.21.15.107:0/2295189882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94b80fff20 0x7f94b8110170 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f94a800b790 tx=0x7f94a8009250 comp rx=0 tx=0).stop 2024-04-22T23:44:10.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.347+0000 7f94c0039700 1 -- 172.21.15.107:0/2295189882 shutdown_connections 2024-04-22T23:44:10.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.347+0000 7f94c0039700 1 --2- 172.21.15.107:0/2295189882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94b8066b80 0x7f94b810fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:10.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.347+0000 7f94c0039700 1 --2- 172.21.15.107:0/2295189882 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f94a406ec10 0x7f94a40710c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:10.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.347+0000 7f94c0039700 1 --2- 172.21.15.107:0/2295189882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94b80fff20 0x7f94b8110170 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:10.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.347+0000 7f94c0039700 1 -- 172.21.15.107:0/2295189882 >> 172.21.15.107:0/2295189882 conn(0x7f94b80f7450 msgr2=0x7f94b8100310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:44:10.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.347+0000 7f94c0039700 1 -- 172.21.15.107:0/2295189882 shutdown_connections 2024-04-22T23:44:10.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.347+0000 7f94c0039700 1 -- 172.21.15.107:0/2295189882 wait complete. 2024-04-22T23:44:10.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.451+0000 7f9cac446700 1 -- 172.21.15.107:0/4255487404 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ca41023d0 msgr2=0x7f9ca41047b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:10.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.451+0000 7f9cac446700 1 --2- 172.21.15.107:0/4255487404 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ca41023d0 0x7f9ca41047b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9c94004940 tx=0x7f9c94009e30 comp rx=0 tx=0).stop 2024-04-22T23:44:10.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.451+0000 7f9cac446700 1 -- 172.21.15.107:0/4255487404 shutdown_connections 2024-04-22T23:44:10.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.451+0000 7f9cac446700 1 --2- 172.21.15.107:0/4255487404 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ca41023d0 0x7f9ca41047b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:10.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.451+0000 7f9cac446700 1 --2- 172.21.15.107:0/4255487404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ca4066290 0x7f9ca4101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:10.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.451+0000 7f9cac446700 1 -- 172.21.15.107:0/4255487404 >> 172.21.15.107:0/4255487404 conn(0x7f9ca40f74c0 msgr2=0x7f9ca40f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:44:10.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.451+0000 7f9cac446700 1 -- 172.21.15.107:0/4255487404 shutdown_connections 2024-04-22T23:44:10.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.451+0000 7f9cac446700 1 -- 172.21.15.107:0/4255487404 wait complete. 2024-04-22T23:44:10.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.452+0000 7f9cac446700 1 Processor -- start 2024-04-22T23:44:10.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.452+0000 7f9cac446700 1 -- start start 2024-04-22T23:44:10.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.452+0000 7f9cac446700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ca4066290 0x7f9ca410fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:10.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.452+0000 7f9cac446700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ca41023d0 0x7f9ca4110170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:10.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.452+0000 7f9cac446700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca4110770 con 0x7f9ca4066290 2024-04-22T23:44:10.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.452+0000 7f9cac446700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca4110fc0 con 0x7f9ca41023d0 2024-04-22T23:44:10.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.452+0000 7f9caa1e2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ca4066290 0x7f9ca410fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:10.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.452+0000 7f9caa1e2700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ca4066290 0x7f9ca410fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35516/0 (socket says 172.21.15.107:35516) 2024-04-22T23:44:10.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.452+0000 7f9caa1e2700 1 -- 172.21.15.107:0/1566609864 learned_addr learned my addr 172.21.15.107:0/1566609864 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:44:10.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.452+0000 7f9ca99e1700 1 --2- 172.21.15.107:0/1566609864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ca41023d0 0x7f9ca4110170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:10.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.452+0000 7f9caa1e2700 1 -- 172.21.15.107:0/1566609864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ca41023d0 msgr2=0x7f9ca4110170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:10.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.452+0000 7f9caa1e2700 1 --2- 172.21.15.107:0/1566609864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ca41023d0 0x7f9ca4110170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:10.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.452+0000 7f9caa1e2700 1 -- 172.21.15.107:0/1566609864 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c94005040 con 0x7f9ca4066290 2024-04-22T23:44:10.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.453+0000 7f9caa1e2700 1 --2- 172.21.15.107:0/1566609864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ca4066290 0x7f9ca410fc30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0002a80 tx=0x7f9ca0004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:44:10.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.453+0000 7f9c9affd700 1 -- 172.21.15.107:0/1566609864 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ca0007cc0 con 0x7f9ca4066290 2024-04-22T23:44:10.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.453+0000 7f9c9affd700 1 -- 172.21.15.107:0/1566609864 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9ca000f040 con 0x7f9ca4066290 2024-04-22T23:44:10.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.453+0000 7f9cac446700 1 -- 172.21.15.107:0/1566609864 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ca41112a0 con 0x7f9ca4066290 2024-04-22T23:44:10.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.453+0000 7f9c9affd700 1 -- 172.21.15.107:0/1566609864 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ca000da30 con 0x7f9ca4066290 2024-04-22T23:44:10.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.453+0000 7f9cac446700 1 -- 172.21.15.107:0/1566609864 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ca41117e0 con 0x7f9ca4066290 2024-04-22T23:44:10.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.454+0000 7f9cac446700 1 -- 172.21.15.107:0/1566609864 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ca405f5f0 con 0x7f9ca4066290 2024-04-22T23:44:10.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.454+0000 7f9c9affd700 1 -- 172.21.15.107:0/1566609864 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f9ca000db90 con 0x7f9ca4066290 2024-04-22T23:44:10.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.455+0000 7f9c9affd700 1 --2- 172.21.15.107:0/1566609864 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f9c9006ec10 0x7f9c900710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:10.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.455+0000 7f9c9affd700 1 -- 172.21.15.107:0/1566609864 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f9ca008e6b0 con 0x7f9ca4066290 2024-04-22T23:44:10.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.455+0000 7f9ca99e1700 1 --2- 172.21.15.107:0/1566609864 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f9c9006ec10 0x7f9c900710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:10.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.455+0000 7f9ca99e1700 1 --2- 172.21.15.107:0/1566609864 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f9c9006ec10 0x7f9c900710c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9c94009fd0 tx=0x7f9c94009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:44:10.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.460+0000 7f9c9affd700 1 -- 172.21.15.107:0/1566609864 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f9ca005b030 con 0x7f9ca4066290 2024-04-22T23:44:10.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.679+0000 7f9cac446700 1 -- 172.21.15.107:0/1566609864 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9ca4111b70 con 0x7f9c9006ec10 2024-04-22T23:44:10.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.690+0000 7f9c9affd700 1 -- 172.21.15.107:0/1566609864 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f9ca4111b70 con 0x7f9c9006ec10 2024-04-22T23:44:10.683 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-22T23:44:10.683 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (7m) 29s ago 17m 24.6M - 0.25.0 c8568f914cd2 ec1a5c6260de 2024-04-22T23:44:10.683 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (5m) 29s ago 17m 6656k - 19.0.0-3252-g43be0201 a0434699c1d7 e074fd2cbb9c 2024-04-22T23:44:10.683 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi157 smithi157 running (5m) 12s ago 16m 6651k - 19.0.0-3252-g43be0201 a0434699c1d7 abc2649dee22 2024-04-22T23:44:10.683 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (7m) 29s ago 16m 86.7M - 9.4.7 954c08fa6188 31449ff442d2 2024-04-22T23:44:10.683 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.dfvvkw smithi107 running (58s) 29s ago 11m 14.1M - 19.0.0-3252-g43be0201 a0434699c1d7 7566b5cc3d01 2024-04-22T23:44:10.683 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.ktuxjl smithi107 running (34s) 29s ago 11m 13.9M - 19.0.0-3252-g43be0201 a0434699c1d7 ce23545f0ae2 2024-04-22T23:44:10.683 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi157.rcaris smithi157 running (15s) 12s ago 11m 11.6M - 19.0.0-3252-g43be0201 a0434699c1d7 12550073a997 2024-04-22T23:44:10.683 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi157.ufywse smithi157 running (11m) 12s ago 11m 71.3M - 17.2.7-884-g70d0cb4b cafd8ddac8fa 8b00d61eb355 2024-04-22T23:44:10.683 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.uipxvm smithi107 *:8443,9283,8765 running (7m) 29s ago 18m 510M - 19.0.0-3252-g43be0201 a0434699c1d7 d20f05dfe4f2 2024-04-22T23:44:10.684 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi157.tuylis smithi157 *:8443,9283,8765 running (6m) 12s ago 16m 452M - 19.0.0-3252-g43be0201 a0434699c1d7 957e598c9001 2024-04-22T23:44:10.684 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (6m) 29s ago 18m 54.5M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 bc72bbbe2c39 2024-04-22T23:44:10.684 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi157 smithi157 running (6m) 12s ago 16m 39.7M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 ea88de8be4fe 2024-04-22T23:44:10.684 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (7m) 29s ago 17m 16.5M - 1.5.0 0da6a335fe13 5205cfd9bd51 2024-04-22T23:44:10.684 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi157 smithi157 *:9100 running (7m) 12s ago 16m 16.8M - 1.5.0 0da6a335fe13 168ca191bb8c 2024-04-22T23:44:10.684 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (4m) 29s ago 15m 104M 1557M 19.0.0-3252-g43be0201 a0434699c1d7 fd0ba8ef0c27 2024-04-22T23:44:10.684 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (4m) 29s ago 14m 102M 1557M 19.0.0-3252-g43be0201 a0434699c1d7 77e33b6122e0 2024-04-22T23:44:10.684 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (3m) 29s ago 14m 97.4M 1557M 19.0.0-3252-g43be0201 a0434699c1d7 c307bd8cdcfc 2024-04-22T23:44:10.684 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi157 running (3m) 12s ago 14m 110M 2581M 19.0.0-3252-g43be0201 a0434699c1d7 e81ca519c831 2024-04-22T23:44:10.684 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi157 running (2m) 12s ago 13m 105M 2581M 19.0.0-3252-g43be0201 a0434699c1d7 d5b5a2e4781a 2024-04-22T23:44:10.684 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi157 running (117s) 12s ago 13m 96.0M 2581M 19.0.0-3252-g43be0201 a0434699c1d7 0ee423135fdb 2024-04-22T23:44:10.684 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (6m) 29s ago 16m 66.1M - 2.43.0 a07b618ecd1d 99a4ea8594da 2024-04-22T23:44:10.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.693+0000 7f9cac446700 1 -- 172.21.15.107:0/1566609864 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f9c9006ec10 msgr2=0x7f9c900710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:10.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.693+0000 7f9cac446700 1 --2- 172.21.15.107:0/1566609864 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f9c9006ec10 0x7f9c900710c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9c94009fd0 tx=0x7f9c94009e30 comp rx=0 tx=0).stop 2024-04-22T23:44:10.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.693+0000 7f9cac446700 1 -- 172.21.15.107:0/1566609864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ca4066290 msgr2=0x7f9ca410fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:10.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.693+0000 7f9cac446700 1 --2- 172.21.15.107:0/1566609864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ca4066290 0x7f9ca410fc30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0002a80 tx=0x7f9ca0004600 comp rx=0 tx=0).stop 2024-04-22T23:44:10.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.693+0000 7f9cac446700 1 -- 172.21.15.107:0/1566609864 shutdown_connections 2024-04-22T23:44:10.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.693+0000 7f9cac446700 1 --2- 172.21.15.107:0/1566609864 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f9c9006ec10 0x7f9c900710c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:10.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.693+0000 7f9cac446700 1 --2- 172.21.15.107:0/1566609864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ca4066290 0x7f9ca410fc30 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:10.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.693+0000 7f9cac446700 1 --2- 172.21.15.107:0/1566609864 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ca41023d0 0x7f9ca4110170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:10.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.693+0000 7f9cac446700 1 -- 172.21.15.107:0/1566609864 >> 172.21.15.107:0/1566609864 conn(0x7f9ca40f74c0 msgr2=0x7f9ca40f7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:44:10.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.693+0000 7f9cac446700 1 -- 172.21.15.107:0/1566609864 shutdown_connections 2024-04-22T23:44:10.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.693+0000 7f9cac446700 1 -- 172.21.15.107:0/1566609864 wait complete. 2024-04-22T23:44:10.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.798+0000 7fedceb13700 1 -- 172.21.15.107:0/3587928436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fedc8066290 msgr2=0x7fedc8101e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:10.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.798+0000 7fedceb13700 1 --2- 172.21.15.107:0/3587928436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fedc8066290 0x7fedc8101e90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fedbc004940 tx=0x7fedbc009e30 comp rx=0 tx=0).stop 2024-04-22T23:44:10.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.798+0000 7fedceb13700 1 -- 172.21.15.107:0/3587928436 shutdown_connections 2024-04-22T23:44:10.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.798+0000 7fedceb13700 1 --2- 172.21.15.107:0/3587928436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedc81023d0 0x7fedc81047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:10.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.798+0000 7fedceb13700 1 --2- 172.21.15.107:0/3587928436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fedc8066290 0x7fedc8101e90 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:10.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.798+0000 7fedceb13700 1 -- 172.21.15.107:0/3587928436 >> 172.21.15.107:0/3587928436 conn(0x7fedc80f74c0 msgr2=0x7fedc80f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:44:10.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.798+0000 7fedceb13700 1 -- 172.21.15.107:0/3587928436 shutdown_connections 2024-04-22T23:44:10.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.798+0000 7fedceb13700 1 -- 172.21.15.107:0/3587928436 wait complete. 2024-04-22T23:44:10.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.799+0000 7fedceb13700 1 Processor -- start 2024-04-22T23:44:10.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.799+0000 7fedceb13700 1 -- start start 2024-04-22T23:44:10.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.799+0000 7fedceb13700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fedc8066290 0x7fedc810fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:10.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.799+0000 7fedceb13700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedc81023d0 0x7fedc81101a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:10.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.799+0000 7fedceb13700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc81107a0 con 0x7fedc8066290 2024-04-22T23:44:10.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.799+0000 7fedceb13700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc8110ff0 con 0x7fedc81023d0 2024-04-22T23:44:10.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.799+0000 7fedcc8af700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fedc8066290 0x7fedc810fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:10.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.799+0000 7fedcc8af700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fedc8066290 0x7fedc810fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35536/0 (socket says 172.21.15.107:35536) 2024-04-22T23:44:10.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.799+0000 7fedcc8af700 1 -- 172.21.15.107:0/1352239730 learned_addr learned my addr 172.21.15.107:0/1352239730 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:44:10.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.800+0000 7fedcc8af700 1 -- 172.21.15.107:0/1352239730 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedc81023d0 msgr2=0x7fedc81101a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:10.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.800+0000 7fedcc8af700 1 --2- 172.21.15.107:0/1352239730 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedc81023d0 0x7fedc81101a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:10.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.800+0000 7fedcc8af700 1 -- 172.21.15.107:0/1352239730 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedbc005040 con 0x7fedc8066290 2024-04-22T23:44:10.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.800+0000 7fedcc8af700 1 --2- 172.21.15.107:0/1352239730 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fedc8066290 0x7fedc810fc60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fedbc00b970 tx=0x7fedbc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:44:10.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.800+0000 7fedc57fa700 1 -- 172.21.15.107:0/1352239730 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fedbc00e040 con 0x7fedc8066290 2024-04-22T23:44:10.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.800+0000 7fedceb13700 1 -- 172.21.15.107:0/1352239730 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedc8111270 con 0x7fedc8066290 2024-04-22T23:44:10.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.800+0000 7fedceb13700 1 -- 172.21.15.107:0/1352239730 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedc81117b0 con 0x7fedc8066290 2024-04-22T23:44:10.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.800+0000 7fedc57fa700 1 -- 172.21.15.107:0/1352239730 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fedbc004290 con 0x7fedc8066290 2024-04-22T23:44:10.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.800+0000 7fedc57fa700 1 -- 172.21.15.107:0/1352239730 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fedbc01c630 con 0x7fedc8066290 2024-04-22T23:44:10.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.801+0000 7fedceb13700 1 -- 172.21.15.107:0/1352239730 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedac0052e0 con 0x7fedc8066290 2024-04-22T23:44:10.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.802+0000 7fedc57fa700 1 -- 172.21.15.107:0/1352239730 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fedbc00bbf0 con 0x7fedc8066290 2024-04-22T23:44:10.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.802+0000 7fedc57fa700 1 --2- 172.21.15.107:0/1352239730 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fedb806ec10 0x7fedb80710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:10.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.802+0000 7fedc57fa700 1 -- 172.21.15.107:0/1352239730 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fedbc08fb50 con 0x7fedc8066290 2024-04-22T23:44:10.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.802+0000 7fedc7fff700 1 --2- 172.21.15.107:0/1352239730 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fedb806ec10 0x7fedb80710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:10.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.803+0000 7fedc7fff700 1 --2- 172.21.15.107:0/1352239730 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fedb806ec10 0x7fedb80710c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fedb4004d00 tx=0x7fedb4009ce0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:44:10.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:10.807+0000 7fedc57fa700 1 -- 172.21.15.107:0/1352239730 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fedbc05c4a0 con 0x7fedc8066290 2024-04-22T23:44:10.981 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:10 smithi107 ceph-mon[90179]: pgmap v224: 65 pgs: 65 active+clean; 165 MiB data, 728 MiB used, 536 GiB / 536 GiB avail; 2.8 MiB/s rd, 0 op/s 2024-04-22T23:44:10.981 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:10 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:10.981 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:10 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:44:11.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.101+0000 7fedceb13700 1 -- 172.21.15.107:0/1352239730 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fedac005da0 con 0x7fedc8066290 2024-04-22T23:44:11.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.101+0000 7fedc57fa700 1 -- 172.21.15.107:0/1352239730 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fedbc013e60 con 0x7fedc8066290 2024-04-22T23:44:11.094 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-22T23:44:11.094 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-04-22T23:44:11.094 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:44:11.094 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:44:11.094 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-04-22T23:44:11.094 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:44:11.094 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:44:11.094 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-04-22T23:44:11.094 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 6 2024-04-22T23:44:11.094 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:44:11.094 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-04-22T23:44:11.094 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 3 2024-04-22T23:44:11.095 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:44:11.095 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-04-22T23:44:11.095 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 13 2024-04-22T23:44:11.095 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T23:44:11.095 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-22T23:44:11.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.104+0000 7fedceb13700 1 -- 172.21.15.107:0/1352239730 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fedb806ec10 msgr2=0x7fedb80710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:11.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.104+0000 7fedceb13700 1 --2- 172.21.15.107:0/1352239730 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fedb806ec10 0x7fedb80710c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fedb4004d00 tx=0x7fedb4009ce0 comp rx=0 tx=0).stop 2024-04-22T23:44:11.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.104+0000 7fedceb13700 1 -- 172.21.15.107:0/1352239730 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fedc8066290 msgr2=0x7fedc810fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:11.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.104+0000 7fedceb13700 1 --2- 172.21.15.107:0/1352239730 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fedc8066290 0x7fedc810fc60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fedbc00b970 tx=0x7fedbc009e30 comp rx=0 tx=0).stop 2024-04-22T23:44:11.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.104+0000 7fedceb13700 1 -- 172.21.15.107:0/1352239730 shutdown_connections 2024-04-22T23:44:11.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.104+0000 7fedceb13700 1 --2- 172.21.15.107:0/1352239730 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fedb806ec10 0x7fedb80710c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:11.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.104+0000 7fedceb13700 1 --2- 172.21.15.107:0/1352239730 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fedc8066290 0x7fedc810fc60 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:11.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.104+0000 7fedceb13700 1 --2- 172.21.15.107:0/1352239730 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fedc81023d0 0x7fedc81101a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:11.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.104+0000 7fedceb13700 1 -- 172.21.15.107:0/1352239730 >> 172.21.15.107:0/1352239730 conn(0x7fedc80f74c0 msgr2=0x7fedc80f7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:44:11.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.104+0000 7fedceb13700 1 -- 172.21.15.107:0/1352239730 shutdown_connections 2024-04-22T23:44:11.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.104+0000 7fedceb13700 1 -- 172.21.15.107:0/1352239730 wait complete. 2024-04-22T23:44:11.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:10 smithi157 ceph-mon[79403]: pgmap v224: 65 pgs: 65 active+clean; 165 MiB data, 728 MiB used, 536 GiB / 536 GiB avail; 2.8 MiB/s rd, 0 op/s 2024-04-22T23:44:11.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:10 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:11.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:10 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:44:11.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.206+0000 7f1d8f510700 1 -- 172.21.15.107:0/883627849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d880fe090 msgr2=0x7f1d880fe500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:11.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.206+0000 7f1d8f510700 1 --2- 172.21.15.107:0/883627849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d880fe090 0x7f1d880fe500 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1d78004940 tx=0x7f1d78009e30 comp rx=0 tx=0).stop 2024-04-22T23:44:11.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.206+0000 7f1d8f510700 1 -- 172.21.15.107:0/883627849 shutdown_connections 2024-04-22T23:44:11.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.206+0000 7f1d8f510700 1 --2- 172.21.15.107:0/883627849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d880fe090 0x7f1d880fe500 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:11.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.206+0000 7f1d8f510700 1 --2- 172.21.15.107:0/883627849 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d880fcf20 0x7f1d880fd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:11.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.206+0000 7f1d8f510700 1 -- 172.21.15.107:0/883627849 >> 172.21.15.107:0/883627849 conn(0x7f1d880f85c0 msgr2=0x7f1d880faa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:44:11.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.206+0000 7f1d8f510700 1 -- 172.21.15.107:0/883627849 shutdown_connections 2024-04-22T23:44:11.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.206+0000 7f1d8f510700 1 -- 172.21.15.107:0/883627849 wait complete. 2024-04-22T23:44:11.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.207+0000 7f1d8f510700 1 Processor -- start 2024-04-22T23:44:11.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.207+0000 7f1d8f510700 1 -- start start 2024-04-22T23:44:11.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.207+0000 7f1d8f510700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d880fcf20 0x7f1d8810da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:11.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.207+0000 7f1d8f510700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d880fe090 0x7f1d8810dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:11.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.207+0000 7f1d8f510700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8810e5b0 con 0x7f1d880fe090 2024-04-22T23:44:11.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.207+0000 7f1d8f510700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8810ee00 con 0x7f1d880fcf20 2024-04-22T23:44:11.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.207+0000 7f1d8caab700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d880fe090 0x7f1d8810dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:11.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.207+0000 7f1d8caab700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d880fe090 0x7f1d8810dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35548/0 (socket says 172.21.15.107:35548) 2024-04-22T23:44:11.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.207+0000 7f1d8caab700 1 -- 172.21.15.107:0/3840868677 learned_addr learned my addr 172.21.15.107:0/3840868677 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:44:11.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.207+0000 7f1d8d2ac700 1 --2- 172.21.15.107:0/3840868677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d880fcf20 0x7f1d8810da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:11.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.207+0000 7f1d8caab700 1 -- 172.21.15.107:0/3840868677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d880fcf20 msgr2=0x7f1d8810da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:11.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.207+0000 7f1d8caab700 1 --2- 172.21.15.107:0/3840868677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d880fcf20 0x7f1d8810da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:11.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.208+0000 7f1d8caab700 1 -- 172.21.15.107:0/3840868677 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d78005040 con 0x7f1d880fe090 2024-04-22T23:44:11.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.208+0000 7f1d8caab700 1 --2- 172.21.15.107:0/3840868677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d880fe090 0x7f1d8810dfb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1d7800b3b0 tx=0x7f1d78009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:44:11.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.208+0000 7f1d7dffb700 1 -- 172.21.15.107:0/3840868677 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d7800e040 con 0x7f1d880fe090 2024-04-22T23:44:11.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.208+0000 7f1d8f510700 1 -- 172.21.15.107:0/3840868677 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d8810f080 con 0x7f1d880fe090 2024-04-22T23:44:11.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.208+0000 7f1d7dffb700 1 -- 172.21.15.107:0/3840868677 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1d7800bbf0 con 0x7f1d880fe090 2024-04-22T23:44:11.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.208+0000 7f1d7dffb700 1 -- 172.21.15.107:0/3840868677 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d7801c630 con 0x7f1d880fe090 2024-04-22T23:44:11.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.208+0000 7f1d8f510700 1 -- 172.21.15.107:0/3840868677 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d8810f5c0 con 0x7f1d880fe090 2024-04-22T23:44:11.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.209+0000 7f1d8f510700 1 -- 172.21.15.107:0/3840868677 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d8805f5f0 con 0x7f1d880fe090 2024-04-22T23:44:11.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.210+0000 7f1d7dffb700 1 -- 172.21.15.107:0/3840868677 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f1d7800bd60 con 0x7f1d880fe090 2024-04-22T23:44:11.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.211+0000 7f1d7dffb700 1 --2- 172.21.15.107:0/3840868677 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f1d7406ec00 0x7f1d740710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:11.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.211+0000 7f1d7dffb700 1 -- 172.21.15.107:0/3840868677 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f1d7808f500 con 0x7f1d880fe090 2024-04-22T23:44:11.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.211+0000 7f1d8d2ac700 1 --2- 172.21.15.107:0/3840868677 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f1d7406ec00 0x7f1d740710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:11.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.211+0000 7f1d8d2ac700 1 --2- 172.21.15.107:0/3840868677 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f1d7406ec00 0x7f1d740710b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1d84004f40 tx=0x7f1d8400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:44:11.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.219+0000 7f1d7dffb700 1 -- 172.21.15.107:0/3840868677 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f1d7805bdb0 con 0x7f1d880fe090 2024-04-22T23:44:11.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.473+0000 7f1d8f510700 1 -- 172.21.15.107:0/3840868677 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1d88105910 con 0x7f1d880fe090 2024-04-22T23:44:11.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.474+0000 7f1d7dffb700 1 -- 172.21.15.107:0/3840868677 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1754 (secure 0 0 0) 0x7f1d7805b940 con 0x7f1d880fe090 2024-04-22T23:44:11.467 INFO:teuthology.orchestra.run.smithi107.stdout:e36 2024-04-22T23:44:11.467 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T23:44:11.467 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:44:11.467 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-04-22T23:44:11.467 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:44:11.467 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-04-22T23:44:11.467 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-04-22T23:44:11.467 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 36 2024-04-22T23:44:11.467 INFO:teuthology.orchestra.run.smithi107.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-22T23:44:11.467 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-04-22T23:32:20.146130+0000 2024-04-22T23:44:11.467 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-04-22T23:44:07.212403+0000 2024-04-22T23:44:11.467 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-04-22T23:44:11.467 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-04-22T23:44:11.467 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-04-22T23:44:11.468 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-04-22T23:44:11.468 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-04-22T23:44:11.468 INFO:teuthology.orchestra.run.smithi107.stdout:max_xattr_size 65536 2024-04-22T23:44:11.468 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-04-22T23:44:11.468 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-04-22T23:44:11.468 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 83 2024-04-22T23:44:11.468 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-22T23:44:11.468 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 1 2024-04-22T23:44:11.468 INFO:teuthology.orchestra.run.smithi107.stdout:in 0 2024-04-22T23:44:11.468 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=34434} 2024-04-22T23:44:11.468 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-04-22T23:44:11.468 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-04-22T23:44:11.468 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 1 2024-04-22T23:44:11.468 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-04-22T23:44:11.469 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-04-22T23:44:11.469 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data disabled 2024-04-22T23:44:11.469 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-04-22T23:44:11.469 INFO:teuthology.orchestra.run.smithi107.stdout:bal_rank_mask -1 2024-04-22T23:44:11.469 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-04-22T23:44:11.469 INFO:teuthology.orchestra.run.smithi107.stdout:qdb_cluster leader: 0 members: 2024-04-22T23:44:11.469 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.dfvvkw{0:34434} state up:replay seq 1 join_fscid=1 addr [v2:172.21.15.107:6826/418145840,v1:172.21.15.107:6827/418145840] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T23:44:11.469 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:44:11.469 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:44:11.469 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-04-22T23:44:11.469 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:44:11.469 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.ktuxjl{-1:34438} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.107:6828/1256497807,v1:172.21.15.107:6829/1256497807] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T23:44:11.469 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi157.rcaris{-1:44323} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.157:6824/3923812398,v1:172.21.15.157:6825/3923812398] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T23:44:11.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.477+0000 7f1d8f510700 1 -- 172.21.15.107:0/3840868677 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f1d7406ec00 msgr2=0x7f1d740710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:11.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.477+0000 7f1d8f510700 1 --2- 172.21.15.107:0/3840868677 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f1d7406ec00 0x7f1d740710b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1d84004f40 tx=0x7f1d8400b040 comp rx=0 tx=0).stop 2024-04-22T23:44:11.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.477+0000 7f1d8f510700 1 -- 172.21.15.107:0/3840868677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d880fe090 msgr2=0x7f1d8810dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:11.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.477+0000 7f1d8f510700 1 --2- 172.21.15.107:0/3840868677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d880fe090 0x7f1d8810dfb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1d7800b3b0 tx=0x7f1d78009e30 comp rx=0 tx=0).stop 2024-04-22T23:44:11.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.477+0000 7f1d8f510700 1 -- 172.21.15.107:0/3840868677 shutdown_connections 2024-04-22T23:44:11.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.477+0000 7f1d8f510700 1 --2- 172.21.15.107:0/3840868677 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d880fcf20 0x7f1d8810da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:11.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.477+0000 7f1d8f510700 1 --2- 172.21.15.107:0/3840868677 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f1d7406ec00 0x7f1d740710b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:11.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.478+0000 7f1d8f510700 1 --2- 172.21.15.107:0/3840868677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d880fe090 0x7f1d8810dfb0 secure :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1d7800b3b0 tx=0x7f1d78009e30 comp rx=0 tx=0).stop 2024-04-22T23:44:11.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.478+0000 7f1d8f510700 1 -- 172.21.15.107:0/3840868677 >> 172.21.15.107:0/3840868677 conn(0x7f1d880f85c0 msgr2=0x7f1d880f9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:44:11.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.478+0000 7f1d8f510700 1 -- 172.21.15.107:0/3840868677 shutdown_connections 2024-04-22T23:44:11.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.478+0000 7f1d8f510700 1 -- 172.21.15.107:0/3840868677 wait complete. 2024-04-22T23:44:11.471 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 36 2024-04-22T23:44:11.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.582+0000 7f31ddb05700 1 -- 172.21.15.107:0/160301200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31d80662f0 msgr2=0x7f31d80ffa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:11.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.582+0000 7f31ddb05700 1 --2- 172.21.15.107:0/160301200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31d80662f0 0x7f31d80ffa30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f31c8004940 tx=0x7f31c8009e30 comp rx=0 tx=0).stop 2024-04-22T23:44:11.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.582+0000 7f31ddb05700 1 -- 172.21.15.107:0/160301200 shutdown_connections 2024-04-22T23:44:11.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.582+0000 7f31ddb05700 1 --2- 172.21.15.107:0/160301200 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31d80fff70 0x7f31d8102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:11.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.582+0000 7f31ddb05700 1 --2- 172.21.15.107:0/160301200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31d80662f0 0x7f31d80ffa30 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:11.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.582+0000 7f31ddb05700 1 -- 172.21.15.107:0/160301200 >> 172.21.15.107:0/160301200 conn(0x7f31d80f74c0 msgr2=0x7f31d80f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:44:11.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.582+0000 7f31ddb05700 1 -- 172.21.15.107:0/160301200 shutdown_connections 2024-04-22T23:44:11.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.582+0000 7f31ddb05700 1 -- 172.21.15.107:0/160301200 wait complete. 2024-04-22T23:44:11.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.583+0000 7f31ddb05700 1 Processor -- start 2024-04-22T23:44:11.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.583+0000 7f31ddb05700 1 -- start start 2024-04-22T23:44:11.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.584+0000 7f31ddb05700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31d80662f0 0x7f31d810da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:11.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.584+0000 7f31ddb05700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31d80fff70 0x7f31d810df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:11.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.584+0000 7f31ddb05700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d810e580 con 0x7f31d80662f0 2024-04-22T23:44:11.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.584+0000 7f31ddb05700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d810ed80 con 0x7f31d80fff70 2024-04-22T23:44:11.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.584+0000 7f31d77fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31d80662f0 0x7f31d810da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:11.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.584+0000 7f31d77fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31d80662f0 0x7f31d810da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34444/0 (socket says 172.21.15.107:34444) 2024-04-22T23:44:11.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.584+0000 7f31d77fe700 1 -- 172.21.15.107:0/2751468142 learned_addr learned my addr 172.21.15.107:0/2751468142 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:44:11.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.584+0000 7f31d6ffd700 1 --2- 172.21.15.107:0/2751468142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31d80fff70 0x7f31d810df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:11.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.584+0000 7f31d77fe700 1 -- 172.21.15.107:0/2751468142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31d80fff70 msgr2=0x7f31d810df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:11.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.584+0000 7f31d77fe700 1 --2- 172.21.15.107:0/2751468142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31d80fff70 0x7f31d810df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:11.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.584+0000 7f31d77fe700 1 -- 172.21.15.107:0/2751468142 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31c8005040 con 0x7f31d80662f0 2024-04-22T23:44:11.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.584+0000 7f31d77fe700 1 --2- 172.21.15.107:0/2751468142 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31d80662f0 0x7f31d810da40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f31c8004090 tx=0x7f31c8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:44:11.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.584+0000 7f31bffff700 1 -- 172.21.15.107:0/2751468142 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31c8021070 con 0x7f31d80662f0 2024-04-22T23:44:11.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.584+0000 7f31ddb05700 1 -- 172.21.15.107:0/2751468142 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31d810f000 con 0x7f31d80662f0 2024-04-22T23:44:11.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.584+0000 7f31bffff700 1 -- 172.21.15.107:0/2751468142 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f31c801c400 con 0x7f31d80662f0 2024-04-22T23:44:11.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.585+0000 7f31bffff700 1 -- 172.21.15.107:0/2751468142 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31c800e070 con 0x7f31d80662f0 2024-04-22T23:44:11.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.585+0000 7f31ddb05700 1 -- 172.21.15.107:0/2751468142 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31d810f4e0 con 0x7f31d80662f0 2024-04-22T23:44:11.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.586+0000 7f31ddb05700 1 -- 172.21.15.107:0/2751468142 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31d805f5f0 con 0x7f31d80662f0 2024-04-22T23:44:11.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.586+0000 7f31bffff700 1 -- 172.21.15.107:0/2751468142 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f31c8013af0 con 0x7f31d80662f0 2024-04-22T23:44:11.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.587+0000 7f31bffff700 1 --2- 172.21.15.107:0/2751468142 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f31c006ea00 0x7f31c0070eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:11.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.587+0000 7f31bffff700 1 -- 172.21.15.107:0/2751468142 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f31c808f5b0 con 0x7f31d80662f0 2024-04-22T23:44:11.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.587+0000 7f31d6ffd700 1 --2- 172.21.15.107:0/2751468142 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f31c006ea00 0x7f31c0070eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:11.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.587+0000 7f31d6ffd700 1 --2- 172.21.15.107:0/2751468142 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f31c006ea00 0x7f31c0070eb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f31cc009b80 tx=0x7f31cc00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:44:11.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.596+0000 7f31bffff700 1 -- 172.21.15.107:0/2751468142 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f31c805be60 con 0x7f31d80662f0 2024-04-22T23:44:11.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.848+0000 7f31ddb05700 1 -- 172.21.15.107:0/2751468142 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f31d810f720 con 0x7f31c006ea00 2024-04-22T23:44:11.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.850+0000 7f31bffff700 1 -- 172.21.15.107:0/2751468142 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f31d810f720 con 0x7f31c006ea00 2024-04-22T23:44:11.842 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-22T23:44:11.842 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c", 2024-04-22T23:44:11.843 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-04-22T23:44:11.843 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-22T23:44:11.843 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [ 2024-04-22T23:44:11.843 INFO:teuthology.orchestra.run.smithi107.stdout: "osd", 2024-04-22T23:44:11.843 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr", 2024-04-22T23:44:11.843 INFO:teuthology.orchestra.run.smithi107.stdout: "crash", 2024-04-22T23:44:11.843 INFO:teuthology.orchestra.run.smithi107.stdout: "mon" 2024-04-22T23:44:11.843 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-22T23:44:11.843 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "15/21 daemons upgraded", 2024-04-22T23:44:11.843 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Currently upgrading mds daemons", 2024-04-22T23:44:11.843 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-04-22T23:44:11.843 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-22T23:44:11.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.852+0000 7f31ddb05700 1 -- 172.21.15.107:0/2751468142 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f31c006ea00 msgr2=0x7f31c0070eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:11.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.852+0000 7f31ddb05700 1 --2- 172.21.15.107:0/2751468142 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f31c006ea00 0x7f31c0070eb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f31cc009b80 tx=0x7f31cc00b040 comp rx=0 tx=0).stop 2024-04-22T23:44:11.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.852+0000 7f31ddb05700 1 -- 172.21.15.107:0/2751468142 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31d80662f0 msgr2=0x7f31d810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:11.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.852+0000 7f31ddb05700 1 --2- 172.21.15.107:0/2751468142 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31d80662f0 0x7f31d810da40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f31c8004090 tx=0x7f31c8009e30 comp rx=0 tx=0).stop 2024-04-22T23:44:11.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.852+0000 7f31ddb05700 1 -- 172.21.15.107:0/2751468142 shutdown_connections 2024-04-22T23:44:11.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.852+0000 7f31ddb05700 1 --2- 172.21.15.107:0/2751468142 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f31c006ea00 0x7f31c0070eb0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:11.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.852+0000 7f31ddb05700 1 --2- 172.21.15.107:0/2751468142 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31d80662f0 0x7f31d810da40 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:11.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.852+0000 7f31ddb05700 1 --2- 172.21.15.107:0/2751468142 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f31d80fff70 0x7f31d810df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:11.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.852+0000 7f31ddb05700 1 -- 172.21.15.107:0/2751468142 >> 172.21.15.107:0/2751468142 conn(0x7f31d80f74c0 msgr2=0x7f31d80f7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:44:11.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.853+0000 7f31ddb05700 1 -- 172.21.15.107:0/2751468142 shutdown_connections 2024-04-22T23:44:11.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.853+0000 7f31ddb05700 1 -- 172.21.15.107:0/2751468142 wait complete. 2024-04-22T23:44:11.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.975+0000 7fab1fd14700 1 -- 172.21.15.107:0/240961549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab18100010 msgr2=0x7fab181023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:11.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.975+0000 7fab1fd14700 1 --2- 172.21.15.107:0/240961549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab18100010 0x7fab181023f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fab08004940 tx=0x7fab08009e30 comp rx=0 tx=0).stop 2024-04-22T23:44:11.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.975+0000 7fab1fd14700 1 -- 172.21.15.107:0/240961549 shutdown_connections 2024-04-22T23:44:11.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.975+0000 7fab1fd14700 1 --2- 172.21.15.107:0/240961549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab18100010 0x7fab181023f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:11.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.975+0000 7fab1fd14700 1 --2- 172.21.15.107:0/240961549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab18066250 0x7fab180ffad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:11.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.975+0000 7fab1fd14700 1 -- 172.21.15.107:0/240961549 >> 172.21.15.107:0/240961549 conn(0x7fab180f74c0 msgr2=0x7fab180f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:44:11.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.975+0000 7fab1fd14700 1 -- 172.21.15.107:0/240961549 shutdown_connections 2024-04-22T23:44:11.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.975+0000 7fab1fd14700 1 -- 172.21.15.107:0/240961549 wait complete. 2024-04-22T23:44:11.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.976+0000 7fab1fd14700 1 Processor -- start 2024-04-22T23:44:11.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.976+0000 7fab1fd14700 1 -- start start 2024-04-22T23:44:11.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.976+0000 7fab1fd14700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab18066250 0x7fab1810d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:11.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.976+0000 7fab1fd14700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab18100010 0x7fab1810df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:11.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.976+0000 7fab1fd14700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1810e530 con 0x7fab18100010 2024-04-22T23:44:11.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.976+0000 7fab1fd14700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1810ed30 con 0x7fab18066250 2024-04-22T23:44:11.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.976+0000 7fab1d2af700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab18100010 0x7fab1810df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:11.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.976+0000 7fab1d2af700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab18100010 0x7fab1810df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34468/0 (socket says 172.21.15.107:34468) 2024-04-22T23:44:11.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.976+0000 7fab1d2af700 1 -- 172.21.15.107:0/2819399928 learned_addr learned my addr 172.21.15.107:0/2819399928 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:44:11.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.976+0000 7fab1dab0700 1 --2- 172.21.15.107:0/2819399928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab18066250 0x7fab1810d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:11.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.977+0000 7fab1d2af700 1 -- 172.21.15.107:0/2819399928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab18066250 msgr2=0x7fab1810d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:11.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.977+0000 7fab1d2af700 1 --2- 172.21.15.107:0/2819399928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab18066250 0x7fab1810d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:11.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.977+0000 7fab1d2af700 1 -- 172.21.15.107:0/2819399928 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab08005040 con 0x7fab18100010 2024-04-22T23:44:11.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.977+0000 7fab1d2af700 1 --2- 172.21.15.107:0/2819399928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab18100010 0x7fab1810df30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fab08013870 tx=0x7fab08009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:44:11.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.977+0000 7fab0e7fc700 1 -- 172.21.15.107:0/2819399928 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab08021070 con 0x7fab18100010 2024-04-22T23:44:11.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.977+0000 7fab0e7fc700 1 -- 172.21.15.107:0/2819399928 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fab08009430 con 0x7fab18100010 2024-04-22T23:44:11.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.977+0000 7fab1fd14700 1 -- 172.21.15.107:0/2819399928 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab1810efb0 con 0x7fab18100010 2024-04-22T23:44:11.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.977+0000 7fab0e7fc700 1 -- 172.21.15.107:0/2819399928 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab0800e070 con 0x7fab18100010 2024-04-22T23:44:11.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.977+0000 7fab1fd14700 1 -- 172.21.15.107:0/2819399928 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab1810f4f0 con 0x7fab18100010 2024-04-22T23:44:11.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.978+0000 7fab1fd14700 1 -- 172.21.15.107:0/2819399928 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab1805f5f0 con 0x7fab18100010 2024-04-22T23:44:11.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.979+0000 7fab0e7fc700 1 -- 172.21.15.107:0/2819399928 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fab0801c7e0 con 0x7fab18100010 2024-04-22T23:44:11.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.979+0000 7fab0e7fc700 1 --2- 172.21.15.107:0/2819399928 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fab0406ec00 0x7fab040710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:11.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.980+0000 7fab0e7fc700 1 -- 172.21.15.107:0/2819399928 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fab0808fad0 con 0x7fab18100010 2024-04-22T23:44:11.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.980+0000 7fab1dab0700 1 --2- 172.21.15.107:0/2819399928 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fab0406ec00 0x7fab040710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:11.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.980+0000 7fab1dab0700 1 --2- 172.21.15.107:0/2819399928 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fab0406ec00 0x7fab040710b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fab14009b80 tx=0x7fab1400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:44:11.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:11.984+0000 7fab0e7fc700 1 -- 172.21.15.107:0/2819399928 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fab0805c420 con 0x7fab18100010 2024-04-22T23:44:12.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:11 smithi157 ceph-mon[79403]: from='client.34472 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:44:12.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:11 smithi157 ceph-mon[79403]: from='client.34476 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:44:12.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:11 smithi157 ceph-mon[79403]: from='client.34480 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:44:12.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:11 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/1352239730' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:12.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:11 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/3840868677' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:44:12.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:11 smithi107 ceph-mon[90179]: from='client.34472 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:44:12.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:11 smithi107 ceph-mon[90179]: from='client.34476 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:44:12.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:11 smithi107 ceph-mon[90179]: from='client.34480 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:44:12.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:11 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/1352239730' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:12.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:11 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/3840868677' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:44:12.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:12.278+0000 7fab1fd14700 1 -- 172.21.15.107:0/2819399928 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fab1804bed0 con 0x7fab18100010 2024-04-22T23:44:12.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:12.279+0000 7fab0e7fc700 1 -- 172.21.15.107:0/2819399928 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+107 (secure 0 0 0) 0x7fab08004490 con 0x7fab18100010 2024-04-22T23:44:12.271 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_WARN 1 filesystem is degraded 2024-04-22T23:44:12.271 INFO:teuthology.orchestra.run.smithi107.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-22T23:44:12.271 INFO:teuthology.orchestra.run.smithi107.stdout: fs cephfs is degraded 2024-04-22T23:44:12.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:12.281+0000 7fab1fd14700 1 -- 172.21.15.107:0/2819399928 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fab0406ec00 msgr2=0x7fab040710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:12.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:12.281+0000 7fab1fd14700 1 --2- 172.21.15.107:0/2819399928 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fab0406ec00 0x7fab040710b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fab14009b80 tx=0x7fab1400b040 comp rx=0 tx=0).stop 2024-04-22T23:44:12.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:12.281+0000 7fab1fd14700 1 -- 172.21.15.107:0/2819399928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab18100010 msgr2=0x7fab1810df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:12.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:12.281+0000 7fab1fd14700 1 --2- 172.21.15.107:0/2819399928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab18100010 0x7fab1810df30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fab08013870 tx=0x7fab08009e30 comp rx=0 tx=0).stop 2024-04-22T23:44:12.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:12.281+0000 7fab1fd14700 1 -- 172.21.15.107:0/2819399928 shutdown_connections 2024-04-22T23:44:12.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:12.281+0000 7fab1fd14700 1 --2- 172.21.15.107:0/2819399928 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab18066250 0x7fab1810d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:12.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:12.281+0000 7fab1fd14700 1 --2- 172.21.15.107:0/2819399928 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fab0406ec00 0x7fab040710b0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:12.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:12.281+0000 7fab1fd14700 1 --2- 172.21.15.107:0/2819399928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab18100010 0x7fab1810df30 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:12.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:12.281+0000 7fab1fd14700 1 -- 172.21.15.107:0/2819399928 >> 172.21.15.107:0/2819399928 conn(0x7fab180f74c0 msgr2=0x7fab180f7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:44:12.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:12.281+0000 7fab1fd14700 1 -- 172.21.15.107:0/2819399928 shutdown_connections 2024-04-22T23:44:12.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:12.281+0000 7fab1fd14700 1 -- 172.21.15.107:0/2819399928 wait complete. 2024-04-22T23:44:13.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:12 smithi157 ceph-mon[79403]: pgmap v225: 65 pgs: 65 active+clean; 165 MiB data, 728 MiB used, 536 GiB / 536 GiB avail; 6.4 MiB/s rd, 2 op/s 2024-04-22T23:44:13.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:12 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/2819399928' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:44:13.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:12 smithi107 ceph-mon[90179]: pgmap v225: 65 pgs: 65 active+clean; 165 MiB data, 728 MiB used, 536 GiB / 536 GiB avail; 6.4 MiB/s rd, 2 op/s 2024-04-22T23:44:13.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:12 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/2819399928' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:44:14.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:13 smithi157 ceph-mon[79403]: from='client.34492 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:44:14.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:13 smithi107 ceph-mon[90179]: from='client.34492 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:44:15.052 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:14 smithi157 ceph-mon[79403]: pgmap v226: 65 pgs: 65 active+clean; 165 MiB data, 728 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 4 op/s 2024-04-22T23:44:15.132 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:14 smithi107 ceph-mon[90179]: pgmap v226: 65 pgs: 65 active+clean; 165 MiB data, 728 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 4 op/s 2024-04-22T23:44:16.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:15 smithi157 ceph-mon[79403]: reconnect by client.14522 v1:192.168.0.1:0/3769680709 after 0.000999984 2024-04-22T23:44:16.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:15 smithi157 ceph-mon[79403]: reconnect by client.14528 v1:172.21.15.157:0/2901108260 after 0.000999984 2024-04-22T23:44:16.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:15 smithi157 ceph-mon[79403]: mds.? [v2:172.21.15.107:6826/418145840,v1:172.21.15.107:6827/418145840] up:reconnect 2024-04-22T23:44:16.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:15 smithi157 ceph-mon[79403]: fsmap cephfs:1/1 {0=cephfs.smithi107.dfvvkw=up:reconnect} 2 up:standby 2024-04-22T23:44:16.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:15 smithi107 ceph-mon[90179]: reconnect by client.14522 v1:192.168.0.1:0/3769680709 after 0.000999984 2024-04-22T23:44:16.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:15 smithi107 ceph-mon[90179]: reconnect by client.14528 v1:172.21.15.157:0/2901108260 after 0.000999984 2024-04-22T23:44:16.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:15 smithi107 ceph-mon[90179]: mds.? [v2:172.21.15.107:6826/418145840,v1:172.21.15.107:6827/418145840] up:reconnect 2024-04-22T23:44:16.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:15 smithi107 ceph-mon[90179]: fsmap cephfs:1/1 {0=cephfs.smithi107.dfvvkw=up:reconnect} 2 up:standby 2024-04-22T23:44:17.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:16 smithi157 ceph-mon[79403]: pgmap v227: 65 pgs: 65 active+clean; 165 MiB data, 728 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-04-22T23:44:17.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:16 smithi157 ceph-mon[79403]: mds.? [v2:172.21.15.107:6826/418145840,v1:172.21.15.107:6827/418145840] up:rejoin 2024-04-22T23:44:17.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:16 smithi157 ceph-mon[79403]: fsmap cephfs:1/1 {0=cephfs.smithi107.dfvvkw=up:rejoin} 2 up:standby 2024-04-22T23:44:17.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:16 smithi157 ceph-mon[79403]: daemon mds.cephfs.smithi107.dfvvkw is now active in filesystem cephfs as rank 0 2024-04-22T23:44:17.201 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:16 smithi107 ceph-mon[90179]: pgmap v227: 65 pgs: 65 active+clean; 165 MiB data, 728 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-04-22T23:44:17.201 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:16 smithi107 ceph-mon[90179]: mds.? [v2:172.21.15.107:6826/418145840,v1:172.21.15.107:6827/418145840] up:rejoin 2024-04-22T23:44:17.201 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:16 smithi107 ceph-mon[90179]: fsmap cephfs:1/1 {0=cephfs.smithi107.dfvvkw=up:rejoin} 2 up:standby 2024-04-22T23:44:17.201 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:16 smithi107 ceph-mon[90179]: daemon mds.cephfs.smithi107.dfvvkw is now active in filesystem cephfs as rank 0 2024-04-22T23:44:18.097 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:17 smithi157 ceph-mon[79403]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-22T23:44:18.097 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:17 smithi157 ceph-mon[79403]: Cluster is now healthy 2024-04-22T23:44:18.097 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:17 smithi157 ceph-mon[79403]: mds.? [v2:172.21.15.107:6826/418145840,v1:172.21.15.107:6827/418145840] up:active 2024-04-22T23:44:18.097 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:17 smithi157 ceph-mon[79403]: fsmap cephfs:1 {0=cephfs.smithi107.dfvvkw=up:active} 2 up:standby 2024-04-22T23:44:18.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:17 smithi107 ceph-mon[90179]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-22T23:44:18.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:17 smithi107 ceph-mon[90179]: Cluster is now healthy 2024-04-22T23:44:18.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:17 smithi107 ceph-mon[90179]: mds.? [v2:172.21.15.107:6826/418145840,v1:172.21.15.107:6827/418145840] up:active 2024-04-22T23:44:18.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:17 smithi107 ceph-mon[90179]: fsmap cephfs:1 {0=cephfs.smithi107.dfvvkw=up:active} 2 up:standby 2024-04-22T23:44:19.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:18 smithi157 ceph-mon[79403]: pgmap v228: 65 pgs: 65 active+clean; 165 MiB data, 728 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 194 B/s wr, 6 op/s 2024-04-22T23:44:19.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:18 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:19.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:18 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:19.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:18 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:44:19.133 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:18 smithi107 ceph-mon[90179]: pgmap v228: 65 pgs: 65 active+clean; 165 MiB data, 728 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 194 B/s wr, 6 op/s 2024-04-22T23:44:19.133 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:18 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:19.133 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:18 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:19.133 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:18 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:44:20.629 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:20 smithi157 ceph-mon[79403]: mds.? [v2:172.21.15.157:6826/3167053094,v1:172.21.15.157:6827/3167053094] up:boot 2024-04-22T23:44:20.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:20 smithi157 ceph-mon[79403]: fsmap cephfs:1 {0=cephfs.smithi107.dfvvkw=up:active} 3 up:standby 2024-04-22T23:44:20.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:20 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi157.ufywse"}]: dispatch 2024-04-22T23:44:20.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:20 smithi157 ceph-mon[79403]: pgmap v229: 65 pgs: 65 active+clean; 165 MiB data, 732 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 170 B/s wr, 5 op/s 2024-04-22T23:44:20.731 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:20 smithi107 ceph-mon[90179]: mds.? [v2:172.21.15.157:6826/3167053094,v1:172.21.15.157:6827/3167053094] up:boot 2024-04-22T23:44:20.731 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:20 smithi107 ceph-mon[90179]: fsmap cephfs:1 {0=cephfs.smithi107.dfvvkw=up:active} 3 up:standby 2024-04-22T23:44:20.731 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:20 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi157.ufywse"}]: dispatch 2024-04-22T23:44:20.731 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:20 smithi107 ceph-mon[90179]: pgmap v229: 65 pgs: 65 active+clean; 165 MiB data, 732 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 170 B/s wr, 5 op/s 2024-04-22T23:44:22.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:21 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:22.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:21 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:22.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:21 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:22.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:21 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:23.059 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:22 smithi157 ceph-mon[79403]: pgmap v230: 65 pgs: 65 active+clean; 165 MiB data, 732 MiB used, 536 GiB / 536 GiB avail; 11 MiB/s rd, 170 B/s wr, 6 op/s 2024-04-22T23:44:23.059 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:22 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:23.059 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:22 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:23.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:22 smithi107 ceph-mon[90179]: pgmap v230: 65 pgs: 65 active+clean; 165 MiB data, 732 MiB used, 536 GiB / 536 GiB avail; 11 MiB/s rd, 170 B/s wr, 6 op/s 2024-04-22T23:44:23.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:22 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:23.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:22 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:25.072 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:24 smithi157 ceph-mon[79403]: pgmap v231: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail; 8.0 MiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-04-22T23:44:25.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:24 smithi107 ceph-mon[90179]: pgmap v231: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail; 8.0 MiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-04-22T23:44:26.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:26 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:26.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:26 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:26.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:26 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:44:26.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:26 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:44:26.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:26 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:26.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:26 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:44:26.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:26 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:26.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:26 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:26.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:26 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:26.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:26 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:26.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:26 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:26.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:26 smithi157 ceph-mon[79403]: Upgrade: Setting container_image for all mds 2024-04-22T23:44:26.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:26 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:26.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:26 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi107.dfvvkw"}]: dispatch 2024-04-22T23:44:26.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:26 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi107.dfvvkw"}]': finished 2024-04-22T23:44:26.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:26 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi107.ktuxjl"}]: dispatch 2024-04-22T23:44:26.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:26 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi107.ktuxjl"}]': finished 2024-04-22T23:44:26.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:26 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi157.rcaris"}]: dispatch 2024-04-22T23:44:26.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:26 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi157.rcaris"}]': finished 2024-04-22T23:44:26.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:26 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi157.ufywse"}]: dispatch 2024-04-22T23:44:26.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:26 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi157.ufywse"}]': finished 2024-04-22T23:44:26.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:26 smithi157 ceph-mon[79403]: Upgrade: Scaling up filesystem cephfs max_mds to 2 2024-04-22T23:44:26.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:26 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-04-22T23:44:26.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:26 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:26.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:26 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:44:26.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:26 smithi157 ceph-mon[79403]: pgmap v232: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-04-22T23:44:26.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:26 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:26.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:26 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:26.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:26 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:44:26.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:26 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:44:26.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:26 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:26.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:26 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:44:26.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:26 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:26.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:26 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:26.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:26 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:26.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:26 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:26.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:26 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:26.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:26 smithi107 ceph-mon[90179]: Upgrade: Setting container_image for all mds 2024-04-22T23:44:26.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:26 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:26.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:26 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi107.dfvvkw"}]: dispatch 2024-04-22T23:44:26.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:26 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi107.dfvvkw"}]': finished 2024-04-22T23:44:26.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:26 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi107.ktuxjl"}]: dispatch 2024-04-22T23:44:26.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:26 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi107.ktuxjl"}]': finished 2024-04-22T23:44:26.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:26 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi157.rcaris"}]: dispatch 2024-04-22T23:44:26.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:26 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi157.rcaris"}]': finished 2024-04-22T23:44:26.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:26 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi157.ufywse"}]: dispatch 2024-04-22T23:44:26.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:26 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi157.ufywse"}]': finished 2024-04-22T23:44:26.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:26 smithi107 ceph-mon[90179]: Upgrade: Scaling up filesystem cephfs max_mds to 2 2024-04-22T23:44:26.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:26 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-04-22T23:44:26.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:26 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:26.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:26 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:44:26.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:26 smithi107 ceph-mon[90179]: pgmap v232: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-04-22T23:44:27.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:27 smithi107 ceph-mon[90179]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-22T23:44:27.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:27 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-04-22T23:44:27.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:27 smithi107 ceph-mon[90179]: daemon mds.cephfs.smithi107.ktuxjl assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-22T23:44:27.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:27 smithi107 ceph-mon[90179]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-22T23:44:27.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:27 smithi107 ceph-mon[90179]: Cluster is now healthy 2024-04-22T23:44:27.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:27 smithi107 ceph-mon[90179]: fsmap cephfs:1 {0=cephfs.smithi107.dfvvkw=up:active} 3 up:standby 2024-04-22T23:44:27.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:27 smithi107 ceph-mon[90179]: fsmap cephfs:2 {0=cephfs.smithi107.dfvvkw=up:active,1=cephfs.smithi107.ktuxjl=up:starting} 2 up:standby 2024-04-22T23:44:27.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:27 smithi107 ceph-mon[90179]: daemon mds.cephfs.smithi107.ktuxjl is now active in filesystem cephfs as rank 1 2024-04-22T23:44:27.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:27 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:27.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:27 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:27.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:27 smithi107 ceph-mon[90179]: Upgrade: Setting container_image for all rgw 2024-04-22T23:44:27.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:27 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:27.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:27 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:27.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:27 smithi107 ceph-mon[90179]: Upgrade: Setting container_image for all rbd-mirror 2024-04-22T23:44:27.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:27 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:27.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:27 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:27.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:27 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:27.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:27 smithi107 ceph-mon[90179]: Upgrade: Setting container_image for all ceph-exporter 2024-04-22T23:44:27.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:27 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:27.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:27 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:27.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:27 smithi107 ceph-mon[90179]: Upgrade: Setting container_image for all iscsi 2024-04-22T23:44:27.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:27 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:27.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:27 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:27.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:27 smithi107 ceph-mon[90179]: Upgrade: Setting container_image for all nfs 2024-04-22T23:44:27.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:27 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:27.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:27 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:27.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:27 smithi107 ceph-mon[90179]: Upgrade: Setting container_image for all nvmeof 2024-04-22T23:44:27.734 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:27 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:27.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:27 smithi157 ceph-mon[79403]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-22T23:44:27.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:27 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-04-22T23:44:27.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:27 smithi157 ceph-mon[79403]: daemon mds.cephfs.smithi107.ktuxjl assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-22T23:44:27.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:27 smithi157 ceph-mon[79403]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-22T23:44:27.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:27 smithi157 ceph-mon[79403]: Cluster is now healthy 2024-04-22T23:44:27.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:27 smithi157 ceph-mon[79403]: fsmap cephfs:1 {0=cephfs.smithi107.dfvvkw=up:active} 3 up:standby 2024-04-22T23:44:27.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:27 smithi157 ceph-mon[79403]: fsmap cephfs:2 {0=cephfs.smithi107.dfvvkw=up:active,1=cephfs.smithi107.ktuxjl=up:starting} 2 up:standby 2024-04-22T23:44:27.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:27 smithi157 ceph-mon[79403]: daemon mds.cephfs.smithi107.ktuxjl is now active in filesystem cephfs as rank 1 2024-04-22T23:44:27.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:27 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:27.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:27 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:27.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:27 smithi157 ceph-mon[79403]: Upgrade: Setting container_image for all rgw 2024-04-22T23:44:27.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:27 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:27.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:27 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:27.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:27 smithi157 ceph-mon[79403]: Upgrade: Setting container_image for all rbd-mirror 2024-04-22T23:44:27.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:27 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:27.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:27 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:27.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:27 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:27.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:27 smithi157 ceph-mon[79403]: Upgrade: Setting container_image for all ceph-exporter 2024-04-22T23:44:27.882 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:27 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:27.886 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:27 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:27.886 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:27 smithi157 ceph-mon[79403]: Upgrade: Setting container_image for all iscsi 2024-04-22T23:44:27.886 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:27 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:27.886 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:27 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:27.886 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:27 smithi157 ceph-mon[79403]: Upgrade: Setting container_image for all nfs 2024-04-22T23:44:27.886 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:27 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:27.887 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:27 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:27.887 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:27 smithi157 ceph-mon[79403]: Upgrade: Setting container_image for all nvmeof 2024-04-22T23:44:27.887 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:27 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:28.981 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:28 smithi107 ceph-mon[90179]: mds.? [v2:172.21.15.107:6828/1256497807,v1:172.21.15.107:6829/1256497807] up:active 2024-04-22T23:44:28.981 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:28 smithi107 ceph-mon[90179]: fsmap cephfs:2 {0=cephfs.smithi107.dfvvkw=up:active,1=cephfs.smithi107.ktuxjl=up:active} 2 up:standby 2024-04-22T23:44:28.981 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:28 smithi107 ceph-mon[90179]: pgmap v233: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail; 3.3 KiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-04-22T23:44:29.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:28 smithi157 ceph-mon[79403]: mds.? [v2:172.21.15.107:6828/1256497807,v1:172.21.15.107:6829/1256497807] up:active 2024-04-22T23:44:29.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:28 smithi157 ceph-mon[79403]: fsmap cephfs:2 {0=cephfs.smithi107.dfvvkw=up:active,1=cephfs.smithi107.ktuxjl=up:active} 2 up:standby 2024-04-22T23:44:29.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:28 smithi157 ceph-mon[79403]: pgmap v233: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail; 3.3 KiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-04-22T23:44:29.807 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:29 smithi107 ceph-mon[90179]: Upgrade: Updating node-exporter.smithi107 (1/2) 2024-04-22T23:44:29.807 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:29 smithi107 ceph-mon[90179]: Deploying daemon node-exporter.smithi107 on smithi107 2024-04-22T23:44:30.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:29 smithi157 ceph-mon[79403]: Upgrade: Updating node-exporter.smithi107 (1/2) 2024-04-22T23:44:30.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:29 smithi157 ceph-mon[79403]: Deploying daemon node-exporter.smithi107 on smithi107 2024-04-22T23:44:30.860 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:30 smithi107 ceph-mon[90179]: pgmap v234: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail; 4.9 KiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-04-22T23:44:31.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:30 smithi157 ceph-mon[79403]: pgmap v234: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail; 4.9 KiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-04-22T23:44:32.342 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:32 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:32.342 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:32 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:32.342 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:32 smithi157 ceph-mon[79403]: pgmap v235: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail; 5.4 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-04-22T23:44:32.481 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:32 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:32.481 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:32 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:32.481 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:32 smithi107 ceph-mon[90179]: pgmap v235: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail; 5.4 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-04-22T23:44:33.380 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:33 smithi157 ceph-mon[79403]: Upgrade: Updating node-exporter.smithi157 (2/2) 2024-04-22T23:44:33.380 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:33 smithi157 ceph-mon[79403]: Deploying daemon node-exporter.smithi157 on smithi157 2024-04-22T23:44:33.481 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:33 smithi107 ceph-mon[90179]: Upgrade: Updating node-exporter.smithi157 (2/2) 2024-04-22T23:44:33.481 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:33 smithi107 ceph-mon[90179]: Deploying daemon node-exporter.smithi157 on smithi157 2024-04-22T23:44:34.380 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:34 smithi157 ceph-mon[79403]: pgmap v236: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail; 6.2 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-04-22T23:44:34.481 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:34 smithi107 ceph-mon[90179]: pgmap v236: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail; 6.2 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-04-22T23:44:36.609 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:36 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:36.609 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:36 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:36.609 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:36 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:44:36.609 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:36 smithi107 ceph-mon[90179]: pgmap v237: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail; 6.2 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-22T23:44:36.629 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:36 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:36.629 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:36 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:36.629 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:36 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:44:36.629 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:36 smithi157 ceph-mon[79403]: pgmap v237: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail; 6.2 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-22T23:44:38.981 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:38 smithi107 ceph-mon[90179]: pgmap v238: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail; 6.2 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-22T23:44:38.981 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:38 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:38.981 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:38 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:39.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:38 smithi157 ceph-mon[79403]: pgmap v238: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail; 6.2 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-22T23:44:39.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:38 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:39.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:38 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:39.981 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:39 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:39.981 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:39 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:39.981 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:39 smithi107 ceph-mon[90179]: pgmap v239: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail; 6.1 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-22T23:44:40.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:39 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:40.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:39 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:40.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:39 smithi157 ceph-mon[79403]: pgmap v239: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail; 6.1 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-22T23:44:41.346 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:41 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:41.346 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:41 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:41.346 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:41 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:44:41.481 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:41 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:41.481 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:41 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:41.481 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:41 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:44:42.265 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:42 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:42.265 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:42 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:42.265 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:42 smithi157 ceph-mon[79403]: pgmap v240: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail; 3.7 KiB/s rd, 170 B/s wr, 4 op/s 2024-04-22T23:44:42.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.386+0000 7f53073ed700 1 -- 172.21.15.107:0/4064310255 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53000fcff0 msgr2=0x7f53000fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:42.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.386+0000 7f53073ed700 1 --2- 172.21.15.107:0/4064310255 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53000fcff0 0x7f53000fd460 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f52f0004940 tx=0x7f52f0009e30 comp rx=0 tx=0).stop 2024-04-22T23:44:42.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.386+0000 7f53073ed700 1 -- 172.21.15.107:0/4064310255 shutdown_connections 2024-04-22T23:44:42.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.386+0000 7f53073ed700 1 --2- 172.21.15.107:0/4064310255 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53000fcff0 0x7f53000fd460 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:42.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.386+0000 7f53073ed700 1 --2- 172.21.15.107:0/4064310255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53000fbd40 0x7f53000fc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:42.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.386+0000 7f53073ed700 1 -- 172.21.15.107:0/4064310255 >> 172.21.15.107:0/4064310255 conn(0x7f53000f74c0 msgr2=0x7f53000f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:44:42.382 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:42 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:42.382 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:42 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:42.382 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:42 smithi107 ceph-mon[90179]: pgmap v240: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail; 3.7 KiB/s rd, 170 B/s wr, 4 op/s 2024-04-22T23:44:42.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.386+0000 7f53073ed700 1 -- 172.21.15.107:0/4064310255 shutdown_connections 2024-04-22T23:44:42.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.386+0000 7f53073ed700 1 -- 172.21.15.107:0/4064310255 wait complete. 2024-04-22T23:44:42.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.387+0000 7f53073ed700 1 Processor -- start 2024-04-22T23:44:42.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.387+0000 7f53073ed700 1 -- start start 2024-04-22T23:44:42.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.387+0000 7f53073ed700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53000fbd40 0x7f53001106e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:42.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.387+0000 7f53073ed700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53000fcff0 0x7f5300112c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:42.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.387+0000 7f53073ed700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5300066330 con 0x7f53000fbd40 2024-04-22T23:44:42.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.387+0000 7f53073ed700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5300066470 con 0x7f53000fcff0 2024-04-22T23:44:42.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.387+0000 7f5305189700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53000fbd40 0x7f53001106e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:42.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.387+0000 7f5305189700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53000fbd40 0x7f53001106e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34086/0 (socket says 172.21.15.107:34086) 2024-04-22T23:44:42.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.387+0000 7f5305189700 1 -- 172.21.15.107:0/2749125094 learned_addr learned my addr 172.21.15.107:0/2749125094 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:44:42.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.387+0000 7f5304988700 1 --2- 172.21.15.107:0/2749125094 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53000fcff0 0x7f5300112c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:42.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.387+0000 7f5305189700 1 -- 172.21.15.107:0/2749125094 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53000fcff0 msgr2=0x7f5300112c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:42.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.387+0000 7f5305189700 1 --2- 172.21.15.107:0/2749125094 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53000fcff0 0x7f5300112c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:42.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.387+0000 7f5305189700 1 -- 172.21.15.107:0/2749125094 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f0005040 con 0x7f53000fbd40 2024-04-22T23:44:42.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.387+0000 7f5304988700 1 --2- 172.21.15.107:0/2749125094 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53000fcff0 0x7f5300112c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T23:44:42.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.388+0000 7f5305189700 1 --2- 172.21.15.107:0/2749125094 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53000fbd40 0x7f53001106e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f52fc00eb40 tx=0x7f52fc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:44:42.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.388+0000 7f52f5ffb700 1 -- 172.21.15.107:0/2749125094 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52fc007cc0 con 0x7f53000fbd40 2024-04-22T23:44:42.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.388+0000 7f52f5ffb700 1 -- 172.21.15.107:0/2749125094 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f52fc00f040 con 0x7f53000fbd40 2024-04-22T23:44:42.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.388+0000 7f53073ed700 1 -- 172.21.15.107:0/2749125094 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5300113170 con 0x7f53000fbd40 2024-04-22T23:44:42.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.388+0000 7f52f5ffb700 1 -- 172.21.15.107:0/2749125094 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52fc009620 con 0x7f53000fbd40 2024-04-22T23:44:42.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.388+0000 7f53073ed700 1 -- 172.21.15.107:0/2749125094 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5300113680 con 0x7f53000fbd40 2024-04-22T23:44:42.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.389+0000 7f53073ed700 1 -- 172.21.15.107:0/2749125094 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f530005f5f0 con 0x7f53000fbd40 2024-04-22T23:44:42.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.389+0000 7f52f5ffb700 1 -- 172.21.15.107:0/2749125094 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f52fc009780 con 0x7f53000fbd40 2024-04-22T23:44:42.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.390+0000 7f52f5ffb700 1 --2- 172.21.15.107:0/2749125094 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f52ec06ec00 0x7f52ec0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:42.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.390+0000 7f52f5ffb700 1 -- 172.21.15.107:0/2749125094 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f52fc08e5f0 con 0x7f53000fbd40 2024-04-22T23:44:42.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.390+0000 7f5304988700 1 --2- 172.21.15.107:0/2749125094 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f52ec06ec00 0x7f52ec0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:42.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.390+0000 7f5304988700 1 --2- 172.21.15.107:0/2749125094 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f52ec06ec00 0x7f52ec0710b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f52f0000c00 tx=0x7f52f0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:44:42.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.395+0000 7f52f5ffb700 1 -- 172.21.15.107:0/2749125094 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f52fc0573a0 con 0x7f53000fbd40 2024-04-22T23:44:42.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.615+0000 7f53073ed700 1 -- 172.21.15.107:0/2749125094 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f53001012e0 con 0x7f52ec06ec00 2024-04-22T23:44:42.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.617+0000 7f52f5ffb700 1 -- 172.21.15.107:0/2749125094 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7f53001012e0 con 0x7f52ec06ec00 2024-04-22T23:44:42.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.619+0000 7f53073ed700 1 -- 172.21.15.107:0/2749125094 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f52ec06ec00 msgr2=0x7f52ec0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:42.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.619+0000 7f53073ed700 1 --2- 172.21.15.107:0/2749125094 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f52ec06ec00 0x7f52ec0710b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f52f0000c00 tx=0x7f52f0009e30 comp rx=0 tx=0).stop 2024-04-22T23:44:42.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.619+0000 7f53073ed700 1 -- 172.21.15.107:0/2749125094 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53000fbd40 msgr2=0x7f53001106e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:42.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.619+0000 7f53073ed700 1 --2- 172.21.15.107:0/2749125094 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53000fbd40 0x7f53001106e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f52fc00eb40 tx=0x7f52fc00b040 comp rx=0 tx=0).stop 2024-04-22T23:44:42.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.619+0000 7f53073ed700 1 -- 172.21.15.107:0/2749125094 shutdown_connections 2024-04-22T23:44:42.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.619+0000 7f53073ed700 1 --2- 172.21.15.107:0/2749125094 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f52ec06ec00 0x7f52ec0710b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:42.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.619+0000 7f53073ed700 1 --2- 172.21.15.107:0/2749125094 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53000fbd40 0x7f53001106e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:42.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.619+0000 7f53073ed700 1 --2- 172.21.15.107:0/2749125094 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53000fcff0 0x7f5300112c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:42.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.619+0000 7f53073ed700 1 -- 172.21.15.107:0/2749125094 >> 172.21.15.107:0/2749125094 conn(0x7f53000f74c0 msgr2=0x7f53000f8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:44:42.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.619+0000 7f53073ed700 1 -- 172.21.15.107:0/2749125094 shutdown_connections 2024-04-22T23:44:42.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.619+0000 7f53073ed700 1 -- 172.21.15.107:0/2749125094 wait complete. 2024-04-22T23:44:42.622 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-04-22T23:44:42.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.720+0000 7f9bcf925700 1 -- 172.21.15.107:0/431908415 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bc80fdff0 msgr2=0x7f9bc80fe460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:42.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.720+0000 7f9bcf925700 1 --2- 172.21.15.107:0/431908415 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bc80fdff0 0x7f9bc80fe460 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8004940 tx=0x7f9bb8009e30 comp rx=0 tx=0).stop 2024-04-22T23:44:42.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.720+0000 7f9bcf925700 1 -- 172.21.15.107:0/431908415 shutdown_connections 2024-04-22T23:44:42.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.720+0000 7f9bcf925700 1 --2- 172.21.15.107:0/431908415 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bc80fdff0 0x7f9bc80fe460 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:42.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.720+0000 7f9bcf925700 1 --2- 172.21.15.107:0/431908415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bc80fce80 0x7f9bc80fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:42.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.720+0000 7f9bcf925700 1 -- 172.21.15.107:0/431908415 >> 172.21.15.107:0/431908415 conn(0x7f9bc80f8580 msgr2=0x7f9bc80fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:44:42.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.721+0000 7f9bcf925700 1 -- 172.21.15.107:0/431908415 shutdown_connections 2024-04-22T23:44:42.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.721+0000 7f9bcf925700 1 -- 172.21.15.107:0/431908415 wait complete. 2024-04-22T23:44:42.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.721+0000 7f9bcf925700 1 Processor -- start 2024-04-22T23:44:42.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.721+0000 7f9bcf925700 1 -- start start 2024-04-22T23:44:42.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.721+0000 7f9bcf925700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bc80fce80 0x7f9bc8071c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:42.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.721+0000 7f9bcf925700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bc80fdff0 0x7f9bc8070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:42.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.721+0000 7f9bcf925700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc8072260 con 0x7f9bc80fce80 2024-04-22T23:44:42.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.721+0000 7f9bcf925700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc8070850 con 0x7f9bc80fdff0 2024-04-22T23:44:42.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.721+0000 7f9bcd6c1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bc80fce80 0x7f9bc8071c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:42.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.721+0000 7f9bcd6c1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bc80fce80 0x7f9bc8071c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34102/0 (socket says 172.21.15.107:34102) 2024-04-22T23:44:42.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.721+0000 7f9bcd6c1700 1 -- 172.21.15.107:0/3311100833 learned_addr learned my addr 172.21.15.107:0/3311100833 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:44:42.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.722+0000 7f9bccec0700 1 --2- 172.21.15.107:0/3311100833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bc80fdff0 0x7f9bc8070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:42.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.722+0000 7f9bcd6c1700 1 -- 172.21.15.107:0/3311100833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bc80fdff0 msgr2=0x7f9bc8070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:42.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.722+0000 7f9bcd6c1700 1 --2- 172.21.15.107:0/3311100833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bc80fdff0 0x7f9bc8070310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:42.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.722+0000 7f9bcd6c1700 1 -- 172.21.15.107:0/3311100833 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb8005040 con 0x7f9bc80fce80 2024-04-22T23:44:42.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.722+0000 7f9bcd6c1700 1 --2- 172.21.15.107:0/3311100833 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bc80fce80 0x7f9bc8071c60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9bc400eb40 tx=0x7f9bc400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:44:42.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.722+0000 7f9bbdffb700 1 -- 172.21.15.107:0/3311100833 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9bc4007cc0 con 0x7f9bc80fce80 2024-04-22T23:44:42.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.722+0000 7f9bbdffb700 1 -- 172.21.15.107:0/3311100833 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9bc400f040 con 0x7f9bc80fce80 2024-04-22T23:44:42.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.722+0000 7f9bbdffb700 1 -- 172.21.15.107:0/3311100833 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9bc400ca10 con 0x7f9bc80fce80 2024-04-22T23:44:42.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.722+0000 7f9bcf925700 1 -- 172.21.15.107:0/3311100833 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bc8070aa0 con 0x7f9bc80fce80 2024-04-22T23:44:42.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.722+0000 7f9bcf925700 1 -- 172.21.15.107:0/3311100833 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bc8070fe0 con 0x7f9bc80fce80 2024-04-22T23:44:42.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.723+0000 7f9bcf925700 1 -- 172.21.15.107:0/3311100833 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bc804c710 con 0x7f9bc80fce80 2024-04-22T23:44:42.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.724+0000 7f9bbdffb700 1 -- 172.21.15.107:0/3311100833 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f9bc400cb70 con 0x7f9bc80fce80 2024-04-22T23:44:42.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.724+0000 7f9bbdffb700 1 --2- 172.21.15.107:0/3311100833 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f9bb406ec00 0x7f9bb40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:42.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.724+0000 7f9bbdffb700 1 -- 172.21.15.107:0/3311100833 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f9bc408e420 con 0x7f9bc80fce80 2024-04-22T23:44:42.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.724+0000 7f9bccec0700 1 --2- 172.21.15.107:0/3311100833 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f9bb406ec00 0x7f9bb40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:42.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.724+0000 7f9bccec0700 1 --2- 172.21.15.107:0/3311100833 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f9bb406ec00 0x7f9bb40710b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8009b30 tx=0x7f9bb8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:44:42.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.729+0000 7f9bbdffb700 1 -- 172.21.15.107:0/3311100833 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f9bc405ad80 con 0x7f9bc80fce80 2024-04-22T23:44:42.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.959+0000 7f9bcf925700 1 -- 172.21.15.107:0/3311100833 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9bc81022c0 con 0x7f9bb406ec00 2024-04-22T23:44:42.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.960+0000 7f9bbdffb700 1 -- 172.21.15.107:0/3311100833 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7f9bc81022c0 con 0x7f9bb406ec00 2024-04-22T23:44:42.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.963+0000 7f9bcf925700 1 -- 172.21.15.107:0/3311100833 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f9bb406ec00 msgr2=0x7f9bb40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:42.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.963+0000 7f9bcf925700 1 --2- 172.21.15.107:0/3311100833 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f9bb406ec00 0x7f9bb40710b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8009b30 tx=0x7f9bb8009e30 comp rx=0 tx=0).stop 2024-04-22T23:44:42.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.963+0000 7f9bcf925700 1 -- 172.21.15.107:0/3311100833 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bc80fce80 msgr2=0x7f9bc8071c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:42.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.963+0000 7f9bcf925700 1 --2- 172.21.15.107:0/3311100833 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bc80fce80 0x7f9bc8071c60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9bc400eb40 tx=0x7f9bc400b040 comp rx=0 tx=0).stop 2024-04-22T23:44:42.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.963+0000 7f9bcf925700 1 -- 172.21.15.107:0/3311100833 shutdown_connections 2024-04-22T23:44:42.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.963+0000 7f9bcf925700 1 --2- 172.21.15.107:0/3311100833 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f9bb406ec00 0x7f9bb40710b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:42.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.963+0000 7f9bcf925700 1 --2- 172.21.15.107:0/3311100833 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bc80fce80 0x7f9bc8071c60 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:42.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.963+0000 7f9bcf925700 1 --2- 172.21.15.107:0/3311100833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bc80fdff0 0x7f9bc8070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:42.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.963+0000 7f9bcf925700 1 -- 172.21.15.107:0/3311100833 >> 172.21.15.107:0/3311100833 conn(0x7f9bc80f8580 msgr2=0x7f9bc80f9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:44:42.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.963+0000 7f9bcf925700 1 -- 172.21.15.107:0/3311100833 shutdown_connections 2024-04-22T23:44:42.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:42.963+0000 7f9bcf925700 1 -- 172.21.15.107:0/3311100833 wait complete. 2024-04-22T23:44:43.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.067+0000 7f931a13b700 1 -- 172.21.15.107:0/2193485258 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93140fce80 msgr2=0x7f93140fd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:43.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.067+0000 7f931a13b700 1 --2- 172.21.15.107:0/2193485258 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93140fce80 0x7f93140fd290 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9304004940 tx=0x7f9304009e30 comp rx=0 tx=0).stop 2024-04-22T23:44:43.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.067+0000 7f931a13b700 1 -- 172.21.15.107:0/2193485258 shutdown_connections 2024-04-22T23:44:43.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.067+0000 7f931a13b700 1 --2- 172.21.15.107:0/2193485258 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93140fdff0 0x7f93140fe460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:43.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.067+0000 7f931a13b700 1 --2- 172.21.15.107:0/2193485258 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93140fce80 0x7f93140fd290 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:43.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.067+0000 7f931a13b700 1 -- 172.21.15.107:0/2193485258 >> 172.21.15.107:0/2193485258 conn(0x7f93140f8540 msgr2=0x7f93140fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:44:43.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.068+0000 7f931a13b700 1 -- 172.21.15.107:0/2193485258 shutdown_connections 2024-04-22T23:44:43.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.068+0000 7f931a13b700 1 -- 172.21.15.107:0/2193485258 wait complete. 2024-04-22T23:44:43.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.068+0000 7f931a13b700 1 Processor -- start 2024-04-22T23:44:43.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.068+0000 7f931a13b700 1 -- start start 2024-04-22T23:44:43.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.068+0000 7f931a13b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93140fce80 0x7f931410d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:43.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.068+0000 7f931a13b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93140fdff0 0x7f931410de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:43.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.068+0000 7f931a13b700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931410e450 con 0x7f93140fdff0 2024-04-22T23:44:43.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.068+0000 7f931a13b700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931410e5c0 con 0x7f93140fce80 2024-04-22T23:44:43.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.068+0000 7f930bfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93140fdff0 0x7f931410de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:43.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.069+0000 7f930bfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93140fdff0 0x7f931410de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34122/0 (socket says 172.21.15.107:34122) 2024-04-22T23:44:43.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.069+0000 7f930bfff700 1 -- 172.21.15.107:0/3366209941 learned_addr learned my addr 172.21.15.107:0/3366209941 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:44:43.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.069+0000 7f93137fe700 1 --2- 172.21.15.107:0/3366209941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93140fce80 0x7f931410d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:43.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.069+0000 7f930bfff700 1 -- 172.21.15.107:0/3366209941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93140fce80 msgr2=0x7f931410d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:43.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.069+0000 7f930bfff700 1 --2- 172.21.15.107:0/3366209941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93140fce80 0x7f931410d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:43.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.069+0000 7f930bfff700 1 -- 172.21.15.107:0/3366209941 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9304005040 con 0x7f93140fdff0 2024-04-22T23:44:43.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.069+0000 7f930bfff700 1 --2- 172.21.15.107:0/3366209941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93140fdff0 0x7f931410de50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f92fc00eb40 tx=0x7f92fc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:44:43.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.069+0000 7f9310ff9700 1 -- 172.21.15.107:0/3366209941 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92fc007cc0 con 0x7f93140fdff0 2024-04-22T23:44:43.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.069+0000 7f9310ff9700 1 -- 172.21.15.107:0/3366209941 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f92fc00f040 con 0x7f93140fdff0 2024-04-22T23:44:43.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.069+0000 7f931a13b700 1 -- 172.21.15.107:0/3366209941 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9314112fc0 con 0x7f93140fdff0 2024-04-22T23:44:43.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.069+0000 7f9310ff9700 1 -- 172.21.15.107:0/3366209941 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92fc00ca10 con 0x7f93140fdff0 2024-04-22T23:44:43.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.069+0000 7f931a13b700 1 -- 172.21.15.107:0/3366209941 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93141134d0 con 0x7f93140fdff0 2024-04-22T23:44:43.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.070+0000 7f931a13b700 1 -- 172.21.15.107:0/3366209941 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f931404c710 con 0x7f93140fdff0 2024-04-22T23:44:43.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.071+0000 7f9310ff9700 1 -- 172.21.15.107:0/3366209941 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f92fc00cb70 con 0x7f93140fdff0 2024-04-22T23:44:43.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.071+0000 7f9310ff9700 1 --2- 172.21.15.107:0/3366209941 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f930006ec00 0x7f93000710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:43.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.071+0000 7f9310ff9700 1 -- 172.21.15.107:0/3366209941 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f92fc08e450 con 0x7f93140fdff0 2024-04-22T23:44:43.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.071+0000 7f93137fe700 1 --2- 172.21.15.107:0/3366209941 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f930006ec00 0x7f93000710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:43.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.072+0000 7f93137fe700 1 --2- 172.21.15.107:0/3366209941 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f930006ec00 0x7f93000710b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9304009b30 tx=0x7f9304009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:44:43.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.079+0000 7f9310ff9700 1 -- 172.21.15.107:0/3366209941 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f92fc05adb0 con 0x7f93140fdff0 2024-04-22T23:44:43.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.306+0000 7f931a13b700 1 -- 172.21.15.107:0/3366209941 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f93141022e0 con 0x7f930006ec00 2024-04-22T23:44:43.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.316+0000 7f9310ff9700 1 -- 172.21.15.107:0/3366209941 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f93141022e0 con 0x7f930006ec00 2024-04-22T23:44:43.309 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-22T23:44:43.309 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (8m) 3s ago 17m 24.7M - 0.25.0 c8568f914cd2 ec1a5c6260de 2024-04-22T23:44:43.309 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (5m) 3s ago 17m 6656k - 19.0.0-3252-g43be0201 a0434699c1d7 e074fd2cbb9c 2024-04-22T23:44:43.309 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi157 smithi157 running (5m) 5s ago 16m 6651k - 19.0.0-3252-g43be0201 a0434699c1d7 abc2649dee22 2024-04-22T23:44:43.309 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (7m) 3s ago 17m 86.4M - 9.4.7 954c08fa6188 31449ff442d2 2024-04-22T23:44:43.309 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.dfvvkw smithi107 running (90s) 3s ago 12m 70.2M - 19.0.0-3252-g43be0201 a0434699c1d7 7566b5cc3d01 2024-04-22T23:44:43.309 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.ktuxjl smithi107 running (67s) 3s ago 12m 16.6M - 19.0.0-3252-g43be0201 a0434699c1d7 ce23545f0ae2 2024-04-22T23:44:43.310 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi157.rcaris smithi157 running (48s) 5s ago 12m 11.9M - 19.0.0-3252-g43be0201 a0434699c1d7 12550073a997 2024-04-22T23:44:43.310 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi157.ufywse smithi157 running (25s) 5s ago 12m 14.1M - 19.0.0-3252-g43be0201 a0434699c1d7 e9f97f60ed21 2024-04-22T23:44:43.310 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.uipxvm smithi107 *:8443,9283,8765 running (7m) 3s ago 19m 512M - 19.0.0-3252-g43be0201 a0434699c1d7 d20f05dfe4f2 2024-04-22T23:44:43.310 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi157.tuylis smithi157 *:8443,9283,8765 running (7m) 5s ago 16m 452M - 19.0.0-3252-g43be0201 a0434699c1d7 957e598c9001 2024-04-22T23:44:43.310 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (6m) 3s ago 19m 58.5M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 bc72bbbe2c39 2024-04-22T23:44:43.310 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi157 smithi157 running (6m) 5s ago 16m 41.2M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 ea88de8be4fe 2024-04-22T23:44:43.310 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (12s) 3s ago 17m 5452k - 1.5.0 0da6a335fe13 c338e5470944 2024-04-22T23:44:43.310 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi157 smithi157 *:9100 running (8s) 5s ago 16m 3389k - 1.5.0 0da6a335fe13 8d0255aff39c 2024-04-22T23:44:43.310 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (5m) 3s ago 15m 104M 1557M 19.0.0-3252-g43be0201 a0434699c1d7 fd0ba8ef0c27 2024-04-22T23:44:43.310 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (4m) 3s ago 15m 104M 1557M 19.0.0-3252-g43be0201 a0434699c1d7 77e33b6122e0 2024-04-22T23:44:43.310 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (4m) 3s ago 15m 98.9M 1557M 19.0.0-3252-g43be0201 a0434699c1d7 c307bd8cdcfc 2024-04-22T23:44:43.310 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi157 running (3m) 5s ago 14m 112M 2581M 19.0.0-3252-g43be0201 a0434699c1d7 e81ca519c831 2024-04-22T23:44:43.310 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi157 running (3m) 5s ago 14m 105M 2581M 19.0.0-3252-g43be0201 a0434699c1d7 d5b5a2e4781a 2024-04-22T23:44:43.311 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi157 running (2m) 5s ago 13m 97.1M 2581M 19.0.0-3252-g43be0201 a0434699c1d7 0ee423135fdb 2024-04-22T23:44:43.311 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (7m) 3s ago 17m 66.1M - 2.43.0 a07b618ecd1d 99a4ea8594da 2024-04-22T23:44:43.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.318+0000 7f931a13b700 1 -- 172.21.15.107:0/3366209941 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f930006ec00 msgr2=0x7f93000710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:43.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.318+0000 7f931a13b700 1 --2- 172.21.15.107:0/3366209941 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f930006ec00 0x7f93000710b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9304009b30 tx=0x7f9304009e30 comp rx=0 tx=0).stop 2024-04-22T23:44:43.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.318+0000 7f931a13b700 1 -- 172.21.15.107:0/3366209941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93140fdff0 msgr2=0x7f931410de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:43.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.318+0000 7f931a13b700 1 --2- 172.21.15.107:0/3366209941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93140fdff0 0x7f931410de50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f92fc00eb40 tx=0x7f92fc00b040 comp rx=0 tx=0).stop 2024-04-22T23:44:43.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.318+0000 7f931a13b700 1 -- 172.21.15.107:0/3366209941 shutdown_connections 2024-04-22T23:44:43.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.318+0000 7f931a13b700 1 --2- 172.21.15.107:0/3366209941 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93140fce80 0x7f931410d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:43.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.318+0000 7f931a13b700 1 --2- 172.21.15.107:0/3366209941 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f930006ec00 0x7f93000710b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:43.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.318+0000 7f931a13b700 1 --2- 172.21.15.107:0/3366209941 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93140fdff0 0x7f931410de50 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:43.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.318+0000 7f931a13b700 1 -- 172.21.15.107:0/3366209941 >> 172.21.15.107:0/3366209941 conn(0x7f93140f8540 msgr2=0x7f93140f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:44:43.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.319+0000 7f931a13b700 1 -- 172.21.15.107:0/3366209941 shutdown_connections 2024-04-22T23:44:43.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.319+0000 7f931a13b700 1 -- 172.21.15.107:0/3366209941 wait complete. 2024-04-22T23:44:43.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.421+0000 7f3a554b4700 1 -- 172.21.15.107:0/2070648777 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a50071e20 msgr2=0x7f3a50070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:43.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.421+0000 7f3a554b4700 1 --2- 172.21.15.107:0/2070648777 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a50071e20 0x7f3a50070330 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3a40004940 tx=0x7f3a40009e30 comp rx=0 tx=0).stop 2024-04-22T23:44:43.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.421+0000 7f3a554b4700 1 -- 172.21.15.107:0/2070648777 shutdown_connections 2024-04-22T23:44:43.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.421+0000 7f3a554b4700 1 --2- 172.21.15.107:0/2070648777 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a50070870 0x7f3a50070ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:43.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.421+0000 7f3a554b4700 1 --2- 172.21.15.107:0/2070648777 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a50071e20 0x7f3a50070330 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:43.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.421+0000 7f3a554b4700 1 -- 172.21.15.107:0/2070648777 >> 172.21.15.107:0/2070648777 conn(0x7f3a500f8580 msgr2=0x7f3a500fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:44:43.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.421+0000 7f3a554b4700 1 -- 172.21.15.107:0/2070648777 shutdown_connections 2024-04-22T23:44:43.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.421+0000 7f3a554b4700 1 -- 172.21.15.107:0/2070648777 wait complete. 2024-04-22T23:44:43.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.421+0000 7f3a554b4700 1 Processor -- start 2024-04-22T23:44:43.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.422+0000 7f3a554b4700 1 -- start start 2024-04-22T23:44:43.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.422+0000 7f3a554b4700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a50070870 0x7f3a5010fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:43.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.422+0000 7f3a554b4700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a50071e20 0x7f3a501100e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:43.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.422+0000 7f3a554b4700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a501106e0 con 0x7f3a50071e20 2024-04-22T23:44:43.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.422+0000 7f3a554b4700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a50110850 con 0x7f3a50070870 2024-04-22T23:44:43.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.422+0000 7f3a4e7fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a50071e20 0x7f3a501100e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:43.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.422+0000 7f3a4e7fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a50071e20 0x7f3a501100e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34154/0 (socket says 172.21.15.107:34154) 2024-04-22T23:44:43.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.422+0000 7f3a4e7fc700 1 -- 172.21.15.107:0/4265284703 learned_addr learned my addr 172.21.15.107:0/4265284703 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:44:43.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.422+0000 7f3a4effd700 1 --2- 172.21.15.107:0/4265284703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a50070870 0x7f3a5010fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:43.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.422+0000 7f3a4e7fc700 1 -- 172.21.15.107:0/4265284703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a50070870 msgr2=0x7f3a5010fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:43.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.422+0000 7f3a4e7fc700 1 --2- 172.21.15.107:0/4265284703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a50070870 0x7f3a5010fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:43.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.422+0000 7f3a4e7fc700 1 -- 172.21.15.107:0/4265284703 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a40005040 con 0x7f3a50071e20 2024-04-22T23:44:43.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.422+0000 7f3a4effd700 1 --2- 172.21.15.107:0/4265284703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a50070870 0x7f3a5010fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T23:44:43.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.422+0000 7f3a4e7fc700 1 --2- 172.21.15.107:0/4265284703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a50071e20 0x7f3a501100e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3a4400c830 tx=0x7f3a44004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:44:43.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.423+0000 7f3a377fe700 1 -- 172.21.15.107:0/4265284703 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a440106f0 con 0x7f3a50071e20 2024-04-22T23:44:43.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.423+0000 7f3a554b4700 1 -- 172.21.15.107:0/4265284703 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a501152b0 con 0x7f3a50071e20 2024-04-22T23:44:43.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.423+0000 7f3a377fe700 1 -- 172.21.15.107:0/4265284703 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3a44010d20 con 0x7f3a50071e20 2024-04-22T23:44:43.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.423+0000 7f3a377fe700 1 -- 172.21.15.107:0/4265284703 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a440113a0 con 0x7f3a50071e20 2024-04-22T23:44:43.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.423+0000 7f3a554b4700 1 -- 172.21.15.107:0/4265284703 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a50115870 con 0x7f3a50071e20 2024-04-22T23:44:43.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.424+0000 7f3a554b4700 1 -- 172.21.15.107:0/4265284703 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a5004c710 con 0x7f3a50071e20 2024-04-22T23:44:43.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.424+0000 7f3a377fe700 1 -- 172.21.15.107:0/4265284703 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f3a44012400 con 0x7f3a50071e20 2024-04-22T23:44:43.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.425+0000 7f3a377fe700 1 --2- 172.21.15.107:0/4265284703 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f3a3806ec00 0x7f3a380710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:43.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.425+0000 7f3a377fe700 1 -- 172.21.15.107:0/4265284703 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f3a4408e7c0 con 0x7f3a50071e20 2024-04-22T23:44:43.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.425+0000 7f3a4effd700 1 --2- 172.21.15.107:0/4265284703 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f3a3806ec00 0x7f3a380710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:43.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.425+0000 7f3a4effd700 1 --2- 172.21.15.107:0/4265284703 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f3a3806ec00 0x7f3a380710b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3a40004890 tx=0x7f3a40009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:44:43.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.430+0000 7f3a377fe700 1 -- 172.21.15.107:0/4265284703 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f3a4405b110 con 0x7f3a50071e20 2024-04-22T23:44:43.629 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:43 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:43.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:43 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:43.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:43 smithi157 ceph-mon[79403]: from='client.34502 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:44:43.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.727+0000 7f3a554b4700 1 -- 172.21.15.107:0/4265284703 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3a50115d70 con 0x7f3a50071e20 2024-04-22T23:44:43.721 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:43 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:43.721 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:43 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:43.721 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:43 smithi107 ceph-mon[90179]: from='client.34502 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:44:43.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.727+0000 7f3a377fe700 1 -- 172.21.15.107:0/4265284703 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f3a4405aca0 con 0x7f3a50071e20 2024-04-22T23:44:43.721 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-22T23:44:43.721 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-04-22T23:44:43.721 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:44:43.722 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:44:43.722 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-04-22T23:44:43.722 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:44:43.722 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:44:43.722 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-04-22T23:44:43.722 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 6 2024-04-22T23:44:43.722 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:44:43.722 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-04-22T23:44:43.722 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 4 2024-04-22T23:44:43.722 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:44:43.722 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-04-22T23:44:43.722 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 14 2024-04-22T23:44:43.722 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T23:44:43.722 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-22T23:44:43.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.730+0000 7f3a554b4700 1 -- 172.21.15.107:0/4265284703 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f3a3806ec00 msgr2=0x7f3a380710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:43.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.730+0000 7f3a554b4700 1 --2- 172.21.15.107:0/4265284703 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f3a3806ec00 0x7f3a380710b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3a40004890 tx=0x7f3a40009e30 comp rx=0 tx=0).stop 2024-04-22T23:44:43.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.730+0000 7f3a554b4700 1 -- 172.21.15.107:0/4265284703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a50071e20 msgr2=0x7f3a501100e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:43.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.730+0000 7f3a554b4700 1 --2- 172.21.15.107:0/4265284703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a50071e20 0x7f3a501100e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3a4400c830 tx=0x7f3a44004470 comp rx=0 tx=0).stop 2024-04-22T23:44:43.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.730+0000 7f3a554b4700 1 -- 172.21.15.107:0/4265284703 shutdown_connections 2024-04-22T23:44:43.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.730+0000 7f3a554b4700 1 --2- 172.21.15.107:0/4265284703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a50070870 0x7f3a5010fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:43.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.730+0000 7f3a554b4700 1 --2- 172.21.15.107:0/4265284703 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f3a3806ec00 0x7f3a380710b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:43.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.730+0000 7f3a554b4700 1 --2- 172.21.15.107:0/4265284703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a50071e20 0x7f3a501100e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:43.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.730+0000 7f3a554b4700 1 -- 172.21.15.107:0/4265284703 >> 172.21.15.107:0/4265284703 conn(0x7f3a500f8580 msgr2=0x7f3a500f9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:44:43.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.730+0000 7f3a554b4700 1 -- 172.21.15.107:0/4265284703 shutdown_connections 2024-04-22T23:44:43.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.730+0000 7f3a554b4700 1 -- 172.21.15.107:0/4265284703 wait complete. 2024-04-22T23:44:43.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.834+0000 7f62f92e5700 1 -- 172.21.15.107:0/701974748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62f40fbe20 msgr2=0x7f62f40fc230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:43.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.834+0000 7f62f92e5700 1 --2- 172.21.15.107:0/701974748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62f40fbe20 0x7f62f40fc230 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f62e4004940 tx=0x7f62e4009e30 comp rx=0 tx=0).stop 2024-04-22T23:44:43.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.834+0000 7f62f92e5700 1 -- 172.21.15.107:0/701974748 shutdown_connections 2024-04-22T23:44:43.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.834+0000 7f62f92e5700 1 --2- 172.21.15.107:0/701974748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62f40fcf90 0x7f62f40fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:43.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.834+0000 7f62f92e5700 1 --2- 172.21.15.107:0/701974748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62f40fbe20 0x7f62f40fc230 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:43.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.834+0000 7f62f92e5700 1 -- 172.21.15.107:0/701974748 >> 172.21.15.107:0/701974748 conn(0x7f62f40f74e0 msgr2=0x7f62f40f9910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:44:43.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.834+0000 7f62f92e5700 1 -- 172.21.15.107:0/701974748 shutdown_connections 2024-04-22T23:44:43.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.834+0000 7f62f92e5700 1 -- 172.21.15.107:0/701974748 wait complete. 2024-04-22T23:44:43.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.835+0000 7f62f92e5700 1 Processor -- start 2024-04-22T23:44:43.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.835+0000 7f62f92e5700 1 -- start start 2024-04-22T23:44:43.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.835+0000 7f62f92e5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62f40fbe20 0x7f62f41107d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:43.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.835+0000 7f62f92e5700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62f40fcf90 0x7f62f4112d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:43.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.835+0000 7f62f92e5700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f4066ba0 con 0x7f62f40fbe20 2024-04-22T23:44:43.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.835+0000 7f62f92e5700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f4066ce0 con 0x7f62f40fcf90 2024-04-22T23:44:43.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.835+0000 7f62f2ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62f40fbe20 0x7f62f41107d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:43.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.835+0000 7f62f2ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62f40fbe20 0x7f62f41107d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34168/0 (socket says 172.21.15.107:34168) 2024-04-22T23:44:43.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.835+0000 7f62f2ffd700 1 -- 172.21.15.107:0/342586328 learned_addr learned my addr 172.21.15.107:0/342586328 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:44:43.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.835+0000 7f62f27fc700 1 --2- 172.21.15.107:0/342586328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62f40fcf90 0x7f62f4112d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:43.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.835+0000 7f62f2ffd700 1 -- 172.21.15.107:0/342586328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62f40fcf90 msgr2=0x7f62f4112d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:43.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.835+0000 7f62f2ffd700 1 --2- 172.21.15.107:0/342586328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62f40fcf90 0x7f62f4112d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:43.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.835+0000 7f62f2ffd700 1 -- 172.21.15.107:0/342586328 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e4005040 con 0x7f62f40fbe20 2024-04-22T23:44:43.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.835+0000 7f62f27fc700 1 --2- 172.21.15.107:0/342586328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62f40fcf90 0x7f62f4112d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T23:44:43.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.836+0000 7f62f2ffd700 1 --2- 172.21.15.107:0/342586328 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62f40fbe20 0x7f62f41107d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f62e4009150 tx=0x7f62e4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:44:43.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.836+0000 7f62db7fe700 1 -- 172.21.15.107:0/342586328 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62e400e040 con 0x7f62f40fbe20 2024-04-22T23:44:43.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.836+0000 7f62db7fe700 1 -- 172.21.15.107:0/342586328 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f62e400bbf0 con 0x7f62f40fbe20 2024-04-22T23:44:43.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.836+0000 7f62f92e5700 1 -- 172.21.15.107:0/342586328 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62f4113260 con 0x7f62f40fbe20 2024-04-22T23:44:43.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.836+0000 7f62db7fe700 1 -- 172.21.15.107:0/342586328 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62e401c630 con 0x7f62f40fbe20 2024-04-22T23:44:43.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.836+0000 7f62f92e5700 1 -- 172.21.15.107:0/342586328 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62f41137d0 con 0x7f62f40fbe20 2024-04-22T23:44:43.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.837+0000 7f62f92e5700 1 -- 172.21.15.107:0/342586328 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62f405f5f0 con 0x7f62f40fbe20 2024-04-22T23:44:43.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.837+0000 7f62db7fe700 1 -- 172.21.15.107:0/342586328 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f62e400bd60 con 0x7f62f40fbe20 2024-04-22T23:44:43.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.838+0000 7f62db7fe700 1 --2- 172.21.15.107:0/342586328 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f62dc06ec10 0x7f62dc0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:43.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.838+0000 7f62db7fe700 1 -- 172.21.15.107:0/342586328 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f62e408fd80 con 0x7f62f40fbe20 2024-04-22T23:44:43.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.838+0000 7f62f27fc700 1 --2- 172.21.15.107:0/342586328 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f62dc06ec10 0x7f62dc0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:43.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.838+0000 7f62f27fc700 1 --2- 172.21.15.107:0/342586328 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f62dc06ec10 0x7f62dc0710c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f62e8009fd0 tx=0x7f62e800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:44:43.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:43.843+0000 7f62db7fe700 1 -- 172.21.15.107:0/342586328 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f62e4058b00 con 0x7f62f40fbe20 2024-04-22T23:44:44.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.098+0000 7f62f92e5700 1 -- 172.21.15.107:0/342586328 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f62f41139f0 con 0x7f62f40fbe20 2024-04-22T23:44:44.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.098+0000 7f62db7fe700 1 -- 172.21.15.107:0/342586328 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 43 v43) v1 ==== 76+0+1957 (secure 0 0 0) 0x7f62e405c1c0 con 0x7f62f40fbe20 2024-04-22T23:44:44.092 INFO:teuthology.orchestra.run.smithi107.stdout:e43 2024-04-22T23:44:44.092 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T23:44:44.092 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:44:44.092 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-04-22T23:44:44.092 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:44:44.092 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-04-22T23:44:44.092 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-04-22T23:44:44.092 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 43 2024-04-22T23:44:44.092 INFO:teuthology.orchestra.run.smithi107.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-22T23:44:44.092 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-04-22T23:32:20.146130+0000 2024-04-22T23:44:44.092 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-04-22T23:44:27.730668+0000 2024-04-22T23:44:44.092 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-04-22T23:44:44.093 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-04-22T23:44:44.093 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-04-22T23:44:44.093 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-04-22T23:44:44.093 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-04-22T23:44:44.093 INFO:teuthology.orchestra.run.smithi107.stdout:max_xattr_size 65536 2024-04-22T23:44:44.093 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-04-22T23:44:44.093 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-04-22T23:44:44.093 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 83 2024-04-22T23:44:44.093 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-22T23:44:44.093 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 2 2024-04-22T23:44:44.093 INFO:teuthology.orchestra.run.smithi107.stdout:in 0,1 2024-04-22T23:44:44.093 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=34434,1=34438} 2024-04-22T23:44:44.094 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-04-22T23:44:44.094 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-04-22T23:44:44.094 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-04-22T23:44:44.094 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-04-22T23:44:44.094 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-04-22T23:44:44.094 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data disabled 2024-04-22T23:44:44.094 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-04-22T23:44:44.094 INFO:teuthology.orchestra.run.smithi107.stdout:bal_rank_mask -1 2024-04-22T23:44:44.094 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-04-22T23:44:44.094 INFO:teuthology.orchestra.run.smithi107.stdout:qdb_cluster leader: 34434 members: 34434,34438 2024-04-22T23:44:44.094 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.dfvvkw{0:34434} state up:active seq 19 join_fscid=1 addr [v2:172.21.15.107:6826/418145840,v1:172.21.15.107:6827/418145840] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T23:44:44.094 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.ktuxjl{1:34438} state up:active seq 14 join_fscid=1 addr [v2:172.21.15.107:6828/1256497807,v1:172.21.15.107:6829/1256497807] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T23:44:44.094 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:44:44.094 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:44:44.095 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-04-22T23:44:44.095 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:44:44.095 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi157.rcaris{-1:44323} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.157:6824/3923812398,v1:172.21.15.157:6825/3923812398] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T23:44:44.095 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi157.ufywse{-1:44349} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.157:6826/3167053094,v1:172.21.15.157:6827/3167053094] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T23:44:44.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.100+0000 7f62f92e5700 1 -- 172.21.15.107:0/342586328 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f62dc06ec10 msgr2=0x7f62dc0710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:44.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.100+0000 7f62f92e5700 1 --2- 172.21.15.107:0/342586328 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f62dc06ec10 0x7f62dc0710c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f62e8009fd0 tx=0x7f62e800b040 comp rx=0 tx=0).stop 2024-04-22T23:44:44.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.100+0000 7f62f92e5700 1 -- 172.21.15.107:0/342586328 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62f40fbe20 msgr2=0x7f62f41107d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:44.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.100+0000 7f62f92e5700 1 --2- 172.21.15.107:0/342586328 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62f40fbe20 0x7f62f41107d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f62e4009150 tx=0x7f62e4009e30 comp rx=0 tx=0).stop 2024-04-22T23:44:44.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.100+0000 7f62f92e5700 1 -- 172.21.15.107:0/342586328 shutdown_connections 2024-04-22T23:44:44.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.100+0000 7f62f92e5700 1 --2- 172.21.15.107:0/342586328 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f62dc06ec10 0x7f62dc0710c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:44.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.100+0000 7f62f92e5700 1 --2- 172.21.15.107:0/342586328 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62f40fbe20 0x7f62f41107d0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:44.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.100+0000 7f62f92e5700 1 --2- 172.21.15.107:0/342586328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62f40fcf90 0x7f62f4112d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:44.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.100+0000 7f62f92e5700 1 -- 172.21.15.107:0/342586328 >> 172.21.15.107:0/342586328 conn(0x7f62f40f74e0 msgr2=0x7f62f40f8be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:44:44.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.100+0000 7f62f92e5700 1 -- 172.21.15.107:0/342586328 shutdown_connections 2024-04-22T23:44:44.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.100+0000 7f62f92e5700 1 -- 172.21.15.107:0/342586328 wait complete. 2024-04-22T23:44:44.096 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 43 2024-04-22T23:44:44.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.203+0000 7f067972e700 1 -- 172.21.15.107:0/2636008580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06740fed90 msgr2=0x7f06740faed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:44.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.203+0000 7f067972e700 1 --2- 172.21.15.107:0/2636008580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06740fed90 0x7f06740faed0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0668004940 tx=0x7f0668009e30 comp rx=0 tx=0).stop 2024-04-22T23:44:44.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.203+0000 7f067972e700 1 -- 172.21.15.107:0/2636008580 shutdown_connections 2024-04-22T23:44:44.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.203+0000 7f067972e700 1 --2- 172.21.15.107:0/2636008580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06740fed90 0x7f06740faed0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:44.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.203+0000 7f067972e700 1 --2- 172.21.15.107:0/2636008580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06740fe440 0x7f06740fe850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:44.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.203+0000 7f067972e700 1 -- 172.21.15.107:0/2636008580 >> 172.21.15.107:0/2636008580 conn(0x7f06740f6be0 msgr2=0x7f06740f9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:44:44.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.203+0000 7f067972e700 1 -- 172.21.15.107:0/2636008580 shutdown_connections 2024-04-22T23:44:44.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.203+0000 7f067972e700 1 -- 172.21.15.107:0/2636008580 wait complete. 2024-04-22T23:44:44.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.204+0000 7f067972e700 1 Processor -- start 2024-04-22T23:44:44.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.204+0000 7f067972e700 1 -- start start 2024-04-22T23:44:44.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.204+0000 7f067972e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06740fe440 0x7f06740fda50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:44.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.204+0000 7f067972e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06740fed90 0x7f06740fc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:44.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.204+0000 7f067972e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06740fe050 con 0x7f06740fed90 2024-04-22T23:44:44.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.204+0000 7f067972e700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06740fc640 con 0x7f06740fe440 2024-04-22T23:44:44.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.204+0000 7f06727fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06740fed90 0x7f06740fc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:44.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.204+0000 7f06727fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06740fed90 0x7f06740fc100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34178/0 (socket says 172.21.15.107:34178) 2024-04-22T23:44:44.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.204+0000 7f06727fc700 1 -- 172.21.15.107:0/3911206622 learned_addr learned my addr 172.21.15.107:0/3911206622 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:44:44.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.204+0000 7f0672ffd700 1 --2- 172.21.15.107:0/3911206622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06740fe440 0x7f06740fda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:44.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.204+0000 7f06727fc700 1 -- 172.21.15.107:0/3911206622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06740fe440 msgr2=0x7f06740fda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:44.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.204+0000 7f06727fc700 1 --2- 172.21.15.107:0/3911206622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06740fe440 0x7f06740fda50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:44.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.204+0000 7f06727fc700 1 -- 172.21.15.107:0/3911206622 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0668005040 con 0x7f06740fed90 2024-04-22T23:44:44.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.204+0000 7f0672ffd700 1 --2- 172.21.15.107:0/3911206622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06740fe440 0x7f06740fda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T23:44:44.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.204+0000 7f06727fc700 1 --2- 172.21.15.107:0/3911206622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06740fed90 0x7f06740fc100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f066800b3b0 tx=0x7f0668009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:44:44.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.204+0000 7f065b7fe700 1 -- 172.21.15.107:0/3911206622 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f066800e040 con 0x7f06740fed90 2024-04-22T23:44:44.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.204+0000 7f067972e700 1 -- 172.21.15.107:0/3911206622 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06740fc890 con 0x7f06740fed90 2024-04-22T23:44:44.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.204+0000 7f065b7fe700 1 -- 172.21.15.107:0/3911206622 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f066800bcc0 con 0x7f06740fed90 2024-04-22T23:44:44.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.204+0000 7f065b7fe700 1 -- 172.21.15.107:0/3911206622 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f066801c630 con 0x7f06740fed90 2024-04-22T23:44:44.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.205+0000 7f067972e700 1 -- 172.21.15.107:0/3911206622 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06740fce50 con 0x7f06740fed90 2024-04-22T23:44:44.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.205+0000 7f067972e700 1 -- 172.21.15.107:0/3911206622 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f067404c710 con 0x7f06740fed90 2024-04-22T23:44:44.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.206+0000 7f065b7fe700 1 -- 172.21.15.107:0/3911206622 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f0668013980 con 0x7f06740fed90 2024-04-22T23:44:44.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.206+0000 7f065b7fe700 1 --2- 172.21.15.107:0/3911206622 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f065c06ec00 0x7f065c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:44.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.206+0000 7f065b7fe700 1 -- 172.21.15.107:0/3911206622 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f066808f650 con 0x7f06740fed90 2024-04-22T23:44:44.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.206+0000 7f0672ffd700 1 --2- 172.21.15.107:0/3911206622 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f065c06ec00 0x7f065c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:44.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.207+0000 7f0672ffd700 1 --2- 172.21.15.107:0/3911206622 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f065c06ec00 0x7f065c0710b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f06640041d0 tx=0x7f0664009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:44:44.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.211+0000 7f065b7fe700 1 -- 172.21.15.107:0/3911206622 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f066805be80 con 0x7f06740fed90 2024-04-22T23:44:44.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.437+0000 7f067972e700 1 -- 172.21.15.107:0/3911206622 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f06741024e0 con 0x7f065c06ec00 2024-04-22T23:44:44.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.439+0000 7f065b7fe700 1 -- 172.21.15.107:0/3911206622 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7f06741024e0 con 0x7f065c06ec00 2024-04-22T23:44:44.432 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-22T23:44:44.432 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c", 2024-04-22T23:44:44.432 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-04-22T23:44:44.432 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-22T23:44:44.432 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [ 2024-04-22T23:44:44.432 INFO:teuthology.orchestra.run.smithi107.stdout: "crash", 2024-04-22T23:44:44.432 INFO:teuthology.orchestra.run.smithi107.stdout: "mds", 2024-04-22T23:44:44.432 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr", 2024-04-22T23:44:44.433 INFO:teuthology.orchestra.run.smithi107.stdout: "osd", 2024-04-22T23:44:44.433 INFO:teuthology.orchestra.run.smithi107.stdout: "mon" 2024-04-22T23:44:44.433 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-22T23:44:44.433 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "16/21 daemons upgraded", 2024-04-22T23:44:44.433 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Currently upgrading node-exporter daemons", 2024-04-22T23:44:44.433 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-04-22T23:44:44.433 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-22T23:44:44.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.441+0000 7f067972e700 1 -- 172.21.15.107:0/3911206622 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f065c06ec00 msgr2=0x7f065c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:44.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.441+0000 7f067972e700 1 --2- 172.21.15.107:0/3911206622 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f065c06ec00 0x7f065c0710b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f06640041d0 tx=0x7f0664009040 comp rx=0 tx=0).stop 2024-04-22T23:44:44.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.441+0000 7f067972e700 1 -- 172.21.15.107:0/3911206622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06740fed90 msgr2=0x7f06740fc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:44.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.441+0000 7f067972e700 1 --2- 172.21.15.107:0/3911206622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06740fed90 0x7f06740fc100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f066800b3b0 tx=0x7f0668009e30 comp rx=0 tx=0).stop 2024-04-22T23:44:44.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.441+0000 7f067972e700 1 -- 172.21.15.107:0/3911206622 shutdown_connections 2024-04-22T23:44:44.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.441+0000 7f067972e700 1 --2- 172.21.15.107:0/3911206622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06740fe440 0x7f06740fda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:44.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.441+0000 7f067972e700 1 --2- 172.21.15.107:0/3911206622 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f065c06ec00 0x7f065c0710b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:44.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.441+0000 7f067972e700 1 --2- 172.21.15.107:0/3911206622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06740fed90 0x7f06740fc100 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:44.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.441+0000 7f067972e700 1 -- 172.21.15.107:0/3911206622 >> 172.21.15.107:0/3911206622 conn(0x7f06740f6be0 msgr2=0x7f06740f8220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:44:44.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.441+0000 7f067972e700 1 -- 172.21.15.107:0/3911206622 shutdown_connections 2024-04-22T23:44:44.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.441+0000 7f067972e700 1 -- 172.21.15.107:0/3911206622 wait complete. 2024-04-22T23:44:44.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.542+0000 7f4f51b7e700 1 -- 172.21.15.107:0/2518238701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f4c0fbd40 msgr2=0x7f4f4c0fc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:44.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.542+0000 7f4f51b7e700 1 --2- 172.21.15.107:0/2518238701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f4c0fbd40 0x7f4f4c0fc150 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c004940 tx=0x7f4f3c009e30 comp rx=0 tx=0).stop 2024-04-22T23:44:44.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.542+0000 7f4f51b7e700 1 -- 172.21.15.107:0/2518238701 shutdown_connections 2024-04-22T23:44:44.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.542+0000 7f4f51b7e700 1 --2- 172.21.15.107:0/2518238701 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f4c0fcff0 0x7f4f4c0fd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:44.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.542+0000 7f4f51b7e700 1 --2- 172.21.15.107:0/2518238701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f4c0fbd40 0x7f4f4c0fc150 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:44.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.542+0000 7f4f51b7e700 1 -- 172.21.15.107:0/2518238701 >> 172.21.15.107:0/2518238701 conn(0x7f4f4c0f74c0 msgr2=0x7f4f4c0f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:44:44.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.542+0000 7f4f51b7e700 1 -- 172.21.15.107:0/2518238701 shutdown_connections 2024-04-22T23:44:44.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.542+0000 7f4f51b7e700 1 -- 172.21.15.107:0/2518238701 wait complete. 2024-04-22T23:44:44.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.542+0000 7f4f51b7e700 1 Processor -- start 2024-04-22T23:44:44.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.543+0000 7f4f51b7e700 1 -- start start 2024-04-22T23:44:44.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.543+0000 7f4f51b7e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f4c0fbd40 0x7f4f4c10fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:44.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.543+0000 7f4f51b7e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f4c0fcff0 0x7f4f4c110130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:44.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.543+0000 7f4f51b7e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4c110730 con 0x7f4f4c0fbd40 2024-04-22T23:44:44.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.543+0000 7f4f51b7e700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4c110f30 con 0x7f4f4c0fcff0 2024-04-22T23:44:44.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.543+0000 7f4f4b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f4c0fbd40 0x7f4f4c10fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:44.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.543+0000 7f4f4b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f4c0fbd40 0x7f4f4c10fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34202/0 (socket says 172.21.15.107:34202) 2024-04-22T23:44:44.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.543+0000 7f4f4b7fe700 1 -- 172.21.15.107:0/2462753490 learned_addr learned my addr 172.21.15.107:0/2462753490 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:44:44.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.543+0000 7f4f42ffd700 1 --2- 172.21.15.107:0/2462753490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f4c0fcff0 0x7f4f4c110130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:44.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.543+0000 7f4f4b7fe700 1 -- 172.21.15.107:0/2462753490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f4c0fcff0 msgr2=0x7f4f4c110130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:44.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.543+0000 7f4f4b7fe700 1 --2- 172.21.15.107:0/2462753490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f4c0fcff0 0x7f4f4c110130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:44.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.543+0000 7f4f4b7fe700 1 -- 172.21.15.107:0/2462753490 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f3c005040 con 0x7f4f4c0fbd40 2024-04-22T23:44:44.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.543+0000 7f4f42ffd700 1 --2- 172.21.15.107:0/2462753490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f4c0fcff0 0x7f4f4c110130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T23:44:44.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.543+0000 7f4f4b7fe700 1 --2- 172.21.15.107:0/2462753490 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f4c0fbd40 0x7f4f4c10fbf0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c009150 tx=0x7f4f3c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:44:44.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.544+0000 7f4f48ff9700 1 -- 172.21.15.107:0/2462753490 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f3c00e040 con 0x7f4f4c0fbd40 2024-04-22T23:44:44.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.544+0000 7f4f48ff9700 1 -- 172.21.15.107:0/2462753490 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4f3c00bcc0 con 0x7f4f4c0fbd40 2024-04-22T23:44:44.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.544+0000 7f4f51b7e700 1 -- 172.21.15.107:0/2462753490 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f4c1111b0 con 0x7f4f4c0fbd40 2024-04-22T23:44:44.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.544+0000 7f4f48ff9700 1 -- 172.21.15.107:0/2462753490 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f3c01c630 con 0x7f4f4c0fbd40 2024-04-22T23:44:44.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.544+0000 7f4f51b7e700 1 -- 172.21.15.107:0/2462753490 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f4c1116f0 con 0x7f4f4c0fbd40 2024-04-22T23:44:44.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.545+0000 7f4f51b7e700 1 -- 172.21.15.107:0/2462753490 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f4c101980 con 0x7f4f4c0fbd40 2024-04-22T23:44:44.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.545+0000 7f4f48ff9700 1 -- 172.21.15.107:0/2462753490 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f4f3c013420 con 0x7f4f4c0fbd40 2024-04-22T23:44:44.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.545+0000 7f4f48ff9700 1 --2- 172.21.15.107:0/2462753490 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f4f3806ec00 0x7f4f380710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:44:44.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.546+0000 7f4f48ff9700 1 -- 172.21.15.107:0/2462753490 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f4f3c08f620 con 0x7f4f4c0fbd40 2024-04-22T23:44:44.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.546+0000 7f4f42ffd700 1 --2- 172.21.15.107:0/2462753490 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f4f3806ec00 0x7f4f380710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:44:44.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.546+0000 7f4f42ffd700 1 --2- 172.21.15.107:0/2462753490 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f4f3806ec00 0x7f4f380710b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4f34007fd0 tx=0x7f4f34009f60 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:44:44.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.550+0000 7f4f48ff9700 1 -- 172.21.15.107:0/2462753490 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f4f3c05bed0 con 0x7f4f4c0fbd40 2024-04-22T23:44:44.629 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:44 smithi157 ceph-mon[79403]: from='client.34506 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:44:44.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:44 smithi157 ceph-mon[79403]: from='client.34510 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:44:44.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:44 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/4265284703' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:44.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:44 smithi157 ceph-mon[79403]: pgmap v241: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-22T23:44:44.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:44 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/342586328' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:44:44.731 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:44 smithi107 ceph-mon[90179]: from='client.34506 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:44:44.731 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:44 smithi107 ceph-mon[90179]: from='client.34510 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:44:44.731 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:44 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/4265284703' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:44.731 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:44 smithi107 ceph-mon[90179]: pgmap v241: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-22T23:44:44.731 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:44 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/342586328' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:44:44.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.860+0000 7f4f51b7e700 1 -- 172.21.15.107:0/2462753490 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f4f4c104860 con 0x7f4f4c0fbd40 2024-04-22T23:44:44.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.860+0000 7f4f48ff9700 1 -- 172.21.15.107:0/2462753490 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f4f3c05ba60 con 0x7f4f4c0fbd40 2024-04-22T23:44:44.854 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_OK 2024-04-22T23:44:44.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.862+0000 7f4f51b7e700 1 -- 172.21.15.107:0/2462753490 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f4f3806ec00 msgr2=0x7f4f380710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:44.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.862+0000 7f4f51b7e700 1 --2- 172.21.15.107:0/2462753490 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f4f3806ec00 0x7f4f380710b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4f34007fd0 tx=0x7f4f34009f60 comp rx=0 tx=0).stop 2024-04-22T23:44:44.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.862+0000 7f4f51b7e700 1 -- 172.21.15.107:0/2462753490 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f4c0fbd40 msgr2=0x7f4f4c10fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:44:44.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.862+0000 7f4f51b7e700 1 --2- 172.21.15.107:0/2462753490 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f4c0fbd40 0x7f4f4c10fbf0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c009150 tx=0x7f4f3c009e30 comp rx=0 tx=0).stop 2024-04-22T23:44:44.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.862+0000 7f4f51b7e700 1 -- 172.21.15.107:0/2462753490 shutdown_connections 2024-04-22T23:44:44.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.862+0000 7f4f51b7e700 1 --2- 172.21.15.107:0/2462753490 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f4f3806ec00 0x7f4f380710b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:44.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.862+0000 7f4f51b7e700 1 --2- 172.21.15.107:0/2462753490 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f4c0fbd40 0x7f4f4c10fbf0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:44.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.862+0000 7f4f51b7e700 1 --2- 172.21.15.107:0/2462753490 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f4c0fcff0 0x7f4f4c110130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:44:44.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.863+0000 7f4f51b7e700 1 -- 172.21.15.107:0/2462753490 >> 172.21.15.107:0/2462753490 conn(0x7f4f4c0f74c0 msgr2=0x7f4f4c065af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:44:44.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.863+0000 7f4f51b7e700 1 -- 172.21.15.107:0/2462753490 shutdown_connections 2024-04-22T23:44:44.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:44:44.863+0000 7f4f51b7e700 1 -- 172.21.15.107:0/2462753490 wait complete. 2024-04-22T23:44:45.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:45 smithi157 ceph-mon[79403]: from='client.34522 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:44:45.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:45 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:45.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:45 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:45.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:45 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:44:45.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:45 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:44:45.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:45 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/2462753490' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:44:45.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:45 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:45.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:45 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:44:45.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:45 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:45.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:45 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:45.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:45 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:45.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:45 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:45.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:45 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:45.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:45 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:45.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:45 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:45.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:45 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:45.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:45 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:45.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:45 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:45.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:45 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:45.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:45 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:45.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:45 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:45.665 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:45 smithi107 ceph-mon[90179]: from='client.34522 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:44:45.666 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:45 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:45.666 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:45 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:45.666 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:45 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:44:45.666 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:45 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:44:45.666 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:45 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/2462753490' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:44:45.666 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:45 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:45.666 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:45 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:44:45.666 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:45 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:45.666 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:45 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:45.666 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:45 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:45.666 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:45 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:45.666 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:45 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:45.666 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:45 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:45.666 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:45 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:45.667 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:45 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:45.667 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:45 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:45.667 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:45 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:45.667 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:45 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:45.667 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:45 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:45.667 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:45 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:44:46.629 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:46 smithi157 ceph-mon[79403]: pgmap v242: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-04-22T23:44:46.731 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:46 smithi107 ceph-mon[90179]: pgmap v242: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-04-22T23:44:47.481 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:47 smithi107 ceph-mon[90179]: Upgrade: Updating prometheus.smithi107 2024-04-22T23:44:47.481 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:47 smithi107 ceph-mon[90179]: Deploying daemon prometheus.smithi107 on smithi107 2024-04-22T23:44:47.629 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:47 smithi157 ceph-mon[79403]: Upgrade: Updating prometheus.smithi107 2024-04-22T23:44:47.629 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:47 smithi157 ceph-mon[79403]: Deploying daemon prometheus.smithi107 on smithi107 2024-04-22T23:44:48.629 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:48 smithi157 ceph-mon[79403]: pgmap v243: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:44:48.731 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:48 smithi107 ceph-mon[90179]: pgmap v243: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:44:50.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:50 smithi107 ceph-mon[90179]: pgmap v244: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:44:51.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:50 smithi157 ceph-mon[79403]: pgmap v244: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:44:53.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:52 smithi157 ceph-mon[79403]: pgmap v245: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:44:53.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:52 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:53.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:52 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:53.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:52 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:44:53.218 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:52 smithi107 ceph-mon[90179]: pgmap v245: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:44:53.218 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:52 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:53.218 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:52 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:53.218 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:52 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:44:55.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:54 smithi157 ceph-mon[79403]: pgmap v246: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:44:55.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:54 smithi107 ceph-mon[90179]: pgmap v246: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:44:56.111 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:55 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:44:56.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:55 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:44:57.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:56 smithi157 ceph-mon[79403]: pgmap v247: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:44:57.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:56 smithi107 ceph-mon[90179]: pgmap v247: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:44:58.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:57 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:58.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:57 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:58.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:57 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:58.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:57 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:59.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:58 smithi157 ceph-mon[79403]: pgmap v248: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:44:59.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:58 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:59.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:44:58 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:59.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:58 smithi107 ceph-mon[90179]: pgmap v248: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:44:59.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:58 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:44:59.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:44:58 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:45:00.981 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:00 smithi107 ceph-mon[90179]: pgmap v249: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:01.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:00 smithi157 ceph-mon[79403]: pgmap v249: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:02.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:02 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:45:02.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:02 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:45:02.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:02 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:45:02.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:02 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:45:02.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:02 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:45:02.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:02 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-22T23:45:02.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:02 smithi107 ceph-mon[90179]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-22T23:45:02.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:02 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:45:02.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:02 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:02.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:02 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:02.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:02 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:02.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:02 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:02.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:02 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:02.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:02 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:02.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:02 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:02.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:02 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:02.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:02 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:02.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:02 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:02.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:02 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:02.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:02 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:02.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:02 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:02.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:02 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:02.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:02 smithi107 ceph-mon[90179]: pgmap v250: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:02.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:02 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:45:02.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:02 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:45:02.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:02 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:45:02.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:02 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:45:02.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:02 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:45:02.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:02 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-22T23:45:02.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:02 smithi157 ceph-mon[79403]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-22T23:45:02.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:02 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:45:02.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:02 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:02.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:02 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:02.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:02 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:02.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:02 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:02.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:02 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:02.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:02 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:02.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:02 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:02.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:02 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:02.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:02 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:02.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:02 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:02.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:02 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:02.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:02 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:02.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:02 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:02.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:02 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:02.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:02 smithi157 ceph-mon[79403]: pgmap v250: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:05.044 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:04 smithi107 ceph-mon[90179]: Upgrade: Updating alertmanager.smithi107 2024-04-22T23:45:05.044 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:04 smithi107 ceph-mon[90179]: Deploying daemon alertmanager.smithi107 on smithi107 2024-04-22T23:45:05.044 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:04 smithi107 ceph-mon[90179]: pgmap v251: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:05.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:04 smithi157 ceph-mon[79403]: Upgrade: Updating alertmanager.smithi107 2024-04-22T23:45:05.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:04 smithi157 ceph-mon[79403]: Deploying daemon alertmanager.smithi107 on smithi107 2024-04-22T23:45:05.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:04 smithi157 ceph-mon[79403]: pgmap v251: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:07.008 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:06 smithi107 ceph-mon[90179]: pgmap v252: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:07.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:06 smithi157 ceph-mon[79403]: pgmap v252: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:09.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:08 smithi157 ceph-mon[79403]: pgmap v253: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:09.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:08 smithi107 ceph-mon[90179]: pgmap v253: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:10.050 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:09 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:45:10.051 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:09 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:45:10.051 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:09 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:45:10.051 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:09 smithi107 ceph-mon[90179]: pgmap v254: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:10.379 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:09 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:45:10.380 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:09 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:45:10.380 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:09 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:45:10.380 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:09 smithi157 ceph-mon[79403]: pgmap v254: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:11.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:10 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:45:11.281 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:10 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:45:12.379 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:11 smithi157 ceph-mon[79403]: pgmap v255: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:12.481 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:11 smithi107 ceph-mon[90179]: pgmap v255: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:14.731 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:14 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:45:14.731 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:14 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:45:14.731 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:14 smithi107 ceph-mon[90179]: pgmap v256: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:14.879 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:14 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:45:14.879 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:14 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:45:14.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:14 smithi157 ceph-mon[79403]: pgmap v256: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:14.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:14.976+0000 7f27eb522700 1 -- 172.21.15.107:0/1552799630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27e40fce50 msgr2=0x7f27e40fd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:14.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:14.976+0000 7f27eb522700 1 --2- 172.21.15.107:0/1552799630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27e40fce50 0x7f27e40fd260 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f27e0004940 tx=0x7f27e0009e30 comp rx=0 tx=0).stop 2024-04-22T23:45:14.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:14.977+0000 7f27eb522700 1 -- 172.21.15.107:0/1552799630 shutdown_connections 2024-04-22T23:45:14.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:14.977+0000 7f27eb522700 1 --2- 172.21.15.107:0/1552799630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27e40fe050 0x7f27e40fe4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:14.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:14.977+0000 7f27eb522700 1 --2- 172.21.15.107:0/1552799630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27e40fce50 0x7f27e40fd260 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:14.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:14.977+0000 7f27eb522700 1 -- 172.21.15.107:0/1552799630 >> 172.21.15.107:0/1552799630 conn(0x7f27e40f8540 msgr2=0x7f27e40fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:45:14.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:14.977+0000 7f27eb522700 1 -- 172.21.15.107:0/1552799630 shutdown_connections 2024-04-22T23:45:14.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:14.977+0000 7f27eb522700 1 -- 172.21.15.107:0/1552799630 wait complete. 2024-04-22T23:45:14.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:14.977+0000 7f27eb522700 1 Processor -- start 2024-04-22T23:45:14.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:14.977+0000 7f27eb522700 1 -- start start 2024-04-22T23:45:14.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:14.977+0000 7f27eb522700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27e40fe050 0x7f27e410da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:45:14.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:14.977+0000 7f27eb522700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27e410dfc0 0x7f27e4112ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:45:14.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:14.977+0000 7f27eb522700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e410e480 con 0x7f27e410dfc0 2024-04-22T23:45:14.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:14.977+0000 7f27eb522700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e410e5c0 con 0x7f27e40fe050 2024-04-22T23:45:14.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:14.978+0000 7f27e8abd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27e410dfc0 0x7f27e4112ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:45:14.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:14.978+0000 7f27e8abd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27e410dfc0 0x7f27e4112ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55622/0 (socket says 172.21.15.107:55622) 2024-04-22T23:45:14.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:14.978+0000 7f27e8abd700 1 -- 172.21.15.107:0/792589751 learned_addr learned my addr 172.21.15.107:0/792589751 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:45:14.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:14.978+0000 7f27e92be700 1 --2- 172.21.15.107:0/792589751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27e40fe050 0x7f27e410da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:45:14.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:14.978+0000 7f27e8abd700 1 -- 172.21.15.107:0/792589751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27e40fe050 msgr2=0x7f27e410da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:14.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:14.978+0000 7f27e8abd700 1 --2- 172.21.15.107:0/792589751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27e40fe050 0x7f27e410da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:14.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:14.978+0000 7f27e8abd700 1 -- 172.21.15.107:0/792589751 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e0005040 con 0x7f27e410dfc0 2024-04-22T23:45:14.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:14.978+0000 7f27e8abd700 1 --2- 172.21.15.107:0/792589751 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27e410dfc0 0x7f27e4112ff0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f27d400eb40 tx=0x7f27d400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:45:14.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:14.978+0000 7f27d9ffb700 1 -- 172.21.15.107:0/792589751 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27d4007cc0 con 0x7f27e410dfc0 2024-04-22T23:45:14.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:14.978+0000 7f27d9ffb700 1 -- 172.21.15.107:0/792589751 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f27d400f040 con 0x7f27e410dfc0 2024-04-22T23:45:14.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:14.978+0000 7f27eb522700 1 -- 172.21.15.107:0/792589751 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27e4113530 con 0x7f27e410dfc0 2024-04-22T23:45:14.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:14.978+0000 7f27d9ffb700 1 -- 172.21.15.107:0/792589751 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27d400ca10 con 0x7f27e410dfc0 2024-04-22T23:45:14.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:14.979+0000 7f27eb522700 1 -- 172.21.15.107:0/792589751 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27e4113a20 con 0x7f27e410dfc0 2024-04-22T23:45:14.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:14.979+0000 7f27eb522700 1 -- 172.21.15.107:0/792589751 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27e404c710 con 0x7f27e410dfc0 2024-04-22T23:45:14.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:14.980+0000 7f27d9ffb700 1 -- 172.21.15.107:0/792589751 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f27d400cb70 con 0x7f27e410dfc0 2024-04-22T23:45:14.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:14.981+0000 7f27d9ffb700 1 --2- 172.21.15.107:0/792589751 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f27d006ec00 0x7f27d00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:45:14.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:14.981+0000 7f27d9ffb700 1 -- 172.21.15.107:0/792589751 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f27d408e4c0 con 0x7f27e410dfc0 2024-04-22T23:45:14.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:14.981+0000 7f27e92be700 1 --2- 172.21.15.107:0/792589751 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f27d006ec00 0x7f27d00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:45:14.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:14.981+0000 7f27e92be700 1 --2- 172.21.15.107:0/792589751 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f27d006ec00 0x7f27d00710b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f27e0009fd0 tx=0x7f27e0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:45:14.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:14.986+0000 7f27d9ffb700 1 -- 172.21.15.107:0/792589751 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f27d405ad80 con 0x7f27e410dfc0 2024-04-22T23:45:15.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.220+0000 7f27eb522700 1 -- 172.21.15.107:0/792589751 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f27e41024b0 con 0x7f27d006ec00 2024-04-22T23:45:15.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.221+0000 7f27d9ffb700 1 -- 172.21.15.107:0/792589751 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+440 (secure 0 0 0) 0x7f27e41024b0 con 0x7f27d006ec00 2024-04-22T23:45:15.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.223+0000 7f27eb522700 1 -- 172.21.15.107:0/792589751 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f27d006ec00 msgr2=0x7f27d00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:15.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.223+0000 7f27eb522700 1 --2- 172.21.15.107:0/792589751 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f27d006ec00 0x7f27d00710b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f27e0009fd0 tx=0x7f27e0009e30 comp rx=0 tx=0).stop 2024-04-22T23:45:15.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.223+0000 7f27eb522700 1 -- 172.21.15.107:0/792589751 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27e410dfc0 msgr2=0x7f27e4112ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:15.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.223+0000 7f27eb522700 1 --2- 172.21.15.107:0/792589751 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27e410dfc0 0x7f27e4112ff0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f27d400eb40 tx=0x7f27d400b040 comp rx=0 tx=0).stop 2024-04-22T23:45:15.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.223+0000 7f27eb522700 1 -- 172.21.15.107:0/792589751 shutdown_connections 2024-04-22T23:45:15.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.223+0000 7f27eb522700 1 --2- 172.21.15.107:0/792589751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27e40fe050 0x7f27e410da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:15.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.223+0000 7f27eb522700 1 --2- 172.21.15.107:0/792589751 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f27d006ec00 0x7f27d00710b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:15.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.223+0000 7f27eb522700 1 --2- 172.21.15.107:0/792589751 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27e410dfc0 0x7f27e4112ff0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:15.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.223+0000 7f27eb522700 1 -- 172.21.15.107:0/792589751 >> 172.21.15.107:0/792589751 conn(0x7f27e40f8540 msgr2=0x7f27e40f9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:45:15.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.223+0000 7f27eb522700 1 -- 172.21.15.107:0/792589751 shutdown_connections 2024-04-22T23:45:15.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.223+0000 7f27eb522700 1 -- 172.21.15.107:0/792589751 wait complete. 2024-04-22T23:45:15.226 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-04-22T23:45:15.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.329+0000 7f5e515f3700 1 -- 172.21.15.107:0/1830853951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e4c0fe090 msgr2=0x7f5e4c0fe500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:15.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.329+0000 7f5e515f3700 1 --2- 172.21.15.107:0/1830853951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e4c0fe090 0x7f5e4c0fe500 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5e40004940 tx=0x7f5e40009e30 comp rx=0 tx=0).stop 2024-04-22T23:45:15.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.329+0000 7f5e515f3700 1 -- 172.21.15.107:0/1830853951 shutdown_connections 2024-04-22T23:45:15.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.329+0000 7f5e515f3700 1 --2- 172.21.15.107:0/1830853951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e4c0fe090 0x7f5e4c0fe500 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:15.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.329+0000 7f5e515f3700 1 --2- 172.21.15.107:0/1830853951 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e4c0fcf20 0x7f5e4c0fd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:15.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.329+0000 7f5e515f3700 1 -- 172.21.15.107:0/1830853951 >> 172.21.15.107:0/1830853951 conn(0x7f5e4c0f85c0 msgr2=0x7f5e4c0faa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:45:15.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.329+0000 7f5e515f3700 1 -- 172.21.15.107:0/1830853951 shutdown_connections 2024-04-22T23:45:15.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.329+0000 7f5e515f3700 1 -- 172.21.15.107:0/1830853951 wait complete. 2024-04-22T23:45:15.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.330+0000 7f5e515f3700 1 Processor -- start 2024-04-22T23:45:15.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.330+0000 7f5e515f3700 1 -- start start 2024-04-22T23:45:15.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.330+0000 7f5e515f3700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e4c0fcf20 0x7f5e4c10da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:45:15.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.330+0000 7f5e515f3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e4c0fe090 0x7f5e4c10dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:45:15.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.330+0000 7f5e515f3700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4c10e5c0 con 0x7f5e4c0fe090 2024-04-22T23:45:15.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.330+0000 7f5e515f3700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4c10ee10 con 0x7f5e4c0fcf20 2024-04-22T23:45:15.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.330+0000 7f5e4a7fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e4c0fe090 0x7f5e4c10dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:45:15.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.330+0000 7f5e4a7fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e4c0fe090 0x7f5e4c10dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55640/0 (socket says 172.21.15.107:55640) 2024-04-22T23:45:15.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.330+0000 7f5e4a7fc700 1 -- 172.21.15.107:0/581084620 learned_addr learned my addr 172.21.15.107:0/581084620 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:45:15.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.330+0000 7f5e4affd700 1 --2- 172.21.15.107:0/581084620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e4c0fcf20 0x7f5e4c10da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:45:15.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.330+0000 7f5e4a7fc700 1 -- 172.21.15.107:0/581084620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e4c0fcf20 msgr2=0x7f5e4c10da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:15.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.330+0000 7f5e4a7fc700 1 --2- 172.21.15.107:0/581084620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e4c0fcf20 0x7f5e4c10da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:15.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.330+0000 7f5e4a7fc700 1 -- 172.21.15.107:0/581084620 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e40005040 con 0x7f5e4c0fe090 2024-04-22T23:45:15.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.331+0000 7f5e4a7fc700 1 --2- 172.21.15.107:0/581084620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e4c0fe090 0x7f5e4c10dfc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5e40009150 tx=0x7f5e40009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:45:15.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.331+0000 7f5e337fe700 1 -- 172.21.15.107:0/581084620 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e4000e040 con 0x7f5e4c0fe090 2024-04-22T23:45:15.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.331+0000 7f5e515f3700 1 -- 172.21.15.107:0/581084620 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e4c10f090 con 0x7f5e4c0fe090 2024-04-22T23:45:15.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.331+0000 7f5e337fe700 1 -- 172.21.15.107:0/581084620 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5e4000bcc0 con 0x7f5e4c0fe090 2024-04-22T23:45:15.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.331+0000 7f5e337fe700 1 -- 172.21.15.107:0/581084620 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e4001c630 con 0x7f5e4c0fe090 2024-04-22T23:45:15.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.331+0000 7f5e515f3700 1 -- 172.21.15.107:0/581084620 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e4c10f5d0 con 0x7f5e4c0fe090 2024-04-22T23:45:15.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.332+0000 7f5e515f3700 1 -- 172.21.15.107:0/581084620 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e4c05f5f0 con 0x7f5e4c0fe090 2024-04-22T23:45:15.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.332+0000 7f5e337fe700 1 -- 172.21.15.107:0/581084620 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f5e40013420 con 0x7f5e4c0fe090 2024-04-22T23:45:15.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.333+0000 7f5e337fe700 1 --2- 172.21.15.107:0/581084620 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f5e3406ec00 0x7f5e340710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:45:15.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.333+0000 7f5e4affd700 1 --2- 172.21.15.107:0/581084620 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f5e3406ec00 0x7f5e340710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:45:15.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.333+0000 7f5e337fe700 1 -- 172.21.15.107:0/581084620 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f5e4008f5e0 con 0x7f5e4c0fe090 2024-04-22T23:45:15.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.333+0000 7f5e4affd700 1 --2- 172.21.15.107:0/581084620 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f5e3406ec00 0x7f5e340710b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c004d60 tx=0x7f5e3c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:45:15.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.338+0000 7f5e337fe700 1 -- 172.21.15.107:0/581084620 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f5e4005be90 con 0x7f5e4c0fe090 2024-04-22T23:45:15.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.561+0000 7f5e515f3700 1 -- 172.21.15.107:0/581084620 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5e4c10f860 con 0x7f5e3406ec00 2024-04-22T23:45:15.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.562+0000 7f5e337fe700 1 -- 172.21.15.107:0/581084620 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+440 (secure 0 0 0) 0x7f5e4c10f860 con 0x7f5e3406ec00 2024-04-22T23:45:15.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.564+0000 7f5e515f3700 1 -- 172.21.15.107:0/581084620 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f5e3406ec00 msgr2=0x7f5e340710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:15.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.564+0000 7f5e515f3700 1 --2- 172.21.15.107:0/581084620 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f5e3406ec00 0x7f5e340710b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c004d60 tx=0x7f5e3c00b040 comp rx=0 tx=0).stop 2024-04-22T23:45:15.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.564+0000 7f5e515f3700 1 -- 172.21.15.107:0/581084620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e4c0fe090 msgr2=0x7f5e4c10dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:15.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.564+0000 7f5e515f3700 1 --2- 172.21.15.107:0/581084620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e4c0fe090 0x7f5e4c10dfc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5e40009150 tx=0x7f5e40009e30 comp rx=0 tx=0).stop 2024-04-22T23:45:15.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.564+0000 7f5e515f3700 1 -- 172.21.15.107:0/581084620 shutdown_connections 2024-04-22T23:45:15.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.564+0000 7f5e515f3700 1 --2- 172.21.15.107:0/581084620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e4c0fcf20 0x7f5e4c10da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:15.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.564+0000 7f5e515f3700 1 --2- 172.21.15.107:0/581084620 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f5e3406ec00 0x7f5e340710b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:15.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.564+0000 7f5e515f3700 1 --2- 172.21.15.107:0/581084620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e4c0fe090 0x7f5e4c10dfc0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:15.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.564+0000 7f5e515f3700 1 -- 172.21.15.107:0/581084620 >> 172.21.15.107:0/581084620 conn(0x7f5e4c0f85c0 msgr2=0x7f5e4c0f9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:45:15.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.564+0000 7f5e515f3700 1 -- 172.21.15.107:0/581084620 shutdown_connections 2024-04-22T23:45:15.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.564+0000 7f5e515f3700 1 -- 172.21.15.107:0/581084620 wait complete. 2024-04-22T23:45:15.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.665+0000 7fe4c9af4700 1 -- 172.21.15.107:0/2736215608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4c4070890 msgr2=0x7fe4c4070d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:15.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.665+0000 7fe4c9af4700 1 --2- 172.21.15.107:0/2736215608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4c4070890 0x7fe4c4070d00 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8004940 tx=0x7fe4b8009e30 comp rx=0 tx=0).stop 2024-04-22T23:45:15.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.665+0000 7fe4c9af4700 1 -- 172.21.15.107:0/2736215608 shutdown_connections 2024-04-22T23:45:15.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.665+0000 7fe4c9af4700 1 --2- 172.21.15.107:0/2736215608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4c4070890 0x7fe4c4070d00 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:15.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.665+0000 7fe4c9af4700 1 --2- 172.21.15.107:0/2736215608 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4c4071e40 0x7fe4c4070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:15.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.665+0000 7fe4c9af4700 1 -- 172.21.15.107:0/2736215608 >> 172.21.15.107:0/2736215608 conn(0x7fe4c40f8570 msgr2=0x7fe4c40fa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:45:15.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.665+0000 7fe4c9af4700 1 -- 172.21.15.107:0/2736215608 shutdown_connections 2024-04-22T23:45:15.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.665+0000 7fe4c9af4700 1 -- 172.21.15.107:0/2736215608 wait complete. 2024-04-22T23:45:15.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.665+0000 7fe4c9af4700 1 Processor -- start 2024-04-22T23:45:15.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.666+0000 7fe4c9af4700 1 -- start start 2024-04-22T23:45:15.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.666+0000 7fe4c9af4700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4c4070890 0x7fe4c410fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:45:15.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.666+0000 7fe4c9af4700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4c4071e40 0x7fe4c4110160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:45:15.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.666+0000 7fe4c9af4700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c4110760 con 0x7fe4c4070890 2024-04-22T23:45:15.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.666+0000 7fe4c9af4700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c4110fb0 con 0x7fe4c4071e40 2024-04-22T23:45:15.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.666+0000 7fe4c37fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4c4070890 0x7fe4c410fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:45:15.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.666+0000 7fe4c37fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4c4070890 0x7fe4c410fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55660/0 (socket says 172.21.15.107:55660) 2024-04-22T23:45:15.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.666+0000 7fe4c37fe700 1 -- 172.21.15.107:0/1741829947 learned_addr learned my addr 172.21.15.107:0/1741829947 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:45:15.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.666+0000 7fe4c2ffd700 1 --2- 172.21.15.107:0/1741829947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4c4071e40 0x7fe4c4110160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:45:15.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.666+0000 7fe4c37fe700 1 -- 172.21.15.107:0/1741829947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4c4071e40 msgr2=0x7fe4c4110160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:15.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.666+0000 7fe4c37fe700 1 --2- 172.21.15.107:0/1741829947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4c4071e40 0x7fe4c4110160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:15.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.666+0000 7fe4c37fe700 1 -- 172.21.15.107:0/1741829947 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4b8005040 con 0x7fe4c4070890 2024-04-22T23:45:15.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.666+0000 7fe4c37fe700 1 --2- 172.21.15.107:0/1741829947 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4c4070890 0x7fe4c410fc20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4002a80 tx=0x7fe4b4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:45:15.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.666+0000 7fe4abfff700 1 -- 172.21.15.107:0/1741829947 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4b4007cc0 con 0x7fe4c4070890 2024-04-22T23:45:15.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.667+0000 7fe4abfff700 1 -- 172.21.15.107:0/1741829947 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe4b400f040 con 0x7fe4c4070890 2024-04-22T23:45:15.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.667+0000 7fe4c9af4700 1 -- 172.21.15.107:0/1741829947 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4c4111290 con 0x7fe4c4070890 2024-04-22T23:45:15.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.667+0000 7fe4abfff700 1 -- 172.21.15.107:0/1741829947 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4b400da30 con 0x7fe4c4070890 2024-04-22T23:45:15.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.667+0000 7fe4c9af4700 1 -- 172.21.15.107:0/1741829947 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4c41117d0 con 0x7fe4c4070890 2024-04-22T23:45:15.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.667+0000 7fe4c9af4700 1 -- 172.21.15.107:0/1741829947 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4c405f5f0 con 0x7fe4c4070890 2024-04-22T23:45:15.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.668+0000 7fe4abfff700 1 -- 172.21.15.107:0/1741829947 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fe4b400db90 con 0x7fe4c4070890 2024-04-22T23:45:15.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.668+0000 7fe4abfff700 1 --2- 172.21.15.107:0/1741829947 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fe4ac06ec00 0x7fe4ac0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:45:15.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.669+0000 7fe4abfff700 1 -- 172.21.15.107:0/1741829947 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fe4b408e6b0 con 0x7fe4c4070890 2024-04-22T23:45:15.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.669+0000 7fe4c2ffd700 1 --2- 172.21.15.107:0/1741829947 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fe4ac06ec00 0x7fe4ac0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:45:15.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.669+0000 7fe4c2ffd700 1 --2- 172.21.15.107:0/1741829947 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fe4ac06ec00 0x7fe4ac0710b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe4b800b970 tx=0x7fe4b8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:45:15.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.673+0000 7fe4abfff700 1 -- 172.21.15.107:0/1741829947 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fe4b405aee0 con 0x7fe4c4070890 2024-04-22T23:45:15.731 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:15 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:45:15.731 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:15 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:45:15.879 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:15 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:45:15.879 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:15 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:45:15.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.893+0000 7fe4c9af4700 1 -- 172.21.15.107:0/1741829947 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe4c4111bb0 con 0x7fe4ac06ec00 2024-04-22T23:45:15.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.903+0000 7fe4abfff700 1 -- 172.21.15.107:0/1741829947 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fe4c4111bb0 con 0x7fe4ac06ec00 2024-04-22T23:45:15.895 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-22T23:45:15.895 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (7s) 2s ago 18m 14.6M - 0.25.0 c8568f914cd2 07fa1ebb7f47 2024-04-22T23:45:15.895 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (6m) 2s ago 18m 6656k - 19.0.0-3252-g43be0201 a0434699c1d7 e074fd2cbb9c 2024-04-22T23:45:15.895 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi157 smithi157 running (6m) 38s ago 17m 6651k - 19.0.0-3252-g43be0201 a0434699c1d7 abc2649dee22 2024-04-22T23:45:15.895 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (8m) 2s ago 17m 86.4M - 9.4.7 954c08fa6188 31449ff442d2 2024-04-22T23:45:15.895 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.dfvvkw smithi107 running (2m) 2s ago 12m 70.3M - 19.0.0-3252-g43be0201 a0434699c1d7 7566b5cc3d01 2024-04-22T23:45:15.895 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.ktuxjl smithi107 running (99s) 2s ago 12m 17.6M - 19.0.0-3252-g43be0201 a0434699c1d7 ce23545f0ae2 2024-04-22T23:45:15.895 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi157.rcaris smithi157 running (80s) 38s ago 12m 11.9M - 19.0.0-3252-g43be0201 a0434699c1d7 12550073a997 2024-04-22T23:45:15.895 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi157.ufywse smithi157 running (57s) 38s ago 12m 14.1M - 19.0.0-3252-g43be0201 a0434699c1d7 e9f97f60ed21 2024-04-22T23:45:15.896 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.uipxvm smithi107 *:8443,9283,8765 running (8m) 2s ago 19m 514M - 19.0.0-3252-g43be0201 a0434699c1d7 d20f05dfe4f2 2024-04-22T23:45:15.896 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi157.tuylis smithi157 *:8443,9283,8765 running (7m) 38s ago 17m 452M - 19.0.0-3252-g43be0201 a0434699c1d7 957e598c9001 2024-04-22T23:45:15.896 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (7m) 2s ago 19m 59.3M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 bc72bbbe2c39 2024-04-22T23:45:15.896 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi157 smithi157 running (7m) 38s ago 17m 41.2M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 ea88de8be4fe 2024-04-22T23:45:15.896 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (45s) 2s ago 18m 8392k - 1.5.0 0da6a335fe13 c338e5470944 2024-04-22T23:45:15.896 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi157 smithi157 *:9100 running (40s) 38s ago 17m 3389k - 1.5.0 0da6a335fe13 8d0255aff39c 2024-04-22T23:45:15.896 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (5m) 2s ago 16m 105M 1557M 19.0.0-3252-g43be0201 a0434699c1d7 fd0ba8ef0c27 2024-04-22T23:45:15.896 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (5m) 2s ago 16m 104M 1557M 19.0.0-3252-g43be0201 a0434699c1d7 77e33b6122e0 2024-04-22T23:45:15.896 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (4m) 2s ago 15m 99.1M 1557M 19.0.0-3252-g43be0201 a0434699c1d7 c307bd8cdcfc 2024-04-22T23:45:15.896 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi157 running (4m) 38s ago 15m 112M 2581M 19.0.0-3252-g43be0201 a0434699c1d7 e81ca519c831 2024-04-22T23:45:15.896 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi157 running (3m) 38s ago 14m 105M 2581M 19.0.0-3252-g43be0201 a0434699c1d7 d5b5a2e4781a 2024-04-22T23:45:15.896 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi157 running (3m) 38s ago 14m 97.1M 2581M 19.0.0-3252-g43be0201 a0434699c1d7 0ee423135fdb 2024-04-22T23:45:15.896 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (23s) 2s ago 17m 46.8M - 2.43.0 a07b618ecd1d f6160052545f 2024-04-22T23:45:15.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.905+0000 7fe4c9af4700 1 -- 172.21.15.107:0/1741829947 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fe4ac06ec00 msgr2=0x7fe4ac0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:15.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.905+0000 7fe4c9af4700 1 --2- 172.21.15.107:0/1741829947 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fe4ac06ec00 0x7fe4ac0710b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe4b800b970 tx=0x7fe4b8009e30 comp rx=0 tx=0).stop 2024-04-22T23:45:15.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.905+0000 7fe4c9af4700 1 -- 172.21.15.107:0/1741829947 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4c4070890 msgr2=0x7fe4c410fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:15.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.905+0000 7fe4c9af4700 1 --2- 172.21.15.107:0/1741829947 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4c4070890 0x7fe4c410fc20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4002a80 tx=0x7fe4b4004600 comp rx=0 tx=0).stop 2024-04-22T23:45:15.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.905+0000 7fe4c9af4700 1 -- 172.21.15.107:0/1741829947 shutdown_connections 2024-04-22T23:45:15.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.905+0000 7fe4c9af4700 1 --2- 172.21.15.107:0/1741829947 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fe4ac06ec00 0x7fe4ac0710b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:15.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.905+0000 7fe4c9af4700 1 --2- 172.21.15.107:0/1741829947 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4c4070890 0x7fe4c410fc20 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:15.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.905+0000 7fe4c9af4700 1 --2- 172.21.15.107:0/1741829947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4c4071e40 0x7fe4c4110160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:15.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.905+0000 7fe4c9af4700 1 -- 172.21.15.107:0/1741829947 >> 172.21.15.107:0/1741829947 conn(0x7fe4c40f8570 msgr2=0x7fe4c40f9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:45:15.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.905+0000 7fe4c9af4700 1 -- 172.21.15.107:0/1741829947 shutdown_connections 2024-04-22T23:45:15.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:15.905+0000 7fe4c9af4700 1 -- 172.21.15.107:0/1741829947 wait complete. 2024-04-22T23:45:15.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.006+0000 7f6b8954d700 1 -- 172.21.15.107:0/3723813146 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b840fbd40 msgr2=0x7f6b840fc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:15.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.006+0000 7f6b8954d700 1 --2- 172.21.15.107:0/3723813146 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b840fbd40 0x7f6b840fc150 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6b74004940 tx=0x7f6b74009e30 comp rx=0 tx=0).stop 2024-04-22T23:45:15.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.006+0000 7f6b8954d700 1 -- 172.21.15.107:0/3723813146 shutdown_connections 2024-04-22T23:45:15.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.006+0000 7f6b8954d700 1 --2- 172.21.15.107:0/3723813146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b840fcff0 0x7f6b840fd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:15.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.006+0000 7f6b8954d700 1 --2- 172.21.15.107:0/3723813146 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b840fbd40 0x7f6b840fc150 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:15.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.006+0000 7f6b8954d700 1 -- 172.21.15.107:0/3723813146 >> 172.21.15.107:0/3723813146 conn(0x7f6b840f74c0 msgr2=0x7f6b840f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:45:15.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.006+0000 7f6b8954d700 1 -- 172.21.15.107:0/3723813146 shutdown_connections 2024-04-22T23:45:15.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.006+0000 7f6b8954d700 1 -- 172.21.15.107:0/3723813146 wait complete. 2024-04-22T23:45:16.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.007+0000 7f6b8954d700 1 Processor -- start 2024-04-22T23:45:16.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.007+0000 7f6b8954d700 1 -- start start 2024-04-22T23:45:16.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.007+0000 7f6b8954d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b840fbd40 0x7f6b8410c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:45:16.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.007+0000 7f6b8954d700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b840fcff0 0x7f6b8410ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:45:16.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.007+0000 7f6b8954d700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8410d460 con 0x7f6b840fbd40 2024-04-22T23:45:16.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.007+0000 7f6b8954d700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8410dcb0 con 0x7f6b840fcff0 2024-04-22T23:45:16.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.007+0000 7f6b82ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b840fbd40 0x7f6b8410c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:45:16.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.007+0000 7f6b82ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b840fbd40 0x7f6b8410c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55680/0 (socket says 172.21.15.107:55680) 2024-04-22T23:45:16.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.007+0000 7f6b82ffd700 1 -- 172.21.15.107:0/1526032686 learned_addr learned my addr 172.21.15.107:0/1526032686 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:45:16.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.007+0000 7f6b827fc700 1 --2- 172.21.15.107:0/1526032686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b840fcff0 0x7f6b8410ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:45:16.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.007+0000 7f6b82ffd700 1 -- 172.21.15.107:0/1526032686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b840fcff0 msgr2=0x7f6b8410ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:16.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.007+0000 7f6b82ffd700 1 --2- 172.21.15.107:0/1526032686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b840fcff0 0x7f6b8410ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:16.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.007+0000 7f6b82ffd700 1 -- 172.21.15.107:0/1526032686 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b74005040 con 0x7f6b840fbd40 2024-04-22T23:45:16.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.007+0000 7f6b827fc700 1 --2- 172.21.15.107:0/1526032686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b840fcff0 0x7f6b8410ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T23:45:16.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.007+0000 7f6b82ffd700 1 --2- 172.21.15.107:0/1526032686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b840fbd40 0x7f6b8410c920 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6b74009fd0 tx=0x7f6b74009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:45:16.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.008+0000 7f6b6b7fe700 1 -- 172.21.15.107:0/1526032686 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b7400e040 con 0x7f6b840fbd40 2024-04-22T23:45:16.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.008+0000 7f6b6b7fe700 1 -- 172.21.15.107:0/1526032686 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6b7400bed0 con 0x7f6b840fbd40 2024-04-22T23:45:16.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.008+0000 7f6b8954d700 1 -- 172.21.15.107:0/1526032686 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b8410df30 con 0x7f6b840fbd40 2024-04-22T23:45:16.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.008+0000 7f6b6b7fe700 1 -- 172.21.15.107:0/1526032686 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b740137c0 con 0x7f6b840fbd40 2024-04-22T23:45:16.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.008+0000 7f6b8954d700 1 -- 172.21.15.107:0/1526032686 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b8410e470 con 0x7f6b840fbd40 2024-04-22T23:45:16.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.009+0000 7f6b8954d700 1 -- 172.21.15.107:0/1526032686 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b8405f5f0 con 0x7f6b840fbd40 2024-04-22T23:45:16.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.009+0000 7f6b6b7fe700 1 -- 172.21.15.107:0/1526032686 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f6b7401f480 con 0x7f6b840fbd40 2024-04-22T23:45:16.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.009+0000 7f6b6b7fe700 1 --2- 172.21.15.107:0/1526032686 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f6b6c06ec00 0x7f6b6c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:45:16.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.010+0000 7f6b6b7fe700 1 -- 172.21.15.107:0/1526032686 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f6b74092350 con 0x7f6b840fbd40 2024-04-22T23:45:16.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.010+0000 7f6b827fc700 1 --2- 172.21.15.107:0/1526032686 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f6b6c06ec00 0x7f6b6c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:45:16.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.010+0000 7f6b827fc700 1 --2- 172.21.15.107:0/1526032686 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f6b6c06ec00 0x7f6b6c0710b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6b78009fd0 tx=0x7f6b78009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:45:16.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.014+0000 7f6b6b7fe700 1 -- 172.21.15.107:0/1526032686 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f6b7405ec30 con 0x7f6b840fbd40 2024-04-22T23:45:16.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.309+0000 7f6b8954d700 1 -- 172.21.15.107:0/1526032686 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6b8410e860 con 0x7f6b840fbd40 2024-04-22T23:45:16.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.310+0000 7f6b6b7fe700 1 -- 172.21.15.107:0/1526032686 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f6b7405e7c0 con 0x7f6b840fbd40 2024-04-22T23:45:16.302 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-22T23:45:16.302 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-04-22T23:45:16.302 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:45:16.302 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:45:16.302 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-04-22T23:45:16.303 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:45:16.303 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:45:16.303 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-04-22T23:45:16.303 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 6 2024-04-22T23:45:16.303 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:45:16.303 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-04-22T23:45:16.303 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 4 2024-04-22T23:45:16.303 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:45:16.303 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-04-22T23:45:16.303 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 14 2024-04-22T23:45:16.303 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T23:45:16.303 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-22T23:45:16.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.312+0000 7f6b8954d700 1 -- 172.21.15.107:0/1526032686 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f6b6c06ec00 msgr2=0x7f6b6c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:16.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.312+0000 7f6b8954d700 1 --2- 172.21.15.107:0/1526032686 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f6b6c06ec00 0x7f6b6c0710b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6b78009fd0 tx=0x7f6b78009ee0 comp rx=0 tx=0).stop 2024-04-22T23:45:16.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.312+0000 7f6b8954d700 1 -- 172.21.15.107:0/1526032686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b840fbd40 msgr2=0x7f6b8410c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:16.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.312+0000 7f6b8954d700 1 --2- 172.21.15.107:0/1526032686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b840fbd40 0x7f6b8410c920 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6b74009fd0 tx=0x7f6b74009e30 comp rx=0 tx=0).stop 2024-04-22T23:45:16.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.312+0000 7f6b8954d700 1 -- 172.21.15.107:0/1526032686 shutdown_connections 2024-04-22T23:45:16.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.312+0000 7f6b8954d700 1 --2- 172.21.15.107:0/1526032686 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f6b6c06ec00 0x7f6b6c0710b0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:16.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.312+0000 7f6b8954d700 1 --2- 172.21.15.107:0/1526032686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b840fbd40 0x7f6b8410c920 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:16.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.312+0000 7f6b8954d700 1 --2- 172.21.15.107:0/1526032686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b840fcff0 0x7f6b8410ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:16.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.312+0000 7f6b8954d700 1 -- 172.21.15.107:0/1526032686 >> 172.21.15.107:0/1526032686 conn(0x7f6b840f74c0 msgr2=0x7f6b840f8bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:45:16.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.312+0000 7f6b8954d700 1 -- 172.21.15.107:0/1526032686 shutdown_connections 2024-04-22T23:45:16.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.312+0000 7f6b8954d700 1 -- 172.21.15.107:0/1526032686 wait complete. 2024-04-22T23:45:16.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.414+0000 7fe794c9b700 1 -- 172.21.15.107:0/1598647122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7900fe410 msgr2=0x7fe7900fe820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:16.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.414+0000 7fe794c9b700 1 --2- 172.21.15.107:0/1598647122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7900fe410 0x7fe7900fe820 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe780009ce0 tx=0x7fe780004d10 comp rx=0 tx=0).stop 2024-04-22T23:45:16.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.415+0000 7fe794c9b700 1 -- 172.21.15.107:0/1598647122 shutdown_connections 2024-04-22T23:45:16.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.415+0000 7fe794c9b700 1 --2- 172.21.15.107:0/1598647122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7900fed60 0x7fe7900faea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:16.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.415+0000 7fe794c9b700 1 --2- 172.21.15.107:0/1598647122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7900fe410 0x7fe7900fe820 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:16.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.415+0000 7fe794c9b700 1 -- 172.21.15.107:0/1598647122 >> 172.21.15.107:0/1598647122 conn(0x7fe7900f6bd0 msgr2=0x7fe7900f8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:45:16.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.415+0000 7fe794c9b700 1 -- 172.21.15.107:0/1598647122 shutdown_connections 2024-04-22T23:45:16.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.415+0000 7fe794c9b700 1 -- 172.21.15.107:0/1598647122 wait complete. 2024-04-22T23:45:16.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.415+0000 7fe794c9b700 1 Processor -- start 2024-04-22T23:45:16.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.416+0000 7fe794c9b700 1 -- start start 2024-04-22T23:45:16.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.416+0000 7fe794c9b700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7900fe410 0x7fe79010dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:45:16.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.416+0000 7fe794c9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7900fed60 0x7fe79010e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:45:16.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.416+0000 7fe794c9b700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79010e6e0 con 0x7fe7900fed60 2024-04-22T23:45:16.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.416+0000 7fe794c9b700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79010ef30 con 0x7fe7900fe410 2024-04-22T23:45:16.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.416+0000 7fe785d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7900fed60 0x7fe79010e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:45:16.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.416+0000 7fe785d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7900fed60 0x7fe79010e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55710/0 (socket says 172.21.15.107:55710) 2024-04-22T23:45:16.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.416+0000 7fe785d9b700 1 -- 172.21.15.107:0/2922494954 learned_addr learned my addr 172.21.15.107:0/2922494954 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:45:16.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.416+0000 7fe78e59c700 1 --2- 172.21.15.107:0/2922494954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7900fe410 0x7fe79010dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:45:16.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.416+0000 7fe785d9b700 1 -- 172.21.15.107:0/2922494954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7900fe410 msgr2=0x7fe79010dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:16.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.416+0000 7fe785d9b700 1 --2- 172.21.15.107:0/2922494954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7900fe410 0x7fe79010dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:16.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.416+0000 7fe785d9b700 1 -- 172.21.15.107:0/2922494954 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe780009940 con 0x7fe7900fed60 2024-04-22T23:45:16.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.416+0000 7fe785d9b700 1 --2- 172.21.15.107:0/2922494954 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7900fed60 0x7fe79010e0e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe77800c8c0 tx=0x7fe7780044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:45:16.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.416+0000 7fe7877fe700 1 -- 172.21.15.107:0/2922494954 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe77800a980 con 0x7fe7900fed60 2024-04-22T23:45:16.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.416+0000 7fe7877fe700 1 -- 172.21.15.107:0/2922494954 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe77800ce60 con 0x7fe7900fed60 2024-04-22T23:45:16.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.417+0000 7fe794c9b700 1 -- 172.21.15.107:0/2922494954 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe79010f210 con 0x7fe7900fed60 2024-04-22T23:45:16.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.417+0000 7fe7877fe700 1 -- 172.21.15.107:0/2922494954 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7780115e0 con 0x7fe7900fed60 2024-04-22T23:45:16.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.417+0000 7fe794c9b700 1 -- 172.21.15.107:0/2922494954 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe79010f750 con 0x7fe7900fed60 2024-04-22T23:45:16.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.418+0000 7fe794c9b700 1 -- 172.21.15.107:0/2922494954 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe79005f5f0 con 0x7fe7900fed60 2024-04-22T23:45:16.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.418+0000 7fe7877fe700 1 -- 172.21.15.107:0/2922494954 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fe778012400 con 0x7fe7900fed60 2024-04-22T23:45:16.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.419+0000 7fe7877fe700 1 --2- 172.21.15.107:0/2922494954 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fe77c06ec00 0x7fe77c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:45:16.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.419+0000 7fe7877fe700 1 -- 172.21.15.107:0/2922494954 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fe77808ea30 con 0x7fe7900fed60 2024-04-22T23:45:16.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.419+0000 7fe78e59c700 1 --2- 172.21.15.107:0/2922494954 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fe77c06ec00 0x7fe77c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:45:16.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.419+0000 7fe78e59c700 1 --2- 172.21.15.107:0/2922494954 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fe77c06ec00 0x7fe77c0710b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe7800095a0 tx=0x7fe780004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:45:16.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.423+0000 7fe7877fe700 1 -- 172.21.15.107:0/2922494954 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fe77805b2e0 con 0x7fe7900fed60 2024-04-22T23:45:16.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.677+0000 7fe794c9b700 1 -- 172.21.15.107:0/2922494954 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe79004bed0 con 0x7fe7900fed60 2024-04-22T23:45:16.670 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:16 smithi107 ceph-mon[90179]: from='client.34530 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:45:16.671 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:16 smithi107 ceph-mon[90179]: from='client.34534 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:45:16.671 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:16 smithi107 ceph-mon[90179]: pgmap v257: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:16.671 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:16 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/1526032686' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:16.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.678+0000 7fe7877fe700 1 -- 172.21.15.107:0/2922494954 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 43 v43) v1 ==== 76+0+1957 (secure 0 0 0) 0x7fe7780126b0 con 0x7fe7900fed60 2024-04-22T23:45:16.671 INFO:teuthology.orchestra.run.smithi107.stdout:e43 2024-04-22T23:45:16.671 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T23:45:16.671 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:45:16.672 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-04-22T23:45:16.672 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:45:16.672 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-04-22T23:45:16.672 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-04-22T23:45:16.672 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 43 2024-04-22T23:45:16.672 INFO:teuthology.orchestra.run.smithi107.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-22T23:45:16.672 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-04-22T23:32:20.146130+0000 2024-04-22T23:45:16.672 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-04-22T23:44:27.730668+0000 2024-04-22T23:45:16.672 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-04-22T23:45:16.672 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-04-22T23:45:16.672 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-04-22T23:45:16.672 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-04-22T23:45:16.672 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-04-22T23:45:16.672 INFO:teuthology.orchestra.run.smithi107.stdout:max_xattr_size 65536 2024-04-22T23:45:16.672 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-04-22T23:45:16.673 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-04-22T23:45:16.673 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 83 2024-04-22T23:45:16.673 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-22T23:45:16.673 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 2 2024-04-22T23:45:16.673 INFO:teuthology.orchestra.run.smithi107.stdout:in 0,1 2024-04-22T23:45:16.673 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=34434,1=34438} 2024-04-22T23:45:16.673 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-04-22T23:45:16.673 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-04-22T23:45:16.673 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-04-22T23:45:16.673 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-04-22T23:45:16.673 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-04-22T23:45:16.673 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data disabled 2024-04-22T23:45:16.673 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-04-22T23:45:16.673 INFO:teuthology.orchestra.run.smithi107.stdout:bal_rank_mask -1 2024-04-22T23:45:16.674 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-04-22T23:45:16.674 INFO:teuthology.orchestra.run.smithi107.stdout:qdb_cluster leader: 34434 members: 34434,34438 2024-04-22T23:45:16.674 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.dfvvkw{0:34434} state up:active seq 19 join_fscid=1 addr [v2:172.21.15.107:6826/418145840,v1:172.21.15.107:6827/418145840] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T23:45:16.674 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.ktuxjl{1:34438} state up:active seq 14 join_fscid=1 addr [v2:172.21.15.107:6828/1256497807,v1:172.21.15.107:6829/1256497807] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T23:45:16.674 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:45:16.674 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:45:16.674 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-04-22T23:45:16.674 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:45:16.674 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi157.rcaris{-1:44323} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.157:6824/3923812398,v1:172.21.15.157:6825/3923812398] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T23:45:16.674 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi157.ufywse{-1:44349} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.157:6826/3167053094,v1:172.21.15.157:6827/3167053094] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T23:45:16.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.681+0000 7fe794c9b700 1 -- 172.21.15.107:0/2922494954 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fe77c06ec00 msgr2=0x7fe77c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:16.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.681+0000 7fe794c9b700 1 --2- 172.21.15.107:0/2922494954 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fe77c06ec00 0x7fe77c0710b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe7800095a0 tx=0x7fe780004d10 comp rx=0 tx=0).stop 2024-04-22T23:45:16.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.681+0000 7fe794c9b700 1 -- 172.21.15.107:0/2922494954 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7900fed60 msgr2=0x7fe79010e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:16.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.681+0000 7fe794c9b700 1 --2- 172.21.15.107:0/2922494954 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7900fed60 0x7fe79010e0e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe77800c8c0 tx=0x7fe7780044c0 comp rx=0 tx=0).stop 2024-04-22T23:45:16.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.681+0000 7fe794c9b700 1 -- 172.21.15.107:0/2922494954 shutdown_connections 2024-04-22T23:45:16.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.681+0000 7fe794c9b700 1 --2- 172.21.15.107:0/2922494954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7900fe410 0x7fe79010dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:16.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.681+0000 7fe794c9b700 1 --2- 172.21.15.107:0/2922494954 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fe77c06ec00 0x7fe77c0710b0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:16.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.681+0000 7fe794c9b700 1 --2- 172.21.15.107:0/2922494954 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7900fed60 0x7fe79010e0e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:16.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.681+0000 7fe794c9b700 1 -- 172.21.15.107:0/2922494954 >> 172.21.15.107:0/2922494954 conn(0x7fe7900f6bd0 msgr2=0x7fe7900f82e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:45:16.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.681+0000 7fe794c9b700 1 -- 172.21.15.107:0/2922494954 shutdown_connections 2024-04-22T23:45:16.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.681+0000 7fe794c9b700 1 -- 172.21.15.107:0/2922494954 wait complete. 2024-04-22T23:45:16.676 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 43 2024-04-22T23:45:16.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.785+0000 7f1511506700 1 -- 172.21.15.107:0/973576106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f150c0fce80 msgr2=0x7f150c0fd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:16.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.785+0000 7f1511506700 1 --2- 172.21.15.107:0/973576106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f150c0fce80 0x7f150c0fd290 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f14fc004940 tx=0x7f14fc009e30 comp rx=0 tx=0).stop 2024-04-22T23:45:16.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.785+0000 7f1511506700 1 -- 172.21.15.107:0/973576106 shutdown_connections 2024-04-22T23:45:16.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.785+0000 7f1511506700 1 --2- 172.21.15.107:0/973576106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f150c0fdff0 0x7f150c0fe460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:16.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.785+0000 7f1511506700 1 --2- 172.21.15.107:0/973576106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f150c0fce80 0x7f150c0fd290 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:16.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.785+0000 7f1511506700 1 -- 172.21.15.107:0/973576106 >> 172.21.15.107:0/973576106 conn(0x7f150c0f8540 msgr2=0x7f150c0fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:45:16.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.785+0000 7f1511506700 1 -- 172.21.15.107:0/973576106 shutdown_connections 2024-04-22T23:45:16.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.785+0000 7f1511506700 1 -- 172.21.15.107:0/973576106 wait complete. 2024-04-22T23:45:16.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.786+0000 7f1511506700 1 Processor -- start 2024-04-22T23:45:16.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.786+0000 7f1511506700 1 -- start start 2024-04-22T23:45:16.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.786+0000 7f1511506700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f150c0fce80 0x7f150c10d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:45:16.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.786+0000 7f1511506700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f150c0fdff0 0x7f150c10ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:45:16.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.786+0000 7f1511506700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150c10e4d0 con 0x7f150c0fdff0 2024-04-22T23:45:16.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.786+0000 7f1511506700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150c10e640 con 0x7f150c0fce80 2024-04-22T23:45:16.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.786+0000 7f150a7fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f150c0fdff0 0x7f150c10ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:45:16.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.786+0000 7f150a7fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f150c0fdff0 0x7f150c10ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55730/0 (socket says 172.21.15.107:55730) 2024-04-22T23:45:16.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.786+0000 7f150a7fc700 1 -- 172.21.15.107:0/1598383204 learned_addr learned my addr 172.21.15.107:0/1598383204 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:45:16.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.786+0000 7f150affd700 1 --2- 172.21.15.107:0/1598383204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f150c0fce80 0x7f150c10d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:45:16.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.786+0000 7f150a7fc700 1 -- 172.21.15.107:0/1598383204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f150c0fce80 msgr2=0x7f150c10d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:16.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.786+0000 7f150a7fc700 1 --2- 172.21.15.107:0/1598383204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f150c0fce80 0x7f150c10d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:16.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.786+0000 7f150a7fc700 1 -- 172.21.15.107:0/1598383204 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14fc005040 con 0x7f150c0fdff0 2024-04-22T23:45:16.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.787+0000 7f150a7fc700 1 --2- 172.21.15.107:0/1598383204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f150c0fdff0 0x7f150c10ded0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1500002a50 tx=0x7f15000046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:45:16.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.787+0000 7f14f37fe700 1 -- 172.21.15.107:0/1598383204 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1500007cc0 con 0x7f150c0fdff0 2024-04-22T23:45:16.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.787+0000 7f1511506700 1 -- 172.21.15.107:0/1598383204 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f150c1130a0 con 0x7f150c0fdff0 2024-04-22T23:45:16.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.787+0000 7f14f37fe700 1 -- 172.21.15.107:0/1598383204 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f150000f040 con 0x7f150c0fdff0 2024-04-22T23:45:16.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.787+0000 7f14f37fe700 1 -- 172.21.15.107:0/1598383204 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1500009640 con 0x7f150c0fdff0 2024-04-22T23:45:16.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.787+0000 7f1511506700 1 -- 172.21.15.107:0/1598383204 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f150c113660 con 0x7f150c0fdff0 2024-04-22T23:45:16.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.788+0000 7f1511506700 1 -- 172.21.15.107:0/1598383204 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f150c04c710 con 0x7f150c0fdff0 2024-04-22T23:45:16.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.788+0000 7f14f37fe700 1 -- 172.21.15.107:0/1598383204 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f15000097a0 con 0x7f150c0fdff0 2024-04-22T23:45:16.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.789+0000 7f14f37fe700 1 --2- 172.21.15.107:0/1598383204 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f14f406ec00 0x7f14f40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:45:16.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.789+0000 7f14f37fe700 1 -- 172.21.15.107:0/1598383204 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f150008e750 con 0x7f150c0fdff0 2024-04-22T23:45:16.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.789+0000 7f150affd700 1 --2- 172.21.15.107:0/1598383204 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f14f406ec00 0x7f14f40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:45:16.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.789+0000 7f150affd700 1 --2- 172.21.15.107:0/1598383204 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f14f406ec00 0x7f14f40710b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f14fc00b3b0 tx=0x7f14fc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:45:16.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:16.793+0000 7f14f37fe700 1 -- 172.21.15.107:0/1598383204 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f150005af80 con 0x7f150c0fdff0 2024-04-22T23:45:16.879 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:16 smithi157 ceph-mon[79403]: from='client.34530 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:45:16.879 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:16 smithi157 ceph-mon[79403]: from='client.34534 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:45:16.879 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:16 smithi157 ceph-mon[79403]: pgmap v257: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:16.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:16 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/1526032686' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:17.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.025+0000 7f1511506700 1 -- 172.21.15.107:0/1598383204 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f150c1023d0 con 0x7f14f406ec00 2024-04-22T23:45:17.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.027+0000 7f14f37fe700 1 -- 172.21.15.107:0/1598383204 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+440 (secure 0 0 0) 0x7f150c1023d0 con 0x7f14f406ec00 2024-04-22T23:45:17.019 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-22T23:45:17.019 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c", 2024-04-22T23:45:17.019 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": true, 2024-04-22T23:45:17.019 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-22T23:45:17.019 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [ 2024-04-22T23:45:17.019 INFO:teuthology.orchestra.run.smithi107.stdout: "crash", 2024-04-22T23:45:17.020 INFO:teuthology.orchestra.run.smithi107.stdout: "mds", 2024-04-22T23:45:17.020 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr", 2024-04-22T23:45:17.020 INFO:teuthology.orchestra.run.smithi107.stdout: "osd", 2024-04-22T23:45:17.020 INFO:teuthology.orchestra.run.smithi107.stdout: "mon" 2024-04-22T23:45:17.020 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-22T23:45:17.020 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": "16/21 daemons upgraded", 2024-04-22T23:45:17.020 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "Currently upgrading alertmanager daemons", 2024-04-22T23:45:17.020 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-04-22T23:45:17.020 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-22T23:45:17.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.029+0000 7f1511506700 1 -- 172.21.15.107:0/1598383204 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f14f406ec00 msgr2=0x7f14f40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:17.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.029+0000 7f1511506700 1 --2- 172.21.15.107:0/1598383204 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f14f406ec00 0x7f14f40710b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f14fc00b3b0 tx=0x7f14fc009e30 comp rx=0 tx=0).stop 2024-04-22T23:45:17.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.029+0000 7f1511506700 1 -- 172.21.15.107:0/1598383204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f150c0fdff0 msgr2=0x7f150c10ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:17.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.029+0000 7f1511506700 1 --2- 172.21.15.107:0/1598383204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f150c0fdff0 0x7f150c10ded0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1500002a50 tx=0x7f15000046b0 comp rx=0 tx=0).stop 2024-04-22T23:45:17.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.029+0000 7f1511506700 1 -- 172.21.15.107:0/1598383204 shutdown_connections 2024-04-22T23:45:17.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.029+0000 7f1511506700 1 --2- 172.21.15.107:0/1598383204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f150c0fce80 0x7f150c10d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:17.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.029+0000 7f1511506700 1 --2- 172.21.15.107:0/1598383204 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f14f406ec00 0x7f14f40710b0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:17.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.029+0000 7f1511506700 1 --2- 172.21.15.107:0/1598383204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f150c0fdff0 0x7f150c10ded0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:17.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.029+0000 7f1511506700 1 -- 172.21.15.107:0/1598383204 >> 172.21.15.107:0/1598383204 conn(0x7f150c0f8540 msgr2=0x7f150c0f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:45:17.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.029+0000 7f1511506700 1 -- 172.21.15.107:0/1598383204 shutdown_connections 2024-04-22T23:45:17.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.029+0000 7f1511506700 1 -- 172.21.15.107:0/1598383204 wait complete. 2024-04-22T23:45:17.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.133+0000 7f05d5ad9700 1 -- 172.21.15.107:0/3790704366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05d00708f0 msgr2=0x7f05d0070d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:17.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.133+0000 7f05d5ad9700 1 --2- 172.21.15.107:0/3790704366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05d00708f0 0x7f05d0070d60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f05c0004940 tx=0x7f05c0009e30 comp rx=0 tx=0).stop 2024-04-22T23:45:17.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.133+0000 7f05d5ad9700 1 -- 172.21.15.107:0/3790704366 shutdown_connections 2024-04-22T23:45:17.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.133+0000 7f05d5ad9700 1 --2- 172.21.15.107:0/3790704366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05d00708f0 0x7f05d0070d60 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:17.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.133+0000 7f05d5ad9700 1 --2- 172.21.15.107:0/3790704366 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05d0071ea0 0x7f05d00703b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:17.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.133+0000 7f05d5ad9700 1 -- 172.21.15.107:0/3790704366 >> 172.21.15.107:0/3790704366 conn(0x7f05d00f8570 msgr2=0x7f05d00fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:45:17.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.133+0000 7f05d5ad9700 1 -- 172.21.15.107:0/3790704366 shutdown_connections 2024-04-22T23:45:17.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.133+0000 7f05d5ad9700 1 -- 172.21.15.107:0/3790704366 wait complete. 2024-04-22T23:45:17.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.133+0000 7f05d5ad9700 1 Processor -- start 2024-04-22T23:45:17.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.134+0000 7f05d5ad9700 1 -- start start 2024-04-22T23:45:17.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.134+0000 7f05d5ad9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05d00708f0 0x7f05d010fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:45:17.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.134+0000 7f05d5ad9700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05d0071ea0 0x7f05d0110170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:45:17.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.134+0000 7f05d5ad9700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d0110770 con 0x7f05d00708f0 2024-04-22T23:45:17.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.134+0000 7f05d5ad9700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d0110fc0 con 0x7f05d0071ea0 2024-04-22T23:45:17.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.134+0000 7f05cf7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05d00708f0 0x7f05d010fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:45:17.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.134+0000 7f05cf7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05d00708f0 0x7f05d010fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55748/0 (socket says 172.21.15.107:55748) 2024-04-22T23:45:17.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.134+0000 7f05cf7fe700 1 -- 172.21.15.107:0/3560378666 learned_addr learned my addr 172.21.15.107:0/3560378666 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:45:17.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.134+0000 7f05ceffd700 1 --2- 172.21.15.107:0/3560378666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05d0071ea0 0x7f05d0110170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:45:17.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.134+0000 7f05cf7fe700 1 -- 172.21.15.107:0/3560378666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05d0071ea0 msgr2=0x7f05d0110170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:17.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.134+0000 7f05cf7fe700 1 --2- 172.21.15.107:0/3560378666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05d0071ea0 0x7f05d0110170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:17.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.134+0000 7f05cf7fe700 1 -- 172.21.15.107:0/3560378666 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05c0005040 con 0x7f05d00708f0 2024-04-22T23:45:17.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.134+0000 7f05cf7fe700 1 --2- 172.21.15.107:0/3560378666 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05d00708f0 0x7f05d010fc30 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f05c800ca90 tx=0x7f05c8007f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:45:17.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.135+0000 7f05b7fff700 1 -- 172.21.15.107:0/3560378666 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05c800a980 con 0x7f05d00708f0 2024-04-22T23:45:17.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.135+0000 7f05d5ad9700 1 -- 172.21.15.107:0/3560378666 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05d01112a0 con 0x7f05d00708f0 2024-04-22T23:45:17.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.135+0000 7f05d5ad9700 1 -- 172.21.15.107:0/3560378666 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05d01117e0 con 0x7f05d00708f0 2024-04-22T23:45:17.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.135+0000 7f05b7fff700 1 -- 172.21.15.107:0/3560378666 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f05c8012440 con 0x7f05d00708f0 2024-04-22T23:45:17.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.135+0000 7f05b7fff700 1 -- 172.21.15.107:0/3560378666 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05c80115e0 con 0x7f05d00708f0 2024-04-22T23:45:17.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.136+0000 7f05d5ad9700 1 -- 172.21.15.107:0/3560378666 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05bc0052e0 con 0x7f05d00708f0 2024-04-22T23:45:17.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.137+0000 7f05b7fff700 1 -- 172.21.15.107:0/3560378666 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f05c8012a90 con 0x7f05d00708f0 2024-04-22T23:45:17.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.137+0000 7f05b7fff700 1 --2- 172.21.15.107:0/3560378666 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f05b806ec00 0x7f05b80710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:45:17.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.137+0000 7f05b7fff700 1 -- 172.21.15.107:0/3560378666 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f05c808eff0 con 0x7f05d00708f0 2024-04-22T23:45:17.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.137+0000 7f05ceffd700 1 --2- 172.21.15.107:0/3560378666 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f05b806ec00 0x7f05b80710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:45:17.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.137+0000 7f05ceffd700 1 --2- 172.21.15.107:0/3560378666 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f05b806ec00 0x7f05b80710b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f05c000b3b0 tx=0x7f05c0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:45:17.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.143+0000 7f05b7fff700 1 -- 172.21.15.107:0/3560378666 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f05c805b820 con 0x7f05d00708f0 2024-04-22T23:45:17.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.475+0000 7f05d5ad9700 1 -- 172.21.15.107:0/3560378666 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f05bc005590 con 0x7f05d00708f0 2024-04-22T23:45:17.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.475+0000 7f05b7fff700 1 -- 172.21.15.107:0/3560378666 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f05c805b3b0 con 0x7f05d00708f0 2024-04-22T23:45:17.468 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_OK 2024-04-22T23:45:17.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.477+0000 7f05d5ad9700 1 -- 172.21.15.107:0/3560378666 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f05b806ec00 msgr2=0x7f05b80710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:17.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.477+0000 7f05d5ad9700 1 --2- 172.21.15.107:0/3560378666 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f05b806ec00 0x7f05b80710b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f05c000b3b0 tx=0x7f05c0009e30 comp rx=0 tx=0).stop 2024-04-22T23:45:17.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.477+0000 7f05d5ad9700 1 -- 172.21.15.107:0/3560378666 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05d00708f0 msgr2=0x7f05d010fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:17.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.477+0000 7f05d5ad9700 1 --2- 172.21.15.107:0/3560378666 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05d00708f0 0x7f05d010fc30 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f05c800ca90 tx=0x7f05c8007f00 comp rx=0 tx=0).stop 2024-04-22T23:45:17.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.478+0000 7f05d5ad9700 1 -- 172.21.15.107:0/3560378666 shutdown_connections 2024-04-22T23:45:17.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.478+0000 7f05d5ad9700 1 --2- 172.21.15.107:0/3560378666 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f05b806ec00 0x7f05b80710b0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:17.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.478+0000 7f05d5ad9700 1 --2- 172.21.15.107:0/3560378666 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05d00708f0 0x7f05d010fc30 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:17.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.478+0000 7f05d5ad9700 1 --2- 172.21.15.107:0/3560378666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05d0071ea0 0x7f05d0110170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:17.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.478+0000 7f05d5ad9700 1 -- 172.21.15.107:0/3560378666 >> 172.21.15.107:0/3560378666 conn(0x7f05d00f8570 msgr2=0x7f05d00f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:45:17.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.478+0000 7f05d5ad9700 1 -- 172.21.15.107:0/3560378666 shutdown_connections 2024-04-22T23:45:17.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:17.478+0000 7f05d5ad9700 1 -- 172.21.15.107:0/3560378666 wait complete. 2024-04-22T23:45:17.679 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:17 smithi107 ceph-mon[90179]: from='client.34538 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:45:17.679 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:17 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/2922494954' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:45:17.679 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:17 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/3560378666' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:45:17.879 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:17 smithi157 ceph-mon[79403]: from='client.34538 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:45:17.879 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:17 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/2922494954' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:45:17.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:17 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/3560378666' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:45:18.731 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:18 smithi107 ceph-mon[90179]: from='client.34550 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:45:18.731 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:18 smithi107 ceph-mon[90179]: pgmap v258: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:18.731 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:18 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:45:18.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:18 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:45:18.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:18 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:45:18.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:18 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:45:18.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:18 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:45:18.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:18 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-22T23:45:18.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:18 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi107:9093"}]: dispatch 2024-04-22T23:45:18.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:18 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:45:18.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:18 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:45:18.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:18 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:18.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:18 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:18.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:18 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:18.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:18 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:18.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:18 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:18.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:18 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:18.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:18 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:18.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:18 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:18.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:18 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:18.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:18 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:18.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:18 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:18.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:18 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:18.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:18 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:18.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:18 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:18.733 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:18 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:18.879 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:18 smithi157 ceph-mon[79403]: from='client.34550 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:45:18.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:18 smithi157 ceph-mon[79403]: pgmap v258: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:18.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:18 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:45:18.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:18 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:45:18.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:18 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:45:18.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:18 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:45:18.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:18 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:45:18.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:18 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-22T23:45:18.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:18 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi107:9093"}]: dispatch 2024-04-22T23:45:18.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:18 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:45:18.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:18 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:45:18.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:18 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:18.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:18 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:18.880 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:18 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:18.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:18 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:18.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:18 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:18.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:18 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:18.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:18 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:18.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:18 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:18.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:18 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:18.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:18 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:18.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:18 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:18.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:18 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:18.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:18 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:18.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:18 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:18.881 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:18 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:19.731 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:19 smithi107 ceph-mon[90179]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-22T23:45:19.731 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:19 smithi107 ceph-mon[90179]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi107:9093"}]: dispatch 2024-04-22T23:45:19.879 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:19 smithi157 ceph-mon[79403]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-22T23:45:19.879 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:19 smithi157 ceph-mon[79403]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi107:9093"}]: dispatch 2024-04-22T23:45:20.731 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:20 smithi107 ceph-mon[90179]: pgmap v259: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:20.879 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:20 smithi157 ceph-mon[79403]: pgmap v259: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:21.879 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:21 smithi157 ceph-mon[79403]: Upgrade: Updating grafana.smithi107 2024-04-22T23:45:21.879 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:21 smithi157 ceph-mon[79403]: Deploying daemon grafana.smithi107 on smithi107 2024-04-22T23:45:21.981 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:21 smithi107 ceph-mon[90179]: Upgrade: Updating grafana.smithi107 2024-04-22T23:45:21.981 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:21 smithi107 ceph-mon[90179]: Deploying daemon grafana.smithi107 on smithi107 2024-04-22T23:45:22.879 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:22 smithi157 ceph-mon[79403]: pgmap v260: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:22.981 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:22 smithi107 ceph-mon[90179]: pgmap v260: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:25.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:24 smithi157 ceph-mon[79403]: pgmap v261: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:25.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:24 smithi107 ceph-mon[90179]: pgmap v261: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:26.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:25 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:45:26.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:25 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:45:27.116 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:26 smithi107 ceph-mon[90179]: pgmap v262: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:27.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:26 smithi157 ceph-mon[79403]: pgmap v262: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:29.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:28 smithi157 ceph-mon[79403]: pgmap v263: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:29.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:28 smithi107 ceph-mon[90179]: pgmap v263: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:31.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:30 smithi157 ceph-mon[79403]: pgmap v264: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:31.184 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:30 smithi107 ceph-mon[90179]: pgmap v264: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:32.981 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:32 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:45:32.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:32 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:45:32.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:32 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:45:32.982 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:32 smithi107 ceph-mon[90179]: pgmap v265: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:33.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:32 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:45:33.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:32 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:45:33.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:32 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:45:33.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:32 smithi157 ceph-mon[79403]: pgmap v265: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:35.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:34 smithi157 ceph-mon[79403]: pgmap v266: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:35.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:34 smithi107 ceph-mon[90179]: pgmap v266: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:37.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:36 smithi157 ceph-mon[79403]: pgmap v267: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:37.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:36 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:45:37.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:36 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:45:37.201 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:36 smithi107 ceph-mon[90179]: pgmap v267: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:37.201 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:36 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:45:37.201 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:36 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:45:38.731 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:38 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:45:38.731 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:38 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:45:38.732 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:38 smithi107 ceph-mon[90179]: pgmap v268: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:38.879 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:38 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:45:38.879 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:38 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:45:38.879 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:38 smithi157 ceph-mon[79403]: pgmap v268: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:41.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:40 smithi157 ceph-mon[79403]: pgmap v269: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:41.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:40 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:45:41.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:40 smithi107 ceph-mon[90179]: pgmap v269: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:41.232 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:40 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:45:43.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:42 smithi157 ceph-mon[79403]: pgmap v270: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:43.211 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:42 smithi107 ceph-mon[90179]: pgmap v270: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:44.274 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:45:44.274 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:45:44.274 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:45:44.274 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:45:44.274 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:45:44.274 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-22T23:45:44.274 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-22T23:45:44.274 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi107:3000"}]: dispatch 2024-04-22T23:45:44.274 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi107:3000"}]: dispatch 2024-04-22T23:45:44.275 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:45:44.275 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:45:44.275 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:44.275 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:44.275 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:44.275 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:44.275 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:44.275 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:44.275 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:44.275 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:44.275 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:44.276 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:44.276 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:44.276 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:44.276 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:44.276 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:44.276 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:44.276 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:44.276 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:44.276 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:44.276 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: Upgrade: Finalizing container_image settings 2024-04-22T23:45:44.276 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:45:44.276 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-22T23:45:44.276 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-22T23:45:44.276 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-22T23:45:44.276 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-22T23:45:44.277 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-22T23:45:44.277 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-04-22T23:45:44.277 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-22T23:45:44.277 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-04-22T23:45:44.277 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-22T23:45:44.277 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-04-22T23:45:44.277 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-22T23:45:44.277 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-22T23:45:44.277 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-22T23:45:44.277 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-22T23:45:44.277 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-22T23:45:44.277 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-22T23:45:44.277 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-04-22T23:45:44.278 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-22T23:45:44.278 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-22T23:45:44.278 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-22T23:45:44.278 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-22T23:45:44.278 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-22T23:45:44.278 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-22T23:45:44.278 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-22T23:45:44.278 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-22T23:45:44.278 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-22T23:45:44.278 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-22T23:45:44.278 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-22T23:45:44.278 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-22T23:45:44.278 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: Upgrade: Complete! 2024-04-22T23:45:44.278 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-22T23:45:44.279 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-22T23:45:44.279 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:45:44.279 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: pgmap v271: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:44.279 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:45:44.279 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:45:44.279 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:45:44.279 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:45:44.279 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:45:44.279 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:45:44.279 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:45:44.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:45:44.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:45:44.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:45:44.630 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:45:44.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:45:44.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-22T23:45:44.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-22T23:45:44.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi107:3000"}]: dispatch 2024-04-22T23:45:44.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi107:3000"}]: dispatch 2024-04-22T23:45:44.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:45:44.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:45:44.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:44.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:44.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:44.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:44.631 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:44.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:44.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:44.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:44.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:44.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:44.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:44.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:44.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:44.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:44.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:44.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:44.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:44.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:45:44.632 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: Upgrade: Finalizing container_image settings 2024-04-22T23:45:44.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:45:44.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-22T23:45:44.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-22T23:45:44.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-22T23:45:44.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-22T23:45:44.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-22T23:45:44.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-04-22T23:45:44.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-22T23:45:44.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-04-22T23:45:44.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-22T23:45:44.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-04-22T23:45:44.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-22T23:45:44.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-22T23:45:44.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-22T23:45:44.633 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-22T23:45:44.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-22T23:45:44.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-22T23:45:44.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-04-22T23:45:44.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-22T23:45:44.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-22T23:45:44.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-22T23:45:44.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-22T23:45:44.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-22T23:45:44.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-22T23:45:44.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-22T23:45:44.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-22T23:45:44.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-22T23:45:44.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-22T23:45:44.634 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-22T23:45:44.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-22T23:45:44.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: Upgrade: Complete! 2024-04-22T23:45:44.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-22T23:45:44.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-22T23:45:44.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:45:44.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: pgmap v271: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:44.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:45:44.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:45:44.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:45:44.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:45:44.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:45:44.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:45:44.635 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:45:47.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:46 smithi157 ceph-mon[79403]: pgmap v272: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:47.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:46 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:45:47.201 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:46 smithi107 ceph-mon[90179]: pgmap v272: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:47.201 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:46 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:45:47.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:47.599+0000 7f28bd4b9700 1 -- 172.21.15.107:0/281910074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28b80fcf90 msgr2=0x7f28b80fd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:47.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:47.599+0000 7f28bd4b9700 1 --2- 172.21.15.107:0/281910074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28b80fcf90 0x7f28b80fd400 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f28ac004940 tx=0x7f28ac009e30 comp rx=0 tx=0).stop 2024-04-22T23:45:47.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:47.599+0000 7f28bd4b9700 1 -- 172.21.15.107:0/281910074 shutdown_connections 2024-04-22T23:45:47.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:47.599+0000 7f28bd4b9700 1 --2- 172.21.15.107:0/281910074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28b80fcf90 0x7f28b80fd400 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:47.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:47.599+0000 7f28bd4b9700 1 --2- 172.21.15.107:0/281910074 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28b80fbce0 0x7f28b80fc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:47.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:47.599+0000 7f28bd4b9700 1 -- 172.21.15.107:0/281910074 >> 172.21.15.107:0/281910074 conn(0x7f28b80f7440 msgr2=0x7f28b80f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:45:47.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:47.599+0000 7f28bd4b9700 1 -- 172.21.15.107:0/281910074 shutdown_connections 2024-04-22T23:45:47.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:47.599+0000 7f28bd4b9700 1 -- 172.21.15.107:0/281910074 wait complete. 2024-04-22T23:45:47.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:47.599+0000 7f28bd4b9700 1 Processor -- start 2024-04-22T23:45:47.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:47.599+0000 7f28bd4b9700 1 -- start start 2024-04-22T23:45:47.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:47.600+0000 7f28bd4b9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28b80fbce0 0x7f28b8110730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:45:47.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:47.600+0000 7f28bd4b9700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28b80fcf90 0x7f28b8112c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:45:47.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:47.600+0000 7f28bd4b9700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b8066bf0 con 0x7f28b80fbce0 2024-04-22T23:45:47.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:47.600+0000 7f28bd4b9700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b8066d30 con 0x7f28b80fcf90 2024-04-22T23:45:47.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:47.600+0000 7f28b6ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28b80fbce0 0x7f28b8110730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:45:47.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:47.600+0000 7f28b6ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28b80fbce0 0x7f28b8110730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47668/0 (socket says 172.21.15.107:47668) 2024-04-22T23:45:47.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:47.600+0000 7f28b6ffd700 1 -- 172.21.15.107:0/4004839837 learned_addr learned my addr 172.21.15.107:0/4004839837 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:45:47.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:47.600+0000 7f28b67fc700 1 --2- 172.21.15.107:0/4004839837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28b80fcf90 0x7f28b8112c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:45:47.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:47.600+0000 7f28b6ffd700 1 -- 172.21.15.107:0/4004839837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28b80fcf90 msgr2=0x7f28b8112c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:47.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:47.600+0000 7f28b6ffd700 1 --2- 172.21.15.107:0/4004839837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28b80fcf90 0x7f28b8112c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:47.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:47.600+0000 7f28b6ffd700 1 -- 172.21.15.107:0/4004839837 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28ac005040 con 0x7f28b80fbce0 2024-04-22T23:45:47.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:47.600+0000 7f28b6ffd700 1 --2- 172.21.15.107:0/4004839837 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28b80fbce0 0x7f28b8110730 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f28a800c830 tx=0x7f28a8004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:45:47.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:47.600+0000 7f289f7fe700 1 -- 172.21.15.107:0/4004839837 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28a800a760 con 0x7f28b80fbce0 2024-04-22T23:45:47.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:47.600+0000 7f289f7fe700 1 -- 172.21.15.107:0/4004839837 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f28a800ad90 con 0x7f28b80fbce0 2024-04-22T23:45:47.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:47.600+0000 7f28bd4b9700 1 -- 172.21.15.107:0/4004839837 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28b8113220 con 0x7f28b80fbce0 2024-04-22T23:45:47.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:47.600+0000 7f289f7fe700 1 -- 172.21.15.107:0/4004839837 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28a80113a0 con 0x7f28b80fbce0 2024-04-22T23:45:47.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:47.601+0000 7f28bd4b9700 1 -- 172.21.15.107:0/4004839837 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28b81137e0 con 0x7f28b80fbce0 2024-04-22T23:45:47.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:47.601+0000 7f28bd4b9700 1 -- 172.21.15.107:0/4004839837 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28b805f5f0 con 0x7f28b80fbce0 2024-04-22T23:45:47.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:47.602+0000 7f289f7fe700 1 -- 172.21.15.107:0/4004839837 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f28a8012400 con 0x7f28b80fbce0 2024-04-22T23:45:47.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:47.602+0000 7f289f7fe700 1 --2- 172.21.15.107:0/4004839837 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f28a006ec00 0x7f28a00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:45:47.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:47.603+0000 7f289f7fe700 1 -- 172.21.15.107:0/4004839837 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f28a808e9a0 con 0x7f28b80fbce0 2024-04-22T23:45:47.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:47.603+0000 7f28b67fc700 1 --2- 172.21.15.107:0/4004839837 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f28a006ec00 0x7f28a00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:45:47.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:47.603+0000 7f28b67fc700 1 --2- 172.21.15.107:0/4004839837 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f28a006ec00 0x7f28a00710b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f28ac00b3b0 tx=0x7f28ac009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:45:47.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:47.607+0000 7f289f7fe700 1 -- 172.21.15.107:0/4004839837 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f28a8057720 con 0x7f28b80fbce0 2024-04-22T23:45:47.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:47.828+0000 7f28bd4b9700 1 -- 172.21.15.107:0/4004839837 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f28b81012b0 con 0x7f28a006ec00 2024-04-22T23:45:47.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:47.829+0000 7f289f7fe700 1 -- 172.21.15.107:0/4004839837 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f28b81012b0 con 0x7f28a006ec00 2024-04-22T23:45:47.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:47.831+0000 7f28bd4b9700 1 -- 172.21.15.107:0/4004839837 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f28a006ec00 msgr2=0x7f28a00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:47.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:47.831+0000 7f28bd4b9700 1 --2- 172.21.15.107:0/4004839837 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f28a006ec00 0x7f28a00710b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f28ac00b3b0 tx=0x7f28ac009e30 comp rx=0 tx=0).stop 2024-04-22T23:45:47.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:47.831+0000 7f28bd4b9700 1 -- 172.21.15.107:0/4004839837 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28b80fbce0 msgr2=0x7f28b8110730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:47.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:47.831+0000 7f28bd4b9700 1 --2- 172.21.15.107:0/4004839837 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28b80fbce0 0x7f28b8110730 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f28a800c830 tx=0x7f28a8004470 comp rx=0 tx=0).stop 2024-04-22T23:45:47.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:47.831+0000 7f28bd4b9700 1 -- 172.21.15.107:0/4004839837 shutdown_connections 2024-04-22T23:45:47.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:47.831+0000 7f28bd4b9700 1 --2- 172.21.15.107:0/4004839837 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f28a006ec00 0x7f28a00710b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:47.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:47.831+0000 7f28bd4b9700 1 --2- 172.21.15.107:0/4004839837 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28b80fbce0 0x7f28b8110730 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:47.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:47.831+0000 7f28bd4b9700 1 --2- 172.21.15.107:0/4004839837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28b80fcf90 0x7f28b8112c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:47.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:47.831+0000 7f28bd4b9700 1 -- 172.21.15.107:0/4004839837 >> 172.21.15.107:0/4004839837 conn(0x7f28b80f7440 msgr2=0x7f28b80f8b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:45:47.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:47.832+0000 7f28bd4b9700 1 -- 172.21.15.107:0/4004839837 shutdown_connections 2024-04-22T23:45:47.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:47.832+0000 7f28bd4b9700 1 -- 172.21.15.107:0/4004839837 wait complete. 2024-04-22T23:45:48.456 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -e sha1=43be020184947e53516056c9931e1ac5bdbbb1a5 -- bash -c 'ceph orch ps' 2024-04-22T23:45:48.754 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:45:49.126 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:48 smithi107 ceph-mon[90179]: pgmap v273: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:49.126 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:48 smithi107 ceph-mon[90179]: from='client.34558 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:45:49.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:48 smithi157 ceph-mon[79403]: pgmap v273: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:49.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:48 smithi157 ceph-mon[79403]: from='client.34558 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:45:50.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:50.417+0000 7f7e9dba8700 1 -- 172.21.15.107:0/2399849549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e981037c0 msgr2=0x7f7e980fce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:50.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:50.417+0000 7f7e9dba8700 1 --2- 172.21.15.107:0/2399849549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e981037c0 0x7f7e980fce30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c004940 tx=0x7f7e8c009e30 comp rx=0 tx=0).stop 2024-04-22T23:45:50.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:50.417+0000 7f7e9dba8700 1 -- 172.21.15.107:0/2399849549 shutdown_connections 2024-04-22T23:45:50.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:50.417+0000 7f7e9dba8700 1 --2- 172.21.15.107:0/2399849549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e981037c0 0x7f7e980fce30 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:50.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:50.417+0000 7f7e9dba8700 1 --2- 172.21.15.107:0/2399849549 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e98102e20 0x7f7e981031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:50.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:50.417+0000 7f7e9dba8700 1 -- 172.21.15.107:0/2399849549 >> 172.21.15.107:0/2399849549 conn(0x7f7e980f89b0 msgr2=0x7f7e980fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:45:50.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:50.418+0000 7f7e9dba8700 1 -- 172.21.15.107:0/2399849549 shutdown_connections 2024-04-22T23:45:50.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:50.418+0000 7f7e9dba8700 1 -- 172.21.15.107:0/2399849549 wait complete. 2024-04-22T23:45:50.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:50.418+0000 7f7e9dba8700 1 Processor -- start 2024-04-22T23:45:50.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:50.418+0000 7f7e9dba8700 1 -- start start 2024-04-22T23:45:50.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:50.418+0000 7f7e9dba8700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e98102e20 0x7f7e981139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:45:50.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:50.419+0000 7f7e9dba8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e981037c0 0x7f7e98113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:45:50.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:50.419+0000 7f7e9dba8700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e98114530 con 0x7f7e981037c0 2024-04-22T23:45:50.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:50.419+0000 7f7e9dba8700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9810db50 con 0x7f7e98102e20 2024-04-22T23:45:50.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:50.419+0000 7f7e96ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e981037c0 0x7f7e98113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:45:50.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:50.419+0000 7f7e96ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e981037c0 0x7f7e98113ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47696/0 (socket says 172.21.15.107:47696) 2024-04-22T23:45:50.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:50.419+0000 7f7e96ffd700 1 -- 172.21.15.107:0/1451484862 learned_addr learned my addr 172.21.15.107:0/1451484862 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:45:50.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:50.419+0000 7f7e96ffd700 1 -- 172.21.15.107:0/1451484862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e98102e20 msgr2=0x7f7e981139a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:50.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:50.419+0000 7f7e96ffd700 1 --2- 172.21.15.107:0/1451484862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e98102e20 0x7f7e981139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:50.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:50.419+0000 7f7e96ffd700 1 -- 172.21.15.107:0/1451484862 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e8c005040 con 0x7f7e981037c0 2024-04-22T23:45:50.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:50.419+0000 7f7e96ffd700 1 --2- 172.21.15.107:0/1451484862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e981037c0 0x7f7e98113ee0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c009100 tx=0x7f7e8c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:45:50.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:50.419+0000 7f7e7ffff700 1 -- 172.21.15.107:0/1451484862 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e8c00e040 con 0x7f7e981037c0 2024-04-22T23:45:50.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:50.419+0000 7f7e9dba8700 1 -- 172.21.15.107:0/1451484862 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e9810ddd0 con 0x7f7e981037c0 2024-04-22T23:45:50.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:50.419+0000 7f7e7ffff700 1 -- 172.21.15.107:0/1451484862 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7e8c013420 con 0x7f7e981037c0 2024-04-22T23:45:50.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:50.419+0000 7f7e7ffff700 1 -- 172.21.15.107:0/1451484862 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e8c01c610 con 0x7f7e981037c0 2024-04-22T23:45:50.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:50.419+0000 7f7e9dba8700 1 -- 172.21.15.107:0/1451484862 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e9810e390 con 0x7f7e981037c0 2024-04-22T23:45:50.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:50.420+0000 7f7e9dba8700 1 -- 172.21.15.107:0/1451484862 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e981052f0 con 0x7f7e981037c0 2024-04-22T23:45:50.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:50.421+0000 7f7e7ffff700 1 -- 172.21.15.107:0/1451484862 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f7e8c00baf0 con 0x7f7e981037c0 2024-04-22T23:45:50.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:50.421+0000 7f7e7ffff700 1 --2- 172.21.15.107:0/1451484862 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f7e8006ec00 0x7f7e800710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:45:50.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:50.421+0000 7f7e7ffff700 1 -- 172.21.15.107:0/1451484862 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f7e8c08fc10 con 0x7f7e981037c0 2024-04-22T23:45:50.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:50.421+0000 7f7e977fe700 1 --2- 172.21.15.107:0/1451484862 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f7e8006ec00 0x7f7e800710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:45:50.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:50.421+0000 7f7e977fe700 1 --2- 172.21.15.107:0/1451484862 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f7e8006ec00 0x7f7e800710b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7e880041d0 tx=0x7f7e88004160 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:45:50.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:50.426+0000 7f7e7ffff700 1 -- 172.21.15.107:0/1451484862 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f7e8c058990 con 0x7f7e981037c0 2024-04-22T23:45:50.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:50.635+0000 7f7e9dba8700 1 -- 172.21.15.107:0/1451484862 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7e9810f060 con 0x7f7e8006ec00 2024-04-22T23:45:50.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:50.645+0000 7f7e7ffff700 1 -- 172.21.15.107:0/1451484862 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f7e9810f060 con 0x7f7e8006ec00 2024-04-22T23:45:50.638 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-22T23:45:50.638 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (42s) 14s ago 18m 14.6M - 0.25.0 c8568f914cd2 07fa1ebb7f47 2024-04-22T23:45:50.638 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (6m) 14s ago 19m 6656k - 19.0.0-3252-g43be0201 a0434699c1d7 e074fd2cbb9c 2024-04-22T23:45:50.638 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi157 smithi157 running (6m) 72s ago 18m 6651k - 19.0.0-3252-g43be0201 a0434699c1d7 abc2649dee22 2024-04-22T23:45:50.638 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (19s) 14s ago 18m 54.8M - 9.4.12 2bacad6d85d8 adc26d364256 2024-04-22T23:45:50.638 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.dfvvkw smithi107 running (2m) 14s ago 13m 70.4M - 19.0.0-3252-g43be0201 a0434699c1d7 7566b5cc3d01 2024-04-22T23:45:50.638 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.ktuxjl smithi107 running (2m) 14s ago 13m 18.3M - 19.0.0-3252-g43be0201 a0434699c1d7 ce23545f0ae2 2024-04-22T23:45:50.638 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi157.rcaris smithi157 running (115s) 72s ago 13m 11.9M - 19.0.0-3252-g43be0201 a0434699c1d7 12550073a997 2024-04-22T23:45:50.638 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi157.ufywse smithi157 running (92s) 72s ago 13m 14.1M - 19.0.0-3252-g43be0201 a0434699c1d7 e9f97f60ed21 2024-04-22T23:45:50.638 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.uipxvm smithi107 *:8443,9283,8765 running (8m) 14s ago 20m 522M - 19.0.0-3252-g43be0201 a0434699c1d7 d20f05dfe4f2 2024-04-22T23:45:50.639 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi157.tuylis smithi157 *:8443,9283,8765 running (8m) 72s ago 17m 452M - 19.0.0-3252-g43be0201 a0434699c1d7 957e598c9001 2024-04-22T23:45:50.639 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (8m) 14s ago 20m 60.1M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 bc72bbbe2c39 2024-04-22T23:45:50.639 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi157 smithi157 running (7m) 72s ago 17m 41.2M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 ea88de8be4fe 2024-04-22T23:45:50.639 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (79s) 14s ago 18m 8870k - 1.5.0 0da6a335fe13 c338e5470944 2024-04-22T23:45:50.639 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi157 smithi157 *:9100 running (75s) 72s ago 18m 3389k - 1.5.0 0da6a335fe13 8d0255aff39c 2024-04-22T23:45:50.639 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (6m) 14s ago 17m 104M 1557M 19.0.0-3252-g43be0201 a0434699c1d7 fd0ba8ef0c27 2024-04-22T23:45:50.639 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (5m) 14s ago 16m 104M 1557M 19.0.0-3252-g43be0201 a0434699c1d7 77e33b6122e0 2024-04-22T23:45:50.639 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (5m) 14s ago 16m 99.2M 1557M 19.0.0-3252-g43be0201 a0434699c1d7 c307bd8cdcfc 2024-04-22T23:45:50.639 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi157 running (4m) 72s ago 15m 112M 2581M 19.0.0-3252-g43be0201 a0434699c1d7 e81ca519c831 2024-04-22T23:45:50.639 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi157 running (4m) 72s ago 15m 105M 2581M 19.0.0-3252-g43be0201 a0434699c1d7 d5b5a2e4781a 2024-04-22T23:45:50.639 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi157 running (3m) 72s ago 14m 97.1M 2581M 19.0.0-3252-g43be0201 a0434699c1d7 0ee423135fdb 2024-04-22T23:45:50.639 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (58s) 14s ago 18m 47.6M - 2.43.0 a07b618ecd1d f6160052545f 2024-04-22T23:45:50.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:50.647+0000 7f7e9dba8700 1 -- 172.21.15.107:0/1451484862 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f7e8006ec00 msgr2=0x7f7e800710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:50.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:50.647+0000 7f7e9dba8700 1 --2- 172.21.15.107:0/1451484862 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f7e8006ec00 0x7f7e800710b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7e880041d0 tx=0x7f7e88004160 comp rx=0 tx=0).stop 2024-04-22T23:45:50.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:50.647+0000 7f7e9dba8700 1 -- 172.21.15.107:0/1451484862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e981037c0 msgr2=0x7f7e98113ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:50.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:50.647+0000 7f7e9dba8700 1 --2- 172.21.15.107:0/1451484862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e981037c0 0x7f7e98113ee0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c009100 tx=0x7f7e8c009e30 comp rx=0 tx=0).stop 2024-04-22T23:45:50.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:50.647+0000 7f7e9dba8700 1 -- 172.21.15.107:0/1451484862 shutdown_connections 2024-04-22T23:45:50.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:50.647+0000 7f7e9dba8700 1 --2- 172.21.15.107:0/1451484862 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e98102e20 0x7f7e981139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:50.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:50.647+0000 7f7e9dba8700 1 --2- 172.21.15.107:0/1451484862 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f7e8006ec00 0x7f7e800710b0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:50.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:50.647+0000 7f7e9dba8700 1 --2- 172.21.15.107:0/1451484862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e981037c0 0x7f7e98113ee0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:50.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:50.647+0000 7f7e9dba8700 1 -- 172.21.15.107:0/1451484862 >> 172.21.15.107:0/1451484862 conn(0x7f7e980f89b0 msgr2=0x7f7e981021e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:45:50.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:50.648+0000 7f7e9dba8700 1 -- 172.21.15.107:0/1451484862 shutdown_connections 2024-04-22T23:45:50.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:50.648+0000 7f7e9dba8700 1 -- 172.21.15.107:0/1451484862 wait complete. 2024-04-22T23:45:50.962 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:50 smithi107 ceph-mon[90179]: pgmap v274: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:51.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:50 smithi157 ceph-mon[79403]: pgmap v274: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:51.279 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -e sha1=43be020184947e53516056c9931e1ac5bdbbb1a5 -- bash -c 'ceph orch upgrade status' 2024-04-22T23:45:51.533 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:45:51.908 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:51 smithi107 ceph-mon[90179]: from='client.34562 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:45:52.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:51 smithi157 ceph-mon[79403]: from='client.34562 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:45:52.858 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:52 smithi107 ceph-mon[90179]: pgmap v275: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:53.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:53.062+0000 7ff5b6506700 1 -- 172.21.15.107:0/2976029302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5b0102e20 msgr2=0x7ff5b01031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:53.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:53.062+0000 7ff5b6506700 1 --2- 172.21.15.107:0/2976029302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5b0102e20 0x7ff5b01031f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0004940 tx=0x7ff5a0009e30 comp rx=0 tx=0).stop 2024-04-22T23:45:53.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:53.062+0000 7ff5b6506700 1 -- 172.21.15.107:0/2976029302 shutdown_connections 2024-04-22T23:45:53.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:53.062+0000 7ff5b6506700 1 --2- 172.21.15.107:0/2976029302 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5b01037c0 0x7ff5b00fce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:53.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:53.062+0000 7ff5b6506700 1 --2- 172.21.15.107:0/2976029302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5b0102e20 0x7ff5b01031f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:53.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:53.062+0000 7ff5b6506700 1 -- 172.21.15.107:0/2976029302 >> 172.21.15.107:0/2976029302 conn(0x7ff5b00f89b0 msgr2=0x7ff5b00fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:45:53.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:53.063+0000 7ff5b6506700 1 -- 172.21.15.107:0/2976029302 shutdown_connections 2024-04-22T23:45:53.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:53.063+0000 7ff5b6506700 1 -- 172.21.15.107:0/2976029302 wait complete. 2024-04-22T23:45:53.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:53.063+0000 7ff5b6506700 1 Processor -- start 2024-04-22T23:45:53.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:53.063+0000 7ff5b6506700 1 -- start start 2024-04-22T23:45:53.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:53.064+0000 7ff5b6506700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5b0102e20 0x7ff5b0115b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:45:53.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:53.064+0000 7ff5b6506700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5b01037c0 0x7ff5b01160d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:45:53.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:53.064+0000 7ff5b6506700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b0116720 con 0x7ff5b01037c0 2024-04-22T23:45:53.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:53.064+0000 7ff5b6506700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b010fd00 con 0x7ff5b0102e20 2024-04-22T23:45:53.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:53.064+0000 7ff5af7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5b01037c0 0x7ff5b01160d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:45:53.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:53.064+0000 7ff5af7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5b01037c0 0x7ff5b01160d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41554/0 (socket says 172.21.15.107:41554) 2024-04-22T23:45:53.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:53.064+0000 7ff5af7fe700 1 -- 172.21.15.107:0/93451234 learned_addr learned my addr 172.21.15.107:0/93451234 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:45:53.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:53.064+0000 7ff5affff700 1 --2- 172.21.15.107:0/93451234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5b0102e20 0x7ff5b0115b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:45:53.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:53.064+0000 7ff5af7fe700 1 -- 172.21.15.107:0/93451234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5b0102e20 msgr2=0x7ff5b0115b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:53.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:53.064+0000 7ff5af7fe700 1 --2- 172.21.15.107:0/93451234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5b0102e20 0x7ff5b0115b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:53.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:53.064+0000 7ff5af7fe700 1 -- 172.21.15.107:0/93451234 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a0005040 con 0x7ff5b01037c0 2024-04-22T23:45:53.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:53.064+0000 7ff5affff700 1 --2- 172.21.15.107:0/93451234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5b0102e20 0x7ff5b0115b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T23:45:53.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:53.064+0000 7ff5af7fe700 1 --2- 172.21.15.107:0/93451234 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5b01037c0 0x7ff5b01160d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4002a50 tx=0x7ff5a4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:45:53.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:53.064+0000 7ff5acff9700 1 -- 172.21.15.107:0/93451234 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5a4007cc0 con 0x7ff5b01037c0 2024-04-22T23:45:53.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:53.064+0000 7ff5b6506700 1 -- 172.21.15.107:0/93451234 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5b010ffe0 con 0x7ff5b01037c0 2024-04-22T23:45:53.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:53.064+0000 7ff5acff9700 1 -- 172.21.15.107:0/93451234 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff5a400f040 con 0x7ff5b01037c0 2024-04-22T23:45:53.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:53.064+0000 7ff5b6506700 1 -- 172.21.15.107:0/93451234 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5b01105a0 con 0x7ff5b01037c0 2024-04-22T23:45:53.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:53.065+0000 7ff5acff9700 1 -- 172.21.15.107:0/93451234 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5a400da40 con 0x7ff5b01037c0 2024-04-22T23:45:53.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:53.065+0000 7ff5b6506700 1 -- 172.21.15.107:0/93451234 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5b0107320 con 0x7ff5b01037c0 2024-04-22T23:45:53.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:53.066+0000 7ff5acff9700 1 -- 172.21.15.107:0/93451234 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7ff5a40097c0 con 0x7ff5b01037c0 2024-04-22T23:45:53.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:53.066+0000 7ff5acff9700 1 --2- 172.21.15.107:0/93451234 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7ff59806ec00 0x7ff5980710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:45:53.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:53.067+0000 7ff5acff9700 1 -- 172.21.15.107:0/93451234 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7ff5a408e760 con 0x7ff5b01037c0 2024-04-22T23:45:53.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:53.067+0000 7ff5affff700 1 --2- 172.21.15.107:0/93451234 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7ff59806ec00 0x7ff5980710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:45:53.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:53.067+0000 7ff5affff700 1 --2- 172.21.15.107:0/93451234 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7ff59806ec00 0x7ff5980710b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0004560 tx=0x7ff5a0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:45:53.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:53.071+0000 7ff5acff9700 1 -- 172.21.15.107:0/93451234 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7ff5a405af90 con 0x7ff5b01037c0 2024-04-22T23:45:53.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:52 smithi157 ceph-mon[79403]: pgmap v275: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:53.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:53.289+0000 7ff5b6506700 1 -- 172.21.15.107:0/93451234 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff5b01111e0 con 0x7ff59806ec00 2024-04-22T23:45:53.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:53.291+0000 7ff5acff9700 1 -- 172.21.15.107:0/93451234 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7ff5b01111e0 con 0x7ff59806ec00 2024-04-22T23:45:53.283 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-22T23:45:53.284 INFO:teuthology.orchestra.run.smithi107.stdout: "target_image": null, 2024-04-22T23:45:53.284 INFO:teuthology.orchestra.run.smithi107.stdout: "in_progress": false, 2024-04-22T23:45:53.284 INFO:teuthology.orchestra.run.smithi107.stdout: "which": "", 2024-04-22T23:45:53.284 INFO:teuthology.orchestra.run.smithi107.stdout: "services_complete": [], 2024-04-22T23:45:53.284 INFO:teuthology.orchestra.run.smithi107.stdout: "progress": null, 2024-04-22T23:45:53.284 INFO:teuthology.orchestra.run.smithi107.stdout: "message": "", 2024-04-22T23:45:53.284 INFO:teuthology.orchestra.run.smithi107.stdout: "is_paused": false 2024-04-22T23:45:53.284 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-22T23:45:53.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:53.293+0000 7ff5b6506700 1 -- 172.21.15.107:0/93451234 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7ff59806ec00 msgr2=0x7ff5980710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:53.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:53.293+0000 7ff5b6506700 1 --2- 172.21.15.107:0/93451234 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7ff59806ec00 0x7ff5980710b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0004560 tx=0x7ff5a0009e30 comp rx=0 tx=0).stop 2024-04-22T23:45:53.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:53.293+0000 7ff5b6506700 1 -- 172.21.15.107:0/93451234 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5b01037c0 msgr2=0x7ff5b01160d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:53.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:53.293+0000 7ff5b6506700 1 --2- 172.21.15.107:0/93451234 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5b01037c0 0x7ff5b01160d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4002a50 tx=0x7ff5a4004600 comp rx=0 tx=0).stop 2024-04-22T23:45:53.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:53.293+0000 7ff5b6506700 1 -- 172.21.15.107:0/93451234 shutdown_connections 2024-04-22T23:45:53.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:53.293+0000 7ff5b6506700 1 --2- 172.21.15.107:0/93451234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5b0102e20 0x7ff5b0115b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:53.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:53.293+0000 7ff5b6506700 1 --2- 172.21.15.107:0/93451234 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7ff59806ec00 0x7ff5980710b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:53.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:53.293+0000 7ff5b6506700 1 --2- 172.21.15.107:0/93451234 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5b01037c0 0x7ff5b01160d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:53.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:53.293+0000 7ff5b6506700 1 -- 172.21.15.107:0/93451234 >> 172.21.15.107:0/93451234 conn(0x7ff5b00f89b0 msgr2=0x7ff5b00fa5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:45:53.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:53.293+0000 7ff5b6506700 1 -- 172.21.15.107:0/93451234 shutdown_connections 2024-04-22T23:45:53.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:53.293+0000 7ff5b6506700 1 -- 172.21.15.107:0/93451234 wait complete. 2024-04-22T23:45:53.934 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -e sha1=43be020184947e53516056c9931e1ac5bdbbb1a5 -- bash -c 'ceph health detail' 2024-04-22T23:45:54.187 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:45:54.981 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:54 smithi107 ceph-mon[90179]: from='client.34566 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:45:54.981 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:54 smithi107 ceph-mon[90179]: pgmap v276: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:55.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:54 smithi157 ceph-mon[79403]: from='client.34566 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:45:55.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:54 smithi157 ceph-mon[79403]: pgmap v276: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:55.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:55.888+0000 7f1303aae700 1 -- 172.21.15.107:0/992705968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12fc101d00 msgr2=0x7f12fc1020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:55.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:55.888+0000 7f1303aae700 1 --2- 172.21.15.107:0/992705968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12fc101d00 0x7f12fc1020d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f12f8009ce0 tx=0x7f12f8004d10 comp rx=0 tx=0).stop 2024-04-22T23:45:55.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:55.888+0000 7f1303aae700 1 -- 172.21.15.107:0/992705968 shutdown_connections 2024-04-22T23:45:55.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:55.888+0000 7f1303aae700 1 --2- 172.21.15.107:0/992705968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12fc1026a0 0x7f12fc0fbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:55.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:55.888+0000 7f1303aae700 1 --2- 172.21.15.107:0/992705968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12fc101d00 0x7f12fc1020d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:55.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:55.888+0000 7f1303aae700 1 -- 172.21.15.107:0/992705968 >> 172.21.15.107:0/992705968 conn(0x7f12fc0f7870 msgr2=0x7f12fc0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:45:55.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:55.888+0000 7f1303aae700 1 -- 172.21.15.107:0/992705968 shutdown_connections 2024-04-22T23:45:55.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:55.888+0000 7f1303aae700 1 -- 172.21.15.107:0/992705968 wait complete. 2024-04-22T23:45:55.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:55.889+0000 7f1303aae700 1 Processor -- start 2024-04-22T23:45:55.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:55.889+0000 7f1303aae700 1 -- start start 2024-04-22T23:45:55.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:55.889+0000 7f1303aae700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12fc101d00 0x7f12fc112a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:45:55.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:55.889+0000 7f1303aae700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12fc1026a0 0x7f12fc112f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:45:55.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:55.889+0000 7f1303aae700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12fc113590 con 0x7f12fc1026a0 2024-04-22T23:45:55.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:55.889+0000 7f1303aae700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12fc10cb90 con 0x7f12fc101d00 2024-04-22T23:45:55.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:55.889+0000 7f1301049700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12fc1026a0 0x7f12fc112f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:45:55.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:55.889+0000 7f1301049700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12fc1026a0 0x7f12fc112f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41576/0 (socket says 172.21.15.107:41576) 2024-04-22T23:45:55.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:55.889+0000 7f1301049700 1 -- 172.21.15.107:0/2758145758 learned_addr learned my addr 172.21.15.107:0/2758145758 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:45:55.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:55.889+0000 7f130184a700 1 --2- 172.21.15.107:0/2758145758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12fc101d00 0x7f12fc112a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:45:55.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:55.889+0000 7f1301049700 1 -- 172.21.15.107:0/2758145758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12fc101d00 msgr2=0x7f12fc112a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:55.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:55.889+0000 7f1301049700 1 --2- 172.21.15.107:0/2758145758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12fc101d00 0x7f12fc112a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:55.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:55.889+0000 7f1301049700 1 -- 172.21.15.107:0/2758145758 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12f8009940 con 0x7f12fc1026a0 2024-04-22T23:45:55.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:55.889+0000 7f1301049700 1 --2- 172.21.15.107:0/2758145758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12fc1026a0 0x7f12fc112f40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f12ec00c800 tx=0x7f12ec004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:45:55.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:55.890+0000 7f12f27fc700 1 -- 172.21.15.107:0/2758145758 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12ec00a760 con 0x7f12fc1026a0 2024-04-22T23:45:55.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:55.890+0000 7f12f27fc700 1 -- 172.21.15.107:0/2758145758 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f12ec00ad90 con 0x7f12fc1026a0 2024-04-22T23:45:55.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:55.890+0000 7f12f27fc700 1 -- 172.21.15.107:0/2758145758 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12ec0113a0 con 0x7f12fc1026a0 2024-04-22T23:45:55.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:55.890+0000 7f1303aae700 1 -- 172.21.15.107:0/2758145758 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12fc10ce70 con 0x7f12fc1026a0 2024-04-22T23:45:55.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:55.890+0000 7f1303aae700 1 -- 172.21.15.107:0/2758145758 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12fc10d430 con 0x7f12fc1026a0 2024-04-22T23:45:55.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:55.891+0000 7f1303aae700 1 -- 172.21.15.107:0/2758145758 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12fc0fd450 con 0x7f12fc1026a0 2024-04-22T23:45:55.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:55.891+0000 7f12f27fc700 1 -- 172.21.15.107:0/2758145758 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f12ec012400 con 0x7f12fc1026a0 2024-04-22T23:45:55.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:55.891+0000 7f12f27fc700 1 --2- 172.21.15.107:0/2758145758 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f12e806ec00 0x7f12e80710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:45:55.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:55.892+0000 7f130184a700 1 --2- 172.21.15.107:0/2758145758 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f12e806ec00 0x7f12e80710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:45:55.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:55.892+0000 7f12f27fc700 1 -- 172.21.15.107:0/2758145758 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f12ec08e830 con 0x7f12fc1026a0 2024-04-22T23:45:55.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:55.892+0000 7f130184a700 1 --2- 172.21.15.107:0/2758145758 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f12e806ec00 0x7f12e80710b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f12f800b740 tx=0x7f12f8004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:45:55.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:55.896+0000 7f12f27fc700 1 -- 172.21.15.107:0/2758145758 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f12ec05b180 con 0x7f12fc1026a0 2024-04-22T23:45:56.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:55 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:45:56.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:56.180+0000 7f1303aae700 1 -- 172.21.15.107:0/2758145758 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f12fc108e00 con 0x7f12fc1026a0 2024-04-22T23:45:56.172 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:55 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:45:56.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:56.180+0000 7f12f27fc700 1 -- 172.21.15.107:0/2758145758 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f12ec05ad10 con 0x7f12fc1026a0 2024-04-22T23:45:56.173 INFO:teuthology.orchestra.run.smithi107.stdout:HEALTH_OK 2024-04-22T23:45:56.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:56.183+0000 7f1303aae700 1 -- 172.21.15.107:0/2758145758 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f12e806ec00 msgr2=0x7f12e80710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:56.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:56.183+0000 7f1303aae700 1 --2- 172.21.15.107:0/2758145758 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f12e806ec00 0x7f12e80710b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f12f800b740 tx=0x7f12f8004d10 comp rx=0 tx=0).stop 2024-04-22T23:45:56.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:56.183+0000 7f1303aae700 1 -- 172.21.15.107:0/2758145758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12fc1026a0 msgr2=0x7f12fc112f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:56.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:56.183+0000 7f1303aae700 1 --2- 172.21.15.107:0/2758145758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12fc1026a0 0x7f12fc112f40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f12ec00c800 tx=0x7f12ec004470 comp rx=0 tx=0).stop 2024-04-22T23:45:56.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:56.183+0000 7f1303aae700 1 -- 172.21.15.107:0/2758145758 shutdown_connections 2024-04-22T23:45:56.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:56.183+0000 7f1303aae700 1 --2- 172.21.15.107:0/2758145758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12fc101d00 0x7f12fc112a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:56.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:56.183+0000 7f1303aae700 1 --2- 172.21.15.107:0/2758145758 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f12e806ec00 0x7f12e80710b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:56.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:56.183+0000 7f1303aae700 1 --2- 172.21.15.107:0/2758145758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12fc1026a0 0x7f12fc112f40 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:56.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:56.183+0000 7f1303aae700 1 -- 172.21.15.107:0/2758145758 >> 172.21.15.107:0/2758145758 conn(0x7f12fc0f7870 msgr2=0x7f12fc0f87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:45:56.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:56.184+0000 7f1303aae700 1 -- 172.21.15.107:0/2758145758 shutdown_connections 2024-04-22T23:45:56.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:56.184+0000 7f1303aae700 1 -- 172.21.15.107:0/2758145758 wait complete. 2024-04-22T23:45:56.776 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -e sha1=43be020184947e53516056c9931e1ac5bdbbb1a5 -- bash -c 'ceph versions' 2024-04-22T23:45:57.036 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:45:57.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:56 smithi157 ceph-mon[79403]: pgmap v277: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:57.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:56 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/2758145758' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:45:57.158 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:56 smithi107 ceph-mon[90179]: pgmap v277: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:57.158 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:56 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/2758145758' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-22T23:45:58.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:58.786+0000 7f92795a0700 1 -- 172.21.15.107:0/3125037784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92740fb4d0 msgr2=0x7f92740fb920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:58.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:58.786+0000 7f92795a0700 1 --2- 172.21.15.107:0/3125037784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92740fb4d0 0x7f92740fb920 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9268004940 tx=0x7f9268009e30 comp rx=0 tx=0).stop 2024-04-22T23:45:58.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:58.786+0000 7f92795a0700 1 -- 172.21.15.107:0/3125037784 shutdown_connections 2024-04-22T23:45:58.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:58.786+0000 7f92795a0700 1 --2- 172.21.15.107:0/3125037784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92740fb4d0 0x7f92740fb920 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:58.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:58.786+0000 7f92795a0700 1 --2- 172.21.15.107:0/3125037784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9274102f60 0x7f9274103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:58.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:58.786+0000 7f92795a0700 1 -- 172.21.15.107:0/3125037784 >> 172.21.15.107:0/3125037784 conn(0x7f92740714c0 msgr2=0x7f92740718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:45:58.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:58.786+0000 7f92795a0700 1 -- 172.21.15.107:0/3125037784 shutdown_connections 2024-04-22T23:45:58.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:58.786+0000 7f92795a0700 1 -- 172.21.15.107:0/3125037784 wait complete. 2024-04-22T23:45:58.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:58.786+0000 7f92795a0700 1 Processor -- start 2024-04-22T23:45:58.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:58.786+0000 7f92795a0700 1 -- start start 2024-04-22T23:45:58.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:58.786+0000 7f92795a0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92740fb4d0 0x7f9274113960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:45:58.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:58.786+0000 7f92795a0700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9274102f60 0x7f9274113ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:45:58.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:58.786+0000 7f92795a0700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92741144f0 con 0x7f92740fb4d0 2024-04-22T23:45:58.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:58.786+0000 7f92795a0700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f927410daf0 con 0x7f9274102f60 2024-04-22T23:45:58.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:58.787+0000 7f9272ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92740fb4d0 0x7f9274113960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:45:58.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:58.787+0000 7f9272ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92740fb4d0 0x7f9274113960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41598/0 (socket says 172.21.15.107:41598) 2024-04-22T23:45:58.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:58.787+0000 7f9272ffd700 1 -- 172.21.15.107:0/2351321575 learned_addr learned my addr 172.21.15.107:0/2351321575 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:45:58.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:58.787+0000 7f92727fc700 1 --2- 172.21.15.107:0/2351321575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9274102f60 0x7f9274113ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:45:58.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:58.787+0000 7f9272ffd700 1 -- 172.21.15.107:0/2351321575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9274102f60 msgr2=0x7f9274113ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:58.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:58.787+0000 7f9272ffd700 1 --2- 172.21.15.107:0/2351321575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9274102f60 0x7f9274113ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:58.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:58.787+0000 7f9272ffd700 1 -- 172.21.15.107:0/2351321575 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9268005040 con 0x7f92740fb4d0 2024-04-22T23:45:58.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:58.787+0000 7f9272ffd700 1 --2- 172.21.15.107:0/2351321575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92740fb4d0 0x7f9274113960 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9264002a50 tx=0x7f92640046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:45:58.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:58.787+0000 7f925b7fe700 1 -- 172.21.15.107:0/2351321575 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9264007cc0 con 0x7f92740fb4d0 2024-04-22T23:45:58.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:58.787+0000 7f92795a0700 1 -- 172.21.15.107:0/2351321575 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f927410ddd0 con 0x7f92740fb4d0 2024-04-22T23:45:58.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:58.787+0000 7f925b7fe700 1 -- 172.21.15.107:0/2351321575 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f926400f040 con 0x7f92740fb4d0 2024-04-22T23:45:58.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:58.787+0000 7f925b7fe700 1 -- 172.21.15.107:0/2351321575 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f926400da30 con 0x7f92740fb4d0 2024-04-22T23:45:58.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:58.787+0000 7f92795a0700 1 -- 172.21.15.107:0/2351321575 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f927410e360 con 0x7f92740fb4d0 2024-04-22T23:45:58.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:58.788+0000 7f92795a0700 1 -- 172.21.15.107:0/2351321575 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9274105340 con 0x7f92740fb4d0 2024-04-22T23:45:58.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:58.789+0000 7f925b7fe700 1 -- 172.21.15.107:0/2351321575 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f926400db90 con 0x7f92740fb4d0 2024-04-22T23:45:58.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:58.789+0000 7f925b7fe700 1 --2- 172.21.15.107:0/2351321575 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f925c06ec00 0x7f925c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:45:58.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:58.789+0000 7f925b7fe700 1 -- 172.21.15.107:0/2351321575 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f926408e6d0 con 0x7f92740fb4d0 2024-04-22T23:45:58.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:58.789+0000 7f92727fc700 1 --2- 172.21.15.107:0/2351321575 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f925c06ec00 0x7f925c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:45:58.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:58.790+0000 7f92727fc700 1 --2- 172.21.15.107:0/2351321575 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f925c06ec00 0x7f925c0710b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f927410edd0 tx=0x7f9268009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:45:58.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:58.794+0000 7f925b7fe700 1 -- 172.21.15.107:0/2351321575 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f926405af00 con 0x7f92740fb4d0 2024-04-22T23:45:59.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:59.078+0000 7f92795a0700 1 -- 172.21.15.107:0/2351321575 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f927410f010 con 0x7f92740fb4d0 2024-04-22T23:45:59.070 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:58 smithi107 ceph-mon[90179]: pgmap v278: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:59.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:59.079+0000 7f925b7fe700 1 -- 172.21.15.107:0/2351321575 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f926405aa90 con 0x7f92740fb4d0 2024-04-22T23:45:59.071 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-22T23:45:59.071 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-04-22T23:45:59.071 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:45:59.071 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:45:59.071 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-04-22T23:45:59.071 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:45:59.071 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:45:59.071 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-04-22T23:45:59.071 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 6 2024-04-22T23:45:59.072 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:45:59.072 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-04-22T23:45:59.072 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 4 2024-04-22T23:45:59.072 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:45:59.072 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-04-22T23:45:59.072 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 14 2024-04-22T23:45:59.072 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T23:45:59.072 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-22T23:45:59.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:59.081+0000 7f92795a0700 1 -- 172.21.15.107:0/2351321575 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f925c06ec00 msgr2=0x7f925c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:59.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:59.081+0000 7f92795a0700 1 --2- 172.21.15.107:0/2351321575 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f925c06ec00 0x7f925c0710b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f927410edd0 tx=0x7f9268009e30 comp rx=0 tx=0).stop 2024-04-22T23:45:59.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:59.081+0000 7f92795a0700 1 -- 172.21.15.107:0/2351321575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92740fb4d0 msgr2=0x7f9274113960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:45:59.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:59.081+0000 7f92795a0700 1 --2- 172.21.15.107:0/2351321575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92740fb4d0 0x7f9274113960 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9264002a50 tx=0x7f92640046b0 comp rx=0 tx=0).stop 2024-04-22T23:45:59.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:59.081+0000 7f92795a0700 1 -- 172.21.15.107:0/2351321575 shutdown_connections 2024-04-22T23:45:59.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:59.081+0000 7f92795a0700 1 --2- 172.21.15.107:0/2351321575 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f925c06ec00 0x7f925c0710b0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:59.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:59.081+0000 7f92795a0700 1 --2- 172.21.15.107:0/2351321575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92740fb4d0 0x7f9274113960 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:59.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:59.081+0000 7f92795a0700 1 --2- 172.21.15.107:0/2351321575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9274102f60 0x7f9274113ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:45:59.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:59.081+0000 7f92795a0700 1 -- 172.21.15.107:0/2351321575 >> 172.21.15.107:0/2351321575 conn(0x7f92740714c0 msgr2=0x7f92740fa7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:45:59.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:59.081+0000 7f92795a0700 1 -- 172.21.15.107:0/2351321575 shutdown_connections 2024-04-22T23:45:59.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:45:59.081+0000 7f92795a0700 1 -- 172.21.15.107:0/2351321575 wait complete. 2024-04-22T23:45:59.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:58 smithi157 ceph-mon[79403]: pgmap v278: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:45:59.681 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -e sha1=43be020184947e53516056c9931e1ac5bdbbb1a5 -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-04-22T23:45:59.942 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:45:59.981 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:45:59 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/2351321575' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:46:00.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:45:59 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/2351321575' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:46:01.073 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:46:00 smithi107 ceph-mon[90179]: pgmap v279: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:01.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:46:00 smithi157 ceph-mon[79403]: pgmap v279: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:01.568 INFO:teuthology.orchestra.run.smithi107.stdout:wait for servicemap items w/ changing names to refresh 2024-04-22T23:46:02.265 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -e sha1=43be020184947e53516056c9931e1ac5bdbbb1a5 -- bash -c 'sleep 60' 2024-04-22T23:46:02.520 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:46:02.900 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:46:02 smithi107 ceph-mon[90179]: pgmap v280: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:03.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:46:02 smithi157 ceph-mon[79403]: pgmap v280: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:05.115 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:46:04 smithi107 ceph-mon[90179]: pgmap v281: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:05.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:46:04 smithi157 ceph-mon[79403]: pgmap v281: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:07.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:46:06 smithi157 ceph-mon[79403]: pgmap v282: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:07.202 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:46:06 smithi107 ceph-mon[90179]: pgmap v282: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:09.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:46:08 smithi157 ceph-mon[79403]: pgmap v283: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:09.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:46:08 smithi107 ceph-mon[90179]: pgmap v283: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:11.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:46:10 smithi157 ceph-mon[79403]: pgmap v284: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:11.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:46:10 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:46:11.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:46:10 smithi107 ceph-mon[90179]: pgmap v284: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:11.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:46:10 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:46:12.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:46:11 smithi157 ceph-mon[79403]: pgmap v285: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:12.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:46:11 smithi107 ceph-mon[90179]: pgmap v285: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:14.981 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:46:14 smithi107 ceph-mon[90179]: pgmap v286: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:15.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:46:14 smithi157 ceph-mon[79403]: pgmap v286: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:17.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:46:16 smithi157 ceph-mon[79403]: pgmap v287: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:17.201 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:46:16 smithi107 ceph-mon[90179]: pgmap v287: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:19.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:46:18 smithi157 ceph-mon[79403]: pgmap v288: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:19.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:46:18 smithi107 ceph-mon[90179]: pgmap v288: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:21.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:46:20 smithi157 ceph-mon[79403]: pgmap v289: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:21.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:46:20 smithi107 ceph-mon[90179]: pgmap v289: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:23.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:46:22 smithi157 ceph-mon[79403]: pgmap v290: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:23.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:46:22 smithi107 ceph-mon[90179]: pgmap v290: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:25.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:46:24 smithi157 ceph-mon[79403]: pgmap v291: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:25.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:46:24 smithi107 ceph-mon[90179]: pgmap v291: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:26.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:46:25 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:46:26.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:46:25 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:46:27.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:46:26 smithi157 ceph-mon[79403]: pgmap v292: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:27.200 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:46:26 smithi107 ceph-mon[90179]: pgmap v292: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:29.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:46:28 smithi157 ceph-mon[79403]: pgmap v293: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:29.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:46:28 smithi107 ceph-mon[90179]: pgmap v293: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:31.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:46:30 smithi157 ceph-mon[79403]: pgmap v294: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:31.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:46:30 smithi107 ceph-mon[90179]: pgmap v294: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:32.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:46:31 smithi107 ceph-mon[90179]: pgmap v295: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:32.379 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:46:31 smithi157 ceph-mon[79403]: pgmap v295: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:35.130 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:46:34 smithi157 ceph-mon[79403]: pgmap v296: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:35.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:46:34 smithi107 ceph-mon[90179]: pgmap v296: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:37.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:46:36 smithi157 ceph-mon[79403]: pgmap v297: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:37.201 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:46:36 smithi107 ceph-mon[90179]: pgmap v297: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:39.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:46:38 smithi157 ceph-mon[79403]: pgmap v298: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:39.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:46:38 smithi107 ceph-mon[90179]: pgmap v298: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:41.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:46:40 smithi157 ceph-mon[79403]: pgmap v299: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:41.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:46:40 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:46:41.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:46:40 smithi107 ceph-mon[90179]: pgmap v299: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:41.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:46:40 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:46:43.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:46:42 smithi157 ceph-mon[79403]: pgmap v300: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:43.235 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:46:42 smithi107 ceph-mon[90179]: pgmap v300: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:45.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:46:44 smithi157 ceph-mon[79403]: pgmap v301: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:45.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:46:44 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:46:45.230 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:46:44 smithi107 ceph-mon[90179]: pgmap v301: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:45.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:46:44 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T23:46:46.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:46:45 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:46:46.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:46:45 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:46:46.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:46:45 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:46:46.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:46:45 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:46:46.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:46:45 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:46:46.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:46:45 smithi107 ceph-mon[90179]: pgmap v302: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:46.379 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:46:45 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:46:46.379 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:46:45 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:46:46.379 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:46:45 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T23:46:46.380 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:46:45 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T23:46:46.380 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:46:45 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' 2024-04-22T23:46:46.380 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:46:45 smithi157 ceph-mon[79403]: pgmap v302: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:49.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:46:48 smithi157 ceph-mon[79403]: pgmap v303: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:49.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:46:48 smithi107 ceph-mon[90179]: pgmap v303: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:51.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:46:50 smithi157 ceph-mon[79403]: pgmap v304: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:51.230 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:46:50 smithi107 ceph-mon[90179]: pgmap v304: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:53.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:46:52 smithi157 ceph-mon[79403]: pgmap v305: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:53.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:46:52 smithi107 ceph-mon[90179]: pgmap v305: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:55.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:46:54 smithi157 ceph-mon[79403]: pgmap v306: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:55.230 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:46:54 smithi107 ceph-mon[90179]: pgmap v306: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:56.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:46:55 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:46:56.230 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:46:55 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:46:57.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:46:56 smithi157 ceph-mon[79403]: pgmap v307: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:57.201 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:46:56 smithi107 ceph-mon[90179]: pgmap v307: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:59.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:46:58 smithi157 ceph-mon[79403]: pgmap v308: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:46:59.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:46:58 smithi107 ceph-mon[90179]: pgmap v308: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:47:01.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:00 smithi157 ceph-mon[79403]: pgmap v309: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:47:01.230 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:00 smithi107 ceph-mon[90179]: pgmap v309: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:47:03.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:02 smithi157 ceph-mon[79403]: pgmap v310: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:47:03.230 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:02 smithi107 ceph-mon[90179]: pgmap v310: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:47:04.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:03 smithi107 ceph-mon[90179]: pgmap v311: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:47:04.379 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:03 smithi157 ceph-mon[79403]: pgmap v311: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:47:04.973 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -e sha1=43be020184947e53516056c9931e1ac5bdbbb1a5 -- bash -c 'ceph orch ps' 2024-04-22T23:47:05.237 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:47:06.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:06.889+0000 7efd0e7be700 1 -- 172.21.15.107:0/1525114015 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd080fe7d0 msgr2=0x7efd08106ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:06.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:06.889+0000 7efd0e7be700 1 --2- 172.21.15.107:0/1525114015 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd080fe7d0 0x7efd08106ea0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7efcfc004940 tx=0x7efcfc009e30 comp rx=0 tx=0).stop 2024-04-22T23:47:06.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:06.889+0000 7efd0e7be700 1 -- 172.21.15.107:0/1525114015 shutdown_connections 2024-04-22T23:47:06.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:06.889+0000 7efd0e7be700 1 --2- 172.21.15.107:0/1525114015 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd080fe7d0 0x7efd08106ea0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:06.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:06.889+0000 7efd0e7be700 1 --2- 172.21.15.107:0/1525114015 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd080fde30 0x7efd080fe200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:06.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:06.889+0000 7efd0e7be700 1 -- 172.21.15.107:0/1525114015 >> 172.21.15.107:0/1525114015 conn(0x7efd080f7870 msgr2=0x7efd080f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:06.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:06.889+0000 7efd0e7be700 1 -- 172.21.15.107:0/1525114015 shutdown_connections 2024-04-22T23:47:06.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:06.889+0000 7efd0e7be700 1 -- 172.21.15.107:0/1525114015 wait complete. 2024-04-22T23:47:06.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:06.890+0000 7efd0e7be700 1 Processor -- start 2024-04-22T23:47:06.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:06.890+0000 7efd0e7be700 1 -- start start 2024-04-22T23:47:06.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:06.890+0000 7efd0e7be700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd080fde30 0x7efd08113a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:06.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:06.890+0000 7efd0e7be700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd080fe7d0 0x7efd08113f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:06.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:06.890+0000 7efd0e7be700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd08114660 con 0x7efd080fde30 2024-04-22T23:47:06.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:06.890+0000 7efd0e7be700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd0810dbb0 con 0x7efd080fe7d0 2024-04-22T23:47:06.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:06.890+0000 7efd07fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd080fde30 0x7efd08113a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:06.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:06.890+0000 7efd07fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd080fde30 0x7efd08113a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34664/0 (socket says 172.21.15.107:34664) 2024-04-22T23:47:06.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:06.890+0000 7efd07fff700 1 -- 172.21.15.107:0/2383194091 learned_addr learned my addr 172.21.15.107:0/2383194091 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:47:06.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:06.891+0000 7efd077fe700 1 --2- 172.21.15.107:0/2383194091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd080fe7d0 0x7efd08113f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:06.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:06.891+0000 7efd07fff700 1 -- 172.21.15.107:0/2383194091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd080fe7d0 msgr2=0x7efd08113f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:06.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:06.891+0000 7efd07fff700 1 --2- 172.21.15.107:0/2383194091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd080fe7d0 0x7efd08113f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:06.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:06.891+0000 7efd07fff700 1 -- 172.21.15.107:0/2383194091 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcfc005040 con 0x7efd080fde30 2024-04-22T23:47:06.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:06.891+0000 7efd07fff700 1 --2- 172.21.15.107:0/2383194091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd080fde30 0x7efd08113a40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7efcf800b580 tx=0x7efcf80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:06.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:06.891+0000 7efd04ff9700 1 -- 172.21.15.107:0/2383194091 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efcf800f6f0 con 0x7efd080fde30 2024-04-22T23:47:06.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:06.891+0000 7efd04ff9700 1 -- 172.21.15.107:0/2383194091 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efcf800fd20 con 0x7efd080fde30 2024-04-22T23:47:06.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:06.891+0000 7efd0e7be700 1 -- 172.21.15.107:0/2383194091 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd0810de30 con 0x7efd080fde30 2024-04-22T23:47:06.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:06.891+0000 7efd04ff9700 1 -- 172.21.15.107:0/2383194091 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efcf8011ea0 con 0x7efd080fde30 2024-04-22T23:47:06.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:06.891+0000 7efd0e7be700 1 -- 172.21.15.107:0/2383194091 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd0810e370 con 0x7efd080fde30 2024-04-22T23:47:06.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:06.892+0000 7efd0e7be700 1 -- 172.21.15.107:0/2383194091 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd08066ae0 con 0x7efd080fde30 2024-04-22T23:47:06.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:06.892+0000 7efd04ff9700 1 -- 172.21.15.107:0/2383194091 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7efcf800f850 con 0x7efd080fde30 2024-04-22T23:47:06.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:06.893+0000 7efd04ff9700 1 --2- 172.21.15.107:0/2383194091 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7efcf006ec00 0x7efcf00710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:06.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:06.893+0000 7efd04ff9700 1 -- 172.21.15.107:0/2383194091 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7efcf808e5f0 con 0x7efd080fde30 2024-04-22T23:47:06.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:06.893+0000 7efd077fe700 1 --2- 172.21.15.107:0/2383194091 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7efcf006ec00 0x7efcf00710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:06.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:06.893+0000 7efd077fe700 1 --2- 172.21.15.107:0/2383194091 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7efcf006ec00 0x7efcf00710b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efcfc00b3b0 tx=0x7efcfc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:06.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:06.898+0000 7efd04ff9700 1 -- 172.21.15.107:0/2383194091 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7efcf805aea0 con 0x7efd080fde30 2024-04-22T23:47:07.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:07.108+0000 7efd0e7be700 1 -- 172.21.15.107:0/2383194091 --> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7efd0810f150 con 0x7efcf006ec00 2024-04-22T23:47:07.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:06 smithi107 ceph-mon[90179]: pgmap v312: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:47:07.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:07.118+0000 7efd04ff9700 1 -- 172.21.15.107:0/2383194091 <== mgr.34104 v2:172.21.15.107:6800/762611949 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7efd0810f150 con 0x7efcf006ec00 2024-04-22T23:47:07.110 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-22T23:47:07.110 INFO:teuthology.orchestra.run.smithi107.stdout:alertmanager.smithi107 smithi107 *:9093,9094 running (118s) 90s ago 20m 14.6M - 0.25.0 c8568f914cd2 07fa1ebb7f47 2024-04-22T23:47:07.110 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi107 smithi107 running (8m) 90s ago 20m 6656k - 19.0.0-3252-g43be0201 a0434699c1d7 e074fd2cbb9c 2024-04-22T23:47:07.110 INFO:teuthology.orchestra.run.smithi107.stdout:crash.smithi157 smithi157 running (8m) 2m ago 19m 6651k - 19.0.0-3252-g43be0201 a0434699c1d7 abc2649dee22 2024-04-22T23:47:07.110 INFO:teuthology.orchestra.run.smithi107.stdout:grafana.smithi107 smithi107 *:3000 running (96s) 90s ago 19m 54.8M - 9.4.12 2bacad6d85d8 adc26d364256 2024-04-22T23:47:07.110 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.dfvvkw smithi107 running (3m) 90s ago 14m 70.4M - 19.0.0-3252-g43be0201 a0434699c1d7 7566b5cc3d01 2024-04-22T23:47:07.110 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi107.ktuxjl smithi107 running (3m) 90s ago 14m 18.3M - 19.0.0-3252-g43be0201 a0434699c1d7 ce23545f0ae2 2024-04-22T23:47:07.110 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi157.rcaris smithi157 running (3m) 2m ago 14m 11.9M - 19.0.0-3252-g43be0201 a0434699c1d7 12550073a997 2024-04-22T23:47:07.110 INFO:teuthology.orchestra.run.smithi107.stdout:mds.cephfs.smithi157.ufywse smithi157 running (2m) 2m ago 14m 14.1M - 19.0.0-3252-g43be0201 a0434699c1d7 e9f97f60ed21 2024-04-22T23:47:07.111 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi107.uipxvm smithi107 *:8443,9283,8765 running (10m) 90s ago 21m 522M - 19.0.0-3252-g43be0201 a0434699c1d7 d20f05dfe4f2 2024-04-22T23:47:07.111 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.smithi157.tuylis smithi157 *:8443,9283,8765 running (9m) 2m ago 19m 452M - 19.0.0-3252-g43be0201 a0434699c1d7 957e598c9001 2024-04-22T23:47:07.111 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi107 smithi107 running (9m) 90s ago 21m 60.1M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 bc72bbbe2c39 2024-04-22T23:47:07.111 INFO:teuthology.orchestra.run.smithi107.stdout:mon.smithi157 smithi157 running (8m) 2m ago 19m 41.2M 2048M 19.0.0-3252-g43be0201 a0434699c1d7 ea88de8be4fe 2024-04-22T23:47:07.111 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi107 smithi107 *:9100 running (2m) 90s ago 20m 8870k - 1.5.0 0da6a335fe13 c338e5470944 2024-04-22T23:47:07.111 INFO:teuthology.orchestra.run.smithi107.stdout:node-exporter.smithi157 smithi157 *:9100 running (2m) 2m ago 19m 3389k - 1.5.0 0da6a335fe13 8d0255aff39c 2024-04-22T23:47:07.111 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (7m) 90s ago 18m 104M 1557M 19.0.0-3252-g43be0201 a0434699c1d7 fd0ba8ef0c27 2024-04-22T23:47:07.111 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi107 running (7m) 90s ago 17m 104M 1557M 19.0.0-3252-g43be0201 a0434699c1d7 77e33b6122e0 2024-04-22T23:47:07.111 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi107 running (6m) 90s ago 17m 99.2M 1557M 19.0.0-3252-g43be0201 a0434699c1d7 c307bd8cdcfc 2024-04-22T23:47:07.111 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi157 running (6m) 2m ago 17m 112M 2581M 19.0.0-3252-g43be0201 a0434699c1d7 e81ca519c831 2024-04-22T23:47:07.111 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi157 running (5m) 2m ago 16m 105M 2581M 19.0.0-3252-g43be0201 a0434699c1d7 d5b5a2e4781a 2024-04-22T23:47:07.111 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi157 running (4m) 2m ago 16m 97.1M 2581M 19.0.0-3252-g43be0201 a0434699c1d7 0ee423135fdb 2024-04-22T23:47:07.111 INFO:teuthology.orchestra.run.smithi107.stdout:prometheus.smithi107 smithi107 *:9095 running (2m) 90s ago 19m 47.6M - 2.43.0 a07b618ecd1d f6160052545f 2024-04-22T23:47:07.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:07.120+0000 7efd0e7be700 1 -- 172.21.15.107:0/2383194091 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7efcf006ec00 msgr2=0x7efcf00710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:07.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:07.120+0000 7efd0e7be700 1 --2- 172.21.15.107:0/2383194091 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7efcf006ec00 0x7efcf00710b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efcfc00b3b0 tx=0x7efcfc009e30 comp rx=0 tx=0).stop 2024-04-22T23:47:07.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:07.120+0000 7efd0e7be700 1 -- 172.21.15.107:0/2383194091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd080fde30 msgr2=0x7efd08113a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:07.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:07.120+0000 7efd0e7be700 1 --2- 172.21.15.107:0/2383194091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd080fde30 0x7efd08113a40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7efcf800b580 tx=0x7efcf80091a0 comp rx=0 tx=0).stop 2024-04-22T23:47:07.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:07.120+0000 7efd0e7be700 1 -- 172.21.15.107:0/2383194091 shutdown_connections 2024-04-22T23:47:07.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:07.120+0000 7efd0e7be700 1 --2- 172.21.15.107:0/2383194091 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7efcf006ec00 0x7efcf00710b0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:07.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:07.120+0000 7efd0e7be700 1 --2- 172.21.15.107:0/2383194091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd080fde30 0x7efd08113a40 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:07.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:07.120+0000 7efd0e7be700 1 --2- 172.21.15.107:0/2383194091 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efd080fe7d0 0x7efd08113f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:07.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:07.120+0000 7efd0e7be700 1 -- 172.21.15.107:0/2383194091 >> 172.21.15.107:0/2383194091 conn(0x7efd080f7870 msgr2=0x7efd080f8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:07.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:07.120+0000 7efd0e7be700 1 -- 172.21.15.107:0/2383194091 shutdown_connections 2024-04-22T23:47:07.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:07.120+0000 7efd0e7be700 1 -- 172.21.15.107:0/2383194091 wait complete. 2024-04-22T23:47:07.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:06 smithi157 ceph-mon[79403]: pgmap v312: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:47:07.746 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -e sha1=43be020184947e53516056c9931e1ac5bdbbb1a5 -- bash -c 'ceph versions' 2024-04-22T23:47:08.009 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:47:09.077 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:08 smithi107 ceph-mon[90179]: from='client.34578 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:47:09.077 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:08 smithi107 ceph-mon[90179]: pgmap v313: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:47:09.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:08 smithi157 ceph-mon[79403]: from='client.34578 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T23:47:09.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:08 smithi157 ceph-mon[79403]: pgmap v313: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:47:09.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:09.569+0000 7f5ebfdb0700 1 -- 172.21.15.107:0/568546333 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5eb80656b0 msgr2=0x7f5eb8065b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:09.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:09.569+0000 7f5ebfdb0700 1 --2- 172.21.15.107:0/568546333 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5eb80656b0 0x7f5eb8065b20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8004940 tx=0x7f5ea8009e30 comp rx=0 tx=0).stop 2024-04-22T23:47:09.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:09.569+0000 7f5ebfdb0700 1 -- 172.21.15.107:0/568546333 shutdown_connections 2024-04-22T23:47:09.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:09.569+0000 7f5ebfdb0700 1 --2- 172.21.15.107:0/568546333 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5eb80656b0 0x7f5eb8065b20 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:09.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:09.569+0000 7f5ebfdb0700 1 --2- 172.21.15.107:0/568546333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5eb8102e60 0x7f5eb8103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:09.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:09.569+0000 7f5ebfdb0700 1 -- 172.21.15.107:0/568546333 >> 172.21.15.107:0/568546333 conn(0x7f5eb8072bc0 msgr2=0x7f5eb8072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:09.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:09.569+0000 7f5ebfdb0700 1 -- 172.21.15.107:0/568546333 shutdown_connections 2024-04-22T23:47:09.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:09.569+0000 7f5ebfdb0700 1 -- 172.21.15.107:0/568546333 wait complete. 2024-04-22T23:47:09.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:09.569+0000 7f5ebfdb0700 1 Processor -- start 2024-04-22T23:47:09.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:09.569+0000 7f5ebfdb0700 1 -- start start 2024-04-22T23:47:09.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:09.570+0000 7f5ebfdb0700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5eb80656b0 0x7f5eb8113ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:09.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:09.570+0000 7f5ebfdb0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5eb8102e60 0x7f5eb8114000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:09.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:09.570+0000 7f5ebfdb0700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb8114650 con 0x7f5eb8102e60 2024-04-22T23:47:09.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:09.570+0000 7f5ebfdb0700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb810da90 con 0x7f5eb80656b0 2024-04-22T23:47:09.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:09.570+0000 7f5ebd34b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5eb8102e60 0x7f5eb8114000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:09.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:09.570+0000 7f5ebdb4c700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5eb80656b0 0x7f5eb8113ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:09.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:09.570+0000 7f5ebd34b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5eb8102e60 0x7f5eb8114000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34682/0 (socket says 172.21.15.107:34682) 2024-04-22T23:47:09.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:09.570+0000 7f5ebd34b700 1 -- 172.21.15.107:0/3378456571 learned_addr learned my addr 172.21.15.107:0/3378456571 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:47:09.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:09.570+0000 7f5ebd34b700 1 -- 172.21.15.107:0/3378456571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5eb80656b0 msgr2=0x7f5eb8113ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:09.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:09.570+0000 7f5ebd34b700 1 --2- 172.21.15.107:0/3378456571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5eb80656b0 0x7f5eb8113ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:09.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:09.570+0000 7f5ebd34b700 1 -- 172.21.15.107:0/3378456571 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea8005040 con 0x7f5eb8102e60 2024-04-22T23:47:09.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:09.570+0000 7f5ebd34b700 1 --2- 172.21.15.107:0/3378456571 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5eb8102e60 0x7f5eb8114000 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8009b30 tx=0x7f5ea8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:09.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:09.571+0000 7f5eae7fc700 1 -- 172.21.15.107:0/3378456571 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ea8021070 con 0x7f5eb8102e60 2024-04-22T23:47:09.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:09.571+0000 7f5eae7fc700 1 -- 172.21.15.107:0/3378456571 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5ea8013c00 con 0x7f5eb8102e60 2024-04-22T23:47:09.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:09.571+0000 7f5ebfdb0700 1 -- 172.21.15.107:0/3378456571 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eb810dd10 con 0x7f5eb8102e60 2024-04-22T23:47:09.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:09.571+0000 7f5eae7fc700 1 -- 172.21.15.107:0/3378456571 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ea800e070 con 0x7f5eb8102e60 2024-04-22T23:47:09.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:09.571+0000 7f5ebfdb0700 1 -- 172.21.15.107:0/3378456571 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eb810e1f0 con 0x7f5eb8102e60 2024-04-22T23:47:09.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:09.572+0000 7f5ebfdb0700 1 -- 172.21.15.107:0/3378456571 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5eb8100130 con 0x7f5eb8102e60 2024-04-22T23:47:09.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:09.572+0000 7f5eae7fc700 1 -- 172.21.15.107:0/3378456571 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f5ea801c400 con 0x7f5eb8102e60 2024-04-22T23:47:09.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:09.572+0000 7f5eae7fc700 1 --2- 172.21.15.107:0/3378456571 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f5ea406ea00 0x7f5ea4070eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:09.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:09.573+0000 7f5eae7fc700 1 -- 172.21.15.107:0/3378456571 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f5ea808f960 con 0x7f5eb8102e60 2024-04-22T23:47:09.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:09.573+0000 7f5ebdb4c700 1 --2- 172.21.15.107:0/3378456571 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f5ea406ea00 0x7f5ea4070eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:09.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:09.573+0000 7f5ebdb4c700 1 --2- 172.21.15.107:0/3378456571 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f5ea406ea00 0x7f5ea4070eb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4009b80 tx=0x7f5eb400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:09.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:09.577+0000 7f5eae7fc700 1 -- 172.21.15.107:0/3378456571 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f5ea805c210 con 0x7f5eb8102e60 2024-04-22T23:47:09.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:09.861+0000 7f5ebfdb0700 1 -- 172.21.15.107:0/3378456571 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5eb805f5f0 con 0x7f5eb8102e60 2024-04-22T23:47:09.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:09.862+0000 7f5eae7fc700 1 -- 172.21.15.107:0/3378456571 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f5ea805bda0 con 0x7f5eb8102e60 2024-04-22T23:47:09.854 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-22T23:47:09.854 INFO:teuthology.orchestra.run.smithi107.stdout: "mon": { 2024-04-22T23:47:09.854 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:47:09.854 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:47:09.854 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr": { 2024-04-22T23:47:09.854 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 2 2024-04-22T23:47:09.854 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:47:09.855 INFO:teuthology.orchestra.run.smithi107.stdout: "osd": { 2024-04-22T23:47:09.855 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 6 2024-04-22T23:47:09.855 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:47:09.855 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-04-22T23:47:09.855 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 4 2024-04-22T23:47:09.855 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-22T23:47:09.855 INFO:teuthology.orchestra.run.smithi107.stdout: "overall": { 2024-04-22T23:47:09.855 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)": 14 2024-04-22T23:47:09.856 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-22T23:47:09.856 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-22T23:47:09.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:09.864+0000 7f5ebfdb0700 1 -- 172.21.15.107:0/3378456571 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f5ea406ea00 msgr2=0x7f5ea4070eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:09.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:09.864+0000 7f5ebfdb0700 1 --2- 172.21.15.107:0/3378456571 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f5ea406ea00 0x7f5ea4070eb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4009b80 tx=0x7f5eb400b040 comp rx=0 tx=0).stop 2024-04-22T23:47:09.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:09.864+0000 7f5ebfdb0700 1 -- 172.21.15.107:0/3378456571 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5eb8102e60 msgr2=0x7f5eb8114000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:09.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:09.864+0000 7f5ebfdb0700 1 --2- 172.21.15.107:0/3378456571 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5eb8102e60 0x7f5eb8114000 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8009b30 tx=0x7f5ea8009e30 comp rx=0 tx=0).stop 2024-04-22T23:47:09.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:09.864+0000 7f5ebfdb0700 1 -- 172.21.15.107:0/3378456571 shutdown_connections 2024-04-22T23:47:09.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:09.864+0000 7f5ebfdb0700 1 --2- 172.21.15.107:0/3378456571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5eb80656b0 0x7f5eb8113ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:09.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:09.864+0000 7f5ebfdb0700 1 --2- 172.21.15.107:0/3378456571 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f5ea406ea00 0x7f5ea4070eb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:09.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:09.864+0000 7f5ebfdb0700 1 --2- 172.21.15.107:0/3378456571 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5eb8102e60 0x7f5eb8114000 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:09.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:09.864+0000 7f5ebfdb0700 1 -- 172.21.15.107:0/3378456571 >> 172.21.15.107:0/3378456571 conn(0x7f5eb8072bc0 msgr2=0x7f5eb80faec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:09.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:09.864+0000 7f5ebfdb0700 1 -- 172.21.15.107:0/3378456571 shutdown_connections 2024-04-22T23:47:09.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:09.864+0000 7f5ebfdb0700 1 -- 172.21.15.107:0/3378456571 wait complete. 2024-04-22T23:47:10.602 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -e sha1=43be020184947e53516056c9931e1ac5bdbbb1a5 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-04-22T23:47:10.859 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:47:11.089 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:10 smithi107 ceph-mon[90179]: pgmap v314: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:47:11.089 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:10 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/3378456571' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:47:11.089 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:10 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:47:11.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:10 smithi157 ceph-mon[79403]: pgmap v314: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:47:11.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:10 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/3378456571' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:47:11.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:10 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:47:12.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:12.450+0000 7fa6aa54e700 1 -- 172.21.15.107:0/571312713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6a41037b0 msgr2=0x7fa6a40fce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:12.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:12.450+0000 7fa6aa54e700 1 --2- 172.21.15.107:0/571312713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6a41037b0 0x7fa6a40fce20 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa694004940 tx=0x7fa694009e30 comp rx=0 tx=0).stop 2024-04-22T23:47:12.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:12.450+0000 7fa6aa54e700 1 -- 172.21.15.107:0/571312713 shutdown_connections 2024-04-22T23:47:12.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:12.450+0000 7fa6aa54e700 1 --2- 172.21.15.107:0/571312713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6a41037b0 0x7fa6a40fce20 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:12.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:12.450+0000 7fa6aa54e700 1 --2- 172.21.15.107:0/571312713 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6a4102e10 0x7fa6a41031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:12.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:12.450+0000 7fa6aa54e700 1 -- 172.21.15.107:0/571312713 >> 172.21.15.107:0/571312713 conn(0x7fa6a40f89a0 msgr2=0x7fa6a40fadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:12.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:12.450+0000 7fa6aa54e700 1 -- 172.21.15.107:0/571312713 shutdown_connections 2024-04-22T23:47:12.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:12.450+0000 7fa6aa54e700 1 -- 172.21.15.107:0/571312713 wait complete. 2024-04-22T23:47:12.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:12.451+0000 7fa6aa54e700 1 Processor -- start 2024-04-22T23:47:12.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:12.451+0000 7fa6aa54e700 1 -- start start 2024-04-22T23:47:12.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:12.451+0000 7fa6aa54e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6a4102e10 0x7fa6a4113b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:12.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:12.451+0000 7fa6aa54e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6a41037b0 0x7fa6a41140d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:12.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:12.451+0000 7fa6aa54e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a41147b0 con 0x7fa6a41037b0 2024-04-22T23:47:12.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:12.451+0000 7fa6aa54e700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a410db10 con 0x7fa6a4102e10 2024-04-22T23:47:12.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:12.451+0000 7fa6a37fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6a41037b0 0x7fa6a41140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:12.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:12.451+0000 7fa6a37fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6a41037b0 0x7fa6a41140d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44826/0 (socket says 172.21.15.107:44826) 2024-04-22T23:47:12.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:12.451+0000 7fa6a37fe700 1 -- 172.21.15.107:0/577297307 learned_addr learned my addr 172.21.15.107:0/577297307 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:47:12.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:12.451+0000 7fa6a3fff700 1 --2- 172.21.15.107:0/577297307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6a4102e10 0x7fa6a4113b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:12.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:12.451+0000 7fa6a37fe700 1 -- 172.21.15.107:0/577297307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6a4102e10 msgr2=0x7fa6a4113b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:12.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:12.451+0000 7fa6a37fe700 1 --2- 172.21.15.107:0/577297307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6a4102e10 0x7fa6a4113b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:12.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:12.451+0000 7fa6a37fe700 1 -- 172.21.15.107:0/577297307 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa694005040 con 0x7fa6a41037b0 2024-04-22T23:47:12.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:12.452+0000 7fa6a37fe700 1 --2- 172.21.15.107:0/577297307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6a41037b0 0x7fa6a41140d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa694009100 tx=0x7fa694009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:12.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:12.452+0000 7fa6a0ff9700 1 -- 172.21.15.107:0/577297307 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa69400e040 con 0x7fa6a41037b0 2024-04-22T23:47:12.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:12.452+0000 7fa6aa54e700 1 -- 172.21.15.107:0/577297307 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6a410dd90 con 0x7fa6a41037b0 2024-04-22T23:47:12.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:12.452+0000 7fa6a0ff9700 1 -- 172.21.15.107:0/577297307 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa694013420 con 0x7fa6a41037b0 2024-04-22T23:47:12.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:12.452+0000 7fa6a0ff9700 1 -- 172.21.15.107:0/577297307 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa69401c610 con 0x7fa6a41037b0 2024-04-22T23:47:12.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:12.452+0000 7fa6aa54e700 1 -- 172.21.15.107:0/577297307 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6a410e350 con 0x7fa6a41037b0 2024-04-22T23:47:12.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:12.453+0000 7fa6aa54e700 1 -- 172.21.15.107:0/577297307 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6a40fe560 con 0x7fa6a41037b0 2024-04-22T23:47:12.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:12.453+0000 7fa6a0ff9700 1 -- 172.21.15.107:0/577297307 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fa694013590 con 0x7fa6a41037b0 2024-04-22T23:47:12.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:12.454+0000 7fa6a0ff9700 1 --2- 172.21.15.107:0/577297307 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fa68c06ec00 0x7fa68c0710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:12.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:12.454+0000 7fa6a0ff9700 1 -- 172.21.15.107:0/577297307 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fa69408fb90 con 0x7fa6a41037b0 2024-04-22T23:47:12.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:12.454+0000 7fa6a3fff700 1 --2- 172.21.15.107:0/577297307 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fa68c06ec00 0x7fa68c0710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:12.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:12.454+0000 7fa6a3fff700 1 --2- 172.21.15.107:0/577297307 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fa68c06ec00 0x7fa68c0710b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa698006fd0 tx=0x7fa698008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:12.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:12.459+0000 7fa6a0ff9700 1 -- 172.21.15.107:0/577297307 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fa69405c440 con 0x7fa6a41037b0 2024-04-22T23:47:12.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:12.741+0000 7fa6aa54e700 1 -- 172.21.15.107:0/577297307 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa6a405f5f0 con 0x7fa6a41037b0 2024-04-22T23:47:12.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:12.741+0000 7fa6a0ff9700 1 -- 172.21.15.107:0/577297307 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fa69405bfd0 con 0x7fa6a41037b0 2024-04-22T23:47:12.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:12.743+0000 7fa6aa54e700 1 -- 172.21.15.107:0/577297307 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fa68c06ec00 msgr2=0x7fa68c0710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:12.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:12.743+0000 7fa6aa54e700 1 --2- 172.21.15.107:0/577297307 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fa68c06ec00 0x7fa68c0710b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa698006fd0 tx=0x7fa698008040 comp rx=0 tx=0).stop 2024-04-22T23:47:12.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:12.743+0000 7fa6aa54e700 1 -- 172.21.15.107:0/577297307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6a41037b0 msgr2=0x7fa6a41140d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:12.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:12.743+0000 7fa6aa54e700 1 --2- 172.21.15.107:0/577297307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6a41037b0 0x7fa6a41140d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa694009100 tx=0x7fa694009e30 comp rx=0 tx=0).stop 2024-04-22T23:47:12.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:12.743+0000 7fa6aa54e700 1 -- 172.21.15.107:0/577297307 shutdown_connections 2024-04-22T23:47:12.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:12.743+0000 7fa6aa54e700 1 --2- 172.21.15.107:0/577297307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa6a4102e10 0x7fa6a4113b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:12.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:12.743+0000 7fa6aa54e700 1 --2- 172.21.15.107:0/577297307 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fa68c06ec00 0x7fa68c0710b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:12.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:12.743+0000 7fa6aa54e700 1 --2- 172.21.15.107:0/577297307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6a41037b0 0x7fa6a41140d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:12.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:12.743+0000 7fa6aa54e700 1 -- 172.21.15.107:0/577297307 >> 172.21.15.107:0/577297307 conn(0x7fa6a40f89a0 msgr2=0x7fa6a40f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:12.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:12.743+0000 7fa6aa54e700 1 -- 172.21.15.107:0/577297307 shutdown_connections 2024-04-22T23:47:12.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:12.743+0000 7fa6aa54e700 1 -- 172.21.15.107:0/577297307 wait complete. 2024-04-22T23:47:12.745 INFO:teuthology.orchestra.run.smithi107.stdout:true 2024-04-22T23:47:13.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:12 smithi157 ceph-mon[79403]: pgmap v315: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:47:13.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:12 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/577297307' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:47:13.131 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:12 smithi107 ceph-mon[90179]: pgmap v315: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:47:13.131 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:12 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/577297307' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:47:13.451 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -e sha1=43be020184947e53516056c9931e1ac5bdbbb1a5 -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-04-22T23:47:13.706 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:47:15.083 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:14 smithi107 ceph-mon[90179]: pgmap v316: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:47:15.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:14 smithi157 ceph-mon[79403]: pgmap v316: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:47:15.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:15.314+0000 7f09bfecc700 1 -- 172.21.15.107:0/2261804537 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09b8102e10 msgr2=0x7f09b81031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:15.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:15.314+0000 7f09bfecc700 1 --2- 172.21.15.107:0/2261804537 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09b8102e10 0x7f09b81031e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f09b4004940 tx=0x7f09b4009e30 comp rx=0 tx=0).stop 2024-04-22T23:47:15.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:15.314+0000 7f09bfecc700 1 -- 172.21.15.107:0/2261804537 shutdown_connections 2024-04-22T23:47:15.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:15.314+0000 7f09bfecc700 1 --2- 172.21.15.107:0/2261804537 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09b81037b0 0x7f09b80fce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:15.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:15.314+0000 7f09bfecc700 1 --2- 172.21.15.107:0/2261804537 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09b8102e10 0x7f09b81031e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:15.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:15.314+0000 7f09bfecc700 1 -- 172.21.15.107:0/2261804537 >> 172.21.15.107:0/2261804537 conn(0x7f09b80f89a0 msgr2=0x7f09b80fadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:15.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:15.314+0000 7f09bfecc700 1 -- 172.21.15.107:0/2261804537 shutdown_connections 2024-04-22T23:47:15.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:15.314+0000 7f09bfecc700 1 -- 172.21.15.107:0/2261804537 wait complete. 2024-04-22T23:47:15.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:15.315+0000 7f09bfecc700 1 Processor -- start 2024-04-22T23:47:15.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:15.315+0000 7f09bfecc700 1 -- start start 2024-04-22T23:47:15.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:15.315+0000 7f09bfecc700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09b8102e10 0x7f09b8113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:15.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:15.315+0000 7f09bfecc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09b81037b0 0x7f09b81140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:15.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:15.315+0000 7f09bfecc700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b8114710 con 0x7f09b81037b0 2024-04-22T23:47:15.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:15.315+0000 7f09bfecc700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b810db00 con 0x7f09b8102e10 2024-04-22T23:47:15.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:15.315+0000 7f09bd467700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09b81037b0 0x7f09b81140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:15.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:15.315+0000 7f09bd467700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09b81037b0 0x7f09b81140c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44832/0 (socket says 172.21.15.107:44832) 2024-04-22T23:47:15.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:15.315+0000 7f09bd467700 1 -- 172.21.15.107:0/1407153671 learned_addr learned my addr 172.21.15.107:0/1407153671 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:47:15.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:15.315+0000 7f09bdc68700 1 --2- 172.21.15.107:0/1407153671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09b8102e10 0x7f09b8113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:15.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:15.315+0000 7f09bd467700 1 -- 172.21.15.107:0/1407153671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09b8102e10 msgr2=0x7f09b8113b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:15.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:15.315+0000 7f09bd467700 1 --2- 172.21.15.107:0/1407153671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09b8102e10 0x7f09b8113b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:15.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:15.315+0000 7f09bd467700 1 -- 172.21.15.107:0/1407153671 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b4005040 con 0x7f09b81037b0 2024-04-22T23:47:15.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:15.315+0000 7f09bdc68700 1 --2- 172.21.15.107:0/1407153671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09b8102e10 0x7f09b8113b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T23:47:15.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:15.315+0000 7f09bd467700 1 --2- 172.21.15.107:0/1407153671 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09b81037b0 0x7f09b81140c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f09a8002a50 tx=0x7f09a8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:15.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:15.316+0000 7f09ae7fc700 1 -- 172.21.15.107:0/1407153671 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f09a8007cc0 con 0x7f09b81037b0 2024-04-22T23:47:15.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:15.316+0000 7f09bfecc700 1 -- 172.21.15.107:0/1407153671 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09b810dde0 con 0x7f09b81037b0 2024-04-22T23:47:15.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:15.316+0000 7f09ae7fc700 1 -- 172.21.15.107:0/1407153671 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f09a800f040 con 0x7f09b81037b0 2024-04-22T23:47:15.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:15.316+0000 7f09ae7fc700 1 -- 172.21.15.107:0/1407153671 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f09a800daf0 con 0x7f09b81037b0 2024-04-22T23:47:15.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:15.316+0000 7f09bfecc700 1 -- 172.21.15.107:0/1407153671 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09b810e3a0 con 0x7f09b81037b0 2024-04-22T23:47:15.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:15.317+0000 7f09bfecc700 1 -- 172.21.15.107:0/1407153671 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09b8100120 con 0x7f09b81037b0 2024-04-22T23:47:15.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:15.317+0000 7f09ae7fc700 1 -- 172.21.15.107:0/1407153671 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f09a8009810 con 0x7f09b81037b0 2024-04-22T23:47:15.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:15.318+0000 7f09ae7fc700 1 --2- 172.21.15.107:0/1407153671 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f09a406ec00 0x7f09a40710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:15.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:15.318+0000 7f09ae7fc700 1 -- 172.21.15.107:0/1407153671 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f09a808e6d0 con 0x7f09b81037b0 2024-04-22T23:47:15.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:15.318+0000 7f09bdc68700 1 --2- 172.21.15.107:0/1407153671 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f09a406ec00 0x7f09a40710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:15.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:15.318+0000 7f09bdc68700 1 --2- 172.21.15.107:0/1407153671 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f09a406ec00 0x7f09a40710b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f09b4004560 tx=0x7f09b4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:15.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:15.323+0000 7f09ae7fc700 1 -- 172.21.15.107:0/1407153671 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f09a805af00 con 0x7f09b81037b0 2024-04-22T23:47:15.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:15.611+0000 7f09bfecc700 1 -- 172.21.15.107:0/1407153671 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f09b810f1d0 con 0x7f09b81037b0 2024-04-22T23:47:15.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:15.612+0000 7f09ae7fc700 1 -- 172.21.15.107:0/1407153671 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f09a805aa90 con 0x7f09b81037b0 2024-04-22T23:47:15.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:15.614+0000 7f09bfecc700 1 -- 172.21.15.107:0/1407153671 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f09a406ec00 msgr2=0x7f09a40710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:15.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:15.614+0000 7f09bfecc700 1 --2- 172.21.15.107:0/1407153671 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f09a406ec00 0x7f09a40710b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f09b4004560 tx=0x7f09b4009e30 comp rx=0 tx=0).stop 2024-04-22T23:47:15.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:15.614+0000 7f09bfecc700 1 -- 172.21.15.107:0/1407153671 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09b81037b0 msgr2=0x7f09b81140c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:15.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:15.614+0000 7f09bfecc700 1 --2- 172.21.15.107:0/1407153671 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09b81037b0 0x7f09b81140c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f09a8002a50 tx=0x7f09a8004600 comp rx=0 tx=0).stop 2024-04-22T23:47:15.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:15.614+0000 7f09bfecc700 1 -- 172.21.15.107:0/1407153671 shutdown_connections 2024-04-22T23:47:15.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:15.614+0000 7f09bfecc700 1 --2- 172.21.15.107:0/1407153671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09b8102e10 0x7f09b8113b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:15.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:15.614+0000 7f09bfecc700 1 --2- 172.21.15.107:0/1407153671 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f09a406ec00 0x7f09a40710b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:15.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:15.614+0000 7f09bfecc700 1 --2- 172.21.15.107:0/1407153671 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09b81037b0 0x7f09b81140c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:15.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:15.614+0000 7f09bfecc700 1 -- 172.21.15.107:0/1407153671 >> 172.21.15.107:0/1407153671 conn(0x7f09b80f89a0 msgr2=0x7f09b8107400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:15.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:15.614+0000 7f09bfecc700 1 -- 172.21.15.107:0/1407153671 shutdown_connections 2024-04-22T23:47:15.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:15.614+0000 7f09bfecc700 1 -- 172.21.15.107:0/1407153671 wait complete. 2024-04-22T23:47:15.616 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph version 19.0.0-3252-g43be0201 (43be020184947e53516056c9931e1ac5bdbbb1a5) squid (dev)" 2024-04-22T23:47:15.979 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:15 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/1407153671' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:47:16.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:15 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/1407153671' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-22T23:47:16.271 DEBUG:teuthology.parallel:result is None 2024-04-22T23:47:16.271 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-22T23:47:16.282 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi107.front.sepia.ceph.com 2024-04-22T23:47:16.282 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-22T23:47:16.534 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/mon.smithi107/config 2024-04-22T23:47:16.981 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:16 smithi107 ceph-mon[90179]: pgmap v317: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:47:17.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:16 smithi157 ceph-mon[79403]: pgmap v317: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:47:18.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:18.222+0000 7f6f5950e700 1 -- 172.21.15.107:0/2601406035 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f540fb4d0 msgr2=0x7f6f540fb920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:18.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:18.222+0000 7f6f5950e700 1 --2- 172.21.15.107:0/2601406035 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f540fb4d0 0x7f6f540fb920 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c004940 tx=0x7f6f3c009e30 comp rx=0 tx=0).stop 2024-04-22T23:47:18.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:18.222+0000 7f6f5950e700 1 -- 172.21.15.107:0/2601406035 shutdown_connections 2024-04-22T23:47:18.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:18.222+0000 7f6f5950e700 1 --2- 172.21.15.107:0/2601406035 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f540fb4d0 0x7f6f540fb920 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:18.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:18.222+0000 7f6f5950e700 1 --2- 172.21.15.107:0/2601406035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f54102f60 0x7f6f54103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:18.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:18.222+0000 7f6f5950e700 1 -- 172.21.15.107:0/2601406035 >> 172.21.15.107:0/2601406035 conn(0x7f6f540714c0 msgr2=0x7f6f540718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:18.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:18.222+0000 7f6f5950e700 1 -- 172.21.15.107:0/2601406035 shutdown_connections 2024-04-22T23:47:18.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:18.222+0000 7f6f5950e700 1 -- 172.21.15.107:0/2601406035 wait complete. 2024-04-22T23:47:18.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:18.222+0000 7f6f5950e700 1 Processor -- start 2024-04-22T23:47:18.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:18.223+0000 7f6f5950e700 1 -- start start 2024-04-22T23:47:18.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:18.223+0000 7f6f5950e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f540fb4d0 0x7f6f540fc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:18.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:18.223+0000 7f6f5950e700 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f54102f60 0x7f6f540fcbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:18.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:18.223+0000 7f6f5950e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f540fe650 con 0x7f6f540fb4d0 2024-04-22T23:47:18.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:18.223+0000 7f6f5950e700 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f540fd130 con 0x7f6f54102f60 2024-04-22T23:47:18.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:18.223+0000 7f6f52ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f540fb4d0 0x7f6f540fc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:18.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:18.223+0000 7f6f52ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f540fb4d0 0x7f6f540fc6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44860/0 (socket says 172.21.15.107:44860) 2024-04-22T23:47:18.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:18.223+0000 7f6f52ffd700 1 -- 172.21.15.107:0/1937457712 learned_addr learned my addr 172.21.15.107:0/1937457712 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:47:18.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:18.223+0000 7f6f4a7fc700 1 --2- 172.21.15.107:0/1937457712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f54102f60 0x7f6f540fcbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:18.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:18.223+0000 7f6f52ffd700 1 -- 172.21.15.107:0/1937457712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f54102f60 msgr2=0x7f6f540fcbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:18.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:18.223+0000 7f6f52ffd700 1 --2- 172.21.15.107:0/1937457712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f54102f60 0x7f6f540fcbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:18.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:18.223+0000 7f6f52ffd700 1 -- 172.21.15.107:0/1937457712 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f3c005040 con 0x7f6f540fb4d0 2024-04-22T23:47:18.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:18.223+0000 7f6f4a7fc700 1 --2- 172.21.15.107:0/1937457712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f54102f60 0x7f6f540fcbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T23:47:18.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:18.223+0000 7f6f52ffd700 1 --2- 172.21.15.107:0/1937457712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f540fb4d0 0x7f6f540fc6b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6f4400c8c0 tx=0x7f6f440044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:18.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:18.223+0000 7f6f4bfff700 1 -- 172.21.15.107:0/1937457712 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f4400a980 con 0x7f6f540fb4d0 2024-04-22T23:47:18.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:18.224+0000 7f6f5950e700 1 -- 172.21.15.107:0/1937457712 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f540fd3e0 con 0x7f6f540fb4d0 2024-04-22T23:47:18.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:18.224+0000 7f6f4bfff700 1 -- 172.21.15.107:0/1937457712 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6f4400ce60 con 0x7f6f540fb4d0 2024-04-22T23:47:18.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:18.224+0000 7f6f4bfff700 1 -- 172.21.15.107:0/1937457712 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f440115e0 con 0x7f6f540fb4d0 2024-04-22T23:47:18.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:18.224+0000 7f6f5950e700 1 -- 172.21.15.107:0/1937457712 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f5411a120 con 0x7f6f540fb4d0 2024-04-22T23:47:18.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:18.224+0000 7f6f5950e700 1 -- 172.21.15.107:0/1937457712 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f54105340 con 0x7f6f540fb4d0 2024-04-22T23:47:18.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:18.225+0000 7f6f4bfff700 1 -- 172.21.15.107:0/1937457712 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f6f44012400 con 0x7f6f540fb4d0 2024-04-22T23:47:18.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:18.225+0000 7f6f4bfff700 1 --2- 172.21.15.107:0/1937457712 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f6f4006ec00 0x7f6f400710b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:18.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:18.226+0000 7f6f4bfff700 1 -- 172.21.15.107:0/1937457712 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f6f4408ea30 con 0x7f6f540fb4d0 2024-04-22T23:47:18.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:18.226+0000 7f6f4a7fc700 1 --2- 172.21.15.107:0/1937457712 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f6f4006ec00 0x7f6f400710b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:18.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:18.226+0000 7f6f4a7fc700 1 --2- 172.21.15.107:0/1937457712 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f6f4006ec00 0x7f6f400710b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c004f90 tx=0x7f6f3c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:18.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:18.230+0000 7f6f4bfff700 1 -- 172.21.15.107:0/1937457712 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f6f440577b0 con 0x7f6f540fb4d0 2024-04-22T23:47:18.470 INFO:teuthology.orchestra.run.smithi107.stdout:e43 2024-04-22T23:47:18.471 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T23:47:18.471 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-22T23:47:18.471 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-04-22T23:47:18.471 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:47:18.471 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-04-22T23:47:18.471 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-04-22T23:47:18.471 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 43 2024-04-22T23:47:18.471 INFO:teuthology.orchestra.run.smithi107.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-22T23:47:18.472 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-04-22T23:32:20.146130+0000 2024-04-22T23:47:18.472 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-04-22T23:44:27.730668+0000 2024-04-22T23:47:18.472 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-04-22T23:47:18.472 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-04-22T23:47:18.472 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 60 2024-04-22T23:47:18.472 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-04-22T23:47:18.472 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-04-22T23:47:18.472 INFO:teuthology.orchestra.run.smithi107.stdout:max_xattr_size 65536 2024-04-22T23:47:18.472 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-04-22T23:47:18.472 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-04-22T23:47:18.472 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 83 2024-04-22T23:47:18.472 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-22T23:47:18.472 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 2 2024-04-22T23:47:18.473 INFO:teuthology.orchestra.run.smithi107.stdout:in 0,1 2024-04-22T23:47:18.473 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=34434,1=34438} 2024-04-22T23:47:18.473 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-04-22T23:47:18.473 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-04-22T23:47:18.473 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-04-22T23:47:18.473 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-04-22T23:47:18.473 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-04-22T23:47:18.473 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data disabled 2024-04-22T23:47:18.473 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-04-22T23:47:18.473 INFO:teuthology.orchestra.run.smithi107.stdout:bal_rank_mask -1 2024-04-22T23:47:18.473 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 1 2024-04-22T23:47:18.473 INFO:teuthology.orchestra.run.smithi107.stdout:qdb_cluster leader: 34434 members: 34434,34438 2024-04-22T23:47:18.473 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.dfvvkw{0:34434} state up:active seq 19 join_fscid=1 addr [v2:172.21.15.107:6826/418145840,v1:172.21.15.107:6827/418145840] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T23:47:18.473 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi107.ktuxjl{1:34438} state up:active seq 14 join_fscid=1 addr [v2:172.21.15.107:6828/1256497807,v1:172.21.15.107:6829/1256497807] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T23:47:18.473 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:47:18.474 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:47:18.474 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-04-22T23:47:18.474 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:47:18.474 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi157.rcaris{-1:44323} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.157:6824/3923812398,v1:172.21.15.157:6825/3923812398] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T23:47:18.474 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.cephfs.smithi157.ufywse{-1:44349} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.157:6826/3167053094,v1:172.21.15.157:6827/3167053094] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T23:47:18.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:18.478+0000 7f6f5950e700 1 -- 172.21.15.107:0/1937457712 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f6f540fdb90 con 0x7f6f540fb4d0 2024-04-22T23:47:18.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:18.478+0000 7f6f4bfff700 1 -- 172.21.15.107:0/1937457712 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 43 v43) v1 ==== 76+0+1957 (secure 0 0 0) 0x7f6f440126b0 con 0x7f6f540fb4d0 2024-04-22T23:47:18.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:18.480+0000 7f6f5950e700 1 -- 172.21.15.107:0/1937457712 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f6f4006ec00 msgr2=0x7f6f400710b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:18.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:18.480+0000 7f6f5950e700 1 --2- 172.21.15.107:0/1937457712 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f6f4006ec00 0x7f6f400710b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c004f90 tx=0x7f6f3c009e30 comp rx=0 tx=0).stop 2024-04-22T23:47:18.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:18.480+0000 7f6f5950e700 1 -- 172.21.15.107:0/1937457712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f540fb4d0 msgr2=0x7f6f540fc6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:18.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:18.480+0000 7f6f5950e700 1 --2- 172.21.15.107:0/1937457712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f540fb4d0 0x7f6f540fc6b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6f4400c8c0 tx=0x7f6f440044c0 comp rx=0 tx=0).stop 2024-04-22T23:47:18.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:18.480+0000 7f6f5950e700 1 -- 172.21.15.107:0/1937457712 shutdown_connections 2024-04-22T23:47:18.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:18.480+0000 7f6f5950e700 1 --2- 172.21.15.107:0/1937457712 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f6f4006ec00 0x7f6f400710b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:18.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:18.480+0000 7f6f5950e700 1 --2- 172.21.15.107:0/1937457712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f540fb4d0 0x7f6f540fc6b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:18.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:18.480+0000 7f6f5950e700 1 --2- 172.21.15.107:0/1937457712 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f54102f60 0x7f6f540fcbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:18.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:18.480+0000 7f6f5950e700 1 -- 172.21.15.107:0/1937457712 >> 172.21.15.107:0/1937457712 conn(0x7f6f540714c0 msgr2=0x7f6f540fa8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:18.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:18.481+0000 7f6f5950e700 1 -- 172.21.15.107:0/1937457712 shutdown_connections 2024-04-22T23:47:18.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:18.481+0000 7f6f5950e700 1 -- 172.21.15.107:0/1937457712 wait complete. 2024-04-22T23:47:18.476 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 43 2024-04-22T23:47:19.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:18 smithi157 ceph-mon[79403]: pgmap v318: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:47:19.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:18 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/1937457712' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:47:19.178 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:18 smithi107 ceph-mon[90179]: pgmap v318: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:47:19.178 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:18 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/1937457712' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T23:47:19.204 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-04-22T23:47:19.216 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T23:47:19.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.323+0000 7f70e3b9d640 1 -- 172.21.15.107:0/1448142881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70dc1021b0 msgr2=0x7f70dc102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:19.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.323+0000 7f70e3b9d640 1 --2- 172.21.15.107:0/1448142881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70dc1021b0 0x7f70dc102580 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f70d80048d0 tx=0x7f70d8009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:19.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.323+0000 7f70e3b9d640 1 -- 172.21.15.107:0/1448142881 shutdown_connections 2024-04-22T23:47:19.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.323+0000 7f70e3b9d640 1 --2- 172.21.15.107:0/1448142881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70dc102b50 0x7f70dc0fc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:19.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.323+0000 7f70e3b9d640 1 --2- 172.21.15.107:0/1448142881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70dc1021b0 0x7f70dc102580 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:19.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.323+0000 7f70e3b9d640 1 -- 172.21.15.107:0/1448142881 >> 172.21.15.107:0/1448142881 conn(0x7f70dc0f7d40 msgr2=0x7f70dc0fa150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:19.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.323+0000 7f70e3b9d640 1 -- 172.21.15.107:0/1448142881 shutdown_connections 2024-04-22T23:47:19.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.323+0000 7f70e3b9d640 1 -- 172.21.15.107:0/1448142881 wait complete. 2024-04-22T23:47:19.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.324+0000 7f70e3b9d640 1 Processor -- start 2024-04-22T23:47:19.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.324+0000 7f70e3b9d640 1 -- start start 2024-04-22T23:47:19.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.324+0000 7f70e3b9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70dc1021b0 0x7f70dc06f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:19.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.324+0000 7f70e3b9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70dc102b50 0x7f70dc06fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:19.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.324+0000 7f70e3b9d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70dc0716b0 con 0x7f70dc1021b0 2024-04-22T23:47:19.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.324+0000 7f70e3b9d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70dc070250 con 0x7f70dc102b50 2024-04-22T23:47:19.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.324+0000 7f70e1939640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70dc1021b0 0x7f70dc06f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:19.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.324+0000 7f70e1939640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70dc1021b0 0x7f70dc06f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44880/0 (socket says 172.21.15.107:44880) 2024-04-22T23:47:19.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.324+0000 7f70e1939640 1 -- 172.21.15.107:0/1441472667 learned_addr learned my addr 172.21.15.107:0/1441472667 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:47:19.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.324+0000 7f70e1138640 1 --2- 172.21.15.107:0/1441472667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70dc102b50 0x7f70dc06fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:19.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.324+0000 7f70e1939640 1 -- 172.21.15.107:0/1441472667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70dc102b50 msgr2=0x7f70dc06fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:19.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.324+0000 7f70e1939640 1 --2- 172.21.15.107:0/1441472667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70dc102b50 0x7f70dc06fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:19.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.324+0000 7f70e1939640 1 -- 172.21.15.107:0/1441472667 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d8005040 con 0x7f70dc1021b0 2024-04-22T23:47:19.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.325+0000 7f70e1939640 1 --2- 172.21.15.107:0/1441472667 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70dc1021b0 0x7f70dc06f7d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f70d8004930 tx=0x7f70d8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:19.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.325+0000 7f70d27fc640 1 -- 172.21.15.107:0/1441472667 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70d80094c0 con 0x7f70dc1021b0 2024-04-22T23:47:19.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.325+0000 7f70d27fc640 1 -- 172.21.15.107:0/1441472667 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f70d8032e50 con 0x7f70dc1021b0 2024-04-22T23:47:19.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.325+0000 7f70e3b9d640 1 -- 172.21.15.107:0/1441472667 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70dc0704d0 con 0x7f70dc1021b0 2024-04-22T23:47:19.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.325+0000 7f70d27fc640 1 -- 172.21.15.107:0/1441472667 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70d80433b0 con 0x7f70dc1021b0 2024-04-22T23:47:19.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.325+0000 7f70e3b9d640 1 -- 172.21.15.107:0/1441472667 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70dc1193a0 con 0x7f70dc1021b0 2024-04-22T23:47:19.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.326+0000 7f70e3b9d640 1 -- 172.21.15.107:0/1441472667 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70dc0fd920 con 0x7f70dc1021b0 2024-04-22T23:47:19.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.326+0000 7f70d27fc640 1 -- 172.21.15.107:0/1441472667 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f70d802f070 con 0x7f70dc1021b0 2024-04-22T23:47:19.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.326+0000 7f70d27fc640 1 --2- 172.21.15.107:0/1441472667 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f70c806eb70 0x7f70c8071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:19.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.327+0000 7f70d27fc640 1 -- 172.21.15.107:0/1441472667 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f70d80b07d0 con 0x7f70dc1021b0 2024-04-22T23:47:19.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.327+0000 7f70e1138640 1 --2- 172.21.15.107:0/1441472667 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f70c806eb70 0x7f70c8071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:19.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.327+0000 7f70e1138640 1 --2- 172.21.15.107:0/1441472667 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f70c806eb70 0x7f70c8071020 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f70cc009fd0 tx=0x7f70cc009e50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:19.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.331+0000 7f70d27fc640 1 -- 172.21.15.107:0/1441472667 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f70d80794d0 con 0x7f70dc1021b0 2024-04-22T23:47:19.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.510+0000 7f70e3b9d640 1 -- 172.21.15.107:0/1441472667 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f70dc119530 con 0x7f70dc1021b0 2024-04-22T23:47:19.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.511+0000 7f70d27fc640 1 -- 172.21.15.107:0/1441472667 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16968 (secure 0 0 0) 0x7f70d807cb90 con 0x7f70dc1021b0 2024-04-22T23:47:19.503 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:47:19.503 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":83,"fsid":"8252f6f2-00ff-11ef-bc93-c7b262605968","created":"2024-04-22T23:25:25.497133+0000","modified":"2024-04-22T23:44:07.204472+0000","last_up_change":"2024-04-22T23:42:20.528617+0000","last_in_change":"2024-04-22T23:30:44.396843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T23:29:48.249903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-22T23:32:19.078061+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-22T23:32:19.378987+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"65404290-734a-4901-bbb4-a87d3d846b06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":53,"up_thru":78,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":149725748},{"type":"v1","addr":"172.21.15.107:6803","nonce":149725748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":149725748},{"type":"v1","addr":"172.21.15.107:6805","nonce":149725748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":149725748},{"type":"v1","addr":"172.21.15.107:6809","nonce":149725748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":149725748},{"type":"v1","addr":"172.21.15.107:6807","nonce":149725748}]},"public_addr":"172.21.15.107:6803/149725748","cluster_addr":"172.21.15.107:6805/149725748","heartbeat_back_addr":"172.21.15.107:6809/149725748","heartbeat_front_addr":"172.21.15.107:6807/149725748","state":["exists","up"]},{"osd":1,"uuid":"6df381f7-b907-4d1b-bd08-fdf34e9729f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":54,"up_from":58,"up_thru":78,"down_at":55,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":997953964},{"type":"v1","addr":"172.21.15.107:6811","nonce":997953964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":997953964},{"type":"v1","addr":"172.21.15.107:6813","nonce":997953964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":997953964},{"type":"v1","addr":"172.21.15.107:6817","nonce":997953964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":997953964},{"type":"v1","addr":"172.21.15.107:6815","nonce":997953964}]},"public_addr":"172.21.15.107:6811/997953964","cluster_addr":"172.21.15.107:6813/997953964","heartbeat_back_addr":"172.21.15.107:6817/997953964","heartbeat_front_addr":"172.21.15.107:6815/997953964","state":["exists","up"]},{"osd":2,"uuid":"2c5e5c37-c1de-4e64-9c21-eacc145f1412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":59,"up_from":63,"up_thru":78,"down_at":60,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1823995026},{"type":"v1","addr":"172.21.15.107:6819","nonce":1823995026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1823995026},{"type":"v1","addr":"172.21.15.107:6821","nonce":1823995026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1823995026},{"type":"v1","addr":"172.21.15.107:6825","nonce":1823995026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1823995026},{"type":"v1","addr":"172.21.15.107:6823","nonce":1823995026}]},"public_addr":"172.21.15.107:6819/1823995026","cluster_addr":"172.21.15.107:6821/1823995026","heartbeat_back_addr":"172.21.15.107:6825/1823995026","heartbeat_front_addr":"172.21.15.107:6823/1823995026","state":["exists","up"]},{"osd":3,"uuid":"b7c9c124-9f43-4b5b-b039-c61ae30a416d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":64,"up_from":68,"up_thru":78,"down_at":65,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1553340859},{"type":"v1","addr":"172.21.15.157:6801","nonce":1553340859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1553340859},{"type":"v1","addr":"172.21.15.157:6803","nonce":1553340859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1553340859},{"type":"v1","addr":"172.21.15.157:6807","nonce":1553340859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1553340859},{"type":"v1","addr":"172.21.15.157:6805","nonce":1553340859}]},"public_addr":"172.21.15.157:6801/1553340859","cluster_addr":"172.21.15.157:6803/1553340859","heartbeat_back_addr":"172.21.15.157:6807/1553340859","heartbeat_front_addr":"172.21.15.157:6805/1553340859","state":["exists","up"]},{"osd":4,"uuid":"34a0cb61-1c2b-4515-9d74-8a3ad3f15fbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":69,"up_from":73,"up_thru":78,"down_at":70,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2579998472},{"type":"v1","addr":"172.21.15.157:6809","nonce":2579998472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2579998472},{"type":"v1","addr":"172.21.15.157:6811","nonce":2579998472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2579998472},{"type":"v1","addr":"172.21.15.157:6815","nonce":2579998472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2579998472},{"type":"v1","addr":"172.21.15.157:6813","nonce":2579998472}]},"public_addr":"172.21.15.157:6809/2579998472","cluster_addr":"172.21.15.157:6811/2579998472","heartbeat_back_addr":"172.21.15.157:6815/2579998472","heartbeat_front_addr":"172.21.15.157:6813/2579998472","state":["exists","up"]},{"osd":5,"uuid":"00934a7f-7e88-45cf-a2cc-e8a94bceaab0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":74,"up_from":78,"up_thru":78,"down_at":75,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3093110961},{"type":"v1","addr":"172.21.15.157:6817","nonce":3093110961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3093110961},{"type":"v1","addr":"172.21.15.157:6819","nonce":3093110961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3093110961},{"type":"v1","addr":"172.21.15.157:6823","nonce":3093110961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3093110961},{"type":"v1","addr":"172.21.15.157:6821","nonce":3093110961}]},"public_addr":"172.21.15.157:6817/3093110961","cluster_addr":"172.21.15.157:6819/3093110961","heartbeat_back_addr":"172.21.15.157:6823/3093110961","heartbeat_front_addr":"172.21.15.157:6821/3093110961","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:28:48.161289+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:29:18.447013+0000","dead_epoch":54},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:29:45.948165+0000","dead_epoch":59},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:30:09.825103+0000","dead_epoch":64},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:30:35.976009+0000","dead_epoch":69},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:31:01.729305+0000","dead_epoch":74}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.157:6827/4072444199":"2024-04-23T23:44:07.204437+0000","172.21.15.157:6826/4072444199":"2024-04-23T23:44:07.204437+0000","172.21.15.107:0/3783622140":"2024-04-23T23:37:55.631534+0000","172.21.15.107:0/3055151095":"2024-04-23T23:37:55.631534+0000","172.21.15.107:0/3377956937":"2024-04-23T23:25:58.816560+0000","172.21.15.107:0/2659503121":"2024-04-23T23:36:18.020700+0000","172.21.15.157:6829/3332906296":"2024-04-23T23:37:10.884172+0000","172.21.15.107:6800/307474287":"2024-04-23T23:26:29.807373+0000","172.21.15.107:6829/3335313253":"2024-04-23T23:43:26.926970+0000","172.21.15.107:6800/467262774":"2024-04-23T23:25:58.816560+0000","172.21.15.107:0/663073386":"2024-04-23T23:26:29.807373+0000","172.21.15.157:0/2260053045":"2024-04-23T23:37:10.884172+0000","172.21.15.107:0/525119004":"2024-04-23T23:36:18.020700+0000","172.21.15.107:6801/307474287":"2024-04-23T23:26:29.807373+0000","172.21.15.107:0/1065200386":"2024-04-23T23:36:18.020700+0000","172.21.15.107:6800/3427816626":"2024-04-23T23:27:33.931895+0000","172.21.15.107:0/3920905056":"2024-04-23T23:25:58.816560+0000","172.21.15.107:0/1734120985":"2024-04-23T23:27:33.931895+0000","172.21.15.157:0/278207195":"2024-04-23T23:37:10.884172+0000","172.21.15.107:0/2444490678":"2024-04-23T23:26:29.807373+0000","172.21.15.107:0/3567543485":"2024-04-23T23:36:18.020700+0000","172.21.15.157:6828/3332906296":"2024-04-23T23:37:10.884172+0000","172.21.15.107:0/1205692115":"2024-04-23T23:27:33.931895+0000","172.21.15.107:6801/467262774":"2024-04-23T23:25:58.816560+0000","172.21.15.107:0/2168608506":"2024-04-23T23:37:55.631534+0000","172.21.15.107:6800/2516490111":"2024-04-23T23:36:18.020700+0000","172.21.15.157:0/3173482279":"2024-04-23T23:37:10.884172+0000","172.21.15.107:6801/2516490111":"2024-04-23T23:36:18.020700+0000","172.21.15.157:0/3140121750":"2024-04-23T23:37:10.884172+0000","172.21.15.107:0/1029440409":"2024-04-23T23:26:29.807373+0000","172.21.15.107:6826/4144299942":"2024-04-23T23:32:35.769448+0000","172.21.15.107:6828/3335313253":"2024-04-23T23:43:26.926970+0000","172.21.15.107:6827/4144299942":"2024-04-23T23:32:35.769448+0000","172.21.15.107:6801/3427816626":"2024-04-23T23:27:33.931895+0000","172.21.15.157:0/2684888571":"2024-04-23T23:37:10.884172+0000","172.21.15.107:0/2873032378":"2024-04-23T23:37:55.631534+0000","172.21.15.107:0/3810721822":"2024-04-23T23:25:58.816560+0000","172.21.15.107:0/1888554706":"2024-04-23T23:37:55.631534+0000","172.21.15.107:0/3608791958":"2024-04-23T23:27:33.931895+0000","172.21.15.157:0/2452797907":"2024-04-23T23:37:10.884172+0000","172.21.15.107:0/3843391359":"2024-04-23T23:37:55.631534+0000","172.21.15.107:6801/2362628021":"2024-04-23T23:37:55.631534+0000","172.21.15.107:6800/2362628021":"2024-04-23T23:37:55.631534+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T23:47:19.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.513+0000 7f70e3b9d640 1 -- 172.21.15.107:0/1441472667 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f70c806eb70 msgr2=0x7f70c8071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:19.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.513+0000 7f70e3b9d640 1 --2- 172.21.15.107:0/1441472667 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f70c806eb70 0x7f70c8071020 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f70cc009fd0 tx=0x7f70cc009e50 comp rx=0 tx=0).stop 2024-04-22T23:47:19.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.513+0000 7f70e3b9d640 1 -- 172.21.15.107:0/1441472667 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70dc1021b0 msgr2=0x7f70dc06f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:19.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.513+0000 7f70e3b9d640 1 --2- 172.21.15.107:0/1441472667 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70dc1021b0 0x7f70dc06f7d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f70d8004930 tx=0x7f70d8009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:19.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.513+0000 7f70e3b9d640 1 -- 172.21.15.107:0/1441472667 shutdown_connections 2024-04-22T23:47:19.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.513+0000 7f70e3b9d640 1 --2- 172.21.15.107:0/1441472667 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f70c806eb70 0x7f70c8071020 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:19.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.513+0000 7f70e3b9d640 1 --2- 172.21.15.107:0/1441472667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70dc102b50 0x7f70dc06fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:19.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.513+0000 7f70e3b9d640 1 --2- 172.21.15.107:0/1441472667 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70dc1021b0 0x7f70dc06f7d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:19.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.513+0000 7f70e3b9d640 1 -- 172.21.15.107:0/1441472667 >> 172.21.15.107:0/1441472667 conn(0x7f70dc0f7d40 msgr2=0x7f70dc0f8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:19.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.513+0000 7f70e3b9d640 1 -- 172.21.15.107:0/1441472667 shutdown_connections 2024-04-22T23:47:19.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.513+0000 7f70e3b9d640 1 -- 172.21.15.107:0/1441472667 wait complete. 2024-04-22T23:47:19.516 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T23:29:48.249903+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-22T23:32:19.078061+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-22T23:32:19.378987+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T23:47:19.516 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T23:47:19.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.621+0000 7f0fd32ae640 1 -- 172.21.15.107:0/2149651611 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fcc1022a0 msgr2=0x7f0fcc102670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:19.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.621+0000 7f0fd32ae640 1 --2- 172.21.15.107:0/2149651611 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fcc1022a0 0x7f0fcc102670 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0fc80048d0 tx=0x7f0fc8009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:19.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.621+0000 7f0fd32ae640 1 -- 172.21.15.107:0/2149651611 shutdown_connections 2024-04-22T23:47:19.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.621+0000 7f0fd32ae640 1 --2- 172.21.15.107:0/2149651611 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fcc0fa8a0 0x7f0fcc0fad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:19.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.621+0000 7f0fd32ae640 1 --2- 172.21.15.107:0/2149651611 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fcc1022a0 0x7f0fcc102670 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:19.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.621+0000 7f0fd32ae640 1 -- 172.21.15.107:0/2149651611 >> 172.21.15.107:0/2149651611 conn(0x7f0fcc071760 msgr2=0x7f0fcc071b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:19.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.622+0000 7f0fd32ae640 1 -- 172.21.15.107:0/2149651611 shutdown_connections 2024-04-22T23:47:19.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.622+0000 7f0fd32ae640 1 -- 172.21.15.107:0/2149651611 wait complete. 2024-04-22T23:47:19.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.622+0000 7f0fd32ae640 1 Processor -- start 2024-04-22T23:47:19.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.622+0000 7f0fd32ae640 1 -- start start 2024-04-22T23:47:19.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.622+0000 7f0fd32ae640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fcc0fa8a0 0x7f0fcc0fba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:19.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.622+0000 7f0fd32ae640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fcc1022a0 0x7f0fcc0fbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:19.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.622+0000 7f0fd32ae640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc0fd960 con 0x7f0fcc0fa8a0 2024-04-22T23:47:19.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.622+0000 7f0fd32ae640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc0fc500 con 0x7f0fcc1022a0 2024-04-22T23:47:19.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.622+0000 7f0fd104a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fcc0fa8a0 0x7f0fcc0fba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:19.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.622+0000 7f0fd104a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fcc0fa8a0 0x7f0fcc0fba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44896/0 (socket says 172.21.15.107:44896) 2024-04-22T23:47:19.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.622+0000 7f0fd104a640 1 -- 172.21.15.107:0/1819970875 learned_addr learned my addr 172.21.15.107:0/1819970875 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:47:19.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.622+0000 7f0fd0849640 1 --2- 172.21.15.107:0/1819970875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fcc1022a0 0x7f0fcc0fbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:19.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.623+0000 7f0fd104a640 1 -- 172.21.15.107:0/1819970875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fcc1022a0 msgr2=0x7f0fcc0fbfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:19.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.623+0000 7f0fd104a640 1 --2- 172.21.15.107:0/1819970875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fcc1022a0 0x7f0fcc0fbfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:19.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.623+0000 7f0fd104a640 1 -- 172.21.15.107:0/1819970875 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc8005040 con 0x7f0fcc0fa8a0 2024-04-22T23:47:19.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.623+0000 7f0fd104a640 1 --2- 172.21.15.107:0/1819970875 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fcc0fa8a0 0x7f0fcc0fba80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0fc802be70 tx=0x7f0fc8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:19.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.623+0000 7f0fc1ffb640 1 -- 172.21.15.107:0/1819970875 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fc80094c0 con 0x7f0fcc0fa8a0 2024-04-22T23:47:19.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.623+0000 7f0fc1ffb640 1 -- 172.21.15.107:0/1819970875 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0fc8009620 con 0x7f0fcc0fa8a0 2024-04-22T23:47:19.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.623+0000 7f0fd32ae640 1 -- 172.21.15.107:0/1819970875 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fcc0fc780 con 0x7f0fcc0fa8a0 2024-04-22T23:47:19.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.623+0000 7f0fc1ffb640 1 -- 172.21.15.107:0/1819970875 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fc80435f0 con 0x7f0fcc0fa8a0 2024-04-22T23:47:19.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.623+0000 7f0fd32ae640 1 -- 172.21.15.107:0/1819970875 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fcc119450 con 0x7f0fcc0fa8a0 2024-04-22T23:47:19.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.624+0000 7f0fd32ae640 1 -- 172.21.15.107:0/1819970875 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fcc065340 con 0x7f0fcc0fa8a0 2024-04-22T23:47:19.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.625+0000 7f0fc1ffb640 1 -- 172.21.15.107:0/1819970875 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f0fc802f070 con 0x7f0fcc0fa8a0 2024-04-22T23:47:19.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.625+0000 7f0fc1ffb640 1 --2- 172.21.15.107:0/1819970875 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f0fb806eb80 0x7f0fb8071030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:19.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.625+0000 7f0fc1ffb640 1 -- 172.21.15.107:0/1819970875 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f0fc80b0860 con 0x7f0fcc0fa8a0 2024-04-22T23:47:19.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.625+0000 7f0fd0849640 1 --2- 172.21.15.107:0/1819970875 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f0fb806eb80 0x7f0fb8071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:19.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.626+0000 7f0fd0849640 1 --2- 172.21.15.107:0/1819970875 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f0fb806eb80 0x7f0fb8071030 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc0fd1b0 tx=0x7f0fbc009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:19.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.630+0000 7f0fc1ffb640 1 -- 172.21.15.107:0/1819970875 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f0fc8079560 con 0x7f0fcc0fa8a0 2024-04-22T23:47:19.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.811+0000 7f0fd32ae640 1 -- 172.21.15.107:0/1819970875 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0fcc0fcf60 con 0x7f0fcc0fa8a0 2024-04-22T23:47:19.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.812+0000 7f0fc1ffb640 1 -- 172.21.15.107:0/1819970875 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0fc807cc20 con 0x7f0fcc0fa8a0 2024-04-22T23:47:19.803 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2024-04-22T23:47:19.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.814+0000 7f0fd32ae640 1 -- 172.21.15.107:0/1819970875 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f0fb806eb80 msgr2=0x7f0fb8071030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:19.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.814+0000 7f0fd32ae640 1 --2- 172.21.15.107:0/1819970875 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f0fb806eb80 0x7f0fb8071030 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc0fd1b0 tx=0x7f0fbc009040 comp rx=0 tx=0).stop 2024-04-22T23:47:19.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.814+0000 7f0fd32ae640 1 -- 172.21.15.107:0/1819970875 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fcc0fa8a0 msgr2=0x7f0fcc0fba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:19.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.814+0000 7f0fd32ae640 1 --2- 172.21.15.107:0/1819970875 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fcc0fa8a0 0x7f0fcc0fba80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0fc802be70 tx=0x7f0fc8009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:19.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.814+0000 7f0fd32ae640 1 -- 172.21.15.107:0/1819970875 shutdown_connections 2024-04-22T23:47:19.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.814+0000 7f0fd32ae640 1 --2- 172.21.15.107:0/1819970875 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f0fb806eb80 0x7f0fb8071030 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:19.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.814+0000 7f0fd32ae640 1 --2- 172.21.15.107:0/1819970875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fcc1022a0 0x7f0fcc0fbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:19.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.814+0000 7f0fd32ae640 1 --2- 172.21.15.107:0/1819970875 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fcc0fa8a0 0x7f0fcc0fba80 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:19.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.814+0000 7f0fd32ae640 1 -- 172.21.15.107:0/1819970875 >> 172.21.15.107:0/1819970875 conn(0x7f0fcc071760 msgr2=0x7f0fcc0f9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:19.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.814+0000 7f0fd32ae640 1 -- 172.21.15.107:0/1819970875 shutdown_connections 2024-04-22T23:47:19.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.814+0000 7f0fd32ae640 1 -- 172.21.15.107:0/1819970875 wait complete. 2024-04-22T23:47:19.816 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-22T23:47:19.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.921+0000 7ff10c8ae640 1 -- 172.21.15.107:0/3457500467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff108102b70 msgr2=0x7ff1080fc200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:19.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.921+0000 7ff10c8ae640 1 --2- 172.21.15.107:0/3457500467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff108102b70 0x7ff1080fc200 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc0048d0 tx=0x7ff0fc009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:19.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.921+0000 7ff10c8ae640 1 -- 172.21.15.107:0/3457500467 shutdown_connections 2024-04-22T23:47:19.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.921+0000 7ff10c8ae640 1 --2- 172.21.15.107:0/3457500467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff108102b70 0x7ff1080fc200 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:19.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.921+0000 7ff10c8ae640 1 --2- 172.21.15.107:0/3457500467 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff1081021d0 0x7ff1081025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:19.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.921+0000 7ff10c8ae640 1 -- 172.21.15.107:0/3457500467 >> 172.21.15.107:0/3457500467 conn(0x7ff1080f7dc0 msgr2=0x7ff1080fa1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:19.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.921+0000 7ff10c8ae640 1 -- 172.21.15.107:0/3457500467 shutdown_connections 2024-04-22T23:47:19.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.921+0000 7ff10c8ae640 1 -- 172.21.15.107:0/3457500467 wait complete. 2024-04-22T23:47:19.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.921+0000 7ff10c8ae640 1 Processor -- start 2024-04-22T23:47:19.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.921+0000 7ff10c8ae640 1 -- start start 2024-04-22T23:47:19.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.922+0000 7ff10c8ae640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1081021d0 0x7ff10806f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:19.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.922+0000 7ff10c8ae640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff108102b70 0x7ff10806fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:19.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.922+0000 7ff10c8ae640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1080716b0 con 0x7ff1081021d0 2024-04-22T23:47:19.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.922+0000 7ff10c8ae640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff108070250 con 0x7ff108102b70 2024-04-22T23:47:19.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.922+0000 7ff10659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1081021d0 0x7ff10806f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:19.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.922+0000 7ff10659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1081021d0 0x7ff10806f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44922/0 (socket says 172.21.15.107:44922) 2024-04-22T23:47:19.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.922+0000 7ff10659c640 1 -- 172.21.15.107:0/3642419239 learned_addr learned my addr 172.21.15.107:0/3642419239 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:47:19.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.922+0000 7ff105d9b640 1 --2- 172.21.15.107:0/3642419239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff108102b70 0x7ff10806fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:19.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.922+0000 7ff10659c640 1 -- 172.21.15.107:0/3642419239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff108102b70 msgr2=0x7ff10806fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:19.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.922+0000 7ff10659c640 1 --2- 172.21.15.107:0/3642419239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff108102b70 0x7ff10806fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:19.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.922+0000 7ff10659c640 1 -- 172.21.15.107:0/3642419239 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0fc005040 con 0x7ff1081021d0 2024-04-22T23:47:19.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.922+0000 7ff10659c640 1 --2- 172.21.15.107:0/3642419239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1081021d0 0x7ff10806f7d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff0f800c830 tx=0x7ff0f8004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:19.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.922+0000 7ff0f6ffd640 1 -- 172.21.15.107:0/3642419239 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0f800a7d0 con 0x7ff1081021d0 2024-04-22T23:47:19.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.922+0000 7ff0f6ffd640 1 -- 172.21.15.107:0/3642419239 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff0f800ae00 con 0x7ff1081021d0 2024-04-22T23:47:19.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.922+0000 7ff10c8ae640 1 -- 172.21.15.107:0/3642419239 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff108070530 con 0x7ff1081021d0 2024-04-22T23:47:19.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.922+0000 7ff0f6ffd640 1 -- 172.21.15.107:0/3642419239 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0f80113a0 con 0x7ff1081021d0 2024-04-22T23:47:19.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.922+0000 7ff10c8ae640 1 -- 172.21.15.107:0/3642419239 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1081193f0 con 0x7ff1081021d0 2024-04-22T23:47:19.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.923+0000 7ff10c8ae640 1 -- 172.21.15.107:0/3642419239 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1080fd940 con 0x7ff1081021d0 2024-04-22T23:47:19.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.924+0000 7ff0f6ffd640 1 -- 172.21.15.107:0/3642419239 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7ff0f8012400 con 0x7ff1081021d0 2024-04-22T23:47:19.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.924+0000 7ff0f6ffd640 1 --2- 172.21.15.107:0/3642419239 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7ff0f006eb80 0x7ff0f0071030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:19.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.924+0000 7ff0f6ffd640 1 -- 172.21.15.107:0/3642419239 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7ff0f808ea30 con 0x7ff1081021d0 2024-04-22T23:47:19.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.924+0000 7ff105d9b640 1 --2- 172.21.15.107:0/3642419239 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7ff0f006eb80 0x7ff0f0071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:19.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.924+0000 7ff105d9b640 1 --2- 172.21.15.107:0/3642419239 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7ff0f006eb80 0x7ff0f0071030 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc0048a0 tx=0x7ff0fc009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:19.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:19.929+0000 7ff0f6ffd640 1 -- 172.21.15.107:0/3642419239 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7ff0f8057850 con 0x7ff1081021d0 2024-04-22T23:47:20.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.106+0000 7ff10c8ae640 1 -- 172.21.15.107:0/3642419239 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7ff10804bcc0 con 0x7ff1081021d0 2024-04-22T23:47:20.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.107+0000 7ff0f6ffd640 1 -- 172.21.15.107:0/3642419239 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v83) v1 ==== 107+0+11 (secure 0 0 0) 0x7ff0f805af10 con 0x7ff1081021d0 2024-04-22T23:47:20.098 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-04-22T23:47:20.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.108+0000 7ff10c8ae640 1 -- 172.21.15.107:0/3642419239 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7ff0f006eb80 msgr2=0x7ff0f0071030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:20.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.109+0000 7ff10c8ae640 1 --2- 172.21.15.107:0/3642419239 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7ff0f006eb80 0x7ff0f0071030 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc0048a0 tx=0x7ff0fc009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:20.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.109+0000 7ff10c8ae640 1 -- 172.21.15.107:0/3642419239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1081021d0 msgr2=0x7ff10806f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:20.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.109+0000 7ff10c8ae640 1 --2- 172.21.15.107:0/3642419239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1081021d0 0x7ff10806f7d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff0f800c830 tx=0x7ff0f8004430 comp rx=0 tx=0).stop 2024-04-22T23:47:20.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.109+0000 7ff10c8ae640 1 -- 172.21.15.107:0/3642419239 shutdown_connections 2024-04-22T23:47:20.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.109+0000 7ff10c8ae640 1 --2- 172.21.15.107:0/3642419239 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7ff0f006eb80 0x7ff0f0071030 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:20.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.109+0000 7ff10c8ae640 1 --2- 172.21.15.107:0/3642419239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff108102b70 0x7ff10806fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:20.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.109+0000 7ff10c8ae640 1 --2- 172.21.15.107:0/3642419239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff1081021d0 0x7ff10806f7d0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:20.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.109+0000 7ff10c8ae640 1 -- 172.21.15.107:0/3642419239 >> 172.21.15.107:0/3642419239 conn(0x7ff1080f7dc0 msgr2=0x7ff1080f8d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:20.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.109+0000 7ff10c8ae640 1 -- 172.21.15.107:0/3642419239 shutdown_connections 2024-04-22T23:47:20.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.109+0000 7ff10c8ae640 1 -- 172.21.15.107:0/3642419239 wait complete. 2024-04-22T23:47:20.110 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:19 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/1441472667' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T23:47:20.110 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:19 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/1819970875' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T23:47:20.111 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-22T23:47:20.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:19 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/1441472667' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T23:47:20.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:19 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/1819970875' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T23:47:20.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.219+0000 7f22b8b9d640 1 -- 172.21.15.107:0/2296381038 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22b4101150 msgr2=0x7f22b4101520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:20.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.219+0000 7f22b8b9d640 1 --2- 172.21.15.107:0/2296381038 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22b4101150 0x7f22b4101520 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f22a80048d0 tx=0x7f22a8009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:20.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.219+0000 7f22b8b9d640 1 -- 172.21.15.107:0/2296381038 shutdown_connections 2024-04-22T23:47:20.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.219+0000 7f22b8b9d640 1 --2- 172.21.15.107:0/2296381038 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22b40fb120 0x7f22b40fb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:20.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.219+0000 7f22b8b9d640 1 --2- 172.21.15.107:0/2296381038 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22b4101150 0x7f22b4101520 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:20.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.219+0000 7f22b8b9d640 1 -- 172.21.15.107:0/2296381038 >> 172.21.15.107:0/2296381038 conn(0x7f22b40710c0 msgr2=0x7f22b40714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:20.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.219+0000 7f22b8b9d640 1 -- 172.21.15.107:0/2296381038 shutdown_connections 2024-04-22T23:47:20.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.219+0000 7f22b8b9d640 1 -- 172.21.15.107:0/2296381038 wait complete. 2024-04-22T23:47:20.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.219+0000 7f22b8b9d640 1 Processor -- start 2024-04-22T23:47:20.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.219+0000 7f22b8b9d640 1 -- start start 2024-04-22T23:47:20.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.220+0000 7f22b8b9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22b40fb120 0x7f22b4111bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:20.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.220+0000 7f22b8b9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22b4101150 0x7f22b4112130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:20.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.220+0000 7f22b8b9d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b4112730 con 0x7f22b40fb120 2024-04-22T23:47:20.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.220+0000 7f22b8b9d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b410bd80 con 0x7f22b4101150 2024-04-22T23:47:20.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.220+0000 7f22b259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22b40fb120 0x7f22b4111bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:20.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.220+0000 7f22b259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22b40fb120 0x7f22b4111bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44938/0 (socket says 172.21.15.107:44938) 2024-04-22T23:47:20.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.220+0000 7f22b259c640 1 -- 172.21.15.107:0/3268642629 learned_addr learned my addr 172.21.15.107:0/3268642629 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:47:20.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.220+0000 7f22b1d9b640 1 --2- 172.21.15.107:0/3268642629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22b4101150 0x7f22b4112130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:20.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.220+0000 7f22b259c640 1 -- 172.21.15.107:0/3268642629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22b4101150 msgr2=0x7f22b4112130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:20.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.220+0000 7f22b259c640 1 --2- 172.21.15.107:0/3268642629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22b4101150 0x7f22b4112130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:20.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.220+0000 7f22b259c640 1 -- 172.21.15.107:0/3268642629 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a8005040 con 0x7f22b40fb120 2024-04-22T23:47:20.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.220+0000 7f22b259c640 1 --2- 172.21.15.107:0/3268642629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22b40fb120 0x7f22b4111bf0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f22a8030640 tx=0x7f22a8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:20.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.220+0000 7f22a2ffd640 1 -- 172.21.15.107:0/3268642629 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22a8030820 con 0x7f22b40fb120 2024-04-22T23:47:20.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.220+0000 7f22a2ffd640 1 -- 172.21.15.107:0/3268642629 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f22a80092e0 con 0x7f22b40fb120 2024-04-22T23:47:20.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.220+0000 7f22a2ffd640 1 -- 172.21.15.107:0/3268642629 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22a8043400 con 0x7f22b40fb120 2024-04-22T23:47:20.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.220+0000 7f22b8b9d640 1 -- 172.21.15.107:0/3268642629 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22b410c000 con 0x7f22b40fb120 2024-04-22T23:47:20.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.221+0000 7f22b8b9d640 1 -- 172.21.15.107:0/3268642629 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22b410c520 con 0x7f22b40fb120 2024-04-22T23:47:20.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.222+0000 7f22b8b9d640 1 -- 172.21.15.107:0/3268642629 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22b40fc810 con 0x7f22b40fb120 2024-04-22T23:47:20.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.222+0000 7f22a2ffd640 1 -- 172.21.15.107:0/3268642629 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f22a803b420 con 0x7f22b40fb120 2024-04-22T23:47:20.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.222+0000 7f22a2ffd640 1 --2- 172.21.15.107:0/3268642629 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f229c06eb70 0x7f229c071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:20.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.223+0000 7f22b1d9b640 1 --2- 172.21.15.107:0/3268642629 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f229c06eb70 0x7f229c071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:20.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.223+0000 7f22a2ffd640 1 -- 172.21.15.107:0/3268642629 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f22a80b16c0 con 0x7f22b40fb120 2024-04-22T23:47:20.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.223+0000 7f22b1d9b640 1 --2- 172.21.15.107:0/3268642629 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f229c06eb70 0x7f229c071020 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f22b410d4b0 tx=0x7f22a400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:20.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.227+0000 7f22a2ffd640 1 -- 172.21.15.107:0/3268642629 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f22a807a500 con 0x7f22b40fb120 2024-04-22T23:47:20.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.414+0000 7f22b8b9d640 1 -- 172.21.15.107:0/3268642629 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f22b404bcc0 con 0x7f22b40fb120 2024-04-22T23:47:20.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.414+0000 7f22a2ffd640 1 -- 172.21.15.107:0/3268642629 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v83) v1 ==== 107+0+11 (secure 0 0 0) 0x7f22a807dbc0 con 0x7f22b40fb120 2024-04-22T23:47:20.406 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-04-22T23:47:20.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.416+0000 7f22b8b9d640 1 -- 172.21.15.107:0/3268642629 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f229c06eb70 msgr2=0x7f229c071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:20.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.416+0000 7f22b8b9d640 1 --2- 172.21.15.107:0/3268642629 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f229c06eb70 0x7f229c071020 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f22b410d4b0 tx=0x7f22a400b040 comp rx=0 tx=0).stop 2024-04-22T23:47:20.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.416+0000 7f22b8b9d640 1 -- 172.21.15.107:0/3268642629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22b40fb120 msgr2=0x7f22b4111bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:20.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.416+0000 7f22b8b9d640 1 --2- 172.21.15.107:0/3268642629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22b40fb120 0x7f22b4111bf0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f22a8030640 tx=0x7f22a8009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:20.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.416+0000 7f22b8b9d640 1 -- 172.21.15.107:0/3268642629 shutdown_connections 2024-04-22T23:47:20.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.416+0000 7f22b8b9d640 1 --2- 172.21.15.107:0/3268642629 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f229c06eb70 0x7f229c071020 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:20.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.416+0000 7f22b8b9d640 1 --2- 172.21.15.107:0/3268642629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22b4101150 0x7f22b4112130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:20.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.416+0000 7f22b8b9d640 1 --2- 172.21.15.107:0/3268642629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22b40fb120 0x7f22b4111bf0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:20.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.416+0000 7f22b8b9d640 1 -- 172.21.15.107:0/3268642629 >> 172.21.15.107:0/3268642629 conn(0x7f22b40710c0 msgr2=0x7f22b40f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:20.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.417+0000 7f22b8b9d640 1 -- 172.21.15.107:0/3268642629 shutdown_connections 2024-04-22T23:47:20.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.417+0000 7f22b8b9d640 1 -- 172.21.15.107:0/3268642629 wait complete. 2024-04-22T23:47:20.419 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T23:47:20.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.525+0000 7fd99c66e640 1 -- 172.21.15.107:0/1207858763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd994070120 msgr2=0x7fd994108560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:20.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.525+0000 7fd99c66e640 1 --2- 172.21.15.107:0/1207858763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd994070120 0x7fd994108560 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd9840048d0 tx=0x7fd984009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:20.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.525+0000 7fd99c66e640 1 -- 172.21.15.107:0/1207858763 shutdown_connections 2024-04-22T23:47:20.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.525+0000 7fd99c66e640 1 --2- 172.21.15.107:0/1207858763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd994070120 0x7fd994108560 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:20.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.525+0000 7fd99c66e640 1 --2- 172.21.15.107:0/1207858763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd99406f810 0x7fd99406fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:20.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.525+0000 7fd99c66e640 1 -- 172.21.15.107:0/1207858763 >> 172.21.15.107:0/1207858763 conn(0x7fd9940f7da0 msgr2=0x7fd9940fa1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:20.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.525+0000 7fd99c66e640 1 -- 172.21.15.107:0/1207858763 shutdown_connections 2024-04-22T23:47:20.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.525+0000 7fd99c66e640 1 -- 172.21.15.107:0/1207858763 wait complete. 2024-04-22T23:47:20.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.526+0000 7fd99c66e640 1 Processor -- start 2024-04-22T23:47:20.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.526+0000 7fd99c66e640 1 -- start start 2024-04-22T23:47:20.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.526+0000 7fd99c66e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd99406f810 0x7fd994115010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:20.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.526+0000 7fd99c66e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd994070120 0x7fd994115550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:20.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.526+0000 7fd99c66e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd994115b50 con 0x7fd99406f810 2024-04-22T23:47:20.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.526+0000 7fd99c66e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd99410f1a0 con 0x7fd994070120 2024-04-22T23:47:20.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.526+0000 7fd99a40a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd99406f810 0x7fd994115010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:20.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.526+0000 7fd99a40a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd99406f810 0x7fd994115010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44958/0 (socket says 172.21.15.107:44958) 2024-04-22T23:47:20.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.526+0000 7fd99a40a640 1 -- 172.21.15.107:0/2205688965 learned_addr learned my addr 172.21.15.107:0/2205688965 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:47:20.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.526+0000 7fd999c09640 1 --2- 172.21.15.107:0/2205688965 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd994070120 0x7fd994115550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:20.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.526+0000 7fd99a40a640 1 -- 172.21.15.107:0/2205688965 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd994070120 msgr2=0x7fd994115550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:20.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.526+0000 7fd99a40a640 1 --2- 172.21.15.107:0/2205688965 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd994070120 0x7fd994115550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:20.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.526+0000 7fd99a40a640 1 -- 172.21.15.107:0/2205688965 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd984005040 con 0x7fd99406f810 2024-04-22T23:47:20.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.526+0000 7fd99a40a640 1 --2- 172.21.15.107:0/2205688965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd99406f810 0x7fd994115010 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd990002a50 tx=0x7fd990004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:20.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.527+0000 7fd98affd640 1 -- 172.21.15.107:0/2205688965 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd990007c30 con 0x7fd99406f810 2024-04-22T23:47:20.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.527+0000 7fd98affd640 1 -- 172.21.15.107:0/2205688965 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd99000f040 con 0x7fd99406f810 2024-04-22T23:47:20.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.527+0000 7fd99c66e640 1 -- 172.21.15.107:0/2205688965 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd99410f480 con 0x7fd99406f810 2024-04-22T23:47:20.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.527+0000 7fd98affd640 1 -- 172.21.15.107:0/2205688965 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd99000db50 con 0x7fd99406f810 2024-04-22T23:47:20.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.527+0000 7fd99c66e640 1 -- 172.21.15.107:0/2205688965 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd99410f970 con 0x7fd99406f810 2024-04-22T23:47:20.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.528+0000 7fd99c66e640 1 -- 172.21.15.107:0/2205688965 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd994070f50 con 0x7fd99406f810 2024-04-22T23:47:20.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.528+0000 7fd98affd640 1 -- 172.21.15.107:0/2205688965 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fd99000dcf0 con 0x7fd99406f810 2024-04-22T23:47:20.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.529+0000 7fd98affd640 1 --2- 172.21.15.107:0/2205688965 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fd98006eb70 0x7fd980071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:20.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.529+0000 7fd98affd640 1 -- 172.21.15.107:0/2205688965 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fd99008e190 con 0x7fd99406f810 2024-04-22T23:47:20.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.529+0000 7fd999c09640 1 --2- 172.21.15.107:0/2205688965 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fd98006eb70 0x7fd980071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:20.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.529+0000 7fd999c09640 1 --2- 172.21.15.107:0/2205688965 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fd98006eb70 0x7fd980071020 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd994100950 tx=0x7fd984009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:20.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.533+0000 7fd98affd640 1 -- 172.21.15.107:0/2205688965 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fd990056f10 con 0x7fd99406f810 2024-04-22T23:47:20.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.765+0000 7fd99c66e640 1 -- 172.21.15.107:0/2205688965 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd994110680 con 0x7fd99406f810 2024-04-22T23:47:20.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.766+0000 7fd98affd640 1 -- 172.21.15.107:0/2205688965 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+5239 (secure 0 0 0) 0x7fd99005a5d0 con 0x7fd99406f810 2024-04-22T23:47:20.758 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:47:20.758 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44323,"name":"cephfs.smithi157.rcaris","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6825/3923812398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3923812398},{"type":"v1","addr":"172.21.15.157:6825","nonce":3923812398}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":44349,"name":"cephfs.smithi157.ufywse","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6827/3167053094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":3167053094},{"type":"v1","addr":"172.21.15.157:6827","nonce":3167053094}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T23:32:20.146130+0000","modified":"2024-04-22T23:44:27.730668+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34434,"mds_1":34438},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34434":{"gid":34434,"name":"cephfs.smithi107.dfvvkw","rank":0,"incarnation":36,"state":"up:active","state_seq":19,"addr":"172.21.15.107:6827/418145840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":418145840},{"type":"v1","addr":"172.21.15.107:6827","nonce":418145840}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34438":{"gid":34438,"name":"cephfs.smithi107.ktuxjl","rank":1,"incarnation":42,"state":"up:active","state_seq":14,"addr":"172.21.15.107:6829/1256497807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1256497807},{"type":"v1","addr":"172.21.15.107:6829","nonce":1256497807}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34434,"qdb_cluster":[34434,34438]},"id":1}]} 2024-04-22T23:47:20.758 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 43 2024-04-22T23:47:20.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.768+0000 7fd99c66e640 1 -- 172.21.15.107:0/2205688965 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fd98006eb70 msgr2=0x7fd980071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:20.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.768+0000 7fd99c66e640 1 --2- 172.21.15.107:0/2205688965 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fd98006eb70 0x7fd980071020 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd994100950 tx=0x7fd984009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:20.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.768+0000 7fd99c66e640 1 -- 172.21.15.107:0/2205688965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd99406f810 msgr2=0x7fd994115010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:20.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.768+0000 7fd99c66e640 1 --2- 172.21.15.107:0/2205688965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd99406f810 0x7fd994115010 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd990002a50 tx=0x7fd990004600 comp rx=0 tx=0).stop 2024-04-22T23:47:20.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.768+0000 7fd99c66e640 1 -- 172.21.15.107:0/2205688965 shutdown_connections 2024-04-22T23:47:20.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.768+0000 7fd99c66e640 1 --2- 172.21.15.107:0/2205688965 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fd98006eb70 0x7fd980071020 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:20.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.768+0000 7fd99c66e640 1 --2- 172.21.15.107:0/2205688965 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd994070120 0x7fd994115550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:20.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.768+0000 7fd99c66e640 1 --2- 172.21.15.107:0/2205688965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd99406f810 0x7fd994115010 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:20.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.768+0000 7fd99c66e640 1 -- 172.21.15.107:0/2205688965 >> 172.21.15.107:0/2205688965 conn(0x7fd9940f7da0 msgr2=0x7fd9940f92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:20.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.768+0000 7fd99c66e640 1 -- 172.21.15.107:0/2205688965 shutdown_connections 2024-04-22T23:47:20.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.768+0000 7fd99c66e640 1 -- 172.21.15.107:0/2205688965 wait complete. 2024-04-22T23:47:20.771 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 17, 'max_mds': 2, 'flags': 18} 2024-04-22T23:47:20.771 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 18 2024-04-22T23:47:20.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.877+0000 7f4c98c6e640 1 -- 172.21.15.107:0/4271221551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c94101c90 msgr2=0x7f4c940fb320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:20.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.877+0000 7f4c98c6e640 1 --2- 172.21.15.107:0/4271221551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c94101c90 0x7f4c940fb320 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c0048d0 tx=0x7f4c7c009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:20.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.877+0000 7f4c98c6e640 1 -- 172.21.15.107:0/4271221551 shutdown_connections 2024-04-22T23:47:20.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.877+0000 7f4c98c6e640 1 --2- 172.21.15.107:0/4271221551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c94101c90 0x7f4c940fb320 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:20.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.877+0000 7f4c98c6e640 1 --2- 172.21.15.107:0/4271221551 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c941012f0 0x7f4c941016c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:20.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.877+0000 7f4c98c6e640 1 -- 172.21.15.107:0/4271221551 >> 172.21.15.107:0/4271221551 conn(0x7f4c940f6e80 msgr2=0x7f4c940f9290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:20.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.877+0000 7f4c98c6e640 1 -- 172.21.15.107:0/4271221551 shutdown_connections 2024-04-22T23:47:20.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.877+0000 7f4c98c6e640 1 -- 172.21.15.107:0/4271221551 wait complete. 2024-04-22T23:47:20.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.878+0000 7f4c98c6e640 1 Processor -- start 2024-04-22T23:47:20.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.878+0000 7f4c98c6e640 1 -- start start 2024-04-22T23:47:20.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.878+0000 7f4c98c6e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c941012f0 0x7f4c94111e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:20.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.878+0000 7f4c98c6e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c94101c90 0x7f4c94112380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:20.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.878+0000 7f4c98c6e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c94112980 con 0x7f4c94101c90 2024-04-22T23:47:20.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.878+0000 7f4c98c6e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9410bfd0 con 0x7f4c941012f0 2024-04-22T23:47:20.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.878+0000 7f4c8bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c94101c90 0x7f4c94112380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:20.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.878+0000 7f4c8bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c94101c90 0x7f4c94112380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44982/0 (socket says 172.21.15.107:44982) 2024-04-22T23:47:20.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.878+0000 7f4c8bfff640 1 -- 172.21.15.107:0/402230686 learned_addr learned my addr 172.21.15.107:0/402230686 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:47:20.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.878+0000 7f4c9259c640 1 --2- 172.21.15.107:0/402230686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c941012f0 0x7f4c94111e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:20.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.878+0000 7f4c8bfff640 1 -- 172.21.15.107:0/402230686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c941012f0 msgr2=0x7f4c94111e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:20.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.878+0000 7f4c8bfff640 1 --2- 172.21.15.107:0/402230686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c941012f0 0x7f4c94111e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:20.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.878+0000 7f4c8bfff640 1 -- 172.21.15.107:0/402230686 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c7c005040 con 0x7f4c94101c90 2024-04-22T23:47:20.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.878+0000 7f4c9259c640 1 --2- 172.21.15.107:0/402230686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c941012f0 0x7f4c94111e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T23:47:20.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.878+0000 7f4c8bfff640 1 --2- 172.21.15.107:0/402230686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c94101c90 0x7f4c94112380 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c004930 tx=0x7f4c7c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:20.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.879+0000 7f4c8affd640 1 -- 172.21.15.107:0/402230686 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c7c0094c0 con 0x7f4c94101c90 2024-04-22T23:47:20.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.879+0000 7f4c98c6e640 1 -- 172.21.15.107:0/402230686 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c9410c250 con 0x7f4c94101c90 2024-04-22T23:47:20.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.879+0000 7f4c8affd640 1 -- 172.21.15.107:0/402230686 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4c7c032e50 con 0x7f4c94101c90 2024-04-22T23:47:20.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.879+0000 7f4c8affd640 1 -- 172.21.15.107:0/402230686 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c7c0433b0 con 0x7f4c94101c90 2024-04-22T23:47:20.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.879+0000 7f4c98c6e640 1 -- 172.21.15.107:0/402230686 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c9410c710 con 0x7f4c94101c90 2024-04-22T23:47:20.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.880+0000 7f4c98c6e640 1 -- 172.21.15.107:0/402230686 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c940fca60 con 0x7f4c94101c90 2024-04-22T23:47:20.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.880+0000 7f4c8affd640 1 -- 172.21.15.107:0/402230686 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f4c7c02f070 con 0x7f4c94101c90 2024-04-22T23:47:20.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.881+0000 7f4c8affd640 1 --2- 172.21.15.107:0/402230686 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f4c8006eb70 0x7f4c80071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:20.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.881+0000 7f4c8affd640 1 -- 172.21.15.107:0/402230686 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f4c7c0b0960 con 0x7f4c94101c90 2024-04-22T23:47:20.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.881+0000 7f4c9259c640 1 --2- 172.21.15.107:0/402230686 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f4c8006eb70 0x7f4c80071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:20.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.881+0000 7f4c9259c640 1 --2- 172.21.15.107:0/402230686 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f4c8006eb70 0x7f4c80071020 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4c84004c80 tx=0x7f4c84009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:20.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:20.889+0000 7f4c8affd640 1 -- 172.21.15.107:0/402230686 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f4c7c079660 con 0x7f4c94101c90 2024-04-22T23:47:21.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.108+0000 7f4c98c6e640 1 -- 172.21.15.107:0/402230686 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 18, "format": "json"} v 0) v1 -- 0x7f4c94108430 con 0x7f4c94101c90 2024-04-22T23:47:21.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.109+0000 7f4c8affd640 1 -- 172.21.15.107:0/402230686 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 18, "format": "json"}]=0 dumped fsmap epoch 18 v43) v1 ==== 107+0+4885 (secure 0 0 0) 0x7f4c7c07cd20 con 0x7f4c94101c90 2024-04-22T23:47:21.101 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:47:21.101 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14484,"name":"cephfs.smithi107.dfvvkw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6827/3548717247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3548717247},{"type":"v1","addr":"172.21.15.107:6827","nonce":3548717247}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":24277,"name":"cephfs.smithi157.ufywse","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.157:6827/4072444199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":4072444199},{"type":"v1","addr":"172.21.15.157:6827","nonce":4072444199}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T23:32:20.146130+0000","modified":"2024-04-22T23:34:31.577195+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14468,"mds_1":24265},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14468":{"gid":14468,"name":"cephfs.smithi107.ktuxjl","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.107:6829/3335313253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3335313253},{"type":"v1","addr":"172.21.15.107:6829","nonce":3335313253}]},"join_fscid":1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24265":{"gid":24265,"name":"cephfs.smithi157.rcaris","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6825/2779251893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2779251893},{"type":"v1","addr":"172.21.15.157:6825","nonce":2779251893}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-22T23:47:21.101 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 18 2024-04-22T23:47:21.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.111+0000 7f4c98c6e640 1 -- 172.21.15.107:0/402230686 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f4c8006eb70 msgr2=0x7f4c80071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:21.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.111+0000 7f4c98c6e640 1 --2- 172.21.15.107:0/402230686 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f4c8006eb70 0x7f4c80071020 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4c84004c80 tx=0x7f4c84009040 comp rx=0 tx=0).stop 2024-04-22T23:47:21.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.111+0000 7f4c98c6e640 1 -- 172.21.15.107:0/402230686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c94101c90 msgr2=0x7f4c94112380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:21.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.111+0000 7f4c98c6e640 1 --2- 172.21.15.107:0/402230686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c94101c90 0x7f4c94112380 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c004930 tx=0x7f4c7c009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:21.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.111+0000 7f4c98c6e640 1 -- 172.21.15.107:0/402230686 shutdown_connections 2024-04-22T23:47:21.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.111+0000 7f4c98c6e640 1 --2- 172.21.15.107:0/402230686 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f4c8006eb70 0x7f4c80071020 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:21.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.111+0000 7f4c98c6e640 1 --2- 172.21.15.107:0/402230686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c94101c90 0x7f4c94112380 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:21.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.111+0000 7f4c98c6e640 1 --2- 172.21.15.107:0/402230686 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4c941012f0 0x7f4c94111e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:21.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.111+0000 7f4c98c6e640 1 -- 172.21.15.107:0/402230686 >> 172.21.15.107:0/402230686 conn(0x7f4c940f6e80 msgr2=0x7f4c940f7e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:21.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.112+0000 7f4c98c6e640 1 -- 172.21.15.107:0/402230686 shutdown_connections 2024-04-22T23:47:21.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.112+0000 7f4c98c6e640 1 -- 172.21.15.107:0/402230686 wait complete. 2024-04-22T23:47:21.113 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:20 smithi107 ceph-mon[90179]: pgmap v319: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:47:21.113 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:20 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/3642419239' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-22T23:47:21.113 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:20 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/3268642629' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-22T23:47:21.113 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:20 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/2205688965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T23:47:21.114 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 19 2024-04-22T23:47:21.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:20 smithi157 ceph-mon[79403]: pgmap v319: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:47:21.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:20 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/3642419239' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-22T23:47:21.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:20 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/3268642629' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-22T23:47:21.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:20 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/2205688965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T23:47:21.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.218+0000 7f054279d640 1 -- 172.21.15.107:0/4065081321 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f053c0fb120 msgr2=0x7f053c0fb570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:21.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.218+0000 7f054279d640 1 --2- 172.21.15.107:0/4065081321 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f053c0fb120 0x7f053c0fb570 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f05300048d0 tx=0x7f0530009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:21.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.218+0000 7f054279d640 1 -- 172.21.15.107:0/4065081321 shutdown_connections 2024-04-22T23:47:21.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.218+0000 7f054279d640 1 --2- 172.21.15.107:0/4065081321 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f053c0fb120 0x7f053c0fb570 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:21.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.218+0000 7f054279d640 1 --2- 172.21.15.107:0/4065081321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f053c101150 0x7f053c101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:21.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.218+0000 7f054279d640 1 -- 172.21.15.107:0/4065081321 >> 172.21.15.107:0/4065081321 conn(0x7f053c0710c0 msgr2=0x7f053c0714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:21.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.218+0000 7f054279d640 1 -- 172.21.15.107:0/4065081321 shutdown_connections 2024-04-22T23:47:21.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.218+0000 7f054279d640 1 -- 172.21.15.107:0/4065081321 wait complete. 2024-04-22T23:47:21.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.218+0000 7f054279d640 1 Processor -- start 2024-04-22T23:47:21.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.218+0000 7f054279d640 1 -- start start 2024-04-22T23:47:21.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.219+0000 7f054279d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f053c0fb120 0x7f053c0fcd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:21.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.219+0000 7f054279d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f053c101150 0x7f053c0fd280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:21.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.219+0000 7f054279d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053c0fec20 con 0x7f053c0fb120 2024-04-22T23:47:21.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.219+0000 7f054279d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053c0fd7c0 con 0x7f053c101150 2024-04-22T23:47:21.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.219+0000 7f053bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f053c0fb120 0x7f053c0fcd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:21.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.219+0000 7f053bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f053c0fb120 0x7f053c0fcd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44992/0 (socket says 172.21.15.107:44992) 2024-04-22T23:47:21.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.219+0000 7f053bfff640 1 -- 172.21.15.107:0/3976655735 learned_addr learned my addr 172.21.15.107:0/3976655735 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:47:21.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.219+0000 7f053b7fe640 1 --2- 172.21.15.107:0/3976655735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f053c101150 0x7f053c0fd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:21.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.219+0000 7f053bfff640 1 -- 172.21.15.107:0/3976655735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f053c101150 msgr2=0x7f053c0fd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:21.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.219+0000 7f053bfff640 1 --2- 172.21.15.107:0/3976655735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f053c101150 0x7f053c0fd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:21.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.219+0000 7f053bfff640 1 -- 172.21.15.107:0/3976655735 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0530005040 con 0x7f053c0fb120 2024-04-22T23:47:21.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.219+0000 7f053bfff640 1 --2- 172.21.15.107:0/3976655735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f053c0fb120 0x7f053c0fcd40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f052c002a50 tx=0x7f052c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:21.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.219+0000 7f0538ff9640 1 -- 172.21.15.107:0/3976655735 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f052c007c30 con 0x7f053c0fb120 2024-04-22T23:47:21.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.219+0000 7f0538ff9640 1 -- 172.21.15.107:0/3976655735 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f052c00f040 con 0x7f053c0fb120 2024-04-22T23:47:21.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.219+0000 7f054279d640 1 -- 172.21.15.107:0/3976655735 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f053c0fdaa0 con 0x7f053c0fb120 2024-04-22T23:47:21.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.219+0000 7f0538ff9640 1 -- 172.21.15.107:0/3976655735 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f052c00db50 con 0x7f053c0fb120 2024-04-22T23:47:21.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.219+0000 7f054279d640 1 -- 172.21.15.107:0/3976655735 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f053c119430 con 0x7f053c0fb120 2024-04-22T23:47:21.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.220+0000 7f054279d640 1 -- 172.21.15.107:0/3976655735 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f053c0fc810 con 0x7f053c0fb120 2024-04-22T23:47:21.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.221+0000 7f0538ff9640 1 -- 172.21.15.107:0/3976655735 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f052c01c400 con 0x7f053c0fb120 2024-04-22T23:47:21.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.222+0000 7f0538ff9640 1 --2- 172.21.15.107:0/3976655735 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f052406eb70 0x7f0524071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:21.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.222+0000 7f0538ff9640 1 -- 172.21.15.107:0/3976655735 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f052c08e290 con 0x7f053c0fb120 2024-04-22T23:47:21.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.222+0000 7f053b7fe640 1 --2- 172.21.15.107:0/3976655735 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f052406eb70 0x7f0524071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:21.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.222+0000 7f053b7fe640 1 --2- 172.21.15.107:0/3976655735 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f052406eb70 0x7f0524071020 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0530009fa0 tx=0x7f0530009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:21.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.226+0000 7f0538ff9640 1 -- 172.21.15.107:0/3976655735 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f052c056f90 con 0x7f053c0fb120 2024-04-22T23:47:21.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.446+0000 7f054279d640 1 -- 172.21.15.107:0/3976655735 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 19, "format": "json"} v 0) v1 -- 0x7f053c04bcc0 con 0x7f053c0fb120 2024-04-22T23:47:21.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.447+0000 7f0538ff9640 1 -- 172.21.15.107:0/3976655735 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 19, "format": "json"}]=0 dumped fsmap epoch 19 v43) v1 ==== 107+0+4886 (secure 0 0 0) 0x7f052c05a650 con 0x7f053c0fb120 2024-04-22T23:47:21.439 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:47:21.439 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14484,"name":"cephfs.smithi107.dfvvkw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6827/3548717247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3548717247},{"type":"v1","addr":"172.21.15.107:6827","nonce":3548717247}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":24277,"name":"cephfs.smithi157.ufywse","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.157:6827/4072444199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":4072444199},{"type":"v1","addr":"172.21.15.157:6827","nonce":4072444199}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T23:32:20.146130+0000","modified":"2024-04-22T23:35:12.338678+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14468,"mds_1":24265},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14468":{"gid":14468,"name":"cephfs.smithi107.ktuxjl","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.107:6829/3335313253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3335313253},{"type":"v1","addr":"172.21.15.107:6829","nonce":3335313253}]},"join_fscid":1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24265":{"gid":24265,"name":"cephfs.smithi157.rcaris","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6825/2779251893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2779251893},{"type":"v1","addr":"172.21.15.157:6825","nonce":2779251893}]},"join_fscid":1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-22T23:47:21.439 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 19 2024-04-22T23:47:21.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.449+0000 7f054279d640 1 -- 172.21.15.107:0/3976655735 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f052406eb70 msgr2=0x7f0524071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:21.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.449+0000 7f054279d640 1 --2- 172.21.15.107:0/3976655735 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f052406eb70 0x7f0524071020 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0530009fa0 tx=0x7f0530009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:21.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.449+0000 7f054279d640 1 -- 172.21.15.107:0/3976655735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f053c0fb120 msgr2=0x7f053c0fcd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:21.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.449+0000 7f054279d640 1 --2- 172.21.15.107:0/3976655735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f053c0fb120 0x7f053c0fcd40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f052c002a50 tx=0x7f052c004600 comp rx=0 tx=0).stop 2024-04-22T23:47:21.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.449+0000 7f054279d640 1 -- 172.21.15.107:0/3976655735 shutdown_connections 2024-04-22T23:47:21.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.449+0000 7f054279d640 1 --2- 172.21.15.107:0/3976655735 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f052406eb70 0x7f0524071020 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:21.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.449+0000 7f054279d640 1 --2- 172.21.15.107:0/3976655735 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f053c101150 0x7f053c0fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:21.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.449+0000 7f054279d640 1 --2- 172.21.15.107:0/3976655735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f053c0fb120 0x7f053c0fcd40 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:21.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.449+0000 7f054279d640 1 -- 172.21.15.107:0/3976655735 >> 172.21.15.107:0/3976655735 conn(0x7f053c0710c0 msgr2=0x7f053c0f9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:21.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.449+0000 7f054279d640 1 -- 172.21.15.107:0/3976655735 shutdown_connections 2024-04-22T23:47:21.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.449+0000 7f054279d640 1 -- 172.21.15.107:0/3976655735 wait complete. 2024-04-22T23:47:21.451 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 20 2024-04-22T23:47:21.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.555+0000 7f573739d640 1 -- 172.21.15.107:0/3968281969 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57300fa8a0 msgr2=0x7f57300fad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:21.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.555+0000 7f573739d640 1 --2- 172.21.15.107:0/3968281969 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57300fa8a0 0x7f57300fad10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f57200048d0 tx=0x7f5720009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:21.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.555+0000 7f573739d640 1 -- 172.21.15.107:0/3968281969 shutdown_connections 2024-04-22T23:47:21.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.555+0000 7f573739d640 1 --2- 172.21.15.107:0/3968281969 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57300fa8a0 0x7f57300fad10 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:21.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.555+0000 7f573739d640 1 --2- 172.21.15.107:0/3968281969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57301022a0 0x7f5730102670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:21.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.555+0000 7f573739d640 1 -- 172.21.15.107:0/3968281969 >> 172.21.15.107:0/3968281969 conn(0x7f5730071760 msgr2=0x7f5730071b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:21.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.555+0000 7f573739d640 1 -- 172.21.15.107:0/3968281969 shutdown_connections 2024-04-22T23:47:21.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.555+0000 7f573739d640 1 -- 172.21.15.107:0/3968281969 wait complete. 2024-04-22T23:47:21.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.555+0000 7f573739d640 1 Processor -- start 2024-04-22T23:47:21.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.555+0000 7f573739d640 1 -- start start 2024-04-22T23:47:21.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.556+0000 7f573739d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57300fa8a0 0x7f5730112d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:21.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.556+0000 7f573739d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57301022a0 0x7f5730113290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:21.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.556+0000 7f573739d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5730113890 con 0x7f57301022a0 2024-04-22T23:47:21.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.556+0000 7f573739d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f573010cf30 con 0x7f57300fa8a0 2024-04-22T23:47:21.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.556+0000 7f5734938640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57301022a0 0x7f5730113290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:21.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.556+0000 7f5734938640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57301022a0 0x7f5730113290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38482/0 (socket says 172.21.15.107:38482) 2024-04-22T23:47:21.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.556+0000 7f5734938640 1 -- 172.21.15.107:0/3034192588 learned_addr learned my addr 172.21.15.107:0/3034192588 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:47:21.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.556+0000 7f5735139640 1 --2- 172.21.15.107:0/3034192588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57300fa8a0 0x7f5730112d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:21.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.556+0000 7f5734938640 1 -- 172.21.15.107:0/3034192588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57300fa8a0 msgr2=0x7f5730112d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:21.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.556+0000 7f5734938640 1 --2- 172.21.15.107:0/3034192588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57300fa8a0 0x7f5730112d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:21.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.556+0000 7f5734938640 1 -- 172.21.15.107:0/3034192588 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5720005040 con 0x7f57301022a0 2024-04-22T23:47:21.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.556+0000 7f5734938640 1 --2- 172.21.15.107:0/3034192588 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57301022a0 0x7f5730113290 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5720004930 tx=0x7f5720009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:21.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.556+0000 7f5725ffb640 1 -- 172.21.15.107:0/3034192588 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57200094c0 con 0x7f57301022a0 2024-04-22T23:47:21.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.556+0000 7f573739d640 1 -- 172.21.15.107:0/3034192588 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f573010d1b0 con 0x7f57301022a0 2024-04-22T23:47:21.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.556+0000 7f5725ffb640 1 -- 172.21.15.107:0/3034192588 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5720032e50 con 0x7f57301022a0 2024-04-22T23:47:21.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.556+0000 7f5725ffb640 1 -- 172.21.15.107:0/3034192588 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57200433b0 con 0x7f57301022a0 2024-04-22T23:47:21.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.557+0000 7f573739d640 1 -- 172.21.15.107:0/3034192588 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f573010d670 con 0x7f57301022a0 2024-04-22T23:47:21.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.558+0000 7f573739d640 1 -- 172.21.15.107:0/3034192588 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5730065340 con 0x7f57301022a0 2024-04-22T23:47:21.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.558+0000 7f5725ffb640 1 -- 172.21.15.107:0/3034192588 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f572002f070 con 0x7f57301022a0 2024-04-22T23:47:21.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.558+0000 7f5725ffb640 1 --2- 172.21.15.107:0/3034192588 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f571c06eb70 0x7f571c071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:21.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.559+0000 7f5725ffb640 1 -- 172.21.15.107:0/3034192588 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f57200b0760 con 0x7f57301022a0 2024-04-22T23:47:21.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.559+0000 7f5735139640 1 --2- 172.21.15.107:0/3034192588 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f571c06eb70 0x7f571c071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:21.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.559+0000 7f5735139640 1 --2- 172.21.15.107:0/3034192588 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f571c06eb70 0x7f571c071020 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5730064ea0 tx=0x7f5728009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:21.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.563+0000 7f5725ffb640 1 -- 172.21.15.107:0/3034192588 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f5720079510 con 0x7f57301022a0 2024-04-22T23:47:21.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.779+0000 7f573739d640 1 -- 172.21.15.107:0/3034192588 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 20, "format": "json"} v 0) v1 -- 0x7f5730109460 con 0x7f57301022a0 2024-04-22T23:47:21.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.780+0000 7f5725ffb640 1 -- 172.21.15.107:0/3034192588 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 20, "format": "json"}]=0 dumped fsmap epoch 20 v43) v1 ==== 107+0+4900 (secure 0 0 0) 0x7f572007cbd0 con 0x7f57301022a0 2024-04-22T23:47:21.772 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:47:21.772 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14484,"name":"cephfs.smithi107.dfvvkw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6827/3548717247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3548717247},{"type":"v1","addr":"172.21.15.107:6827","nonce":3548717247}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":24277,"name":"cephfs.smithi157.ufywse","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.157:6827/4072444199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":4072444199},{"type":"v1","addr":"172.21.15.157:6827","nonce":4072444199}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T23:32:20.146130+0000","modified":"2024-04-22T23:39:02.713696+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14468,"mds_1":24265},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14468":{"gid":14468,"name":"cephfs.smithi107.ktuxjl","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.107:6829/3335313253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3335313253},{"type":"v1","addr":"172.21.15.107:6829","nonce":3335313253}]},"join_fscid":1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24265":{"gid":24265,"name":"cephfs.smithi157.rcaris","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6825/2779251893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2779251893},{"type":"v1","addr":"172.21.15.157:6825","nonce":2779251893}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14468,"qdb_cluster":[14468,24265]},"id":1}]} 2024-04-22T23:47:21.772 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 20 2024-04-22T23:47:21.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.782+0000 7f573739d640 1 -- 172.21.15.107:0/3034192588 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f571c06eb70 msgr2=0x7f571c071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:21.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.782+0000 7f573739d640 1 --2- 172.21.15.107:0/3034192588 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f571c06eb70 0x7f571c071020 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5730064ea0 tx=0x7f5728009040 comp rx=0 tx=0).stop 2024-04-22T23:47:21.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.782+0000 7f573739d640 1 -- 172.21.15.107:0/3034192588 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57301022a0 msgr2=0x7f5730113290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:21.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.782+0000 7f573739d640 1 --2- 172.21.15.107:0/3034192588 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57301022a0 0x7f5730113290 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5720004930 tx=0x7f5720009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:21.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.782+0000 7f573739d640 1 -- 172.21.15.107:0/3034192588 shutdown_connections 2024-04-22T23:47:21.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.782+0000 7f573739d640 1 --2- 172.21.15.107:0/3034192588 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f571c06eb70 0x7f571c071020 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:21.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.782+0000 7f573739d640 1 --2- 172.21.15.107:0/3034192588 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57301022a0 0x7f5730113290 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:21.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.782+0000 7f573739d640 1 --2- 172.21.15.107:0/3034192588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57300fa8a0 0x7f5730112d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:21.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.782+0000 7f573739d640 1 -- 172.21.15.107:0/3034192588 >> 172.21.15.107:0/3034192588 conn(0x7f5730071760 msgr2=0x7f57300f9a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:21.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.782+0000 7f573739d640 1 -- 172.21.15.107:0/3034192588 shutdown_connections 2024-04-22T23:47:21.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.782+0000 7f573739d640 1 -- 172.21.15.107:0/3034192588 wait complete. 2024-04-22T23:47:21.785 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 21 2024-04-22T23:47:21.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.890+0000 7f961315d640 1 -- 172.21.15.107:0/1168068559 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f960c102250 msgr2=0x7f960c102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:21.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.890+0000 7f961315d640 1 --2- 172.21.15.107:0/1168068559 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f960c102250 0x7f960c102620 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f96000048d0 tx=0x7f9600009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:21.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.890+0000 7f961315d640 1 -- 172.21.15.107:0/1168068559 shutdown_connections 2024-04-22T23:47:21.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.890+0000 7f961315d640 1 --2- 172.21.15.107:0/1168068559 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f960c0643b0 0x7f960c0647e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:21.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.890+0000 7f961315d640 1 --2- 172.21.15.107:0/1168068559 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f960c102250 0x7f960c102620 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:21.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.890+0000 7f961315d640 1 -- 172.21.15.107:0/1168068559 >> 172.21.15.107:0/1168068559 conn(0x7f960c0710c0 msgr2=0x7f960c0714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:21.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.890+0000 7f961315d640 1 -- 172.21.15.107:0/1168068559 shutdown_connections 2024-04-22T23:47:21.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.890+0000 7f961315d640 1 -- 172.21.15.107:0/1168068559 wait complete. 2024-04-22T23:47:21.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.891+0000 7f961315d640 1 Processor -- start 2024-04-22T23:47:21.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.891+0000 7f961315d640 1 -- start start 2024-04-22T23:47:21.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.891+0000 7f961315d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f960c0643b0 0x7f960c112d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:21.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.891+0000 7f961315d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f960c102250 0x7f960c113270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:21.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.891+0000 7f961315d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f960c113900 con 0x7f960c102250 2024-04-22T23:47:21.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.891+0000 7f961315d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f960c10cec0 con 0x7f960c0643b0 2024-04-22T23:47:21.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.891+0000 7f960bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f960c102250 0x7f960c113270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:21.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.891+0000 7f960bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f960c102250 0x7f960c113270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38504/0 (socket says 172.21.15.107:38504) 2024-04-22T23:47:21.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.891+0000 7f960bfff640 1 -- 172.21.15.107:0/2867859718 learned_addr learned my addr 172.21.15.107:0/2867859718 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:47:21.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.892+0000 7f9610ef9640 1 --2- 172.21.15.107:0/2867859718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f960c0643b0 0x7f960c112d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:21.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.892+0000 7f960bfff640 1 -- 172.21.15.107:0/2867859718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f960c0643b0 msgr2=0x7f960c112d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:21.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.892+0000 7f960bfff640 1 --2- 172.21.15.107:0/2867859718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f960c0643b0 0x7f960c112d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:21.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.892+0000 7f960bfff640 1 -- 172.21.15.107:0/2867859718 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9600005040 con 0x7f960c102250 2024-04-22T23:47:21.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.892+0000 7f960bfff640 1 --2- 172.21.15.107:0/2867859718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f960c102250 0x7f960c113270 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f95f800b790 tx=0x7f95f80091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:21.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.892+0000 7f96097fa640 1 -- 172.21.15.107:0/2867859718 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95f800c700 con 0x7f960c102250 2024-04-22T23:47:21.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.892+0000 7f961315d640 1 -- 172.21.15.107:0/2867859718 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f960c10d1a0 con 0x7f960c102250 2024-04-22T23:47:21.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.892+0000 7f96097fa640 1 -- 172.21.15.107:0/2867859718 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f95f800cd30 con 0x7f960c102250 2024-04-22T23:47:21.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.892+0000 7f96097fa640 1 -- 172.21.15.107:0/2867859718 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95f8013370 con 0x7f960c102250 2024-04-22T23:47:21.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.892+0000 7f961315d640 1 -- 172.21.15.107:0/2867859718 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f960c10d6c0 con 0x7f960c102250 2024-04-22T23:47:21.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.893+0000 7f961315d640 1 -- 172.21.15.107:0/2867859718 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f960c0fd950 con 0x7f960c102250 2024-04-22T23:47:21.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.893+0000 7f96097fa640 1 -- 172.21.15.107:0/2867859718 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f95f8012070 con 0x7f960c102250 2024-04-22T23:47:21.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.894+0000 7f96097fa640 1 --2- 172.21.15.107:0/2867859718 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f95fc06eb70 0x7f95fc071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:21.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.894+0000 7f96097fa640 1 -- 172.21.15.107:0/2867859718 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f95f808e3c0 con 0x7f960c102250 2024-04-22T23:47:21.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.894+0000 7f9610ef9640 1 --2- 172.21.15.107:0/2867859718 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f95fc06eb70 0x7f95fc071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:21.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.894+0000 7f9610ef9640 1 --2- 172.21.15.107:0/2867859718 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f95fc06eb70 0x7f95fc071020 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9600003080 tx=0x7f9600009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:21.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:21.898+0000 7f96097fa640 1 -- 172.21.15.107:0/2867859718 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f95f80570c0 con 0x7f960c102250 2024-04-22T23:47:22.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.112+0000 7f961315d640 1 -- 172.21.15.107:0/2867859718 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) v1 -- 0x7f960c10d330 con 0x7f960c102250 2024-04-22T23:47:22.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.113+0000 7f96097fa640 1 -- 172.21.15.107:0/2867859718 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v43) v1 ==== 107+0+4899 (secure 0 0 0) 0x7f960c10d330 con 0x7f960c102250 2024-04-22T23:47:22.105 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:47:22.105 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14484,"name":"cephfs.smithi107.dfvvkw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6827/3548717247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3548717247},{"type":"v1","addr":"172.21.15.107:6827","nonce":3548717247}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":24277,"name":"cephfs.smithi157.ufywse","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.157:6827/4072444199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":4072444199},{"type":"v1","addr":"172.21.15.157:6827","nonce":4072444199}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T23:32:20.146130+0000","modified":"2024-04-22T23:39:06.006385+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14468,"mds_1":24265},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14468":{"gid":14468,"name":"cephfs.smithi107.ktuxjl","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.107:6829/3335313253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3335313253},{"type":"v1","addr":"172.21.15.107:6829","nonce":3335313253}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24265":{"gid":24265,"name":"cephfs.smithi157.rcaris","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.157:6825/2779251893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2779251893},{"type":"v1","addr":"172.21.15.157:6825","nonce":2779251893}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14468,"qdb_cluster":[24265,14468]},"id":1}]} 2024-04-22T23:47:22.105 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 21 2024-04-22T23:47:22.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.115+0000 7f961315d640 1 -- 172.21.15.107:0/2867859718 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f95fc06eb70 msgr2=0x7f95fc071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:22.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.115+0000 7f961315d640 1 --2- 172.21.15.107:0/2867859718 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f95fc06eb70 0x7f95fc071020 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9600003080 tx=0x7f9600009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:22.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.115+0000 7f961315d640 1 -- 172.21.15.107:0/2867859718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f960c102250 msgr2=0x7f960c113270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:22.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.115+0000 7f961315d640 1 --2- 172.21.15.107:0/2867859718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f960c102250 0x7f960c113270 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f95f800b790 tx=0x7f95f80091d0 comp rx=0 tx=0).stop 2024-04-22T23:47:22.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.115+0000 7f961315d640 1 -- 172.21.15.107:0/2867859718 shutdown_connections 2024-04-22T23:47:22.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.115+0000 7f961315d640 1 --2- 172.21.15.107:0/2867859718 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f95fc06eb70 0x7f95fc071020 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:22.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.115+0000 7f961315d640 1 --2- 172.21.15.107:0/2867859718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f960c102250 0x7f960c113270 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:22.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.115+0000 7f961315d640 1 --2- 172.21.15.107:0/2867859718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f960c0643b0 0x7f960c112d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:22.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.115+0000 7f961315d640 1 -- 172.21.15.107:0/2867859718 >> 172.21.15.107:0/2867859718 conn(0x7f960c0710c0 msgr2=0x7f960c0f9510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:22.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.115+0000 7f961315d640 1 -- 172.21.15.107:0/2867859718 shutdown_connections 2024-04-22T23:47:22.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.115+0000 7f961315d640 1 -- 172.21.15.107:0/2867859718 wait complete. 2024-04-22T23:47:22.117 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:21 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/402230686' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-04-22T23:47:22.117 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:21 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/3976655735' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-04-22T23:47:22.117 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:21 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/3034192588' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-04-22T23:47:22.117 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:21 smithi107 ceph-mon[90179]: pgmap v320: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:47:22.121 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 22 2024-04-22T23:47:22.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:21 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/402230686' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-04-22T23:47:22.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:21 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/3976655735' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-04-22T23:47:22.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:21 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/3034192588' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-04-22T23:47:22.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:21 smithi157 ceph-mon[79403]: pgmap v320: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:47:22.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.225+0000 7f393802e640 1 -- 172.21.15.107:0/1054103573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3930102400 msgr2=0x7f39301027d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:22.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.225+0000 7f393802e640 1 --2- 172.21.15.107:0/1054103573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3930102400 0x7f39301027d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f392c0048d0 tx=0x7f392c009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:22.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.225+0000 7f393802e640 1 -- 172.21.15.107:0/1054103573 shutdown_connections 2024-04-22T23:47:22.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.225+0000 7f393802e640 1 --2- 172.21.15.107:0/1054103573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3930102da0 0x7f39300fc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:22.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.225+0000 7f393802e640 1 --2- 172.21.15.107:0/1054103573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3930102400 0x7f39301027d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:22.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.225+0000 7f393802e640 1 -- 172.21.15.107:0/1054103573 >> 172.21.15.107:0/1054103573 conn(0x7f39300f7fb0 msgr2=0x7f39300fa3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:22.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.225+0000 7f393802e640 1 -- 172.21.15.107:0/1054103573 shutdown_connections 2024-04-22T23:47:22.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.225+0000 7f393802e640 1 -- 172.21.15.107:0/1054103573 wait complete. 2024-04-22T23:47:22.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.225+0000 7f393802e640 1 Processor -- start 2024-04-22T23:47:22.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.225+0000 7f393802e640 1 -- start start 2024-04-22T23:47:22.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.226+0000 7f393802e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3930102400 0x7f3930112f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:22.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.226+0000 7f393802e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3930102da0 0x7f3930113490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:22.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.226+0000 7f393802e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3930113a90 con 0x7f3930102da0 2024-04-22T23:47:22.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.226+0000 7f393802e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f393010d0e0 con 0x7f3930102400 2024-04-22T23:47:22.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.226+0000 7f39355c9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3930102da0 0x7f3930113490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:22.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.226+0000 7f39355c9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3930102da0 0x7f3930113490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38532/0 (socket says 172.21.15.107:38532) 2024-04-22T23:47:22.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.226+0000 7f39355c9640 1 -- 172.21.15.107:0/2008337748 learned_addr learned my addr 172.21.15.107:0/2008337748 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:47:22.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.226+0000 7f3935dca640 1 --2- 172.21.15.107:0/2008337748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3930102400 0x7f3930112f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:22.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.226+0000 7f39355c9640 1 -- 172.21.15.107:0/2008337748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3930102400 msgr2=0x7f3930112f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:22.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.226+0000 7f39355c9640 1 --2- 172.21.15.107:0/2008337748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3930102400 0x7f3930112f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:22.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.226+0000 7f39355c9640 1 -- 172.21.15.107:0/2008337748 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f392c005040 con 0x7f3930102da0 2024-04-22T23:47:22.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.226+0000 7f39355c9640 1 --2- 172.21.15.107:0/2008337748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3930102da0 0x7f3930113490 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f392000c790 tx=0x7f39200043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:22.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.226+0000 7f39267fc640 1 -- 172.21.15.107:0/2008337748 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f392000a820 con 0x7f3930102da0 2024-04-22T23:47:22.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.226+0000 7f39267fc640 1 -- 172.21.15.107:0/2008337748 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f392000ae50 con 0x7f3930102da0 2024-04-22T23:47:22.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.226+0000 7f393802e640 1 -- 172.21.15.107:0/2008337748 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f393010d3c0 con 0x7f3930102da0 2024-04-22T23:47:22.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.226+0000 7f39267fc640 1 -- 172.21.15.107:0/2008337748 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39200115e0 con 0x7f3930102da0 2024-04-22T23:47:22.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.226+0000 7f393802e640 1 -- 172.21.15.107:0/2008337748 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f393010d8e0 con 0x7f3930102da0 2024-04-22T23:47:22.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.227+0000 7f393802e640 1 -- 172.21.15.107:0/2008337748 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39300fdb70 con 0x7f3930102da0 2024-04-22T23:47:22.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.228+0000 7f39267fc640 1 -- 172.21.15.107:0/2008337748 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f3920012400 con 0x7f3930102da0 2024-04-22T23:47:22.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.228+0000 7f39267fc640 1 --2- 172.21.15.107:0/2008337748 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f391c06eb70 0x7f391c071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:22.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.228+0000 7f3935dca640 1 --2- 172.21.15.107:0/2008337748 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f391c06eb70 0x7f391c071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:22.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.228+0000 7f39267fc640 1 -- 172.21.15.107:0/2008337748 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f392008eb20 con 0x7f3930102da0 2024-04-22T23:47:22.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.229+0000 7f3935dca640 1 --2- 172.21.15.107:0/2008337748 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f391c06eb70 0x7f391c071020 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f392c003080 tx=0x7f392c009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:22.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.233+0000 7f39267fc640 1 -- 172.21.15.107:0/2008337748 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f39200578a0 con 0x7f3930102da0 2024-04-22T23:47:22.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.448+0000 7f393802e640 1 -- 172.21.15.107:0/2008337748 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7f3930109590 con 0x7f3930102da0 2024-04-22T23:47:22.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.449+0000 7f39267fc640 1 -- 172.21.15.107:0/2008337748 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v43) v1 ==== 107+0+4895 (secure 0 0 0) 0x7f3920057030 con 0x7f3930102da0 2024-04-22T23:47:22.440 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:47:22.441 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14484,"name":"cephfs.smithi107.dfvvkw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6827/3548717247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3548717247},{"type":"v1","addr":"172.21.15.107:6827","nonce":3548717247}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":24277,"name":"cephfs.smithi157.ufywse","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.157:6827/4072444199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":4072444199},{"type":"v1","addr":"172.21.15.157:6827","nonce":4072444199}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T23:32:20.146130+0000","modified":"2024-04-22T23:42:22.327522+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0,1],"up":{"mds_0":14468,"mds_1":24265},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14468":{"gid":14468,"name":"cephfs.smithi107.ktuxjl","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.107:6829/3335313253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3335313253},{"type":"v1","addr":"172.21.15.107:6829","nonce":3335313253}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24265":{"gid":24265,"name":"cephfs.smithi157.rcaris","rank":1,"incarnation":7,"state":"up:stopping","state_seq":4,"addr":"172.21.15.157:6825/2779251893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2779251893},{"type":"v1","addr":"172.21.15.157:6825","nonce":2779251893}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14468,"qdb_cluster":[14468]},"id":1}]} 2024-04-22T23:47:22.441 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 22 2024-04-22T23:47:22.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.451+0000 7f393802e640 1 -- 172.21.15.107:0/2008337748 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f391c06eb70 msgr2=0x7f391c071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:22.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.451+0000 7f393802e640 1 --2- 172.21.15.107:0/2008337748 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f391c06eb70 0x7f391c071020 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f392c003080 tx=0x7f392c009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:22.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.451+0000 7f393802e640 1 -- 172.21.15.107:0/2008337748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3930102da0 msgr2=0x7f3930113490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:22.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.451+0000 7f393802e640 1 --2- 172.21.15.107:0/2008337748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3930102da0 0x7f3930113490 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f392000c790 tx=0x7f39200043c0 comp rx=0 tx=0).stop 2024-04-22T23:47:22.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.451+0000 7f393802e640 1 -- 172.21.15.107:0/2008337748 shutdown_connections 2024-04-22T23:47:22.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.451+0000 7f393802e640 1 --2- 172.21.15.107:0/2008337748 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f391c06eb70 0x7f391c071020 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:22.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.451+0000 7f393802e640 1 --2- 172.21.15.107:0/2008337748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3930102da0 0x7f3930113490 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:22.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.451+0000 7f393802e640 1 --2- 172.21.15.107:0/2008337748 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3930102400 0x7f3930112f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:22.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.451+0000 7f393802e640 1 -- 172.21.15.107:0/2008337748 >> 172.21.15.107:0/2008337748 conn(0x7f39300f7fb0 msgr2=0x7f39300f8f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:22.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.451+0000 7f393802e640 1 -- 172.21.15.107:0/2008337748 shutdown_connections 2024-04-22T23:47:22.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.451+0000 7f393802e640 1 -- 172.21.15.107:0/2008337748 wait complete. 2024-04-22T23:47:22.454 DEBUG:tasks.fs:max_mds reduced in epoch 22 2024-04-22T23:47:22.454 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 23 2024-04-22T23:47:22.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.558+0000 7effffa2e640 1 -- 172.21.15.107:0/2267585318 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efff8102260 msgr2=0x7efff8102630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:22.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.558+0000 7effffa2e640 1 --2- 172.21.15.107:0/2267585318 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efff8102260 0x7efff8102630 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7efff40048d0 tx=0x7efff4009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:22.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.558+0000 7effffa2e640 1 -- 172.21.15.107:0/2267585318 shutdown_connections 2024-04-22T23:47:22.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.558+0000 7effffa2e640 1 --2- 172.21.15.107:0/2267585318 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efff80fa8d0 0x7efff80fad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:22.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.558+0000 7effffa2e640 1 --2- 172.21.15.107:0/2267585318 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efff8102260 0x7efff8102630 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:22.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.558+0000 7effffa2e640 1 -- 172.21.15.107:0/2267585318 >> 172.21.15.107:0/2267585318 conn(0x7efff80708f0 msgr2=0x7efff8070cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:22.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.558+0000 7effffa2e640 1 -- 172.21.15.107:0/2267585318 shutdown_connections 2024-04-22T23:47:22.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.558+0000 7effffa2e640 1 -- 172.21.15.107:0/2267585318 wait complete. 2024-04-22T23:47:22.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.558+0000 7effffa2e640 1 Processor -- start 2024-04-22T23:47:22.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.558+0000 7effffa2e640 1 -- start start 2024-04-22T23:47:22.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.559+0000 7effffa2e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efff80fa8d0 0x7efff8112d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:22.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.559+0000 7effffa2e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efff8102260 0x7efff8113270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:22.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.559+0000 7effffa2e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff8113870 con 0x7efff80fa8d0 2024-04-22T23:47:22.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.559+0000 7effffa2e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff810cec0 con 0x7efff8102260 2024-04-22T23:47:22.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.559+0000 7efffd7ca640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efff80fa8d0 0x7efff8112d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:22.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.559+0000 7efffd7ca640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efff80fa8d0 0x7efff8112d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38554/0 (socket says 172.21.15.107:38554) 2024-04-22T23:47:22.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.559+0000 7efffd7ca640 1 -- 172.21.15.107:0/4013547148 learned_addr learned my addr 172.21.15.107:0/4013547148 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:47:22.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.559+0000 7efffd7ca640 1 -- 172.21.15.107:0/4013547148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efff8102260 msgr2=0x7efff8113270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:22.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.559+0000 7efffd7ca640 1 --2- 172.21.15.107:0/4013547148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efff8102260 0x7efff8113270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:22.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.559+0000 7efffd7ca640 1 -- 172.21.15.107:0/4013547148 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efff4005040 con 0x7efff80fa8d0 2024-04-22T23:47:22.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.559+0000 7efffd7ca640 1 --2- 172.21.15.107:0/4013547148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efff80fa8d0 0x7efff8112d30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7efff4004930 tx=0x7efff4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:22.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.559+0000 7effedffb640 1 -- 172.21.15.107:0/4013547148 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efff40094c0 con 0x7efff80fa8d0 2024-04-22T23:47:22.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.559+0000 7effffa2e640 1 -- 172.21.15.107:0/4013547148 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efff810d140 con 0x7efff80fa8d0 2024-04-22T23:47:22.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.559+0000 7effedffb640 1 -- 172.21.15.107:0/4013547148 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efff4032e50 con 0x7efff80fa8d0 2024-04-22T23:47:22.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.559+0000 7effedffb640 1 -- 172.21.15.107:0/4013547148 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efff40433b0 con 0x7efff80fa8d0 2024-04-22T23:47:22.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.560+0000 7effffa2e640 1 -- 172.21.15.107:0/4013547148 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efff810d600 con 0x7efff80fa8d0 2024-04-22T23:47:22.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.560+0000 7effffa2e640 1 -- 172.21.15.107:0/4013547148 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efff80652e0 con 0x7efff80fa8d0 2024-04-22T23:47:22.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.561+0000 7effedffb640 1 -- 172.21.15.107:0/4013547148 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7efff402f070 con 0x7efff80fa8d0 2024-04-22T23:47:22.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.561+0000 7effedffb640 1 --2- 172.21.15.107:0/4013547148 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7effe406eb70 0x7effe4071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:22.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.562+0000 7effedffb640 1 -- 172.21.15.107:0/4013547148 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7efff40b0900 con 0x7efff80fa8d0 2024-04-22T23:47:22.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.562+0000 7efffcfc9640 1 --2- 172.21.15.107:0/4013547148 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7effe406eb70 0x7effe4071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:22.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.562+0000 7efffcfc9640 1 --2- 172.21.15.107:0/4013547148 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7effe406eb70 0x7effe4071020 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efff810e5f0 tx=0x7effe8007cb0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:22.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.566+0000 7effedffb640 1 -- 172.21.15.107:0/4013547148 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7efff4079600 con 0x7efff80fa8d0 2024-04-22T23:47:22.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.780+0000 7effffa2e640 1 -- 172.21.15.107:0/4013547148 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7efff8109400 con 0x7efff80fa8d0 2024-04-22T23:47:22.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.781+0000 7effedffb640 1 -- 172.21.15.107:0/4013547148 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v43) v1 ==== 107+0+4896 (secure 0 0 0) 0x7efff407ccc0 con 0x7efff80fa8d0 2024-04-22T23:47:22.773 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:47:22.773 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14484,"name":"cephfs.smithi107.dfvvkw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6827/3548717247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3548717247},{"type":"v1","addr":"172.21.15.107:6827","nonce":3548717247}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":24277,"name":"cephfs.smithi157.ufywse","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.157:6827/4072444199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":4072444199},{"type":"v1","addr":"172.21.15.157:6827","nonce":4072444199}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T23:32:20.146130+0000","modified":"2024-04-22T23:42:31.924457+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0,1],"up":{"mds_0":14468,"mds_1":24265},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14468":{"gid":14468,"name":"cephfs.smithi107.ktuxjl","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.107:6829/3335313253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3335313253},{"type":"v1","addr":"172.21.15.107:6829","nonce":3335313253}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24265":{"gid":24265,"name":"cephfs.smithi157.rcaris","rank":1,"incarnation":7,"state":"up:stopping","state_seq":4,"addr":"172.21.15.157:6825/2779251893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":2779251893},{"type":"v1","addr":"172.21.15.157:6825","nonce":2779251893}]},"join_fscid":1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14468,"qdb_cluster":[14468]},"id":1}]} 2024-04-22T23:47:22.773 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 23 2024-04-22T23:47:22.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.783+0000 7effffa2e640 1 -- 172.21.15.107:0/4013547148 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7effe406eb70 msgr2=0x7effe4071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:22.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.783+0000 7effffa2e640 1 --2- 172.21.15.107:0/4013547148 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7effe406eb70 0x7effe4071020 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efff810e5f0 tx=0x7effe8007cb0 comp rx=0 tx=0).stop 2024-04-22T23:47:22.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.783+0000 7effffa2e640 1 -- 172.21.15.107:0/4013547148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efff80fa8d0 msgr2=0x7efff8112d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:22.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.783+0000 7effffa2e640 1 --2- 172.21.15.107:0/4013547148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efff80fa8d0 0x7efff8112d30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7efff4004930 tx=0x7efff4009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:22.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.783+0000 7effffa2e640 1 -- 172.21.15.107:0/4013547148 shutdown_connections 2024-04-22T23:47:22.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.783+0000 7effffa2e640 1 --2- 172.21.15.107:0/4013547148 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7effe406eb70 0x7effe4071020 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:22.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.783+0000 7effffa2e640 1 --2- 172.21.15.107:0/4013547148 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efff8102260 0x7efff8113270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:22.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.783+0000 7effffa2e640 1 --2- 172.21.15.107:0/4013547148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efff80fa8d0 0x7efff8112d30 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:22.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.783+0000 7effffa2e640 1 -- 172.21.15.107:0/4013547148 >> 172.21.15.107:0/4013547148 conn(0x7efff80708f0 msgr2=0x7efff80f9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:22.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.783+0000 7effffa2e640 1 -- 172.21.15.107:0/4013547148 shutdown_connections 2024-04-22T23:47:22.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.783+0000 7effffa2e640 1 -- 172.21.15.107:0/4013547148 wait complete. 2024-04-22T23:47:22.786 DEBUG:tasks.fs:max_mds reduced in epoch 23 2024-04-22T23:47:22.786 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 24 2024-04-22T23:47:22.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.890+0000 7ffa3359d640 1 -- 172.21.15.107:0/2901490533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa2c101150 msgr2=0x7ffa2c101520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:22.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.890+0000 7ffa3359d640 1 --2- 172.21.15.107:0/2901490533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa2c101150 0x7ffa2c101520 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ffa240048d0 tx=0x7ffa24009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:22.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.890+0000 7ffa3359d640 1 -- 172.21.15.107:0/2901490533 shutdown_connections 2024-04-22T23:47:22.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.890+0000 7ffa3359d640 1 --2- 172.21.15.107:0/2901490533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa2c0fb120 0x7ffa2c0fb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:22.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.890+0000 7ffa3359d640 1 --2- 172.21.15.107:0/2901490533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa2c101150 0x7ffa2c101520 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:22.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.890+0000 7ffa3359d640 1 -- 172.21.15.107:0/2901490533 >> 172.21.15.107:0/2901490533 conn(0x7ffa2c0710c0 msgr2=0x7ffa2c0714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:22.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.891+0000 7ffa3359d640 1 -- 172.21.15.107:0/2901490533 shutdown_connections 2024-04-22T23:47:22.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.891+0000 7ffa3359d640 1 -- 172.21.15.107:0/2901490533 wait complete. 2024-04-22T23:47:22.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.891+0000 7ffa3359d640 1 Processor -- start 2024-04-22T23:47:22.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.891+0000 7ffa3359d640 1 -- start start 2024-04-22T23:47:22.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.891+0000 7ffa3359d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa2c0fb120 0x7ffa2c112d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:22.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.891+0000 7ffa3359d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa2c101150 0x7ffa2c113250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:22.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.891+0000 7ffa3359d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2c1138e0 con 0x7ffa2c101150 2024-04-22T23:47:22.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.891+0000 7ffa3359d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2c10cee0 con 0x7ffa2c0fb120 2024-04-22T23:47:22.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.891+0000 7ffa30b38640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa2c101150 0x7ffa2c113250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:22.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.891+0000 7ffa30b38640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa2c101150 0x7ffa2c113250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38564/0 (socket says 172.21.15.107:38564) 2024-04-22T23:47:22.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.891+0000 7ffa30b38640 1 -- 172.21.15.107:0/1024887574 learned_addr learned my addr 172.21.15.107:0/1024887574 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:47:22.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.892+0000 7ffa31339640 1 --2- 172.21.15.107:0/1024887574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa2c0fb120 0x7ffa2c112d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:22.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.892+0000 7ffa30b38640 1 -- 172.21.15.107:0/1024887574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa2c0fb120 msgr2=0x7ffa2c112d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:22.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.892+0000 7ffa30b38640 1 --2- 172.21.15.107:0/1024887574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa2c0fb120 0x7ffa2c112d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:22.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.892+0000 7ffa30b38640 1 -- 172.21.15.107:0/1024887574 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa24005040 con 0x7ffa2c101150 2024-04-22T23:47:22.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.892+0000 7ffa30b38640 1 --2- 172.21.15.107:0/1024887574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa2c101150 0x7ffa2c113250 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ffa2000bbb0 tx=0x7ffa20009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:22.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.892+0000 7ffa29ffb640 1 -- 172.21.15.107:0/1024887574 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa2000c9c0 con 0x7ffa2c101150 2024-04-22T23:47:22.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.892+0000 7ffa29ffb640 1 -- 172.21.15.107:0/1024887574 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffa200092e0 con 0x7ffa2c101150 2024-04-22T23:47:22.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.892+0000 7ffa3359d640 1 -- 172.21.15.107:0/1024887574 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa2c10d1c0 con 0x7ffa2c101150 2024-04-22T23:47:22.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.892+0000 7ffa29ffb640 1 -- 172.21.15.107:0/1024887574 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa20007640 con 0x7ffa2c101150 2024-04-22T23:47:22.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.892+0000 7ffa3359d640 1 -- 172.21.15.107:0/1024887574 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa2c10d6e0 con 0x7ffa2c101150 2024-04-22T23:47:22.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.893+0000 7ffa3359d640 1 -- 172.21.15.107:0/1024887574 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa2c0fc810 con 0x7ffa2c101150 2024-04-22T23:47:22.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.894+0000 7ffa29ffb640 1 -- 172.21.15.107:0/1024887574 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7ffa20012070 con 0x7ffa2c101150 2024-04-22T23:47:22.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.894+0000 7ffa29ffb640 1 --2- 172.21.15.107:0/1024887574 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7ffa1406eb70 0x7ffa14071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:22.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.894+0000 7ffa29ffb640 1 -- 172.21.15.107:0/1024887574 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7ffa2008e9c0 con 0x7ffa2c101150 2024-04-22T23:47:22.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.894+0000 7ffa31339640 1 --2- 172.21.15.107:0/1024887574 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7ffa1406eb70 0x7ffa14071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:22.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.894+0000 7ffa31339640 1 --2- 172.21.15.107:0/1024887574 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7ffa1406eb70 0x7ffa14071020 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ffa24007ac0 tx=0x7ffa24009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:22.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:22.899+0000 7ffa29ffb640 1 -- 172.21.15.107:0/1024887574 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7ffa200576c0 con 0x7ffa2c101150 2024-04-22T23:47:23.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.115+0000 7ffa3359d640 1 -- 172.21.15.107:0/1024887574 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7ffa2c04bcc0 con 0x7ffa2c101150 2024-04-22T23:47:23.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.115+0000 7ffa29ffb640 1 -- 172.21.15.107:0/1024887574 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v43) v1 ==== 107+0+4097 (secure 0 0 0) 0x7ffa2005ad80 con 0x7ffa2c101150 2024-04-22T23:47:23.107 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:47:23.107 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14484,"name":"cephfs.smithi107.dfvvkw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6827/3548717247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3548717247},{"type":"v1","addr":"172.21.15.107:6827","nonce":3548717247}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":24277,"name":"cephfs.smithi157.ufywse","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.157:6827/4072444199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":4072444199},{"type":"v1","addr":"172.21.15.157:6827","nonce":4072444199}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T23:32:20.146130+0000","modified":"2024-04-22T23:42:56.925708+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14468},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_14468":{"gid":14468,"name":"cephfs.smithi107.ktuxjl","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.107:6829/3335313253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3335313253},{"type":"v1","addr":"172.21.15.107:6829","nonce":3335313253}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14468,"qdb_cluster":[14468]},"id":1}]} 2024-04-22T23:47:23.107 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 24 2024-04-22T23:47:23.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.117+0000 7ffa3359d640 1 -- 172.21.15.107:0/1024887574 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7ffa1406eb70 msgr2=0x7ffa14071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:23.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.117+0000 7ffa3359d640 1 --2- 172.21.15.107:0/1024887574 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7ffa1406eb70 0x7ffa14071020 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ffa24007ac0 tx=0x7ffa24009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:23.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.117+0000 7ffa3359d640 1 -- 172.21.15.107:0/1024887574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa2c101150 msgr2=0x7ffa2c113250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:23.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.117+0000 7ffa3359d640 1 --2- 172.21.15.107:0/1024887574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa2c101150 0x7ffa2c113250 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ffa2000bbb0 tx=0x7ffa20009840 comp rx=0 tx=0).stop 2024-04-22T23:47:23.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.117+0000 7ffa3359d640 1 -- 172.21.15.107:0/1024887574 shutdown_connections 2024-04-22T23:47:23.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.117+0000 7ffa3359d640 1 --2- 172.21.15.107:0/1024887574 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7ffa1406eb70 0x7ffa14071020 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:23.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.117+0000 7ffa3359d640 1 --2- 172.21.15.107:0/1024887574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa2c101150 0x7ffa2c113250 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:23.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.117+0000 7ffa3359d640 1 --2- 172.21.15.107:0/1024887574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa2c0fb120 0x7ffa2c112d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:23.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.117+0000 7ffa3359d640 1 -- 172.21.15.107:0/1024887574 >> 172.21.15.107:0/1024887574 conn(0x7ffa2c0710c0 msgr2=0x7ffa2c064380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:23.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.118+0000 7ffa3359d640 1 -- 172.21.15.107:0/1024887574 shutdown_connections 2024-04-22T23:47:23.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.118+0000 7ffa3359d640 1 -- 172.21.15.107:0/1024887574 wait complete. 2024-04-22T23:47:23.119 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:22 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/2867859718' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-04-22T23:47:23.119 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:22 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/2008337748' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-04-22T23:47:23.119 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:22 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/4013547148' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-04-22T23:47:23.120 DEBUG:tasks.fs:max_mds reduced in epoch 24 2024-04-22T23:47:23.120 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 25 2024-04-22T23:47:23.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:22 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/2867859718' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-04-22T23:47:23.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:22 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/2008337748' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-04-22T23:47:23.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:22 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/4013547148' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-04-22T23:47:23.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.223+0000 7f2b7942e640 1 -- 172.21.15.107:0/1451880793 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b740fd400 msgr2=0x7f2b740fd7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:23.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.223+0000 7f2b7942e640 1 --2- 172.21.15.107:0/1451880793 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b740fd400 0x7f2b740fd7d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2b640048d0 tx=0x7f2b64009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:23.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.223+0000 7f2b7942e640 1 -- 172.21.15.107:0/1451880793 shutdown_connections 2024-04-22T23:47:23.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.223+0000 7f2b7942e640 1 --2- 172.21.15.107:0/1451880793 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b740fdd10 0x7f2b741063e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:23.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.223+0000 7f2b7942e640 1 --2- 172.21.15.107:0/1451880793 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b740fd400 0x7f2b740fd7d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:23.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.223+0000 7f2b7942e640 1 -- 172.21.15.107:0/1451880793 >> 172.21.15.107:0/1451880793 conn(0x7f2b740f6eb0 msgr2=0x7f2b740f92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:23.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.223+0000 7f2b7942e640 1 -- 172.21.15.107:0/1451880793 shutdown_connections 2024-04-22T23:47:23.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.223+0000 7f2b7942e640 1 -- 172.21.15.107:0/1451880793 wait complete. 2024-04-22T23:47:23.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.223+0000 7f2b7942e640 1 Processor -- start 2024-04-22T23:47:23.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.223+0000 7f2b7942e640 1 -- start start 2024-04-22T23:47:23.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.224+0000 7f2b7942e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b740fd400 0x7f2b74112f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:23.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.224+0000 7f2b7942e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b740fdd10 0x7f2b741134a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:23.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.224+0000 7f2b7942e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b74113aa0 con 0x7f2b740fd400 2024-04-22T23:47:23.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.224+0000 7f2b7942e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7410d0f0 con 0x7f2b740fdd10 2024-04-22T23:47:23.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.224+0000 7f2b72ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b740fd400 0x7f2b74112f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:23.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.224+0000 7f2b72ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b740fd400 0x7f2b74112f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38578/0 (socket says 172.21.15.107:38578) 2024-04-22T23:47:23.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.224+0000 7f2b72ffd640 1 -- 172.21.15.107:0/4020808156 learned_addr learned my addr 172.21.15.107:0/4020808156 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:47:23.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.224+0000 7f2b6bfff640 1 --2- 172.21.15.107:0/4020808156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b740fdd10 0x7f2b741134a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:23.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.224+0000 7f2b72ffd640 1 -- 172.21.15.107:0/4020808156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b740fdd10 msgr2=0x7f2b741134a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:23.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.224+0000 7f2b72ffd640 1 --2- 172.21.15.107:0/4020808156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b740fdd10 0x7f2b741134a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:23.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.224+0000 7f2b72ffd640 1 -- 172.21.15.107:0/4020808156 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b64005040 con 0x7f2b740fd400 2024-04-22T23:47:23.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.224+0000 7f2b72ffd640 1 --2- 172.21.15.107:0/4020808156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b740fd400 0x7f2b74112f60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2b640041d0 tx=0x7f2b64009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:23.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.224+0000 7f2b6b7fe640 1 -- 172.21.15.107:0/4020808156 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b640092e0 con 0x7f2b740fd400 2024-04-22T23:47:23.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.224+0000 7f2b6b7fe640 1 -- 172.21.15.107:0/4020808156 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2b64009440 con 0x7f2b740fd400 2024-04-22T23:47:23.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.224+0000 7f2b7942e640 1 -- 172.21.15.107:0/4020808156 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b7410d370 con 0x7f2b740fd400 2024-04-22T23:47:23.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.225+0000 7f2b6b7fe640 1 -- 172.21.15.107:0/4020808156 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b64043640 con 0x7f2b740fd400 2024-04-22T23:47:23.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.225+0000 7f2b7942e640 1 -- 172.21.15.107:0/4020808156 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b7410d890 con 0x7f2b740fd400 2024-04-22T23:47:23.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.225+0000 7f2b7942e640 1 -- 172.21.15.107:0/4020808156 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b74065e60 con 0x7f2b740fd400 2024-04-22T23:47:23.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.226+0000 7f2b6b7fe640 1 -- 172.21.15.107:0/4020808156 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f2b6403b9f0 con 0x7f2b740fd400 2024-04-22T23:47:23.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.226+0000 7f2b6b7fe640 1 --2- 172.21.15.107:0/4020808156 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f2b6006eb70 0x7f2b60071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:23.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.226+0000 7f2b6b7fe640 1 -- 172.21.15.107:0/4020808156 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f2b640b16c0 con 0x7f2b740fd400 2024-04-22T23:47:23.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.226+0000 7f2b6bfff640 1 --2- 172.21.15.107:0/4020808156 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f2b6006eb70 0x7f2b60071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:23.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.227+0000 7f2b6bfff640 1 --2- 172.21.15.107:0/4020808156 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f2b6006eb70 0x7f2b60071020 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2b7410e820 tx=0x7f2b5c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:23.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.231+0000 7f2b6b7fe640 1 -- 172.21.15.107:0/4020808156 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f2b6407a500 con 0x7f2b740fd400 2024-04-22T23:47:23.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.442+0000 7f2b7942e640 1 -- 172.21.15.107:0/4020808156 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7f2b740638d0 con 0x7f2b740fd400 2024-04-22T23:47:23.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.443+0000 7f2b6b7fe640 1 -- 172.21.15.107:0/4020808156 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v43) v1 ==== 107+0+4879 (secure 0 0 0) 0x7f2b6407dbc0 con 0x7f2b740fd400 2024-04-22T23:47:23.435 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:47:23.435 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14484,"name":"cephfs.smithi107.dfvvkw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6827/3548717247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3548717247},{"type":"v1","addr":"172.21.15.107:6827","nonce":3548717247}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":24277,"name":"cephfs.smithi157.ufywse","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.157:6827/4072444199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":4072444199},{"type":"v1","addr":"172.21.15.157:6827","nonce":4072444199}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":44279,"name":"cephfs.smithi157.rcaris","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6825/1722945322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1722945322},{"type":"v1","addr":"172.21.15.157:6825","nonce":1722945322}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T23:32:20.146130+0000","modified":"2024-04-22T23:42:56.925708+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14468},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_14468":{"gid":14468,"name":"cephfs.smithi107.ktuxjl","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.107:6829/3335313253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3335313253},{"type":"v1","addr":"172.21.15.107:6829","nonce":3335313253}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14468,"qdb_cluster":[14468]},"id":1}]} 2024-04-22T23:47:23.435 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 25 2024-04-22T23:47:23.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.445+0000 7f2b7942e640 1 -- 172.21.15.107:0/4020808156 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f2b6006eb70 msgr2=0x7f2b60071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:23.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.445+0000 7f2b7942e640 1 --2- 172.21.15.107:0/4020808156 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f2b6006eb70 0x7f2b60071020 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2b7410e820 tx=0x7f2b5c009040 comp rx=0 tx=0).stop 2024-04-22T23:47:23.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.445+0000 7f2b7942e640 1 -- 172.21.15.107:0/4020808156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b740fd400 msgr2=0x7f2b74112f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:23.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.445+0000 7f2b7942e640 1 --2- 172.21.15.107:0/4020808156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b740fd400 0x7f2b74112f60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2b640041d0 tx=0x7f2b64009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:23.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.445+0000 7f2b7942e640 1 -- 172.21.15.107:0/4020808156 shutdown_connections 2024-04-22T23:47:23.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.445+0000 7f2b7942e640 1 --2- 172.21.15.107:0/4020808156 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f2b6006eb70 0x7f2b60071020 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:23.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.445+0000 7f2b7942e640 1 --2- 172.21.15.107:0/4020808156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b740fdd10 0x7f2b741134a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:23.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.445+0000 7f2b7942e640 1 --2- 172.21.15.107:0/4020808156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b740fd400 0x7f2b74112f60 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:23.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.445+0000 7f2b7942e640 1 -- 172.21.15.107:0/4020808156 >> 172.21.15.107:0/4020808156 conn(0x7f2b740f6eb0 msgr2=0x7f2b740f83a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:23.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.445+0000 7f2b7942e640 1 -- 172.21.15.107:0/4020808156 shutdown_connections 2024-04-22T23:47:23.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.445+0000 7f2b7942e640 1 -- 172.21.15.107:0/4020808156 wait complete. 2024-04-22T23:47:23.448 DEBUG:tasks.fs:max_mds reduced in epoch 25 2024-04-22T23:47:23.448 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 26 2024-04-22T23:47:23.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.554+0000 7f80e319d640 1 -- 172.21.15.107:0/3169868742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80dc0fbae0 msgr2=0x7f80dc108840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:23.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.554+0000 7f80e319d640 1 --2- 172.21.15.107:0/3169868742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80dc0fbae0 0x7f80dc108840 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f80c8004900 tx=0x7f80c8009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:23.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.554+0000 7f80e319d640 1 -- 172.21.15.107:0/3169868742 shutdown_connections 2024-04-22T23:47:23.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.554+0000 7f80e319d640 1 --2- 172.21.15.107:0/3169868742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80dc0fbae0 0x7f80dc108840 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:23.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.554+0000 7f80e319d640 1 --2- 172.21.15.107:0/3169868742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80dc0fb1d0 0x7f80dc0fb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:23.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.554+0000 7f80e319d640 1 -- 172.21.15.107:0/3169868742 >> 172.21.15.107:0/3169868742 conn(0x7f80dc0f6ec0 msgr2=0x7f80dc0f92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:23.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.554+0000 7f80e319d640 1 -- 172.21.15.107:0/3169868742 shutdown_connections 2024-04-22T23:47:23.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.555+0000 7f80e319d640 1 -- 172.21.15.107:0/3169868742 wait complete. 2024-04-22T23:47:23.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.555+0000 7f80e319d640 1 Processor -- start 2024-04-22T23:47:23.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.555+0000 7f80e319d640 1 -- start start 2024-04-22T23:47:23.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.555+0000 7f80e319d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80dc0fb1d0 0x7f80dc115150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:23.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.555+0000 7f80e319d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80dc0fbae0 0x7f80dc115690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:23.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.555+0000 7f80e319d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80dc115c90 con 0x7f80dc0fb1d0 2024-04-22T23:47:23.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.555+0000 7f80e319d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80dc10f2e0 con 0x7f80dc0fbae0 2024-04-22T23:47:23.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.555+0000 7f80e0f39640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80dc0fb1d0 0x7f80dc115150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:23.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.555+0000 7f80e0f39640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80dc0fb1d0 0x7f80dc115150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38602/0 (socket says 172.21.15.107:38602) 2024-04-22T23:47:23.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.555+0000 7f80e0f39640 1 -- 172.21.15.107:0/198542096 learned_addr learned my addr 172.21.15.107:0/198542096 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:47:23.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.556+0000 7f80dbfff640 1 --2- 172.21.15.107:0/198542096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80dc0fbae0 0x7f80dc115690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:23.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.556+0000 7f80e0f39640 1 -- 172.21.15.107:0/198542096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80dc0fbae0 msgr2=0x7f80dc115690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:23.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.556+0000 7f80e0f39640 1 --2- 172.21.15.107:0/198542096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80dc0fbae0 0x7f80dc115690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:23.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.556+0000 7f80e0f39640 1 -- 172.21.15.107:0/198542096 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80c8005040 con 0x7f80dc0fb1d0 2024-04-22T23:47:23.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.556+0000 7f80e0f39640 1 --2- 172.21.15.107:0/198542096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80dc0fb1d0 0x7f80dc115150 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f80d000eb70 tx=0x7f80d000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:23.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.556+0000 7f80d97fa640 1 -- 172.21.15.107:0/198542096 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80d0007c30 con 0x7f80dc0fb1d0 2024-04-22T23:47:23.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.556+0000 7f80e319d640 1 -- 172.21.15.107:0/198542096 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80dc10f560 con 0x7f80dc0fb1d0 2024-04-22T23:47:23.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.556+0000 7f80d97fa640 1 -- 172.21.15.107:0/198542096 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f80d000f040 con 0x7f80dc0fb1d0 2024-04-22T23:47:23.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.556+0000 7f80d97fa640 1 -- 172.21.15.107:0/198542096 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80d00095b0 con 0x7f80dc0fb1d0 2024-04-22T23:47:23.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.556+0000 7f80e319d640 1 -- 172.21.15.107:0/198542096 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80dc10fa80 con 0x7f80dc0fb1d0 2024-04-22T23:47:23.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.558+0000 7f80d97fa640 1 -- 172.21.15.107:0/198542096 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f80d000c6a0 con 0x7f80dc0fb1d0 2024-04-22T23:47:23.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.558+0000 7f80e319d640 1 -- 172.21.15.107:0/198542096 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80dc065e60 con 0x7f80dc0fb1d0 2024-04-22T23:47:23.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.558+0000 7f80d97fa640 1 --2- 172.21.15.107:0/198542096 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f80cc06eb70 0x7f80cc071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:23.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.558+0000 7f80d97fa640 1 -- 172.21.15.107:0/198542096 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f80d008e6e0 con 0x7f80dc0fb1d0 2024-04-22T23:47:23.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.558+0000 7f80dbfff640 1 --2- 172.21.15.107:0/198542096 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f80cc06eb70 0x7f80cc071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:23.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.559+0000 7f80dbfff640 1 --2- 172.21.15.107:0/198542096 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f80cc06eb70 0x7f80cc071020 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f80c8009990 tx=0x7f80c8009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:23.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.563+0000 7f80d97fa640 1 -- 172.21.15.107:0/198542096 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f80d00573e0 con 0x7f80dc0fb1d0 2024-04-22T23:47:23.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.778+0000 7f80e319d640 1 -- 172.21.15.107:0/198542096 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7f80dc0638d0 con 0x7f80dc0fb1d0 2024-04-22T23:47:23.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.778+0000 7f80d97fa640 1 -- 172.21.15.107:0/198542096 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v43) v1 ==== 107+0+4097 (secure 0 0 0) 0x7f80d005aaa0 con 0x7f80dc0fb1d0 2024-04-22T23:47:23.770 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:47:23.770 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24277,"name":"cephfs.smithi157.ufywse","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.157:6827/4072444199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":4072444199},{"type":"v1","addr":"172.21.15.157:6827","nonce":4072444199}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":44279,"name":"cephfs.smithi157.rcaris","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6825/1722945322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1722945322},{"type":"v1","addr":"172.21.15.157:6825","nonce":1722945322}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T23:32:20.146130+0000","modified":"2024-04-22T23:42:56.925708+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14468},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_14468":{"gid":14468,"name":"cephfs.smithi107.ktuxjl","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.107:6829/3335313253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3335313253},{"type":"v1","addr":"172.21.15.107:6829","nonce":3335313253}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14468,"qdb_cluster":[14468]},"id":1}]} 2024-04-22T23:47:23.771 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 26 2024-04-22T23:47:23.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.780+0000 7f80e319d640 1 -- 172.21.15.107:0/198542096 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f80cc06eb70 msgr2=0x7f80cc071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:23.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.780+0000 7f80e319d640 1 --2- 172.21.15.107:0/198542096 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f80cc06eb70 0x7f80cc071020 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f80c8009990 tx=0x7f80c8009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:23.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.780+0000 7f80e319d640 1 -- 172.21.15.107:0/198542096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80dc0fb1d0 msgr2=0x7f80dc115150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:23.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.780+0000 7f80e319d640 1 --2- 172.21.15.107:0/198542096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80dc0fb1d0 0x7f80dc115150 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f80d000eb70 tx=0x7f80d000b040 comp rx=0 tx=0).stop 2024-04-22T23:47:23.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.780+0000 7f80e319d640 1 -- 172.21.15.107:0/198542096 shutdown_connections 2024-04-22T23:47:23.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.780+0000 7f80e319d640 1 --2- 172.21.15.107:0/198542096 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f80cc06eb70 0x7f80cc071020 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:23.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.780+0000 7f80e319d640 1 --2- 172.21.15.107:0/198542096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80dc0fbae0 0x7f80dc115690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:23.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.780+0000 7f80e319d640 1 --2- 172.21.15.107:0/198542096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80dc0fb1d0 0x7f80dc115150 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:23.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.780+0000 7f80e319d640 1 -- 172.21.15.107:0/198542096 >> 172.21.15.107:0/198542096 conn(0x7f80dc0f6ec0 msgr2=0x7f80dc0f72f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:23.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.780+0000 7f80e319d640 1 -- 172.21.15.107:0/198542096 shutdown_connections 2024-04-22T23:47:23.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.780+0000 7f80e319d640 1 -- 172.21.15.107:0/198542096 wait complete. 2024-04-22T23:47:23.783 DEBUG:tasks.fs:max_mds reduced in epoch 26 2024-04-22T23:47:23.783 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 27 2024-04-22T23:47:23.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.888+0000 7f130dcae640 1 -- 172.21.15.107:0/4260002582 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13081012e0 msgr2=0x7f13081016b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:23.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.888+0000 7f130dcae640 1 --2- 172.21.15.107:0/4260002582 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13081012e0 0x7f13081016b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f12f80048d0 tx=0x7f12f8009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:23.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.888+0000 7f130dcae640 1 -- 172.21.15.107:0/4260002582 shutdown_connections 2024-04-22T23:47:23.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.888+0000 7f130dcae640 1 --2- 172.21.15.107:0/4260002582 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1308101c80 0x7f13080fb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:23.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.888+0000 7f130dcae640 1 --2- 172.21.15.107:0/4260002582 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13081012e0 0x7f13081016b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:23.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.888+0000 7f130dcae640 1 -- 172.21.15.107:0/4260002582 >> 172.21.15.107:0/4260002582 conn(0x7f13080f6eb0 msgr2=0x7f13080f92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:23.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.888+0000 7f130dcae640 1 -- 172.21.15.107:0/4260002582 shutdown_connections 2024-04-22T23:47:23.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.888+0000 7f130dcae640 1 -- 172.21.15.107:0/4260002582 wait complete. 2024-04-22T23:47:23.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.889+0000 7f130dcae640 1 Processor -- start 2024-04-22T23:47:23.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.889+0000 7f130dcae640 1 -- start start 2024-04-22T23:47:23.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.889+0000 7f130dcae640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13081012e0 0x7f130810e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:23.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.889+0000 7f130dcae640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1308101c80 0x7f130810e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:23.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.889+0000 7f130dcae640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1308066520 con 0x7f1308101c80 2024-04-22T23:47:23.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.889+0000 7f130dcae640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130810eb70 con 0x7f13081012e0 2024-04-22T23:47:23.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.889+0000 7f1306ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1308101c80 0x7f130810e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:23.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.889+0000 7f1306ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1308101c80 0x7f130810e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38622/0 (socket says 172.21.15.107:38622) 2024-04-22T23:47:23.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.889+0000 7f1306ffd640 1 -- 172.21.15.107:0/980368365 learned_addr learned my addr 172.21.15.107:0/980368365 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:47:23.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.890+0000 7f13077fe640 1 --2- 172.21.15.107:0/980368365 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13081012e0 0x7f130810e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:23.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.890+0000 7f1306ffd640 1 -- 172.21.15.107:0/980368365 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13081012e0 msgr2=0x7f130810e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:23.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.890+0000 7f1306ffd640 1 --2- 172.21.15.107:0/980368365 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13081012e0 0x7f130810e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:23.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.890+0000 7f1306ffd640 1 -- 172.21.15.107:0/980368365 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12f8005040 con 0x7f1308101c80 2024-04-22T23:47:23.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.890+0000 7f1306ffd640 1 --2- 172.21.15.107:0/980368365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1308101c80 0x7f130810e630 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f12fc002a50 tx=0x7f12fc004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:23.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.890+0000 7f12effff640 1 -- 172.21.15.107:0/980368365 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12fc007c30 con 0x7f1308101c80 2024-04-22T23:47:23.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.890+0000 7f12effff640 1 -- 172.21.15.107:0/980368365 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f12fc00f040 con 0x7f1308101c80 2024-04-22T23:47:23.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.890+0000 7f130dcae640 1 -- 172.21.15.107:0/980368365 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f130810ee50 con 0x7f1308101c80 2024-04-22T23:47:23.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.890+0000 7f12effff640 1 -- 172.21.15.107:0/980368365 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12fc00db50 con 0x7f1308101c80 2024-04-22T23:47:23.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.890+0000 7f130dcae640 1 -- 172.21.15.107:0/980368365 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1308117410 con 0x7f1308101c80 2024-04-22T23:47:23.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.891+0000 7f130dcae640 1 -- 172.21.15.107:0/980368365 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13080fca50 con 0x7f1308101c80 2024-04-22T23:47:23.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.891+0000 7f12effff640 1 -- 172.21.15.107:0/980368365 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f12fc01c400 con 0x7f1308101c80 2024-04-22T23:47:23.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.892+0000 7f12effff640 1 --2- 172.21.15.107:0/980368365 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f12f006eb80 0x7f12f0071030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:23.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.892+0000 7f13077fe640 1 --2- 172.21.15.107:0/980368365 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f12f006eb80 0x7f12f0071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:23.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.892+0000 7f12effff640 1 -- 172.21.15.107:0/980368365 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f12fc08e240 con 0x7f1308101c80 2024-04-22T23:47:23.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.892+0000 7f13077fe640 1 --2- 172.21.15.107:0/980368365 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f12f006eb80 0x7f12f0071030 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f12f8031870 tx=0x7f12f8009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:23.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:23.897+0000 7f12effff640 1 -- 172.21.15.107:0/980368365 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f12fc056f40 con 0x7f1308101c80 2024-04-22T23:47:24.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.111+0000 7f130dcae640 1 -- 172.21.15.107:0/980368365 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7f13080638d0 con 0x7f1308101c80 2024-04-22T23:47:24.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.111+0000 7f12effff640 1 -- 172.21.15.107:0/980368365 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v43) v1 ==== 107+0+4944 (secure 0 0 0) 0x7f12fc05a600 con 0x7f1308101c80 2024-04-22T23:47:24.103 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:47:24.103 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24277,"name":"cephfs.smithi157.ufywse","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.157:6827/4072444199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":4072444199},{"type":"v1","addr":"172.21.15.157:6827","nonce":4072444199}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":34434,"name":"cephfs.smithi107.dfvvkw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6827/418145840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":418145840},{"type":"v1","addr":"172.21.15.107:6827","nonce":418145840}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44279,"name":"cephfs.smithi157.rcaris","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6825/1722945322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1722945322},{"type":"v1","addr":"172.21.15.157:6825","nonce":1722945322}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T23:32:20.146130+0000","modified":"2024-04-22T23:42:56.925708+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14468},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_14468":{"gid":14468,"name":"cephfs.smithi107.ktuxjl","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.107:6829/3335313253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3335313253},{"type":"v1","addr":"172.21.15.107:6829","nonce":3335313253}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14468,"qdb_cluster":[14468]},"id":1}]} 2024-04-22T23:47:24.103 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 27 2024-04-22T23:47:24.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.113+0000 7f130dcae640 1 -- 172.21.15.107:0/980368365 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f12f006eb80 msgr2=0x7f12f0071030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:24.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.113+0000 7f130dcae640 1 --2- 172.21.15.107:0/980368365 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f12f006eb80 0x7f12f0071030 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f12f8031870 tx=0x7f12f8009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:24.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.113+0000 7f130dcae640 1 -- 172.21.15.107:0/980368365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1308101c80 msgr2=0x7f130810e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:24.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.113+0000 7f130dcae640 1 --2- 172.21.15.107:0/980368365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1308101c80 0x7f130810e630 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f12fc002a50 tx=0x7f12fc004600 comp rx=0 tx=0).stop 2024-04-22T23:47:24.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.113+0000 7f130dcae640 1 -- 172.21.15.107:0/980368365 shutdown_connections 2024-04-22T23:47:24.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.113+0000 7f130dcae640 1 --2- 172.21.15.107:0/980368365 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f12f006eb80 0x7f12f0071030 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:24.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.113+0000 7f130dcae640 1 --2- 172.21.15.107:0/980368365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1308101c80 0x7f130810e630 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:24.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.113+0000 7f130dcae640 1 --2- 172.21.15.107:0/980368365 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13081012e0 0x7f130810e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:24.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.113+0000 7f130dcae640 1 -- 172.21.15.107:0/980368365 >> 172.21.15.107:0/980368365 conn(0x7f13080f6eb0 msgr2=0x7f13080f7e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:24.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.113+0000 7f130dcae640 1 -- 172.21.15.107:0/980368365 shutdown_connections 2024-04-22T23:47:24.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.114+0000 7f130dcae640 1 -- 172.21.15.107:0/980368365 wait complete. 2024-04-22T23:47:24.115 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:23 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/1024887574' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-04-22T23:47:24.115 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:23 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/4020808156' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-04-22T23:47:24.115 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:23 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/198542096' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-04-22T23:47:24.115 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:23 smithi107 ceph-mon[90179]: pgmap v321: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:47:24.116 DEBUG:tasks.fs:max_mds reduced in epoch 27 2024-04-22T23:47:24.116 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 28 2024-04-22T23:47:24.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:23 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/1024887574' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-04-22T23:47:24.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:23 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/4020808156' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-04-22T23:47:24.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:23 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/198542096' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-04-22T23:47:24.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:23 smithi157 ceph-mon[79403]: pgmap v321: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:47:24.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.218+0000 7f6a3351d640 1 -- 172.21.15.107:0/1767474555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a2c0fb120 msgr2=0x7f6a2c0fb570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:24.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.218+0000 7f6a3351d640 1 --2- 172.21.15.107:0/1767474555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a2c0fb120 0x7f6a2c0fb570 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c0048d0 tx=0x7f6a1c009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:24.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.219+0000 7f6a3351d640 1 -- 172.21.15.107:0/1767474555 shutdown_connections 2024-04-22T23:47:24.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.219+0000 7f6a3351d640 1 --2- 172.21.15.107:0/1767474555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a2c0fb120 0x7f6a2c0fb570 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:24.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.219+0000 7f6a3351d640 1 --2- 172.21.15.107:0/1767474555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a2c101150 0x7f6a2c101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:24.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.219+0000 7f6a3351d640 1 -- 172.21.15.107:0/1767474555 >> 172.21.15.107:0/1767474555 conn(0x7f6a2c0710c0 msgr2=0x7f6a2c0714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:24.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.219+0000 7f6a3351d640 1 -- 172.21.15.107:0/1767474555 shutdown_connections 2024-04-22T23:47:24.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.219+0000 7f6a3351d640 1 -- 172.21.15.107:0/1767474555 wait complete. 2024-04-22T23:47:24.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.219+0000 7f6a3351d640 1 Processor -- start 2024-04-22T23:47:24.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.219+0000 7f6a3351d640 1 -- start start 2024-04-22T23:47:24.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.219+0000 7f6a3351d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a2c0fb120 0x7f6a2c112d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:24.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.219+0000 7f6a3351d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a2c101150 0x7f6a2c113250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:24.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.219+0000 7f6a3351d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2c113850 con 0x7f6a2c101150 2024-04-22T23:47:24.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.219+0000 7f6a3351d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2c10cea0 con 0x7f6a2c0fb120 2024-04-22T23:47:24.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.219+0000 7f6a30ab8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a2c101150 0x7f6a2c113250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:24.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.219+0000 7f6a30ab8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a2c101150 0x7f6a2c113250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38636/0 (socket says 172.21.15.107:38636) 2024-04-22T23:47:24.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.219+0000 7f6a30ab8640 1 -- 172.21.15.107:0/3017025177 learned_addr learned my addr 172.21.15.107:0/3017025177 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:47:24.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.220+0000 7f6a30ab8640 1 -- 172.21.15.107:0/3017025177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a2c0fb120 msgr2=0x7f6a2c112d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:24.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.220+0000 7f6a30ab8640 1 --2- 172.21.15.107:0/3017025177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a2c0fb120 0x7f6a2c112d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:24.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.220+0000 7f6a30ab8640 1 -- 172.21.15.107:0/3017025177 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a1c005040 con 0x7f6a2c101150 2024-04-22T23:47:24.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.220+0000 7f6a30ab8640 1 --2- 172.21.15.107:0/3017025177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a2c101150 0x7f6a2c113250 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c0041d0 tx=0x7f6a1c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:24.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.220+0000 7f6a21ffb640 1 -- 172.21.15.107:0/3017025177 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a1c030870 con 0x7f6a2c101150 2024-04-22T23:47:24.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.220+0000 7f6a21ffb640 1 -- 172.21.15.107:0/3017025177 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6a1c0092e0 con 0x7f6a2c101150 2024-04-22T23:47:24.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.220+0000 7f6a3351d640 1 -- 172.21.15.107:0/3017025177 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a2c10d120 con 0x7f6a2c101150 2024-04-22T23:47:24.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.220+0000 7f6a21ffb640 1 -- 172.21.15.107:0/3017025177 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a1c043400 con 0x7f6a2c101150 2024-04-22T23:47:24.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.220+0000 7f6a3351d640 1 -- 172.21.15.107:0/3017025177 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a2c10d640 con 0x7f6a2c101150 2024-04-22T23:47:24.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.221+0000 7f6a3351d640 1 -- 172.21.15.107:0/3017025177 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a2c0fc810 con 0x7f6a2c101150 2024-04-22T23:47:24.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.222+0000 7f6a21ffb640 1 -- 172.21.15.107:0/3017025177 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f6a1c03b420 con 0x7f6a2c101150 2024-04-22T23:47:24.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.222+0000 7f6a21ffb640 1 --2- 172.21.15.107:0/3017025177 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f6a1806eb70 0x7f6a18071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:24.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.222+0000 7f6a21ffb640 1 -- 172.21.15.107:0/3017025177 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f6a1c0b1720 con 0x7f6a2c101150 2024-04-22T23:47:24.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.222+0000 7f6a312b9640 1 --2- 172.21.15.107:0/3017025177 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f6a1806eb70 0x7f6a18071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:24.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.222+0000 7f6a312b9640 1 --2- 172.21.15.107:0/3017025177 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f6a1806eb70 0x7f6a18071020 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6a28004c80 tx=0x7f6a28009c50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:24.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.227+0000 7f6a21ffb640 1 -- 172.21.15.107:0/3017025177 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f6a1c07a560 con 0x7f6a2c101150 2024-04-22T23:47:24.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.443+0000 7f6a3351d640 1 -- 172.21.15.107:0/3017025177 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7f6a2c04bcc0 con 0x7f6a2c101150 2024-04-22T23:47:24.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.444+0000 7f6a21ffb640 1 -- 172.21.15.107:0/3017025177 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v43) v1 ==== 107+0+4142 (secure 0 0 0) 0x7f6a1c07dc20 con 0x7f6a2c101150 2024-04-22T23:47:24.435 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:47:24.435 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24277,"name":"cephfs.smithi157.ufywse","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.157:6827/4072444199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":4072444199},{"type":"v1","addr":"172.21.15.157:6827","nonce":4072444199}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":34434,"name":"cephfs.smithi107.dfvvkw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6827/418145840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":418145840},{"type":"v1","addr":"172.21.15.107:6827","nonce":418145840}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44279,"name":"cephfs.smithi157.rcaris","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6825/1722945322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1722945322},{"type":"v1","addr":"172.21.15.157:6825","nonce":1722945322}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T23:32:20.146130+0000","modified":"2024-04-22T23:43:26.927227+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[1],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-22T23:47:24.436 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 28 2024-04-22T23:47:24.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.446+0000 7f6a3351d640 1 -- 172.21.15.107:0/3017025177 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f6a1806eb70 msgr2=0x7f6a18071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:24.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.446+0000 7f6a3351d640 1 --2- 172.21.15.107:0/3017025177 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f6a1806eb70 0x7f6a18071020 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6a28004c80 tx=0x7f6a28009c50 comp rx=0 tx=0).stop 2024-04-22T23:47:24.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.446+0000 7f6a3351d640 1 -- 172.21.15.107:0/3017025177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a2c101150 msgr2=0x7f6a2c113250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:24.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.446+0000 7f6a3351d640 1 --2- 172.21.15.107:0/3017025177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a2c101150 0x7f6a2c113250 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c0041d0 tx=0x7f6a1c009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:24.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.446+0000 7f6a3351d640 1 -- 172.21.15.107:0/3017025177 shutdown_connections 2024-04-22T23:47:24.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.446+0000 7f6a3351d640 1 --2- 172.21.15.107:0/3017025177 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f6a1806eb70 0x7f6a18071020 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:24.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.446+0000 7f6a3351d640 1 --2- 172.21.15.107:0/3017025177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a2c101150 0x7f6a2c113250 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:24.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.446+0000 7f6a3351d640 1 --2- 172.21.15.107:0/3017025177 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a2c0fb120 0x7f6a2c112d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:24.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.446+0000 7f6a3351d640 1 -- 172.21.15.107:0/3017025177 >> 172.21.15.107:0/3017025177 conn(0x7f6a2c0710c0 msgr2=0x7f6a2c0f9490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:24.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.446+0000 7f6a3351d640 1 -- 172.21.15.107:0/3017025177 shutdown_connections 2024-04-22T23:47:24.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.446+0000 7f6a3351d640 1 -- 172.21.15.107:0/3017025177 wait complete. 2024-04-22T23:47:24.448 DEBUG:tasks.fs:max_mds reduced in epoch 28 2024-04-22T23:47:24.449 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 29 2024-04-22T23:47:24.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.551+0000 7f9df8cae640 1 -- 172.21.15.107:0/924942986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9df4101a70 msgr2=0x7f9df40fb100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:24.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.551+0000 7f9df8cae640 1 --2- 172.21.15.107:0/924942986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9df4101a70 0x7f9df40fb100 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc0048d0 tx=0x7f9ddc009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:24.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.551+0000 7f9df8cae640 1 -- 172.21.15.107:0/924942986 shutdown_connections 2024-04-22T23:47:24.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.551+0000 7f9df8cae640 1 --2- 172.21.15.107:0/924942986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9df4101a70 0x7f9df40fb100 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:24.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.551+0000 7f9df8cae640 1 --2- 172.21.15.107:0/924942986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9df41010d0 0x7f9df41014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:24.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.551+0000 7f9df8cae640 1 -- 172.21.15.107:0/924942986 >> 172.21.15.107:0/924942986 conn(0x7f9df40f6cc0 msgr2=0x7f9df40f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:24.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.551+0000 7f9df8cae640 1 -- 172.21.15.107:0/924942986 shutdown_connections 2024-04-22T23:47:24.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.551+0000 7f9df8cae640 1 -- 172.21.15.107:0/924942986 wait complete. 2024-04-22T23:47:24.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.552+0000 7f9df8cae640 1 Processor -- start 2024-04-22T23:47:24.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.552+0000 7f9df8cae640 1 -- start start 2024-04-22T23:47:24.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.552+0000 7f9df8cae640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9df41010d0 0x7f9df410dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:24.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.552+0000 7f9df8cae640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9df4101a70 0x7f9df410e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:24.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.552+0000 7f9df8cae640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df4066520 con 0x7f9df41010d0 2024-04-22T23:47:24.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.552+0000 7f9df8cae640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df410e970 con 0x7f9df4101a70 2024-04-22T23:47:24.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.552+0000 7f9df259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9df41010d0 0x7f9df410dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:24.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.552+0000 7f9df259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9df41010d0 0x7f9df410dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38652/0 (socket says 172.21.15.107:38652) 2024-04-22T23:47:24.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.552+0000 7f9df259c640 1 -- 172.21.15.107:0/1367748586 learned_addr learned my addr 172.21.15.107:0/1367748586 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:47:24.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.552+0000 7f9de9d9b640 1 --2- 172.21.15.107:0/1367748586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9df4101a70 0x7f9df410e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:24.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.552+0000 7f9df259c640 1 -- 172.21.15.107:0/1367748586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9df4101a70 msgr2=0x7f9df410e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:24.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.552+0000 7f9df259c640 1 --2- 172.21.15.107:0/1367748586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9df4101a70 0x7f9df410e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:24.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.552+0000 7f9df259c640 1 -- 172.21.15.107:0/1367748586 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ddc005040 con 0x7f9df41010d0 2024-04-22T23:47:24.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.552+0000 7f9df259c640 1 --2- 172.21.15.107:0/1367748586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9df41010d0 0x7f9df410dec0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9de400c790 tx=0x7f9de40043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:24.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.553+0000 7f9deb7fe640 1 -- 172.21.15.107:0/1367748586 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9de400a820 con 0x7f9df41010d0 2024-04-22T23:47:24.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.553+0000 7f9df8cae640 1 -- 172.21.15.107:0/1367748586 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9df410ec50 con 0x7f9df41010d0 2024-04-22T23:47:24.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.553+0000 7f9deb7fe640 1 -- 172.21.15.107:0/1367748586 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9de400ae50 con 0x7f9df41010d0 2024-04-22T23:47:24.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.553+0000 7f9deb7fe640 1 -- 172.21.15.107:0/1367748586 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9de40115e0 con 0x7f9df41010d0 2024-04-22T23:47:24.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.553+0000 7f9df8cae640 1 -- 172.21.15.107:0/1367748586 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9df41171e0 con 0x7f9df41010d0 2024-04-22T23:47:24.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.554+0000 7f9df8cae640 1 -- 172.21.15.107:0/1367748586 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9df40fc840 con 0x7f9df41010d0 2024-04-22T23:47:24.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.554+0000 7f9deb7fe640 1 -- 172.21.15.107:0/1367748586 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f9de4012400 con 0x7f9df41010d0 2024-04-22T23:47:24.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.554+0000 7f9deb7fe640 1 --2- 172.21.15.107:0/1367748586 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f9de006eb70 0x7f9de0071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:24.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.555+0000 7f9de9d9b640 1 --2- 172.21.15.107:0/1367748586 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f9de006eb70 0x7f9de0071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:24.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.555+0000 7f9deb7fe640 1 -- 172.21.15.107:0/1367748586 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f9de408ea50 con 0x7f9df41010d0 2024-04-22T23:47:24.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.555+0000 7f9de9d9b640 1 --2- 172.21.15.107:0/1367748586 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f9de006eb70 0x7f9de0071020 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9df4065d70 tx=0x7f9ddc009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:24.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.559+0000 7f9deb7fe640 1 -- 172.21.15.107:0/1367748586 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f9de40577d0 con 0x7f9df41010d0 2024-04-22T23:47:24.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.772+0000 7f9df8cae640 1 -- 172.21.15.107:0/1367748586 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) v1 -- 0x7f9df404bcc0 con 0x7f9df41010d0 2024-04-22T23:47:24.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.773+0000 7f9deb7fe640 1 -- 172.21.15.107:0/1367748586 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v43) v1 ==== 107+0+4153 (secure 0 0 0) 0x7f9de405ae90 con 0x7f9df41010d0 2024-04-22T23:47:24.765 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:47:24.765 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34434,"name":"cephfs.smithi107.dfvvkw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6827/418145840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":418145840},{"type":"v1","addr":"172.21.15.107:6827","nonce":418145840}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44279,"name":"cephfs.smithi157.rcaris","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6825/1722945322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1722945322},{"type":"v1","addr":"172.21.15.157:6825","nonce":1722945322}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T23:32:20.146130+0000","modified":"2024-04-22T23:43:26.935679+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24277},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_24277":{"gid":24277,"name":"cephfs.smithi157.ufywse","rank":0,"incarnation":29,"state":"up:replay","state_seq":2,"addr":"172.21.15.157:6827/4072444199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":4072444199},{"type":"v1","addr":"172.21.15.157:6827","nonce":4072444199}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-22T23:47:24.765 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 29 2024-04-22T23:47:24.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.775+0000 7f9df8cae640 1 -- 172.21.15.107:0/1367748586 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f9de006eb70 msgr2=0x7f9de0071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:24.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.775+0000 7f9df8cae640 1 --2- 172.21.15.107:0/1367748586 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f9de006eb70 0x7f9de0071020 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9df4065d70 tx=0x7f9ddc009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:24.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.775+0000 7f9df8cae640 1 -- 172.21.15.107:0/1367748586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9df41010d0 msgr2=0x7f9df410dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:24.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.775+0000 7f9df8cae640 1 --2- 172.21.15.107:0/1367748586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9df41010d0 0x7f9df410dec0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9de400c790 tx=0x7f9de40043c0 comp rx=0 tx=0).stop 2024-04-22T23:47:24.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.775+0000 7f9df8cae640 1 -- 172.21.15.107:0/1367748586 shutdown_connections 2024-04-22T23:47:24.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.775+0000 7f9df8cae640 1 --2- 172.21.15.107:0/1367748586 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f9de006eb70 0x7f9de0071020 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:24.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.775+0000 7f9df8cae640 1 --2- 172.21.15.107:0/1367748586 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9df4101a70 0x7f9df410e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:24.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.775+0000 7f9df8cae640 1 --2- 172.21.15.107:0/1367748586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9df41010d0 0x7f9df410dec0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:24.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.775+0000 7f9df8cae640 1 -- 172.21.15.107:0/1367748586 >> 172.21.15.107:0/1367748586 conn(0x7f9df40f6cc0 msgr2=0x7f9df40f7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:24.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.775+0000 7f9df8cae640 1 -- 172.21.15.107:0/1367748586 shutdown_connections 2024-04-22T23:47:24.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.775+0000 7f9df8cae640 1 -- 172.21.15.107:0/1367748586 wait complete. 2024-04-22T23:47:24.778 DEBUG:tasks.fs:max_mds reduced in epoch 29 2024-04-22T23:47:24.779 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 30 2024-04-22T23:47:24.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.882+0000 7f7ac795d640 1 -- 172.21.15.107:0/1295101763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ac01010d0 msgr2=0x7f7ac01014a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:24.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.882+0000 7f7ac795d640 1 --2- 172.21.15.107:0/1295101763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ac01010d0 0x7f7ac01014a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7abc0048d0 tx=0x7f7abc009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:24.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.882+0000 7f7ac795d640 1 -- 172.21.15.107:0/1295101763 shutdown_connections 2024-04-22T23:47:24.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.882+0000 7f7ac795d640 1 --2- 172.21.15.107:0/1295101763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ac0101a70 0x7f7ac00fb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:24.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.882+0000 7f7ac795d640 1 --2- 172.21.15.107:0/1295101763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ac01010d0 0x7f7ac01014a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:24.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.882+0000 7f7ac795d640 1 -- 172.21.15.107:0/1295101763 >> 172.21.15.107:0/1295101763 conn(0x7f7ac00f6ca0 msgr2=0x7f7ac00f90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:24.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.882+0000 7f7ac795d640 1 -- 172.21.15.107:0/1295101763 shutdown_connections 2024-04-22T23:47:24.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.882+0000 7f7ac795d640 1 -- 172.21.15.107:0/1295101763 wait complete. 2024-04-22T23:47:24.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.882+0000 7f7ac795d640 1 Processor -- start 2024-04-22T23:47:24.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.882+0000 7f7ac795d640 1 -- start start 2024-04-22T23:47:24.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.883+0000 7f7ac795d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ac01010d0 0x7f7ac010de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:24.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.883+0000 7f7ac795d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ac0101a70 0x7f7ac010e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:24.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.883+0000 7f7ac795d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac010e8b0 con 0x7f7ac01010d0 2024-04-22T23:47:24.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.883+0000 7f7ac795d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac010ea20 con 0x7f7ac0101a70 2024-04-22T23:47:24.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.883+0000 7f7ac56f9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ac01010d0 0x7f7ac010de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:24.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.883+0000 7f7ac56f9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ac01010d0 0x7f7ac010de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38678/0 (socket says 172.21.15.107:38678) 2024-04-22T23:47:24.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.883+0000 7f7ac56f9640 1 -- 172.21.15.107:0/1326572190 learned_addr learned my addr 172.21.15.107:0/1326572190 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:47:24.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.883+0000 7f7ac4ef8640 1 --2- 172.21.15.107:0/1326572190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ac0101a70 0x7f7ac010e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:24.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.883+0000 7f7ac56f9640 1 -- 172.21.15.107:0/1326572190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ac0101a70 msgr2=0x7f7ac010e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:24.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.883+0000 7f7ac56f9640 1 --2- 172.21.15.107:0/1326572190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ac0101a70 0x7f7ac010e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:24.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.883+0000 7f7ac56f9640 1 -- 172.21.15.107:0/1326572190 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7abc005040 con 0x7f7ac01010d0 2024-04-22T23:47:24.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.883+0000 7f7ac4ef8640 1 --2- 172.21.15.107:0/1326572190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ac0101a70 0x7f7ac010e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T23:47:24.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.883+0000 7f7ac56f9640 1 --2- 172.21.15.107:0/1326572190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ac01010d0 0x7f7ac010de30 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7abc034040 tx=0x7f7abc009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:24.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.883+0000 7f7ab5ffb640 1 -- 172.21.15.107:0/1326572190 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7abc0092b0 con 0x7f7ac01010d0 2024-04-22T23:47:24.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.883+0000 7f7ab5ffb640 1 -- 172.21.15.107:0/1326572190 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7abc03b400 con 0x7f7ac01010d0 2024-04-22T23:47:24.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.883+0000 7f7ac795d640 1 -- 172.21.15.107:0/1326572190 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ac010eca0 con 0x7f7ac01010d0 2024-04-22T23:47:24.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.883+0000 7f7ab5ffb640 1 -- 172.21.15.107:0/1326572190 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7abc0435d0 con 0x7f7ac01010d0 2024-04-22T23:47:24.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.884+0000 7f7ac795d640 1 -- 172.21.15.107:0/1326572190 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ac01170f0 con 0x7f7ac01010d0 2024-04-22T23:47:24.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.884+0000 7f7ac795d640 1 -- 172.21.15.107:0/1326572190 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ac00fc840 con 0x7f7ac01010d0 2024-04-22T23:47:24.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.885+0000 7f7ab5ffb640 1 -- 172.21.15.107:0/1326572190 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f7abc02f070 con 0x7f7ac01010d0 2024-04-22T23:47:24.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.885+0000 7f7ab5ffb640 1 --2- 172.21.15.107:0/1326572190 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f7aac06eb70 0x7f7aac071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:24.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.885+0000 7f7ab5ffb640 1 -- 172.21.15.107:0/1326572190 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f7abc0b0530 con 0x7f7ac01010d0 2024-04-22T23:47:24.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.885+0000 7f7ac4ef8640 1 --2- 172.21.15.107:0/1326572190 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f7aac06eb70 0x7f7aac071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:24.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.886+0000 7f7ac4ef8640 1 --2- 172.21.15.107:0/1326572190 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f7aac06eb70 0x7f7aac071020 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0065880 tx=0x7f7ab000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:24.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:24.890+0000 7f7ab5ffb640 1 -- 172.21.15.107:0/1326572190 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f7abc079230 con 0x7f7ac01010d0 2024-04-22T23:47:25.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.105+0000 7f7ac795d640 1 -- 172.21.15.107:0/1326572190 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) v1 -- 0x7f7ac004bcc0 con 0x7f7ac01010d0 2024-04-22T23:47:25.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.106+0000 7f7ab5ffb640 1 -- 172.21.15.107:0/1326572190 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v43) v1 ==== 107+0+4158 (secure 0 0 0) 0x7f7abc07c8f0 con 0x7f7ac01010d0 2024-04-22T23:47:25.098 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:47:25.098 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34434,"name":"cephfs.smithi107.dfvvkw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6827/418145840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":418145840},{"type":"v1","addr":"172.21.15.107:6827","nonce":418145840}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44279,"name":"cephfs.smithi157.rcaris","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6825/1722945322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1722945322},{"type":"v1","addr":"172.21.15.157:6825","nonce":1722945322}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T23:32:20.146130+0000","modified":"2024-04-22T23:43:34.077025+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24277},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_24277":{"gid":24277,"name":"cephfs.smithi157.ufywse","rank":0,"incarnation":29,"state":"up:reconnect","state_seq":167,"addr":"172.21.15.157:6827/4072444199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":4072444199},{"type":"v1","addr":"172.21.15.157:6827","nonce":4072444199}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-22T23:47:25.098 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 30 2024-04-22T23:47:25.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.108+0000 7f7ac795d640 1 -- 172.21.15.107:0/1326572190 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f7aac06eb70 msgr2=0x7f7aac071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:25.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.108+0000 7f7ac795d640 1 --2- 172.21.15.107:0/1326572190 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f7aac06eb70 0x7f7aac071020 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0065880 tx=0x7f7ab000b040 comp rx=0 tx=0).stop 2024-04-22T23:47:25.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.108+0000 7f7ac795d640 1 -- 172.21.15.107:0/1326572190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ac01010d0 msgr2=0x7f7ac010de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:25.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.108+0000 7f7ac795d640 1 --2- 172.21.15.107:0/1326572190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ac01010d0 0x7f7ac010de30 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7abc034040 tx=0x7f7abc009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:25.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.108+0000 7f7ac795d640 1 -- 172.21.15.107:0/1326572190 shutdown_connections 2024-04-22T23:47:25.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.108+0000 7f7ac795d640 1 --2- 172.21.15.107:0/1326572190 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f7aac06eb70 0x7f7aac071020 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:25.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.108+0000 7f7ac795d640 1 --2- 172.21.15.107:0/1326572190 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ac0101a70 0x7f7ac010e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:25.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.108+0000 7f7ac795d640 1 --2- 172.21.15.107:0/1326572190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ac01010d0 0x7f7ac010de30 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:25.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.108+0000 7f7ac795d640 1 -- 172.21.15.107:0/1326572190 >> 172.21.15.107:0/1326572190 conn(0x7f7ac00f6ca0 msgr2=0x7f7ac00f7c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:25.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.108+0000 7f7ac795d640 1 -- 172.21.15.107:0/1326572190 shutdown_connections 2024-04-22T23:47:25.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.108+0000 7f7ac795d640 1 -- 172.21.15.107:0/1326572190 wait complete. 2024-04-22T23:47:25.109 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:24 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/980368365' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-04-22T23:47:25.109 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:24 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/3017025177' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-04-22T23:47:25.109 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:24 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/1367748586' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-04-22T23:47:25.111 DEBUG:tasks.fs:max_mds reduced in epoch 30 2024-04-22T23:47:25.111 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 31 2024-04-22T23:47:25.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:24 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/980368365' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-04-22T23:47:25.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:24 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/3017025177' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-04-22T23:47:25.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:24 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/1367748586' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-04-22T23:47:25.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.214+0000 7f84d806e640 1 -- 172.21.15.107:0/3843057669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84d0101100 msgr2=0x7f84d01014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:25.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.214+0000 7f84d806e640 1 --2- 172.21.15.107:0/3843057669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84d0101100 0x7f84d01014d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f84cc0048d0 tx=0x7f84cc009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:25.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.214+0000 7f84d806e640 1 -- 172.21.15.107:0/3843057669 shutdown_connections 2024-04-22T23:47:25.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.214+0000 7f84d806e640 1 --2- 172.21.15.107:0/3843057669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84d0101aa0 0x7f84d00fb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:25.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.214+0000 7f84d806e640 1 --2- 172.21.15.107:0/3843057669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84d0101100 0x7f84d01014d0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:25.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.214+0000 7f84d806e640 1 -- 172.21.15.107:0/3843057669 >> 172.21.15.107:0/3843057669 conn(0x7f84d00f6cd0 msgr2=0x7f84d00f90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:25.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.214+0000 7f84d806e640 1 -- 172.21.15.107:0/3843057669 shutdown_connections 2024-04-22T23:47:25.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.214+0000 7f84d806e640 1 -- 172.21.15.107:0/3843057669 wait complete. 2024-04-22T23:47:25.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.214+0000 7f84d806e640 1 Processor -- start 2024-04-22T23:47:25.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.214+0000 7f84d806e640 1 -- start start 2024-04-22T23:47:25.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.214+0000 7f84d806e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84d0101100 0x7f84d010dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:25.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.214+0000 7f84d806e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84d0101aa0 0x7f84d010e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:25.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.214+0000 7f84d806e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d0066520 con 0x7f84d0101aa0 2024-04-22T23:47:25.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.214+0000 7f84d806e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d010e970 con 0x7f84d0101100 2024-04-22T23:47:25.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.214+0000 7f84d5609640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84d0101aa0 0x7f84d010e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:25.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.214+0000 7f84d5609640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84d0101aa0 0x7f84d010e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38696/0 (socket says 172.21.15.107:38696) 2024-04-22T23:47:25.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.214+0000 7f84d5609640 1 -- 172.21.15.107:0/4052687803 learned_addr learned my addr 172.21.15.107:0/4052687803 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:47:25.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.215+0000 7f84d5e0a640 1 --2- 172.21.15.107:0/4052687803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84d0101100 0x7f84d010dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:25.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.215+0000 7f84d5609640 1 -- 172.21.15.107:0/4052687803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84d0101100 msgr2=0x7f84d010dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:25.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.215+0000 7f84d5609640 1 --2- 172.21.15.107:0/4052687803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84d0101100 0x7f84d010dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:25.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.215+0000 7f84d5609640 1 -- 172.21.15.107:0/4052687803 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84cc005040 con 0x7f84d0101aa0 2024-04-22T23:47:25.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.215+0000 7f84d5609640 1 --2- 172.21.15.107:0/4052687803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84d0101aa0 0x7f84d010e400 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f84c0002a50 tx=0x7f84c0004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:25.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.215+0000 7f84c67fc640 1 -- 172.21.15.107:0/4052687803 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84c0007c30 con 0x7f84d0101aa0 2024-04-22T23:47:25.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.215+0000 7f84c67fc640 1 -- 172.21.15.107:0/4052687803 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f84c000f040 con 0x7f84d0101aa0 2024-04-22T23:47:25.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.215+0000 7f84d806e640 1 -- 172.21.15.107:0/4052687803 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84d010ec50 con 0x7f84d0101aa0 2024-04-22T23:47:25.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.215+0000 7f84c67fc640 1 -- 172.21.15.107:0/4052687803 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84c000db50 con 0x7f84d0101aa0 2024-04-22T23:47:25.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.215+0000 7f84d806e640 1 -- 172.21.15.107:0/4052687803 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84d01171e0 con 0x7f84d0101aa0 2024-04-22T23:47:25.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.216+0000 7f84d806e640 1 -- 172.21.15.107:0/4052687803 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84d00fc870 con 0x7f84d0101aa0 2024-04-22T23:47:25.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.216+0000 7f84c67fc640 1 -- 172.21.15.107:0/4052687803 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f84c000dcf0 con 0x7f84d0101aa0 2024-04-22T23:47:25.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.217+0000 7f84c67fc640 1 --2- 172.21.15.107:0/4052687803 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f84bc06eb70 0x7f84bc071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:25.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.217+0000 7f84c67fc640 1 -- 172.21.15.107:0/4052687803 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f84c008e180 con 0x7f84d0101aa0 2024-04-22T23:47:25.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.217+0000 7f84d5e0a640 1 --2- 172.21.15.107:0/4052687803 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f84bc06eb70 0x7f84bc071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:25.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.217+0000 7f84d5e0a640 1 --2- 172.21.15.107:0/4052687803 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f84bc06eb70 0x7f84bc071020 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f84cc003080 tx=0x7f84cc009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:25.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.221+0000 7f84c67fc640 1 -- 172.21.15.107:0/4052687803 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f84c0056f00 con 0x7f84d0101aa0 2024-04-22T23:47:25.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.435+0000 7f84d806e640 1 -- 172.21.15.107:0/4052687803 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 31, "format": "json"} v 0) v1 -- 0x7f84d0065b20 con 0x7f84d0101aa0 2024-04-22T23:47:25.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.436+0000 7f84c67fc640 1 -- 172.21.15.107:0/4052687803 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 31, "format": "json"}]=0 dumped fsmap epoch 31 v43) v1 ==== 107+0+4155 (secure 0 0 0) 0x7f84c005a5c0 con 0x7f84d0101aa0 2024-04-22T23:47:25.428 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:47:25.428 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34434,"name":"cephfs.smithi107.dfvvkw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6827/418145840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":418145840},{"type":"v1","addr":"172.21.15.107:6827","nonce":418145840}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44279,"name":"cephfs.smithi157.rcaris","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6825/1722945322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1722945322},{"type":"v1","addr":"172.21.15.157:6825","nonce":1722945322}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T23:32:20.146130+0000","modified":"2024-04-22T23:43:34.798931+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24277},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_24277":{"gid":24277,"name":"cephfs.smithi157.ufywse","rank":0,"incarnation":29,"state":"up:rejoin","state_seq":168,"addr":"172.21.15.157:6827/4072444199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":4072444199},{"type":"v1","addr":"172.21.15.157:6827","nonce":4072444199}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-22T23:47:25.428 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 31 2024-04-22T23:47:25.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.438+0000 7f84d806e640 1 -- 172.21.15.107:0/4052687803 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f84bc06eb70 msgr2=0x7f84bc071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:25.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.438+0000 7f84d806e640 1 --2- 172.21.15.107:0/4052687803 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f84bc06eb70 0x7f84bc071020 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f84cc003080 tx=0x7f84cc009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:25.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.438+0000 7f84d806e640 1 -- 172.21.15.107:0/4052687803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84d0101aa0 msgr2=0x7f84d010e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:25.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.438+0000 7f84d806e640 1 --2- 172.21.15.107:0/4052687803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84d0101aa0 0x7f84d010e400 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f84c0002a50 tx=0x7f84c0004600 comp rx=0 tx=0).stop 2024-04-22T23:47:25.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.438+0000 7f84d806e640 1 -- 172.21.15.107:0/4052687803 shutdown_connections 2024-04-22T23:47:25.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.438+0000 7f84d806e640 1 --2- 172.21.15.107:0/4052687803 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f84bc06eb70 0x7f84bc071020 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:25.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.438+0000 7f84d806e640 1 --2- 172.21.15.107:0/4052687803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84d0101aa0 0x7f84d010e400 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:25.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.438+0000 7f84d806e640 1 --2- 172.21.15.107:0/4052687803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84d0101100 0x7f84d010dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:25.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.438+0000 7f84d806e640 1 -- 172.21.15.107:0/4052687803 >> 172.21.15.107:0/4052687803 conn(0x7f84d00f6cd0 msgr2=0x7f84d00f7b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:25.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.438+0000 7f84d806e640 1 -- 172.21.15.107:0/4052687803 shutdown_connections 2024-04-22T23:47:25.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.438+0000 7f84d806e640 1 -- 172.21.15.107:0/4052687803 wait complete. 2024-04-22T23:47:25.441 DEBUG:tasks.fs:max_mds reduced in epoch 31 2024-04-22T23:47:25.441 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 32 2024-04-22T23:47:25.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.544+0000 7fdc7322e640 1 -- 172.21.15.107:0/4177091193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc6c102b90 msgr2=0x7fdc6c0fa8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:25.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.544+0000 7fdc7322e640 1 --2- 172.21.15.107:0/4177091193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc6c102b90 0x7fdc6c0fa8d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fdc580048d0 tx=0x7fdc58009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:25.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.544+0000 7fdc7322e640 1 -- 172.21.15.107:0/4177091193 shutdown_connections 2024-04-22T23:47:25.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.544+0000 7fdc7322e640 1 --2- 172.21.15.107:0/4177091193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc6c102b90 0x7fdc6c0fa8d0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:25.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.544+0000 7fdc7322e640 1 --2- 172.21.15.107:0/4177091193 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc6c102280 0x7fdc6c102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:25.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.544+0000 7fdc7322e640 1 -- 172.21.15.107:0/4177091193 >> 172.21.15.107:0/4177091193 conn(0x7fdc6c0708f0 msgr2=0x7fdc6c070cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:25.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.544+0000 7fdc7322e640 1 -- 172.21.15.107:0/4177091193 shutdown_connections 2024-04-22T23:47:25.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.544+0000 7fdc7322e640 1 -- 172.21.15.107:0/4177091193 wait complete. 2024-04-22T23:47:25.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.545+0000 7fdc7322e640 1 Processor -- start 2024-04-22T23:47:25.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.545+0000 7fdc7322e640 1 -- start start 2024-04-22T23:47:25.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.545+0000 7fdc7322e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc6c102280 0x7fdc6c112d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:25.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.545+0000 7fdc7322e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc6c102b90 0x7fdc6c1132a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:25.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.545+0000 7fdc7322e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c1138a0 con 0x7fdc6c102280 2024-04-22T23:47:25.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.545+0000 7fdc7322e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c10cef0 con 0x7fdc6c102b90 2024-04-22T23:47:25.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.545+0000 7fdc70fca640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc6c102280 0x7fdc6c112d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:25.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.545+0000 7fdc70fca640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc6c102280 0x7fdc6c112d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38720/0 (socket says 172.21.15.107:38720) 2024-04-22T23:47:25.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.545+0000 7fdc70fca640 1 -- 172.21.15.107:0/3130998434 learned_addr learned my addr 172.21.15.107:0/3130998434 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:47:25.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.546+0000 7fdc6bfff640 1 --2- 172.21.15.107:0/3130998434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc6c102b90 0x7fdc6c1132a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:25.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.546+0000 7fdc70fca640 1 -- 172.21.15.107:0/3130998434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc6c102b90 msgr2=0x7fdc6c1132a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:25.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.546+0000 7fdc70fca640 1 --2- 172.21.15.107:0/3130998434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc6c102b90 0x7fdc6c1132a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:25.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.546+0000 7fdc70fca640 1 -- 172.21.15.107:0/3130998434 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc58005040 con 0x7fdc6c102280 2024-04-22T23:47:25.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.546+0000 7fdc6bfff640 1 --2- 172.21.15.107:0/3130998434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc6c102b90 0x7fdc6c1132a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T23:47:25.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.546+0000 7fdc70fca640 1 --2- 172.21.15.107:0/3130998434 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc6c102280 0x7fdc6c112d60 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fdc6000c830 tx=0x7fdc60004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:25.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.546+0000 7fdc697fa640 1 -- 172.21.15.107:0/3130998434 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc6000a850 con 0x7fdc6c102280 2024-04-22T23:47:25.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.546+0000 7fdc7322e640 1 -- 172.21.15.107:0/3130998434 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc6c10d1d0 con 0x7fdc6c102280 2024-04-22T23:47:25.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.546+0000 7fdc697fa640 1 -- 172.21.15.107:0/3130998434 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdc6000ae80 con 0x7fdc6c102280 2024-04-22T23:47:25.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.546+0000 7fdc697fa640 1 -- 172.21.15.107:0/3130998434 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc60012ea0 con 0x7fdc6c102280 2024-04-22T23:47:25.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.546+0000 7fdc7322e640 1 -- 172.21.15.107:0/3130998434 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc6c10d6f0 con 0x7fdc6c102280 2024-04-22T23:47:25.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.547+0000 7fdc7322e640 1 -- 172.21.15.107:0/3130998434 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc6c065330 con 0x7fdc6c102280 2024-04-22T23:47:25.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.547+0000 7fdc697fa640 1 -- 172.21.15.107:0/3130998434 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fdc60012400 con 0x7fdc6c102280 2024-04-22T23:47:25.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.548+0000 7fdc697fa640 1 --2- 172.21.15.107:0/3130998434 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fdc5c06eb70 0x7fdc5c071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:25.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.548+0000 7fdc697fa640 1 -- 172.21.15.107:0/3130998434 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fdc6008ebd0 con 0x7fdc6c102280 2024-04-22T23:47:25.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.548+0000 7fdc6bfff640 1 --2- 172.21.15.107:0/3130998434 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fdc5c06eb70 0x7fdc5c071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:25.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.548+0000 7fdc6bfff640 1 --2- 172.21.15.107:0/3130998434 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fdc5c06eb70 0x7fdc5c071020 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fdc58000c00 tx=0x7fdc58009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:25.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.552+0000 7fdc697fa640 1 -- 172.21.15.107:0/3130998434 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fdc60057950 con 0x7fdc6c102280 2024-04-22T23:47:25.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.769+0000 7fdc7322e640 1 -- 172.21.15.107:0/3130998434 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 32, "format": "json"} v 0) v1 -- 0x7fdc6c04bcc0 con 0x7fdc6c102280 2024-04-22T23:47:25.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.769+0000 7fdc697fa640 1 -- 172.21.15.107:0/3130998434 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 32, "format": "json"}]=0 dumped fsmap epoch 32 v43) v1 ==== 107+0+5014 (secure 0 0 0) 0x7fdc600126b0 con 0x7fdc6c102280 2024-04-22T23:47:25.761 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:47:25.761 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34434,"name":"cephfs.smithi107.dfvvkw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6827/418145840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":418145840},{"type":"v1","addr":"172.21.15.107:6827","nonce":418145840}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":34438,"name":"cephfs.smithi107.ktuxjl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6829/1256497807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1256497807},{"type":"v1","addr":"172.21.15.107:6829","nonce":1256497807}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":44279,"name":"cephfs.smithi157.rcaris","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6825/1722945322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":1722945322},{"type":"v1","addr":"172.21.15.157:6825","nonce":1722945322}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T23:32:20.146130+0000","modified":"2024-04-22T23:43:36.801717+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24277},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_24277":{"gid":24277,"name":"cephfs.smithi157.ufywse","rank":0,"incarnation":29,"state":"up:active","state_seq":169,"addr":"172.21.15.157:6827/4072444199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":4072444199},{"type":"v1","addr":"172.21.15.157:6827","nonce":4072444199}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24277,"qdb_cluster":[24277]},"id":1}]} 2024-04-22T23:47:25.762 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 32 2024-04-22T23:47:25.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.772+0000 7fdc7322e640 1 -- 172.21.15.107:0/3130998434 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fdc5c06eb70 msgr2=0x7fdc5c071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:25.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.772+0000 7fdc7322e640 1 --2- 172.21.15.107:0/3130998434 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fdc5c06eb70 0x7fdc5c071020 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fdc58000c00 tx=0x7fdc58009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:25.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.772+0000 7fdc7322e640 1 -- 172.21.15.107:0/3130998434 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc6c102280 msgr2=0x7fdc6c112d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:25.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.772+0000 7fdc7322e640 1 --2- 172.21.15.107:0/3130998434 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc6c102280 0x7fdc6c112d60 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fdc6000c830 tx=0x7fdc60004430 comp rx=0 tx=0).stop 2024-04-22T23:47:25.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.772+0000 7fdc7322e640 1 -- 172.21.15.107:0/3130998434 shutdown_connections 2024-04-22T23:47:25.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.772+0000 7fdc7322e640 1 --2- 172.21.15.107:0/3130998434 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fdc5c06eb70 0x7fdc5c071020 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:25.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.772+0000 7fdc7322e640 1 --2- 172.21.15.107:0/3130998434 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc6c102b90 0x7fdc6c1132a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:25.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.772+0000 7fdc7322e640 1 --2- 172.21.15.107:0/3130998434 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc6c102280 0x7fdc6c112d60 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:25.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.772+0000 7fdc7322e640 1 -- 172.21.15.107:0/3130998434 >> 172.21.15.107:0/3130998434 conn(0x7fdc6c0708f0 msgr2=0x7fdc6c0ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:25.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.772+0000 7fdc7322e640 1 -- 172.21.15.107:0/3130998434 shutdown_connections 2024-04-22T23:47:25.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.772+0000 7fdc7322e640 1 -- 172.21.15.107:0/3130998434 wait complete. 2024-04-22T23:47:25.775 DEBUG:tasks.fs:max_mds reduced in epoch 32 2024-04-22T23:47:25.775 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 33 2024-04-22T23:47:25.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.879+0000 7f34efe6e640 1 -- 172.21.15.107:0/3126690414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34e8101150 msgr2=0x7f34e8101520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:25.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.879+0000 7f34efe6e640 1 --2- 172.21.15.107:0/3126690414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34e8101150 0x7f34e8101520 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f34e40048d0 tx=0x7f34e4009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:25.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.879+0000 7f34efe6e640 1 -- 172.21.15.107:0/3126690414 shutdown_connections 2024-04-22T23:47:25.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.879+0000 7f34efe6e640 1 --2- 172.21.15.107:0/3126690414 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34e80fb120 0x7f34e80fb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:25.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.879+0000 7f34efe6e640 1 --2- 172.21.15.107:0/3126690414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34e8101150 0x7f34e8101520 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:25.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.879+0000 7f34efe6e640 1 -- 172.21.15.107:0/3126690414 >> 172.21.15.107:0/3126690414 conn(0x7f34e80710c0 msgr2=0x7f34e80714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:25.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.879+0000 7f34efe6e640 1 -- 172.21.15.107:0/3126690414 shutdown_connections 2024-04-22T23:47:25.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.879+0000 7f34efe6e640 1 -- 172.21.15.107:0/3126690414 wait complete. 2024-04-22T23:47:25.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.880+0000 7f34efe6e640 1 Processor -- start 2024-04-22T23:47:25.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.880+0000 7f34efe6e640 1 -- start start 2024-04-22T23:47:25.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.880+0000 7f34efe6e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34e80fb120 0x7f34e8101f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:25.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.880+0000 7f34efe6e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34e8101150 0x7f34e8102440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:25.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.880+0000 7f34efe6e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e8103e60 con 0x7f34e80fb120 2024-04-22T23:47:25.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.880+0000 7f34efe6e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e8102980 con 0x7f34e8101150 2024-04-22T23:47:25.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.880+0000 7f34edc0a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34e80fb120 0x7f34e8101f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:25.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.880+0000 7f34edc0a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34e80fb120 0x7f34e8101f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38726/0 (socket says 172.21.15.107:38726) 2024-04-22T23:47:25.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.880+0000 7f34edc0a640 1 -- 172.21.15.107:0/827542657 learned_addr learned my addr 172.21.15.107:0/827542657 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:47:25.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.880+0000 7f34ed409640 1 --2- 172.21.15.107:0/827542657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34e8101150 0x7f34e8102440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:25.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.880+0000 7f34edc0a640 1 -- 172.21.15.107:0/827542657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34e8101150 msgr2=0x7f34e8102440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:25.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.880+0000 7f34edc0a640 1 --2- 172.21.15.107:0/827542657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34e8101150 0x7f34e8102440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:25.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.880+0000 7f34edc0a640 1 -- 172.21.15.107:0/827542657 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e4005040 con 0x7f34e80fb120 2024-04-22T23:47:25.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.881+0000 7f34edc0a640 1 --2- 172.21.15.107:0/827542657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34e80fb120 0x7f34e8101f00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f34e40043d0 tx=0x7f34e4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:25.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.882+0000 7f34de7fc640 1 -- 172.21.15.107:0/827542657 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34e4030840 con 0x7f34e80fb120 2024-04-22T23:47:25.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.882+0000 7f34de7fc640 1 -- 172.21.15.107:0/827542657 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f34e40092e0 con 0x7f34e80fb120 2024-04-22T23:47:25.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.882+0000 7f34efe6e640 1 -- 172.21.15.107:0/827542657 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34e8102c00 con 0x7f34e80fb120 2024-04-22T23:47:25.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.882+0000 7f34de7fc640 1 -- 172.21.15.107:0/827542657 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34e4043640 con 0x7f34e80fb120 2024-04-22T23:47:25.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.882+0000 7f34efe6e640 1 -- 172.21.15.107:0/827542657 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34e81193d0 con 0x7f34e80fb120 2024-04-22T23:47:25.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.883+0000 7f34efe6e640 1 -- 172.21.15.107:0/827542657 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34e80fc810 con 0x7f34e80fb120 2024-04-22T23:47:25.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.883+0000 7f34de7fc640 1 -- 172.21.15.107:0/827542657 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f34e403b420 con 0x7f34e80fb120 2024-04-22T23:47:25.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.884+0000 7f34de7fc640 1 --2- 172.21.15.107:0/827542657 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f34d406eb80 0x7f34d4071030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:25.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.884+0000 7f34de7fc640 1 -- 172.21.15.107:0/827542657 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f34e40b1650 con 0x7f34e80fb120 2024-04-22T23:47:25.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.884+0000 7f34ed409640 1 --2- 172.21.15.107:0/827542657 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f34d406eb80 0x7f34d4071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:25.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.884+0000 7f34ed409640 1 --2- 172.21.15.107:0/827542657 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f34d406eb80 0x7f34d4071030 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f34d8009fd0 tx=0x7f34d8009e50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:25.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:25.889+0000 7f34de7fc640 1 -- 172.21.15.107:0/827542657 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f34e407a3d0 con 0x7f34e80fb120 2024-04-22T23:47:26.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.102+0000 7f34efe6e640 1 -- 172.21.15.107:0/827542657 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 33, "format": "json"} v 0) v1 -- 0x7f34e804bcc0 con 0x7f34e80fb120 2024-04-22T23:47:26.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.102+0000 7f34de7fc640 1 -- 172.21.15.107:0/827542657 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 33, "format": "json"}]=0 dumped fsmap epoch 33 v43) v1 ==== 107+0+4232 (secure 0 0 0) 0x7f34e407da90 con 0x7f34e80fb120 2024-04-22T23:47:26.094 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:47:26.094 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34434,"name":"cephfs.smithi107.dfvvkw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6827/418145840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":418145840},{"type":"v1","addr":"172.21.15.107:6827","nonce":418145840}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":34438,"name":"cephfs.smithi107.ktuxjl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6829/1256497807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1256497807},{"type":"v1","addr":"172.21.15.107:6829","nonce":1256497807}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T23:32:20.146130+0000","modified":"2024-04-22T23:43:36.801717+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24277},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_24277":{"gid":24277,"name":"cephfs.smithi157.ufywse","rank":0,"incarnation":29,"state":"up:active","state_seq":169,"addr":"172.21.15.157:6827/4072444199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":4072444199},{"type":"v1","addr":"172.21.15.157:6827","nonce":4072444199}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24277,"qdb_cluster":[24277]},"id":1}]} 2024-04-22T23:47:26.094 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 33 2024-04-22T23:47:26.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.104+0000 7f34efe6e640 1 -- 172.21.15.107:0/827542657 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f34d406eb80 msgr2=0x7f34d4071030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:26.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.104+0000 7f34efe6e640 1 --2- 172.21.15.107:0/827542657 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f34d406eb80 0x7f34d4071030 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f34d8009fd0 tx=0x7f34d8009e50 comp rx=0 tx=0).stop 2024-04-22T23:47:26.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.104+0000 7f34efe6e640 1 -- 172.21.15.107:0/827542657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34e80fb120 msgr2=0x7f34e8101f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:26.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.104+0000 7f34efe6e640 1 --2- 172.21.15.107:0/827542657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34e80fb120 0x7f34e8101f00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f34e40043d0 tx=0x7f34e4009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:26.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.104+0000 7f34efe6e640 1 -- 172.21.15.107:0/827542657 shutdown_connections 2024-04-22T23:47:26.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.104+0000 7f34efe6e640 1 --2- 172.21.15.107:0/827542657 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f34d406eb80 0x7f34d4071030 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:26.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.104+0000 7f34efe6e640 1 --2- 172.21.15.107:0/827542657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34e8101150 0x7f34e8102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:26.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.104+0000 7f34efe6e640 1 --2- 172.21.15.107:0/827542657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34e80fb120 0x7f34e8101f00 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:26.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.104+0000 7f34efe6e640 1 -- 172.21.15.107:0/827542657 >> 172.21.15.107:0/827542657 conn(0x7f34e80710c0 msgr2=0x7f34e80643c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:26.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.104+0000 7f34efe6e640 1 -- 172.21.15.107:0/827542657 shutdown_connections 2024-04-22T23:47:26.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.104+0000 7f34efe6e640 1 -- 172.21.15.107:0/827542657 wait complete. 2024-04-22T23:47:26.106 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:25 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/1326572190' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-04-22T23:47:26.106 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:25 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/4052687803' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-04-22T23:47:26.106 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:25 smithi107 ceph-mon[90179]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:47:26.106 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:25 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/3130998434' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-04-22T23:47:26.106 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:25 smithi107 ceph-mon[90179]: pgmap v322: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:47:26.107 DEBUG:tasks.fs:max_mds reduced in epoch 33 2024-04-22T23:47:26.107 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 34 2024-04-22T23:47:26.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:25 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/1326572190' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-04-22T23:47:26.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:25 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/4052687803' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-04-22T23:47:26.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:25 smithi157 ceph-mon[79403]: from='mgr.34104 172.21.15.107:0/376113948' entity='mgr.smithi107.uipxvm' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-22T23:47:26.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:25 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/3130998434' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-04-22T23:47:26.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:25 smithi157 ceph-mon[79403]: pgmap v322: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:47:26.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.209+0000 7f5809b1d640 1 -- 172.21.15.107:0/755852096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5804102b90 msgr2=0x7f58040fa8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:26.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.209+0000 7f5809b1d640 1 --2- 172.21.15.107:0/755852096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5804102b90 0x7f58040fa8d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f57ec0048d0 tx=0x7f57ec009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:26.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.209+0000 7f5809b1d640 1 -- 172.21.15.107:0/755852096 shutdown_connections 2024-04-22T23:47:26.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.209+0000 7f5809b1d640 1 --2- 172.21.15.107:0/755852096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5804102b90 0x7f58040fa8d0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:26.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.209+0000 7f5809b1d640 1 --2- 172.21.15.107:0/755852096 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5804102280 0x7f5804102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:26.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.209+0000 7f5809b1d640 1 -- 172.21.15.107:0/755852096 >> 172.21.15.107:0/755852096 conn(0x7f58040708f0 msgr2=0x7f5804070cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:26.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.209+0000 7f5809b1d640 1 -- 172.21.15.107:0/755852096 shutdown_connections 2024-04-22T23:47:26.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.209+0000 7f5809b1d640 1 -- 172.21.15.107:0/755852096 wait complete. 2024-04-22T23:47:26.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.209+0000 7f5809b1d640 1 Processor -- start 2024-04-22T23:47:26.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.209+0000 7f5809b1d640 1 -- start start 2024-04-22T23:47:26.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.210+0000 7f5809b1d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5804102280 0x7f5804112d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:26.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.210+0000 7f5809b1d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5804102b90 0x7f58041132a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:26.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.210+0000 7f5809b1d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58041138a0 con 0x7f5804102280 2024-04-22T23:47:26.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.210+0000 7f5809b1d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f580410cef0 con 0x7f5804102b90 2024-04-22T23:47:26.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.210+0000 7f58037fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5804102280 0x7f5804112d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:26.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.210+0000 7f58037fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5804102280 0x7f5804112d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38748/0 (socket says 172.21.15.107:38748) 2024-04-22T23:47:26.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.210+0000 7f58037fe640 1 -- 172.21.15.107:0/1566424583 learned_addr learned my addr 172.21.15.107:0/1566424583 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:47:26.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.210+0000 7f57fbfff640 1 --2- 172.21.15.107:0/1566424583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5804102b90 0x7f58041132a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:26.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.210+0000 7f58037fe640 1 -- 172.21.15.107:0/1566424583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5804102b90 msgr2=0x7f58041132a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:26.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.210+0000 7f58037fe640 1 --2- 172.21.15.107:0/1566424583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5804102b90 0x7f58041132a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:26.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.210+0000 7f58037fe640 1 -- 172.21.15.107:0/1566424583 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57ec005040 con 0x7f5804102280 2024-04-22T23:47:26.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.210+0000 7f57fbfff640 1 --2- 172.21.15.107:0/1566424583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5804102b90 0x7f58041132a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T23:47:26.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.210+0000 7f58037fe640 1 --2- 172.21.15.107:0/1566424583 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5804102280 0x7f5804112d60 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f57f400c830 tx=0x7f57f4004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:26.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.210+0000 7f5800ff9640 1 -- 172.21.15.107:0/1566424583 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57f400a850 con 0x7f5804102280 2024-04-22T23:47:26.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.210+0000 7f5800ff9640 1 -- 172.21.15.107:0/1566424583 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f57f400ae80 con 0x7f5804102280 2024-04-22T23:47:26.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.210+0000 7f5809b1d640 1 -- 172.21.15.107:0/1566424583 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f580410d1d0 con 0x7f5804102280 2024-04-22T23:47:26.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.210+0000 7f5800ff9640 1 -- 172.21.15.107:0/1566424583 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57f4012ea0 con 0x7f5804102280 2024-04-22T23:47:26.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.210+0000 7f5809b1d640 1 -- 172.21.15.107:0/1566424583 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f580410d6f0 con 0x7f5804102280 2024-04-22T23:47:26.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.211+0000 7f5809b1d640 1 -- 172.21.15.107:0/1566424583 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5804065330 con 0x7f5804102280 2024-04-22T23:47:26.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.212+0000 7f5800ff9640 1 -- 172.21.15.107:0/1566424583 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f57f4012400 con 0x7f5804102280 2024-04-22T23:47:26.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.212+0000 7f5800ff9640 1 --2- 172.21.15.107:0/1566424583 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f57f006eb80 0x7f57f0071030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:26.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.212+0000 7f5800ff9640 1 -- 172.21.15.107:0/1566424583 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f57f408ebd0 con 0x7f5804102280 2024-04-22T23:47:26.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.212+0000 7f57fbfff640 1 --2- 172.21.15.107:0/1566424583 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f57f006eb80 0x7f57f0071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:26.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.213+0000 7f57fbfff640 1 --2- 172.21.15.107:0/1566424583 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f57f006eb80 0x7f57f0071030 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f57ec000c00 tx=0x7f57ec009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:26.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.217+0000 7f5800ff9640 1 -- 172.21.15.107:0/1566424583 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f57f4057950 con 0x7f5804102280 2024-04-22T23:47:26.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.431+0000 7f5809b1d640 1 -- 172.21.15.107:0/1566424583 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 34, "format": "json"} v 0) v1 -- 0x7f580404bcc0 con 0x7f5804102280 2024-04-22T23:47:26.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.432+0000 7f5800ff9640 1 -- 172.21.15.107:0/1566424583 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 34, "format": "json"}]=0 dumped fsmap epoch 34 v43) v1 ==== 107+0+5082 (secure 0 0 0) 0x7f57f40126b0 con 0x7f5804102280 2024-04-22T23:47:26.424 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:47:26.424 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34434,"name":"cephfs.smithi107.dfvvkw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6827/418145840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":418145840},{"type":"v1","addr":"172.21.15.107:6827","nonce":418145840}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":34438,"name":"cephfs.smithi107.ktuxjl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6829/1256497807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1256497807},{"type":"v1","addr":"172.21.15.107:6829","nonce":1256497807}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":44323,"name":"cephfs.smithi157.rcaris","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6825/3923812398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3923812398},{"type":"v1","addr":"172.21.15.157:6825","nonce":3923812398}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T23:32:20.146130+0000","modified":"2024-04-22T23:43:36.801717+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24277},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_24277":{"gid":24277,"name":"cephfs.smithi157.ufywse","rank":0,"incarnation":29,"state":"up:active","state_seq":169,"addr":"172.21.15.157:6827/4072444199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":4072444199},{"type":"v1","addr":"172.21.15.157:6827","nonce":4072444199}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24277,"qdb_cluster":[24277]},"id":1}]} 2024-04-22T23:47:26.424 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 34 2024-04-22T23:47:26.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.434+0000 7f5809b1d640 1 -- 172.21.15.107:0/1566424583 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f57f006eb80 msgr2=0x7f57f0071030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:26.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.434+0000 7f5809b1d640 1 --2- 172.21.15.107:0/1566424583 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f57f006eb80 0x7f57f0071030 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f57ec000c00 tx=0x7f57ec009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:26.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.434+0000 7f5809b1d640 1 -- 172.21.15.107:0/1566424583 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5804102280 msgr2=0x7f5804112d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:26.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.434+0000 7f5809b1d640 1 --2- 172.21.15.107:0/1566424583 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5804102280 0x7f5804112d60 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f57f400c830 tx=0x7f57f4004430 comp rx=0 tx=0).stop 2024-04-22T23:47:26.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.434+0000 7f5809b1d640 1 -- 172.21.15.107:0/1566424583 shutdown_connections 2024-04-22T23:47:26.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.434+0000 7f5809b1d640 1 --2- 172.21.15.107:0/1566424583 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f57f006eb80 0x7f57f0071030 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:26.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.434+0000 7f5809b1d640 1 --2- 172.21.15.107:0/1566424583 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5804102b90 0x7f58041132a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:26.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.434+0000 7f5809b1d640 1 --2- 172.21.15.107:0/1566424583 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5804102280 0x7f5804112d60 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:26.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.434+0000 7f5809b1d640 1 -- 172.21.15.107:0/1566424583 >> 172.21.15.107:0/1566424583 conn(0x7f58040708f0 msgr2=0x7f58040ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:26.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.434+0000 7f5809b1d640 1 -- 172.21.15.107:0/1566424583 shutdown_connections 2024-04-22T23:47:26.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.434+0000 7f5809b1d640 1 -- 172.21.15.107:0/1566424583 wait complete. 2024-04-22T23:47:26.437 DEBUG:tasks.fs:max_mds reduced in epoch 34 2024-04-22T23:47:26.437 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 35 2024-04-22T23:47:26.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.540+0000 7f4dce2ae640 1 -- 172.21.15.107:0/2061837088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4dc81008e0 msgr2=0x7f4dc8100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:26.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.540+0000 7f4dce2ae640 1 --2- 172.21.15.107:0/2061837088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4dc81008e0 0x7f4dc8100cb0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4db80049c0 tx=0x7f4db800aeb0 comp rx=0 tx=0).stop 2024-04-22T23:47:26.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.540+0000 7f4dce2ae640 1 -- 172.21.15.107:0/2061837088 shutdown_connections 2024-04-22T23:47:26.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.540+0000 7f4dce2ae640 1 --2- 172.21.15.107:0/2061837088 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4dc80fa8d0 0x7f4dc80fad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:26.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.540+0000 7f4dce2ae640 1 --2- 172.21.15.107:0/2061837088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4dc81008e0 0x7f4dc8100cb0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:26.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.540+0000 7f4dce2ae640 1 -- 172.21.15.107:0/2061837088 >> 172.21.15.107:0/2061837088 conn(0x7f4dc8070880 msgr2=0x7f4dc8070c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:26.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.540+0000 7f4dce2ae640 1 -- 172.21.15.107:0/2061837088 shutdown_connections 2024-04-22T23:47:26.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.540+0000 7f4dce2ae640 1 -- 172.21.15.107:0/2061837088 wait complete. 2024-04-22T23:47:26.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.541+0000 7f4dce2ae640 1 Processor -- start 2024-04-22T23:47:26.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.541+0000 7f4dce2ae640 1 -- start start 2024-04-22T23:47:26.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.541+0000 7f4dce2ae640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4dc80fa8d0 0x7f4dc8112d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:26.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.541+0000 7f4dce2ae640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4dc81008e0 0x7f4dc8113240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:26.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.541+0000 7f4dce2ae640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc8113840 con 0x7f4dc81008e0 2024-04-22T23:47:26.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.541+0000 7f4dce2ae640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc810ce90 con 0x7f4dc80fa8d0 2024-04-22T23:47:26.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.541+0000 7f4dc77fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4dc81008e0 0x7f4dc8113240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:26.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.541+0000 7f4dc77fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4dc81008e0 0x7f4dc8113240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38762/0 (socket says 172.21.15.107:38762) 2024-04-22T23:47:26.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.541+0000 7f4dc77fe640 1 -- 172.21.15.107:0/1890323628 learned_addr learned my addr 172.21.15.107:0/1890323628 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:47:26.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.542+0000 7f4dc7fff640 1 --2- 172.21.15.107:0/1890323628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4dc80fa8d0 0x7f4dc8112d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:26.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.542+0000 7f4dc77fe640 1 -- 172.21.15.107:0/1890323628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4dc80fa8d0 msgr2=0x7f4dc8112d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:26.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.542+0000 7f4dc77fe640 1 --2- 172.21.15.107:0/1890323628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4dc80fa8d0 0x7f4dc8112d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:26.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.542+0000 7f4dc77fe640 1 -- 172.21.15.107:0/1890323628 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4db8005040 con 0x7f4dc81008e0 2024-04-22T23:47:26.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.542+0000 7f4dc77fe640 1 --2- 172.21.15.107:0/1890323628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4dc81008e0 0x7f4dc8113240 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc002a50 tx=0x7f4dbc004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:26.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.542+0000 7f4dc4ff9640 1 -- 172.21.15.107:0/1890323628 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4dbc007c30 con 0x7f4dc81008e0 2024-04-22T23:47:26.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.542+0000 7f4dce2ae640 1 -- 172.21.15.107:0/1890323628 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dc810d170 con 0x7f4dc81008e0 2024-04-22T23:47:26.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.542+0000 7f4dc4ff9640 1 -- 172.21.15.107:0/1890323628 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4dbc00f040 con 0x7f4dc81008e0 2024-04-22T23:47:26.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.542+0000 7f4dc4ff9640 1 -- 172.21.15.107:0/1890323628 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4dbc00dbe0 con 0x7f4dc81008e0 2024-04-22T23:47:26.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.542+0000 7f4dce2ae640 1 -- 172.21.15.107:0/1890323628 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dc810d710 con 0x7f4dc81008e0 2024-04-22T23:47:26.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.543+0000 7f4dce2ae640 1 -- 172.21.15.107:0/1890323628 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dc80fbfc0 con 0x7f4dc81008e0 2024-04-22T23:47:26.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.543+0000 7f4dc4ff9640 1 -- 172.21.15.107:0/1890323628 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f4dbc00dd40 con 0x7f4dc81008e0 2024-04-22T23:47:26.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.544+0000 7f4dc4ff9640 1 --2- 172.21.15.107:0/1890323628 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f4db006eb70 0x7f4db0071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:26.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.544+0000 7f4dc4ff9640 1 -- 172.21.15.107:0/1890323628 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f4dbc08e910 con 0x7f4dc81008e0 2024-04-22T23:47:26.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.544+0000 7f4dc7fff640 1 --2- 172.21.15.107:0/1890323628 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f4db006eb70 0x7f4db0071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:26.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.544+0000 7f4dc7fff640 1 --2- 172.21.15.107:0/1890323628 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f4db006eb70 0x7f4db0071020 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4db800abd0 tx=0x7f4db800aeb0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:26.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.548+0000 7f4dc4ff9640 1 -- 172.21.15.107:0/1890323628 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f4dbc057610 con 0x7f4dc81008e0 2024-04-22T23:47:26.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.765+0000 7f4dce2ae640 1 -- 172.21.15.107:0/1890323628 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 35, "format": "json"} v 0) v1 -- 0x7f4dc804bcc0 con 0x7f4dc81008e0 2024-04-22T23:47:26.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.766+0000 7f4dc4ff9640 1 -- 172.21.15.107:0/1890323628 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 35, "format": "json"}]=0 dumped fsmap epoch 35 v43) v1 ==== 107+0+4278 (secure 0 0 0) 0x7f4dbc05acd0 con 0x7f4dc81008e0 2024-04-22T23:47:26.758 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:47:26.758 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34434,"name":"cephfs.smithi107.dfvvkw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6827/418145840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":418145840},{"type":"v1","addr":"172.21.15.107:6827","nonce":418145840}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":34438,"name":"cephfs.smithi107.ktuxjl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6829/1256497807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1256497807},{"type":"v1","addr":"172.21.15.107:6829","nonce":1256497807}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":44323,"name":"cephfs.smithi157.rcaris","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6825/3923812398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3923812398},{"type":"v1","addr":"172.21.15.157:6825","nonce":3923812398}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T23:32:20.146130+0000","modified":"2024-04-22T23:44:07.204703+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[1],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-22T23:47:26.758 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 35 2024-04-22T23:47:26.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.768+0000 7f4dce2ae640 1 -- 172.21.15.107:0/1890323628 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f4db006eb70 msgr2=0x7f4db0071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:26.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.768+0000 7f4dce2ae640 1 --2- 172.21.15.107:0/1890323628 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f4db006eb70 0x7f4db0071020 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4db800abd0 tx=0x7f4db800aeb0 comp rx=0 tx=0).stop 2024-04-22T23:47:26.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.768+0000 7f4dce2ae640 1 -- 172.21.15.107:0/1890323628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4dc81008e0 msgr2=0x7f4dc8113240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:26.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.768+0000 7f4dce2ae640 1 --2- 172.21.15.107:0/1890323628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4dc81008e0 0x7f4dc8113240 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc002a50 tx=0x7f4dbc004600 comp rx=0 tx=0).stop 2024-04-22T23:47:26.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.768+0000 7f4dce2ae640 1 -- 172.21.15.107:0/1890323628 shutdown_connections 2024-04-22T23:47:26.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.768+0000 7f4dce2ae640 1 --2- 172.21.15.107:0/1890323628 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f4db006eb70 0x7f4db0071020 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:26.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.768+0000 7f4dce2ae640 1 --2- 172.21.15.107:0/1890323628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4dc81008e0 0x7f4dc8113240 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:26.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.768+0000 7f4dce2ae640 1 --2- 172.21.15.107:0/1890323628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4dc80fa8d0 0x7f4dc8112d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:26.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.768+0000 7f4dce2ae640 1 -- 172.21.15.107:0/1890323628 >> 172.21.15.107:0/1890323628 conn(0x7f4dc8070880 msgr2=0x7f4dc81037f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:26.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.768+0000 7f4dce2ae640 1 -- 172.21.15.107:0/1890323628 shutdown_connections 2024-04-22T23:47:26.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.768+0000 7f4dce2ae640 1 -- 172.21.15.107:0/1890323628 wait complete. 2024-04-22T23:47:26.771 DEBUG:tasks.fs:max_mds reduced in epoch 35 2024-04-22T23:47:26.771 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 36 2024-04-22T23:47:26.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.874+0000 7fdfd4f9d640 1 -- 172.21.15.107:0/2070613636 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfd00fb120 msgr2=0x7fdfd00fb570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:26.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.874+0000 7fdfd4f9d640 1 --2- 172.21.15.107:0/2070613636 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfd00fb120 0x7fdfd00fb570 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdfc40048d0 tx=0x7fdfc4009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:26.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.874+0000 7fdfd4f9d640 1 -- 172.21.15.107:0/2070613636 shutdown_connections 2024-04-22T23:47:26.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.874+0000 7fdfd4f9d640 1 --2- 172.21.15.107:0/2070613636 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfd00fb120 0x7fdfd00fb570 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:26.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.874+0000 7fdfd4f9d640 1 --2- 172.21.15.107:0/2070613636 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfd0101150 0x7fdfd0101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:26.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.875+0000 7fdfd4f9d640 1 -- 172.21.15.107:0/2070613636 >> 172.21.15.107:0/2070613636 conn(0x7fdfd00710c0 msgr2=0x7fdfd00714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:26.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.875+0000 7fdfd4f9d640 1 -- 172.21.15.107:0/2070613636 shutdown_connections 2024-04-22T23:47:26.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.875+0000 7fdfd4f9d640 1 -- 172.21.15.107:0/2070613636 wait complete. 2024-04-22T23:47:26.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.875+0000 7fdfd4f9d640 1 Processor -- start 2024-04-22T23:47:26.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.875+0000 7fdfd4f9d640 1 -- start start 2024-04-22T23:47:26.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.875+0000 7fdfd4f9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfd00fb120 0x7fdfd010df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:26.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.875+0000 7fdfd4f9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfd0101150 0x7fdfd010e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:26.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.875+0000 7fdfd4f9d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd010e9e0 con 0x7fdfd00fb120 2024-04-22T23:47:26.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.875+0000 7fdfd4f9d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd010eb50 con 0x7fdfd0101150 2024-04-22T23:47:26.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.875+0000 7fdfce59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfd00fb120 0x7fdfd010df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:26.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.875+0000 7fdfce59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfd00fb120 0x7fdfd010df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38766/0 (socket says 172.21.15.107:38766) 2024-04-22T23:47:26.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.875+0000 7fdfce59c640 1 -- 172.21.15.107:0/2801163260 learned_addr learned my addr 172.21.15.107:0/2801163260 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:47:26.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.876+0000 7fdfce59c640 1 -- 172.21.15.107:0/2801163260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfd0101150 msgr2=0x7fdfd010e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:26.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.876+0000 7fdfce59c640 1 --2- 172.21.15.107:0/2801163260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfd0101150 0x7fdfd010e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:26.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.876+0000 7fdfce59c640 1 -- 172.21.15.107:0/2801163260 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc4005040 con 0x7fdfd00fb120 2024-04-22T23:47:26.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.876+0000 7fdfce59c640 1 --2- 172.21.15.107:0/2801163260 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfd00fb120 0x7fdfd010df60 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fdfc000b790 tx=0x7fdfc00091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:26.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.876+0000 7fdfbeffd640 1 -- 172.21.15.107:0/2801163260 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdfc000c700 con 0x7fdfd00fb120 2024-04-22T23:47:26.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.876+0000 7fdfbeffd640 1 -- 172.21.15.107:0/2801163260 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdfc000cd30 con 0x7fdfd00fb120 2024-04-22T23:47:26.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.876+0000 7fdfd4f9d640 1 -- 172.21.15.107:0/2801163260 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfd0116fd0 con 0x7fdfd00fb120 2024-04-22T23:47:26.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.876+0000 7fdfbeffd640 1 -- 172.21.15.107:0/2801163260 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdfc0013370 con 0x7fdfd00fb120 2024-04-22T23:47:26.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.876+0000 7fdfd4f9d640 1 -- 172.21.15.107:0/2801163260 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfd0117300 con 0x7fdfd00fb120 2024-04-22T23:47:26.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.877+0000 7fdfd4f9d640 1 -- 172.21.15.107:0/2801163260 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfd00fc810 con 0x7fdfd00fb120 2024-04-22T23:47:26.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.877+0000 7fdfbeffd640 1 -- 172.21.15.107:0/2801163260 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fdfc0012070 con 0x7fdfd00fb120 2024-04-22T23:47:26.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.878+0000 7fdfbeffd640 1 --2- 172.21.15.107:0/2801163260 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fdfb806eb80 0x7fdfb8071030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:26.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.878+0000 7fdfbeffd640 1 -- 172.21.15.107:0/2801163260 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fdfc008e4b0 con 0x7fdfd00fb120 2024-04-22T23:47:26.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.878+0000 7fdfcdd9b640 1 --2- 172.21.15.107:0/2801163260 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fdfb806eb80 0x7fdfb8071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:26.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.878+0000 7fdfcdd9b640 1 --2- 172.21.15.107:0/2801163260 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fdfb806eb80 0x7fdfb8071030 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4009a10 tx=0x7fdfc4009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:26.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:26.882+0000 7fdfbeffd640 1 -- 172.21.15.107:0/2801163260 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fdfc0057230 con 0x7fdfd00fb120 2024-04-22T23:47:27.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.100+0000 7fdfd4f9d640 1 -- 172.21.15.107:0/2801163260 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 36, "format": "json"} v 0) v1 -- 0x7fdfd010ed10 con 0x7fdfd00fb120 2024-04-22T23:47:27.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.101+0000 7fdfbeffd640 1 -- 172.21.15.107:0/2801163260 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 36, "format": "json"}]=0 dumped fsmap epoch 36 v43) v1 ==== 107+0+4357 (secure 0 0 0) 0x7fdfd010ed10 con 0x7fdfd00fb120 2024-04-22T23:47:27.092 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:47:27.093 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34438,"name":"cephfs.smithi107.ktuxjl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6829/1256497807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1256497807},{"type":"v1","addr":"172.21.15.107:6829","nonce":1256497807}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":44323,"name":"cephfs.smithi157.rcaris","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6825/3923812398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3923812398},{"type":"v1","addr":"172.21.15.157:6825","nonce":3923812398}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T23:32:20.146130+0000","modified":"2024-04-22T23:44:07.212403+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34434},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_34434":{"gid":34434,"name":"cephfs.smithi107.dfvvkw","rank":0,"incarnation":36,"state":"up:replay","state_seq":1,"addr":"172.21.15.107:6827/418145840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":418145840},{"type":"v1","addr":"172.21.15.107:6827","nonce":418145840}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-22T23:47:27.093 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 36 2024-04-22T23:47:27.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.103+0000 7fdfd4f9d640 1 -- 172.21.15.107:0/2801163260 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fdfb806eb80 msgr2=0x7fdfb8071030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:27.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.103+0000 7fdfd4f9d640 1 --2- 172.21.15.107:0/2801163260 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fdfb806eb80 0x7fdfb8071030 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4009a10 tx=0x7fdfc4009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:27.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.103+0000 7fdfd4f9d640 1 -- 172.21.15.107:0/2801163260 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfd00fb120 msgr2=0x7fdfd010df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:27.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.103+0000 7fdfd4f9d640 1 --2- 172.21.15.107:0/2801163260 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfd00fb120 0x7fdfd010df60 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fdfc000b790 tx=0x7fdfc00091d0 comp rx=0 tx=0).stop 2024-04-22T23:47:27.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.103+0000 7fdfd4f9d640 1 -- 172.21.15.107:0/2801163260 shutdown_connections 2024-04-22T23:47:27.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.103+0000 7fdfd4f9d640 1 --2- 172.21.15.107:0/2801163260 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fdfb806eb80 0x7fdfb8071030 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:27.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.103+0000 7fdfd4f9d640 1 --2- 172.21.15.107:0/2801163260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdfd0101150 0x7fdfd010e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:27.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.103+0000 7fdfd4f9d640 1 --2- 172.21.15.107:0/2801163260 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfd00fb120 0x7fdfd010df60 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:27.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.103+0000 7fdfd4f9d640 1 -- 172.21.15.107:0/2801163260 >> 172.21.15.107:0/2801163260 conn(0x7fdfd00710c0 msgr2=0x7fdfd00f9490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:27.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.103+0000 7fdfd4f9d640 1 -- 172.21.15.107:0/2801163260 shutdown_connections 2024-04-22T23:47:27.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.103+0000 7fdfd4f9d640 1 -- 172.21.15.107:0/2801163260 wait complete. 2024-04-22T23:47:27.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:26 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/827542657' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-04-22T23:47:27.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:26 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/1566424583' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-04-22T23:47:27.104 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:26 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/1890323628' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-04-22T23:47:27.105 DEBUG:tasks.fs:max_mds reduced in epoch 36 2024-04-22T23:47:27.106 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 37 2024-04-22T23:47:27.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:26 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/827542657' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-04-22T23:47:27.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:26 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/1566424583' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-04-22T23:47:27.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:26 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/1890323628' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-04-22T23:47:27.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.209+0000 7efe30d10640 1 -- 172.21.15.107:0/3836186136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe2c0fb120 msgr2=0x7efe2c0fb570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:27.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.209+0000 7efe30d10640 1 --2- 172.21.15.107:0/3836186136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe2c0fb120 0x7efe2c0fb570 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7efe140048d0 tx=0x7efe14009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:27.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.209+0000 7efe30d10640 1 -- 172.21.15.107:0/3836186136 shutdown_connections 2024-04-22T23:47:27.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.209+0000 7efe30d10640 1 --2- 172.21.15.107:0/3836186136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe2c0fb120 0x7efe2c0fb570 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:27.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.209+0000 7efe30d10640 1 --2- 172.21.15.107:0/3836186136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe2c101150 0x7efe2c101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:27.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.209+0000 7efe30d10640 1 -- 172.21.15.107:0/3836186136 >> 172.21.15.107:0/3836186136 conn(0x7efe2c0710c0 msgr2=0x7efe2c0714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:27.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.209+0000 7efe30d10640 1 -- 172.21.15.107:0/3836186136 shutdown_connections 2024-04-22T23:47:27.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.210+0000 7efe30d10640 1 -- 172.21.15.107:0/3836186136 wait complete. 2024-04-22T23:47:27.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.210+0000 7efe30d10640 1 Processor -- start 2024-04-22T23:47:27.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.210+0000 7efe30d10640 1 -- start start 2024-04-22T23:47:27.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.210+0000 7efe30d10640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe2c0fb120 0x7efe2c111c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:27.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.210+0000 7efe30d10640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe2c101150 0x7efe2c112150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:27.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.210+0000 7efe30d10640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2c112750 con 0x7efe2c101150 2024-04-22T23:47:27.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.210+0000 7efe30d10640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2c10bda0 con 0x7efe2c0fb120 2024-04-22T23:47:27.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.210+0000 7efe23fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe2c101150 0x7efe2c112150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:27.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.210+0000 7efe23fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe2c101150 0x7efe2c112150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38772/0 (socket says 172.21.15.107:38772) 2024-04-22T23:47:27.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.210+0000 7efe23fff640 1 -- 172.21.15.107:0/54940342 learned_addr learned my addr 172.21.15.107:0/54940342 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:47:27.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.211+0000 7efe23fff640 1 -- 172.21.15.107:0/54940342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe2c0fb120 msgr2=0x7efe2c111c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:27.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.211+0000 7efe23fff640 1 --2- 172.21.15.107:0/54940342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe2c0fb120 0x7efe2c111c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:27.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.211+0000 7efe23fff640 1 -- 172.21.15.107:0/54940342 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe14005040 con 0x7efe2c101150 2024-04-22T23:47:27.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.211+0000 7efe23fff640 1 --2- 172.21.15.107:0/54940342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe2c101150 0x7efe2c112150 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7efe140041d0 tx=0x7efe14009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:27.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.211+0000 7efe22ffd640 1 -- 172.21.15.107:0/54940342 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe14030850 con 0x7efe2c101150 2024-04-22T23:47:27.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.211+0000 7efe22ffd640 1 -- 172.21.15.107:0/54940342 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efe140092e0 con 0x7efe2c101150 2024-04-22T23:47:27.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.211+0000 7efe30d10640 1 -- 172.21.15.107:0/54940342 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe2c10c020 con 0x7efe2c101150 2024-04-22T23:47:27.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.211+0000 7efe22ffd640 1 -- 172.21.15.107:0/54940342 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe14043640 con 0x7efe2c101150 2024-04-22T23:47:27.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.211+0000 7efe30d10640 1 -- 172.21.15.107:0/54940342 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe2c10c540 con 0x7efe2c101150 2024-04-22T23:47:27.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.212+0000 7efe30d10640 1 -- 172.21.15.107:0/54940342 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe2c0fc810 con 0x7efe2c101150 2024-04-22T23:47:27.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.212+0000 7efe22ffd640 1 -- 172.21.15.107:0/54940342 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7efe14009450 con 0x7efe2c101150 2024-04-22T23:47:27.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.213+0000 7efe22ffd640 1 --2- 172.21.15.107:0/54940342 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7efe1806eb80 0x7efe18071030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:27.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.213+0000 7efe22ffd640 1 -- 172.21.15.107:0/54940342 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7efe140b15a0 con 0x7efe2c101150 2024-04-22T23:47:27.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.213+0000 7efe2a59c640 1 --2- 172.21.15.107:0/54940342 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7efe1806eb80 0x7efe18071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:27.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.213+0000 7efe2a59c640 1 --2- 172.21.15.107:0/54940342 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7efe1806eb80 0x7efe18071030 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7efe1c004d40 tx=0x7efe1c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:27.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.218+0000 7efe22ffd640 1 -- 172.21.15.107:0/54940342 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7efe1407a3e0 con 0x7efe2c101150 2024-04-22T23:47:27.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.431+0000 7efe30d10640 1 -- 172.21.15.107:0/54940342 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 37, "format": "json"} v 0) v1 -- 0x7efe2c04bcc0 con 0x7efe2c101150 2024-04-22T23:47:27.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.432+0000 7efe22ffd640 1 -- 172.21.15.107:0/54940342 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 37, "format": "json"}]=0 dumped fsmap epoch 37 v43) v1 ==== 107+0+4361 (secure 0 0 0) 0x7efe1407daa0 con 0x7efe2c101150 2024-04-22T23:47:27.424 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:47:27.424 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34438,"name":"cephfs.smithi107.ktuxjl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6829/1256497807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1256497807},{"type":"v1","addr":"172.21.15.107:6829","nonce":1256497807}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":44323,"name":"cephfs.smithi157.rcaris","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6825/3923812398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3923812398},{"type":"v1","addr":"172.21.15.157:6825","nonce":3923812398}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T23:32:20.146130+0000","modified":"2024-04-22T23:44:14.029604+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34434},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_34434":{"gid":34434,"name":"cephfs.smithi107.dfvvkw","rank":0,"incarnation":36,"state":"up:reconnect","state_seq":17,"addr":"172.21.15.107:6827/418145840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":418145840},{"type":"v1","addr":"172.21.15.107:6827","nonce":418145840}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-22T23:47:27.424 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 37 2024-04-22T23:47:27.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.434+0000 7efe30d10640 1 -- 172.21.15.107:0/54940342 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7efe1806eb80 msgr2=0x7efe18071030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:27.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.434+0000 7efe30d10640 1 --2- 172.21.15.107:0/54940342 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7efe1806eb80 0x7efe18071030 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7efe1c004d40 tx=0x7efe1c00b040 comp rx=0 tx=0).stop 2024-04-22T23:47:27.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.434+0000 7efe30d10640 1 -- 172.21.15.107:0/54940342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe2c101150 msgr2=0x7efe2c112150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:27.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.434+0000 7efe30d10640 1 --2- 172.21.15.107:0/54940342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe2c101150 0x7efe2c112150 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7efe140041d0 tx=0x7efe14009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:27.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.434+0000 7efe30d10640 1 -- 172.21.15.107:0/54940342 shutdown_connections 2024-04-22T23:47:27.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.434+0000 7efe30d10640 1 --2- 172.21.15.107:0/54940342 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7efe1806eb80 0x7efe18071030 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:27.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.434+0000 7efe30d10640 1 --2- 172.21.15.107:0/54940342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe2c101150 0x7efe2c112150 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:27.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.434+0000 7efe30d10640 1 --2- 172.21.15.107:0/54940342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efe2c0fb120 0x7efe2c111c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:27.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.434+0000 7efe30d10640 1 -- 172.21.15.107:0/54940342 >> 172.21.15.107:0/54940342 conn(0x7efe2c0710c0 msgr2=0x7efe2c0f9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:27.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.434+0000 7efe30d10640 1 -- 172.21.15.107:0/54940342 shutdown_connections 2024-04-22T23:47:27.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.434+0000 7efe30d10640 1 -- 172.21.15.107:0/54940342 wait complete. 2024-04-22T23:47:27.436 DEBUG:tasks.fs:max_mds reduced in epoch 37 2024-04-22T23:47:27.437 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 38 2024-04-22T23:47:27.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.539+0000 7f5f0e814640 1 -- 172.21.15.107:0/2504413340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f08102250 msgr2=0x7f5f08102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:27.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.539+0000 7f5f0e814640 1 --2- 172.21.15.107:0/2504413340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f08102250 0x7f5f08102620 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5ef80048d0 tx=0x7f5ef8009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:27.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.539+0000 7f5f0e814640 1 -- 172.21.15.107:0/2504413340 shutdown_connections 2024-04-22T23:47:27.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.539+0000 7f5f0e814640 1 --2- 172.21.15.107:0/2504413340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f080643b0 0x7f5f080647e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:27.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.539+0000 7f5f0e814640 1 --2- 172.21.15.107:0/2504413340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f08102250 0x7f5f08102620 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:27.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.539+0000 7f5f0e814640 1 -- 172.21.15.107:0/2504413340 >> 172.21.15.107:0/2504413340 conn(0x7f5f080710c0 msgr2=0x7f5f080714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:27.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.539+0000 7f5f0e814640 1 -- 172.21.15.107:0/2504413340 shutdown_connections 2024-04-22T23:47:27.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.539+0000 7f5f0e814640 1 -- 172.21.15.107:0/2504413340 wait complete. 2024-04-22T23:47:27.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.540+0000 7f5f0e814640 1 Processor -- start 2024-04-22T23:47:27.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.540+0000 7f5f0e814640 1 -- start start 2024-04-22T23:47:27.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.540+0000 7f5f0e814640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f080643b0 0x7f5f08112d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:27.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.540+0000 7f5f0e814640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f08102250 0x7f5f08113270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:27.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.540+0000 7f5f0e814640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f08113900 con 0x7f5f08102250 2024-04-22T23:47:27.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.540+0000 7f5f0e814640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0810cec0 con 0x7f5f080643b0 2024-04-22T23:47:27.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.540+0000 7f5f077fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f08102250 0x7f5f08113270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:27.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.540+0000 7f5f077fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f08102250 0x7f5f08113270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38790/0 (socket says 172.21.15.107:38790) 2024-04-22T23:47:27.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.540+0000 7f5f077fe640 1 -- 172.21.15.107:0/2496044525 learned_addr learned my addr 172.21.15.107:0/2496044525 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:47:27.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.541+0000 7f5f077fe640 1 -- 172.21.15.107:0/2496044525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f080643b0 msgr2=0x7f5f08112d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:27.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.541+0000 7f5f07fff640 1 --2- 172.21.15.107:0/2496044525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f080643b0 0x7f5f08112d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:27.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.541+0000 7f5f077fe640 1 --2- 172.21.15.107:0/2496044525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f080643b0 0x7f5f08112d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:27.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.541+0000 7f5f077fe640 1 -- 172.21.15.107:0/2496044525 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef8005040 con 0x7f5f08102250 2024-04-22T23:47:27.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.541+0000 7f5f077fe640 1 --2- 172.21.15.107:0/2496044525 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f08102250 0x7f5f08113270 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5efc00b790 tx=0x7f5efc0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:27.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.541+0000 7f5f04ff9640 1 -- 172.21.15.107:0/2496044525 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5efc00c700 con 0x7f5f08102250 2024-04-22T23:47:27.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.541+0000 7f5f04ff9640 1 -- 172.21.15.107:0/2496044525 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5efc00cd30 con 0x7f5f08102250 2024-04-22T23:47:27.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.541+0000 7f5f0e814640 1 -- 172.21.15.107:0/2496044525 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f0810d1a0 con 0x7f5f08102250 2024-04-22T23:47:27.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.541+0000 7f5f04ff9640 1 -- 172.21.15.107:0/2496044525 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5efc013370 con 0x7f5f08102250 2024-04-22T23:47:27.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.541+0000 7f5f0e814640 1 -- 172.21.15.107:0/2496044525 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f0810d6c0 con 0x7f5f08102250 2024-04-22T23:47:27.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.542+0000 7f5f0e814640 1 -- 172.21.15.107:0/2496044525 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f080fd950 con 0x7f5f08102250 2024-04-22T23:47:27.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.542+0000 7f5f04ff9640 1 -- 172.21.15.107:0/2496044525 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f5efc012070 con 0x7f5f08102250 2024-04-22T23:47:27.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.543+0000 7f5f04ff9640 1 --2- 172.21.15.107:0/2496044525 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f5ef006eb70 0x7f5ef0071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:27.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.543+0000 7f5f04ff9640 1 -- 172.21.15.107:0/2496044525 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f5efc08e5a0 con 0x7f5f08102250 2024-04-22T23:47:27.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.543+0000 7f5f07fff640 1 --2- 172.21.15.107:0/2496044525 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f5ef006eb70 0x7f5ef0071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:27.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.543+0000 7f5f07fff640 1 --2- 172.21.15.107:0/2496044525 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f5ef006eb70 0x7f5ef0071020 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8003080 tx=0x7f5ef8009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:27.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.548+0000 7f5f04ff9640 1 -- 172.21.15.107:0/2496044525 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f5efc0572a0 con 0x7f5f08102250 2024-04-22T23:47:27.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.764+0000 7f5f0e814640 1 -- 172.21.15.107:0/2496044525 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 38, "format": "json"} v 0) v1 -- 0x7f5f0804bcc0 con 0x7f5f08102250 2024-04-22T23:47:27.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.765+0000 7f5f04ff9640 1 -- 172.21.15.107:0/2496044525 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 38, "format": "json"}]=0 dumped fsmap epoch 38 v43) v1 ==== 107+0+4358 (secure 0 0 0) 0x7f5efc05a960 con 0x7f5f08102250 2024-04-22T23:47:27.757 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:47:27.757 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34438,"name":"cephfs.smithi107.ktuxjl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6829/1256497807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1256497807},{"type":"v1","addr":"172.21.15.107:6829","nonce":1256497807}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":44323,"name":"cephfs.smithi157.rcaris","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6825/3923812398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3923812398},{"type":"v1","addr":"172.21.15.157:6825","nonce":3923812398}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T23:32:20.146130+0000","modified":"2024-04-22T23:44:14.808440+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34434},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_34434":{"gid":34434,"name":"cephfs.smithi107.dfvvkw","rank":0,"incarnation":36,"state":"up:rejoin","state_seq":18,"addr":"172.21.15.107:6827/418145840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":418145840},{"type":"v1","addr":"172.21.15.107:6827","nonce":418145840}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-22T23:47:27.757 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 38 2024-04-22T23:47:27.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.767+0000 7f5f0e814640 1 -- 172.21.15.107:0/2496044525 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f5ef006eb70 msgr2=0x7f5ef0071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:27.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.767+0000 7f5f0e814640 1 --2- 172.21.15.107:0/2496044525 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f5ef006eb70 0x7f5ef0071020 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8003080 tx=0x7f5ef8009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:27.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.767+0000 7f5f0e814640 1 -- 172.21.15.107:0/2496044525 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f08102250 msgr2=0x7f5f08113270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:27.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.767+0000 7f5f0e814640 1 --2- 172.21.15.107:0/2496044525 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f08102250 0x7f5f08113270 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5efc00b790 tx=0x7f5efc0091d0 comp rx=0 tx=0).stop 2024-04-22T23:47:27.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.767+0000 7f5f0e814640 1 -- 172.21.15.107:0/2496044525 shutdown_connections 2024-04-22T23:47:27.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.767+0000 7f5f0e814640 1 --2- 172.21.15.107:0/2496044525 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f5ef006eb70 0x7f5ef0071020 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:27.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.767+0000 7f5f0e814640 1 --2- 172.21.15.107:0/2496044525 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f08102250 0x7f5f08113270 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:27.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.767+0000 7f5f0e814640 1 --2- 172.21.15.107:0/2496044525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f080643b0 0x7f5f08112d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:27.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.767+0000 7f5f0e814640 1 -- 172.21.15.107:0/2496044525 >> 172.21.15.107:0/2496044525 conn(0x7f5f080710c0 msgr2=0x7f5f080f9510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:27.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.767+0000 7f5f0e814640 1 -- 172.21.15.107:0/2496044525 shutdown_connections 2024-04-22T23:47:27.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.767+0000 7f5f0e814640 1 -- 172.21.15.107:0/2496044525 wait complete. 2024-04-22T23:47:27.769 DEBUG:tasks.fs:max_mds reduced in epoch 38 2024-04-22T23:47:27.770 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 39 2024-04-22T23:47:27.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.873+0000 7f02dcc6e640 1 -- 172.21.15.107:0/2614011575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02d81019f0 msgr2=0x7f02d80fb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:27.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.873+0000 7f02dcc6e640 1 --2- 172.21.15.107:0/2614011575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02d81019f0 0x7f02d80fb080 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f02c00048d0 tx=0x7f02c0009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:27.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.873+0000 7f02dcc6e640 1 -- 172.21.15.107:0/2614011575 shutdown_connections 2024-04-22T23:47:27.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.873+0000 7f02dcc6e640 1 --2- 172.21.15.107:0/2614011575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02d81019f0 0x7f02d80fb080 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:27.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.873+0000 7f02dcc6e640 1 --2- 172.21.15.107:0/2614011575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02d8101050 0x7f02d8101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:27.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.873+0000 7f02dcc6e640 1 -- 172.21.15.107:0/2614011575 >> 172.21.15.107:0/2614011575 conn(0x7f02d80f6c20 msgr2=0x7f02d80f9030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:27.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.873+0000 7f02dcc6e640 1 -- 172.21.15.107:0/2614011575 shutdown_connections 2024-04-22T23:47:27.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.873+0000 7f02dcc6e640 1 -- 172.21.15.107:0/2614011575 wait complete. 2024-04-22T23:47:27.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.874+0000 7f02dcc6e640 1 Processor -- start 2024-04-22T23:47:27.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.874+0000 7f02dcc6e640 1 -- start start 2024-04-22T23:47:27.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.874+0000 7f02dcc6e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02d8101050 0x7f02d8112c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:27.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.874+0000 7f02dcc6e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02d81019f0 0x7f02d81131c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:27.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.874+0000 7f02dcc6e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d81137c0 con 0x7f02d81019f0 2024-04-22T23:47:27.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.874+0000 7f02dcc6e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d810ce10 con 0x7f02d8101050 2024-04-22T23:47:27.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.874+0000 7f02cffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02d81019f0 0x7f02d81131c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:27.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.874+0000 7f02cffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02d81019f0 0x7f02d81131c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38818/0 (socket says 172.21.15.107:38818) 2024-04-22T23:47:27.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.874+0000 7f02cffff640 1 -- 172.21.15.107:0/3803079203 learned_addr learned my addr 172.21.15.107:0/3803079203 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:47:27.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.874+0000 7f02d659c640 1 --2- 172.21.15.107:0/3803079203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02d8101050 0x7f02d8112c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:27.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.874+0000 7f02cffff640 1 -- 172.21.15.107:0/3803079203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02d8101050 msgr2=0x7f02d8112c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:27.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.874+0000 7f02cffff640 1 --2- 172.21.15.107:0/3803079203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02d8101050 0x7f02d8112c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:27.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.874+0000 7f02cffff640 1 -- 172.21.15.107:0/3803079203 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02c0005040 con 0x7f02d81019f0 2024-04-22T23:47:27.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.874+0000 7f02cffff640 1 --2- 172.21.15.107:0/3803079203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02d81019f0 0x7f02d81131c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f02c0004930 tx=0x7f02c0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:27.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.875+0000 7f02ceffd640 1 -- 172.21.15.107:0/3803079203 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02c00094c0 con 0x7f02d81019f0 2024-04-22T23:47:27.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.875+0000 7f02dcc6e640 1 -- 172.21.15.107:0/3803079203 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02d810d090 con 0x7f02d81019f0 2024-04-22T23:47:27.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.875+0000 7f02ceffd640 1 -- 172.21.15.107:0/3803079203 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f02c0032e50 con 0x7f02d81019f0 2024-04-22T23:47:27.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.875+0000 7f02ceffd640 1 -- 172.21.15.107:0/3803079203 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02c00433b0 con 0x7f02d81019f0 2024-04-22T23:47:27.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.875+0000 7f02dcc6e640 1 -- 172.21.15.107:0/3803079203 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02d810d550 con 0x7f02d81019f0 2024-04-22T23:47:27.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.876+0000 7f02dcc6e640 1 -- 172.21.15.107:0/3803079203 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02d80fc7c0 con 0x7f02d81019f0 2024-04-22T23:47:27.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.876+0000 7f02ceffd640 1 -- 172.21.15.107:0/3803079203 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f02c002f070 con 0x7f02d81019f0 2024-04-22T23:47:27.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.876+0000 7f02ceffd640 1 --2- 172.21.15.107:0/3803079203 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f02c406eb70 0x7f02c4071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:27.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.877+0000 7f02ceffd640 1 -- 172.21.15.107:0/3803079203 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f02c00b0960 con 0x7f02d81019f0 2024-04-22T23:47:27.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.877+0000 7f02d659c640 1 --2- 172.21.15.107:0/3803079203 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f02c406eb70 0x7f02c4071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:27.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.877+0000 7f02d659c640 1 --2- 172.21.15.107:0/3803079203 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f02c406eb70 0x7f02c4071020 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f02c8004c40 tx=0x7f02c8009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:27.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:27.881+0000 7f02ceffd640 1 -- 172.21.15.107:0/3803079203 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f02c0079660 con 0x7f02d81019f0 2024-04-22T23:47:28.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.099+0000 7f02dcc6e640 1 -- 172.21.15.107:0/3803079203 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 39, "format": "json"} v 0) v1 -- 0x7f02d8108190 con 0x7f02d81019f0 2024-04-22T23:47:28.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.099+0000 7f02ceffd640 1 -- 172.21.15.107:0/3803079203 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 39, "format": "json"}]=0 dumped fsmap epoch 39 v43) v1 ==== 107+0+4367 (secure 0 0 0) 0x7f02c007cd20 con 0x7f02d81019f0 2024-04-22T23:47:28.091 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:47:28.091 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34438,"name":"cephfs.smithi107.ktuxjl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6829/1256497807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1256497807},{"type":"v1","addr":"172.21.15.107:6829","nonce":1256497807}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":44323,"name":"cephfs.smithi157.rcaris","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6825/3923812398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3923812398},{"type":"v1","addr":"172.21.15.157:6825","nonce":3923812398}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T23:32:20.146130+0000","modified":"2024-04-22T23:44:16.815341+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34434},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_34434":{"gid":34434,"name":"cephfs.smithi107.dfvvkw","rank":0,"incarnation":36,"state":"up:active","state_seq":19,"addr":"172.21.15.107:6827/418145840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":418145840},{"type":"v1","addr":"172.21.15.107:6827","nonce":418145840}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34434,"qdb_cluster":[34434]},"id":1}]} 2024-04-22T23:47:28.091 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 39 2024-04-22T23:47:28.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.102+0000 7f02dcc6e640 1 -- 172.21.15.107:0/3803079203 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f02c406eb70 msgr2=0x7f02c4071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:28.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.102+0000 7f02dcc6e640 1 --2- 172.21.15.107:0/3803079203 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f02c406eb70 0x7f02c4071020 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f02c8004c40 tx=0x7f02c8009040 comp rx=0 tx=0).stop 2024-04-22T23:47:28.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.102+0000 7f02dcc6e640 1 -- 172.21.15.107:0/3803079203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02d81019f0 msgr2=0x7f02d81131c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:28.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.102+0000 7f02dcc6e640 1 --2- 172.21.15.107:0/3803079203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02d81019f0 0x7f02d81131c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f02c0004930 tx=0x7f02c0009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:28.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.102+0000 7f02dcc6e640 1 -- 172.21.15.107:0/3803079203 shutdown_connections 2024-04-22T23:47:28.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.102+0000 7f02dcc6e640 1 --2- 172.21.15.107:0/3803079203 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f02c406eb70 0x7f02c4071020 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:28.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.102+0000 7f02dcc6e640 1 --2- 172.21.15.107:0/3803079203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02d81019f0 0x7f02d81131c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:28.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.102+0000 7f02dcc6e640 1 --2- 172.21.15.107:0/3803079203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02d8101050 0x7f02d8112c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:28.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.102+0000 7f02dcc6e640 1 -- 172.21.15.107:0/3803079203 >> 172.21.15.107:0/3803079203 conn(0x7f02d80f6c20 msgr2=0x7f02d80f7b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:28.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.102+0000 7f02dcc6e640 1 -- 172.21.15.107:0/3803079203 shutdown_connections 2024-04-22T23:47:28.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.102+0000 7f02dcc6e640 1 -- 172.21.15.107:0/3803079203 wait complete. 2024-04-22T23:47:28.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:27 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/2801163260' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-04-22T23:47:28.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:27 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/54940342' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-04-22T23:47:28.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:27 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/2496044525' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 38, "format": "json"}]: dispatch 2024-04-22T23:47:28.103 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:27 smithi107 ceph-mon[90179]: pgmap v323: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:47:28.104 DEBUG:tasks.fs:max_mds reduced in epoch 39 2024-04-22T23:47:28.105 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 40 2024-04-22T23:47:28.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:27 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/2801163260' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-04-22T23:47:28.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:27 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/54940342' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-04-22T23:47:28.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:27 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/2496044525' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 38, "format": "json"}]: dispatch 2024-04-22T23:47:28.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:27 smithi157 ceph-mon[79403]: pgmap v323: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:47:28.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.207+0000 7f2b4055d640 1 -- 172.21.15.107:0/370513728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b38101aa0 msgr2=0x7f2b380fb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:28.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.207+0000 7f2b4055d640 1 --2- 172.21.15.107:0/370513728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b38101aa0 0x7f2b380fb130 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2b280048d0 tx=0x7f2b28009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:28.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.207+0000 7f2b4055d640 1 -- 172.21.15.107:0/370513728 shutdown_connections 2024-04-22T23:47:28.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.207+0000 7f2b4055d640 1 --2- 172.21.15.107:0/370513728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b38101aa0 0x7f2b380fb130 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:28.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.207+0000 7f2b4055d640 1 --2- 172.21.15.107:0/370513728 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b38101100 0x7f2b381014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:28.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.207+0000 7f2b4055d640 1 -- 172.21.15.107:0/370513728 >> 172.21.15.107:0/370513728 conn(0x7f2b380f6cd0 msgr2=0x7f2b380f90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:28.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.207+0000 7f2b4055d640 1 -- 172.21.15.107:0/370513728 shutdown_connections 2024-04-22T23:47:28.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.207+0000 7f2b4055d640 1 -- 172.21.15.107:0/370513728 wait complete. 2024-04-22T23:47:28.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.208+0000 7f2b4055d640 1 Processor -- start 2024-04-22T23:47:28.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.208+0000 7f2b4055d640 1 -- start start 2024-04-22T23:47:28.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.208+0000 7f2b4055d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b38101100 0x7f2b38111c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:28.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.208+0000 7f2b4055d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b38101aa0 0x7f2b38112170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:28.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.208+0000 7f2b4055d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b38112770 con 0x7f2b38101aa0 2024-04-22T23:47:28.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.208+0000 7f2b4055d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3810bdc0 con 0x7f2b38101100 2024-04-22T23:47:28.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.208+0000 7f2b3daf8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b38101aa0 0x7f2b38112170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:28.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.208+0000 7f2b3daf8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b38101aa0 0x7f2b38112170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38834/0 (socket says 172.21.15.107:38834) 2024-04-22T23:47:28.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.208+0000 7f2b3daf8640 1 -- 172.21.15.107:0/2672862082 learned_addr learned my addr 172.21.15.107:0/2672862082 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:47:28.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.208+0000 7f2b3e2f9640 1 --2- 172.21.15.107:0/2672862082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b38101100 0x7f2b38111c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:28.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.208+0000 7f2b3daf8640 1 -- 172.21.15.107:0/2672862082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b38101100 msgr2=0x7f2b38111c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:28.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.208+0000 7f2b3daf8640 1 --2- 172.21.15.107:0/2672862082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b38101100 0x7f2b38111c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:28.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.208+0000 7f2b3daf8640 1 -- 172.21.15.107:0/2672862082 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b28005040 con 0x7f2b38101aa0 2024-04-22T23:47:28.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.208+0000 7f2b3daf8640 1 --2- 172.21.15.107:0/2672862082 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b38101aa0 0x7f2b38112170 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2b28004930 tx=0x7f2b28009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:28.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.208+0000 7f2b2effd640 1 -- 172.21.15.107:0/2672862082 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b280094c0 con 0x7f2b38101aa0 2024-04-22T23:47:28.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.208+0000 7f2b4055d640 1 -- 172.21.15.107:0/2672862082 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b3810c040 con 0x7f2b38101aa0 2024-04-22T23:47:28.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.209+0000 7f2b2effd640 1 -- 172.21.15.107:0/2672862082 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2b28032e50 con 0x7f2b38101aa0 2024-04-22T23:47:28.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.209+0000 7f2b4055d640 1 -- 172.21.15.107:0/2672862082 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b3810c500 con 0x7f2b38101aa0 2024-04-22T23:47:28.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.209+0000 7f2b2effd640 1 -- 172.21.15.107:0/2672862082 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b28044450 con 0x7f2b38101aa0 2024-04-22T23:47:28.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.210+0000 7f2b4055d640 1 -- 172.21.15.107:0/2672862082 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b380fc870 con 0x7f2b38101aa0 2024-04-22T23:47:28.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.210+0000 7f2b2effd640 1 -- 172.21.15.107:0/2672862082 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f2b2802f070 con 0x7f2b38101aa0 2024-04-22T23:47:28.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.210+0000 7f2b2effd640 1 --2- 172.21.15.107:0/2672862082 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f2b2406eb70 0x7f2b24071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:28.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.211+0000 7f2b2effd640 1 -- 172.21.15.107:0/2672862082 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f2b280b0720 con 0x7f2b38101aa0 2024-04-22T23:47:28.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.211+0000 7f2b3e2f9640 1 --2- 172.21.15.107:0/2672862082 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f2b2406eb70 0x7f2b24071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:28.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.211+0000 7f2b3e2f9640 1 --2- 172.21.15.107:0/2672862082 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f2b2406eb70 0x7f2b24071020 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2b34004eb0 tx=0x7f2b3400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:28.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.215+0000 7f2b2effd640 1 -- 172.21.15.107:0/2672862082 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f2b280794d0 con 0x7f2b38101aa0 2024-04-22T23:47:28.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.432+0000 7f2b4055d640 1 -- 172.21.15.107:0/2672862082 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 40, "format": "json"} v 0) v1 -- 0x7f2b38108240 con 0x7f2b38101aa0 2024-04-22T23:47:28.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.432+0000 7f2b2effd640 1 -- 172.21.15.107:0/2672862082 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 40, "format": "json"}]=0 dumped fsmap epoch 40 v43) v1 ==== 107+0+5217 (secure 0 0 0) 0x7f2b2807cb90 con 0x7f2b38101aa0 2024-04-22T23:47:28.425 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:47:28.425 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34438,"name":"cephfs.smithi107.ktuxjl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6829/1256497807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1256497807},{"type":"v1","addr":"172.21.15.107:6829","nonce":1256497807}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":44323,"name":"cephfs.smithi157.rcaris","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6825/3923812398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3923812398},{"type":"v1","addr":"172.21.15.157:6825","nonce":3923812398}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":44349,"name":"cephfs.smithi157.ufywse","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6827/3167053094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":3167053094},{"type":"v1","addr":"172.21.15.157:6827","nonce":3167053094}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T23:32:20.146130+0000","modified":"2024-04-22T23:44:16.815341+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34434},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_34434":{"gid":34434,"name":"cephfs.smithi107.dfvvkw","rank":0,"incarnation":36,"state":"up:active","state_seq":19,"addr":"172.21.15.107:6827/418145840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":418145840},{"type":"v1","addr":"172.21.15.107:6827","nonce":418145840}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34434,"qdb_cluster":[34434]},"id":1}]} 2024-04-22T23:47:28.425 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 40 2024-04-22T23:47:28.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.435+0000 7f2b4055d640 1 -- 172.21.15.107:0/2672862082 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f2b2406eb70 msgr2=0x7f2b24071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:28.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.435+0000 7f2b4055d640 1 --2- 172.21.15.107:0/2672862082 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f2b2406eb70 0x7f2b24071020 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2b34004eb0 tx=0x7f2b3400b040 comp rx=0 tx=0).stop 2024-04-22T23:47:28.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.435+0000 7f2b4055d640 1 -- 172.21.15.107:0/2672862082 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b38101aa0 msgr2=0x7f2b38112170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:28.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.435+0000 7f2b4055d640 1 --2- 172.21.15.107:0/2672862082 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b38101aa0 0x7f2b38112170 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2b28004930 tx=0x7f2b28009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:28.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.435+0000 7f2b4055d640 1 -- 172.21.15.107:0/2672862082 shutdown_connections 2024-04-22T23:47:28.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.435+0000 7f2b4055d640 1 --2- 172.21.15.107:0/2672862082 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f2b2406eb70 0x7f2b24071020 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:28.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.435+0000 7f2b4055d640 1 --2- 172.21.15.107:0/2672862082 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b38101aa0 0x7f2b38112170 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:28.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.435+0000 7f2b4055d640 1 --2- 172.21.15.107:0/2672862082 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b38101100 0x7f2b38111c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:28.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.435+0000 7f2b4055d640 1 -- 172.21.15.107:0/2672862082 >> 172.21.15.107:0/2672862082 conn(0x7f2b380f6cd0 msgr2=0x7f2b380f7ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:28.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.435+0000 7f2b4055d640 1 -- 172.21.15.107:0/2672862082 shutdown_connections 2024-04-22T23:47:28.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.435+0000 7f2b4055d640 1 -- 172.21.15.107:0/2672862082 wait complete. 2024-04-22T23:47:28.437 DEBUG:tasks.fs:max_mds reduced in epoch 40 2024-04-22T23:47:28.437 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 41 2024-04-22T23:47:28.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.540+0000 7f270646e640 1 -- 172.21.15.107:0/485906159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27001023f0 msgr2=0x7f27001027c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:28.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.540+0000 7f270646e640 1 --2- 172.21.15.107:0/485906159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27001023f0 0x7f27001027c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f26f00048d0 tx=0x7f26f0009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:28.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.540+0000 7f270646e640 1 -- 172.21.15.107:0/485906159 shutdown_connections 2024-04-22T23:47:28.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.540+0000 7f270646e640 1 --2- 172.21.15.107:0/485906159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2700102d90 0x7f27000fc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:28.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.540+0000 7f270646e640 1 --2- 172.21.15.107:0/485906159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27001023f0 0x7f27001027c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:28.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.540+0000 7f270646e640 1 -- 172.21.15.107:0/485906159 >> 172.21.15.107:0/485906159 conn(0x7f27000f7fe0 msgr2=0x7f27000fa3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:28.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.540+0000 7f270646e640 1 -- 172.21.15.107:0/485906159 shutdown_connections 2024-04-22T23:47:28.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.540+0000 7f270646e640 1 -- 172.21.15.107:0/485906159 wait complete. 2024-04-22T23:47:28.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.541+0000 7f270646e640 1 Processor -- start 2024-04-22T23:47:28.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.541+0000 7f270646e640 1 -- start start 2024-04-22T23:47:28.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.541+0000 7f270646e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27001023f0 0x7f2700112ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:28.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.541+0000 7f270646e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2700102d90 0x7f2700113420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:28.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.541+0000 7f270646e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2700113a20 con 0x7f2700102d90 2024-04-22T23:47:28.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.541+0000 7f270646e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f270010d070 con 0x7f27001023f0 2024-04-22T23:47:28.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.541+0000 7f26ff7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2700102d90 0x7f2700113420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:28.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.541+0000 7f26ff7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2700102d90 0x7f2700113420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38852/0 (socket says 172.21.15.107:38852) 2024-04-22T23:47:28.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.541+0000 7f26ff7fe640 1 -- 172.21.15.107:0/106771672 learned_addr learned my addr 172.21.15.107:0/106771672 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:47:28.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.541+0000 7f26fffff640 1 --2- 172.21.15.107:0/106771672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27001023f0 0x7f2700112ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:28.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.541+0000 7f26ff7fe640 1 -- 172.21.15.107:0/106771672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27001023f0 msgr2=0x7f2700112ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:28.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.541+0000 7f26ff7fe640 1 --2- 172.21.15.107:0/106771672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27001023f0 0x7f2700112ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:28.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.541+0000 7f26ff7fe640 1 -- 172.21.15.107:0/106771672 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f0005040 con 0x7f2700102d90 2024-04-22T23:47:28.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.541+0000 7f26fffff640 1 --2- 172.21.15.107:0/106771672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27001023f0 0x7f2700112ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T23:47:28.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.542+0000 7f26ff7fe640 1 --2- 172.21.15.107:0/106771672 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2700102d90 0x7f2700113420 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f26f4002a80 tx=0x7f26f4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:28.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.542+0000 7f26fcff9640 1 -- 172.21.15.107:0/106771672 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26f4007c30 con 0x7f2700102d90 2024-04-22T23:47:28.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.542+0000 7f270646e640 1 -- 172.21.15.107:0/106771672 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f270010d350 con 0x7f2700102d90 2024-04-22T23:47:28.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.542+0000 7f26fcff9640 1 -- 172.21.15.107:0/106771672 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f26f400f040 con 0x7f2700102d90 2024-04-22T23:47:28.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.542+0000 7f26fcff9640 1 -- 172.21.15.107:0/106771672 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26f40096b0 con 0x7f2700102d90 2024-04-22T23:47:28.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.542+0000 7f270646e640 1 -- 172.21.15.107:0/106771672 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f270010d870 con 0x7f2700102d90 2024-04-22T23:47:28.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.543+0000 7f270646e640 1 -- 172.21.15.107:0/106771672 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27000fdb60 con 0x7f2700102d90 2024-04-22T23:47:28.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.543+0000 7f26fcff9640 1 -- 172.21.15.107:0/106771672 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f26f400d6b0 con 0x7f2700102d90 2024-04-22T23:47:28.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.544+0000 7f26fcff9640 1 --2- 172.21.15.107:0/106771672 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f26e806eb80 0x7f26e8071030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:28.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.544+0000 7f26fcff9640 1 -- 172.21.15.107:0/106771672 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f26f408e370 con 0x7f2700102d90 2024-04-22T23:47:28.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.544+0000 7f26fffff640 1 --2- 172.21.15.107:0/106771672 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f26e806eb80 0x7f26e8071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:28.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.544+0000 7f26fffff640 1 --2- 172.21.15.107:0/106771672 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f26e806eb80 0x7f26e8071030 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f26f0000c00 tx=0x7f26f0009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:28.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.548+0000 7f26fcff9640 1 -- 172.21.15.107:0/106771672 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f26f4057070 con 0x7f2700102d90 2024-04-22T23:47:28.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.762+0000 7f270646e640 1 -- 172.21.15.107:0/106771672 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 41, "format": "json"} v 0) v1 -- 0x7f270004bcc0 con 0x7f2700102d90 2024-04-22T23:47:28.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.763+0000 7f26fcff9640 1 -- 172.21.15.107:0/106771672 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 41, "format": "json"}]=0 dumped fsmap epoch 41 v43) v1 ==== 107+0+5217 (secure 0 0 0) 0x7f26f405a730 con 0x7f2700102d90 2024-04-22T23:47:28.755 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:47:28.755 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34438,"name":"cephfs.smithi107.ktuxjl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6829/1256497807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1256497807},{"type":"v1","addr":"172.21.15.107:6829","nonce":1256497807}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":44323,"name":"cephfs.smithi157.rcaris","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6825/3923812398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3923812398},{"type":"v1","addr":"172.21.15.157:6825","nonce":3923812398}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":44349,"name":"cephfs.smithi157.ufywse","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6827/3167053094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":3167053094},{"type":"v1","addr":"172.21.15.157:6827","nonce":3167053094}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T23:32:20.146130+0000","modified":"2024-04-22T23:44:25.724056+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0],"up":{"mds_0":34434},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_34434":{"gid":34434,"name":"cephfs.smithi107.dfvvkw","rank":0,"incarnation":36,"state":"up:active","state_seq":19,"addr":"172.21.15.107:6827/418145840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":418145840},{"type":"v1","addr":"172.21.15.107:6827","nonce":418145840}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34434,"qdb_cluster":[34434]},"id":1}]} 2024-04-22T23:47:28.755 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 41 2024-04-22T23:47:28.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.765+0000 7f270646e640 1 -- 172.21.15.107:0/106771672 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f26e806eb80 msgr2=0x7f26e8071030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:28.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.765+0000 7f270646e640 1 --2- 172.21.15.107:0/106771672 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f26e806eb80 0x7f26e8071030 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f26f0000c00 tx=0x7f26f0009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:28.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.765+0000 7f270646e640 1 -- 172.21.15.107:0/106771672 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2700102d90 msgr2=0x7f2700113420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:28.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.765+0000 7f270646e640 1 --2- 172.21.15.107:0/106771672 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2700102d90 0x7f2700113420 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f26f4002a80 tx=0x7f26f4004600 comp rx=0 tx=0).stop 2024-04-22T23:47:28.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.765+0000 7f270646e640 1 -- 172.21.15.107:0/106771672 shutdown_connections 2024-04-22T23:47:28.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.765+0000 7f270646e640 1 --2- 172.21.15.107:0/106771672 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f26e806eb80 0x7f26e8071030 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:28.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.765+0000 7f270646e640 1 --2- 172.21.15.107:0/106771672 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2700102d90 0x7f2700113420 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:28.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.765+0000 7f270646e640 1 --2- 172.21.15.107:0/106771672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27001023f0 0x7f2700112ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:28.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.765+0000 7f270646e640 1 -- 172.21.15.107:0/106771672 >> 172.21.15.107:0/106771672 conn(0x7f27000f7fe0 msgr2=0x7f27000f8fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:28.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.766+0000 7f270646e640 1 -- 172.21.15.107:0/106771672 shutdown_connections 2024-04-22T23:47:28.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.766+0000 7f270646e640 1 -- 172.21.15.107:0/106771672 wait complete. 2024-04-22T23:47:28.768 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 42 2024-04-22T23:47:28.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.871+0000 7eff5946e640 1 -- 172.21.15.107:0/3914916990 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff54102250 msgr2=0x7eff54102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:28.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.871+0000 7eff5946e640 1 --2- 172.21.15.107:0/3914916990 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff54102250 0x7eff54102620 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7eff440048d0 tx=0x7eff44009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:28.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.871+0000 7eff5946e640 1 -- 172.21.15.107:0/3914916990 shutdown_connections 2024-04-22T23:47:28.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.871+0000 7eff5946e640 1 --2- 172.21.15.107:0/3914916990 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff540643b0 0x7eff540647e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:28.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.871+0000 7eff5946e640 1 --2- 172.21.15.107:0/3914916990 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff54102250 0x7eff54102620 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:28.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.871+0000 7eff5946e640 1 -- 172.21.15.107:0/3914916990 >> 172.21.15.107:0/3914916990 conn(0x7eff540710c0 msgr2=0x7eff540714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:28.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.872+0000 7eff5946e640 1 -- 172.21.15.107:0/3914916990 shutdown_connections 2024-04-22T23:47:28.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.872+0000 7eff5946e640 1 -- 172.21.15.107:0/3914916990 wait complete. 2024-04-22T23:47:28.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.872+0000 7eff5946e640 1 Processor -- start 2024-04-22T23:47:28.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.872+0000 7eff5946e640 1 -- start start 2024-04-22T23:47:28.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.872+0000 7eff5946e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff540643b0 0x7eff54112ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:28.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.872+0000 7eff5946e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff54102250 0x7eff54113220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:28.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.872+0000 7eff5946e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff54113820 con 0x7eff54102250 2024-04-22T23:47:28.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.872+0000 7eff5946e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff5410ce70 con 0x7eff540643b0 2024-04-22T23:47:28.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.872+0000 7eff4bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff54102250 0x7eff54113220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:28.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.872+0000 7eff4bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff54102250 0x7eff54113220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38870/0 (socket says 172.21.15.107:38870) 2024-04-22T23:47:28.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.872+0000 7eff4bfff640 1 -- 172.21.15.107:0/2709436839 learned_addr learned my addr 172.21.15.107:0/2709436839 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:47:28.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.873+0000 7eff4bfff640 1 -- 172.21.15.107:0/2709436839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff540643b0 msgr2=0x7eff54112ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:28.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.873+0000 7eff4bfff640 1 --2- 172.21.15.107:0/2709436839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff540643b0 0x7eff54112ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:28.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.873+0000 7eff4bfff640 1 -- 172.21.15.107:0/2709436839 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff44005040 con 0x7eff54102250 2024-04-22T23:47:28.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.873+0000 7eff4bfff640 1 --2- 172.21.15.107:0/2709436839 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff54102250 0x7eff54113220 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7eff3c002a50 tx=0x7eff3c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:28.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.873+0000 7eff4b7fe640 1 -- 172.21.15.107:0/2709436839 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff3c007c30 con 0x7eff54102250 2024-04-22T23:47:28.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.873+0000 7eff4b7fe640 1 -- 172.21.15.107:0/2709436839 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7eff3c00f040 con 0x7eff54102250 2024-04-22T23:47:28.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.873+0000 7eff5946e640 1 -- 172.21.15.107:0/2709436839 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff5410d150 con 0x7eff54102250 2024-04-22T23:47:28.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.873+0000 7eff4b7fe640 1 -- 172.21.15.107:0/2709436839 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff3c00db50 con 0x7eff54102250 2024-04-22T23:47:28.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.873+0000 7eff5946e640 1 -- 172.21.15.107:0/2709436839 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff5410d670 con 0x7eff54102250 2024-04-22T23:47:28.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.874+0000 7eff5946e640 1 -- 172.21.15.107:0/2709436839 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff540fd950 con 0x7eff54102250 2024-04-22T23:47:28.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.874+0000 7eff4b7fe640 1 -- 172.21.15.107:0/2709436839 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7eff3c00dcf0 con 0x7eff54102250 2024-04-22T23:47:28.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.875+0000 7eff4b7fe640 1 --2- 172.21.15.107:0/2709436839 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7eff4006eb70 0x7eff40071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:28.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.875+0000 7eff4b7fe640 1 -- 172.21.15.107:0/2709436839 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7eff3c08e190 con 0x7eff54102250 2024-04-22T23:47:28.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.875+0000 7eff52ffd640 1 --2- 172.21.15.107:0/2709436839 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7eff4006eb70 0x7eff40071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:28.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.875+0000 7eff52ffd640 1 --2- 172.21.15.107:0/2709436839 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7eff4006eb70 0x7eff40071020 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7eff440043d0 tx=0x7eff44009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:28.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:28.879+0000 7eff4b7fe640 1 -- 172.21.15.107:0/2709436839 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7eff3c056f10 con 0x7eff54102250 2024-04-22T23:47:29.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.096+0000 7eff5946e640 1 -- 172.21.15.107:0/2709436839 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 42, "format": "json"} v 0) v1 -- 0x7eff5404bcc0 con 0x7eff54102250 2024-04-22T23:47:29.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.097+0000 7eff4b7fe640 1 -- 172.21.15.107:0/2709436839 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 42, "format": "json"}]=0 dumped fsmap epoch 42 v43) v1 ==== 107+0+5234 (secure 0 0 0) 0x7eff3c05a5d0 con 0x7eff54102250 2024-04-22T23:47:29.089 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:47:29.089 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44323,"name":"cephfs.smithi157.rcaris","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6825/3923812398","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3923812398},{"type":"v1","addr":"172.21.15.157:6825","nonce":3923812398}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34},{"gid":44349,"name":"cephfs.smithi157.ufywse","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6827/3167053094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":3167053094},{"type":"v1","addr":"172.21.15.157:6827","nonce":3167053094}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T23:32:20.146130+0000","modified":"2024-04-22T23:44:26.725403+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34434,"mds_1":34438},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34434":{"gid":34434,"name":"cephfs.smithi107.dfvvkw","rank":0,"incarnation":36,"state":"up:active","state_seq":19,"addr":"172.21.15.107:6827/418145840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":418145840},{"type":"v1","addr":"172.21.15.107:6827","nonce":418145840}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34438":{"gid":34438,"name":"cephfs.smithi107.ktuxjl","rank":1,"incarnation":42,"state":"up:starting","state_seq":1,"addr":"172.21.15.107:6829/1256497807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1256497807},{"type":"v1","addr":"172.21.15.107:6829","nonce":1256497807}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34434,"qdb_cluster":[34434]},"id":1}]} 2024-04-22T23:47:29.089 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 42 2024-04-22T23:47:29.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.099+0000 7eff5946e640 1 -- 172.21.15.107:0/2709436839 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7eff4006eb70 msgr2=0x7eff40071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:29.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.099+0000 7eff5946e640 1 --2- 172.21.15.107:0/2709436839 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7eff4006eb70 0x7eff40071020 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7eff440043d0 tx=0x7eff44009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:29.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.099+0000 7eff5946e640 1 -- 172.21.15.107:0/2709436839 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff54102250 msgr2=0x7eff54113220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:29.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.099+0000 7eff5946e640 1 --2- 172.21.15.107:0/2709436839 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff54102250 0x7eff54113220 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7eff3c002a50 tx=0x7eff3c004600 comp rx=0 tx=0).stop 2024-04-22T23:47:29.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.099+0000 7eff5946e640 1 -- 172.21.15.107:0/2709436839 shutdown_connections 2024-04-22T23:47:29.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.099+0000 7eff5946e640 1 --2- 172.21.15.107:0/2709436839 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7eff4006eb70 0x7eff40071020 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:29.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.099+0000 7eff5946e640 1 --2- 172.21.15.107:0/2709436839 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff54102250 0x7eff54113220 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:29.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.099+0000 7eff5946e640 1 --2- 172.21.15.107:0/2709436839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff540643b0 0x7eff54112ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:29.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.099+0000 7eff5946e640 1 -- 172.21.15.107:0/2709436839 >> 172.21.15.107:0/2709436839 conn(0x7eff540710c0 msgr2=0x7eff540f9550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:29.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.099+0000 7eff5946e640 1 -- 172.21.15.107:0/2709436839 shutdown_connections 2024-04-22T23:47:29.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.099+0000 7eff5946e640 1 -- 172.21.15.107:0/2709436839 wait complete. 2024-04-22T23:47:29.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:28 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/3803079203' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 39, "format": "json"}]: dispatch 2024-04-22T23:47:29.100 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:28 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/2672862082' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 40, "format": "json"}]: dispatch 2024-04-22T23:47:29.101 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:28 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/106771672' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 41, "format": "json"}]: dispatch 2024-04-22T23:47:29.102 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-22T23:47:29.116 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-22T23:47:29.116 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-22T23:47:29.116 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-22T23:47:29.133 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-22T23:47:29.133 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-22T23:47:29.189 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T23:47:29.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.332+0000 7fec1206e640 1 -- 172.21.15.107:0/1316211133 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec0c102b90 msgr2=0x7fec0c0fc220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:29.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.332+0000 7fec1206e640 1 --2- 172.21.15.107:0/1316211133 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec0c102b90 0x7fec0c0fc220 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fec000048d0 tx=0x7fec00009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:29.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.332+0000 7fec1206e640 1 -- 172.21.15.107:0/1316211133 shutdown_connections 2024-04-22T23:47:29.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.332+0000 7fec1206e640 1 --2- 172.21.15.107:0/1316211133 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec0c102b90 0x7fec0c0fc220 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:29.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.332+0000 7fec1206e640 1 --2- 172.21.15.107:0/1316211133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec0c1021f0 0x7fec0c1025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:29.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.332+0000 7fec1206e640 1 -- 172.21.15.107:0/1316211133 >> 172.21.15.107:0/1316211133 conn(0x7fec0c0f7d60 msgr2=0x7fec0c0fa170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:29.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.332+0000 7fec1206e640 1 -- 172.21.15.107:0/1316211133 shutdown_connections 2024-04-22T23:47:29.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.332+0000 7fec1206e640 1 -- 172.21.15.107:0/1316211133 wait complete. 2024-04-22T23:47:29.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.332+0000 7fec1206e640 1 Processor -- start 2024-04-22T23:47:29.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.332+0000 7fec1206e640 1 -- start start 2024-04-22T23:47:29.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.333+0000 7fec1206e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec0c1021f0 0x7fec0c112d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:29.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.333+0000 7fec1206e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec0c102b90 0x7fec0c113290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:29.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.333+0000 7fec1206e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec0c113890 con 0x7fec0c1021f0 2024-04-22T23:47:29.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.333+0000 7fec1206e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec0c10cee0 con 0x7fec0c102b90 2024-04-22T23:47:29.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.333+0000 7fec0b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec0c1021f0 0x7fec0c112d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:29.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.333+0000 7fec0b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec0c1021f0 0x7fec0c112d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38892/0 (socket says 172.21.15.107:38892) 2024-04-22T23:47:29.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.333+0000 7fec0b7fe640 1 -- 172.21.15.107:0/1393030180 learned_addr learned my addr 172.21.15.107:0/1393030180 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:47:29.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.333+0000 7fec0affd640 1 --2- 172.21.15.107:0/1393030180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec0c102b90 0x7fec0c113290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:29.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.333+0000 7fec0b7fe640 1 -- 172.21.15.107:0/1393030180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec0c102b90 msgr2=0x7fec0c113290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:29.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.333+0000 7fec0b7fe640 1 --2- 172.21.15.107:0/1393030180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec0c102b90 0x7fec0c113290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:29.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.333+0000 7fec0b7fe640 1 -- 172.21.15.107:0/1393030180 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec00005040 con 0x7fec0c1021f0 2024-04-22T23:47:29.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.333+0000 7fec0affd640 1 --2- 172.21.15.107:0/1393030180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec0c102b90 0x7fec0c113290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T23:47:29.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.333+0000 7fec0b7fe640 1 --2- 172.21.15.107:0/1393030180 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec0c1021f0 0x7fec0c112d50 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7febfc002a80 tx=0x7febfc004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:29.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.333+0000 7febf3fff640 1 -- 172.21.15.107:0/1393030180 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febfc007c30 con 0x7fec0c1021f0 2024-04-22T23:47:29.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.333+0000 7fec1206e640 1 -- 172.21.15.107:0/1393030180 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec0c10d1c0 con 0x7fec0c1021f0 2024-04-22T23:47:29.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.333+0000 7febf3fff640 1 -- 172.21.15.107:0/1393030180 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7febfc00f040 con 0x7fec0c1021f0 2024-04-22T23:47:29.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.333+0000 7febf3fff640 1 -- 172.21.15.107:0/1393030180 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febfc0096b0 con 0x7fec0c1021f0 2024-04-22T23:47:29.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.333+0000 7fec1206e640 1 -- 172.21.15.107:0/1393030180 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec0c10d6b0 con 0x7fec0c1021f0 2024-04-22T23:47:29.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.334+0000 7fec1206e640 1 -- 172.21.15.107:0/1393030180 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec0c0fd960 con 0x7fec0c1021f0 2024-04-22T23:47:29.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.335+0000 7febf3fff640 1 -- 172.21.15.107:0/1393030180 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7febfc00d6b0 con 0x7fec0c1021f0 2024-04-22T23:47:29.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.335+0000 7febf3fff640 1 --2- 172.21.15.107:0/1393030180 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7febf406eb70 0x7febf4071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:29.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.335+0000 7febf3fff640 1 -- 172.21.15.107:0/1393030180 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7febfc08e370 con 0x7fec0c1021f0 2024-04-22T23:47:29.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.335+0000 7fec0affd640 1 --2- 172.21.15.107:0/1393030180 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7febf406eb70 0x7febf4071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:29.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.336+0000 7fec0affd640 1 --2- 172.21.15.107:0/1393030180 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7febf406eb70 0x7febf4071020 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fec0c064c00 tx=0x7fec00009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:29.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.340+0000 7febf3fff640 1 -- 172.21.15.107:0/1393030180 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7febfc057070 con 0x7fec0c1021f0 2024-04-22T23:47:29.379 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:28 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/3803079203' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 39, "format": "json"}]: dispatch 2024-04-22T23:47:29.379 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:28 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/2672862082' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 40, "format": "json"}]: dispatch 2024-04-22T23:47:29.379 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:28 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/106771672' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 41, "format": "json"}]: dispatch 2024-04-22T23:47:29.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.522+0000 7fec1206e640 1 -- 172.21.15.107:0/1393030180 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fec0c04bcc0 con 0x7fec0c1021f0 2024-04-22T23:47:29.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.523+0000 7febf3fff640 1 -- 172.21.15.107:0/1393030180 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16968 (secure 0 0 0) 0x7febfc05a730 con 0x7fec0c1021f0 2024-04-22T23:47:29.515 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:47:29.515 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":83,"fsid":"8252f6f2-00ff-11ef-bc93-c7b262605968","created":"2024-04-22T23:25:25.497133+0000","modified":"2024-04-22T23:44:07.204472+0000","last_up_change":"2024-04-22T23:42:20.528617+0000","last_in_change":"2024-04-22T23:30:44.396843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T23:29:48.249903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-22T23:32:19.078061+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-22T23:32:19.378987+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"65404290-734a-4901-bbb4-a87d3d846b06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":53,"up_thru":78,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":149725748},{"type":"v1","addr":"172.21.15.107:6803","nonce":149725748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":149725748},{"type":"v1","addr":"172.21.15.107:6805","nonce":149725748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":149725748},{"type":"v1","addr":"172.21.15.107:6809","nonce":149725748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":149725748},{"type":"v1","addr":"172.21.15.107:6807","nonce":149725748}]},"public_addr":"172.21.15.107:6803/149725748","cluster_addr":"172.21.15.107:6805/149725748","heartbeat_back_addr":"172.21.15.107:6809/149725748","heartbeat_front_addr":"172.21.15.107:6807/149725748","state":["exists","up"]},{"osd":1,"uuid":"6df381f7-b907-4d1b-bd08-fdf34e9729f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":54,"up_from":58,"up_thru":78,"down_at":55,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":997953964},{"type":"v1","addr":"172.21.15.107:6811","nonce":997953964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":997953964},{"type":"v1","addr":"172.21.15.107:6813","nonce":997953964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":997953964},{"type":"v1","addr":"172.21.15.107:6817","nonce":997953964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":997953964},{"type":"v1","addr":"172.21.15.107:6815","nonce":997953964}]},"public_addr":"172.21.15.107:6811/997953964","cluster_addr":"172.21.15.107:6813/997953964","heartbeat_back_addr":"172.21.15.107:6817/997953964","heartbeat_front_addr":"172.21.15.107:6815/997953964","state":["exists","up"]},{"osd":2,"uuid":"2c5e5c37-c1de-4e64-9c21-eacc145f1412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":59,"up_from":63,"up_thru":78,"down_at":60,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1823995026},{"type":"v1","addr":"172.21.15.107:6819","nonce":1823995026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1823995026},{"type":"v1","addr":"172.21.15.107:6821","nonce":1823995026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1823995026},{"type":"v1","addr":"172.21.15.107:6825","nonce":1823995026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1823995026},{"type":"v1","addr":"172.21.15.107:6823","nonce":1823995026}]},"public_addr":"172.21.15.107:6819/1823995026","cluster_addr":"172.21.15.107:6821/1823995026","heartbeat_back_addr":"172.21.15.107:6825/1823995026","heartbeat_front_addr":"172.21.15.107:6823/1823995026","state":["exists","up"]},{"osd":3,"uuid":"b7c9c124-9f43-4b5b-b039-c61ae30a416d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":64,"up_from":68,"up_thru":78,"down_at":65,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1553340859},{"type":"v1","addr":"172.21.15.157:6801","nonce":1553340859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1553340859},{"type":"v1","addr":"172.21.15.157:6803","nonce":1553340859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1553340859},{"type":"v1","addr":"172.21.15.157:6807","nonce":1553340859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1553340859},{"type":"v1","addr":"172.21.15.157:6805","nonce":1553340859}]},"public_addr":"172.21.15.157:6801/1553340859","cluster_addr":"172.21.15.157:6803/1553340859","heartbeat_back_addr":"172.21.15.157:6807/1553340859","heartbeat_front_addr":"172.21.15.157:6805/1553340859","state":["exists","up"]},{"osd":4,"uuid":"34a0cb61-1c2b-4515-9d74-8a3ad3f15fbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":69,"up_from":73,"up_thru":78,"down_at":70,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2579998472},{"type":"v1","addr":"172.21.15.157:6809","nonce":2579998472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2579998472},{"type":"v1","addr":"172.21.15.157:6811","nonce":2579998472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2579998472},{"type":"v1","addr":"172.21.15.157:6815","nonce":2579998472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2579998472},{"type":"v1","addr":"172.21.15.157:6813","nonce":2579998472}]},"public_addr":"172.21.15.157:6809/2579998472","cluster_addr":"172.21.15.157:6811/2579998472","heartbeat_back_addr":"172.21.15.157:6815/2579998472","heartbeat_front_addr":"172.21.15.157:6813/2579998472","state":["exists","up"]},{"osd":5,"uuid":"00934a7f-7e88-45cf-a2cc-e8a94bceaab0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":74,"up_from":78,"up_thru":78,"down_at":75,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3093110961},{"type":"v1","addr":"172.21.15.157:6817","nonce":3093110961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3093110961},{"type":"v1","addr":"172.21.15.157:6819","nonce":3093110961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3093110961},{"type":"v1","addr":"172.21.15.157:6823","nonce":3093110961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3093110961},{"type":"v1","addr":"172.21.15.157:6821","nonce":3093110961}]},"public_addr":"172.21.15.157:6817/3093110961","cluster_addr":"172.21.15.157:6819/3093110961","heartbeat_back_addr":"172.21.15.157:6823/3093110961","heartbeat_front_addr":"172.21.15.157:6821/3093110961","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:28:48.161289+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:29:18.447013+0000","dead_epoch":54},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:29:45.948165+0000","dead_epoch":59},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:30:09.825103+0000","dead_epoch":64},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:30:35.976009+0000","dead_epoch":69},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:31:01.729305+0000","dead_epoch":74}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.157:6827/4072444199":"2024-04-23T23:44:07.204437+0000","172.21.15.157:6826/4072444199":"2024-04-23T23:44:07.204437+0000","172.21.15.107:0/3783622140":"2024-04-23T23:37:55.631534+0000","172.21.15.107:0/3055151095":"2024-04-23T23:37:55.631534+0000","172.21.15.107:0/3377956937":"2024-04-23T23:25:58.816560+0000","172.21.15.107:0/2659503121":"2024-04-23T23:36:18.020700+0000","172.21.15.157:6829/3332906296":"2024-04-23T23:37:10.884172+0000","172.21.15.107:6800/307474287":"2024-04-23T23:26:29.807373+0000","172.21.15.107:6829/3335313253":"2024-04-23T23:43:26.926970+0000","172.21.15.107:6800/467262774":"2024-04-23T23:25:58.816560+0000","172.21.15.107:0/663073386":"2024-04-23T23:26:29.807373+0000","172.21.15.157:0/2260053045":"2024-04-23T23:37:10.884172+0000","172.21.15.107:0/525119004":"2024-04-23T23:36:18.020700+0000","172.21.15.107:6801/307474287":"2024-04-23T23:26:29.807373+0000","172.21.15.107:0/1065200386":"2024-04-23T23:36:18.020700+0000","172.21.15.107:6800/3427816626":"2024-04-23T23:27:33.931895+0000","172.21.15.107:0/3920905056":"2024-04-23T23:25:58.816560+0000","172.21.15.107:0/1734120985":"2024-04-23T23:27:33.931895+0000","172.21.15.157:0/278207195":"2024-04-23T23:37:10.884172+0000","172.21.15.107:0/2444490678":"2024-04-23T23:26:29.807373+0000","172.21.15.107:0/3567543485":"2024-04-23T23:36:18.020700+0000","172.21.15.157:6828/3332906296":"2024-04-23T23:37:10.884172+0000","172.21.15.107:0/1205692115":"2024-04-23T23:27:33.931895+0000","172.21.15.107:6801/467262774":"2024-04-23T23:25:58.816560+0000","172.21.15.107:0/2168608506":"2024-04-23T23:37:55.631534+0000","172.21.15.107:6800/2516490111":"2024-04-23T23:36:18.020700+0000","172.21.15.157:0/3173482279":"2024-04-23T23:37:10.884172+0000","172.21.15.107:6801/2516490111":"2024-04-23T23:36:18.020700+0000","172.21.15.157:0/3140121750":"2024-04-23T23:37:10.884172+0000","172.21.15.107:0/1029440409":"2024-04-23T23:26:29.807373+0000","172.21.15.107:6826/4144299942":"2024-04-23T23:32:35.769448+0000","172.21.15.107:6828/3335313253":"2024-04-23T23:43:26.926970+0000","172.21.15.107:6827/4144299942":"2024-04-23T23:32:35.769448+0000","172.21.15.107:6801/3427816626":"2024-04-23T23:27:33.931895+0000","172.21.15.157:0/2684888571":"2024-04-23T23:37:10.884172+0000","172.21.15.107:0/2873032378":"2024-04-23T23:37:55.631534+0000","172.21.15.107:0/3810721822":"2024-04-23T23:25:58.816560+0000","172.21.15.107:0/1888554706":"2024-04-23T23:37:55.631534+0000","172.21.15.107:0/3608791958":"2024-04-23T23:27:33.931895+0000","172.21.15.157:0/2452797907":"2024-04-23T23:37:10.884172+0000","172.21.15.107:0/3843391359":"2024-04-23T23:37:55.631534+0000","172.21.15.107:6801/2362628021":"2024-04-23T23:37:55.631534+0000","172.21.15.107:6800/2362628021":"2024-04-23T23:37:55.631534+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T23:47:29.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.525+0000 7fec1206e640 1 -- 172.21.15.107:0/1393030180 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7febf406eb70 msgr2=0x7febf4071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:29.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.525+0000 7fec1206e640 1 --2- 172.21.15.107:0/1393030180 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7febf406eb70 0x7febf4071020 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fec0c064c00 tx=0x7fec00009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:29.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.525+0000 7fec1206e640 1 -- 172.21.15.107:0/1393030180 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec0c1021f0 msgr2=0x7fec0c112d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:29.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.525+0000 7fec1206e640 1 --2- 172.21.15.107:0/1393030180 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec0c1021f0 0x7fec0c112d50 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7febfc002a80 tx=0x7febfc004600 comp rx=0 tx=0).stop 2024-04-22T23:47:29.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.525+0000 7fec1206e640 1 -- 172.21.15.107:0/1393030180 shutdown_connections 2024-04-22T23:47:29.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.525+0000 7fec1206e640 1 --2- 172.21.15.107:0/1393030180 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7febf406eb70 0x7febf4071020 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:29.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.525+0000 7fec1206e640 1 --2- 172.21.15.107:0/1393030180 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fec0c102b90 0x7fec0c113290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:29.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.525+0000 7fec1206e640 1 --2- 172.21.15.107:0/1393030180 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec0c1021f0 0x7fec0c112d50 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:29.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.525+0000 7fec1206e640 1 -- 172.21.15.107:0/1393030180 >> 172.21.15.107:0/1393030180 conn(0x7fec0c0f7d60 msgr2=0x7fec0c0f8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:29.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.525+0000 7fec1206e640 1 -- 172.21.15.107:0/1393030180 shutdown_connections 2024-04-22T23:47:29.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.525+0000 7fec1206e640 1 -- 172.21.15.107:0/1393030180 wait complete. 2024-04-22T23:47:29.528 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T23:29:48.249903+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-22T23:32:19.078061+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-22T23:32:19.378987+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T23:47:29.528 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T23:47:29.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.633+0000 7fa0d5c6e640 1 -- 172.21.15.107:0/2185020502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0d0102b90 msgr2=0x7fa0d00fa8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:29.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.633+0000 7fa0d5c6e640 1 --2- 172.21.15.107:0/2185020502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0d0102b90 0x7fa0d00fa8d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa0c40048d0 tx=0x7fa0c4009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:29.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.633+0000 7fa0d5c6e640 1 -- 172.21.15.107:0/2185020502 shutdown_connections 2024-04-22T23:47:29.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.633+0000 7fa0d5c6e640 1 --2- 172.21.15.107:0/2185020502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0d0102b90 0x7fa0d00fa8d0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:29.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.633+0000 7fa0d5c6e640 1 --2- 172.21.15.107:0/2185020502 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0d0102280 0x7fa0d0102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:29.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.633+0000 7fa0d5c6e640 1 -- 172.21.15.107:0/2185020502 >> 172.21.15.107:0/2185020502 conn(0x7fa0d00708f0 msgr2=0x7fa0d0070cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:29.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.633+0000 7fa0d5c6e640 1 -- 172.21.15.107:0/2185020502 shutdown_connections 2024-04-22T23:47:29.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.633+0000 7fa0d5c6e640 1 -- 172.21.15.107:0/2185020502 wait complete. 2024-04-22T23:47:29.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.633+0000 7fa0d5c6e640 1 Processor -- start 2024-04-22T23:47:29.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.633+0000 7fa0d5c6e640 1 -- start start 2024-04-22T23:47:29.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.634+0000 7fa0d5c6e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0d0102280 0x7fa0d0112d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:29.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.634+0000 7fa0d5c6e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0d0102b90 0x7fa0d01132c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:29.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.634+0000 7fa0d5c6e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d01138c0 con 0x7fa0d0102b90 2024-04-22T23:47:29.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.634+0000 7fa0d5c6e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d010cf10 con 0x7fa0d0102280 2024-04-22T23:47:29.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.634+0000 7fa0ceffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0d0102b90 0x7fa0d01132c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:29.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.634+0000 7fa0ceffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0d0102b90 0x7fa0d01132c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38920/0 (socket says 172.21.15.107:38920) 2024-04-22T23:47:29.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.634+0000 7fa0ceffd640 1 -- 172.21.15.107:0/3947050759 learned_addr learned my addr 172.21.15.107:0/3947050759 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:47:29.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.634+0000 7fa0cf7fe640 1 --2- 172.21.15.107:0/3947050759 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0d0102280 0x7fa0d0112d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:29.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.634+0000 7fa0ceffd640 1 -- 172.21.15.107:0/3947050759 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0d0102280 msgr2=0x7fa0d0112d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:29.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.634+0000 7fa0ceffd640 1 --2- 172.21.15.107:0/3947050759 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0d0102280 0x7fa0d0112d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:29.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.634+0000 7fa0ceffd640 1 -- 172.21.15.107:0/3947050759 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c4005040 con 0x7fa0d0102b90 2024-04-22T23:47:29.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.634+0000 7fa0ceffd640 1 --2- 172.21.15.107:0/3947050759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0d0102b90 0x7fa0d01132c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa0c40306c0 tx=0x7fa0c4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:29.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.634+0000 7fa0b7fff640 1 -- 172.21.15.107:0/3947050759 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0c4004290 con 0x7fa0d0102b90 2024-04-22T23:47:29.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.634+0000 7fa0d5c6e640 1 -- 172.21.15.107:0/3947050759 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0d010d190 con 0x7fa0d0102b90 2024-04-22T23:47:29.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.634+0000 7fa0b7fff640 1 -- 172.21.15.107:0/3947050759 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa0c40092e0 con 0x7fa0d0102b90 2024-04-22T23:47:29.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.634+0000 7fa0b7fff640 1 -- 172.21.15.107:0/3947050759 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0c4043400 con 0x7fa0d0102b90 2024-04-22T23:47:29.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.634+0000 7fa0d5c6e640 1 -- 172.21.15.107:0/3947050759 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0d010d6b0 con 0x7fa0d0102b90 2024-04-22T23:47:29.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.635+0000 7fa0d5c6e640 1 -- 172.21.15.107:0/3947050759 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0d0065330 con 0x7fa0d0102b90 2024-04-22T23:47:29.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.636+0000 7fa0b7fff640 1 -- 172.21.15.107:0/3947050759 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fa0c403b420 con 0x7fa0d0102b90 2024-04-22T23:47:29.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.636+0000 7fa0b7fff640 1 --2- 172.21.15.107:0/3947050759 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fa0b806eb70 0x7fa0b8071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:29.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.636+0000 7fa0b7fff640 1 -- 172.21.15.107:0/3947050759 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fa0c40b1600 con 0x7fa0d0102b90 2024-04-22T23:47:29.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.636+0000 7fa0cf7fe640 1 --2- 172.21.15.107:0/3947050759 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fa0b806eb70 0x7fa0b8071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:29.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.637+0000 7fa0cf7fe640 1 --2- 172.21.15.107:0/3947050759 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fa0b806eb70 0x7fa0b8071020 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0004a70 tx=0x7fa0c000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:29.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.641+0000 7fa0b7fff640 1 -- 172.21.15.107:0/3947050759 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fa0c407a440 con 0x7fa0d0102b90 2024-04-22T23:47:29.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.817+0000 7fa0d5c6e640 1 -- 172.21.15.107:0/3947050759 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa0d0109370 con 0x7fa0d0102b90 2024-04-22T23:47:29.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.817+0000 7fa0b7fff640 1 -- 172.21.15.107:0/3947050759 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa0c407db00 con 0x7fa0d0102b90 2024-04-22T23:47:29.809 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2024-04-22T23:47:29.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.819+0000 7fa0d5c6e640 1 -- 172.21.15.107:0/3947050759 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fa0b806eb70 msgr2=0x7fa0b8071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:29.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.819+0000 7fa0d5c6e640 1 --2- 172.21.15.107:0/3947050759 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fa0b806eb70 0x7fa0b8071020 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0004a70 tx=0x7fa0c000b040 comp rx=0 tx=0).stop 2024-04-22T23:47:29.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.819+0000 7fa0d5c6e640 1 -- 172.21.15.107:0/3947050759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0d0102b90 msgr2=0x7fa0d01132c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:29.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.819+0000 7fa0d5c6e640 1 --2- 172.21.15.107:0/3947050759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0d0102b90 0x7fa0d01132c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa0c40306c0 tx=0x7fa0c4009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:29.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.820+0000 7fa0d5c6e640 1 -- 172.21.15.107:0/3947050759 shutdown_connections 2024-04-22T23:47:29.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.820+0000 7fa0d5c6e640 1 --2- 172.21.15.107:0/3947050759 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fa0b806eb70 0x7fa0b8071020 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:29.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.820+0000 7fa0d5c6e640 1 --2- 172.21.15.107:0/3947050759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0d0102b90 0x7fa0d01132c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:29.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.820+0000 7fa0d5c6e640 1 --2- 172.21.15.107:0/3947050759 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0d0102280 0x7fa0d0112d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:29.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.820+0000 7fa0d5c6e640 1 -- 172.21.15.107:0/3947050759 >> 172.21.15.107:0/3947050759 conn(0x7fa0d00708f0 msgr2=0x7fa0d00ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:29.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.820+0000 7fa0d5c6e640 1 -- 172.21.15.107:0/3947050759 shutdown_connections 2024-04-22T23:47:29.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.820+0000 7fa0d5c6e640 1 -- 172.21.15.107:0/3947050759 wait complete. 2024-04-22T23:47:29.822 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-22T23:47:29.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.928+0000 7f296a35d640 1 -- 172.21.15.107:0/1258956958 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29641012f0 msgr2=0x7f29641016c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:29.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.928+0000 7f296a35d640 1 --2- 172.21.15.107:0/1258956958 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29641012f0 0x7f29641016c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f29540048d0 tx=0x7f2954009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:29.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.928+0000 7f296a35d640 1 -- 172.21.15.107:0/1258956958 shutdown_connections 2024-04-22T23:47:29.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.928+0000 7f296a35d640 1 --2- 172.21.15.107:0/1258956958 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2964101c90 0x7f29640fb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:29.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.928+0000 7f296a35d640 1 --2- 172.21.15.107:0/1258956958 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29641012f0 0x7f29641016c0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:29.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.928+0000 7f296a35d640 1 -- 172.21.15.107:0/1258956958 >> 172.21.15.107:0/1258956958 conn(0x7f29640f6e80 msgr2=0x7f29640f9290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:29.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.928+0000 7f296a35d640 1 -- 172.21.15.107:0/1258956958 shutdown_connections 2024-04-22T23:47:29.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.928+0000 7f296a35d640 1 -- 172.21.15.107:0/1258956958 wait complete. 2024-04-22T23:47:29.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.929+0000 7f296a35d640 1 Processor -- start 2024-04-22T23:47:29.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.929+0000 7f296a35d640 1 -- start start 2024-04-22T23:47:29.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.929+0000 7f296a35d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29641012f0 0x7f2964111df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:29.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.929+0000 7f296a35d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2964101c90 0x7f2964112330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:29.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.929+0000 7f296a35d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2964112930 con 0x7f2964101c90 2024-04-22T23:47:29.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.929+0000 7f296a35d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2964112aa0 con 0x7f29641012f0 2024-04-22T23:47:29.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.929+0000 7f29637fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2964101c90 0x7f2964112330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:29.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.929+0000 7f29637fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2964101c90 0x7f2964112330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38948/0 (socket says 172.21.15.107:38948) 2024-04-22T23:47:29.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.929+0000 7f29637fe640 1 -- 172.21.15.107:0/790399130 learned_addr learned my addr 172.21.15.107:0/790399130 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:47:29.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.930+0000 7f29637fe640 1 -- 172.21.15.107:0/790399130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29641012f0 msgr2=0x7f2964111df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:29.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.930+0000 7f29637fe640 1 --2- 172.21.15.107:0/790399130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29641012f0 0x7f2964111df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:29.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.930+0000 7f29637fe640 1 -- 172.21.15.107:0/790399130 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2954005040 con 0x7f2964101c90 2024-04-22T23:47:29.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.930+0000 7f29637fe640 1 --2- 172.21.15.107:0/790399130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2964101c90 0x7f2964112330 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2958002a50 tx=0x7f2958004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:29.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.930+0000 7f2960ff9640 1 -- 172.21.15.107:0/790399130 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2958007c30 con 0x7f2964101c90 2024-04-22T23:47:29.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.930+0000 7f296a35d640 1 -- 172.21.15.107:0/790399130 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f296410c130 con 0x7f2964101c90 2024-04-22T23:47:29.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.930+0000 7f2960ff9640 1 -- 172.21.15.107:0/790399130 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f295800f040 con 0x7f2964101c90 2024-04-22T23:47:29.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.930+0000 7f2960ff9640 1 -- 172.21.15.107:0/790399130 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f295800db50 con 0x7f2964101c90 2024-04-22T23:47:29.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.930+0000 7f296a35d640 1 -- 172.21.15.107:0/790399130 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f296410c650 con 0x7f2964101c90 2024-04-22T23:47:29.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.931+0000 7f296a35d640 1 -- 172.21.15.107:0/790399130 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29640fca60 con 0x7f2964101c90 2024-04-22T23:47:29.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.931+0000 7f2960ff9640 1 -- 172.21.15.107:0/790399130 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f295801c400 con 0x7f2964101c90 2024-04-22T23:47:29.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.932+0000 7f2960ff9640 1 --2- 172.21.15.107:0/790399130 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f294c06eb70 0x7f294c071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:29.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.932+0000 7f2960ff9640 1 -- 172.21.15.107:0/790399130 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f295808e240 con 0x7f2964101c90 2024-04-22T23:47:29.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.932+0000 7f2963fff640 1 --2- 172.21.15.107:0/790399130 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f294c06eb70 0x7f294c071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:29.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.932+0000 7f2963fff640 1 --2- 172.21.15.107:0/790399130 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f294c06eb70 0x7f294c071020 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f29540043d0 tx=0x7f2954009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:29.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:29.936+0000 7f2960ff9640 1 -- 172.21.15.107:0/790399130 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f2958056f40 con 0x7f2964101c90 2024-04-22T23:47:30.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.113+0000 7f296a35d640 1 -- 172.21.15.107:0/790399130 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f296404bcc0 con 0x7f2964101c90 2024-04-22T23:47:30.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.114+0000 7f2960ff9640 1 -- 172.21.15.107:0/790399130 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v83) v1 ==== 107+0+11 (secure 0 0 0) 0x7f295805a600 con 0x7f2964101c90 2024-04-22T23:47:30.106 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-04-22T23:47:30.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.116+0000 7f296a35d640 1 -- 172.21.15.107:0/790399130 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f294c06eb70 msgr2=0x7f294c071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:30.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.116+0000 7f296a35d640 1 --2- 172.21.15.107:0/790399130 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f294c06eb70 0x7f294c071020 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f29540043d0 tx=0x7f2954009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:30.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.116+0000 7f296a35d640 1 -- 172.21.15.107:0/790399130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2964101c90 msgr2=0x7f2964112330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:30.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.116+0000 7f296a35d640 1 --2- 172.21.15.107:0/790399130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2964101c90 0x7f2964112330 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2958002a50 tx=0x7f2958004600 comp rx=0 tx=0).stop 2024-04-22T23:47:30.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.116+0000 7f296a35d640 1 -- 172.21.15.107:0/790399130 shutdown_connections 2024-04-22T23:47:30.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.116+0000 7f296a35d640 1 --2- 172.21.15.107:0/790399130 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f294c06eb70 0x7f294c071020 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:30.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.116+0000 7f296a35d640 1 --2- 172.21.15.107:0/790399130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2964101c90 0x7f2964112330 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:30.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.116+0000 7f296a35d640 1 --2- 172.21.15.107:0/790399130 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29641012f0 0x7f2964111df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:30.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.116+0000 7f296a35d640 1 -- 172.21.15.107:0/790399130 >> 172.21.15.107:0/790399130 conn(0x7f29640f6e80 msgr2=0x7f29640f7d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:30.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.116+0000 7f296a35d640 1 -- 172.21.15.107:0/790399130 shutdown_connections 2024-04-22T23:47:30.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.116+0000 7f296a35d640 1 -- 172.21.15.107:0/790399130 wait complete. 2024-04-22T23:47:30.117 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:29 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/2709436839' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 42, "format": "json"}]: dispatch 2024-04-22T23:47:30.117 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:29 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/1393030180' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T23:47:30.117 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:29 smithi107 ceph-mon[90179]: pgmap v324: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:47:30.117 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:29 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/3947050759' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T23:47:30.118 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-22T23:47:30.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.224+0000 7f01b251d640 1 -- 172.21.15.107:0/3203158761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01ac102b90 msgr2=0x7f01ac0fc220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:30.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.224+0000 7f01b251d640 1 --2- 172.21.15.107:0/3203158761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01ac102b90 0x7f01ac0fc220 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f01a00048d0 tx=0x7f01a0009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:30.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.224+0000 7f01b251d640 1 -- 172.21.15.107:0/3203158761 shutdown_connections 2024-04-22T23:47:30.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.224+0000 7f01b251d640 1 --2- 172.21.15.107:0/3203158761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01ac102b90 0x7f01ac0fc220 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:30.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.224+0000 7f01b251d640 1 --2- 172.21.15.107:0/3203158761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01ac1021f0 0x7f01ac1025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:30.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.224+0000 7f01b251d640 1 -- 172.21.15.107:0/3203158761 >> 172.21.15.107:0/3203158761 conn(0x7f01ac0f7dc0 msgr2=0x7f01ac0fa1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:30.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.224+0000 7f01b251d640 1 -- 172.21.15.107:0/3203158761 shutdown_connections 2024-04-22T23:47:30.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.224+0000 7f01b251d640 1 -- 172.21.15.107:0/3203158761 wait complete. 2024-04-22T23:47:30.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.224+0000 7f01b251d640 1 Processor -- start 2024-04-22T23:47:30.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.224+0000 7f01b251d640 1 -- start start 2024-04-22T23:47:30.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.224+0000 7f01b251d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01ac1021f0 0x7f01ac112ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:30.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.224+0000 7f01b251d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01ac102b90 0x7f01ac113220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:30.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.224+0000 7f01b251d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac113820 con 0x7f01ac1021f0 2024-04-22T23:47:30.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.224+0000 7f01b251d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac10ce70 con 0x7f01ac102b90 2024-04-22T23:47:30.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.225+0000 7f01abfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01ac1021f0 0x7f01ac112ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:30.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.225+0000 7f01abfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01ac1021f0 0x7f01ac112ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38974/0 (socket says 172.21.15.107:38974) 2024-04-22T23:47:30.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.225+0000 7f01abfff640 1 -- 172.21.15.107:0/2923124994 learned_addr learned my addr 172.21.15.107:0/2923124994 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:47:30.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.225+0000 7f01ab7fe640 1 --2- 172.21.15.107:0/2923124994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01ac102b90 0x7f01ac113220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:30.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.225+0000 7f01abfff640 1 -- 172.21.15.107:0/2923124994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01ac102b90 msgr2=0x7f01ac113220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:30.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.225+0000 7f01abfff640 1 --2- 172.21.15.107:0/2923124994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01ac102b90 0x7f01ac113220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:30.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.225+0000 7f01abfff640 1 -- 172.21.15.107:0/2923124994 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a0005040 con 0x7f01ac1021f0 2024-04-22T23:47:30.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.225+0000 7f01ab7fe640 1 --2- 172.21.15.107:0/2923124994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01ac102b90 0x7f01ac113220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T23:47:30.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.225+0000 7f01abfff640 1 --2- 172.21.15.107:0/2923124994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01ac1021f0 0x7f01ac112ce0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f019c002a50 tx=0x7f019c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:30.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.225+0000 7f01a8ff9640 1 -- 172.21.15.107:0/2923124994 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f019c007c30 con 0x7f01ac1021f0 2024-04-22T23:47:30.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.225+0000 7f01b251d640 1 -- 172.21.15.107:0/2923124994 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01ac10d150 con 0x7f01ac1021f0 2024-04-22T23:47:30.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.225+0000 7f01a8ff9640 1 -- 172.21.15.107:0/2923124994 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f019c00f040 con 0x7f01ac1021f0 2024-04-22T23:47:30.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.225+0000 7f01a8ff9640 1 -- 172.21.15.107:0/2923124994 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f019c00db50 con 0x7f01ac1021f0 2024-04-22T23:47:30.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.225+0000 7f01b251d640 1 -- 172.21.15.107:0/2923124994 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01ac10d640 con 0x7f01ac1021f0 2024-04-22T23:47:30.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.226+0000 7f01b251d640 1 -- 172.21.15.107:0/2923124994 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01ac0fd960 con 0x7f01ac1021f0 2024-04-22T23:47:30.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.227+0000 7f01a8ff9640 1 -- 172.21.15.107:0/2923124994 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f019c01c400 con 0x7f01ac1021f0 2024-04-22T23:47:30.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.227+0000 7f01a8ff9640 1 --2- 172.21.15.107:0/2923124994 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f019406eb70 0x7f0194071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:30.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.227+0000 7f01a8ff9640 1 -- 172.21.15.107:0/2923124994 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f019c08e240 con 0x7f01ac1021f0 2024-04-22T23:47:30.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.227+0000 7f01ab7fe640 1 --2- 172.21.15.107:0/2923124994 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f019406eb70 0x7f0194071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:30.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.228+0000 7f01ab7fe640 1 --2- 172.21.15.107:0/2923124994 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f019406eb70 0x7f0194071020 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f01ac0fa850 tx=0x7f01a0009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:30.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.232+0000 7f01a8ff9640 1 -- 172.21.15.107:0/2923124994 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f019c056f40 con 0x7f01ac1021f0 2024-04-22T23:47:30.379 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:29 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/2709436839' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 42, "format": "json"}]: dispatch 2024-04-22T23:47:30.379 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:29 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/1393030180' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T23:47:30.379 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:29 smithi157 ceph-mon[79403]: pgmap v324: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:47:30.379 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:29 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/3947050759' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T23:47:30.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.410+0000 7f01b251d640 1 -- 172.21.15.107:0/2923124994 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f01ac109350 con 0x7f01ac1021f0 2024-04-22T23:47:30.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.410+0000 7f01a8ff9640 1 -- 172.21.15.107:0/2923124994 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v83) v1 ==== 107+0+11 (secure 0 0 0) 0x7f019c05a600 con 0x7f01ac1021f0 2024-04-22T23:47:30.402 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-04-22T23:47:30.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.412+0000 7f01b251d640 1 -- 172.21.15.107:0/2923124994 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f019406eb70 msgr2=0x7f0194071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:30.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.412+0000 7f01b251d640 1 --2- 172.21.15.107:0/2923124994 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f019406eb70 0x7f0194071020 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f01ac0fa850 tx=0x7f01a0009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:30.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.412+0000 7f01b251d640 1 -- 172.21.15.107:0/2923124994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01ac1021f0 msgr2=0x7f01ac112ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:30.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.412+0000 7f01b251d640 1 --2- 172.21.15.107:0/2923124994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01ac1021f0 0x7f01ac112ce0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f019c002a50 tx=0x7f019c004600 comp rx=0 tx=0).stop 2024-04-22T23:47:30.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.412+0000 7f01b251d640 1 -- 172.21.15.107:0/2923124994 shutdown_connections 2024-04-22T23:47:30.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.412+0000 7f01b251d640 1 --2- 172.21.15.107:0/2923124994 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f019406eb70 0x7f0194071020 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:30.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.412+0000 7f01b251d640 1 --2- 172.21.15.107:0/2923124994 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01ac102b90 0x7f01ac113220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:30.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.412+0000 7f01b251d640 1 --2- 172.21.15.107:0/2923124994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01ac1021f0 0x7f01ac112ce0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:30.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.412+0000 7f01b251d640 1 -- 172.21.15.107:0/2923124994 >> 172.21.15.107:0/2923124994 conn(0x7f01ac0f7dc0 msgr2=0x7f01ac0f8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:30.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.412+0000 7f01b251d640 1 -- 172.21.15.107:0/2923124994 shutdown_connections 2024-04-22T23:47:30.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.412+0000 7f01b251d640 1 -- 172.21.15.107:0/2923124994 wait complete. 2024-04-22T23:47:30.415 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-22T23:47:30.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.517+0000 7fa54b71d640 1 -- 172.21.15.107:0/3240841297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa544102400 msgr2=0x7fa5441027d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:30.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.517+0000 7fa54b71d640 1 --2- 172.21.15.107:0/3240841297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa544102400 0x7fa5441027d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa5400048d0 tx=0x7fa540009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:30.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.517+0000 7fa54b71d640 1 -- 172.21.15.107:0/3240841297 shutdown_connections 2024-04-22T23:47:30.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.517+0000 7fa54b71d640 1 --2- 172.21.15.107:0/3240841297 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa544102da0 0x7fa5440fc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:30.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.517+0000 7fa54b71d640 1 --2- 172.21.15.107:0/3240841297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa544102400 0x7fa5441027d0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:30.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.517+0000 7fa54b71d640 1 -- 172.21.15.107:0/3240841297 >> 172.21.15.107:0/3240841297 conn(0x7fa5440f7fb0 msgr2=0x7fa5440fa3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:30.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.517+0000 7fa54b71d640 1 -- 172.21.15.107:0/3240841297 shutdown_connections 2024-04-22T23:47:30.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.517+0000 7fa54b71d640 1 -- 172.21.15.107:0/3240841297 wait complete. 2024-04-22T23:47:30.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.517+0000 7fa54b71d640 1 Processor -- start 2024-04-22T23:47:30.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.517+0000 7fa54b71d640 1 -- start start 2024-04-22T23:47:30.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.518+0000 7fa54b71d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa544102400 0x7fa544112ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:30.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.518+0000 7fa54b71d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa544102da0 0x7fa544113430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:30.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.518+0000 7fa54b71d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa544113a30 con 0x7fa544102400 2024-04-22T23:47:30.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.518+0000 7fa54b71d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54410d080 con 0x7fa544102da0 2024-04-22T23:47:30.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.518+0000 7fa5494b9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa544102400 0x7fa544112ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:30.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.518+0000 7fa5494b9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa544102400 0x7fa544112ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38994/0 (socket says 172.21.15.107:38994) 2024-04-22T23:47:30.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.518+0000 7fa5494b9640 1 -- 172.21.15.107:0/2342895705 learned_addr learned my addr 172.21.15.107:0/2342895705 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:47:30.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.518+0000 7fa548cb8640 1 --2- 172.21.15.107:0/2342895705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa544102da0 0x7fa544113430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:30.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.518+0000 7fa5494b9640 1 -- 172.21.15.107:0/2342895705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa544102da0 msgr2=0x7fa544113430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:30.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.518+0000 7fa5494b9640 1 --2- 172.21.15.107:0/2342895705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa544102da0 0x7fa544113430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:30.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.518+0000 7fa5494b9640 1 -- 172.21.15.107:0/2342895705 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa540005040 con 0x7fa544102400 2024-04-22T23:47:30.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.518+0000 7fa548cb8640 1 --2- 172.21.15.107:0/2342895705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa544102da0 0x7fa544113430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T23:47:30.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.518+0000 7fa5494b9640 1 --2- 172.21.15.107:0/2342895705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa544102400 0x7fa544112ef0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa540004930 tx=0x7fa540009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:30.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.518+0000 7fa539ffb640 1 -- 172.21.15.107:0/2342895705 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5400094c0 con 0x7fa544102400 2024-04-22T23:47:30.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.518+0000 7fa539ffb640 1 -- 172.21.15.107:0/2342895705 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa540032e50 con 0x7fa544102400 2024-04-22T23:47:30.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.518+0000 7fa54b71d640 1 -- 172.21.15.107:0/2342895705 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa54410d300 con 0x7fa544102400 2024-04-22T23:47:30.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.518+0000 7fa539ffb640 1 -- 172.21.15.107:0/2342895705 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5400433b0 con 0x7fa544102400 2024-04-22T23:47:30.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.519+0000 7fa54b71d640 1 -- 172.21.15.107:0/2342895705 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa54410d7c0 con 0x7fa544102400 2024-04-22T23:47:30.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.519+0000 7fa54b71d640 1 -- 172.21.15.107:0/2342895705 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5440fdb70 con 0x7fa544102400 2024-04-22T23:47:30.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.520+0000 7fa539ffb640 1 -- 172.21.15.107:0/2342895705 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7fa54002f070 con 0x7fa544102400 2024-04-22T23:47:30.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.520+0000 7fa539ffb640 1 --2- 172.21.15.107:0/2342895705 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fa53006eb70 0x7fa530071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:30.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.520+0000 7fa539ffb640 1 -- 172.21.15.107:0/2342895705 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fa5400b0960 con 0x7fa544102400 2024-04-22T23:47:30.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.520+0000 7fa548cb8640 1 --2- 172.21.15.107:0/2342895705 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fa53006eb70 0x7fa530071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:30.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.521+0000 7fa548cb8640 1 --2- 172.21.15.107:0/2342895705 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fa53006eb70 0x7fa530071020 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa54410e7b0 tx=0x7fa534009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:30.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.525+0000 7fa539ffb640 1 -- 172.21.15.107:0/2342895705 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7fa540079660 con 0x7fa544102400 2024-04-22T23:47:30.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.712+0000 7fa54b71d640 1 -- 172.21.15.107:0/2342895705 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa5440638d0 con 0x7fa544102400 2024-04-22T23:47:30.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.712+0000 7fa539ffb640 1 -- 172.21.15.107:0/2342895705 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 43 entries v83) v1 ==== 81+0+2584 (secure 0 0 0) 0x7fa54007cd20 con 0x7fa544102400 2024-04-22T23:47:30.704 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.157:6827/4072444199 2024-04-23T23:44:07.204437+0000 2024-04-22T23:47:30.704 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.157:6826/4072444199 2024-04-23T23:44:07.204437+0000 2024-04-22T23:47:30.704 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3783622140 2024-04-23T23:37:55.631534+0000 2024-04-22T23:47:30.705 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3055151095 2024-04-23T23:37:55.631534+0000 2024-04-22T23:47:30.705 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3377956937 2024-04-23T23:25:58.816560+0000 2024-04-22T23:47:30.705 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/2659503121 2024-04-23T23:36:18.020700+0000 2024-04-22T23:47:30.705 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.157:6829/3332906296 2024-04-23T23:37:10.884172+0000 2024-04-22T23:47:30.705 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/307474287 2024-04-23T23:26:29.807373+0000 2024-04-22T23:47:30.705 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6829/3335313253 2024-04-23T23:43:26.926970+0000 2024-04-22T23:47:30.705 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/467262774 2024-04-23T23:25:58.816560+0000 2024-04-22T23:47:30.705 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/663073386 2024-04-23T23:26:29.807373+0000 2024-04-22T23:47:30.705 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.157:0/2260053045 2024-04-23T23:37:10.884172+0000 2024-04-22T23:47:30.705 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/525119004 2024-04-23T23:36:18.020700+0000 2024-04-22T23:47:30.705 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/307474287 2024-04-23T23:26:29.807373+0000 2024-04-22T23:47:30.705 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/1065200386 2024-04-23T23:36:18.020700+0000 2024-04-22T23:47:30.705 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/3427816626 2024-04-23T23:27:33.931895+0000 2024-04-22T23:47:30.706 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3920905056 2024-04-23T23:25:58.816560+0000 2024-04-22T23:47:30.706 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/1734120985 2024-04-23T23:27:33.931895+0000 2024-04-22T23:47:30.706 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.157:0/278207195 2024-04-23T23:37:10.884172+0000 2024-04-22T23:47:30.706 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/2444490678 2024-04-23T23:26:29.807373+0000 2024-04-22T23:47:30.706 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3567543485 2024-04-23T23:36:18.020700+0000 2024-04-22T23:47:30.706 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.157:6828/3332906296 2024-04-23T23:37:10.884172+0000 2024-04-22T23:47:30.706 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/1205692115 2024-04-23T23:27:33.931895+0000 2024-04-22T23:47:30.706 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/467262774 2024-04-23T23:25:58.816560+0000 2024-04-22T23:47:30.706 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/2168608506 2024-04-23T23:37:55.631534+0000 2024-04-22T23:47:30.706 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/2516490111 2024-04-23T23:36:18.020700+0000 2024-04-22T23:47:30.706 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.157:0/3173482279 2024-04-23T23:37:10.884172+0000 2024-04-22T23:47:30.706 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/2516490111 2024-04-23T23:36:18.020700+0000 2024-04-22T23:47:30.706 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.157:0/3140121750 2024-04-23T23:37:10.884172+0000 2024-04-22T23:47:30.707 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/1029440409 2024-04-23T23:26:29.807373+0000 2024-04-22T23:47:30.707 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6826/4144299942 2024-04-23T23:32:35.769448+0000 2024-04-22T23:47:30.707 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6828/3335313253 2024-04-23T23:43:26.926970+0000 2024-04-22T23:47:30.707 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6827/4144299942 2024-04-23T23:32:35.769448+0000 2024-04-22T23:47:30.707 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/3427816626 2024-04-23T23:27:33.931895+0000 2024-04-22T23:47:30.707 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.157:0/2684888571 2024-04-23T23:37:10.884172+0000 2024-04-22T23:47:30.707 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/2873032378 2024-04-23T23:37:55.631534+0000 2024-04-22T23:47:30.707 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3810721822 2024-04-23T23:25:58.816560+0000 2024-04-22T23:47:30.707 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/1888554706 2024-04-23T23:37:55.631534+0000 2024-04-22T23:47:30.707 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3608791958 2024-04-23T23:27:33.931895+0000 2024-04-22T23:47:30.707 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.157:0/2452797907 2024-04-23T23:37:10.884172+0000 2024-04-22T23:47:30.708 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3843391359 2024-04-23T23:37:55.631534+0000 2024-04-22T23:47:30.708 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/2362628021 2024-04-23T23:37:55.631534+0000 2024-04-22T23:47:30.708 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/2362628021 2024-04-23T23:37:55.631534+0000 2024-04-22T23:47:30.708 INFO:teuthology.orchestra.run.smithi107.stderr:listed 43 entries 2024-04-22T23:47:30.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.714+0000 7fa54b71d640 1 -- 172.21.15.107:0/2342895705 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fa53006eb70 msgr2=0x7fa530071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:30.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.714+0000 7fa54b71d640 1 --2- 172.21.15.107:0/2342895705 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fa53006eb70 0x7fa530071020 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa54410e7b0 tx=0x7fa534009040 comp rx=0 tx=0).stop 2024-04-22T23:47:30.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.714+0000 7fa54b71d640 1 -- 172.21.15.107:0/2342895705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa544102400 msgr2=0x7fa544112ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:30.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.714+0000 7fa54b71d640 1 --2- 172.21.15.107:0/2342895705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa544102400 0x7fa544112ef0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa540004930 tx=0x7fa540009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:30.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.714+0000 7fa54b71d640 1 -- 172.21.15.107:0/2342895705 shutdown_connections 2024-04-22T23:47:30.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.714+0000 7fa54b71d640 1 --2- 172.21.15.107:0/2342895705 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7fa53006eb70 0x7fa530071020 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:30.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.714+0000 7fa54b71d640 1 --2- 172.21.15.107:0/2342895705 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa544102da0 0x7fa544113430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:30.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.714+0000 7fa54b71d640 1 --2- 172.21.15.107:0/2342895705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa544102400 0x7fa544112ef0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:30.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.714+0000 7fa54b71d640 1 -- 172.21.15.107:0/2342895705 >> 172.21.15.107:0/2342895705 conn(0x7fa5440f7fb0 msgr2=0x7fa5440f8f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:30.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.714+0000 7fa54b71d640 1 -- 172.21.15.107:0/2342895705 shutdown_connections 2024-04-22T23:47:30.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.714+0000 7fa54b71d640 1 -- 172.21.15.107:0/2342895705 wait complete. 2024-04-22T23:47:30.717 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-22T23:47:30.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T23:47:30.717 DEBUG:teuthology.orchestra.run.smithi107:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-22T23:47:30.741 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi107.front.sepia.ceph.com 2024-04-22T23:47:30.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T23:47:30.742 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-22T23:47:30.805 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-22T23:47:30.805 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-22T23:47:30.826 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-04-22T23:47:30.826 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-22T23:47:30.885 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T23:47:30.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.987+0000 7f7fb1a6e640 1 -- 172.21.15.107:0/2416460205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fac0fb130 msgr2=0x7f7fac0fb580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:30.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.987+0000 7f7fb1a6e640 1 --2- 172.21.15.107:0/2416460205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fac0fb130 0x7f7fac0fb580 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7fa00048d0 tx=0x7f7fa0009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:30.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.987+0000 7f7fb1a6e640 1 -- 172.21.15.107:0/2416460205 shutdown_connections 2024-04-22T23:47:30.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.987+0000 7f7fb1a6e640 1 --2- 172.21.15.107:0/2416460205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fac0fb130 0x7f7fac0fb580 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:30.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.987+0000 7f7fb1a6e640 1 --2- 172.21.15.107:0/2416460205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fac101160 0x7f7fac101530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:30.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.987+0000 7f7fb1a6e640 1 -- 172.21.15.107:0/2416460205 >> 172.21.15.107:0/2416460205 conn(0x7f7fac071190 msgr2=0x7f7fac071590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:30.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.987+0000 7f7fb1a6e640 1 -- 172.21.15.107:0/2416460205 shutdown_connections 2024-04-22T23:47:30.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.987+0000 7f7fb1a6e640 1 -- 172.21.15.107:0/2416460205 wait complete. 2024-04-22T23:47:30.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.988+0000 7f7fb1a6e640 1 Processor -- start 2024-04-22T23:47:30.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.988+0000 7f7fb1a6e640 1 -- start start 2024-04-22T23:47:30.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.988+0000 7f7fb1a6e640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fac0fb130 0x7f7fac10df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:30.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.988+0000 7f7fb1a6e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fac101160 0x7f7fac10e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:30.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.988+0000 7f7fb1a6e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fac10e980 con 0x7f7fac101160 2024-04-22T23:47:30.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.988+0000 7f7fb1a6e640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fac10eaf0 con 0x7f7fac0fb130 2024-04-22T23:47:30.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.988+0000 7f7faaffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fac101160 0x7f7fac10e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:30.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.988+0000 7f7faaffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fac101160 0x7f7fac10e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39020/0 (socket says 172.21.15.107:39020) 2024-04-22T23:47:30.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.988+0000 7f7faaffd640 1 -- 172.21.15.107:0/292041444 learned_addr learned my addr 172.21.15.107:0/292041444 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:47:30.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.988+0000 7f7fab7fe640 1 --2- 172.21.15.107:0/292041444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fac0fb130 0x7f7fac10df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:30.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.988+0000 7f7faaffd640 1 -- 172.21.15.107:0/292041444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fac0fb130 msgr2=0x7f7fac10df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:30.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.988+0000 7f7faaffd640 1 --2- 172.21.15.107:0/292041444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fac0fb130 0x7f7fac10df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:30.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.989+0000 7f7faaffd640 1 -- 172.21.15.107:0/292041444 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa0005040 con 0x7f7fac101160 2024-04-22T23:47:30.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.989+0000 7f7faaffd640 1 --2- 172.21.15.107:0/292041444 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fac101160 0x7f7fac10e440 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7fa00041d0 tx=0x7f7fa0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:30.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.989+0000 7f7f93fff640 1 -- 172.21.15.107:0/292041444 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fa00092e0 con 0x7f7fac101160 2024-04-22T23:47:30.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.989+0000 7f7f93fff640 1 -- 172.21.15.107:0/292041444 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7fa0009440 con 0x7f7fac101160 2024-04-22T23:47:30.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:30 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/790399130' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-22T23:47:30.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:30 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/2923124994' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-22T23:47:30.983 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:30 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/2342895705' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-22T23:47:30.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.989+0000 7f7fb1a6e640 1 -- 172.21.15.107:0/292041444 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fac10ed70 con 0x7f7fac101160 2024-04-22T23:47:30.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.989+0000 7f7f93fff640 1 -- 172.21.15.107:0/292041444 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fa0043640 con 0x7f7fac101160 2024-04-22T23:47:30.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.989+0000 7f7fb1a6e640 1 -- 172.21.15.107:0/292041444 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fac117250 con 0x7f7fac101160 2024-04-22T23:47:30.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.990+0000 7f7fb1a6e640 1 -- 172.21.15.107:0/292041444 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fac0fc820 con 0x7f7fac101160 2024-04-22T23:47:30.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.990+0000 7f7f93fff640 1 -- 172.21.15.107:0/292041444 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f7fa003b9f0 con 0x7f7fac101160 2024-04-22T23:47:30.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.991+0000 7f7f93fff640 1 --2- 172.21.15.107:0/292041444 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f7f9406eb70 0x7f7f94071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:30.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.991+0000 7f7f93fff640 1 -- 172.21.15.107:0/292041444 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f7fa00b1440 con 0x7f7fac101160 2024-04-22T23:47:30.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.991+0000 7f7fab7fe640 1 --2- 172.21.15.107:0/292041444 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f7f9406eb70 0x7f7f94071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:30.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.991+0000 7f7fab7fe640 1 --2- 172.21.15.107:0/292041444 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f7f9406eb70 0x7f7f94071020 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c006fd0 tx=0x7f7f9c008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:30.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:30.996+0000 7f7f93fff640 1 -- 172.21.15.107:0/292041444 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f7fa007a260 con 0x7f7fac101160 2024-04-22T23:47:31.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.175+0000 7f7fb1a6e640 1 -- 172.21.15.107:0/292041444 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7fac10ed10 con 0x7f7fac101160 2024-04-22T23:47:31.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.177+0000 7f7f93fff640 1 -- 172.21.15.107:0/292041444 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16968 (secure 0 0 0) 0x7f7fa007d920 con 0x7f7fac101160 2024-04-22T23:47:31.168 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-22T23:47:31.169 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":83,"fsid":"8252f6f2-00ff-11ef-bc93-c7b262605968","created":"2024-04-22T23:25:25.497133+0000","modified":"2024-04-22T23:44:07.204472+0000","last_up_change":"2024-04-22T23:42:20.528617+0000","last_in_change":"2024-04-22T23:30:44.396843+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T23:29:48.249903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-22T23:32:19.078061+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-22T23:32:19.378987+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"65404290-734a-4901-bbb4-a87d3d846b06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":53,"up_thru":78,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":149725748},{"type":"v1","addr":"172.21.15.107:6803","nonce":149725748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":149725748},{"type":"v1","addr":"172.21.15.107:6805","nonce":149725748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":149725748},{"type":"v1","addr":"172.21.15.107:6809","nonce":149725748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":149725748},{"type":"v1","addr":"172.21.15.107:6807","nonce":149725748}]},"public_addr":"172.21.15.107:6803/149725748","cluster_addr":"172.21.15.107:6805/149725748","heartbeat_back_addr":"172.21.15.107:6809/149725748","heartbeat_front_addr":"172.21.15.107:6807/149725748","state":["exists","up"]},{"osd":1,"uuid":"6df381f7-b907-4d1b-bd08-fdf34e9729f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":54,"up_from":58,"up_thru":78,"down_at":55,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":997953964},{"type":"v1","addr":"172.21.15.107:6811","nonce":997953964}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":997953964},{"type":"v1","addr":"172.21.15.107:6813","nonce":997953964}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":997953964},{"type":"v1","addr":"172.21.15.107:6817","nonce":997953964}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":997953964},{"type":"v1","addr":"172.21.15.107:6815","nonce":997953964}]},"public_addr":"172.21.15.107:6811/997953964","cluster_addr":"172.21.15.107:6813/997953964","heartbeat_back_addr":"172.21.15.107:6817/997953964","heartbeat_front_addr":"172.21.15.107:6815/997953964","state":["exists","up"]},{"osd":2,"uuid":"2c5e5c37-c1de-4e64-9c21-eacc145f1412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":59,"up_from":63,"up_thru":78,"down_at":60,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1823995026},{"type":"v1","addr":"172.21.15.107:6819","nonce":1823995026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1823995026},{"type":"v1","addr":"172.21.15.107:6821","nonce":1823995026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1823995026},{"type":"v1","addr":"172.21.15.107:6825","nonce":1823995026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1823995026},{"type":"v1","addr":"172.21.15.107:6823","nonce":1823995026}]},"public_addr":"172.21.15.107:6819/1823995026","cluster_addr":"172.21.15.107:6821/1823995026","heartbeat_back_addr":"172.21.15.107:6825/1823995026","heartbeat_front_addr":"172.21.15.107:6823/1823995026","state":["exists","up"]},{"osd":3,"uuid":"b7c9c124-9f43-4b5b-b039-c61ae30a416d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":64,"up_from":68,"up_thru":78,"down_at":65,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1553340859},{"type":"v1","addr":"172.21.15.157:6801","nonce":1553340859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":1553340859},{"type":"v1","addr":"172.21.15.157:6803","nonce":1553340859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":1553340859},{"type":"v1","addr":"172.21.15.157:6807","nonce":1553340859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":1553340859},{"type":"v1","addr":"172.21.15.157:6805","nonce":1553340859}]},"public_addr":"172.21.15.157:6801/1553340859","cluster_addr":"172.21.15.157:6803/1553340859","heartbeat_back_addr":"172.21.15.157:6807/1553340859","heartbeat_front_addr":"172.21.15.157:6805/1553340859","state":["exists","up"]},{"osd":4,"uuid":"34a0cb61-1c2b-4515-9d74-8a3ad3f15fbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":69,"up_from":73,"up_thru":78,"down_at":70,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2579998472},{"type":"v1","addr":"172.21.15.157:6809","nonce":2579998472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2579998472},{"type":"v1","addr":"172.21.15.157:6811","nonce":2579998472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2579998472},{"type":"v1","addr":"172.21.15.157:6815","nonce":2579998472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2579998472},{"type":"v1","addr":"172.21.15.157:6813","nonce":2579998472}]},"public_addr":"172.21.15.157:6809/2579998472","cluster_addr":"172.21.15.157:6811/2579998472","heartbeat_back_addr":"172.21.15.157:6815/2579998472","heartbeat_front_addr":"172.21.15.157:6813/2579998472","state":["exists","up"]},{"osd":5,"uuid":"00934a7f-7e88-45cf-a2cc-e8a94bceaab0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":74,"up_from":78,"up_thru":78,"down_at":75,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":3093110961},{"type":"v1","addr":"172.21.15.157:6817","nonce":3093110961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3093110961},{"type":"v1","addr":"172.21.15.157:6819","nonce":3093110961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3093110961},{"type":"v1","addr":"172.21.15.157:6823","nonce":3093110961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3093110961},{"type":"v1","addr":"172.21.15.157:6821","nonce":3093110961}]},"public_addr":"172.21.15.157:6817/3093110961","cluster_addr":"172.21.15.157:6819/3093110961","heartbeat_back_addr":"172.21.15.157:6823/3093110961","heartbeat_front_addr":"172.21.15.157:6821/3093110961","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:28:48.161289+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:29:18.447013+0000","dead_epoch":54},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:29:45.948165+0000","dead_epoch":59},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:30:09.825103+0000","dead_epoch":64},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:30:35.976009+0000","dead_epoch":69},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T23:31:01.729305+0000","dead_epoch":74}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.157:6827/4072444199":"2024-04-23T23:44:07.204437+0000","172.21.15.157:6826/4072444199":"2024-04-23T23:44:07.204437+0000","172.21.15.107:0/3783622140":"2024-04-23T23:37:55.631534+0000","172.21.15.107:0/3055151095":"2024-04-23T23:37:55.631534+0000","172.21.15.107:0/3377956937":"2024-04-23T23:25:58.816560+0000","172.21.15.107:0/2659503121":"2024-04-23T23:36:18.020700+0000","172.21.15.157:6829/3332906296":"2024-04-23T23:37:10.884172+0000","172.21.15.107:6800/307474287":"2024-04-23T23:26:29.807373+0000","172.21.15.107:6829/3335313253":"2024-04-23T23:43:26.926970+0000","172.21.15.107:6800/467262774":"2024-04-23T23:25:58.816560+0000","172.21.15.107:0/663073386":"2024-04-23T23:26:29.807373+0000","172.21.15.157:0/2260053045":"2024-04-23T23:37:10.884172+0000","172.21.15.107:0/525119004":"2024-04-23T23:36:18.020700+0000","172.21.15.107:6801/307474287":"2024-04-23T23:26:29.807373+0000","172.21.15.107:0/1065200386":"2024-04-23T23:36:18.020700+0000","172.21.15.107:6800/3427816626":"2024-04-23T23:27:33.931895+0000","172.21.15.107:0/3920905056":"2024-04-23T23:25:58.816560+0000","172.21.15.107:0/1734120985":"2024-04-23T23:27:33.931895+0000","172.21.15.157:0/278207195":"2024-04-23T23:37:10.884172+0000","172.21.15.107:0/2444490678":"2024-04-23T23:26:29.807373+0000","172.21.15.107:0/3567543485":"2024-04-23T23:36:18.020700+0000","172.21.15.157:6828/3332906296":"2024-04-23T23:37:10.884172+0000","172.21.15.107:0/1205692115":"2024-04-23T23:27:33.931895+0000","172.21.15.107:6801/467262774":"2024-04-23T23:25:58.816560+0000","172.21.15.107:0/2168608506":"2024-04-23T23:37:55.631534+0000","172.21.15.107:6800/2516490111":"2024-04-23T23:36:18.020700+0000","172.21.15.157:0/3173482279":"2024-04-23T23:37:10.884172+0000","172.21.15.107:6801/2516490111":"2024-04-23T23:36:18.020700+0000","172.21.15.157:0/3140121750":"2024-04-23T23:37:10.884172+0000","172.21.15.107:0/1029440409":"2024-04-23T23:26:29.807373+0000","172.21.15.107:6826/4144299942":"2024-04-23T23:32:35.769448+0000","172.21.15.107:6828/3335313253":"2024-04-23T23:43:26.926970+0000","172.21.15.107:6827/4144299942":"2024-04-23T23:32:35.769448+0000","172.21.15.107:6801/3427816626":"2024-04-23T23:27:33.931895+0000","172.21.15.157:0/2684888571":"2024-04-23T23:37:10.884172+0000","172.21.15.107:0/2873032378":"2024-04-23T23:37:55.631534+0000","172.21.15.107:0/3810721822":"2024-04-23T23:25:58.816560+0000","172.21.15.107:0/1888554706":"2024-04-23T23:37:55.631534+0000","172.21.15.107:0/3608791958":"2024-04-23T23:27:33.931895+0000","172.21.15.157:0/2452797907":"2024-04-23T23:37:10.884172+0000","172.21.15.107:0/3843391359":"2024-04-23T23:37:55.631534+0000","172.21.15.107:6801/2362628021":"2024-04-23T23:37:55.631534+0000","172.21.15.107:6800/2362628021":"2024-04-23T23:37:55.631534+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T23:47:31.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.178+0000 7f7fb1a6e640 1 -- 172.21.15.107:0/292041444 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f7f9406eb70 msgr2=0x7f7f94071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:31.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.178+0000 7f7fb1a6e640 1 --2- 172.21.15.107:0/292041444 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f7f9406eb70 0x7f7f94071020 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c006fd0 tx=0x7f7f9c008040 comp rx=0 tx=0).stop 2024-04-22T23:47:31.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.178+0000 7f7fb1a6e640 1 -- 172.21.15.107:0/292041444 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fac101160 msgr2=0x7f7fac10e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:31.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.178+0000 7f7fb1a6e640 1 --2- 172.21.15.107:0/292041444 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fac101160 0x7f7fac10e440 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7fa00041d0 tx=0x7f7fa0009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:31.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.179+0000 7f7fb1a6e640 1 -- 172.21.15.107:0/292041444 shutdown_connections 2024-04-22T23:47:31.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.179+0000 7f7fb1a6e640 1 --2- 172.21.15.107:0/292041444 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f7f9406eb70 0x7f7f94071020 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:31.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.179+0000 7f7fb1a6e640 1 --2- 172.21.15.107:0/292041444 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fac101160 0x7f7fac10e440 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:31.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.179+0000 7f7fb1a6e640 1 --2- 172.21.15.107:0/292041444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7fac0fb130 0x7f7fac10df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:31.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.179+0000 7f7fb1a6e640 1 -- 172.21.15.107:0/292041444 >> 172.21.15.107:0/292041444 conn(0x7f7fac071190 msgr2=0x7f7fac0f94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:31.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.179+0000 7f7fb1a6e640 1 -- 172.21.15.107:0/292041444 shutdown_connections 2024-04-22T23:47:31.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.179+0000 7f7fb1a6e640 1 -- 172.21.15.107:0/292041444 wait complete. 2024-04-22T23:47:31.182 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T23:29:48.249903+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-22T23:32:19.078061+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-22T23:32:19.378987+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T23:47:31.182 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T23:47:31.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.288+0000 7f6f03aae640 1 -- 172.21.15.107:0/257194801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6efc101c90 msgr2=0x7f6efc0fb320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:31.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.288+0000 7f6f03aae640 1 --2- 172.21.15.107:0/257194801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6efc101c90 0x7f6efc0fb320 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6eec0048d0 tx=0x7f6eec009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:31.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.288+0000 7f6f03aae640 1 -- 172.21.15.107:0/257194801 shutdown_connections 2024-04-22T23:47:31.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.288+0000 7f6f03aae640 1 --2- 172.21.15.107:0/257194801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6efc101c90 0x7f6efc0fb320 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:31.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.288+0000 7f6f03aae640 1 --2- 172.21.15.107:0/257194801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6efc1012f0 0x7f6efc1016c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:31.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.288+0000 7f6f03aae640 1 -- 172.21.15.107:0/257194801 >> 172.21.15.107:0/257194801 conn(0x7f6efc0f6ec0 msgr2=0x7f6efc0f92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:31.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.288+0000 7f6f03aae640 1 -- 172.21.15.107:0/257194801 shutdown_connections 2024-04-22T23:47:31.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.288+0000 7f6f03aae640 1 -- 172.21.15.107:0/257194801 wait complete. 2024-04-22T23:47:31.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.288+0000 7f6f03aae640 1 Processor -- start 2024-04-22T23:47:31.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.288+0000 7f6f03aae640 1 -- start start 2024-04-22T23:47:31.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.288+0000 7f6f03aae640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6efc1012f0 0x7f6efc10e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:31.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.288+0000 7f6f03aae640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6efc101c90 0x7f6efc10e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:31.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.288+0000 7f6f03aae640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6efc066580 con 0x7f6efc1012f0 2024-04-22T23:47:31.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.288+0000 7f6f03aae640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6efc10eba0 con 0x7f6efc101c90 2024-04-22T23:47:31.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.288+0000 7f6f0184a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6efc1012f0 0x7f6efc10e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:31.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.288+0000 7f6f0184a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6efc1012f0 0x7f6efc10e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39024/0 (socket says 172.21.15.107:39024) 2024-04-22T23:47:31.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.288+0000 7f6f0184a640 1 -- 172.21.15.107:0/878190235 learned_addr learned my addr 172.21.15.107:0/878190235 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:47:31.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.289+0000 7f6f01049640 1 --2- 172.21.15.107:0/878190235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6efc101c90 0x7f6efc10e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:31.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.289+0000 7f6f0184a640 1 -- 172.21.15.107:0/878190235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6efc101c90 msgr2=0x7f6efc10e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:31.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.289+0000 7f6f0184a640 1 --2- 172.21.15.107:0/878190235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6efc101c90 0x7f6efc10e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:31.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.289+0000 7f6f0184a640 1 -- 172.21.15.107:0/878190235 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eec005040 con 0x7f6efc1012f0 2024-04-22T23:47:31.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.289+0000 7f6f0184a640 1 --2- 172.21.15.107:0/878190235 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6efc1012f0 0x7f6efc10e120 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6ef800c810 tx=0x7f6ef80043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:31.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.289+0000 7f6ef27fc640 1 -- 172.21.15.107:0/878190235 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ef800a820 con 0x7f6efc1012f0 2024-04-22T23:47:31.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.289+0000 7f6ef27fc640 1 -- 172.21.15.107:0/878190235 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6ef800ae50 con 0x7f6efc1012f0 2024-04-22T23:47:31.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.289+0000 7f6f03aae640 1 -- 172.21.15.107:0/878190235 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6efc10ee80 con 0x7f6efc1012f0 2024-04-22T23:47:31.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.289+0000 7f6ef27fc640 1 -- 172.21.15.107:0/878190235 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ef80115c0 con 0x7f6efc1012f0 2024-04-22T23:47:31.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.289+0000 7f6f03aae640 1 -- 172.21.15.107:0/878190235 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6efc117440 con 0x7f6efc1012f0 2024-04-22T23:47:31.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.290+0000 7f6f03aae640 1 -- 172.21.15.107:0/878190235 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6efc0fca60 con 0x7f6efc1012f0 2024-04-22T23:47:31.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.290+0000 7f6ef27fc640 1 -- 172.21.15.107:0/878190235 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f6ef800a980 con 0x7f6efc1012f0 2024-04-22T23:47:31.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.291+0000 7f6ef27fc640 1 --2- 172.21.15.107:0/878190235 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f6ee806eb70 0x7f6ee8071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:31.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.291+0000 7f6ef27fc640 1 -- 172.21.15.107:0/878190235 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f6ef808ed90 con 0x7f6efc1012f0 2024-04-22T23:47:31.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.291+0000 7f6f01049640 1 --2- 172.21.15.107:0/878190235 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f6ee806eb70 0x7f6ee8071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:31.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.291+0000 7f6f01049640 1 --2- 172.21.15.107:0/878190235 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f6ee806eb70 0x7f6ee8071020 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6efc065dd0 tx=0x7f6eec009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:31.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.296+0000 7f6ef27fc640 1 -- 172.21.15.107:0/878190235 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f6ef8057b10 con 0x7f6efc1012f0 2024-04-22T23:47:31.379 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:30 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/790399130' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-22T23:47:31.379 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:30 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/2923124994' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-22T23:47:31.379 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:30 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/2342895705' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-22T23:47:31.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.477+0000 7f6f03aae640 1 -- 172.21.15.107:0/878190235 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6efc065b80 con 0x7f6efc1012f0 2024-04-22T23:47:31.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.477+0000 7f6ef27fc640 1 -- 172.21.15.107:0/878190235 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6ef805b1d0 con 0x7f6efc1012f0 2024-04-22T23:47:31.469 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2024-04-22T23:47:31.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.479+0000 7f6f03aae640 1 -- 172.21.15.107:0/878190235 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f6ee806eb70 msgr2=0x7f6ee8071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:31.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.480+0000 7f6f03aae640 1 --2- 172.21.15.107:0/878190235 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f6ee806eb70 0x7f6ee8071020 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6efc065dd0 tx=0x7f6eec009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:31.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.480+0000 7f6f03aae640 1 -- 172.21.15.107:0/878190235 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6efc1012f0 msgr2=0x7f6efc10e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:31.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.480+0000 7f6f03aae640 1 --2- 172.21.15.107:0/878190235 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6efc1012f0 0x7f6efc10e120 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6ef800c810 tx=0x7f6ef80043c0 comp rx=0 tx=0).stop 2024-04-22T23:47:31.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.480+0000 7f6f03aae640 1 -- 172.21.15.107:0/878190235 shutdown_connections 2024-04-22T23:47:31.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.480+0000 7f6f03aae640 1 --2- 172.21.15.107:0/878190235 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f6ee806eb70 0x7f6ee8071020 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:31.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.480+0000 7f6f03aae640 1 --2- 172.21.15.107:0/878190235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6efc101c90 0x7f6efc10e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:31.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.480+0000 7f6f03aae640 1 --2- 172.21.15.107:0/878190235 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6efc1012f0 0x7f6efc10e120 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:31.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.480+0000 7f6f03aae640 1 -- 172.21.15.107:0/878190235 >> 172.21.15.107:0/878190235 conn(0x7f6efc0f6ec0 msgr2=0x7f6efc0f7f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:31.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.480+0000 7f6f03aae640 1 -- 172.21.15.107:0/878190235 shutdown_connections 2024-04-22T23:47:31.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.480+0000 7f6f03aae640 1 -- 172.21.15.107:0/878190235 wait complete. 2024-04-22T23:47:31.482 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-22T23:47:31.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.587+0000 7f5746221640 1 -- 172.21.15.107:0/3072644453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57401022a0 msgr2=0x7f5740102670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:31.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.587+0000 7f5746221640 1 --2- 172.21.15.107:0/3072644453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57401022a0 0x7f5740102670 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f57300048d0 tx=0x7f5730009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:31.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.587+0000 7f5746221640 1 -- 172.21.15.107:0/3072644453 shutdown_connections 2024-04-22T23:47:31.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.587+0000 7f5746221640 1 --2- 172.21.15.107:0/3072644453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57400fa8a0 0x7f57400fad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:31.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.587+0000 7f5746221640 1 --2- 172.21.15.107:0/3072644453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57401022a0 0x7f5740102670 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:31.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.587+0000 7f5746221640 1 -- 172.21.15.107:0/3072644453 >> 172.21.15.107:0/3072644453 conn(0x7f5740071760 msgr2=0x7f5740071b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:31.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.587+0000 7f5746221640 1 -- 172.21.15.107:0/3072644453 shutdown_connections 2024-04-22T23:47:31.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.587+0000 7f5746221640 1 -- 172.21.15.107:0/3072644453 wait complete. 2024-04-22T23:47:31.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.588+0000 7f5746221640 1 Processor -- start 2024-04-22T23:47:31.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.588+0000 7f5746221640 1 -- start start 2024-04-22T23:47:31.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.588+0000 7f5746221640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57400fa8a0 0x7f5740112da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:31.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.588+0000 7f5746221640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57401022a0 0x7f57401132e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:31.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.588+0000 7f5746221640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57401138e0 con 0x7f57400fa8a0 2024-04-22T23:47:31.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.588+0000 7f5746221640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574010cf30 con 0x7f57401022a0 2024-04-22T23:47:31.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.588+0000 7f573f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57400fa8a0 0x7f5740112da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:31.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.588+0000 7f573f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57400fa8a0 0x7f5740112da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33458/0 (socket says 172.21.15.107:33458) 2024-04-22T23:47:31.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.588+0000 7f573f7fe640 1 -- 172.21.15.107:0/1836107234 learned_addr learned my addr 172.21.15.107:0/1836107234 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:47:31.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.588+0000 7f573f7fe640 1 -- 172.21.15.107:0/1836107234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57401022a0 msgr2=0x7f57401132e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:31.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.588+0000 7f573f7fe640 1 --2- 172.21.15.107:0/1836107234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57401022a0 0x7f57401132e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:31.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.588+0000 7f573f7fe640 1 -- 172.21.15.107:0/1836107234 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5730005040 con 0x7f57400fa8a0 2024-04-22T23:47:31.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.589+0000 7f573f7fe640 1 --2- 172.21.15.107:0/1836107234 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57400fa8a0 0x7f5740112da0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5730031720 tx=0x7f5730009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:31.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.589+0000 7f5727fff640 1 -- 172.21.15.107:0/1836107234 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57300094c0 con 0x7f57400fa8a0 2024-04-22T23:47:31.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.589+0000 7f5746221640 1 -- 172.21.15.107:0/1836107234 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f574010d1b0 con 0x7f57400fa8a0 2024-04-22T23:47:31.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.589+0000 7f5727fff640 1 -- 172.21.15.107:0/1836107234 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5730032e50 con 0x7f57400fa8a0 2024-04-22T23:47:31.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.589+0000 7f5746221640 1 -- 172.21.15.107:0/1836107234 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f574010d6d0 con 0x7f57400fa8a0 2024-04-22T23:47:31.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.589+0000 7f5727fff640 1 -- 172.21.15.107:0/1836107234 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f573003dea0 con 0x7f57400fa8a0 2024-04-22T23:47:31.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.590+0000 7f5746221640 1 -- 172.21.15.107:0/1836107234 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5740065340 con 0x7f57400fa8a0 2024-04-22T23:47:31.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.590+0000 7f5727fff640 1 -- 172.21.15.107:0/1836107234 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f573002f070 con 0x7f57400fa8a0 2024-04-22T23:47:31.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.591+0000 7f5727fff640 1 --2- 172.21.15.107:0/1836107234 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f572806eb70 0x7f5728071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:31.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.591+0000 7f5727fff640 1 -- 172.21.15.107:0/1836107234 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f57300b0e30 con 0x7f57400fa8a0 2024-04-22T23:47:31.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.591+0000 7f573effd640 1 --2- 172.21.15.107:0/1836107234 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f572806eb70 0x7f5728071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:31.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.591+0000 7f573effd640 1 --2- 172.21.15.107:0/1836107234 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f572806eb70 0x7f5728071020 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f574010e660 tx=0x7f5734007cb0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:31.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.595+0000 7f5727fff640 1 -- 172.21.15.107:0/1836107234 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f5730079b30 con 0x7f57400fa8a0 2024-04-22T23:47:31.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.773+0000 7f5746221640 1 -- 172.21.15.107:0/1836107234 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f5740109480 con 0x7f57400fa8a0 2024-04-22T23:47:31.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.774+0000 7f5727fff640 1 -- 172.21.15.107:0/1836107234 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v83) v1 ==== 107+0+11 (secure 0 0 0) 0x7f573007d1f0 con 0x7f57400fa8a0 2024-04-22T23:47:31.766 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-04-22T23:47:31.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.776+0000 7f5746221640 1 -- 172.21.15.107:0/1836107234 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f572806eb70 msgr2=0x7f5728071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:31.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.776+0000 7f5746221640 1 --2- 172.21.15.107:0/1836107234 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f572806eb70 0x7f5728071020 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f574010e660 tx=0x7f5734007cb0 comp rx=0 tx=0).stop 2024-04-22T23:47:31.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.776+0000 7f5746221640 1 -- 172.21.15.107:0/1836107234 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57400fa8a0 msgr2=0x7f5740112da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:31.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.776+0000 7f5746221640 1 --2- 172.21.15.107:0/1836107234 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57400fa8a0 0x7f5740112da0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5730031720 tx=0x7f5730009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:31.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.776+0000 7f5746221640 1 -- 172.21.15.107:0/1836107234 shutdown_connections 2024-04-22T23:47:31.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.776+0000 7f5746221640 1 --2- 172.21.15.107:0/1836107234 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f572806eb70 0x7f5728071020 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:31.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.776+0000 7f5746221640 1 --2- 172.21.15.107:0/1836107234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f57401022a0 0x7f57401132e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:31.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.776+0000 7f5746221640 1 --2- 172.21.15.107:0/1836107234 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57400fa8a0 0x7f5740112da0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:31.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.776+0000 7f5746221640 1 -- 172.21.15.107:0/1836107234 >> 172.21.15.107:0/1836107234 conn(0x7f5740071760 msgr2=0x7f57400f9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:31.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.776+0000 7f5746221640 1 -- 172.21.15.107:0/1836107234 shutdown_connections 2024-04-22T23:47:31.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.776+0000 7f5746221640 1 -- 172.21.15.107:0/1836107234 wait complete. 2024-04-22T23:47:31.779 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-22T23:47:31.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.884+0000 7ff5a9d9d640 1 -- 172.21.15.107:0/4132649400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5a41021d0 msgr2=0x7ff5a41025a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:31.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.884+0000 7ff5a9d9d640 1 --2- 172.21.15.107:0/4132649400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5a41021d0 0x7ff5a41025a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff5940048d0 tx=0x7ff594009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:31.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.884+0000 7ff5a9d9d640 1 -- 172.21.15.107:0/4132649400 shutdown_connections 2024-04-22T23:47:31.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.884+0000 7ff5a9d9d640 1 --2- 172.21.15.107:0/4132649400 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5a4102b70 0x7ff5a40fc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:31.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.884+0000 7ff5a9d9d640 1 --2- 172.21.15.107:0/4132649400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5a41021d0 0x7ff5a41025a0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:31.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.884+0000 7ff5a9d9d640 1 -- 172.21.15.107:0/4132649400 >> 172.21.15.107:0/4132649400 conn(0x7ff5a40f7dc0 msgr2=0x7ff5a40fa1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:31.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.884+0000 7ff5a9d9d640 1 -- 172.21.15.107:0/4132649400 shutdown_connections 2024-04-22T23:47:31.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.884+0000 7ff5a9d9d640 1 -- 172.21.15.107:0/4132649400 wait complete. 2024-04-22T23:47:31.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.885+0000 7ff5a9d9d640 1 Processor -- start 2024-04-22T23:47:31.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.885+0000 7ff5a9d9d640 1 -- start start 2024-04-22T23:47:31.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.885+0000 7ff5a9d9d640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5a41021d0 0x7ff5a4112d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:31.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.885+0000 7ff5a9d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5a4102b70 0x7ff5a4113260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:31.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.885+0000 7ff5a9d9d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a4113860 con 0x7ff5a4102b70 2024-04-22T23:47:31.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.885+0000 7ff5a9d9d640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a410ceb0 con 0x7ff5a41021d0 2024-04-22T23:47:31.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.885+0000 7ff59bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5a4102b70 0x7ff5a4113260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:31.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.885+0000 7ff59bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5a4102b70 0x7ff5a4113260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33478/0 (socket says 172.21.15.107:33478) 2024-04-22T23:47:31.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.885+0000 7ff59bfff640 1 -- 172.21.15.107:0/1651244818 learned_addr learned my addr 172.21.15.107:0/1651244818 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:47:31.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.885+0000 7ff5a37fe640 1 --2- 172.21.15.107:0/1651244818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5a41021d0 0x7ff5a4112d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:31.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.885+0000 7ff59bfff640 1 -- 172.21.15.107:0/1651244818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5a41021d0 msgr2=0x7ff5a4112d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:31.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.885+0000 7ff59bfff640 1 --2- 172.21.15.107:0/1651244818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5a41021d0 0x7ff5a4112d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:31.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.885+0000 7ff59bfff640 1 -- 172.21.15.107:0/1651244818 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff594005040 con 0x7ff5a4102b70 2024-04-22T23:47:31.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.885+0000 7ff5a37fe640 1 --2- 172.21.15.107:0/1651244818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5a41021d0 0x7ff5a4112d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T23:47:31.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.886+0000 7ff59bfff640 1 --2- 172.21.15.107:0/1651244818 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5a4102b70 0x7ff5a4113260 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff58c002a50 tx=0x7ff58c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:31.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.886+0000 7ff5a0ff9640 1 -- 172.21.15.107:0/1651244818 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff58c007c30 con 0x7ff5a4102b70 2024-04-22T23:47:31.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.886+0000 7ff5a0ff9640 1 -- 172.21.15.107:0/1651244818 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff58c00f040 con 0x7ff5a4102b70 2024-04-22T23:47:31.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.886+0000 7ff5a9d9d640 1 -- 172.21.15.107:0/1651244818 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5a410d190 con 0x7ff5a4102b70 2024-04-22T23:47:31.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.886+0000 7ff5a0ff9640 1 -- 172.21.15.107:0/1651244818 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff58c00db50 con 0x7ff5a4102b70 2024-04-22T23:47:31.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.886+0000 7ff5a9d9d640 1 -- 172.21.15.107:0/1651244818 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5a410d6b0 con 0x7ff5a4102b70 2024-04-22T23:47:31.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.887+0000 7ff5a9d9d640 1 -- 172.21.15.107:0/1651244818 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5a40fd940 con 0x7ff5a4102b70 2024-04-22T23:47:31.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.887+0000 7ff5a0ff9640 1 -- 172.21.15.107:0/1651244818 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7ff58c01c400 con 0x7ff5a4102b70 2024-04-22T23:47:31.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.887+0000 7ff5a0ff9640 1 --2- 172.21.15.107:0/1651244818 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7ff59006eb70 0x7ff590071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:31.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.888+0000 7ff5a0ff9640 1 -- 172.21.15.107:0/1651244818 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7ff58c08e240 con 0x7ff5a4102b70 2024-04-22T23:47:31.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.888+0000 7ff5a37fe640 1 --2- 172.21.15.107:0/1651244818 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7ff59006eb70 0x7ff590071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:31.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.888+0000 7ff5a37fe640 1 --2- 172.21.15.107:0/1651244818 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7ff59006eb70 0x7ff590071020 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff594000c00 tx=0x7ff594009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:31.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:31.892+0000 7ff5a0ff9640 1 -- 172.21.15.107:0/1651244818 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7ff58c056f40 con 0x7ff5a4102b70 2024-04-22T23:47:32.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.070+0000 7ff5a9d9d640 1 -- 172.21.15.107:0/1651244818 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7ff5a410e390 con 0x7ff5a4102b70 2024-04-22T23:47:32.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.071+0000 7ff5a0ff9640 1 -- 172.21.15.107:0/1651244818 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v83) v1 ==== 107+0+11 (secure 0 0 0) 0x7ff58c05a600 con 0x7ff5a4102b70 2024-04-22T23:47:32.063 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-04-22T23:47:32.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.073+0000 7ff5a9d9d640 1 -- 172.21.15.107:0/1651244818 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7ff59006eb70 msgr2=0x7ff590071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:32.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.073+0000 7ff5a9d9d640 1 --2- 172.21.15.107:0/1651244818 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7ff59006eb70 0x7ff590071020 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff594000c00 tx=0x7ff594009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:32.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.073+0000 7ff5a9d9d640 1 -- 172.21.15.107:0/1651244818 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5a4102b70 msgr2=0x7ff5a4113260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:32.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.073+0000 7ff5a9d9d640 1 --2- 172.21.15.107:0/1651244818 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5a4102b70 0x7ff5a4113260 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff58c002a50 tx=0x7ff58c004600 comp rx=0 tx=0).stop 2024-04-22T23:47:32.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.073+0000 7ff5a9d9d640 1 -- 172.21.15.107:0/1651244818 shutdown_connections 2024-04-22T23:47:32.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.073+0000 7ff5a9d9d640 1 --2- 172.21.15.107:0/1651244818 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7ff59006eb70 0x7ff590071020 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:32.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.073+0000 7ff5a9d9d640 1 --2- 172.21.15.107:0/1651244818 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5a4102b70 0x7ff5a4113260 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:32.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.073+0000 7ff5a9d9d640 1 --2- 172.21.15.107:0/1651244818 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5a41021d0 0x7ff5a4112d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:32.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.073+0000 7ff5a9d9d640 1 -- 172.21.15.107:0/1651244818 >> 172.21.15.107:0/1651244818 conn(0x7ff5a40f7dc0 msgr2=0x7ff5a40f8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:32.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.073+0000 7ff5a9d9d640 1 -- 172.21.15.107:0/1651244818 shutdown_connections 2024-04-22T23:47:32.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.073+0000 7ff5a9d9d640 1 -- 172.21.15.107:0/1651244818 wait complete. 2024-04-22T23:47:32.074 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:31 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/292041444' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T23:47:32.075 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:31 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/878190235' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T23:47:32.075 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:31 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/1836107234' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-22T23:47:32.075 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:31 smithi107 ceph-mon[90179]: pgmap v325: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:47:32.076 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-22T23:47:32.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.178+0000 7f998da14640 1 -- 172.21.15.107:0/3790373574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99880fb120 msgr2=0x7f99880fb570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:32.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.178+0000 7f998da14640 1 --2- 172.21.15.107:0/3790373574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99880fb120 0x7f99880fb570 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f99700048d0 tx=0x7f9970009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:32.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.179+0000 7f998da14640 1 -- 172.21.15.107:0/3790373574 shutdown_connections 2024-04-22T23:47:32.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.179+0000 7f998da14640 1 --2- 172.21.15.107:0/3790373574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99880fb120 0x7f99880fb570 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:32.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.179+0000 7f998da14640 1 --2- 172.21.15.107:0/3790373574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9988101150 0x7f9988101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:32.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.179+0000 7f998da14640 1 -- 172.21.15.107:0/3790373574 >> 172.21.15.107:0/3790373574 conn(0x7f99880710c0 msgr2=0x7f99880714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:32.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.179+0000 7f998da14640 1 -- 172.21.15.107:0/3790373574 shutdown_connections 2024-04-22T23:47:32.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.179+0000 7f998da14640 1 -- 172.21.15.107:0/3790373574 wait complete. 2024-04-22T23:47:32.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.179+0000 7f998da14640 1 Processor -- start 2024-04-22T23:47:32.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.179+0000 7f998da14640 1 -- start start 2024-04-22T23:47:32.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.179+0000 7f998da14640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99880fb120 0x7f9988111bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:32.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.179+0000 7f998da14640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9988101150 0x7f9988112130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:32.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.179+0000 7f998da14640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9988112730 con 0x7f9988101150 2024-04-22T23:47:32.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.179+0000 7f998da14640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f998810bd80 con 0x7f99880fb120 2024-04-22T23:47:32.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.179+0000 7f997ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9988101150 0x7f9988112130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:32.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.179+0000 7f997ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9988101150 0x7f9988112130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33498/0 (socket says 172.21.15.107:33498) 2024-04-22T23:47:32.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.179+0000 7f997ffff640 1 -- 172.21.15.107:0/3434660299 learned_addr learned my addr 172.21.15.107:0/3434660299 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-22T23:47:32.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.180+0000 7f9986ffd640 1 --2- 172.21.15.107:0/3434660299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99880fb120 0x7f9988111bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:32.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.180+0000 7f997ffff640 1 -- 172.21.15.107:0/3434660299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99880fb120 msgr2=0x7f9988111bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:32.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.180+0000 7f997ffff640 1 --2- 172.21.15.107:0/3434660299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99880fb120 0x7f9988111bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:32.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.180+0000 7f997ffff640 1 -- 172.21.15.107:0/3434660299 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9970005040 con 0x7f9988101150 2024-04-22T23:47:32.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.180+0000 7f997ffff640 1 --2- 172.21.15.107:0/3434660299 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9988101150 0x7f9988112130 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f99700041d0 tx=0x7f9970009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:32.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.180+0000 7f997f7fe640 1 -- 172.21.15.107:0/3434660299 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9970030870 con 0x7f9988101150 2024-04-22T23:47:32.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.180+0000 7f997f7fe640 1 -- 172.21.15.107:0/3434660299 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f99700092e0 con 0x7f9988101150 2024-04-22T23:47:32.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.180+0000 7f998da14640 1 -- 172.21.15.107:0/3434660299 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f998810c000 con 0x7f9988101150 2024-04-22T23:47:32.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.180+0000 7f997f7fe640 1 -- 172.21.15.107:0/3434660299 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9970043400 con 0x7f9988101150 2024-04-22T23:47:32.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.180+0000 7f998da14640 1 -- 172.21.15.107:0/3434660299 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f998810c520 con 0x7f9988101150 2024-04-22T23:47:32.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.181+0000 7f998da14640 1 -- 172.21.15.107:0/3434660299 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99880fc810 con 0x7f9988101150 2024-04-22T23:47:32.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.181+0000 7f997f7fe640 1 -- 172.21.15.107:0/3434660299 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 37) v1 ==== 92747+0+0 (secure 0 0 0) 0x7f997003b420 con 0x7f9988101150 2024-04-22T23:47:32.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.182+0000 7f997f7fe640 1 --2- 172.21.15.107:0/3434660299 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f997406eb70 0x7f9974071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T23:47:32.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.182+0000 7f997f7fe640 1 -- 172.21.15.107:0/3434660299 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f99700b15e0 con 0x7f9988101150 2024-04-22T23:47:32.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.182+0000 7f9986ffd640 1 --2- 172.21.15.107:0/3434660299 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f997406eb70 0x7f9974071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T23:47:32.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.182+0000 7f9986ffd640 1 --2- 172.21.15.107:0/3434660299 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f997406eb70 0x7f9974071020 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9978004ce0 tx=0x7f997800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T23:47:32.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.186+0000 7f997f7fe640 1 -- 172.21.15.107:0/3434660299 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188728 (secure 0 0 0) 0x7f997007a4c0 con 0x7f9988101150 2024-04-22T23:47:32.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.375+0000 7f998da14640 1 -- 172.21.15.107:0/3434660299 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f998810d290 con 0x7f9988101150 2024-04-22T23:47:32.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.375+0000 7f997f7fe640 1 -- 172.21.15.107:0/3434660299 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 43 entries v83) v1 ==== 81+0+2584 (secure 0 0 0) 0x7f997007db80 con 0x7f9988101150 2024-04-22T23:47:32.367 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.157:6827/4072444199 2024-04-23T23:44:07.204437+0000 2024-04-22T23:47:32.367 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.157:6826/4072444199 2024-04-23T23:44:07.204437+0000 2024-04-22T23:47:32.367 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3783622140 2024-04-23T23:37:55.631534+0000 2024-04-22T23:47:32.367 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3055151095 2024-04-23T23:37:55.631534+0000 2024-04-22T23:47:32.368 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3377956937 2024-04-23T23:25:58.816560+0000 2024-04-22T23:47:32.368 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/2659503121 2024-04-23T23:36:18.020700+0000 2024-04-22T23:47:32.368 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.157:6829/3332906296 2024-04-23T23:37:10.884172+0000 2024-04-22T23:47:32.368 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/307474287 2024-04-23T23:26:29.807373+0000 2024-04-22T23:47:32.368 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6829/3335313253 2024-04-23T23:43:26.926970+0000 2024-04-22T23:47:32.368 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/467262774 2024-04-23T23:25:58.816560+0000 2024-04-22T23:47:32.368 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/663073386 2024-04-23T23:26:29.807373+0000 2024-04-22T23:47:32.368 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.157:0/2260053045 2024-04-23T23:37:10.884172+0000 2024-04-22T23:47:32.368 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/525119004 2024-04-23T23:36:18.020700+0000 2024-04-22T23:47:32.368 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/307474287 2024-04-23T23:26:29.807373+0000 2024-04-22T23:47:32.368 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/1065200386 2024-04-23T23:36:18.020700+0000 2024-04-22T23:47:32.368 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/3427816626 2024-04-23T23:27:33.931895+0000 2024-04-22T23:47:32.368 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3920905056 2024-04-23T23:25:58.816560+0000 2024-04-22T23:47:32.369 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/1734120985 2024-04-23T23:27:33.931895+0000 2024-04-22T23:47:32.369 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.157:0/278207195 2024-04-23T23:37:10.884172+0000 2024-04-22T23:47:32.369 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/2444490678 2024-04-23T23:26:29.807373+0000 2024-04-22T23:47:32.369 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3567543485 2024-04-23T23:36:18.020700+0000 2024-04-22T23:47:32.369 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.157:6828/3332906296 2024-04-23T23:37:10.884172+0000 2024-04-22T23:47:32.369 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/1205692115 2024-04-23T23:27:33.931895+0000 2024-04-22T23:47:32.369 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/467262774 2024-04-23T23:25:58.816560+0000 2024-04-22T23:47:32.369 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/2168608506 2024-04-23T23:37:55.631534+0000 2024-04-22T23:47:32.369 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/2516490111 2024-04-23T23:36:18.020700+0000 2024-04-22T23:47:32.369 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.157:0/3173482279 2024-04-23T23:37:10.884172+0000 2024-04-22T23:47:32.369 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/2516490111 2024-04-23T23:36:18.020700+0000 2024-04-22T23:47:32.369 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.157:0/3140121750 2024-04-23T23:37:10.884172+0000 2024-04-22T23:47:32.369 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/1029440409 2024-04-23T23:26:29.807373+0000 2024-04-22T23:47:32.369 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6826/4144299942 2024-04-23T23:32:35.769448+0000 2024-04-22T23:47:32.370 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6828/3335313253 2024-04-23T23:43:26.926970+0000 2024-04-22T23:47:32.370 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6827/4144299942 2024-04-23T23:32:35.769448+0000 2024-04-22T23:47:32.370 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/3427816626 2024-04-23T23:27:33.931895+0000 2024-04-22T23:47:32.370 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.157:0/2684888571 2024-04-23T23:37:10.884172+0000 2024-04-22T23:47:32.370 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/2873032378 2024-04-23T23:37:55.631534+0000 2024-04-22T23:47:32.370 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3810721822 2024-04-23T23:25:58.816560+0000 2024-04-22T23:47:32.370 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/1888554706 2024-04-23T23:37:55.631534+0000 2024-04-22T23:47:32.370 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3608791958 2024-04-23T23:27:33.931895+0000 2024-04-22T23:47:32.370 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.157:0/2452797907 2024-04-23T23:37:10.884172+0000 2024-04-22T23:47:32.370 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3843391359 2024-04-23T23:37:55.631534+0000 2024-04-22T23:47:32.370 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/2362628021 2024-04-23T23:37:55.631534+0000 2024-04-22T23:47:32.370 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/2362628021 2024-04-23T23:37:55.631534+0000 2024-04-22T23:47:32.371 INFO:teuthology.orchestra.run.smithi107.stderr:listed 43 entries 2024-04-22T23:47:32.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.377+0000 7f998da14640 1 -- 172.21.15.107:0/3434660299 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f997406eb70 msgr2=0x7f9974071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:32.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.377+0000 7f998da14640 1 --2- 172.21.15.107:0/3434660299 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f997406eb70 0x7f9974071020 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9978004ce0 tx=0x7f997800b040 comp rx=0 tx=0).stop 2024-04-22T23:47:32.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.377+0000 7f998da14640 1 -- 172.21.15.107:0/3434660299 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9988101150 msgr2=0x7f9988112130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T23:47:32.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.377+0000 7f998da14640 1 --2- 172.21.15.107:0/3434660299 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9988101150 0x7f9988112130 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f99700041d0 tx=0x7f9970009da0 comp rx=0 tx=0).stop 2024-04-22T23:47:32.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.377+0000 7f998da14640 1 -- 172.21.15.107:0/3434660299 shutdown_connections 2024-04-22T23:47:32.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.377+0000 7f998da14640 1 --2- 172.21.15.107:0/3434660299 >> [v2:172.21.15.107:6800/762611949,v1:172.21.15.107:6801/762611949] conn(0x7f997406eb70 0x7f9974071020 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:32.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.377+0000 7f998da14640 1 --2- 172.21.15.107:0/3434660299 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9988101150 0x7f9988112130 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:32.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.377+0000 7f998da14640 1 --2- 172.21.15.107:0/3434660299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99880fb120 0x7f9988111bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T23:47:32.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.377+0000 7f998da14640 1 -- 172.21.15.107:0/3434660299 >> 172.21.15.107:0/3434660299 conn(0x7f99880710c0 msgr2=0x7f99880f9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T23:47:32.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.377+0000 7f998da14640 1 -- 172.21.15.107:0/3434660299 shutdown_connections 2024-04-22T23:47:32.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-22T23:47:32.377+0000 7f998da14640 1 -- 172.21.15.107:0/3434660299 wait complete. 2024-04-22T23:47:32.379 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:31 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/292041444' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T23:47:32.379 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:31 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/878190235' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T23:47:32.379 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:31 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/1836107234' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-22T23:47:32.379 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:31 smithi157 ceph-mon[79403]: pgmap v325: 65 pgs: 65 active+clean; 165 MiB data, 733 MiB used, 536 GiB / 536 GiB avail 2024-04-22T23:47:32.380 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-22T23:47:32.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T23:47:32.380 DEBUG:teuthology.orchestra.run.smithi157:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-22T23:47:32.415 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-04-22T23:47:32.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T23:47:32.415 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-22T23:47:32.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T23:47:32.492 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list 2024-04-22T23:47:32.507 INFO:teuthology.orchestra.run.smithi107.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-22T23:47:32.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T23:47:32.508 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-22T23:47:32.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T23:47:32.573 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link delete ceph-brx 2024-04-22T23:47:32.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T23:47:32.675 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-04-22T23:47:32.693 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-22T23:47:32.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T23:47:32.694 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-22T23:47:32.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T23:47:32.762 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link delete ceph-brx 2024-04-22T23:47:32.859 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-22T23:47:32.871 INFO:tasks.cephadm:Teardown begin 2024-04-22T23:47:32.871 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-22T23:47:32.897 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-22T23:47:32.924 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-22T23:47:32.924 DEBUG:teuthology.orchestra.run.smithi107:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-22T23:47:32.952 DEBUG:teuthology.orchestra.run.smithi157:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-22T23:47:32.980 INFO:tasks.cephadm:Stopping all daemons... 2024-04-22T23:47:32.980 INFO:tasks.cephadm.mon.smithi107:Stopping mon.smithi107... 2024-04-22T23:47:32.981 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-8252f6f2-00ff-11ef-bc93-c7b262605968@mon.smithi107 2024-04-22T23:47:33.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:32 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/1651244818' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-22T23:47:33.129 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:32 smithi157 ceph-mon[79403]: from='client.? 172.21.15.107:0/3434660299' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-22T23:47:33.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:32 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/1651244818' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-22T23:47:33.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:32 smithi107 ceph-mon[90179]: from='client.? 172.21.15.107:0/3434660299' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-22T23:47:33.231 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:33 smithi107 systemd[1]: Stopping Ceph mon.smithi107 for 8252f6f2-00ff-11ef-bc93-c7b262605968... 2024-04-22T23:47:33.676 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:33 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-mon-smithi107[90175]: 2024-04-22T23:47:33.317+0000 7f2942126700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi107 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T23:47:33.676 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:33 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-mon-smithi107[90175]: 2024-04-22T23:47:33.317+0000 7f2942126700 -1 mon.smithi107@0(leader) e3 *** Got Signal Terminated *** 2024-04-22T23:47:33.676 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:33 smithi107 podman[144520]: 2024-04-22 23:47:33.419987903 +0000 UTC m=+0.237335402 container died bc72bbbe2c399f31aa0a3d630e519517ed8839fd6fe6e73777b79263f7812193 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-mon-smithi107, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-22T23:47:33.980 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:33 smithi107 podman[144520]: 2024-04-22 23:47:33.784400506 +0000 UTC m=+0.601748010 container cleanup bc72bbbe2c399f31aa0a3d630e519517ed8839fd6fe6e73777b79263f7812193 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-mon-smithi107, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T23:47:33.981 INFO:journalctl@ceph.mon.smithi107.smithi107.stdout:Apr 22 23:47:33 smithi107 bash[144520]: ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-mon-smithi107 2024-04-22T23:47:34.220 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8252f6f2-00ff-11ef-bc93-c7b262605968@mon.smithi107.service' 2024-04-22T23:47:34.269 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T23:47:34.269 INFO:tasks.cephadm.mon.smithi107:Stopped mon.smithi107 2024-04-22T23:47:34.269 INFO:tasks.cephadm.mon.smithi157:Stopping mon.smithi157... 2024-04-22T23:47:34.269 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-8252f6f2-00ff-11ef-bc93-c7b262605968@mon.smithi157 2024-04-22T23:47:34.425 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:34 smithi157 systemd[1]: Stopping Ceph mon.smithi157 for 8252f6f2-00ff-11ef-bc93-c7b262605968... 2024-04-22T23:47:34.879 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:34 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-mon-smithi157[79398]: 2024-04-22T23:47:34.681+0000 7fe4ebbac700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi157 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T23:47:34.879 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:34 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-mon-smithi157[79398]: 2024-04-22T23:47:34.681+0000 7fe4ebbac700 -1 mon.smithi157@1(peon) e3 *** Got Signal Terminated *** 2024-04-22T23:47:35.282 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:34 smithi157 podman[115537]: 2024-04-22 23:47:34.925834221 +0000 UTC m=+0.440033969 container died ea88de8be4fe435a0feb9f285fb754e48d54b8968e6b04287acea8a05fb49275 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-mon-smithi157, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-22T23:47:35.629 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:35 smithi157 podman[115537]: 2024-04-22 23:47:35.498846794 +0000 UTC m=+1.013046573 container cleanup ea88de8be4fe435a0feb9f285fb754e48d54b8968e6b04287acea8a05fb49275 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-mon-smithi157, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-22T23:47:35.629 INFO:journalctl@ceph.mon.smithi157.smithi157.stdout:Apr 22 23:47:35 smithi157 bash[115537]: ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-mon-smithi157 2024-04-22T23:47:35.909 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8252f6f2-00ff-11ef-bc93-c7b262605968@mon.smithi157.service' 2024-04-22T23:47:35.965 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T23:47:35.965 INFO:tasks.cephadm.mon.smithi157:Stopped mon.smithi157 2024-04-22T23:47:35.965 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-22T23:47:35.965 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-8252f6f2-00ff-11ef-bc93-c7b262605968@osd.0 2024-04-22T23:47:36.299 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:47:36 smithi107 systemd[1]: Stopping Ceph osd.0 for 8252f6f2-00ff-11ef-bc93-c7b262605968... 2024-04-22T23:47:36.731 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:47:36 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-0[98610]: 2024-04-22T23:47:36.307+0000 7fe198af7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T23:47:36.731 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:47:36 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-0[98610]: 2024-04-22T23:47:36.307+0000 7fe198af7700 -1 osd.0 83 *** Got signal Terminated *** 2024-04-22T23:47:36.731 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:47:36 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-0[98610]: 2024-04-22T23:47:36.307+0000 7fe198af7700 -1 osd.0 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T23:47:42.139 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:47:41 smithi107 podman[144653]: 2024-04-22 23:47:41.849648049 +0000 UTC m=+5.682448428 container died fd0ba8ef0c2748d981390c273ce32b055ed4923db55a95004e5f33783271fb18 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-adk-testing-2024-04-22-1618-43be020) 2024-04-22T23:47:42.481 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:47:42 smithi107 podman[144653]: 2024-04-22 23:47:42.246845448 +0000 UTC m=+6.079645825 container cleanup fd0ba8ef0c2748d981390c273ce32b055ed4923db55a95004e5f33783271fb18 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-0, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-22T23:47:42.481 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:47:42 smithi107 bash[144653]: ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-0 2024-04-22T23:47:42.855 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:47:42 smithi107 podman[144670]: 2024-04-22 23:47:42.548454672 +0000 UTC m=+0.690329085 container remove fd0ba8ef0c2748d981390c273ce32b055ed4923db55a95004e5f33783271fb18 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-0, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T23:47:43.176 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:47:42 smithi107 podman[144759]: 2024-04-22 23:47:42.76700258 +0000 UTC m=+0.018907498 image pull a0434699c1d76af4eea1fc95ff0a22ea73d1e04e609464fbf658d6f39bc7f2fc quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c 2024-04-22T23:47:43.176 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:47:42 smithi107 podman[144759]: 2024-04-22 23:47:42.882639243 +0000 UTC m=+0.134544142 container create 0efed564ef936b5c8e8b86480046d9616eb73637b4cf49db50a202a65bc6e1d9 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T23:47:43.176 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:47:43 smithi107 podman[144759]: 2024-04-22 23:47:43.110625187 +0000 UTC m=+0.362530105 container init 0efed564ef936b5c8e8b86480046d9616eb73637b4cf49db50a202a65bc6e1d9 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-22T23:47:43.176 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:47:43 smithi107 podman[144759]: 2024-04-22 23:47:43.114145147 +0000 UTC m=+0.366050048 container start 0efed564ef936b5c8e8b86480046d9616eb73637b4cf49db50a202a65bc6e1d9 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0) 2024-04-22T23:47:43.176 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:47:43 smithi107 podman[144759]: 2024-04-22 23:47:43.185119456 +0000 UTC m=+0.437024365 container attach 0efed564ef936b5c8e8b86480046d9616eb73637b4cf49db50a202a65bc6e1d9 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True) 2024-04-22T23:47:43.481 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 22 23:47:43 smithi107 podman[144759]: 2024-04-22 23:47:43.405710398 +0000 UTC m=+0.657615300 container died 0efed564ef936b5c8e8b86480046d9616eb73637b4cf49db50a202a65bc6e1d9 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-0-deactivate, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-22T23:47:44.000 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8252f6f2-00ff-11ef-bc93-c7b262605968@osd.0.service' 2024-04-22T23:47:44.043 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T23:47:44.044 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-22T23:47:44.044 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-22T23:47:44.044 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-8252f6f2-00ff-11ef-bc93-c7b262605968@osd.1 2024-04-22T23:47:44.231 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:47:44 smithi107 systemd[1]: Stopping Ceph osd.1 for 8252f6f2-00ff-11ef-bc93-c7b262605968... 2024-04-22T23:47:44.731 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:47:44 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-1[104313]: 2024-04-22T23:47:44.408+0000 7f6e27b1b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T23:47:44.731 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:47:44 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-1[104313]: 2024-04-22T23:47:44.408+0000 7f6e27b1b700 -1 osd.1 83 *** Got signal Terminated *** 2024-04-22T23:47:44.731 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:47:44 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-1[104313]: 2024-04-22T23:47:44.408+0000 7f6e27b1b700 -1 osd.1 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T23:47:49.981 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:47:49 smithi107 podman[144864]: 2024-04-22 23:47:49.739760007 +0000 UTC m=+5.463038185 container died 77e33b6122e04756b221502e60067adc62348b80981f89757eb58f775365ed1d (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-1, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T23:47:50.274 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:47:50 smithi107 podman[144864]: 2024-04-22 23:47:50.083392404 +0000 UTC m=+5.806670588 container cleanup 77e33b6122e04756b221502e60067adc62348b80981f89757eb58f775365ed1d (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-1, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T23:47:50.275 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:47:50 smithi107 bash[144864]: ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-1 2024-04-22T23:47:50.686 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:47:50 smithi107 podman[144892]: 2024-04-22 23:47:50.2838033 +0000 UTC m=+0.542373097 container remove 77e33b6122e04756b221502e60067adc62348b80981f89757eb58f775365ed1d (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-1, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-22T23:47:50.981 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:47:50 smithi107 podman[144987]: 2024-04-22 23:47:50.598959179 +0000 UTC m=+0.018275309 image pull a0434699c1d76af4eea1fc95ff0a22ea73d1e04e609464fbf658d6f39bc7f2fc quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c 2024-04-22T23:47:50.981 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:47:50 smithi107 podman[144987]: 2024-04-22 23:47:50.718981102 +0000 UTC m=+0.138297254 container create 7ae3a2d07ec32ea775b17b86b0a2178dfbcb268157e38cbb60ec175ac3fce3a1 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0) 2024-04-22T23:47:50.981 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:47:50 smithi107 podman[144987]: 2024-04-22 23:47:50.938260955 +0000 UTC m=+0.357577065 container init 7ae3a2d07ec32ea775b17b86b0a2178dfbcb268157e38cbb60ec175ac3fce3a1 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T23:47:50.981 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:47:50 smithi107 podman[144987]: 2024-04-22 23:47:50.941361764 +0000 UTC m=+0.360677884 container start 7ae3a2d07ec32ea775b17b86b0a2178dfbcb268157e38cbb60ec175ac3fce3a1 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T23:47:51.269 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:47:51 smithi107 podman[144987]: 2024-04-22 23:47:51.027203635 +0000 UTC m=+0.446519745 container attach 7ae3a2d07ec32ea775b17b86b0a2178dfbcb268157e38cbb60ec175ac3fce3a1 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-1-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240416) 2024-04-22T23:47:51.269 INFO:journalctl@ceph.osd.1.smithi107.stdout:Apr 22 23:47:51 smithi107 podman[144987]: 2024-04-22 23:47:51.245590117 +0000 UTC m=+0.664906230 container died 7ae3a2d07ec32ea775b17b86b0a2178dfbcb268157e38cbb60ec175ac3fce3a1 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-1-deactivate, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T23:47:51.867 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8252f6f2-00ff-11ef-bc93-c7b262605968@osd.1.service' 2024-04-22T23:47:51.905 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T23:47:51.905 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-22T23:47:51.906 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-22T23:47:51.906 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-8252f6f2-00ff-11ef-bc93-c7b262605968@osd.2 2024-04-22T23:47:52.230 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:47:51 smithi107 systemd[1]: Stopping Ceph osd.2 for 8252f6f2-00ff-11ef-bc93-c7b262605968... 2024-04-22T23:47:52.730 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:47:52 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-2[109829]: 2024-04-22T23:47:52.243+0000 7fa802d98700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T23:47:52.731 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:47:52 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-2[109829]: 2024-04-22T23:47:52.243+0000 7fa802d98700 -1 osd.2 83 *** Got signal Terminated *** 2024-04-22T23:47:52.731 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:47:52 smithi107 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-2[109829]: 2024-04-22T23:47:52.243+0000 7fa802d98700 -1 osd.2 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T23:47:57.870 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:47:57 smithi107 podman[145093]: 2024-04-22 23:47:57.580059732 +0000 UTC m=+5.449142448 container died c307bd8cdcfcf2460761891e53a002ded7355fac92d94fe6311f36fd97444d85 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-22T23:47:58.203 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:47:57 smithi107 podman[145093]: 2024-04-22 23:47:57.97790599 +0000 UTC m=+5.846988696 container cleanup c307bd8cdcfcf2460761891e53a002ded7355fac92d94fe6311f36fd97444d85 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True) 2024-04-22T23:47:58.203 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:47:57 smithi107 bash[145093]: ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-2 2024-04-22T23:47:58.481 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:47:58 smithi107 podman[145115]: 2024-04-22 23:47:58.212217054 +0000 UTC m=+0.625809570 container remove c307bd8cdcfcf2460761891e53a002ded7355fac92d94fe6311f36fd97444d85 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-2, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-22T23:47:58.823 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:47:58 smithi107 podman[145208]: 2024-04-22 23:47:58.455186053 +0000 UTC m=+0.018299402 image pull a0434699c1d76af4eea1fc95ff0a22ea73d1e04e609464fbf658d6f39bc7f2fc quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c 2024-04-22T23:47:58.823 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:47:58 smithi107 podman[145208]: 2024-04-22 23:47:58.554642296 +0000 UTC m=+0.117755635 container create 49ead56e7bd61d73e01680006713405fe6ba0167f09998aa2e0efce85081bec5 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T23:47:58.823 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:47:58 smithi107 podman[145208]: 2024-04-22 23:47:58.765608428 +0000 UTC m=+0.328721772 container init 49ead56e7bd61d73e01680006713405fe6ba0167f09998aa2e0efce85081bec5 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-22T23:47:58.823 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:47:58 smithi107 podman[145208]: 2024-04-22 23:47:58.76901817 +0000 UTC m=+0.332131515 container start 49ead56e7bd61d73e01680006713405fe6ba0167f09998aa2e0efce85081bec5 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-2-deactivate, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-22T23:47:58.823 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:47:58 smithi107 podman[145208]: 2024-04-22 23:47:58.831897088 +0000 UTC m=+0.395010442 container attach 49ead56e7bd61d73e01680006713405fe6ba0167f09998aa2e0efce85081bec5 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-22T23:47:59.231 INFO:journalctl@ceph.osd.2.smithi107.stdout:Apr 22 23:47:59 smithi107 podman[145208]: 2024-04-22 23:47:59.064077305 +0000 UTC m=+0.627190645 container died 49ead56e7bd61d73e01680006713405fe6ba0167f09998aa2e0efce85081bec5 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-2-deactivate, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-22T23:47:59.645 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8252f6f2-00ff-11ef-bc93-c7b262605968@osd.2.service' 2024-04-22T23:47:59.681 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T23:47:59.682 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-22T23:47:59.682 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-22T23:47:59.682 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-8252f6f2-00ff-11ef-bc93-c7b262605968@osd.3 2024-04-22T23:48:00.010 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:47:59 smithi157 systemd[1]: Stopping Ceph osd.3 for 8252f6f2-00ff-11ef-bc93-c7b262605968... 2024-04-22T23:48:00.379 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:48:00 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-3[88833]: 2024-04-22T23:48:00.018+0000 7f2274e64700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T23:48:00.379 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:48:00 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-3[88833]: 2024-04-22T23:48:00.018+0000 7f2274e64700 -1 osd.3 83 *** Got signal Terminated *** 2024-04-22T23:48:00.379 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:48:00 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-3[88833]: 2024-04-22T23:48:00.018+0000 7f2274e64700 -1 osd.3 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T23:48:04.129 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:48:03 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4[94007]: 2024-04-22T23:48:03.877+0000 7f30d5ab8700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-22T23:47:40.874043+0000 front 2024-04-22T23:47:40.874231+0000 (oldest deadline 2024-04-22T23:48:03.773936+0000) 2024-04-22T23:48:05.129 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:48:04 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4[94007]: 2024-04-22T23:48:04.857+0000 7f30d5ab8700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-22T23:47:40.874043+0000 front 2024-04-22T23:47:40.874231+0000 (oldest deadline 2024-04-22T23:48:03.773936+0000) 2024-04-22T23:48:05.825 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:48:05 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5[99177]: 2024-04-22T23:48:05.651+0000 7f5d43792700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-22T23:47:40.764198+0000 front 2024-04-22T23:47:40.764247+0000 (oldest deadline 2024-04-22T23:48:04.863993+0000) 2024-04-22T23:48:05.826 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:48:05 smithi157 podman[115672]: 2024-04-22 23:48:05.446731008 +0000 UTC m=+5.560770335 container died e81ca519c8318f203dcc7937b3566041fdddfd735147df2d9a41f7a22b19b590 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-22T23:48:06.129 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:48:05 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4[94007]: 2024-04-22T23:48:05.834+0000 7f30d5ab8700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-22T23:47:40.874043+0000 front 2024-04-22T23:47:40.874231+0000 (oldest deadline 2024-04-22T23:48:03.773936+0000) 2024-04-22T23:48:06.129 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:48:06 smithi157 podman[115672]: 2024-04-22 23:48:06.001574376 +0000 UTC m=+6.115613702 container cleanup e81ca519c8318f203dcc7937b3566041fdddfd735147df2d9a41f7a22b19b590 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-3, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True) 2024-04-22T23:48:06.129 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:48:06 smithi157 bash[115672]: ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-3 2024-04-22T23:48:06.612 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:48:06 smithi157 podman[115689]: 2024-04-22 23:48:06.269581345 +0000 UTC m=+0.821361715 container remove e81ca519c8318f203dcc7937b3566041fdddfd735147df2d9a41f7a22b19b590 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416) 2024-04-22T23:48:06.879 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:48:06 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4[94007]: 2024-04-22T23:48:06.832+0000 7f30d5ab8700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-22T23:47:40.874043+0000 front 2024-04-22T23:47:40.874231+0000 (oldest deadline 2024-04-22T23:48:03.773936+0000) 2024-04-22T23:48:06.879 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:48:06 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5[99177]: 2024-04-22T23:48:06.620+0000 7f5d43792700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-22T23:47:40.764198+0000 front 2024-04-22T23:47:40.764247+0000 (oldest deadline 2024-04-22T23:48:04.863993+0000) 2024-04-22T23:48:06.880 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:48:06 smithi157 podman[115781]: 2024-04-22 23:48:06.628975411 +0000 UTC m=+0.110583802 container create 902443a9c1936a7436b6a88548b0cb6cfa75006b2ad12fd03daf3bd7cd0cc10a (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T23:48:06.880 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:48:06 smithi157 podman[115781]: 2024-04-22 23:48:06.536315212 +0000 UTC m=+0.017923614 image pull a0434699c1d76af4eea1fc95ff0a22ea73d1e04e609464fbf658d6f39bc7f2fc quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c 2024-04-22T23:48:07.177 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:48:06 smithi157 podman[115781]: 2024-04-22 23:48:06.889614196 +0000 UTC m=+0.371222589 container init 902443a9c1936a7436b6a88548b0cb6cfa75006b2ad12fd03daf3bd7cd0cc10a (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T23:48:07.177 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:48:06 smithi157 podman[115781]: 2024-04-22 23:48:06.892724383 +0000 UTC m=+0.374332781 container start 902443a9c1936a7436b6a88548b0cb6cfa75006b2ad12fd03daf3bd7cd0cc10a (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T23:48:07.177 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:48:07 smithi157 podman[115781]: 2024-04-22 23:48:07.014846707 +0000 UTC m=+0.496455099 container attach 902443a9c1936a7436b6a88548b0cb6cfa75006b2ad12fd03daf3bd7cd0cc10a (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T23:48:07.450 INFO:journalctl@ceph.osd.3.smithi157.stdout:Apr 22 23:48:07 smithi157 podman[115781]: 2024-04-22 23:48:07.18667401 +0000 UTC m=+0.668282403 container died 902443a9c1936a7436b6a88548b0cb6cfa75006b2ad12fd03daf3bd7cd0cc10a (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T23:48:07.713 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:48:07 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5[99177]: 2024-04-22T23:48:07.629+0000 7f5d43792700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-22T23:47:40.764198+0000 front 2024-04-22T23:47:40.764247+0000 (oldest deadline 2024-04-22T23:48:04.863993+0000) 2024-04-22T23:48:07.786 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8252f6f2-00ff-11ef-bc93-c7b262605968@osd.3.service' 2024-04-22T23:48:07.824 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T23:48:07.824 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-22T23:48:07.825 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-22T23:48:07.825 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-8252f6f2-00ff-11ef-bc93-c7b262605968@osd.4 2024-04-22T23:48:08.129 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:48:07 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4[94007]: 2024-04-22T23:48:07.822+0000 7f30d5ab8700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-22T23:47:40.874043+0000 front 2024-04-22T23:47:40.874231+0000 (oldest deadline 2024-04-22T23:48:03.773936+0000) 2024-04-22T23:48:08.129 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:48:07 smithi157 systemd[1]: Stopping Ceph osd.4 for 8252f6f2-00ff-11ef-bc93-c7b262605968... 2024-04-22T23:48:08.439 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:48:08 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4[94007]: 2024-04-22T23:48:08.188+0000 7f30dd4cc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T23:48:08.439 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:48:08 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4[94007]: 2024-04-22T23:48:08.188+0000 7f30dd4cc700 -1 osd.4 83 *** Got signal Terminated *** 2024-04-22T23:48:08.439 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:48:08 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4[94007]: 2024-04-22T23:48:08.188+0000 7f30dd4cc700 -1 osd.4 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T23:48:08.812 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:48:08 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5[99177]: 2024-04-22T23:48:08.661+0000 7f5d43792700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-22T23:47:40.764198+0000 front 2024-04-22T23:47:40.764247+0000 (oldest deadline 2024-04-22T23:48:04.863993+0000) 2024-04-22T23:48:09.129 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:48:08 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4[94007]: 2024-04-22T23:48:08.820+0000 7f30d5ab8700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-22T23:47:40.874043+0000 front 2024-04-22T23:47:40.874231+0000 (oldest deadline 2024-04-22T23:48:03.773936+0000) 2024-04-22T23:48:10.129 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:48:09 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4[94007]: 2024-04-22T23:48:09.846+0000 7f30d5ab8700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-22T23:47:40.874043+0000 front 2024-04-22T23:47:40.874231+0000 (oldest deadline 2024-04-22T23:48:03.773936+0000) 2024-04-22T23:48:10.129 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:48:09 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5[99177]: 2024-04-22T23:48:09.699+0000 7f5d43792700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-22T23:47:40.764198+0000 front 2024-04-22T23:47:40.764247+0000 (oldest deadline 2024-04-22T23:48:04.863993+0000) 2024-04-22T23:48:11.129 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:48:10 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4[94007]: 2024-04-22T23:48:10.850+0000 7f30d5ab8700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-22T23:47:40.874043+0000 front 2024-04-22T23:47:40.874231+0000 (oldest deadline 2024-04-22T23:48:03.773936+0000) 2024-04-22T23:48:11.129 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:48:10 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5[99177]: 2024-04-22T23:48:10.733+0000 7f5d43792700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-22T23:47:40.764198+0000 front 2024-04-22T23:47:40.764247+0000 (oldest deadline 2024-04-22T23:48:04.863993+0000) 2024-04-22T23:48:11.129 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:48:10 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5[99177]: 2024-04-22T23:48:10.733+0000 7f5d43792700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-04-22T23:47:44.864471+0000 front 2024-04-22T23:47:44.864428+0000 (oldest deadline 2024-04-22T23:48:10.164326+0000) 2024-04-22T23:48:12.129 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:48:11 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4[94007]: 2024-04-22T23:48:11.886+0000 7f30d5ab8700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-22T23:47:40.874043+0000 front 2024-04-22T23:47:40.874231+0000 (oldest deadline 2024-04-22T23:48:03.773936+0000) 2024-04-22T23:48:12.129 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:48:11 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5[99177]: 2024-04-22T23:48:11.722+0000 7f5d43792700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-22T23:47:40.764198+0000 front 2024-04-22T23:47:40.764247+0000 (oldest deadline 2024-04-22T23:48:04.863993+0000) 2024-04-22T23:48:12.129 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:48:11 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5[99177]: 2024-04-22T23:48:11.722+0000 7f5d43792700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-04-22T23:47:44.864471+0000 front 2024-04-22T23:47:44.864428+0000 (oldest deadline 2024-04-22T23:48:10.164326+0000) 2024-04-22T23:48:13.030 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:48:12 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4[94007]: 2024-04-22T23:48:12.915+0000 7f30d5ab8700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-22T23:47:40.874043+0000 front 2024-04-22T23:47:40.874231+0000 (oldest deadline 2024-04-22T23:48:03.773936+0000) 2024-04-22T23:48:13.031 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:48:12 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5[99177]: 2024-04-22T23:48:12.701+0000 7f5d43792700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-22T23:47:40.764198+0000 front 2024-04-22T23:47:40.764247+0000 (oldest deadline 2024-04-22T23:48:04.863993+0000) 2024-04-22T23:48:13.031 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:48:12 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5[99177]: 2024-04-22T23:48:12.701+0000 7f5d43792700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-04-22T23:47:44.864471+0000 front 2024-04-22T23:47:44.864428+0000 (oldest deadline 2024-04-22T23:48:10.164326+0000) 2024-04-22T23:48:13.814 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:48:13 smithi157 podman[115895]: 2024-04-22 23:48:13.537300443 +0000 UTC m=+5.466241436 container died d5b5a2e4781af2f41eab518af21506bfa1e3fb42f5f040299d734b29679c4e5b (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-22T23:48:13.814 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:48:13 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5[99177]: 2024-04-22T23:48:13.742+0000 7f5d43792700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-22T23:47:40.764198+0000 front 2024-04-22T23:47:40.764247+0000 (oldest deadline 2024-04-22T23:48:04.863993+0000) 2024-04-22T23:48:13.815 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:48:13 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5[99177]: 2024-04-22T23:48:13.742+0000 7f5d43792700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-04-22T23:47:44.864471+0000 front 2024-04-22T23:47:44.864428+0000 (oldest deadline 2024-04-22T23:48:10.164326+0000) 2024-04-22T23:48:14.129 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:48:13 smithi157 podman[115895]: 2024-04-22 23:48:13.938758386 +0000 UTC m=+5.867699378 container cleanup d5b5a2e4781af2f41eab518af21506bfa1e3fb42f5f040299d734b29679c4e5b (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-22T23:48:14.129 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:48:13 smithi157 bash[115895]: ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4 2024-04-22T23:48:14.473 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:48:14 smithi157 podman[115914]: 2024-04-22 23:48:14.165048551 +0000 UTC m=+0.627413456 container remove d5b5a2e4781af2f41eab518af21506bfa1e3fb42f5f040299d734b29679c4e5b (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-22T23:48:14.784 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:48:14 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5[99177]: 2024-04-22T23:48:14.703+0000 7f5d43792700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-22T23:47:40.764198+0000 front 2024-04-22T23:47:40.764247+0000 (oldest deadline 2024-04-22T23:48:04.863993+0000) 2024-04-22T23:48:14.784 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:48:14 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5[99177]: 2024-04-22T23:48:14.703+0000 7f5d43792700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-04-22T23:47:44.864471+0000 front 2024-04-22T23:47:44.864428+0000 (oldest deadline 2024-04-22T23:48:10.164326+0000) 2024-04-22T23:48:14.784 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:48:14 smithi157 podman[116001]: 2024-04-22 23:48:14.482583946 +0000 UTC m=+0.113564772 container create e6c9162b2f4e9075ea8cf592b88b1391ea928b83ea8565255b0b24765bdcb17a (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-22T23:48:14.784 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:48:14 smithi157 podman[116001]: 2024-04-22 23:48:14.387248872 +0000 UTC m=+0.018229711 image pull a0434699c1d76af4eea1fc95ff0a22ea73d1e04e609464fbf658d6f39bc7f2fc quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c 2024-04-22T23:48:14.784 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:48:14 smithi157 podman[116001]: 2024-04-22 23:48:14.693436011 +0000 UTC m=+0.324416838 container init e6c9162b2f4e9075ea8cf592b88b1391ea928b83ea8565255b0b24765bdcb17a (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T23:48:14.785 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:48:14 smithi157 podman[116001]: 2024-04-22 23:48:14.696654619 +0000 UTC m=+0.327635445 container start e6c9162b2f4e9075ea8cf592b88b1391ea928b83ea8565255b0b24765bdcb17a (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5) 2024-04-22T23:48:14.785 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:48:14 smithi157 podman[116001]: 2024-04-22 23:48:14.793314511 +0000 UTC m=+0.424295338 container attach e6c9162b2f4e9075ea8cf592b88b1391ea928b83ea8565255b0b24765bdcb17a (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-22T23:48:15.129 INFO:journalctl@ceph.osd.4.smithi157.stdout:Apr 22 23:48:14 smithi157 podman[116001]: 2024-04-22 23:48:14.993183211 +0000 UTC m=+0.624164043 container died e6c9162b2f4e9075ea8cf592b88b1391ea928b83ea8565255b0b24765bdcb17a (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True) 2024-04-22T23:48:15.588 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8252f6f2-00ff-11ef-bc93-c7b262605968@osd.4.service' 2024-04-22T23:48:15.626 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T23:48:15.626 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-22T23:48:15.627 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-22T23:48:15.627 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ceph-8252f6f2-00ff-11ef-bc93-c7b262605968@osd.5 2024-04-22T23:48:15.879 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:48:15 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5[99177]: 2024-04-22T23:48:15.696+0000 7f5d43792700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-22T23:47:40.764198+0000 front 2024-04-22T23:47:40.764247+0000 (oldest deadline 2024-04-22T23:48:04.863993+0000) 2024-04-22T23:48:15.879 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:48:15 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5[99177]: 2024-04-22T23:48:15.696+0000 7f5d43792700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-04-22T23:47:44.864471+0000 front 2024-04-22T23:47:44.864428+0000 (oldest deadline 2024-04-22T23:48:10.164326+0000) 2024-04-22T23:48:15.880 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:48:15 smithi157 systemd[1]: Stopping Ceph osd.5 for 8252f6f2-00ff-11ef-bc93-c7b262605968... 2024-04-22T23:48:16.379 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:48:15 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5[99177]: 2024-04-22T23:48:15.990+0000 7f5d4b1a6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T23:48:16.379 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:48:15 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5[99177]: 2024-04-22T23:48:15.990+0000 7f5d4b1a6700 -1 osd.5 83 *** Got signal Terminated *** 2024-04-22T23:48:16.379 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:48:15 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5[99177]: 2024-04-22T23:48:15.990+0000 7f5d4b1a6700 -1 osd.5 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T23:48:17.129 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:48:16 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5[99177]: 2024-04-22T23:48:16.731+0000 7f5d43792700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-22T23:47:40.764198+0000 front 2024-04-22T23:47:40.764247+0000 (oldest deadline 2024-04-22T23:48:04.863993+0000) 2024-04-22T23:48:17.129 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:48:16 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5[99177]: 2024-04-22T23:48:16.731+0000 7f5d43792700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-04-22T23:47:44.864471+0000 front 2024-04-22T23:47:44.864428+0000 (oldest deadline 2024-04-22T23:48:10.164326+0000) 2024-04-22T23:48:18.129 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:48:17 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5[99177]: 2024-04-22T23:48:17.693+0000 7f5d43792700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-22T23:47:40.764198+0000 front 2024-04-22T23:47:40.764247+0000 (oldest deadline 2024-04-22T23:48:04.863993+0000) 2024-04-22T23:48:18.129 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:48:17 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5[99177]: 2024-04-22T23:48:17.693+0000 7f5d43792700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-04-22T23:47:44.864471+0000 front 2024-04-22T23:47:44.864428+0000 (oldest deadline 2024-04-22T23:48:10.164326+0000) 2024-04-22T23:48:19.129 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:48:18 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5[99177]: 2024-04-22T23:48:18.697+0000 7f5d43792700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-22T23:47:40.764198+0000 front 2024-04-22T23:47:40.764247+0000 (oldest deadline 2024-04-22T23:48:04.863993+0000) 2024-04-22T23:48:19.129 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:48:18 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5[99177]: 2024-04-22T23:48:18.697+0000 7f5d43792700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-04-22T23:47:44.864471+0000 front 2024-04-22T23:47:44.864428+0000 (oldest deadline 2024-04-22T23:48:10.164326+0000) 2024-04-22T23:48:20.129 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:48:19 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5[99177]: 2024-04-22T23:48:19.679+0000 7f5d43792700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-22T23:47:40.764198+0000 front 2024-04-22T23:47:40.764247+0000 (oldest deadline 2024-04-22T23:48:04.863993+0000) 2024-04-22T23:48:20.129 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:48:19 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5[99177]: 2024-04-22T23:48:19.679+0000 7f5d43792700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-04-22T23:47:44.864471+0000 front 2024-04-22T23:47:44.864428+0000 (oldest deadline 2024-04-22T23:48:10.164326+0000) 2024-04-22T23:48:21.129 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:48:20 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5[99177]: 2024-04-22T23:48:20.702+0000 7f5d43792700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-22T23:47:40.764198+0000 front 2024-04-22T23:47:40.764247+0000 (oldest deadline 2024-04-22T23:48:04.863993+0000) 2024-04-22T23:48:21.129 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:48:20 smithi157 ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5[99177]: 2024-04-22T23:48:20.702+0000 7f5d43792700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-04-22T23:47:44.864471+0000 front 2024-04-22T23:47:44.864428+0000 (oldest deadline 2024-04-22T23:48:10.164326+0000) 2024-04-22T23:48:21.750 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:48:21 smithi157 podman[116105]: 2024-04-22 23:48:21.501572174 +0000 UTC m=+5.651393478 container died 0ee423135fdb3187335ab2befe55b4aee05754903fe6b26051f148ffea5a0588 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T23:48:22.125 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:48:21 smithi157 podman[116105]: 2024-04-22 23:48:21.858114422 +0000 UTC m=+6.007935733 container cleanup 0ee423135fdb3187335ab2befe55b4aee05754903fe6b26051f148ffea5a0588 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5, org.label-schema.schema-version=1.0, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T23:48:22.125 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:48:21 smithi157 bash[116105]: ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5 2024-04-22T23:48:22.379 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:48:22 smithi157 podman[116123]: 2024-04-22 23:48:22.134310685 +0000 UTC m=+0.626943854 container remove 0ee423135fdb3187335ab2befe55b4aee05754903fe6b26051f148ffea5a0588 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS) 2024-04-22T23:48:22.721 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:48:22 smithi157 podman[116218]: 2024-04-22 23:48:22.374055147 +0000 UTC m=+0.017993753 image pull a0434699c1d76af4eea1fc95ff0a22ea73d1e04e609464fbf658d6f39bc7f2fc quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c 2024-04-22T23:48:22.721 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:48:22 smithi157 podman[116218]: 2024-04-22 23:48:22.502096044 +0000 UTC m=+0.146034631 container create 632a0c8169bfeef4876578d604d2a402e35de57b326980b9614434c6967edcd8 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-22T23:48:22.722 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:48:22 smithi157 podman[116218]: 2024-04-22 23:48:22.730522397 +0000 UTC m=+0.374460986 container init 632a0c8169bfeef4876578d604d2a402e35de57b326980b9614434c6967edcd8 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True) 2024-04-22T23:48:23.017 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:48:22 smithi157 podman[116218]: 2024-04-22 23:48:22.733844622 +0000 UTC m=+0.377783217 container start 632a0c8169bfeef4876578d604d2a402e35de57b326980b9614434c6967edcd8 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T23:48:23.017 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:48:22 smithi157 podman[116218]: 2024-04-22 23:48:22.804389037 +0000 UTC m=+0.448327635 container attach 632a0c8169bfeef4876578d604d2a402e35de57b326980b9614434c6967edcd8 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-22T23:48:23.356 INFO:journalctl@ceph.osd.5.smithi157.stdout:Apr 22 23:48:23 smithi157 podman[116218]: 2024-04-22 23:48:23.026088501 +0000 UTC m=+0.670027081 container died 632a0c8169bfeef4876578d604d2a402e35de57b326980b9614434c6967edcd8 (image=quay.ceph.io/ceph-ci/ceph@sha256:b8bc99cfccdf4e584cbe106d55b5c5dc774312148ba54d6ca5b660445cf4506c, name=ceph-8252f6f2-00ff-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-adk-testing-2024-04-22-1618-43be020, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-22T23:48:23.635 DEBUG:teuthology.orchestra.run.smithi157:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8252f6f2-00ff-11ef-bc93-c7b262605968@osd.5.service' 2024-04-22T23:48:23.673 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T23:48:23.673 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-22T23:48:23.674 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-22T23:48:49.217 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-22T23:48:59.790 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-22T23:48:59.822 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-22T23:48:59.848 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-22T23:48:59.852 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/adking-2024-04-22_22:45:33-orch:cephadm-wip-adk-testing-2024-04-22-1618-distro-default-smithi/7669223/remote/smithi107/crash 2024-04-22T23:48:59.853 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/crash -- . 2024-04-22T23:48:59.888 INFO:teuthology.orchestra.run.smithi107.stderr:tar: /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-22T23:48:59.888 INFO:teuthology.orchestra.run.smithi107.stderr:tar: Error is not recoverable: exiting now 2024-04-22T23:48:59.890 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/adking-2024-04-22_22:45:33-orch:cephadm-wip-adk-testing-2024-04-22-1618-distro-default-smithi/7669223/remote/smithi157/crash 2024-04-22T23:48:59.891 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/crash -- . 2024-04-22T23:48:59.913 INFO:teuthology.orchestra.run.smithi157.stderr:tar: /var/lib/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-22T23:48:59.913 INFO:teuthology.orchestra.run.smithi157.stderr:tar: Error is not recoverable: exiting now 2024-04-22T23:48:59.914 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-22T23:48:59.914 DEBUG:teuthology.orchestra.run.smithi107:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-22T23:48:59.961 INFO:teuthology.orchestra.run.smithi107.stdout:2024-04-22T23:40:00.000110+0000 mon.smithi107 (mon.0) 316 : cluster [WRN] Health detail: HEALTH_WARN 1 osds down; Degraded data redundancy: 54/339 objects degraded (15.929%), 16 pgs degraded 2024-04-22T23:48:59.961 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-04-22T23:48:59.962 DEBUG:teuthology.orchestra.run.smithi107:> sudo egrep '\[SEC\]' /var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-22T23:49:00.032 DEBUG:teuthology.orchestra.run.smithi107:> sudo egrep '\[ERR\]' /var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-22T23:49:00.104 DEBUG:teuthology.orchestra.run.smithi107:> sudo egrep '\[WRN\]' /var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-22T23:49:00.175 INFO:teuthology.orchestra.run.smithi107.stdout:2024-04-22T23:40:00.000110+0000 mon.smithi107 (mon.0) 316 : cluster [WRN] Health detail: HEALTH_WARN 1 osds down; Degraded data redundancy: 54/339 objects degraded (15.929%), 16 pgs degraded 2024-04-22T23:49:00.175 INFO:tasks.cephadm:Compressing logs... 2024-04-22T23:49:00.176 DEBUG:teuthology.orchestra.run.smithi107:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-22T23:49:00.218 DEBUG:teuthology.orchestra.run.smithi157:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-22T23:49:00.239 INFO:teuthology.orchestra.run.smithi107.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log: No such file or directory 2024-04-22T23:49:00.239 INFO:teuthology.orchestra.run.smithi107.stderr: 2024-04-22T23:49:00.239 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph.log 2024-04-22T23:49:00.239 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-22T23:49:00.240 INFO:teuthology.orchestra.run.smithi107.stderr:gzip/var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph.log: -5 --verbose -- /var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-22T23:49:00.240 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-22T23:49:00.240 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-22T23:49:00.241 INFO:teuthology.orchestra.run.smithi157.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-22T23:49:00.241 INFO:teuthology.orchestra.run.smithi157.stderr:: No such file or directory 2024-04-22T23:49:00.241 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph.log 2024-04-22T23:49:00.241 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-22T23:49:00.242 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-22T23:49:00.243 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-22T23:49:00.243 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-22T23:49:00.243 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi157.ufywse.log 2024-04-22T23:49:00.243 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph-osd.3.log: /var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph-mgr.smithi157.tuylis.log 2024-04-22T23:49:00.243 INFO:teuthology.orchestra.run.smithi157.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz/var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi157.ufywse.log: 2024-04-22T23:49:00.243 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-22T23:49:00.246 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi107.ktuxjl.log 2024-04-22T23:49:00.246 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph-osd.0.log: /var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi107.ktuxjl.log: 87.5% -- replaced with /var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-22T23:49:00.246 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph-mgr.smithi107.uipxvm.log 2024-04-22T23:49:00.246 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph-mon.smithi107.log 2024-04-22T23:49:00.246 INFO:teuthology.orchestra.run.smithi107.stderr: 91.2% -- replaced with /var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-22T23:49:00.247 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph-mgr.smithi107.uipxvm.log: gzip -5 --verbose -- /var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-22T23:49:00.248 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph-mgr.smithi157.tuylis.log: 87.5% -- replaced with /var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-22T23:49:00.248 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi157.rcaris.log 2024-04-22T23:49:00.248 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph-mon.smithi157.log 2024-04-22T23:49:00.248 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi157.rcaris.log: 85.4% -- replaced with /var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-22T23:49:00.248 INFO:teuthology.orchestra.run.smithi157.stderr: 91.4%gzip -5 --verbose -- /var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-22T23:49:00.248 INFO:teuthology.orchestra.run.smithi157.stderr: -- replaced with /var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-22T23:49:00.267 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph-mon.smithi107.log: gzip -5 --verbose -- /var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi107.dfvvkw.log 2024-04-22T23:49:00.268 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph-mon.smithi157.log: /var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph-volume.log: 91.1% -- replaced with /var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-22T23:49:00.268 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph.cephadm.log: 85.5% -- replaced with /var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-22T23:49:00.274 INFO:teuthology.orchestra.run.smithi107.stderr: 92.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-22T23:49:00.275 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-22T23:49:00.295 INFO:teuthology.orchestra.run.smithi157.stderr: 89.6% -- replaced with /var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph-mgr.smithi157.tuylis.log.gz 2024-04-22T23:49:00.334 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi107.dfvvkw.log: /var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph-volume.log: 91.1% -- replaced with /var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-22T23:49:00.624 INFO:teuthology.orchestra.run.smithi107.stderr: 88.5% -- replaced with /var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph-mgr.smithi107.uipxvm.log.gz 2024-04-22T23:49:00.631 INFO:teuthology.orchestra.run.smithi157.stderr: 92.6% -- replaced with /var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph-mon.smithi157.log.gz 2024-04-22T23:49:01.145 INFO:teuthology.orchestra.run.smithi107.stderr: 90.6% -- replaced with /var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph-mon.smithi107.log.gz 2024-04-22T23:49:04.018 INFO:teuthology.orchestra.run.smithi157.stderr: 93.2% -- replaced with /var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-22T23:49:04.481 INFO:teuthology.orchestra.run.smithi107.stderr: 93.3% -- replaced with /var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-22T23:49:04.985 INFO:teuthology.orchestra.run.smithi107.stderr: 93.3% -- replaced with /var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-22T23:49:05.458 INFO:teuthology.orchestra.run.smithi157.stderr: 93.4% -- replaced with /var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-22T23:49:05.995 INFO:teuthology.orchestra.run.smithi157.stderr: 95.0% -- replaced with /var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi157.ufywse.log.gz 2024-04-22T23:49:06.124 INFO:teuthology.orchestra.run.smithi157.stderr: 93.3% -- replaced with /var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-22T23:49:06.148 INFO:teuthology.orchestra.run.smithi107.stderr: 95.0% -- replaced with /var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi107.dfvvkw.log.gz 2024-04-22T23:49:06.614 INFO:teuthology.orchestra.run.smithi107.stderr: 93.3% -- replaced with /var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-22T23:49:35.391 INFO:teuthology.orchestra.run.smithi157.stderr: 93.1% -- replaced with /var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi157.rcaris.log.gz 2024-04-22T23:49:35.391 INFO:teuthology.orchestra.run.smithi157.stderr: 2024-04-22T23:49:35.391 INFO:teuthology.orchestra.run.smithi157.stderr:real 0m35.157s 2024-04-22T23:49:35.391 INFO:teuthology.orchestra.run.smithi157.stderr:user 0m54.295s 2024-04-22T23:49:35.391 INFO:teuthology.orchestra.run.smithi157.stderr:sys 0m1.863s 2024-04-22T23:49:50.071 INFO:teuthology.orchestra.run.smithi107.stderr: 93.0% -- replaced with /var/log/ceph/8252f6f2-00ff-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi107.ktuxjl.log.gz 2024-04-22T23:49:50.073 INFO:teuthology.orchestra.run.smithi107.stderr: 2024-04-22T23:49:50.073 INFO:teuthology.orchestra.run.smithi107.stderr:real 0m49.844s 2024-04-22T23:49:50.073 INFO:teuthology.orchestra.run.smithi107.stderr:user 1m9.939s 2024-04-22T23:49:50.073 INFO:teuthology.orchestra.run.smithi107.stderr:sys 0m2.324s 2024-04-22T23:49:50.073 INFO:tasks.cephadm:Archiving logs... 2024-04-22T23:49:50.074 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/log/ceph to /home/teuthworker/archive/adking-2024-04-22_22:45:33-orch:cephadm-wip-adk-testing-2024-04-22-1618-distro-default-smithi/7669223/remote/smithi107/log 2024-04-22T23:49:50.075 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-22T23:50:00.571 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/log/ceph to /home/teuthworker/archive/adking-2024-04-22_22:45:33-orch:cephadm-wip-adk-testing-2024-04-22-1618-distro-default-smithi/7669223/remote/smithi157/log 2024-04-22T23:50:00.572 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-22T23:50:08.571 INFO:tasks.cephadm:Removing cluster... 2024-04-22T23:50:08.571 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 --force 2024-04-22T23:50:09.251 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8252f6f2-00ff-11ef-bc93-c7b262605968 --force 2024-04-22T23:50:09.795 INFO:tasks.cephadm:Removing cephadm ... 2024-04-22T23:50:09.795 DEBUG:teuthology.orchestra.run.smithi107:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-22T23:50:09.812 DEBUG:teuthology.orchestra.run.smithi157:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-22T23:50:09.827 INFO:tasks.cephadm:Teardown complete 2024-04-22T23:50:09.828 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-22T23:50:09.840 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-22T23:50:09.841 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-22T23:50:09.857 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-22T23:50:09.940 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-22T23:50:09.951 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-22T23:50:09.951 DEBUG:teuthology.orchestra.run.smithi107:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-22T23:50:09.954 DEBUG:teuthology.orchestra.run.smithi157:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-22T23:50:09.966 INFO:teuthology.orchestra.run.smithi107.stderr:bash: line 1: ntpq: command not found 2024-04-22T23:50:09.970 INFO:teuthology.orchestra.run.smithi107.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-22T23:50:09.970 INFO:teuthology.orchestra.run.smithi107.stdout:=============================================================================== 2024-04-22T23:50:09.970 INFO:teuthology.orchestra.run.smithi107.stdout:^- hv01.front.sepia.ceph.com 2 6 377 43 +383us[ +383us] +/- 582ms 2024-04-22T23:50:09.970 INFO:teuthology.orchestra.run.smithi107.stdout:^* hv02.front.sepia.ceph.com 3 6 377 44 -174us[ -158us] +/- 72ms 2024-04-22T23:50:09.970 INFO:teuthology.orchestra.run.smithi107.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 44 +142us[ +142us] +/- 66ms 2024-04-22T23:50:09.970 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T23:50:09.972 INFO:teuthology.orchestra.run.smithi157.stderr:bash: line 1: ntpq: command not found 2024-04-22T23:50:09.975 INFO:teuthology.orchestra.run.smithi157.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-22T23:50:09.975 INFO:teuthology.orchestra.run.smithi157.stdout:=============================================================================== 2024-04-22T23:50:09.975 INFO:teuthology.orchestra.run.smithi157.stdout:^- hv01.front.sepia.ceph.com 2 6 377 45 +389us[ +397us] +/- 582ms 2024-04-22T23:50:09.975 INFO:teuthology.orchestra.run.smithi157.stdout:^* hv02.front.sepia.ceph.com 3 6 377 45 -156us[ -148us] +/- 72ms 2024-04-22T23:50:09.975 INFO:teuthology.orchestra.run.smithi157.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 45 +154us[ +154us] +/- 66ms 2024-04-22T23:50:09.975 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T23:50:09.976 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-22T23:50:09.987 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-22T23:50:09.988 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-22T23:50:10.016 DEBUG:teuthology.orchestra.run.smithi107:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-22T23:50:10.095 DEBUG:teuthology.orchestra.run.smithi157:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-22T23:50:10.167 DEBUG:teuthology.orchestra.run.smithi107:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-22T23:50:10.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T23:50:10.195 DEBUG:teuthology.orchestra.run.smithi157:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-22T23:50:10.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T23:50:10.222 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-22T23:50:10.234 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-22T23:50:10.282 INFO:teuthology.task.internal:Duration was 1876.348685 seconds 2024-04-22T23:50:10.282 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-22T23:50:10.315 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-22T23:50:10.315 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-22T23:50:10.318 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-22T23:50:10.352 INFO:teuthology.orchestra.run.smithi107.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-22T23:50:10.355 INFO:teuthology.orchestra.run.smithi157.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-22T23:50:10.670 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-22T23:50:10.670 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi107.front.sepia.ceph.com 2024-04-22T23:50:10.670 DEBUG:teuthology.orchestra.run.smithi107:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-22T23:50:10.690 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi157.front.sepia.ceph.com 2024-04-22T23:50:10.690 DEBUG:teuthology.orchestra.run.smithi157:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-22T23:50:10.738 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-22T23:50:10.738 DEBUG:teuthology.orchestra.run.smithi107:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-22T23:50:10.740 DEBUG:teuthology.orchestra.run.smithi157:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-22T23:50:10.803 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-22T23:50:10.803 DEBUG:teuthology.orchestra.run.smithi107:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-22T23:50:10.805 DEBUG:teuthology.orchestra.run.smithi157:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-22T23:50:11.455 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-22T23:50:11.468 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-22T23:50:11.469 DEBUG:teuthology.orchestra.run.smithi107:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-22T23:50:11.492 DEBUG:teuthology.orchestra.run.smithi157:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-22T23:50:11.517 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-22T23:50:11.531 DEBUG:teuthology.orchestra.run.smithi107:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-22T23:50:11.534 DEBUG:teuthology.orchestra.run.smithi157:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-22T23:50:11.553 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern = core 2024-04-22T23:50:11.579 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern = core 2024-04-22T23:50:11.591 DEBUG:teuthology.orchestra.run.smithi107:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-22T23:50:11.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T23:50:11.620 DEBUG:teuthology.orchestra.run.smithi157:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-22T23:50:11.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T23:50:11.646 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-22T23:50:11.659 INFO:teuthology.task.internal:Transferring archived files... 2024-04-22T23:50:11.660 DEBUG:teuthology.misc:Transferring archived files from smithi107:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/adking-2024-04-22_22:45:33-orch:cephadm-wip-adk-testing-2024-04-22-1618-distro-default-smithi/7669223/remote/smithi107 2024-04-22T23:50:11.660 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-22T23:50:11.712 DEBUG:teuthology.misc:Transferring archived files from smithi157:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/adking-2024-04-22_22:45:33-orch:cephadm-wip-adk-testing-2024-04-22-1618-distro-default-smithi/7669223/remote/smithi157 2024-04-22T23:50:11.712 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-22T23:50:11.750 INFO:teuthology.task.internal:Removing archive directory... 2024-04-22T23:50:11.750 DEBUG:teuthology.orchestra.run.smithi107:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-22T23:50:11.752 DEBUG:teuthology.orchestra.run.smithi157:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-22T23:50:11.800 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-22T23:50:11.817 INFO:teuthology.task.internal:Not uploading archives. 2024-04-22T23:50:11.817 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-22T23:50:11.843 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-22T23:50:11.843 DEBUG:teuthology.orchestra.run.smithi107:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-22T23:50:11.845 DEBUG:teuthology.orchestra.run.smithi157:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-22T23:50:11.858 INFO:teuthology.orchestra.run.smithi107.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 22 23:50 /home/ubuntu/cephtest 2024-04-22T23:50:11.859 INFO:teuthology.orchestra.run.smithi157.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 22 23:50 /home/ubuntu/cephtest 2024-04-22T23:50:11.861 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-22T23:50:11.874 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-22T23:50:11.942 INFO:teuthology.nuke:Checking targets against current locks 2024-04-22T23:50:11.964 DEBUG:teuthology.nuke:shortname: smithi107 2024-04-22T23:50:11.965 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-22T23:50:11.988 DEBUG:teuthology.nuke:shortname: smithi157 2024-04-22T23:50:11.989 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-22T23:50:11.992 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi107.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/adking-2024-04-22_22:45:33-orch:cephadm-wip-adk-testing-2024-04-22-1618-distro-default-smithi/7669223', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-22 23:12:21.079576', 'locked_by': 'scheduled_adking@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB+DTvQAnUBeyS/bLqLhzS7sorox8muSN8gqBNapoUj1mivuWT3erhKYzFcA+NzRrRmz3Qw6fWIKkUU7eebmS0I='} 2024-04-22T23:50:12.017 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi157.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/adking-2024-04-22_22:45:33-orch:cephadm-wip-adk-testing-2024-04-22-1618-distro-default-smithi/7669223', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-22 23:12:21.078218', 'locked_by': 'scheduled_adking@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMeZUlG6mdz2mxTsSJmYKj0JPQJm6K4MO+DH8xaIK+f8c8Dfx+Y34lUB3z8FFDMUOmXEL2TsyqIsZ3zFX5yvkAk='} 2024-04-22T23:50:12.046 INFO:teuthology.orchestra.console.smithi107:Power off 2024-04-22T23:50:12.047 DEBUG:teuthology.orchestra.console.smithi107:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-22T23:50:12.080 DEBUG:teuthology.orchestra.console.smithi107:power off output: Chassis Power Control: Down/Off 2024-04-22T23:50:12.080 DEBUG:teuthology.orchestra.console.smithi107:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-22T23:50:12.102 INFO:teuthology.orchestra.console.smithi157:Power off 2024-04-22T23:50:12.102 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-22T23:50:12.113 DEBUG:teuthology.orchestra.console.smithi107:check power output: Chassis Power is on 2024-04-22T23:50:12.128 DEBUG:teuthology.orchestra.console.smithi157:power off output: Chassis Power Control: Down/Off 2024-04-22T23:50:12.128 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-22T23:50:12.154 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is on 2024-04-22T23:50:16.114 DEBUG:teuthology.orchestra.console.smithi107:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-22T23:50:16.156 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-22T23:50:16.224 DEBUG:teuthology.orchestra.console.smithi107:check power output: Chassis Power is on 2024-04-22T23:50:16.270 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is on 2024-04-22T23:50:20.229 DEBUG:teuthology.orchestra.console.smithi107:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-22T23:50:20.273 DEBUG:teuthology.orchestra.console.smithi157:pexpect command: ipmitool -H smithi157.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-22T23:50:20.340 DEBUG:teuthology.orchestra.console.smithi107:check power output: Chassis Power is off 2024-04-22T23:50:20.384 DEBUG:teuthology.orchestra.console.smithi157:check power output: Chassis Power is off 2024-04-22T23:50:20.442 INFO:teuthology.orchestra.console.smithi107:Power off completed 2024-04-22T23:50:20.485 INFO:teuthology.orchestra.console.smithi157:Power off completed 2024-04-22T23:50:20.590 INFO:teuthology.run:Summary data: description: orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 1876.3486847877502 failure_reason: '"2024-04-22T23:40:00.000110+0000 mon.smithi107 (mon.0) 316 : cluster [WRN] Health detail: HEALTH_WARN 1 osds down; Degraded data redundancy: 54/339 objects degraded (15.929%), 16 pgs degraded" in cluster log' flavor: default owner: scheduled_adking@teuthology success: false 2024-04-22T23:50:20.590 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-22T23:50:20.678 INFO:teuthology.run:FAIL